Contiguity of Probability Measures: Some Applications in Statistics (Cambridge Tracts in Mathematics (No. 63))

CAMBRIDGE TRACTS IN MATHEMATICS AND MATHEMATICAL PHYSICS GENERAL EDITOBS H. BASS, J. F. C. KINGMAN, F. SMITHIES J. A. ...

Author: George G. Roussas

137 downloads 540 Views 3MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

CAMBRIDGE TRACTS IN MATHEMATICS AND MATHEMATICAL PHYSICS GENERAL EDITOBS

H. BASS, J. F. C. KINGMAN, F. SMITHIES J. A. TODD & C. T. C. WALL

63. Contiguity of probability measures: some applications in statistics

GEORGE G. ROUSSAS Professor of Statistics University of Wisconsin

Contiguity of probability measures: some applications in statistics

CAMBRIDGE AT THE UNIVERSITY PRESS 1972

CAMBRIDGE UNIVERSITY PRESS Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, Sao Paulo, Delhi Cambridge University Press The Edinburgh Building, Cambridge CB2 8RU, UK Published in the United States of America by Cambridge University Press, New York www.cambridge.org Information on this title: www.cambridge.org/9780521083546 © Cambridge University Press 1972 This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published 1972 This digitally printed version 2008 A catalogue record for this publication is available from the British Library Library of Congress Catalogue Card Number: 71—171682 ISBN 978-0-521-08354-6 hardback ISBN 978-0-521-09095-7 paperback

To HERCULES ANGELIQUE DEMETRA STELLA

Contents

Acknowledgement Preface 1

page x xi

On the concept of contiguity and related theorems Summary

1 1

1 Some preliminary definitions and results 2 Contiguity and its relation to other concepts of 'nearness' of sequences of probability measures 3 Alternative characterizations of contiguity 4 Some auxiliary results 5 Proof of Proposition 3.1 6 An additional characterization of contiguity 7 Some results following from contiguity Exercises

2 7 10 18 25 31 33 39

2 Asymptotic expansion and asymptotic distribution of likelihood functions Summary

41 41

1 Preliminaries 2 Assumptions 3 Some examples 4 Asymptotic expansion and asymptotic normality of likelihood functions 5 Some lemmas 6 Proof of theorems of Section 4 Exercise

42 45 47 52 54 63 66 vii

Contents 3 Approximation of a given family of probability measures by an exponential family - asymptotic sufficiency Summary 1 2 3 4 5

4

page 67 67

Formulation of the problem and some preliminary results Some auxiliary results The proof of the theorem Differential equivalence of sequences of probability measures and differential sufficiency Some statistical implications of Theorem 1.1 Exercises

Some statistical applications: AUMP and AUMPU tests for certain testing hypotheses problems Summary 1 Additional assumptions - Examples 2 Some lemmas 3 Testing a simple hypothesis against one-sided alternatives 4 AUMP tests for the examples of Section 1 5 Testing a simple hypothesis against two-sided alternatives 6 Testing a one-sided hypothesis against one-sided alternatives Exercises

Vlll

68 72 76 79 81 84

85 85 86 96 99 105 107 123 127

Some statistical applications: asymptotic efficiency of estimates Summary

128 128

1 W-efficiency - preliminaries 2 Some lemmas 3 A representation theorem

128 131 135

Contents 4

W-efficiency of estimates: upper bounds via Theorem 3.1 page 5 W-efficiency of estimates: upper bounds 6 Asymptotic efficiency of estimates: the classical approach 7 Classical efficiency of estimates: the multiparameter case Exercise 6 1 2 3 4 5 6 7

141 147 157 160 166

Multiparameter asymptotically optimal tests

167

Some notation and preliminary results Formulation of some of the main results Restriction to the class of tests IF Proof of the first main result Proof of the second main result Formulation and proof of the third main result Behaviour of the power under non-local alternatives Exercises

167 169 171 177 184 188 196 197

Appendix 1 Some theorems employed in Chapter 1 2 Some theorems employed in Chapter 2 Exercise 3 A theorem employed in Chapter 5 4 Some theorems employed in Chapter 6 Exercises

199 204 223 224 225 239

Bibliography

240

Index

246

IX

Acknowledgement

Grateful thanks are due to the University of California Press for permission to use the material in Chapter 6, which is a version of a talk given by the author at the Sixth Berkeley Symposium on Mathematical Statistics and Probability. A paper based on this address has appeared in Sixth Symposium of Mathematical Statistics and Probability, edited by Lucien LeCam and published by the University of California Press.

Preface

' Although this may seem a paradox, all science is dominated by the idea of approximation.' BertrvnA Russell

This monograph represents a modest attempt on my part to introduce the concept of contiguity, elaborate on the mathematical theory behind it, and also indicate some of its statistical applications. It lays no claim in containing an exhaustive discussion of results pertaining to contiguity. In fact, there are already new results available which, however, could not have been included in this book. It is simply the result of an attempt to make the concept of contiguity and some of its statistical applications more familiar to several kinds of research workers. These include Theoretical Statisticians, Probabilists, Mathematicians whose primary interest lies in measure theory or approximation theory, and perhaps to practitioners of Statistics as well. It is my belief that contiguity deserves more attention than it has received. I hope that this monograph will be a step in that direction, anticipating the appearance of a more comprehensive treatise on the subject. The concept of contiguity was introduced by Professor Lucien LeCam as a criterion of nearness of sequences of probability measures. In addition to its purely mathematical interest, contiguity is a powerful and very useful tool in Statistics, when one is concerned with asymptotic theory, leading to elegant derivations of the asymptotic properties of tests and estimates under less restrictive assumptions than usual. Of course, this presupposes that one subscribes to the usefulness of large sample theory results. Frequently, however, this is the only alternative open to the Statistician in an unfriendly real world. xi

Preface The concept of contiguity is introduced in Chapter 1 and its relationship to some other modes of nearness of sequences of probability measures is investigated. Some general results which are based on contiguity and which form the backbone of the later chapters are also derived here. In Chapter 2, the statistical model to be employed throughout this monograph is introduced. This is that of a Markov process satisfying certain mild regularity conditions. Of course, the important independent identically distributed case is included in the Markovian model as a special case. Among the assumptions made here the most important one is that of differentiability in quadratic mean of a certain random function. This replaces, in effect, certain of the classical assumptions made in the literature. We refer to those assumptions which deal with the existence of pointwise derivatives up to the third order, and their boundedness by integrable random variables, of essentially the same random function referred to earlier. Under the regularity conditions imposed on the model some general results regarding the asymptotic expansion, in the probability sense, and the asymptotic normality of the loglikelihood function are derived under both a fixed and a moving sequence of parameter points. Incidentally, these derivations attest to the powerfulness and the elegance of the contiguity approach. The essence of Chapter 3 rests on an exponential approximation to the likelihood function with a view to taking advantage of the optimal inference procedures available for the exponential family in a wide variety of problems. The subsequent Chapters 4 and 5 exhibit some statistical applications. Some testing hypotheses problems for real-valued parameters are discussed in Chapter 4. In Chapter 5, the problem of asymptotic efficiency of sequences of estimates is discussed to a certain extent both from the classical point of view as well as along the lines suggested by Wolfowitz. Section 3 of Chapter 4, may be studied directly after Chapter 2. The results included in Sections 5 and 6, however, presuppose knowledge of the material in Chapter 3. In Chapter 6, a multi-parameter testing hypothesis problem is discussed and, finally, in the Appendix various results used in the body of the monograph are gathered together. Most of them are also proved, xii

Preface The first serious thought and attempt to organize the results of this monograph in their present form was made when I was visiting the Mathematics Institute of Aarhus University, Denmark, during the spring semester of 1969. The financial support of the Institute is gratefully acknowledged here. Some financial support was also provided by the University of Wisconsin Research Committee and the National Science Foundation when many of the results included here were written up as Technical Reports. This is also gratefully acknowledged. Some of the material of this monograph has been discussed in seminars both in Aarhus University and the University of Wisconsin in Madison. Comments of the participants in these seminars helped in modifying the original proofs of some theorems. In connection with this, I wish to thank O. BarndorffNielsen, R. A. Johnson, E. SpJ0tvoll and G. K. Bhattacharyya. Many thanks are due to B. Lind for his invaluable contribution in helping clarify and organize various parts of this monograph. Thanks also go to A. Philippou and A. Soms for their many helpful suggestions and constructive comments. W. Davis is also to be thanked for a number of useful comments as well as all those who attended my lectures and contributed their comments. Professor J. F. C. Kingman read the first draft of this monograph on behalf of the Cambridge University Press. His thoughtful comments have contributed to improve the presentation of the material in the monograph. Also the Cambridge University Press itself has been very helpful throughout the editorial and publication process. I extend my sincere thanks to both. Last, but not least, I am grateful to Professor Lucien LeCam who introduced me to the subject of contiguity. Madison, Wisconsin February 1972

GEORGE G. ROUSSAS

xni

1. On the concept of contiguity and related theorems

Summary The main purpose of this chapter is to present the concept of contiguity (see Definition 2.1) introduced by LeCam [4] and study some alternative characterizations of it (see Theorem 6.1). In the process of doing so, some auxiliary concepts such as weak convergence, relative compactness and tightness of a sequence of probability measures are needed. These concepts are introduced in this chapter, as we go along, and also some of their relationships are stated and/or proved. For the omitted proofs, the reader is always referred to appropriate sources. The various characterizations of continguity provide alternative methods one may employ in establishing the presence (or absence) of contiguity in a given case. Some concrete examples are used for illustrative purposes. Contiguity is a concept of' nearness' of sequences of probability measures. It would then be appropriate to relate it to other more familiar concepts of the same nature such as ' nearness' of two sequences of probability measures expressed by the norm (Z^-norm) associated with convergence in variation. By means of examples, it is shown, as one would expect, that 'nearness' of two sequences of probability measures expressed by contiguity is weaker than that expressed by the I^-norm. Some attention is also focused to possible relationships between contiguity on the one hand, and mutual absolute continuity and tightness on the other. In connection with this, it is shown, by means of examples, that mutual absolute continuity of the (corresponding) measures in two sequences of probability measures need not imply contiguity of the sequences. Although the converse is not true either, it is always possible to replace two I

[ 1 ]

RCO

2

Contiguity and related theorems

contiguous sequences of probability measures by two other contiguous sequences whose (corresponding) members are absolutely continuous with respect to one another and these latter sequences lie close to the given ones in the I^-norm sense (see Theorem 5.1). Also it is shown that tightness need not imply contiguity and contiguous sequences of probability measures need not be tight. Finally, a number of important theorems (see Theorems 7.1, 7.2 and their corollaries), based on the assumption of contiguity, are formulated and proved. As will be seen in later chapters, these results are very essential for the statistical applications to be discussed in this monograph. Loosely speaking, these results provide the asymptotic distribution of a sequence of statistics, under a given sequence of probability measures {P^}> ^ ^ ne s a n i e sequence of statistics has a limiting distribution, under a sequence of probability measures {Pn} contiguous to {P^}- For example, {Pn} may correspond to a hypothesis being tested and {P^} to 6 close' alternatives. In order to avoid undue repetition, we should like to mention at the outset that in this chapter, as well as in the subsequent ones, all limits are taken as {n}, or subsequences thereof, converges to infinity through the positive (or non-negative) integers unless otherwise specified. Also, integrals without limits are understood to be taken over the entire (appropriate) space. 1 Some preliminary definitions and results In all that follows, (S,^) is a topological space, where the topology ^ is defined by a metric on S. £f is the topological Borel cr-field generated by ^. Let {J^n} and JS? be a sequence of probability measures and a probability measure, respectively, defined on 5^. Then DEFINITION 1.1 We say that {^n} converges weakly to <£ and we write Sen^Se if J/dJ^-> J/dJ§? for all real-valued, bounded and continuous functions / defined on S. m If (8,ST) = (B»,&") = n &„#,) (m, = 1,2,...), j= l

where {Rj} dSj) = (J?, 3$), the Borel real line, and if Fn and F are the distribution functions (d.f.s) corresponding to o§?n and JSf, re-

Preliminary definitions and results

3

spectively, then weak convergence is equivalent to the convergence Fn(x) ->F(x), xeC(F), the set of continuity points of F (this is a generalization of the Helly-Bray theorem discussed, e.g. in Loeve [1], p. 182; see also Billingsley [4], p. 18). Theorems 1.1A, 1.2 A in the appendix (and also Theorem 1.1 below along with Remark 1.2) provide alternative characterizations of weak convergence. 1.1 Suppose (S,S?) = (Rm,&m) and let (JSfJ and J§? be a sequence of probability measures and a probability measure, respectively, defined on 9 such that j£?n •=> «=§?. Let / be a real-valued, bounded function defined on S such that its restriction to a compact set K is continuous, / vanishes outside K and / need not vanish on the boundary 8K of K, provided SP(dK) = 0. Then jfd^n -> J/djSP. THEOREM

Proof Of course, if/ is continuous, the conclusion holds true whether or not 3?(dK) = 0. Thus it suffices to restrict ourselves to/s which are discontinuous on dK, provided J?(dK) = 0. Since

f /dJ2?B = f /dJSP = 0,

J Kc

J K*

it suffices to show that

f /d^

r

(i.i)

JK

By Theorem 1.1A, &n{K)-+&{K) \iSe{K) = 0, then

since &{8K) = 0. Therefore,

M 0 =

jjdJ?,

where M is an upper bound for | / | on K, and hence (1.1) is true. Thus we assume that 3?(K) > 0 and on 9 n K, define JSf * for all sufficiently large n (so that S£n(K) > 0), and JSf * by

Let (dB)r be the boundary of a set B with respect to the induced (in K) topology. Then it is clear that SB c (dB)r U dK, so that

4

Contiguity and related theorems

JS?*[(&B)r] = 0 implies S?{dB) = 0. Thus ^-continuity sets are also ^-continuity sets. Therefore ££* => J5f *. It follows that f fdJ?n = Xn(K) f /d§?* -> J2?(Z) f /dJ?* = f /dJ2?. JE JB: JK Jz This establishes (1.1) and hence the theorem itself. J COROLLARY 1.1 L e t / b e as in the theorem except t h a t / i s equal to a constant c on Kc and / need not be equal to c on dK, provided &{dK) = 0. Then J/dJS?n -> J/dJS?.

Proof Replace fhjf—c

and apply the theorem. ]

R E M A R K 1.1 By Theorem 1.2 A, it follows that the converse of Theorem 1.1 is also true. That is, if J/dJSfn -> J/dJSf for each/as described in Theorem 1.1, then eS?n => JS?. We recall that {<&„} is a sequence of probability measures defined on £P. Then D E F I N I T I O N 1.2 The sequence {j£?n} is said to be relatively compact if for every subsequence {V} c {^} there exists a further subsequence {V'} c {^'} such that {<£Cn>^) converges weakly to a probability measure (which, in general, depends on {n"}). The following definition will also be useful. D E F I N I T I O N 1.3 The sequence {J^} is said to be tight if for every e > 0 there is a compact set K = K(e) such that

for all n. If (S,^) = (Rm,&m), this condition can be replaced by:

for all n, where [a, 6] is a closed interval in Rm. This is so because any compact set in Rm can be enclosed in a bounded, closed interval, and bounded, closed intervals are compact. The concepts of relative compactness and tightness are related as follows. 1.2 Let(flf,«^) = (Bm,&m). Then {jg?n} is relatively compact if and only if it is tight. THEOREM

Preliminary definitions and results

5

By Theorem 1.3 A, tightness always implies relative compactness. The reverse implication is also true, by the same theorem, provided S is separable and complete. This requirement is, clearly, satisfied in the present situation, where, of course, ^ is assumed to be the usual topology in Rm. So Theorem 1.2 is a special case of Theorem 1.3 A. However, a simple proof of Theorem 1.2 can be presented along the following lines. Proof of Theorem 1.2 Suppose that {j£?n} is tight. Then, for every e > 0, there exists a closed interval in Rm, [a, b], depending on e, such that JS?w([a, b]) > 1 — e for all n. Let Fn be the d.f. corresponding to JSfn. Then by the weak compactness theorem for d.f.s, for every {n'} <= {n} there exists {n"} c= {n'} such that Fn.(x) ->F*(x) for all xeC(F*), where F* is a d.f. in all other respects except that it might fail to have variation equal to one. We shall, actually, show that this does not occur. To this end, let o5f * be the measure induced by F*. Then «J§?* is a probability measure. In fact, =2V([&, &]) > 1— e for all n" and a,b can be taken to be continuity points of F*. Taking the limit as n" -> oo, we have <J§?*([a, b]) ^ 1 — e. Thus, for every e > 0, there exists a closed interval [a, b] in Rm, depending on e, such that &*([a9b])> 1-e, and this implies that «§?* is a probability measure. Hence, tightness implies relative compactness. Now assume that {J§?n} is relatively compact. Then, for every e > 0, we claim that there exists a closed interval [a, b] in Rm, depending on e, such that oSfn([a, b]) > l — e for all n. In fact, if this were not true, then for some e > 0 and for every [a, b] in Rm there would exist a k, depending on e and [a, 6], such that J§?fc([a,b]) ^ 1 — e. Apply this argument for an,bneRm, where an = ( — n,...9 — n)'\ bn = (n,...,ri)' and ' ' ' denotes transpose. Thus, for every n, there exists n' such that <&n>{[an, bn]) ^ 1 — e, where {n'} c {n} and n' -> oo. Then there cannot exist {n") c= {V} such that {J^n*} converges (weakly) to a probability measure J§?, say. This is so because, if F is the d.f. corresponding to JSf, then for every x, y e G(F) with x < y (to be understood in the coordinatewise sense), we have

6

Contiguity and related theorems

where n* is the subscript of the interval associated with n" and the limits are taken as n* -> oo which implies n" -> oo. Thus o§?((#,?/]) ^ 1 — e for every x, y eC(F), which implies that <j£? is not a probability measure. This contradicts our assumption of {&n} being relatively compact. The proof of the theorem is completed. ] Now let {(3?,s/n,Pn)} be a sequence of probability spaces and let {Tn} be a sequence of m-dimensional random vectors such that Tn is ^-measurable. Set J&?^ = &{Tn\Pn). Then the following proposition will prove useful on many occasions. P R O P O S I T I O N 1.1 The sequence {J5fn} just defined is relatively compact (equivalently, tight) if and only if, for every e > 0, there exists b = b(e) > 0 such that i^dl^fl > b) < e for all n, where || • || denotes the usual Euclidean norm in Rm.

Proof By Theorem 1.2, {J§?n} is relatively compact if and only if it is tight. On the other hand, tightness of {JS?W} is equivalent to the existence of an interval [a, b] in Rm, depending on e, suchthatj£?n([a, b]) > 1 — e. This is so by the comments following Definition 1.3. But &n([a, b]) = Pn(Tn e [a, 6]). Thus {Jg?w} is tight if and only if Pn(Tne[a, b]) > 1— e for all n. This, however, is equivalent to the existence of a c(e) = c = (cl9 ...,c m )' with positive coordinates such that Pn(Tn e [ — c, c]) > 1 — e for all n. Taking b(e) = ||c(e)||, we get the desired result. J Let now P and Q be two probability measures on the cr-field •s/ of subsets of X. Then the L^norm of P - Q, denoted by \\P - Q\\, is defined by ||P-Q|| = 2m-p{\P(A)-Q{A)\;Aes/}. Next, if / and g are densities of P and Q, respectively, relative to a dominating cr-finite measure /JL (e.g. fi = P + Q), then

This is shown in Theorem 1.4 A. The following result is isolated here for convenient reference. 1.2 For each n, let Pn and Qn be probability measures on s/n and let {Tn} be a sequence of m-dimensional random vectors such that Tn is j/ n -measurable. Set PROPOSITION

2>r=
and 3* =
Preliminary definitions and results

7

and suppose that ||Pn —Qn|| ~> 0. Then we have

(i)

n^p-^ii^o.

(ii) If jg?£ => Se, a probability measure, then jSfg => Se and conversely. Proof (i) For Be&m, one has so that

(ii) Let 5 be a continuity set of o5f. Then we have

If JS?£ => <J5f, then the second term on the right-hand side of the above inequality tends to zero, by Theorem 1.1A. The first term also converges to zero by (i). Therefore, jSfg => JSf. That J§?# => j£? implies J^^ => ,5? follows by symmetry. The proof of the proposition is concluded. ]

2 Contiguity and its relation to other concepts of 'nearness' of sequences of probability measures In this section, the concept of contiguity is introduced and some alternative characterizations of it are studied. Its relationship to other familiar concepts of 'nearness' of sequences of probability measures is investigated, and, finally, a number of illustrative examples are discussed. Let {(&,£/„)} be a sequence of measurable spaces, and let Pn9 P'n be probability measures onj/ w . Also, let {Tn} be a sequence of random variables (r.v.s) such that Tn is j/ n -measurable. D E F I N I T I O N 2.1 The sequences of probability measures {Pn} and {P^} are said to be contiguous if the following is true: for any j^-measurable r.v.s Tn, Tn-> 0 in Pn-probability if and only if Tn -> 0 in P^-probability. The concept of contiguity is a concept expressing i closeness'

8

Contiguity and related theorems 5

or 'nearness between the sequences of probability measures {Pn} and {P^} m the sense of the definition just given. Some more light is shed on this concept by the fact that, if Anes/n, then Pn(An) -> 0 if and only i£P'n(An) -> 0, provided {Pn} and {P;} are contiguous, as will be shown below. Also some concrete examples will further illustrate the point. It is worth noticing that contiguity is transitive. That is, if {Pn}, {Qn} and {Qn}, {Rn} are contiguous, then {Pn}, {Rn} a r e contiguous. This is an immediate consequence of Definition 2.1. REMARK2.1

P R O P O S I T I O N 2.1 The sequences of probability measures {Pn} and {P^} are contiguous if and only if, for An estfn, Pn(An) -> 0 if a n d o n l y i f P ; ( ^ J - > 0 .

Proof Assume {Pn} and {P^} to be contiguous, and let Pn(An) -> 0 with Ane^n. Set Tn = IAn. Then, for every (l>)e>0,

Pn(\Tn\ >e) = Pn(An),

so that Tn -> 0 in P^-probability. This implies Tn->0 in P'nprobability by Definition 2.1. Since P'J\Tn\ > e) = P'n{An), we have then P^(An) -> 0. The fact that P'n(An) -> 0 implies Pn(An) -> 0 is treated entirely symmetrically. For the converse, we have: let Tn -> 0 in P^-probability and, for e > 0, set An =

(\Tn\>e).

Then Pn{An) -> 0 and this implies P'n(An) -> 0. However, this last convergence is equivalent to the convergence Tn -> 0 in P;-probability. That Tn -> 0 in P;-probability implies Tn -> 0 in Pn-probability follows by symmetry. ] Of course, if j | P n - P ; | | -> 0, then the sequences {Pn} and {P'n} are as close together as they can be, apart from having identical elements. One would then expect that convergence in L r norm would imply contiguity. That this is, actually, the case is seen in the following lemma. LEMMA

guous.

2.1 If | | P n - P ; | | -> 0, then {Pn} and {P;} are conti-

;

Nearness' of sequences

9

Proof The convergence \\Pn — P'n\ -> 0 implies that, for every e > 0, \Pn(An) — P'n(An)\ < e for every Ane R a n d a l l sufficiently large n, n ^ %, say. Thus Pn(^4n) < e implies P^{An) < 2e and Pn(An) < eimpliesPn{An) < 2 e f o r e v e r y A n e R a n d a l l n ^ nv Then Proposition 2.1 applies and gives the desired result. ] R E M A R K 2 . 2 It is demonstrated by means of examples (see, e.g. Example 3.1 (i)) that the converse of Lemma 2.1 is not true. Thus contiguity is a weaker i measure of nearness' of sequences of probability measures than that expressed by convergence in Li-norm. We now consider the following example as a simple application of Lemma 2.1. E X A M P L E 2.1 Let (2C,sfn) = (R,38), Pn = J7(-l/w,l), the uniform measure over ( —1/w, 1) and P'n = U(0, l + l/n), the uniform measure over (0, l + l/n).

Then, if

fn = dPJdl

and gn = dP;/dZ,

where I is the Lebesgue measure in R, we have

\\Pn-K\\ = Thus {Pn} and {P^} are contiguous, by Lemma 2.1. R E M A R K 2.3 Example 2.1 also shows that contiguity need not imply mutual absolute continuity (either for all n or only for all sufficiently large n), as one might be tempted to (wrongly) infer by Proposition 2.1. It will be shown later, however, that any given pair of contiguous sequences of probability measures can always be replaced by another pair of contiguous sequences whose members are mutually absolutely continuous and lie arbitrarily close to the given ones (for this, see Theorem 5.1). On the other hand, mutual absolute continuity need not imply contiguity either, as is demonstrated by the following example. E X A M P L E 2.2

Let

i)

and

where /in -> — oo and fi'n -> oo. Then, clearly, Pn « P'n for all n. (As usual, by P « Q we express the fact that P <^Q and Q <§ P.) Consider the set An defined by

10 An = (/in-l,/in+l).

Contiguity and related theorems Then Pn(An) = c, a positive constant

(c « 0-68), so that Pn(An) does not tend to zero. But P'n{An) -> 0, clearly. Thus, by Proposition 2.1, {Pn} and {P^} cannot be contiguous. R E M A R K 2.4 From this example, we might (wrongly) conclude that the absence of contiguity of {Pn}, {P^} is due to the lack of tightness. (It is clear that the sequences are not tight.) That tightness is not a necessary condition for contiguity is shown in Example 3.1 (i), where the sequences involved are not tight (for a special choice of the parameters) and yet contiguous. In the same example and for another choice of the parameters, it is also shown that tightness is not a sufficient condition for contiguity. Summarizing some of the results obtained so far, we have: If | | P n - P ; | | -> 0, then {Pn}, {P;} are contiguous. The converse need not be true. Contiguity of {PJ, {P^} need not imply Pn « P'n for all or sufficiently large n (see, however, Theorem 5.1). The converse is also true. Contiguity of {Pn}, {P^} need not imply their tightness. The converse is also true. The following remark is relevant here. R E M A R K 2.5 If {Pn} and {P^} are contiguous, but Pn fy P'n, then it follows from Proposition 2.1 that the support of the singular part of Pn with respect to P'n has P^-probability tending to zero, and the support of the singular part of P'n with respect to Pn has P^-probability tending to zero.

3 Alternative characterizations of contiguity Proposition 2.1 provides an alternative characterization of contiguity. Additional ones will be given in the sequel but for their formulation and proof, we need some more notation and also some auxiliary results. To this end, for each n, let Pn, P'n be probability measures on stfn and let /in be a
Alternative characterizations

11

Define the sets Bn, Cn, Dn, En by

and let the r.v. (log-likelihood) An be defined as follows

/J

(3.3)

on Bn and arbitrary (but measurable) on Cn\J Dn[] En. Finally, for each determination of An by (3.3), let Pn)-

(3.4)

Let 8t (i = 1,2, 3), stand for the following statements: (8±) {Pn} and {PQ are contiguous. (52) {&<„} a n ( l {^n} a r e relatively compact (for each determination of An). (53) {^n} is relatively compact (for each determination. of An) and for any {m} c= {71} for which j£?m => JS?, a probability measure (depending, in general, on (J5fn}), we have Jexp A dJ§? = 1, where A is a dummy variable. i In terms of the notation introduced so far, we may formulate the following result. P R O P O S I T I O N 3.1 The statements 8i (i = 1,2,3) given by (3.5), are equivalent. The proof of the proposition is found in Sections 3 and 5. In the meantime, we consider some illustrative examples.

3.1 Let (3F,s/n) = (R938). (i) Here we take Pn = N(/in,
12

Contiguity and related theorems

so that J?n ' \ _ A7 \\fln~lln) n) ~

iV

^=2

L

Z(T

\fln~ ftn) \ >

•

^2 a

n

n

J

Suppose that I——— > is bounded and let the subsequence \ converge to or, say. Then, clearly, ) £>m=>£> = N(- Jo-2, a2)

and

&'m => Jg?'

It follows that {«£^}, {jSf^} are relatively compact and hence {Pn}, {Pn} a r e contiguous on account of the implication S2=> 8V The contiguity of {Pn}, {P^} may also be concluded, on the basis of the implication S3 => Sl9 from the relative compactness of and the fact that

f expAdJSf -

1,

as is easily seen. As a special case, let crn = 1, /in-> JLL, firn -> /^', where both /£, /^' are finite and/£ 4= /^'. Then {Pn}, {P!^ are contiguous but, clearly, Also, let (rn = 1 as above and let /i'n = /in + c, where c is a constant #= Oand/^n-> —oo. Then again {Pn},{P^} are contiguous, but, clearly, they are not tight. Finally, let /i'n = /in + c with c as above, let /in -> /JL,finite,and let orn -> 0. Then {Pn}, {P^} are not contiguous and yet they are tight. (ii) Here we take Pn and P'n to be the probability measures corresponding to one-parameter exponential densities; that is, dP dP' JS A ( A ) ^ A ; ( A » (x > 0). -r^j is bounded from above and also away from 0. Without loss of generality, we may assume that it is bounded away from 1 since if {m} c= {n} is such that A^/Am -> 1, then ||^m~^m|| -> ® ( see Exercise 1). Then we have

Alternative characterizations

13

so that ^ ) exp (A

J

\ A

n~ An I

The domain of y is (log^,oo) if Xn > X'n and (-oOjlog^i if A \ J \ An I Xn < X'n. Let {m} be a subsequence of {n} such that X'mjXm -> A. Then

and

Se'm => .ST:^'

exp ( y ^ l o g AJ exp I - Y ^ 2 / J .

The domain of y is (logA,oo) if A < 1 and ( — 00, log A) if A > 1. Thus we have that {JS^}, {^n} are relatively compact and hence {Pn}, {P^} are contiguous since #2 => Sv The same conclusion can be reached on the basis of the implication $ 3 => Sl9 since flog A Jlc /•log A

for the case that A < 1 and

exp^/dJSf = 1 for the case that

J -oo

A > 1, as is seen by integration. (iii) Let now Pn and P'n be Poisson measures; i.e.

P n* ll{nA\ rji

— pvn / _ ;\ ' \ n •x r _ u 0 x 1 "~ e x P V A n/ | J — ? '

Then we have An = anX + bn, where so that

- A J ^ ; ? / m = ama; + 6m (a; = 0,1,...),

14

Contiguity and related theorems A' x - A ; ) ^ - : yn = anx + bn

(a = 0,1,...).

Suppose that {A^}, {A^} are bounded from above and away from 0. Then, if {m} is a subsequence of {n} such that Am -> A, A^ -> A', it follows that Sem^£e, &'m^&', where J^JS?' are probability distributions over the points y = ax + b, x = 0,1, ..., Sv The implication $ 3 => 8X could also be used since

co

I

= exp ( - A') 2 —. (A exp a) x = exp ( — A') exp (A exp a) = 1. (For another example on contiguity of sequences of probability measures, see Exercise 2.) We recall that the main objective of this section is to establish the equivalences stated in Propostion 3.1. The implication S2 => S± will be shown here (see Corollary 3.2). In the process of doing so, some auxiliary results useful in the sequel will be established. The remaining parts of the proposition are dealt with in Section 5. L E M M A 3.1 Let {&n}, {&'n} and Cn, Dn be defined by (3.4) and (3.2), respectively. Then if {J?n} is relatively compact (equivalently, tight), it follows that Pn(Cn) -> 0. Also, if (JSP;) is relatively compact (equivalently, tight), it follows that

P'niDn) ~> 0.

Proof If {^n} is relatively compact for any determination of An, then it is so for An = — n on Cn. This being the case, suppose that Pn{Cn) 4> 0. Then there exists {m} c {n} such that Pm(Cm) -> 28 > 0

Alternative characterizations

15

and therefore for m ^ some ml9

From this it follows by Proposition 1.1 that {j£?m}, and hence {J2?n},is not relatively compact, a contradiction. Thus the relative compactness of {^n} implies that Pn(Gn) -> 0. The relative compactness of {&„} implies that Pn(Dn) -> 0 by taking An = n on Dn and arguing as above. ] COROLLARY 3.1 If {J^} is relatively compact (equivalently, tight), it follows that Pn{Bn) -> 1, where Bn is given by (3.2). Also, if {&n} is relatively compact (equivalently, tight), then P;(B B )-*I.

Proof It follows from the lemma and the equations 1 = Pn{Bn) + Pn(Cn) = P'n{ LEMMA 3.2 Let j£?n, £P'n be defined by (3.4) and suppose that {jSfn} is relatively compact (equivalently, tight). Then for any Anssfn for which P'n{An) -> 0, it follows that Pn(An) -> 0. Also, if {j5f^} is relatively compact (equivalently, tight), then for any Anes/n for which Pn(An) -» 0, it follows that P'n(An) -> 0.

Proof Let P'n(An) -> 0 and suppose that Pn(An)-\>0. Then there exists {m} c {^} and S > 0 such that P m (^ m ) > 35, m ^ somem2, P'm(Am) -> 0.

(3.6)

The relative compactness of {<&„} implies that of {J§?m}. Thus there exists c = c(S) > 0 such that Pm(|Am| < c) > 1 — 8 for all m, or Pm(Fm) > 1 - S for all m, where i ^ = ( | A m |
(3.7)

The relative compactness of {&„} also implies that Pm(Bm) -> 1 by Corollary 3.1. Thus PJBJ

>l-S

(m ^ somem3).

From (3.6) and (3.8), it follows that PJGJ

>2S

(m ^ somera4),

(3.8)

16

Contiguity and related theorems

where Gm = Am n Bm, and this, together with (3.7), gives that PJHm) >8 (m& m4), where

Hm = FmnGm = Am(]Bmf] Fm.

(3.9)

On Hm, Am = log (gjfm) and also | Am| < c which is equivalent to / m exp(-c)
^ P'JHJ < PJHJexpc.

(3.10)

From (3.9) and (3.10), P'm(Hm) > #exp ( - c), whereas by assumption, P'm{Hm) -> 0 which is a contradiction. Thus, by assuming that P'n(An) -> 0 and utilizing the relative compactness of {J5fn} alone, we were able to conclude that Pn(An) -> 0. The relative compactness of {J£?^} implies, by symmetry, that P'n(An) -> 0 whenever Pn{An) -> 0. ] COROLLARY 3.2 If {JSf^} and {J§^} are relatively compact (equivalently, tight), it follows that {Pn} and {P^} a r e contiguous.

Proof This follows from the lemma and Proposition 2.1.] The proof of the converse of this corollary, i.e. the proof of the implication Sx => S2, as well as that of the equivalence S2 o S3 is considerably easier if Pn « P'n for all sufficiently large n. This, however, need not be true. Our programme then is to replace Pn9 P'n by the probability measures Qn, Q'n such that Qn « Q'n for all sufficiently large n and which are close to Pni P'n, in the sense that ||P n -G n || + \\Pn-Qn\\ ~> 0. In connection with Qn, Q^ one establishes the equivalence and the implication cited above and then one concludes that S2oSz and S2 => Sv For the implementation of these assertions, we need additional notation and also some auxiliary results. For each n, let Qn9 Q'n be probability measures on stfn dominated by the c-finite measure ju,n mentioned in connection with (2.1) (e.g. iin = Pn + P'n + Qn + Q'n)9 and set

Alternative characterizations

17

Next, define the sets Bn, Cn, Dn, En as in (3.2); i.e. B_n = {xe3?;fn(x)gn(x)>0} Cn = {xea?;fn(x)>O,gn(x) =O

(3.12)

and let the r.v. (log-likelihood) An be defined as follows An = l o g ( £ n / / n ) o n 5 n

(3.13)

and arbitrary (but measurable) on Cn U Dn U En. Finally, for each determination of An by (3.13), let cp _ (Pi~K \n \

cp1 — sP'CR \O'\

Let St(i = 1,2, 3), stand for statements similar to namely,

(3.14) (i = 1, 2, 3),

{Qn} and {Q^} are contiguous. ' are {JS?^} and {^n} relatively compact (for each determination of A n ). ($3) {jSf^} is relatively compact (for each determina(3.15) tion of An) and for any {m} <= {^} for which J5fm=>j5f, a probability measure (depending, in general, on {J?^}), we have J exp AdJSf = 1, j where A is a dummy variable. In terms of the notation introduced so far, we may then formulate the following result. P R O P O S I T I O N 3.2 For each n, let Pn, P'n and Qn, Q'n be probability measures defined on stfn, and let oSfn, J§?^ and JSfn, J^f^ be defined by (3.4) and (3.14), respectively. Suppose that

\\Pn-Qn\\ + \\Pn-Qn\\ "> °(3-16) Then for i = 1, 2, 3, Si is fulfilled if and only if Si is, where Si and Si are defined by (3.5) and (3.15), respectively. The proof of this result will follow after some lemmas and propositions have been established in the next section.

18

Contiguity and related theorems

4 Some auxiliary results The following lemma is a variant of Corollary 3.1 in that the relative compactness of {j£?4} is replaced by the assumption that JexpAdJS? = 1. The conclusion is the same. More precisely, one has L E M M A 4.1 Let {i?^}, defined by (3.4), be relatively compact (equivalently, tight) and suppose that for any {m} c= \n) for which J5fm=>i?, a probability measure, one has Jexp AdJSf = 1. Then Pn(Bn) -> 1 and Prn{Bn) -> 1, where Bn is given by (3.2).

Proof In the first place, the relative compactness of {&n} implies that Pn(Bn) -> 1 by Corollary 3.1. We next show that P ; ( £ J - > 1 . We have fexpA n dP^= f expA n dP n +f expAndPn J

J Bn

J Cn

= f ^dP,+ f expA^dP, J BnJn

J Cn

= P'n{Bn)+\

exVAndPn.

JCn

That is,

fexpAndPn = P^(Bn)+ f expA^P,. J

(4.1)

J Cn

This is true for any determination of A^ on Cn and therefore, in particular, for A n — —n. One has then, from (4.1) JexpAdJ2?n = JexpA.dP, = P;(Bw) + exp(-n)Pn(Cw).

(4.2)

In order to show that P'n{Bn) -> 1, it suffices to show that for every {n'} c {n} there exists {m} c {n'} such that P'm(Bm) -> 1. Let {n'} be an arbitrary subsequence of {n}. Then there exists {r} c {n'} such that P'r{Cr) -> a (0 < a ^ 1). By the relative compactness of {&n}, there exists {m} c {/•} such that JSfm => JSf, a probability measure. For any —co<x
J

expAdJ£?m,

J iz,y]

Auxiliary results

19

so that lim inf exp A dJSfm ^ lim inf J

exp A dJ? m

J ix, y]

= lim I

exp A d^m = f

J [x, y]

exp A dJSf.

J [x, y]

The last equality holds because of Theorem 1.1. Thus we have liminf fexpAdjSfm ^ f

expAd.5?.

(4.3)

J [x, y]

J

Letting now x -> — oo, y -> co through continuity points of J§? (see also Exercise 3) and utilizing the dominated convergence theorem, we obtain I

exp AdJSf -> fexp AdJSf = 1.

J[x,y]

(4.4)

J

Relations (4.3) and (4.4) yield liminf f exp AdJ5fm ^ 1.

(4.5)

Now replacing n by m in (4.2) and taking the limits, we obtain on account of (4.5), 1 < l i m m f P ^ J , so that P'm(Bm)-> 1. Therefore P'n{Bn) -> 1, as was to be seen. ] COROLLARY 4.1 The conclusion of the lemma holds with J£?n, Pn, P ; and Bn being replaced by Sny Qn, Q'n and Bni respectively.

Proof Immediate from the lemma. ] The lemma below will be needed in the sequel. L E M M A 4.2 Let/ n , gn and/ n , gn be given by (3.1) and (3.11), respectively, and for any e > 0, define An{e), A'n{e)

/.(*)

-l >e

20

Contiguity and related theorems

where Bn and Bn are given by (3.2) and (3.12), respectively. Then if | | ^ n " G J + IIP-~Qn\\ -> °> itfollows that Pn(An) + P'n{A'n) -> 0 Proof Let Then

/»K

A < K = (1 + e) Pn(Fn). (4.6)

Fn

J Fn

Now suppose that Pn(Fn)-\>0. Then there exists {m} c {^} such that Pm(Fm) -> 25, some 5 > 0. Thus for all sufficiently large m, m ^ ml5 say, Pm(Fm) > S. Then one has from (4.6)

QJFJ-PJFJ

> ePm(FJ >e8 (m > mx).

However, this contradicts the assumption that ||Pw — Qn|| -> 0. Next, let Then it is shown in a similar fashion that Pn(Gn) -> 0, so that Pn(An) -> 0. That P'n(A'n) -> 0 follows by symmetry on account of ||P4~Qn|| ~* ^- Finally, the last assertion of the lemma is immediate. ] The following result asserts that for probability measures which are close together statements 82 and S2 are equivalent. More precisely, one has P R O P O S I T I O N . 1 Consider the statements S2 and $ 2 given by (3.5) and (3.15), respectively. Then if

it follows that S2 and 82 are equivalent. Proof We first show that the relative compactness of {i?^} implies that of {i?^}- The proof is by contradiction. In fact, if {JS?'n} is not relatively compact, there exists an e > 0 and

{mn} c \n)

with mn \ oo such that Q'mn{\Amn| > n) > 8e, n ^ 1. Setting Amn = (|AmJ > »),

(4.7)

Auxiliary results

21

we have then that |A m J > n on Amn Let

and Q'mn{Amn) > Se (n > 1). (4.8)

A+n = (Amn > 7i), ^

= (Kmn < -n).

Then at least one of the inequalities Q'mn(A+J > 4e, Q'mn(A-J > 4e holds true for infinitely many ns, say ni \ oo. For simplicity, set ri ~ mn. and let Fr (i ^ 1), be sets for which one of the inequalities (always the same) Ar. > ni9 Ar. < — ni holds. Of course, Q'rj(Fri) > 4e. In other words, one has Q'r.(Fri) > 4e, and on FreAr.>ni Now

or Ar. < — %

(always the same), (i ^ 1). (4.9)

4e < Q'r.(Fn) = Q'n(Fr. n Brt) + Q'n(Fr. n Dr.).

Hence at least one of the inequalities Q'r.(Fr. n Br.) > 2e, ^ ( ^ n Dr.) > 2e

holds true for infinitely many is, say, i,- f oo. For simplicity, set ri. = ifc3- and consider the following two cases. Gasel That is,

Q'k.(FkMDk.)>2e Q'k.(Gk.)>2e

(j > 1).

(j > 1), Gfc. = ^ . n 5 f c . .

Prom (4.10) and ||P^-Q4|| ^ 0, it follows that for some j

(4.10) v

Now Gk. £ Dk. and Qkj{Dk) = 0. Thus Qft.(Gfc.) = 0 and then \\pn ~ Qn\\ -+ ° implies that Pkj(Gkj) -> 0. That is, for Gk: es/kj, one has Pk.(Gk.) -> 0, whereas P'k.(Gk.) > e (j ^ j^). However, this cannot happen because of Lemma 3.2. Case 2 That is

Q'kj(Fk n Bk.) > 2e 0 > 1). ^(^)>2e

(j > 1), Gk. = Fk.nBk,

(4.12)

From (4.12) and | | P ^ - ^ | | ->• 0, it follows that for some j 2 , P'k(Gk)>e

(j>jt).

(4.13)

22

Contiguity and related theorems

Next, from (4.9), it follows that on Fk, Ak/ > n\ or 7Lk. < —n\ (always the same inequality), where n^ = ^.foo. Suppose first that Ak. > n'j. Then since Gk. g 5fc. on fffc, we have

A*j; = logy,

hj so that Ak. > n'j is equivalent to gk. > fk. exp n^. Integrating both members of this inequality with respect to [ik. over Gk.y one obtains Qkj{Gk.) ^ Qkj(Gkj) exp n] and hence Qkj(Gkj) ^ exp( J -^)Letting j -> oo, we have Qk/(Gk.) -> 0. This result, together with I I ^ - O J "^ °» implies that Pkj{Gkj) -> 0. That is, for GkjejtfkjJ one has P%(Gfc.) -> 0, whereas' P^(Gfcy) > e ( j > j a ), by '(4.13). This is, however, a contradiction on the basis of Lemma 3.2. Next we show that in the present case, the inequality Ak. < — n'^ may not occur except possibly for finitely many Js. In fact, if Ak. < —n'p then on Gk., one has as above that or, by means of (4.12), 2eexp^. ^ Qki&k)- Since n'j->ao as j -> oo, this last inequality, and therefore the original one, Ak. < —n'p cannot be true except possibly for finitely many js. Thus it has been established that (4.8) cannot occur and hence {j£?4} i s relatively compact. That relative compactness of {J?n} implies that of {jSf^} follows similarly. This shows that S2 implies S2. The converse follows by symmetry. ] It is to be noted that the result just established is part of Proposition 3.2. The following results establish another part of the same proposition. P R O P O S I T I O N 4 . 2 Consider the statements £ 3 and Sz given by (3.5) and (3.15), respectively. Then if

it follows that $ 3 and $ 3 are equivalent. Proof From the proof of the previous proposition, it follows that {^n} is relatively compact if and only if {<&n} is so. Let {<J£?n} be relatively compact and let {m} c \n} be such that jSfm => J£?, a probability measure. Then there exists {r} c {m} such that

Auxiliary results

23

3?r => jSf, a probability measure. We shall actually show that J5f = J§?. The proof will then be completed by symmetry. Now in order to show that JSf = jSf, it suffices to prove that Ar - Ar -> 0 in Pr-probability.

(4.14)

7

This is so because ££r => J2 and this, together with (4.14), implies that oSf (Ar| JJ.) => JS? by the standard Slutsky's theorems (see also Exercise 4). This result, together with \\Pn — Qn\\ -> 0 and Proposition 1.2, yields the convergence ^(Ar\Qr) => i?. But Hence j£? = JSf by the uniqueness of weak convergence limits. For the proof of (4.14), let e > 0 be arbitrary and define Ar,A'rhy Ar = Ar(e) = (A,-A, > 6), J ; = A'r(e) = (A r -A r < - e ) . (4.15) Then we have to show that Pr(Ar) + Pr(A'r) -> 0. It will be shown that Pr{Ar) -> 0. The proof that Pr( J;) -> 0 follows in a similar manner. We have

= Pr{Ar n 5 r n Er)+PAA n s r n 4 ) + ^ ( ^ n 5 r n 4 ) +Pr(Ir n 5 r n Er)+p^Ar n cr n 4 ) +pr( Ir n c r n o,)+Pr(Ir n c r n 4 ) jc r nf r ).

(4.16)

From the relative compactness of {=£?„} and Lemma 3.1, one has that Pr(Cr) ->• 0, so that

Pr( 4 . n Or n Er)+Pr(Ir

n Cr n Dr)+Pr(Ir n Cr n 4 ) +P r (4nC r nJ r )^o. (4.17)

Next, Qr(Dr) = Qr(Er) = 0 and hence Pr(Dr) + Pr(Er)-+0 since

n 5 r n 4 ) + W r n 5 r n J r ) -* o.

(4.18)

Now apply Lemma 3.1 with {^Cn} and Cn instead of {SCn} and Cn, which can be done since {&„} is relatively compact. We then have Qr(Cr) -> 0 and this, together with ||Pn — Qn\\ ->• 0, implies that Pr(Cr) -> 0. Therefore Pr(Arf]BrnCr)->0.

(4.19)

24

Contiguity and related theorems

Relations (4.16)-(4.19) imply that Pr{Ar) -> 0 if Pr(Fr)->0,

Fr = lr(]Br(]Br.

(4.20)

We have

Pr(Fr) = Pr[(K-K

>e)0Brf] Br] = Pr Wj > S^j n Br n 5 r J,

(4.21) where S = expe (> 1). Next for 0 < ^ < (8— 1/8) arbitrary, let e2 > 0 be such that (1 + e2j8) < 1 - ev Then P, \\-T > 8—

Pr[Ar(e2)]+Pr [ ( l + e

fy n Br n JSr] (4.22)

provided r is sufficiently large; this is so by Lemma 4.2. From (4.20)-(4.22), it follows that in order to complete the proof, it suffices to show that Pr(Gr)-+0,

Gr

For contradiction, suppose that Pr(Or)-\>0. Then there exists {s} c {r} such that P8(08) -> 280, some 80 > 0, so that Ps(^s)>»o

(*>*x) say.

(4.23)

Now

by Lemma 4.2. Therefore P'r(Gr) -> 0 and hence P'S(GS) - > 0.

(4.24)

However, in view of the relative compactness of {&n}, {^n} and Corollary 3.2, relations (4.23) and (4.24) lead to a contradiction. The proof is completed. ]

Auxiliary results

25

R E M A R K 4.1 It is to be noted that in the proposition just proved, one has that if {r} c: [%} is such that JSfr => JSf, J£?r => JSf, where j£?, ££ are probability measures, then jSf = JSf. We may now conclude the proof of Proposition 3.2 as follows.

Proof of Proposition 3.2 The equivalence of Sl9 Sx follows from Lemma 2.1 and the transitive property of contiguity (see Remark 2.1). The equivalence of S2, S2 is established by Proposition 4.1, and, finally, the equivalence of #3, Ss is the content of Proposition 4.2. J 5 Proof of Proposition 3.1 Consider the statements Si (i = 1, 2,3), given by (3.5) and recall that it is our purpose to prove their equivalence, thus providing additional characterizations of contiguity. In proving these equivalences, one may assume that Pn « P'n for all sufficiently large n. This is so because of Proposition 3.2 and also Theorem 5.1 below. For the proof of this last theorem, the following simple lemma is needed. L E M M A 5.1 Under any one of the statements Si (i = 1, 2, 3), given by (3.5), it follows that Pn(Bn) -+ 1, P'n{Bn) -> 1, where Bn is defined in (3.2).

Proof From (3.2) one has 1 = Pn{Bn) + Pn(Gn) + Pn(Dn) + Pn(En) = Pn(Bn) + Pn(Cn), 1 = P'n{Bn) + P^Cn) + P'n{Dn) + P'n{En) = Pi{Bn) + P'n(D (5.1) Let 8X hold. Then by Proposition 2.1, the fact that

pn(Dn) = p;(cj = o implies that P'n(Dn) -> 0, Pn(Cn) -> 0, so that the conclusion follows from (5.1). Under 82 and $3, the conclusion follows from Corollary 3.1 and Lemma 4.1, respectively. ] THEOREM

5.1 Under any one of the statements Si

(*= 1,2,3),

26

Contiguity and related theorems

as defined by (3.5), there exist sequences of probability measures {Qn} and {Qrn} such that (i) Qn « Q'n for all sufficiently large n. (ii) ||p n -Q w || + | | p ; _ Q ; | | ^ o . _ . (iii) For i = 1,2, 3, St implies 8i9 where the Sfi are given by (3.15). Proof Under any one of the statements Si(i= Lemma 5.1 implies that Pn(Bn)->l

and P;(B n )->l.

1,2,3), (5.2)

For all sufficiently large n (so that Pn(Bn), P'n{Bn) > 0), define the probability measures Qn and Q'n on^/ n as follows:

^n\Bn)i

AnBH

^n\^n)J

AnBn

(5.3) Then we claim that these probability measures satisfy the requirements of the theorem. In fact, (i) Clearly, Qn{A) = 0 if and only if [

fnd/in

= 0. Since

J AnBn

fn > 0 on JS^, this can happen only if [in{A n Bn) = 0. But then Qn^n = 0, which implies that Q;(4) = 0. Thus Qfn < Qn

f J An Bn

and by symmetry, Qn <§ Q'n. (ii) By (5.3), we have

-rf=f'n,

where fa = ——f I

Hence by Theorem 1.4 A,

IK - Qn\ = J |/» -/;! d/^m = ^ L _ J |/ nP B (5 n ) -/ n / B J d^n. By (5.2), it suffices then to show that/ |/ m P ra (£ m )-/ m 4j d^-> 0.

Proof of Proposition 3.1 We have J \fnPn(Bn) -fJBJ

27 dfin

= f fn\Pn(Bn)-IBJd/in+

\fn\Pn(Bn)-IBn\Aixn

= Pn(Bn) - Pl(Bn) + Pn(Bn) Pn{Bl) and this converges to 1 — 12+ 1-0 = 0; that is, \\Pn — Qn\\ -> 0, as was to be seen. The proof of \P'n — Q'n\ -» 0 is entirely analogous, (iii) This is a consequence of (ii) and Proposition 3.2. ] We may now proceed with the proof of the equivalence of the statements Si (i = 1,2, 3). To this effect, we have P R O P O S I T I O N 5.1 Suppose that Pn « P'n for all sufficiently large n. Then the statements Si (i = 1,2, 3), given by (3.5), are equivalent.

Proof (i) S±oS2. That 82=> 8X follows from Corollary 3.2 (without making use of the assumption Pn « P'n). The converse is shown as follows. For ceB and all sufficiently large n, we have Pn(An >c)=(

dPn=[ J (A«>c)

exp ( - A J d P ; J (An>c)

exp(-AJdP; < exp ( - c) P'n{Kn > c) ^ exp ( - c). (The second equality from the left is where the assumption Pn « P'n is used.) Thus, replacing c by (0 < ) cn f oo, we have Pn(An>cn)-*0,

(5.4)

which, by contiguity and Proposition 2.1, implies P'niK > On) "> 0.

(5.5)

In a similar way, we prove that P;(Afc<-cn)->0,

(5.6)

which, again by contiguity and Proposition 2.1, implies Pn(An<-cn)->0.

(5.7)

28

Contiguity and related theorems

Relations (5.4) and (5.7) imply that Pn(\K\ >CJ->O. Therefore, for every e > 0, there exists 6* = b*(e) > 0 such that P n (|A n | > &*) < e for all sufficiently large n. Increasing 6* to take care of the finitely many exceptional ws and setting b — b(e), we have that P n (|AJ > b) < e for all n. This establishes the relative compactness of {J5fn}, by Proposition 1.1. The relative compactness of {i?^} is concluded in a similar fashion, by using relations (5.5) and (5.6). (ii) S2oS3. We first show that S2=> Ss. By the tightness of {J2^}, we have that, for every e > 0, there exists a compact set K = K(e) such that £"n{K) > 1 - e for all n and 3?(dK) = 0 (see expAdjS?^ > 1 — e for JK all n, by the fact that &'n(K) = \ expAdJS?n. (Here is one JK instance where the assumption Pn « P'n is employed.) Now j£?m=>J§? implies expAdJ? m -> expAdJSP, by Theorem 1.1. JK JK Therefore „ Exercise 3). This can be rewritten as

expAdJSf ^ 1 - e . JK Next, J exp AdjSfn = J &P'n = 1 and hence f exp AcLS?n < 1

for any BE38.

(5.8)

(5.9)

JB

Let {Kn} be a sequence of compact sets such that Kn \ R and such that JS?{dKn) = 0 for all n (e.g. take ^Tn to be closed intervals). Then 0 ^ expA/g- (A) f expA and the monotone convergence theorem implies f expAdJSf-> fexpAd«Sf. J Kn J But

exp AdJSf = lim

(5.10)

exp AdJ5fm as m -> oo, and therefore

Proof of Proposition 3.1

29

expAcLS? ^ 1 for all n, by (5.9). Relation (5.10) then gives that Jexp AdJSf ^ 1. We now claim that Jexp AdJSf = 1. This is a consequence of (5.8). The proof of the implication 82=> S3 is complete. Next, we show that Ss => 82. To this end, let {m} be an arbitrary subsequence of {n}. Then by the relative compactness of {Sfn}, there exists {r} c {m} such that JSfr => JSf, a probability measure. Since <£'r = exp AJ?r, we have that for all real-valued, continuous functions/on E each of which vanishes outside a compact set, f/dJ^; = f/expAdLS?r-> f/expAdJ§?= f/dJS?', where the probability measure J§?' is denned by JS?' = So oSf^. => JSf' and the proof is completed. J We may now conclude the proof of Proposition 3.1. Namely, Proof of Proposition 3.1 As was mentioned in the course of the proof of Proposition 5.1, the implications 82 ==> S1 has already been established in Corollary 3.2. Next in discussing the equivalences under consideration, there is no loss of generality by assuming that Pn « P'n for all sufficiently large n. This is so because of Theorem 5.1 and Proposition 3.2. Thus the proof is concluded by Proposition 5.1. This section is closed with an alternative proof of the implication 82 => S± (see Corollary 3.2) by making use of the additional assumption in Proposition 5.1 that Pn « P'n for all sufficiently large n. In the course of the proof we utilize a certain fact which also has an interesting statistical interpretation. Consider Anes/n with 0 < Pn{An) < 1. Then there exists a test 0W defined by (5.11) 10

if

where Ln = dPjJdPn, and 0 < yn < 1 and cn are determined so that j(pndPn = Pn(An), with the property that > P'niAn).

(5.12)

30

Contiguity and related theorems

In order to see this, we set Pn{An) = ocn and, for each n, we consider the problem of testing the hypothesis #*: the underlying probability measure is Pn against the alternative A*: the underlying probability measure is P'n, at level of significance an. Then the test defined by (5.11) is most powerful of level ocn and its power, J3n, is j(f>ndP^. Now, P'n{An) must be ^ fin because otherwise the test 0* defined as follows on

An9

on

Acn

is of level J ^ d P ^ = Pn(An) = an, and has power J>*dp; = p'n{An) > fin. But this contradicts the optimality of (j>n, by the NeymanPearson fundamental lemma. Therefore J ^ d P ^ ^ P^(An), as was to be seen. J We are now in a position to prove the following result. Alternative proof of Corollary 3.2 (S2=>Sx) under the additional assumption that Pn « P'n for all sufficiently large n. Let Ane<s/n be such that Pn(An)->0. We will show that P'n{An) -> O.By (5.12),it suffices to show that J^ w dP;-> 0, where n is defined by (5.11). For 1 < c ( < oo), we have

f

(Ln>c)

dp; )

Now, by the relative compactness of {jSf^}, we can choose c sufficiently large, so that P^(|AJ > logc) < e for all n. This is so by Proposition 1.1. Since also Pn{An) -> 0, we get j^>ndP^ < 2e. Thus P'n{An) -> 0. By symmetry, P;(^ n ) ^ 0impKesPw(iln) -> 0. Then Proposition 2.1 applies and completes the proof. ]

Proof of Proposition 3.1

31

R E M A R K 5.1 Let Anestfn with 0 < Pn(An) < 1, and set an = Pn(An). Now suppose an -> 0 and, for each n, consider the problem of testing the hypothesis #*: the underlying probability measure is Pn against the alternative A%: the underlying probability measure is P'n, at level of significance ocn. The optimal test is defined by (5.11) and its power, J$n, is fin = ^ndP'n. Since an -> 0, it would be reasonable to expect that jSn -> 0. The presence of contiguity does secure that this happens. To see this, suppose {Pn}, {P^} a r e contiguous. Then {JS^}, {^n} are relatively compact, as has been shown, and /?n -> 0, as was seen in the course of this proof.

6

An additional characterization of contiguity

In this section, we establish a fourth characterization of contiguity. We also present the four characterizations of contiguity together for easy reference. In proving the proposition below, it is to be noted that the (unnecessary but convenient) assumption Pn « P'n is not required. P R O P O S I T I O N 6.1 Let {Tn} be any sequence of m-dimensional random vectors. Then the sequences {Pn} and {P^} are contiguous if and only if the following happens: whenever {J£(Tn\Pn)} is relatively compact (equivalently, tight), so is J P ; ) } , and conversely. Proof Suppose {Pn} and {P^} are contiguous and assume that {^(Tn\Pn)} is tight. Then we have to show that {jSf (Tn\P'n)} is also tight. The tightness of {2?(Tn\Pn)} implies that for ei 10, there exists 6i = &^(ej f oo such that Pn(\\Tn\\ > bt) ^ e, for all n,

(t=l,2,...).

(6.1)

Now suppose that {^f(Tn\P^)} is not tight. Then there exists e > 0 and {wj c: \n) with ni \ oo such that P'ni{\\Tni\

> bt ) > e

(»=1,2,...).

(6.2)

Set An. = (||Tm.|| > bt). Then relations (6.1) and (6.2) imply that, aS

*~>°0>

Pni(An)^0

while

P^(Ant) + O.

(6.3)

32

Contiguity and related theorems

Now, since contiguity of {Pn} and {P^}, clearly, implies contiguity of {PnJ and {P^J> we arrive at a contradiction to (6.3), by Proposition 2.1. Therefore {^(Tn\P'n)} is tight. That tightness of {^{Tn\P'n)} implies tightness of {&(Tn\Pn)} is proved in a symmetric way. In order to establish the other direction of the proposition, assume that {££(Tn\Pn)} is tight if and only if {JS?{Tn\P'n)} is tight, and suppose that Vn -> 0 in P^-probability; here it suffices for {1^} to be a sequence of r.v.s such that Vn is j/^-measurable. We would like to show that Vn ~> 0 in P^-probability. Suppose that Vn-\>0 in P^-probability. Then there exists {n'} c {n} and e > 0 such that

nf /\T7 \

\

r

n/

m A\

Pn>^yn\ > e) ^ e for all n'. (6.4) Consider {V}. Then, by the fact that T^, -> 0 in Pn.-probability, it follows that there exists {m} c {V} and am | 0 such that Pm(K\ > O " > 0 .

(6.5)

Set Vm = Fm/am. Then {^(?m\Pm)} is, clearly, tight by virtue of (6.5). But 6 ^ P;(|F m | > e) = P'm(\?m\ > ejam) on account of (6.4), and therefore {^(ym\Prm)} cannot be tight by Proposition 1.1. Setting Vn = Vm for n = m and equal to zero otherwise, we have that {J§?(Vn\Pn)\ is tight while {^{fn\P'n)} is not. This, however, is a contradiction to our assumption. Therefore Vn -> 0 in P'nprobability. That Vn ~> 0 in P^-probability implies Vn -> 0 in P^-probability, follows by symmetry. Thus {Pn} and {P^} are contiguous and the proof of the proposition is completed. ] R E M A R K 6.1 According to the proposition just proved, contiguity of {Pn} and {P^} expresses nearness of these sequences in the sense that they produce distributions, which either simultaneously do not escape to infinity or they simultaneously do. This section is closed with a theorem, where the various characterizations of contiguity are gathered together for easy reference. T H E O R E M 6.1 Let JS?n, <e'n be defined by (3.4), and let {Tn} be any sequence of m-dimensional random vectors such that Tn is j^-measurable. Then the following statements are equivalent.

An additional characterization

33

(i) {Pn} and {P^} are contiguous. (ii) For An e s/n9 Pn{An) -> 0 if and only if P'n{An) -> 0. (iii) {JS?n} and {J§?^} are relatively compact (equivalently, tight). (iv) {&n} is a relatively compact (equivalently, tight) and if [ni] s= {n} is such that 3?m =>«§?, a probability measure, then JexpAdJSf = 1. (v) {J?(Tn\Pn)} is tight if and only if {J?(Tn\P^)} is so. Proof It follows from Proposition 3.1 and Proposition 6.1. J All four characterizations of contiguity summarized in Theorem 6.1 have been mentioned by LeCam in [4], where there is also some discussion regarding their equivalence. In connection with this, the reader is also referred to Roussas and Soms [8], Roussas [6] and LeCam [8].

7 Some results following from contiguity In this section, we discuss some important consequences of the concept of contiguity with a view of using them in statistical applications. Although their statistical significance may not be apparent in the present context, it will become so in the subsequent chapters. In some of the following derivations, we make use of the convenient but unnecessary assumption Pn « P'n for all sufficiently large n. Under this assumption An = log (dP^/dPn) is well-defined a.s. [PJ, \P'nl Recall that &n = Se(AJPJ, J£'n = Jg?(An|P^). Next, let{Tn} be a sequence of ^-dimensional random vectors such that Tn is j^-measurable and set J?n = J?[(An,Tn)\Pn],

(7.1)

(Here and in the sequel, by (An,Tn) we mean the (&+ l)-dimensional random vector (An, T^)'.) Then the first main result of this section relates to the determination of the (weak) limit of the sequence {.J^} in terms of the (weak) limit of {J^}. More precisely, we have the following theorem. Suppose {Pn} and {P^} are contiguous and let ££n and J^' be defined by (7.1). It is further assumed that THEOEEM7.1

34

Contiguity and related theorems

&n=>J£, a probability measure. Then «^ => J£\ £' = exp AJ^ in the sense that d ^ ' / d J ^ = exp A.

where

Proof In this proof we work with sufficiently large n9 so that the assumption Pn « P'n is satisfied. The contiguity assumption of {Pn} and {P^} implies that {J§?n} is relatively compact. This is so by Theorem 6.1. Therefore there exists a subsequence {m} c: \n) such that £?m=>3?, a probability measure, and JexpAdJS? = 1 by the theorem just cited. On the other hand, by setting £ J^(A, t), we have that &n => £ implies JSfn => J^(A), where is a marginal of J^(A,J). Hence «^(A) = JS?(A) = <J5f, by Theorem 1.5 A, and therefore

f

(7.2)

Next, let / be real-valued, continuous, defined on Rk+1 and vanishing outside a compact set. Then we have

; = J/(A, 0 d^[(Aw, Tn)\P£ = J/(A n , Tn

= f/(A, where JS?' = expAJ^. By (7.2), it follows that J^' is a probability measure. Since J/dJ?^ -> J/dJ^' for all real-valued, continuous functions defined on Bk+1 each of which vanishes outside a compact set, one has that 3?'n •=>£?', as was to be shown. ] To this theorem we have the following two important corollaries. COROLLARY

7.1 If j£?(An|PJ => N(ji, cr2), then /^ = -|cr 2 .

Proof By taking 2^ = 1 in the theorem, we have that => exp AN(/i, a2) and hence

Some results

35 2

- I = 1, so that [i = — §
was to be seen. ] COROLLARY

7.2 If J^(A n |PJ => iV(-±
Proof By taking again ym = 1 in the theorem, we have that &{An\P'n) => JSP', where .2" = J*'(A) is a marginal of and, by Corollary 7.1, its d.f. F' is given by

F'{x)=[X J -0

-2/2)2

Thus J§?' is iV(|cr2, iV^O, Y), where V is a non-singular Jcxk covariance matrix, while

and An-h'Tn ~> -\
36

Contiguity and related theorems

L E M M A 7.1 Suppose that P'n depends on h e Rk in a specified way. Let {T^} be a sequence of ^-dimensional random vectors such that Tn is j/ n -measurable, and assume that the following convergences hold true:

&(Tn\Pn)^N(0,T),

(7.3)

where F is a 1c x k non-singular covariance matrix, and An - h'Tn -> - |(r 2 in P^-probability, 2

(7.4)

2

where a = cr (h) = hTh. Then &[(An,Tn)\Pn] converges (weakly) to a ((fc+ l)-dimensional) normal law with mean and covariance given by (-|o- 2 ,0, ...,0)'

and

(^

r

j , respectively.

Proof In order to somewhat simplify the notation, we write c for — |cr2. For ueR and v e R k , we Have mAn + WTn = iu(An - h'Tn - c) + i(v + uh)' Tn + me. Then

4= = J {exp [iu(An - h'Tn - c) + i(v + %A)' Tn + me] - exp [i{v + wA)' y n + me]} dP = I Jexp [i(t> + M*)' r w + iuc] {exp [m(An - ATn - c) - 1]} dP n

J|exp(mZ w )-l|dP n , where we set

Z n = A n - A'Tn - c.

But, for e > 0,

f |exp (mZn) - 11 dPw = f

exp ( M J - 11 dPn

+f J(|Z«l>e)

(7.5)

Some results

37

ption Since this is true for every e > 0, equation (7.5) and assumption (7.4) show that 4 > Q (76) Next relation (7.3) (see also Exercises 4 and 5) implies that Se\(v + uh)' Tn + uc\Pn] => N[uc, (v + uh)' T(v + uh)] from which we obtain exp [i(v + uh)' Tn + me] dPn -> exp [iuc - \{v + uh)' T(v + uh)]. (7.7) Relations (7.6) and (7.7) taken together imply that the characteristic function of (An, Tn) converges to the function exp [me - \(v + uh)' F(v + uh)]. But

(7.8)

uc = (U\ (c, 0,..., 0)' and (v + uh)' F(v + uh) _M'/cr2

-\v) \Th

AT\ /'u\

r)[v)'

as is easily seen by utilizing the fact that cr2 = h'Yh. Therefore the function in (7.8) is the characteristic function of the (k + 1)-dimensional normal distribution with mean (c, 0,..., 0)' and covariance . 2 ,™

(r/a

ryj

and this completes the proof of the lemma. ] For later use, we denote by ££ the (k+ 1)-dimensional normal distribution with mean and covariance as above, that is, (7.9) By an application of Theorem 7.1, one has the following corollary to the lemma just proved. COROLLARY 7.3 Let <£ be defined by relation (7.9). Then, if {PJ, {P'n} are contiguous,

where J^' =

38

Contiguity and related theorems

We are now in a position to prove the second main result of this section. 7.2 Under the same assumptions as those in Lemma 7.1, and the condition that {Pn} and {P^} are contiguous, THEOREM

(heRk).

&(Tn\P'n) => N(Th, T)

we have

Proof It is convenient to use also the notation J^(A,£) for the normal distribution defined by (7.9), J^(A|£) for the conditional distribution, given t, and J&(t) for the normal distribution with mean 0 and covariance F. It is well known that J^(A|£) is normal with mean given by

-ic^ + SuSi^-O) = -ihTh + hTT-H = -tyTh + h't and variance given by

= -hTh + hTh = 0. (For this, see, e.g. Rao [3], p. 441 (v).) We have then

fexpAdJ^ = fexpAJ^(dA,d/) = fexp (7.10) Taking into consideration the form of J^(A|£), the first integration in (7.10) with respect to A over the interval (— oo, A'], gives

T expA^(dA|<) = («P(-**T* + *'O if A' > -WTM't J -a ' \0 otherwise. Next

P'n{An < A', Tn < t') -> fA

f'

exp A^(dA, At),

J — oo J — oo

by Corollary 7.3, and this implies that J — oo J —o

= f [ f" J -oo LJ - o

=f J -0

Some results

39

where |F| denotes the determinant of F. Therefore as was to be seen. | R E M A R K 7.2 In [1], p. 202, Hajek and Sidak define contiguity as follows: {P^} is said to be contiguous to {Pn} if, for Anes/n, Pn(An) -> 0 implies P^(An) -> 0. However, in this work we have adopted and used throughout the more symmetric definition of contiguity as was introduced by LeCam [4], p. 41.

Exercises 1. Consider the measurable space (Q, ^) and let P, Q be two probability measures on IF dominated by a or-finite measure [i (e.g. ju, = P + Q). Set / = dP/d/i, g = dQjd/JL and define the quantities d and p as follows

d=\\P-Q\\=j\f-g\d/t,

p=j(fg)id/i.

2(1 -p 2 )* ^ d > 2(1 -p).

Then show that

(See, e.g. Lemma 1 in Kraft [1].) 2. For each integer n ^ 1, consider the measurable space (&,£/n). Also consider the probability density f{x d)

''

=

WTT)x9

(o<*0>-i)

and for 6n -> 2, 6'n -> 0, let P6n, P$n be the probability measures o n j / n determined b y / ( - ; 0n),f(-; 6'n), respectively. Determine whether or not the sequences {Pn} and {P^} are contiguous. 3. Consider the probability space (£i,^",P) and suppose that (Ai} ie I) is a collection of events such that At ft Aj = 0 for i =f= j ; I is an uncountable index set. Then show that P ( ^ ) is positive for only countably many events. 4. For each integer n ^ 1, let Xn, Yn be r.v.s defined on the probability space (Q,n, e^, Pn) and let X be an r.v. defined on the probability space (£}, J*\ P). Suppose that JSf (Xn\Pn) => &(X\P). Then:

40

Contiguity and related theorems (i) If Yn — Xn -> c in P^-probability, it follows that

(ii) If Yn -> c (> 0) in Pn-probability, it follows that 5. For each integer n ^ 1, consider the probability space and let Tn be an j^-measurable ^-dimensional (Qn^n,P^) random vector such that J?(Tn\Pn) ^N(/i, F). Then for any (constant) i-dimensional vector c, one has that

2. Asymptotic expansion and asymptotic distribution of likelihood functions

Summary In this chapter, some fundamental results regarding the asymptotic expansion, in the probability sense, and also the asymptotic distribution of certain likelihood functions are derived. These results constitute the backbone of the remaining chapters in this monograph and their derivation rests heavily on the material discussed in Chapter 1. The underlying probability model involved in our discussions is that of a Markov process satisfying certain reasonable regularity conditions. This model includes, of course, as a special but important case the model consisting of independent identically distribution (i.i.d.) random variables (r.v.s) which is assumed more often in statistical literature. We now proceed to present a brief outline of what is done in this chapter, since the various derivations are rather involved and the reader might lose sight of the essence of the results. In Section 2, we gather together the various assumptions which are used in the present chapter and which also are basic for what is discussed in the subsequent chapters. The new element here is the assumption of differentiability in quadratic mean of the square root of the probability density function. It replaces the assumption usually made in statistical literature about the existence of two or three pointwise derivatives of the logarithm of the density. As is shown by LeCam [6], the classical Cramertype assumptions imply the one made here. The underlying conditions are then verified in a number of examples which are used throughout this monograph for illustrative purposes. Additional examples where these conditions were found to hold true may be found in Lind and Roussas [1] and Philippou [1], [ 41 ]

42

Asymptotic expansion and distribution

In anticipation of the statistical applications discussed in Chapters 4, 5 and 6, it should be mentioned here that the covariance function T(d) defined by (2.1) plays the role of Fisher's information matrix I{6) under the standard assumptions. Also the random vector An(#) defined by (4.2) replaces the maximum likelihood estimate when 6 is replaced by a known parameter point OQ. Section 4 is concerned with the asymptotic behaviour of the log-likelihood function corresponding to an arbitrary but fixed parameter point 6 and a 'neighbouring' point. It is shown in Theorem 4.1 (see also Remark 4.1) that asymptotically and in the neighbourhood of any parameter point the likelihood function behaves as if it were an exponential family. Actually, this is an involved assertion and its precise formulation and rigorous justification is the main objective of Chapter 3. The statistical implications of this fact are apparent: in the neighbourhood of each parameter point and for certain statistical purposes, the given family of probability measures may be used as if it were an exponential family. Prom a statistical point of view, one is, clearly, interested in the asymptotic distribution of the log-likelihood function and the random vector An(#) mentioned above. These distributions are provided by Theorems 4.2-4.6. The proofs of these theorems are given in Section 6 and they are based on a series of lemmas established in Section 5. 1 Preliminaries In this section, we introduce the basic notation and definitions required for the formulation of the assumptions made below. To start with, let ( « ^ , J / ) be a measurable space and suppose 0 is a ^-dimensional open subset of the Euclidean space Rk, Jc a positive integer. The underlying topology in Rk is the usual one. The pair (Tt,88) stands for the Borel real line and for each de®, one assumes the existence of a probability measure, denoted by pe( •) and defined on 8% and also of a transition probability measure pe( •, •) defined on Rx&. Then by the Kolmogorov consistency theorem (see, e.g. Loeve [1], p. 93),

Preliminaries

43

pe( •) and pd( •, •) determine a probability measure on the pro00

duct cr-field

n ^

00

of subsets of n Rp where Rj = R and

j=I

j=I

^ = «^? for all j . Let P# denote the so-defined probability measure. Without loss of generality (see, e.g. Doob [1], p. 14, and other references cited there), we may and shall assume from now on that (SC^) = ( f[ Rj9 U ®\ • Thus for each 6 e 0, one has the probability space (3£,£/,Pe). Next let {Xn}, n ^ 0, n an integer, be the coordinate process defined on (3C9s/)\ i.e. -3Tn(G>) = xn9 where co = (xl9x2,

...)e&.

Then it is well known that for each 6 e &, this process is a Markov process with initial distribution pd( •) and transition measure pd{ •, •). This is the stochastic process we are going to deal with from now on. Suppressing the random element G), let g(d) be an r. v. defined on (3£,stf,P) for each 6e®, where P is some probability measure on jtf. Then we can give the following definition which will be needed in the formulation of our assumptions. 1.1 Suppose jg2(0) dP < oo. The random function (r.f.) g is said to be dijferentiable in quadratic mean (q.m.) at 6 when the probability measure P is employed if DEFINITION

^\g(O + h)-9(0)-hW)]->O

in q.m. [P] as (0 4=) ||A|| -> 0,

where || • || is the usual Euclidean norm and g(6) is the derivative in q.m. ( a i x l r . vector) oig{6) at 6. Here and in similar situations, ' ' ' denotes transpose and h'g(d) is, of course, the inner product of the vectors indicated. R E M A R K 1.1 It is clear that Definition 1.1 is equivalent to the following one, which is used more conveniently in actually checking differentiability in q.m. Namely,

1.1' Suppose jg2(6)dP < oo. Then the r.f. g is said to be dijferentiable in q.m. at 6 when P is employed if DEFINITION

\ [g(d + Ah) - g{6)] -> h'g{6) in q.m. [P] as

A

(0 < ) A -> 0

44

Asymptotic expansion and distribution

uniformly on bounded sets of values of h; again g(6) is the derivative in q.m. oig{6) at 6 (see also Exercise 1). Now, in the case that 0 is a one-dimensional set, g(6) is simply an r.v. Then it is clear that Definition 1.1 is equivalent to the following one. 1.1" Suppose jg2(6)dP < oo. Then the r.f. g is said to be differentiable in q.m. at 6 when P is employed if DEFINITION

\[g{0 + h)-g{d)-]->g{6) in q.m. [P] as (0*)A-*0, where g(6) is the derivative in q.m. of g(d) at 6. Let now j / n be the cr-field induced by the r.v.s {Xo, Xlt..., Xn} and denote by Pn e the restriction of Pe to stfn. It will be assumed below that for any two 6,6*G0, Pnd and Pn>6* are mutually absolutely continuous for all n. Therefore we may set M]

= q(X0, X x ; 6, d*),

where q(X0; 6,0*) and q{X0,Xx; 6,6*) are well-defined except perhaps on a null set. We also set X,_ i; 6,6*) = ? (X,_ lf Xf, 6,0*)/g(Z,_i; 0,5*) and

0,(6,6*) = [q(Xi\Xj_1;6,6*)]h.

(1.2) (1.3)

It follows that

) fi fiV) Of course, the quantities defined by (1.1) and (1.2) are simply the initial and transition density, respectively, of the process, both considered as r.v.s (which are obtained by replacing the arguments Xj_x and Xj by the r.v.s X^x and Xjy respectively). Then the quantity defined by (1.4) is the (n+ 1)-dimensional density of the process, again considered as an r.v.

Assumptions

45

2 Assumptions In this section all of the assumptions, under which various results are going to be derived, are gathered together for easy reference. (Al) For each de®, the Markov process {Xn}, n ^ 0 is (strictly) stationary and metrically transitive (ergodic). (A2) The probability measures {Pn6; de®} are mutually absolutely continuous for all n ^ 0. (A3) (i) For each de®, the r.f. fa&d*) is differentiable in q.m. with respect to 6* at the point (d, 6) when Pe is employed. Let 4>x(6) be the derivative in q.m. of (fix(d, 6*) with respect to 0* at (0,0). Then (ii) ^j(0) is ja^ x ^-measurable, where ^ is the cr-field of Borel subsets of 0. Let F(0) be the covariance function defined by (2.1) Then (iii) T(d) is positive definite for every de®. (A 4) (i) For each de®, q(X0, Xx, 6, 0*) -» 1 in P1§ ^-probability as 0* -> 6. (ii) For each fixed de®, q(X0; d, d*) iss/ Q x ^-measurable and q(X0, Xx; d, 0*) is s/x x ^-measurable. COMMENTS ON T H E A S S U M P T I O N S For the definition of strict stationarity and ergodicity, the reader is referred to, e.g. Doob [1], p. 191 and p. 460. Although (A 1) is a basic assumption, (A 2) is made to simplify the derivations. One could dispense with it. From the definition of ^(0,0*) by (1.3), one notices that j(pl(d, 0*) dP0 is finite and indeed equal to 1; therefore it makes sense to talk about differentiability in q.m. (see Definition 1.1). Actually this is the reason for considering the square root of the transition density rather than the density itself. Furthermore it is felt that differentiability in q.m. is more probabilistic an assumption than the classical assumption of pointwise differentiability. The remaining parts of (A3), are in the nature of regularity conditions which are likely to be satisfied in most

46

Asymptotic expansion and distribution

cases of practical importance. Finally notice that in forming the quotient p r [^(0,0 + K)- &(0,0)] (or i [^(0,0 + AA) - ^(0,0)]) the term ^ ( 0 , 0) = 1. As for (A 4) (i), this is evidently an exceedingly mild restriction on the two-dimensional density of the process. Assumption (A4) (ii) is of the same nature as (A3) (ii) and is, actually used only in the proof of Lemma 7.1 in Chapter 5. Assumptions (A 1-4) take a simpler form if the underlying process consists of i.i.d. r.v.s. They are listed below for the sake of record. In the first place, (A 1) is automatically satisfied. For this, the reader is referred to Doob [1], Example 2 and Theorem 1.2, p. 460. Next, in the present case, clearly, h(d,d*) = [q(Xj;d,d*)]i

(2.2)

and (A4) (i) follows from (A3) (i), since differentiability in q.m. obviously implies continuity in probability. Thus the simplified assumptions are as follows. (AT) The probability measures {Pnt6; de®} are mutually absolutely continuous for all n ^ 1. (A2')(i) For each 0e@, the r.f. ^(0,0*) defined by (2.2) is differentiate in q.m. with respect to 0* at the point (0,0) when Pe is employed. Let x(d) be the derivative in q.m. of ^(0,0*) with respect to 0* at (0,0). Then (ii) <j)x{d) is X\\38) x ^-measurable. Let F(0) be the covariance function defined by (2.1), where ^(0) is given by (2.2). Then (iii) F(0) is positive definite for every 0e 0. (iv) For each fixed 0 e 0 , q{X0; 0,0*) isstf0 x ^-measurable. In order to avoid trivial repetition, in what follows limits are always taken as {n}, or subsequences thereof, converges to oo unless otherwise specified. All results in this monograph will be derived under assumptions (A 1-4) (or (Al'-2') for the i.i.d. case) and this will not be mentioned explicitly again. Whenever additional assumptions are required, they will be spelled out explicitly.

Some examples

47

3 Some examples In this section, a number of examples are presented which are shown to satisfy assumptions (A 1-4), or (A l'-2') for the i.i.d. case. Some versions of these examples are also discussed in Roussas [2] and Johnson and Roussas [1]. In showing differentiability in q.m., Theorem 2.1A (Vitali's theorem) is instrumental. 3.1 Let {Xn}, n ^ 1 be i.i.d. from i^(6>,cr2), deR, where
The (pointwise) derivative of
Also where

But so that and therefore

->-—-z 4(T

2

as

h -> 0.

48

Asymptotic expansion and distribution

This result together with (3.1) and Theorem 2.1A, implies that 1(d)=±(X1-d).

(3.2)

Furthermore by (2.1) and (3.1), T(6) = tfAfaO)]* = ^ ,

(3.3)

so that (A2') (i)-(iii) are fulfilled. 3.2 Let {Xn}, n ^ 1 be i.i.d. from N(/i, 6), 6 > 0, where /i is assumed to be known. Since (A 1') obviously holds true, it suffices to check (A 2'). We have

and the (pointwise) derivative of
IMrthermore

^ [ - ^ + 2 ^ 2 ] ' _ ±,

(3.5)

as is easily seen. Next

so that Therefore

2f ^ as is easily checked.

as A->0,

(3.6)

Some examples

49

Relations (3.4)-(3.6) and Theorem 2.1 A show that

Since also by means of (2.1) and (3.5),

T(0) = ±£0[U0)f = ^ ,

(3-8)

it follows that (A2') (i)-(iii) are fulfilled. 3.3 Let {^n}, n ^ 1 be i.i.d. with the double exponential densitv p(z;0) = £ e x p [ - | s - 0 | ] (0ei2). Once more (Al') is trivially true and therefore we occupy ourselves with the task of verifying (A2'). Clearly one has

However, the pointwise derivative of i(0,6*) with respect to 0* does not exist at (0,0) when 0 = Xv We will show that the derivative in q.m. does exist. In fact, for each de®, define the r.v. Zj(d) as follows Zj(d) =

r - i if XjKd 0 if I i = I | otherwise

]

fl \ (j = 1, ...,n).)

(3.9)

Then SQZ^O) = &eZix{0) = 0 and <^ZJ(0) - ^ 2 | ( 0 ) = J, since Pe(X1 < 0) = P^(ZX > 0) = J. Furthermore i[^1(0,0 + A ) l ] = i so that for each 0 e 0 , one has t h a t T [§^(0,0 + A) - 1] -> Zx(6) with ft

i^-probability 1, as h -> 0 which implies that - 1] -> ZX(0) [0(

Next ^ J i ^ x

in P^-probability, as A -> 0.

(3.10)

50

Asymptotic expansion and distribution

where

f J(2 - A) exp p

if A < 0

as is easily seen by integration. Considering the case h < 0 first, one has

by setting \h = £, and this is equal to i + | o (1) -> | as t -> 0. That is,

{J

j 2 > i asA->0,

(3.11)

provided h < 0, and the same is seen to be true when h > 0. The fact that <^£f(#) = J and relations (3.10) and (3.11) together with Theorem 2.1 A imply then that

Furthermore

Ud) = Z^d).

(3.12)

T{6) = 4^[^(6>)]2 = 1,

(3.13)

so that (A2') (i)-(iii) are also fulfilled. 3.4 Let {Xn}, n ^ 0 be centred at expectation and suppose they constitute a Gaussian process with covariance given by *e(XmXn) = exp(-0|m-ra|), 6 > 0. Then the process is a stationary Markov process, as follows from Doob [1], Example 4, p. 218 and pp. 233-4. Furthermore the process is metrically transitive according to statements on p. 476 and p. 637, section 7 of the reference just cited. Therefore (Al) is satisfied. Assumption (A2) is clearly true. As for (A3), from the fact that the Xs have expectation 0 and variance 1, it follows that the stationary initial distribution is iV(0,1). By setting p(6) = exp( — 6), we have that the transition density here is given by

Some

51

examples

Therefore it follows that 0X(0,0*) = 0?[p(0),;p(0*)] =

[l-2>2(0*)J

From (3.15), one has for the (pointwise) derivatives indicated below

'«=« 2[l-p*(d)f

2[l-

f-1.

(3.16)

Since the Xs are distributed as N(0f 1) under i^, we have £eXl = geX\=l

and

^ 0 XJ = SdX\ = 3,

(3.17)

while by our assumption about the covariance of successive Xs, it follows that £{XX){d) (3.18) Next by means of (3.14) and (3.17), one has

i.e.

(3.19)

and in a similar fashion (3.20) Then by means of (3.16)-(3.20), one finds that

4-2

52

Asymptotic expansion and distribution

Next

.ill h*\

[{2-|

(3.22)

as follows from (3.15) by replacing 6* by 6 + h and carrying out the integration. Finally, letting h -> 0 in (3.22), we see that the limit on the right-hand side is equal to

This result together with (3.21) and Theorem 2.1A implies that i

m 9l{0)

P2(0) [Xj - 2P(d) xoxx+x\] ~ 2[l-p\6)f

PHd)+p(d)Xox1 a K

2[l-;p (0)]

•

'

From (3.21), it follows that H6)[i+P*m

(324)

so that (A3 (i)-(iii)) are satisfied. Finally it is clear that q(XQ,Xi;

6,6*) = $1(6,6*) = t*[p(6),p(6*)],

so that, by (3.15), (A4) is also satisfied. 4 Asymptotic expansion and asymptotic normality of likelihood functions In this section we formulate the basic results of the present chapter which relate to the asymptotic expansion in the probability sense, and also asymptotic normality of certain likelihood functions. In what follows, Xo, Xv ..., Xn are n + 1 observations from the

Asymptotic expansion and normality

53

underlying Markov process and 6 is an arbitrarily chosen but thereafter kept fixed point of 0. Let {hn} be a sequence of points and set 0n = 6 + hnn~b, so that of Rk such that hn-*heRk, 6n G 0 for sufficiently large n, since 0 is open and 6 e 0. Then on account of (A 2), the log-likelihood of [dPn ^n/dPn d] is well-defined except perhaps on a i^*-null set for all 0*e0. In the sequel, we will work outside this exceptional set. Set

= log
(4.1)

3= 1

and also define the quantities An(0) and A (h, 6) as follows An(6) = 2/i-i S ^,(0), ,4 (A, 6) = iAT((?)*,

(4.2)

where, as it follows from (2.1),

hT(6) h = MJLK'faO)]*.

(4.3)

Then one has the following result. 4.1 Let hn,heRk be such that hn->h and set 6>w = d + hnn~i. Let A(#,6>J, An(6>) and ^4(M) be defined by (4.1) and (4.2), respectively. Then THEOREM

A(0, dn)-h'kn(d)

-> -4(A, 5) in P nt .-probability.

R E M A R K 4.1 The following is a heuristic interpretation of the theorem. It says that one has

[dPn>eJdPn>e] = exj>A(6,dn) » exp[A'A n (0)-4(M)]; i.e. the likelihood behaves approximately as if it were an exponential family. This is made precise and also proved in Chapter 3, Theorem 3.1. From the loose interpretation of Theorem 4.1 just presented, it is evident that the r. vector An(#), being the r. vector appearing in the exponent of an exponential family, is bound to play a very important role in the sequel. Therefore it is eminently important to derive its asymptotic distribution. This also is done in the present chapter and the relevant result goes as follows.

54

Asymptotic expansion and distribution

T H E O R E M 4.2 Let An(0) and T(6) be defined by (4.2) and (2.1), respectively. Then

Then as a consequence of Theorems 4.1 and 4.2, one has 4.3 Let hn, heRk be such that hn-^h and set dn = d + hnn-i. Let A(0,6n) and AT(0) A be defined by (4.1) and (4.3), respectively. Then THEOREM

Now for the purpose of statistical applications to be discussed in other chapters, the asymptotic distribution of A(0, dn) as well as An(0), under Pn>0 , is also needed. These distributions are provided by Theorems 4.5 and 4.6 below. Also Theorem 4.1 remains true when Pn)0 is replaced by Pnyen- This result is stated as Theorem 4.4 below. THEOREM

4.4 With the same notation as that of Theorem

4.1, one has A(0, dn)-h'An(d) THEOREM

-> -A(h, d) in PWf^-probability.

4.5 With the same notation as that of Theorem

4.3, one has

THEOREM

4.2, one has

4.6 With the same notation as that of Theorem ^ ^ p j ^ N(F(6)h, T(d)).

5

Some lemmas Since the proofs of the fundamental results stated in Section 4, and, in particular, that of Theorem 4.1, are quite long, it would be suggested from an organizational viewpoint to present the intermediate steps as lemmas. Actually some of these lemmas are also of independent interest. Unless otherwise specified, in what follows hn, heRk

with

hn -> h and

6n = 6 + hnn~i.

(5.1)

Some lemmas

55

We also set n (l/)

= i\.(l/, C/ )

and

0ni\y)

==

^PiV^i "n/9

\"*^/

where A(8,8n) and 0/0,8 n ) are given by (4.1) and (1.3), respectively, so that 0\8,8n)+

21og0 2 ,.(0).

(5.3)

Occasionally we shall also use the simpler notation Ani nj and j instead of An(d)9
5.1 With 0 w /0) defined by (5.2), one has in ^-probability.

Proof Since, clearly, §^(0,8) = 1, assumption (A3) (i) implies by virtue of Definition 1.1', T [j(0,8 + Ah)-l]-> h'faO)

in q.m. [P0] as A -> 0 (5.4)

uniformly on bounded sets of values of h. This is true for all j = 1,..., n. By taking A to be n~i, and replacing A by a sequence {hn} which converges to h and also using the notation introduced in (5.1) and (5.2), relation (5.4) implies ni(nj - 1) -> h'cjyj in q.m. [P0], for each j .

(5.5)

From (5.5) it follows then that n{nl- I) 2 -> {h'^f Seea(5.9), (5.10), (5.11) below.

Next

in the first mean [P0].

- J (h'fa)* -> ^(A^ x ) 2 a.s. [P6]

(5.6)

(5.7)

by the ergodic theorem and the stationarity of the process, and

56

Asymptotic expansion and distribution

from which it follows, by the Markov inequality that, for every e > 0, > > e

nj=1

However, this last expression converges to zero on account of (5.6). Thus 2 (tnj ~ l )2 - - 2 WW

-> 0 in ^-probability.

This relation together with (5.7) implies the desired result. ] From (5.5) it follows that nj—l -> 0 in P^-probability. The following lemma strengthens this result as follows. LEMMA

5.2 With ni{0) as defined by (5.2), one has

m a x [\nj{6) — l\;l^j^n]->0

in i m p r o b a b i l i t y .

Proof In fact, RnP

(5.8)

where Rnj = Rni(6, h) = ni(
fie\nHtnl-

1)-^|«

0

(5.9)

by (5.5). From (5.8), one has (\nj- 1|; 1 ^ j ^ n) > e] e] r

^ | ; 1 ^ j ^ n) > \en*

+ Pe[max(\Rnj\) 1 < j ^ n) > ±e (5.10) But

nPe{\R nl\

0

(5.11)

Some lemmas

57

by means of (5.9), and if F denotes the distribution of the r.v. l^'^l under P09 one has

3 f* Jo This implies t h a t Next

f°° x2dF-*0. J ien*

(5.12)

= ra f °° dF = ^[™ J It-en*

{\enifdF

G

"J Jen*

>0

(5.13)

by means of (5.12). Thus relations (5.10), (5.11) and (5.13) complete the proof of the lemma. ] LEMMA 5.3 (5.3), one has

AJ0)-

With Aw(0) and nj{0) as defined by (5.2) and

2( S [^(8)- 1]-\

£ [^(0)

- I] -> 0 in P^-probability.

Proof This lemma states, in effect, t h a t An(d) may be replaced asymptotically - and in the probability sense - by certain sums involving the r.f.s nj(O), j = l,...9n which might be easier to deal with. Now for a given 0 < e ( < J), set An = An{6) = [ m a x ( | 0 n , - l | ; 1 ^ j < n) > e\.

(5.14)

Then Lemma 5.2 implies t h a t Pe{Acn) > 1 — e

for all sufficiently large n,n^

n0,

say.

Consider the expansion

(5.15)

58

Asymptotic expansion and distribution

Then (5.14) implies that the expansion in (5.15) holds uniformly in j = 1, ...,n,n ^ n0 on the set Acn of probability > 1 — e if x is replaced by
= (*«-!)(j = l , . . . , w ) , c

on A n with

n) > 1 ~~e

(n

Therefore on ^ and with n ^ TI0, we have

But

3[max (\$ni- 11; 1 ^ j ^ n)] S {fa- I)2 -> 0 in Pfl-probability by Lemmas 5.1 and 5.2. Therefore

in Pfl-probability, or equivalently,

S in Pfl-probability.

(5.16)

Now by (5.3) and (A4) K~

S l o g ^ = logg(Z 0 ; 5,5W) -> 0 in ^-probability, (5.17) 3= 1

so that relations (5.16) and (5.17) imply

K-2 f S ( ^ - 1 ) 4 S ( ^ - 1)21 -> 0 in ^-probability, which is what the lemma asserts. ] The following lemma will also be needed.

Some lemmas LEMMA

59

5.4 With ^>nj(0) as defined by (5.3), one has that,

for each j , ni[2nj(d) - 1] -> 2h'$j{0)

in the first mean [P0].

Proof Consider the following identity

n*(#, -1) - Wfa = {^[n*(^ - 1) - h'fa) + [h'fa(
(5.18)

Now by taking absolute values on both sides of (5.18) and the expectations under Pg, we obtain

Next by means of Holder inequality, applied to the first two terms on the right-hand side of the last expression above, and the fact that &e<$>\$ = 1, we have

But

^|^(^.-i)-

and hence (5.19) becomes -l)*. Now by (5.5)

^[wi(^,.- 1) -A'^.] 2 -> 0,

(5.20) (5.21)

which by Theorem 2.1 A implies &g[nl($ni—l)]* -> ^(A'^) 2 , and (5.22) By (5.21), (5.22) and the fact that tfoih'fa)2 < oo (which follows from (A3) (i)), relation (5.20) gives as was to be shown. J

60

Asymptotic expansion and distribution

We recall that s#$ denotes the o"-field induced by the r.v.s ...,XJ. Then by taking into consideration the definition of nj and utilizing the Markov property, it is an easy matter to see that ,2 , . . fi p /KOQ, ^ ( 0 K ) 1 |7y (5.23) Now we formulate and prove a lemma which together with the previous ones will help to establish parts of the main results stated in Section 4. To this end, we set XQ,XV

!M
0' = 1 . - . » ) .

(5-24)

and then one has the following lemma, keeping in mind that
(ii) S foM0) - ! ] • > - K^t*'^^)] 2 (iii) 5 {[^(») - 1] - n-W^O)}

inp

n, .-Probability.

- S [ ^ ( 5 ) - 1] -> 0 in P n ^-probability.

Proof

(i) By Lemma 5.4, we have that for each j ,

in the first mean [P0]. Then the same convergence is true by conditioning both sides, given s^j_x; i.e. i n

^-l)

t h e

first

m e a n

(5.25) This follows from well-known properties of conditioning (see, e.g. Loeve [1], p. 348). But the left-hand side of (5.25) is equal to 0 a.s. [P0] by virtue of (5.23). Therefore

Since this is true for any h e Rk, it follows that <^(0,-K-i) = 0

a.s.

61

Some lemmas

as was to be seen. (Of course, it is understood that this latter 0 is the fcxl zero vector.) (ii) In the first place, the ergodic theorem implies that

Next

(5.26)

nj=1 > €

> en

(5-27) But by (5.5), ^ ( ^ n l — 1) -> Theorem 2.1A implies that n((j)nl— I) 2 -> {h'

in q.m. [P6] which by means of in the first mean [P0].

Thus (5.27) yields

1

n

-i] ->0 in Probability. (5.28)

Relations (5.26) and (5.28) taken together give then in

^-probability. (5.29)

Now consider the identity and condition both sides, given ^•_ 1 . By taking into consideration (5.23), one obtains then 0 =

62

Asymptotic expansion and distribution

or, by virtue of (5.24), 2(fnj -1) + <^[(^, -1) V y - J = °

a s

- - [Pel

Hence 2 2 ( ^ - 1) + S ^ [ ( ^ -1) V / - J = 0 a-B- [Pel Letting n -> oo and taking into consideration (5.29), one has 2 £ ( ^ - 1) -> - **(* Vi)2

in Probability,

as was to be shown, (iii) Consider the r.v.s Tt = (tm-V-n-ih'fa-^-l)

(j= l , . . . , n ) .

(5.30)

Then

= 0 a. the first equality being true on the basis of well-known properties of conditioning (see, e.g. Loeve [1], p. 350) and the second equality holding by (5.24) and the first part of the lemma. So

^ ( r | r r )

o

[P] (j = i,...,n).

(5.31)

Then the extended Kolmogorov inequality (see, e.g. Loeve [1], p. 386) applies and gives

(5.32)

Some lemmas

63

At the right-hand side above, we also used the stationarity of the Ys which follows from that of the Xs in the first equality sign, the fact that SeYx = 0 which follows from (5.31), and also the Minkowski inequality in order to obtain the last inequality. Now the convergence in q.m. [Pd] in (5.5) implies that ^[^ nl -l)-^J a ->0.

(5.33)

By also taking into consideration (5.24) and well-known conditioning properties also cited before (see Loeve [1], p. 348), (5.5) implies, in conjunction with the first part of the lemma, that -l)->0

in q.m. [Pe],

^ M ( ^ n i ~ l)f -+ °-

so that

( 5 - 34 )

Thus relation (5.32) becomes by means of (5.33) and (5.34) n

2Z->0

in Pfl-probability.

3= 1

The definition of Yj9 j = 1, ...,n by (5.30) completes the proof of (iii) and hence the proof of the lemma itself. ] 6 Proof of theorems of Section 4 The lemmas established in Section 5 will suffice for the proof of some of the main theorems stated in Section 4. In order to complete their proof, however, some additional results will be needed. These auxiliary facts will be established in the present section and some of them will also be employed on other occasions as well. Proof of Theorem 4.1 We want to show that n

An - 2n~b 2 h' - 2£'6(h'(f>1)2 in P^-probability.

(6.1)

By Lemma 5.3, we have A

n

- 2 ^ (nj- 1) + 2 (nj- l)2 _> o

in P r o b a b i l i t y ,

(6.2)

64

Asymptotic expansion and distribution

while by Lemma 5.1, 2 (ni- I) 2 -> &eW$i)2

m P^-probability.

(6.3)

Subtracting (6.3) from (6.2), one obtains An - 2 S ( ^ - 1) -> - ^ ( A ' ^ ) a

m ^-probability.

(6.4)

Next adding up the relationships in (ii) and (iii) of Lemma 5.5, one has S (nj - 1) - n-* S * ' ^ -> - i < W & ) a

in

^-probability,

so that i=i

- 1) - 2n-ii S * ' ^ -> - ^(*'^i) 2

in ^-probability.

i=i

(6.5) From (6.4) and (6.5), it follows that An - 2n-i S A'$5y -> - 2Se{hr(f>1Y in P^-probability which is (6.1).] Proof of Theorem 4.2 By writing A^ and Y instead of An(6) and T(6), we wish to show that ).

(6.6)

This convergence is equivalent to the following one &(t'An\P6) => N{0, t'Tt)

for every «eR k ,

or equivalently, ^ ? L - i S 2t'i\Po) => ^(0, «T0 \

Set

i=i

for every *e JS*.

(6.7)

/

Zj = 2t'(f>j

(j=l,...,n).

Then by (A 1), the process {Zn}, ^ ^ 0 is stationary and ergodic, while S>6Z1 = 0, as it follows from Lemma 5.5 (i). Also SQZ\ < oo

Proof of theorems of Section 4

65

by (A3)(i). Finally, utilizing well-known properties of conditioning (see Loeve [1], p. 350) and Lemma 5.5 (i), one obtains

j_l)

= 0 a.s. Therefore the assumptions of Theorem 2.2A are fulfilled and hence its conclusion implies that

However, this convergence is the same as (6.7) which in turn is equivalent to (6.6). The proof of Theorem 4.2 is completed.] Proof of Theorem 4.3 This is an immediate consequence of Theorems 4.1, 4.2 and the standard Slutsky theorems (see, e.g. Rao [3], p. 102 (x,b)). In fact, by (6.7), we have

yl*)

°> hTh)>

(6-8)

while by (6.1) = - \h'Th

in P^-probability. (6.9)

Thus, (6.8) and (6.9) imply

as the theorem asserts. ] For the proof of Theorem 4.4, we need the following proposition which will also be employed on other occasions. 6.1 Let {A*} be a bounded sequence in Rk and set 6% = 6 + h*n-i. Then the sequences {Pntd} and {Pnf6*} are contiguous. PROPOSITION

Proof Because of the boundedness of {A*}, for any subsequence {n'} c= \n} there exists a further subsequence {m} <= {n'} such that hm-+h, some heRk. Then Theorem 4.3 applies and gives that -2?[A(0, di)\Pe] =>N(- \h'Th, hTh). 5

RCO

66

Asymptotic expansion and distribution

It follows that {&[A(0, Ot)\Pn,0]} is relatively compact. Furthermore, it is easily seen as in the proof of Corollary 3.1 of Chapter 1, that exp AdJSf = 1, where Jg? = N( - \h'Yh, hTh). / •

Then Corollary 2.2 in Chapter 1 applies and gives the desired result. ] Proof of Theorem 4.4 By Proposition 6.1, {Pnt0} and {Pn,^n} are contiguous. Then Theorem 4.1 together with the definition of contiguity (Definition 2.1, Chaper 1) implies the assertion of the theorem. Proof of Theorem 4.5 It is an immediate consequence of Theorem 4.3, Proposition 6.1 and Corollary 3.2 in Chapter 1.] Finally we have all we need in order to establish the last theorem of Section 4. Proof of Theorem 4.6 Assumptions (3.3)-(3.5) in Lemma 3.1 of Chapter 1 are satisfied with Pn = Pn)6, P'n = Pn> v Tn = An(0) and F = T(d). This is so because of Theorems 4.2, 4.3 and 4.1, respectively. Furthermore, {Pn> e} and {Pn eJ are contiguous by Proposition 6.1. Therefore, the assumptions of Theorem 3.2 in Chapter 1 are fulfilled and then its conclusion gives

as was to be shown. ]

Exercise Justify the equivalence of Definitions 1.1 and 1.1' asserted in Remark 1.1.

3. Approximation of a given family of probability measures by an exponential family - asymptotic sufficiency

Summary It was mentioned in Chapter 2 that, under the basic assumptions (A 1-4), one may approximate a given family of probability measures by an exponential family of probability measures in the neighbourhood of each parameter point. By this, we mean the following. For 6e&, set 6n = 6 + hn~i9 so that 6ne@ for all sufficiently large n. Then there is a probability measure Rn h, where the parameter now involved is heRk, such that Rn h is close to Pn e in the 2^-norm sense. More precisely, one has mV[\\Pn>dn-Rnth\\;heB,0n

= d + hn~i] -> 0,

where B is any bounded set of As. The statistical implications of this approximation were also mentioned in Chapter 2. In the present chapter, we give an explicit way of constructing the measures Bnh (see (1.7)) and also establish the approximation mentioned above (see Corollary 3.1). In the process of doing so, one has to introduce a certain truncated version, A*(#), of the random vector An(#), defined by (4.2) in Chapter 2 and also establish some auxiliary results. This is done in Sections 1 and 2. The proof of the main result is presented in Section 3. In Section 4, we introduce the concept of differential (asymptotic) sufficiency of a sequence of statistics at a given parameter point 60i say, for the given family of probability measures. According to this concept, the sequence of statistics {Vn}, say, is differentially (asymptotically) sufficient at 60 for the given family of probability measures if for each n, Vn is sufficient for a family of probability measures indexed by 6 and in the neighbourhood of 60 they lie close to the given family in the 2^-norm sense (see [ 67 ]

5-2

68

Approximation by an exponential family

Definition 4.2). It is then shown that the random vector An(#0) mentioned above is actually differentially (asymptotically) sufficient for the given family at the point 60 (see Corollary 4.2). In the final section, we consider some of the statistical implications of the results obtained in the previous two sections. It is shown that for certain testing hypotheses problems and from an asymptotic power point of view, one may base his tests on the random vector Aw(#0) above, where 60 is known (see Theorem 5.1). It is also shown that tests may actually be based on A*(#o) rather than An(#0), when again asymptotic power is the criterion of optimality of a test (see Theorem 5.2). This result is important in that it provides a way of setting up tests, since A*(#o) happens to be the all-important statistic which appears in the exponent of the approximating exponential family. However, the actual random vector used is An(#0) rather than Aj(
1 Formulation of the problem and some preliminary results In Chapter 2, Remark 4.1, it was heuristically stated that, under suitable conditions

J

)» k

exV[h'An(d)-A(h,e)l

where 6n = d + hnn~i and hn-^heR . That is, the likelihood behaves approximately as if it were an exponential family with h playing the role of the parameter and 6 being kept fixed. One of the main purposes of the present chapter is to make this statement precise and also prove it. To start with, in order for exp [A'AW(#) — A(h, 6)] to be a candidate for an exponential family, it has to have a finite integral with respect to Pnt6, so that when properly normalized

The problem and some preliminary results

69

it will, indeed, be an exponential family. There is no guarantee, however, that Jexp h'An(d) dPn e is finite. Then the appropriate thing to do would be to replace An(d) by something else, A*(#), say, so that Jexp A'A*(#) dPn e is finite. This is done in (1.5). Next, a probability measure Rn hn onstfn is defined in terms of A*(#) (see relation (1.7)), which has a density of the exponential form with respect to Pnd. It is then shown in Theorem 3.1 that

It is in this sense that the likelihood mentioned above is approximately exponential. Namely, the original family of probability measures is approximated by another family of probability measures in the 2^-norm sense and in a neighbourhood of 6. Furthermore, the members of the approximating family are absolutely continuous with respect to Pn e and their densities are of an exponential form with parameter h and 6 being kept fixed. In what follows, an arbitrary point 60 in © will be chosen and kept fixed thereafter, and we set An for An(60). We shall also write PQ rather than Pn d. Next define the real-valued function p onBkhj

' PW-N,

(i.D k

where || • || is the usual Euclidean norm in R , and p is obviously continuous. Q = N(0, T), A = identity mapping on Rk,

(1.2)

sothatJ§?(A|Q) = Q. Then we have the following proposition. P R O P O S I T I O N 1.1 Let p and Q, A be defined by (1.1) and (1.2), respectively. Then

Proof It follows from Theorem 4.2, Chapter 2, and the continuity of p (see, e.g. Rao [3], p. 104 (xii)).] The following result is also true. PROPOSITION

1.2

For every A > 0, we have oo.

70

Approximation by an exponential family

Proof LetA = (A1,...,Afc)/.Then||A|| < 2 | A, | and therefore 3=1

(A)] =
j=i

= #Q \ ft exp A|A,|1 ^ ft by a generalized version of Holder inequality (see Exercise 1). But $g[exp&A|Ay|] < oo since A;- is normal under Q,j= 1, ...,&. This completes the proof of the proposition. ] Consider the random vectors Aw, A and for each a > 0, define the following truncated versions of them denoted by A* and Aa. An if p(AJ < « 0 otherwise '

Aa =

(A if p(A) ^ a J (0 otherwise j

Then the following proposition is true. PROPOSITION

1.3 For every a > 0, we have

(i) and (ii) Proof (i) For every real-valued, bounded and continuous function/on Bk, one has

j f(z

and this last integral converges to

= J/{A/ [/)(A)
The problem and some preliminary results

71

= J/( since <SP(l±n\PeQ) => Q and the set of discontinuities of is a subset of the surface of the sphere $(0, a) which is assigned (^-dimensional) Lebesgue measure zero and hence Q-measure zero. (See Corollary 1.1, Chapter 1.) (ii) This follows from (i) and the continuity of p. (See also proof of Proposition 1.1.)] By the definition of p and Aa, A*, it follows that />(Aa) ^ a, p(A£) ^ a for every a > 0. Thus, whereas <^o[exp Ap(AJ] need not be finite, the expectations <^Q[exp Ap(Aa)] and 0 and all n. What is more, the following proposition holds true. PROPOSITION

1.4

For every a, A > 0, we have

(A*)]. Proof Since the distributions J2?[/o(A*)|P^o] and J§?[p(Aa)|Q] have supports confined to the interval [0, a], it suffices to restrict the integration to this interval. Then we obtain

=f e J[0,a]

f

[0,a]

by Proposition 1.3 (ii) and the fact that the integrand is bounded and continuous over the interval [0, a]. ] Now let {xv} and {ev} be two sequences such that 0 < av \ oo and 0 < ev\ 0 as v -> oo. Consider Ao > 0 to be specified later on (see relation (3.10)). For every avi ev and Ao as above, there exists a positive integer JV^ = N(av, ev, Ao) such that v,v=l,2,...).

(1.4)

72

Approximation by an exponential family

This is so by Proposition 1.4. Clearly, {iV^} can be chosen so that Nv f oo. Then we define a subsequence of ^-dimensional random vectors A* by A* = A«* for n such that Nv ^ n < Nv+1 (i; = 1,2,...).

(1.5)

Set A* = 0 for n < Nv Since for any heRk, it follows that S>6o(exp h'/S%) < oo. In particular, by virtue of (1.5). Thus we may define the following quantity expBn(A) = ^0(expA/A*) and also the probability measure Rn

h

(JeB*)

(1.6)

on sfn by

Rn,h(A) = exp [-Bn(h)] J^exp (A'AJ)dP^.

(1.7)

Then we can formulate the following fundamental result. T H E O R E M 1.1 LetRn h be defined by (1.7) and for a bounded sequence {hn} in Rk, set 6n = 60 + hnn~i. Then we have

\\Pn,en-Rn,K\\^0.

(1.8)

The proof of the theorem is presented in Section 3 after some auxiliary results have been established in the following section. 2 Some auxiliary results In this section, four propositions and two lemmas are formulated and proved. Their purpose is to facilitate the proof of Theorem 1.1. Some of them, however, such as Propositions 2.3 and 2.4, will be employed on other occasions as well. PROPOSITION

2.1 For every A > 0, we have

oo. Proof One has

Some auxiliary results

73

Now expA/)(A)7[/9(A) exp A/?(A) as a -> oo, the dominated convergence theorem applies and gives the result. ] PROPOSITION

2.2 Let A* be defined by (1.5) and let Ao be

as in (1.4). Then <^0[exp A0/>(A*)] ->
Thus

^o[expAo/o(A*)]-^g[expAoy9(A^)]->O

1,2,...).

as p->oo

(2.1)

(which implies n->co because of Nv->co and (1.5)). But by Proposition 2.1, ^ Q [exp Aop(A)].

(2.2)

Relations (2.1) and (2.2) complete the proof of the proposition. ] Now it is, of course, important to know what is the relationship of A* to An. According to the following fact, they differ only on a set whose probability tends to zero. More precisely, we have P R O P O S I T I O N 2.3 Let A* be defined by (1.5) and let 6n be as in Theorem 1.1. Then (i) P,o(A* * A J -* 0, and (ii) P,n(A* # An) -• 0.

Proof From the definition of A*, one has A* = AS' = A m / [ ^ ^ ( A J Thus and therefore

(Nr
Ny+1,

(A* + A J = [p(AJ > a,] P,0(A* + A J = P,0[p(AJ > a j .

v=l,2,...). (2.3)

But ^[/o(AJ|Peo]=>jS?[/o(A)|Q] by Proposition 1.1 and the convergence of the corresponding distribution functions (d.f.s) is

74

Approximation by an exponential family

uniform. This is so by Polya's lemma (see, e.g. Rao [3], p. 100, (vi)), since Q is an absolutely continuous measure. Since <%„ -> oo as *>->oo (which imply n-*co), by taking the limits in (2.3), we obtain the first of the proposition. The second conclusion follows from the first part and contiguity of {P0Q} and {P0J by Proposition 6.1, Chapter 2.] By Proposition 2.3 and Theorems 4.2 and 4.6, Chapter 2, we obtain PROPOSITION

2.4 Let A* be defined by (1.5). Then one has

and (ii) js?(A*|p MJIiin -o=>.ar(rft,r), provided hn -> h. This section is closed with two lemmas and one corollary which will be used in the following section. For n ^ 1, let Xn be r.v.s defined on the probability space (£}, ^", P). Then we recall that the r.v.s \Xn\9 n ^ 1, are said to be uniiformly integrable if

|X n |dP->0 uniformly in n as

J(\Xn\>Ct)

a -> oo (see, e.g. Loeve [1], p. 162). The following lemma provides sufficient conditions for uniform integrability. Namely, L E M M A 2.1 For n ^ 1, let Xn be r.v.s defined on the probability space (Q,^",P) and let X be an r.v. defined on the probability space (Q^J^'jP'). Furthermore, suppose that &{X\P'). Then the r.v.s |XW|, n ^ 1, are uniformly integrable. Proof For simplicity, set Fn9 F for the distribution functions corresponding to J§?(Xn\P), <£(X\P'), respectively. Then we have

f

1*1 dJ^ < f 1*1 dFn- \|*| dF

\Xn\ dP = f + \[

\x\6Fn-[

\x\dF\+(

|*| dF

\x\dFn-\

|*| dF (|a:|
f

+ f (\x\>a)

|a?| dF.

Some auxiliary

results

75

Now | ^ | X n | — (?|X|| < -Je for all n ^ nl9 say, by assumption; \x\ dF < ^e for all sufficiently large a, by the assumption J (\x\>a) that ^\X\ < oo. By choosing a as just stated and also such that both a and — a are continuity points of F, one has Id dF < Je U\x\
If

that I

|X n | dP < e. Increasing a so that to make this

integral < e for n = 1,..., n3— 1, we obtain the desired result. ] LEMMA 2.2 For n ^ 1, let X n , F n be r.v.s defined on the probability space (£},#",P) and suppose that the r.v.s \Xn\, n ^ 1, and \Yn\, n ^ 1, are uniformly integrable and that Then the r.v.s |Xw — Yn\, n^l, are uniformly Xn — Yn^0. integrable.

Proof We have

r

U

J(\Xn-rn\>a)

U

r ^ J (\Xn~Yn\>a)

+f I /ly

ftJdP

(2.4)

-i

J (\JLn— J

and / .

|Z w |dP

•

/

.

f

|Z n |dP + cP(|Z n -r n |>a).

J (\Xn\>c)

Thus for all sufficiently large c, we have

\Xn\ d P J (\Xn\>e) for all 71 and for all sufficiently large a, we have P(\Xn-Yn\ > a) < el2c

(2.5)

76

Approximation by an exponential family

for all n ^ n0, say. Therefore for c, a as just stated and for all n ^ n0 relation (2.4) gives

|XW| dP < e. Increasing J (\Xn-Yn\>a)

a further so that this integral becomes < e for n = 1, ...,n0— 1, we obtain the desired result. ] The lemma just proved has the following corollary. C O R O L L A R Y 2 . 1 Under the assumptions of Lemma 2.2, one hasthat(f | X n - 7 n | -> 0. Proof It follows from the assumption that Xn — Yn -^>0, the uniform integrability of the r.v.s \Xn — Yn\, n ^ 1, as concluded in the lemma, and the Z^-convergence theorem (see Loeve [1],

p. 163).] 3 The proof of the theorem On the basis of the results of the first two sections, we may now present the proof of Theorem 1.1. P R O O F OF T H E O R E M 1.1 The proof is by contradiction. Assume that (1.8) is not true. Then there exists a subsequence {m} c {n} and {Am} c: {hn} with hm->h such that h

||-)>0.

(3.1)

From Theorem 4.1 in Chapter 2, one, clearly, has that [dPJdPeo] = expA(0o,0m) = LJhm) ' hm)l

(3.2)

where A(hm) = \h'mYhm

and

Zm(hJ ~> 0 in P^-probability.

(3.3)

From (1.6) and (1.7), one also has that [di?m> JdPeo] = L*m (hm) = exp [ - BJhJ

+ h'm A* ].

(3,4)

Then from (3.2) and (3.4), one has that \LJhm)-L*(hm)\ \ZJhm)\ + \Bm{hm) -A(hm)\] exp Tm, (3.5)

Proof of the theorem

77

where Tm is an r.v. lying between the r.v.s h'm Am - A (hm) + Zm(hm) and -BJhm) + h'm&t The first and the second term inside the brackets on the righthand side of (3.5) converge to 0 in P^-probability by Proposition 2.3 and relation (3.3), respectively. In the following, we shall «„,,.,,, , also show that Bm(hm)-A{hm)->0. (3.6) These last three conclusions imply that {Tm} is bounded in P^-probability and therefore relation (3.5) implies that Lm(hm) -L*(hm) -> 0 in P^-probability.

(3.7)

Next Theorem 4.3 in Chapter 2 implies that

where A* is the identity mapping on B and

whereas ^ Q .expA* = 1 by Corollary 7.1, Chapter 1. Therefore Lemma 2.1 applies with {n} replaced by {m},

and

Xm = LJhJ

(L*(hJ), X = exp A*

and gives that the r.v.s \Lm(hm)\ = Lm(hm),

\LZ(hm)\ = L*(hm)

corresponding to the sequence {m} are uniformly integrable. This result, together with relation (3.7) and Corollary 2.1, gives that j\Lm(hm)-L^(hm)\ dP0Q -> 0. By Theorem 1.4 A and relations (3.2) and (3.4), this is equivalent to a contradiction of (3.1). Thus the proof of the theorem will be completed by establishing (3.6). Now f exp h'A AQ is the moment generating function of A'A evaluated at t = 1. Since this is equal to exp A (h) and, clearly, A(hm) -» A(h), in order to prove (3.6), it suffices to show that

expBm(hm) -> fexpA'AdQ (= f exp&'zd(A .

(3.8)

78

Approximation by an exponential family

For simplicity, set Sen = J§f(A*|P^o), so that ^n=>Q by Proposition 2.4 (i). Next let Sr be the (closed) sphere centred at the origin and having radius r, and let Mr be a constant such that | e x p ^ z - e x p A ' z | ^ Mr\\hm-h\\, provided zeSr. The fact that Q(dSr) = 0, Sem->Q and Theorem 1.1, Chapter 1, imply that

f

ex^hfzd^m->

i exp&'zdQ. Therefore

J $r

+

exp h'z dJ5fm -

exp A'z dQ

sr so t h a t

- I expA'zdQ. J Sr

(3.9)

Ao.

(3.10)

Let Ao be defined by

Next

I

c

exvh'mzd£>m-

I ^ exiph'zdQ

< Lc e x P Kz\ ^ I

c

djSf

m+ f „ exp \h'z\ dQ

expA o / o(2:)d^ m + | ^ exp Ao/o(2;) dQ. (3.10')

Now since J exp A0/o(z) dQ < oo by Proposition 1.2, we may choose r sufficiently large, so that I Since

exipAop{z)dQ < e.

expA0/>(2;)dJ^m -> J Sr

expAop(z)dQ, J Sr

t follows from (1.4) and (1.5) that c

exp\ o p(z)d<e m ->

(3.11)

Proof of the theorem

79

so that for all sufficiently large m, one has that I

expAo/o(z)
(3.12)

By (3.10'), (3.11), and (3.12), it follows that exp h'm z dJ§?m ->

I

J sn

exp ln'z dQ

J ser

which, together with (3.9), implies that \exph'mzd£?m-+

lexph'zdQ.

(3.13)

Finally,

expBm{hm) = J exp A^A* dP0Q = J exph'mzd£>m -> J exph'zdQ by (3.13), and then (3.8) completes the proof of the theorem. ] The theorem just established has the following corollary. COROLLARY

3.1 Let B be any bounded set in Rk. Then one

has that sup[||P, w -i? n , J ; heB,6n

= do + hn~i] -> 0.

(3.14)

Proof The proof is by contradiction. Suppose that (3.14) is not true. Then there exists a subsequence {m} c {n} and {Am} in B such that || P ^ — Rm )hJ\ -> S, some 8 > 0, where 0j* = ^O + Amm~i. But this contradicts Theorem 1.1. Thus the corollary holds true. ] R E M A R K 3.1 A contradiction argument as the one used to establish (3.17) will be used on many occasions.

4 Differential equivalence of sequences of probability measures and differential sufficiency Two sequences of probability measures which converge in Z^-norm are indistinguishable asymptotically. Such sequences ought to be equivalent asymptotically. For two sequences of probability measures which depend on a parameter, we have the following definition of asymptotic equivalence. D E F I N I T I O N 4.1

For

0e0,

let {Q$e}

and

{Q%}

be

two

sequences of probability measures on stfn. Then the two sequences

80

Approximation by an exponential family

are said to be differentially (asymptotically) equivalent at the point 60 E 0 if for every bounded set B in Rk, the following convergence (4.1) holdstrue. m)g_Qm4. {0-e0)nleB]->0. On account of (4.1), if dn e 0 and 6n -> dQ at the appropriate rate, In other words, in a neighbourhood of d0, either one of the sequences {Q$dJ, {Q^eJ is as good as the other. By setting (6 — 60) n$ = h and writing dn instead of 6, one has 6n = do + hn~i. Therefore the defining relation (4.1) becomes as follows: , dn = 60 + hn-i] -> 0.

(4.2)

At this point it will be convenient to introduce the following notation. For each n and all 6 e 0, define the probability measure B*t0 as follows: 7?* 7? IA Q\ Then we can present the following important corollary to Theorem 1.1. 4.1 Let R^e t>e given by (4.3). Then the sequences of probability measures {Pnf0} and {Rn,e) ar© differentially equivalent at d0. COROLLARY

Proof According to (4.2), one has to show that B u p [ | | P , n - < , J ; heB, 6n = 60 + hn-i] -> 0 for every bounded set B in Rk. By virtue of (4.3), this is equivalent to proving that sup[\\P0n-RnJ;heB9

6n = do + hn~i] -> 0.

(4.4)

However, (4.4) is true because of Corollary 3.1. J The following definition introduces the concept of asymptotic sufficiency of a sequence of statistics, or er-fields, for a given family of probability measures. Specifically, one has D E F I N I T I O N 4.2 For # e 0 , let {Q^e} be a sequence of probability measures on<stfn and let {Vn} be a sequence of i-dimensional, j/^-measurable random vectors. Denote by &n the o*-field

Differential equivalence and sufficiency

81

induced by Vn. Then the sequence {Vn} or {&n} is said to be differentially (asymptotically) sufficient at 60 for the family {Qnjdl QE&}, if there exists a family of probability measures {Qnjel Ge®} such that, for each n, Vn or 3Sn is sufficient for the family {Q%]e\ Be®} and {©£>*}, {Q$d} are differentially equivalent at0 o . The interpretation of asymptotic sufficiency is obvious on account of Definition 4.1 and the comments following it. Namely, for sufficiently large n, in a neighbourhood of 60, Vn, or SSn, may be used as if it were sufficient for {Q$te; ^ in a neighbourhood of 60}. Now according to the following result, the random vector A* possesses the property of being differentially sufficient at 60 for the family {Pnt6; de®}. More precisely COROLLARY 4.2 Let A* be defined by (1.5). Then the sequence {A*} is differentially sufficient at 6Q for the family

Proof By (4.3) and the definition of Rn

h

in (1.7), one has

From this it follows that, for each n, A* is sufficient for the family {l?*^; de®}. Since {Pnyd} and {R%,e} a r e differentially equivalent at 60 by Corollary 4.1, the desired result follows. ]

5 Some statistical implications of Theorem 1.1 Some of the statistical implications of Theorem 1.1 are summarized in the following two theorems. The first of these results states that, for testing the hypothesis HQ: 6 = 60 and from asymptotic power viewpoint, we may confine ourselves to tests which depend on An alone. That is, T H E O R E M 5.1 Let {Zn} be a sequence of r.v.s such that \Zn\ ^ 1 for all n and set Zn = <^ o (Z n |AJ. Then

82

Approximation by an exponential family

where 6n = 60 + hn~i and, for each n, the sup is taken over all r.v.s Zn bounded by 1 in absolute value and over all hs in any bounded subset B of Bk. Proof We have S6nZn - SeZn = I^n, h) + I2(n9 h) + Iz(n, h), where

1,(71, h) = *(Zn\P$n) - £{Zn\Bn>

h)

I2(n,h) = From the assumption that \Zn\ ^ 1, the definition of Zn and the fact that Bn h <^ P6Q, P0n <4 P0Q, one has that Zn is also bounded by 1 a.s. [Bnh] and [Pe ]. Then with the sup as above, both sup \Ixin, h)\ and sup |/3(w, h)\ are bounded by

mV[\\Pdn-BnJ;heB] which tends to 0 by (4.4). We next show that I2(n, h) = 0, which will complete the proof of the theorem. In fact,

the last equality holding true because A* is a function of An. By rewriting the last expression in terms of expectations and using the definition of Bn h once more, one obtains «feo[
^(2 n |i? n>h ) = ^(Zji? w>ft ),

as was to be seen. ] Again for testing the hypothesis Ho and when our interest lies in the asymptotic behaviour of the power of tests, the theorem

Statistical implications

83

below is quite relevant. It states that one can actually base all tests on A* alone. The significance of this statement is apparent by taking into consideration that A* plays the all-important role of the statistic appearing in the exponent of an exponential family. The exact statement of the theorem is as follows. T H E O R E M 5.2 Let {Zn} be a sequence of tests defined on Rk and set dn = d0 + hnrl, heRk. Then for any bounded subset B of Rk, we have

sup [ I ^ Z J A J - ^.Z n (A*)|; h eB] -> 0. Proof We have

Hence | ^ n Z m ( A J - ^ 2 J A * ) | e? 2P6n^n * A*) and therefore s u p [ | ^ Z l l ( A n ) - ^ Z n ( A * ) | ; * e B ] ^ 2sup[P,n(An #= A*); (5.1) Now let {An} be a sequence in 5 and set 6% = #0 + hnn~i. Then, by Proposition 2.3 (ii), we obtain ^

0

.

(5.2)

From (5.2), it easily follows by a contradiction argument, that sup[P, w (A n *A*);/>GJ5]->0.

(5.3)

Relations (5.1) and (5.3) give the desired result. J 6-2

84

Approximation by an exponential family

R E M A R K 5.1 The various tests shall be actually expressed in terms of An. The introduction of A* is a mathematical device by which the original family was approximated by an exponential family defined in terms of A*. Exploiting the distinguished role that A* plays in connection with this exponential family, one is able to set up tests based on A*. Then Theorem 5.2 allows one to pass on to tests based on An. Furthermore, one need not be concerned with tests not based on Aw. This is so by Theorem 5.1. Concrete statistical applications will illustrate the point even further.

Exercises 1. Let Xv ...,Xn be r.v.s defined on the probability space (Q, , F , P). Then show that g\Xx...Xn\

where

?x>-..,?n>°

< ^«i|Z1|«i...^/«»|Zn|fli, and

1

lii + ••• + 1/?n = l-

2. (i) For each n ^ 1, let Xn9 Yn be two r.v.s defined on the probability space (ii, ^n, Pn) and let Z be an r.v. defined on the probability space (£}', JF^P'). Suppose that

&(Xn\Pn)*>&(Z\P')

and Yn-Xn->c

in P^-probability,

where c is a constant. Then show that

(ii) Let {Xn}, {Yn} be as above and let X, Y be r.v.s defined on the probability space (Q', ^F',P'). Then show that if and only if j2?(A1Xn +A.FJPJ => JS?(A1X +A, Y\P')

for every A, A2 in B.

4. Some statistical applications: A UMP and A UMPU tests for certain testing hypotheses problems

Summary In the present chapter, we apply results obtained in the last two chapters to certain testing hypotheses problems and for the case that 0 is a subset of R. All results derived so far in Chapters 2 and 3 were obtained under the basic assumptions (A 1-4). According to these results, the asymptotic expansion of the log-likelihood given in Theorem 4.1, Chapter 2, as well as the asymptotic distribution of the loglikelihood and also of the fundamental random vector A^(^) (for its definition, see (1.1) below) given by Theorems 4.2-4.6, Chapter 2, were valid for moving parameter points 6n essentially of the following form: 6n = d + hn~b9 heB. Thus as n -> oo, 6n approaches the fixed parameter point 6 and at a prescribed rate. If dn either does not approach Q at this rate or it stays away from 6, equivalently, if ni(dn — 6) -> oo( — oo), the above mentioned theorems need not be true. However, in the problems to be considered in this chapter and also Chapter 6, one has to consider dns for which ni(dn — d) -> oo( — oo). This suggests the need for additional assumptions to accommodate this case. These assumptions, labelled as (A5) and (A5'), are introduced in the first section of the present chapter and require that the random vector 60) converges to oo( — oo) in /^-probability whenever Assumptions (A 5), (A 5') are rather mild assumptions to make and for the sake of illustrations, they have been checked and found to be true for the examples of Chapter 2. The relevant discussion is presented in Examples 1.1-1.4 below. [ 85 ]

86

A UMP and A UMPU tests

The testing hypotheses problems referred to above and discussed in this chapter are essentially the following ones: H0:6 = 60(E0) against A: 6 > do(de©) Ho: 6 = d0 against A": 6 * 00 Hx:d ^ 60 against A: 6 > d0. On the basis of results obtained in Chapter 2, it is shown in Theorem 3.1 below that the test defined by (3.2) and (3.3) and based on the random vector An(#0) is asymptotically uniformly most powerful (AUMP; see Definition 3.1) for testing Ho against A above, provided assumptions (A 1-4), (A5) hold. Under assumptions (A 1-4) alone, the test in question is asymptotically locally most powerful (see Corollary 3.1). In the following section, Section 4, we derive the AUMP tests for the illustrative Examples 1.1-1.4 and also compare them with existing tests for the same examples. For testing Ho against A", it is shown in Theorem 5.1 that the test defined by (5.17) and (5.18) is AUMP unbiased (AUMPU; Definition 5.1) under assumptions (A 1-4), (A5), (A5'). Again under (A 1-4) alone, the test is asymptotically locally most powerful unbiased, as Corollary 5.4 asserts. For the proof of Theorem 5.1, one utilizes results regarding the exponential approximation discussed in Chapter 3. At this point, one may refer again to the examples of Section 4 in order to see what the AUMPU tests look like for the concrete examples discussed there. Finally, in Section 6, we consider the third testing hypothesis problem described above and show (see Theorem 6.1) that, under (A 1-4), (A5), (A5'), the test defined by (3.2) and (3.3) is AUMP among all tests which satisfy (6.1). As before, under (A 1-4), the test in question is asymptotically locally most powerful (see Corollary 6.1). Again for the proof of Theorem 6.1, one utilizes results of Chapter 3. Also tests for concrete cases are provided by the tests for Examples 1.1-1.4.

1 Additional assumptions - Examples In the framework assumed in Chapter 2, let XO,XV ...,Xn be n+l r.v.s from the underlying Markov process and let

Additional

assumptions

and examples

87

(f>j(6),j = 1, ...,n be the derivatives in q.m. introduced in (A3). For each de®, define the ^-dimensional random vector An(#) as in (4.2); i.e. n . An(d) = 2n~i S 0,(0). (1.1) 3= 1

In all of this chapter except for Section 9, it will be assumed that 0 is an open subset of R, so that Aw(0) is simply an r.v. Now let doe® and define a) = {de@; 6 > d0}, o)r = {de@; 6 < 60}. (1.2) We can then formulate the following assumptions. (A5) Consider a sequence {6n} with 6neco for all n. Then A^(0O) -> oo in Pn ^-probability whenever ASSUMPTIONS

(A5') Consider a sequence {dn} with dneo)r for all n. Then An(^o) -> — oo in Pnd -probability whenever ni(dn — 60) -> — oo. Of course, o; and o>' above are defined by (1.2) and Aw(#0) is taken from (1.1) for 6 = d0. Before we go any further, it would be appropriate to check and see whether (A5) and (A5r) are satisfied in the Examples 3.1-3.4 discussed in Chapter 2. EXAMPLES

1.1 Refer to Example 3.1 in Chapter 2. Then, by means of (3.2), one has i n AJ0

Clearly

O

)=-2TH2(X,-0

O

).

(1.3)

(T2AJ0O) = n~i £ (X, - 6n) + n*{On - 00) j=i X

and, under Pdn, n~i S ( j - On)is distributed as N(0, a2). There3= 1

fore, for any M > 0,

> M] = P^n-ii^Xj-dJ

> M_n*(0 n -0 o )]

as ni(dn — 60) ->oo. Thus (A 5) is fulfilled. Assumption (A5') is also satisfied because < -if]

as above, whenever ni(8n — 0O)^-— oo.

88

A UMP and A UMP U tests

1.2 Refer to Example 3.2 in Chapter 2. Then by virtue of (3.7), one has

^[j

(1.4)

As it is easily checked, relation (1.4) can be rewritten as follows

(1.5) Now, under Pe , the r.v. 1 j,

j is distributed as x\ a n ( i there-

fore its mean and variance are 1 and 2, respectively. It follows }

-

(L6)

This is so by the normal convergence criterion (see, e.g. Loeve [1], p. 245). Suppose first that 6n < 60 and ni(dn — 60) -> — oo. Then for any M > 0, one has

(1.7) M - -«**-»»('-*) " Mn-

where where

Next for all n > nx = %(ilf), -20jjif > i»*(0 n -0 o ). s o M

.

ni(6n-d0)

tliat

nH0n-

This result, together with (1.6), (1.7) and Polya's lemma (see, e.g. Rao [3], p. 100, (vi)) implies that An(#0) -> — oo in P^-probability. So (A5') is satisfied. Suppose now that 0n>60 and ni(dn — 00) -> oo. Then we distinguish two cases: (i) {dn} stays bounded and let G be a bound of it. In this case and with M as before, one has

Additional assumptions and examples where

26% M M =

"

Next for all n^n2

89

-ni{dn-60)

6J2

(L9)

•

= n2(M), \vh{dn - d0) > 26% M, so that ni(dn-d0)

-*o)

p

*

G2p ^ °°-

Again this result together with (1.6), (1.8) and Polya's lemma implies that An(#0) -> oo in P^w-probability; i.e. (A5) is fulfilled in case (i) above. Consider now (ii) {dn} is unbounded. From (1.8), we have

M] = P6n [ i j i ^ f f

> M'n V(2n) + n]

f(t)dt,

(1.10)

where/is the density of a Xn r-v-> a n ( i i n a similar manner

From (1.9) and (1.12), it readily follows that M*>M'n

if and only if 6n > 6%.

Define {6%} as follows

if 9* —K 8* * 1\260 ot] "90 otherwise. Then 6n ^ 6*. On the other hand, {#*} remains bounded and, clearly, n^{6% — 60) -> oo. But for such a choice of {#*}, case (i) applies and gives Pe*[An(60) > M] -> 1, for any M > 0. This result, together with (1.10), (1.11), and the fact that if* ^ Mn for the above choice of {#*}, implies that P^n[An(^0) > M] -> 1 for any M > 0. The verification of (A 5) is completed. ]

90

A UMP and A UMPU tests

1.3 Refer to Example 3.3 in Chapter 2. Then by means of (3.9) and (3.12) one has that, with improbability equal to one, so that with improbability 1 for all deR, An(0o) =

»

3— 1

Suppose now 6n > 60 and ni(6n - 80) -> oo. Then -flo) = 1] = Pe^-e*

> 0) = Pen(Xx > 0O)

= P0(X± > 60-6n) = Po(X1 + 0 ? l -0 o > 0) the third equality being true because 6 is a location parameter. This result shows that the Pe -distribution of An(#0) is identical n

to the P0-distribution of n~i 2 sgn (Xj + 6n — 60). Consequently 3= 1

for any M > 0, one has Pdn[An(d0) >M] =

Next

0) - 1

as is easily seen, and the second moment of sgn (X,-+ #„ — 6Q), clearly, is equal to 1. It follows that a%sgn (Xt + 6n-60) = exp (
and ^

=^ ~

^

PJAJflo) > if] = Po ( S Z ^ > if r e j, M M n-

[

^

W

M-ni{l-exy[-(dn-60)]} { 2 [ ( 0 W

(1.14) (L15)

Additional assumptions and examples

91

At this point we employ the following inequalities

and

1 — exp ( — x)^^x

if x < log 2

1 — exp { — x)^\

if x ^ log 2.

On the basis of these inequalities, niil - expT nt{l exp L - Id (0 n -
.f

^ _

Thus by setting dn for the denominator in (1.15), one has (2M-ni(dn-60)

.

;f

if

a

^ -

For all sufficiently large n, n ^ ns = ns(M), say, one has lni(0n-dn)>2M Therefore

-ni(6n-60)

and . if if

dn-d0>log2.

Finally by taking into consideration the fact that dn ^ ^2, one has ni(6n-60) , if 0 n - 0 o < l o g 2 4V2 "4 V 2

if
(1.16)

Letting M'n stand for either one of the expressions at the righthand side of (1.16), we have then (1.17) and

(1.18) i=i

j=i

92

A UMP and A UMP U tests

But for each n, Znj,j

= l,...,n are independent with S'QZ^ = 0

n

and 2 0*0 ^n; — !• Then the normal convergence criterion (see 3= 1

Loeve [1], p. 295) applies and gives that nj\0)

j=l

(1.19) 1

This result together with (1.17) and Polya's lemma implies that

Finally relations (1.14), (1.18) and (1.20) show that AJ0 o ) -> oo in i^w-probability; i.e. (A5) is fulfilled. The verification of (A 5') is done in a similar fashion and is left as an exercise (see Exercise 1).J 1.4 Refer to Example 3.4 in Chapter 2. Then by virtueof (3.23), one has

6o)+p{6o)

and as is easily seen, this is rewritten as follows

P (^o) g0)] 2

, *

*s

Additional assumptions and examples

93

Introduce the notation 3

~

Yn = n~\ S (^!-i-l), ^ = n-i J (ZJ- 1), ii

(1.22) and also set Wn = c1Yn + c2Zn + csVn, fin = c 4 a w . Then we have

AJp(0 o )] = Wn +

fin.

(1.23) (1.24)

rv For 0 < x < y, one has exp (— x) — exp ( — y)= exp (— t) At. Jx

Since also exp (— y) ^ exp (-f),we get (y-a)exp(-y) < exp(-x)-exp(-2/). Let now 0 n < 60 and w.*(^w — 60) -> — oo. Then with x = dn and 2/ = 60, the above inequality gives - 0 J - exp ( - d0)] ^ exp ( - 60) ni(dn - 60) -> - oo. Next let #n > 0O and ni(dn — 60) -> oo. Consider two cases: (i) 6n < c. Then with x = 60 and i/ = 0W, the above inequality ] = n*[exp ( - 50) - exp ( - tfj] ^ exp ( - c) ni(dn - 00) -> oo. (ii) Let now {dn} be unbounded. Then for bounded subsequences {dm}, one has by (i) that mi[p(60) —p{0m)] -> oo, whereas for 6m -> oo, one has m*[exp ( - 60) - exp ( - 6J] -> oo exp ( - 60) = oo. Thus with 6n as above, we have ni(6n - 0O) -> oo( - oo) implies ni[p(d0) -p(9n)] -> oo( - oo). (1.25)

94

A UMP and A UMP U tests

At this point we assume for a moment that {Wn} is bounded in P^-probability; i.e. for every e > 0, there exists M — M{e) sufficiently large such that P0n{\Wn\ >M)<e

for all n.

(1.26)

Let us suppose first that ni(dn — d0) -> oo. Then (1.24) in conjunction with (1.25) implies that > M'} = PeJWn+Pn > M') >Pen(\Wn\<M',fin>2M')

^ =

M')-Pen(fin < 2M')

l-P0n(\Wn\>M>)

for any M' > 0 and all sufficiently large n. By choosing Mr > M, we get 1 -P6n(\Wn\ > M') > l-P6n(\Wn\ >M)>\-e

for all n

by means of (1.26). Therefore P0n{An[p(do)] > M'} > 1 - e for all sufficiently large n, which is equivalent to saying that AJ#(# 0 )] -> oo in P^-probability. If we now suppose that ni(dn — do)-> — oo, then it is shown in a similar fashion that An[p(00)] -> — oo in P^-probability. Thus all that remains for us to do is to establish (1.26). We have

+2

S

as follows from Anderson [1], pp. 38-9.

Additional assumptions and examples

95

Now since

[ &W-VI
A™

1

(L27)

71

Next

l

as it follows from the reference last cited. Therefore it follows as before that

((

n

\

y

£[*,_!*,-^(OJ < T^pHdj' By means of (1.22), (1.27)-(1.29), we have then >

and

P,J|c3FM| * Jf)

(L29)

96

A UMP and A UMPU tests

On the other hand, P$n(\Wn\ < 3Jf)

PeJ\cJn\ < M, \c2Zn\ < M, \czVn\ < M) l-P6n(\ClYn\ > M)-Pen(\c2Zn\

so that

> M)-PeJ\czVn\ > M),

Pdn(\Wn\ < 3M) > 1 - T l ^ f A -

(L30)

Therefore if dn > d0, equivalently, p2(0n) < P2(OO), one has by means of

*^±pi.

(1.81)

It follows that if for a given e > 0, M is chosen so that

then i^n(|W^| < 3if) > 1 - e , which is what (1.26) asserts. If we are interested in testing the hypothesis Ho: 6 > d0 against the alternative A': 6 < d0, in which case dn < 60, or equivalently, P2(6n) > p2(d0), then we impose the (not serious) restriction that p{6) is bounded away from one, i.e. p\d) ^ 1 — S, 6 e 0, for some S > 0. Then Pl\W\

and a repetition of the previous argument establishes relation (1.26) again. To summarize: (A5') is satisfied and so is (A5) under the restriction that p{6) is bounded away from one. ] 2 Some lemmas In the testing hypotheses problems to be discussed in this monograph, the following auxiliary results will prove useful. They are formulated and proved here for easy reference.

Some lemmas

97

L E M M A 2.1 Let {Yn} be a sequence of r.v.s defined on some measurable space (Q, J^), and for each n, let Qn be a probability measure on J^. It is assumed that w | r a ) . Then (i) For any numerical sequence {yn}, one has

(2.1)

Let now the sequences of numbers {cn} and {yn} with 0 ^ yn ^ 1 for all n be denned by CW(FW > cn) + 7nQn(Yn = cn) = a

(0 < a < 1).

Then (ii) cn -> fi + 0, one has

(2.2)

The first two terms on the right-hand side of (2.2) converge to zero by (2.1) and Polya's lemma. As for the third term, we have

4\ 2) for some zn with while

e l / z*\ e 1 - ;,„ ,exp( - - g l ^ - „„ —. exD I — - I ^ —; a* (2TT)

\

2/

v ->

0 as >0

e->0.

cr /(2TT)

Therefore bounding the third term on the right-hand side of (2.2) by

lic% x—

and then taking the limits first as n -> oo and then as

e -> 0, it follows that # j r w = yn) -> 0.

98

A UMP and A UMP U tests

(ii) We have a = lim [Qn(Yn > cn) + yn Qn(Yn = cj] = lim Qn(Yn > c j or

by (i);

l-lim^(cj = a

from which it follows that lim Fn(cn) = 1-a. But as above

lim \Fn(cn) -
Therefore hence

<J> ( ^ ^ ) -> 1 - a, C — II JL L

-^r -+i*

and

as was to be shown. | The following result was stated in LeCam [5] without proof. LEMMA 2.2 Consider the measurable space (Q,,^) and let P and Q be two probability measures on !F such that P x, Q. Let / = dP/d/6, g = dQ/d/i for some dominating cr-finite measure /i (e.g. /I = P or Q or P + Q) and set Z = log (g/f). Then, for every e > 0, one has

||P-G|| ^ 2 [ l - e x p ( - e ) ] + 2P(|Z| > e). Proof By Theorem 1.4 A, we have \\P-Q\\= 2sup[\P(A)-Q(A)\;

Ae^]=j\f-g\d,i.

Let now B = (f-g > 0). Then

!\f-9\fy=f \f-9\ty+( \ J

JB

J B°

= f (f-g)dfi+( (gJB

J Bc

= [P(B) - Q(B)] + [Q(B") - P(B°)] = 2[P(B) so that

\\P-Q\\ = 2[P(B)-Q(B)l

(2.3)

Some lemmas

99

SetC = (|Z| >e). Then P(B) - Q(B) = P(B n G)+P(B n Cc) - Q(B n 0) - Q(B n <7C) ^ P ( 0 ) + P ( 5 n c c ) - Q(B n Cc) ^P(|Z| >e) + P(B[\Cc)-Q(Bf)Ce).

(2.4)

But C

f

(1 Cc) =

J Br\Qe

gd/i =

Q C |/d/^ +

J BnCen (/> 0) /

J f i n ^ n (/= C

exp Z dP ^ exp ( - e) P ( 5 n Gc)

(2.5)

because onB,f>g and hence/ > 0, and onOc, \Z\ ^ e, so that exp( —e) ^ exp(Z). Relations (2.3)-(2.5) show that <2[l-exp(-e)] + 2P(|Z| > e), as was to be seen. ] 3 Testing a simple hypothesis against one-sided alternatives In this section, we consider the problem of testing the simple hypothesis Ho: 6 = 60(e&) against the (composite) alternative A: deo), where OJ is defined by (1.2). We shall restrict ourselves to sequences of tests whose level of significance is a and we shall exhibit such a sequence with the optimal property of being asymptotically uniformly most powerful (or asymptotically most powerful in Wald's [2] terminology), according to the following definition. DEFINITION

3.1 For testing Ho: 6 = d0 against A:dea) = {6e@;6 > d0},

the sequence of level cc tests {An} is said to be asymptotically uniformly most powerful (AUMP) if for any other sequence of level a tests {o)n}y one has limsup[sup(^a> n ~^A n ; dea))] ^ 0.

(3.1) 7"2

100

A UMP and A UMP U tests

R E M A R K 3.1 Now objections have been raised regarding the optimality of an AUMP sequence of tests. Actually Lehmann [1] (see also Kendall and Stuart [1], p. 263, Example 25.1) constructed an example for which two sequences of tests, which are both AUMP, have the property that the quotient of the sequence of type II errors converges to infinity. This shows that the criterion for a sequence of tests of being AUMP is not a very selective one. Perhaps an additional criterion ought to be superimposed which would single out an 'efficient' sequence of AUMP tests. Since the purpose of the present section is to illustrate how the results obtained in Chapter 2 may be used for discussing statistical problems, rather than breaking new ground (in the sense of selecting an' efficient' AUMP test), we adhere to Definition 3.1. I am indebted to Dr O. Krafft for bringing to my attention the references cited in the remark. Let An(#) be given by (1.1) and define the sequence {cn\ tn = faW(6o)] = \Y» if K(Go) = ^n\ (3-2) 10 otherwise, J

where the sequences {cn} and {yn} are determined by the requirement (3-3) ^ $e$n = a f° r aU nThen we have the following result. T H E O R E M 3.1 Under assumptions (A 1-4), (A5), the sequence of tests {^J defined by (3.2) and (3.3) is AUMP for testing Ho: 6 = 60 against A: deco, where co is given by (1.2).

Proof The proof is by contradiction. Suppose that {n} is not AUMP. Then (3.1) is violated for some sequence of level oc tests {o)n}, and let the left-hand side of (3.1) have the value S > 0. limsup [sup (&eo)n-£'dn;de(o)]

= 8.

(3.4)

Then there exists a subsequence {m} c {n} and a sequence {dm} with dmeo) for all m such that

Simple against one-sided alternatives

101

Prom hereafter, the main steps in the proof may be summarized as follows. We first show that (3.5) cannot happen if the sequence {6m} is such that {mi(dm — 60)} is unbounded. For this, we employ assumption (A5), Theorem 4.2 in Chapter 2 and Lemma 2.1 herein. Next we show that (3.5) cannot occur either if the sequence {6m} is such that {mi(6m — 6Q)} remains bounded. For this purpose, we employ Theorems 4.1, 4.5, 4.6 of Chapter 2, Lemmas 2.1, 2.2 herein and the Neyman-Pearson fundamental lemma. Consider the sequence {mi(dm — 60)} and suppose first that it is unbounded. Then there exists a subsequence {r} c {m} such that ri(dr — d0) -> oo. Then assumption (A 5) implies Ar(<90) -> oo in P^-probability

(3.6)

(this is the only instance where (A 5) is employed). Now by setting for simplicity a2(60) = 4^o[^1(<9o)]2, Theorem 4.2 in Chapter 2 ^r/r. 9/n xx implies that r/?rK /n x . ^ ., Then Lemma 2.1 applies with Yn = An(d0),

Qn = P6o,

fi = O a n d

cr* =
and gives that {cn} stays bounded. This fact together with (3.6) implies that > Gri ~> 1

so that

and

P6r[Ar(d0)

= cr] -> 0,

Se^)r-> 1.

This result together with (3.5) (with m replaced by r) implies that (3.4) cannot occur. Now consider the case that {mi(6m-60)} is bounded. Then there exists a subsequence {s} c {m} such that s§(ds — 60)->h ^ 0. First we deal with the case that h > 0. To this end, set so that ds = do + hss~i with hs->h. Then by Theorem 4.6 in Chapter 2, one gets

Then Lemma 2.1 applies again with {n} replaced by {s} and with Ys = h&s(60),

Q. = P9t

and fi = a* =

102

A UMP and A UMPU tests

It follows then that Pes[hAs(d0) = hcs]^0.

(3.7)

Also by the same lemma applied with Ys as above, Qs = P0Q and fi = 0, we get hcs -> hor(60) £a, so that

= 1-«»[£,-Aor( hcs] -+ 1 -
i.e.

(3.8)

Eelations (3.7) and (3.8) imply that * , , & - • l-[£a-fto-(<90)].

(3.9)

Next define the sequence {i/rs} as follows fl if = \SS if A(^ O) ^) + p V ^ o ) = ^ lO otherwise, J

(310)

where the sequences {d^ and {S^ are determined by the requirement

*rf.

= a for all,,

(3.11)

and recall that A(00,0.) = log [dPs> ^/dPs> 0 J. By Theorem 4.5 in Chapter 2, J?[A(60,6S) + lh*a*(do)\Pes] =>N(h* Thus by virtue of (3.9) and (3.12),

(3.12)

(3-13) * « , & - * » > . - • °Replacing m by s in (3.5) (which can be done since {s} £ {m}), (3.14)

Simple against one-sided alternatives

103

Thus, on account of (3.13) and (3.14), S. Hence for all sufficiently large s, s ^ sv say, one has For each s ^ sl9 consider the problem of testing the simple hypothesis Ho: 6 = 60 against the simple alternative As: 6 = # s at level a. Then by the Neyman-Pearson fundamental lemma the test denned by (3.10) and (3.11) is the most powerful one. Accordingly, relation (3.15) cannot occur. We arrived at (3.15) by assuming (3.5) and the existence of {s} c= {m} and dseo) such that si(ds-6Q) -> h > 0. Therefore (3.5) and hence (3.4) cannot be true. Finally, it remains for us to show that (3.4) is also not true for the case that {mi(6m — d0)} is bounded and the only convergent subsequence converges to zero. To this effect, let {s} ^ \m) and eseo) for all s be such that si(6s- 60) = hs -> 0. Then Theorem 4.1 in Chapter 2 implies that A(#o> @s) -> °

in

^-probability,

so that, for every e > 0, P,o[|A(0o,0a)| > e ] - > 0 .

(3.16)

Now Lemma 2.2 applies with P = P6o,

Q = P6s

and Z = A(do,ds)

and gives

||P, o -PJ < 2[l-exp(-e)]

+ 2PeJi\A(8o,0t)\ > e\.

This result together with (3.16) implies that HA, ~PeJi\^O.

(3.17)

Next > cJ + r,P,o[A.(0o) = c j i-PflJ^0

so that

by (3.17),

&e<j>s-+a.

(3.18)

104

A UMP and A UMP U tests

In a manner entirely similar to the one used for obtaining (3.18), one also concludes that

so that

&0/f>8-&e8ft8 -* °-

( 3 - 19 )

But relation (3.19) is the same as (3.13) which led us to a contradiction to (3.4). Therefore, (3.19) implies that (3.4) cannot hold and the proof of the theorem is concluded. ] As was pointed out in the course of the proof of Theorem 3.1, assumption (A5) was only used for the purpose of arriving at a contradiction to (3.4) in the case that the alternatives 6n either do not approach 60 sufficiently fast or stay away from it, in the sense that {ni(6n — 60)} is unbounded. Therefore, if we restrict ourselves to alternatives 6n which tend to 60 at the appropriate rate, in the sense that {ni(dn — 60)} stays bounded, the proof of Theorem 3.1 has the following corollary. COROLLARY 3.1 Under the assumptions (A 1-4), the sequence of tests {fin} defined by (3.2) and (3.3) is asymptotically locally most powerful for testing Ho: 6 = 60 against A: deo). That is, the sequence {n} satisfies (3.1) when the parameter values are restricted to those 6s in a) for which {ni(6 — 60)} remains bounded from above. When the alternatives lie on the other side of the hypothesis being tested, a theorem and a corollary analogous to Theorem 3.1 and Corollary 3.1 also hold true; their proof is entirely analogous to the one given. More precisely, define {0^} by

(3.20)

where the sequences {c^} and {y^} are determined by the requirement <^ o 0; = a for all n. (3.21) Then we have T H E O R E M 3.2 Under assumptions (Al-4), (A5'), the sequence of tests {fa} defined by (3.20) and (3.21) is AUMP for testing Ho: 6 = 60 against A': dew', where G/ is given by (1.2).

Simple against one-sided alternatives

105

COROLLARY 3.2 Under the assumptions (A 1-4), the sequence of tests {^^} defined by (3.20) and (3.21) is asymptotically locally most powerful for testing Ho: 6 = 60 against A': deco'. That is, the sequence {^} satisfies (3.1) when the parameter values are restricted to those 6s in o)' for which {ni(d — 6Q)} remains bounded from below. A discussion of the testing problem just concluded is found in Johnson and Roussas [1].

4 AUMP tests for the examples of Section 1 In [2], Wald considered the problem of constructing an AUMP test when the underlying process consists of i.i.d. r.v.s. The tests are based on the maximum likelihood estimate (MLE) and the regularity conditions used are quite stringent. Wald's result has not been carried over to the Markov case to the best of our knowledge. In the present framework, the Markov case is also covered and no reference whatsoever is made to the MLE; its role is played here by the r.v. An(#0). Furthermore the assumptions employed are substantially weaker than those used by Wald. For the purpose of comparing tests, it should be mentioned that for testing HQ: 6 = 60 against A: deco at level a, Wald's tests {wn} are defined as follows rl if Qn>cn\ (4.1) wn = wn(X1,...,Xn) = \yn if Bn = cA lo otherwise, J where 6n is the MLE of 6 and the sequences {cn} and {yn} are defined by the requirement £0QWn = a

for all n.

(4.2)

In the following examples only tests for the alternatives A are considered since the tests for the alternatives A' are constructed symmetrically.

106

A UMP and A UMPU tests

EXAMPLES

4.1 Refer to Example 1.1. In this case, &n(00) is given by (1.3) and is as follows

Therefore the test in (3.2) rejects Ho whenever &n{00) > cn, or equivalently, whenever n%(Xn — 60) > c*, where c* = cr2cn. Since Xn = 0n the present test is the same as that of Wald and both coincide with the existing UMP test for all n. 4.2

Refer to Example 1.2. Here by virture of (1.4), one has

n

where Si = n~x S (Xj-J^)2- Therefore the test in (3.2) rejects Ho 5= 1

whenever An(^0) > cn, or equivalently, whenever n%(S\ — d^ > c*, where c* = %d\cn. Again 8\ = B\ and therefore the present test is the same as that of Wald and both coincide with the existing UMP test for all n. 4.3 Refer to Example 1.3. From (1.9), one has with Pe probability equal to one

and therefore the test in (3.2) rejects Ho whenever An(60) > cn, or equivalently, whenever

where

2

Now

2 ^

Pe<)[IK ^(X,) = 1] = P, o (X, > 60) = \.

Therefore in (4.3) the constant c* can be determined exactly by means of the binomial tables for a moderate sample size, since, n

under Pe , 2 he ao)(Xj) is distributed as B(n} | ) . Of course, both 0

5=1

°'

AUMP tests

107

here and in all other examples the cut-off points can always be determined approximately, since An(#0) is asymptotically normal, under Pe . Actually, here An(#0) is asymptotically N(0,1), under P6Q, because F(d) = 1 by (3.13) in Chapter 2. By taking into consideration the fact that 60 is the population median (mean), the test in (4.3) has an intuitive interpretation; namely, it rejects Ho if too many Xs exceed 6Q. It is well known that the MLE of 6 in the present case is the sample median. Therefore the test based on the MLE rejects Ho whenever ni(Qn — #0) > cn, or equivalently, whenever Furthermore, it is shown (see, e.g. Rao [3], p. 355) that so that the two tests are asymptotically equivalent. For any n, it can be shown (see Exercise 2) that S / a «)(*,)> iw if and only if 0n > 0O;

(4.4)

that is, the two tests are equivalent, provided the adjusting constants - -r and -^ are omitted. 2yw ^n 4.4 Refer to Example 1.4. Here &n[p(d0)] *s given by (1.21) and the test in (3.2) can be carried out explicitly. In the present case, Wald's test based on the MLE is not directly applicable, since the relevant theory has not been carried over to the Markov case (to the best of our knowledge). Also the MLE is not easy to obtain (see Exercise 3).

5 Testing a simple hypothesis against two-sided alternatives Consider the sets o) and o)f defined by (1.2) and set G)" =

0) U
In the present section, the problem of testing the hypothesis Ho: 6 = 6Q(E@) against the alternative A": deo)" is taken up.

108

A UMP and A UMP U tests

One would not possibly expect the existence of an AUMP test. However, an AUMP unbiased (or AMP unbiased in Wald's [2] terminology), according to the following definition, does exist, as is shown herein. DEFINITION

5.1 For testing Ho: 6 = 60 against

the sequence of tests {An} is said to be asymptotically unbiased if liminf [inf (^A n ; flew")] ^ a and is said to be AUMP unbiased (AUMPU) of asymptotic level of significance a if it is asymptotically unbiased and of asymptotic level a and limsupfsup^w^-^A^; dew")] < 0 (5.1) for any sequence of asymptotically unbiased and of asymptotic level of significance a tests {o)n}. When the sample size n is fixed, the unbiasedness of a test requires that its power remains ^ a. When n is allowed to increase indefinitely, it seems natural to replace the concept of unbiasedness of a test by asymptotic unbiasedness, as is done in Definition 5.1. The approach used in this section (by employing results of Chapter 3) also makes it possible to consider sequences of tests of asymptotic level of significance CL rather than of exact level a. The main result in this section is Theorem 5.1. The auxiliary results presented below will facilitate the proof of the theorem. The following lemma is a version of the Helly-Bray theorem (see Loeve [1], p. 182) and is of some interest in its own right. L E MM A 5.1 Let {Fn} be a sequence of d.f.s on R and let {fn} be a sequence of real-valued, measurable functions defined on R. Also let F be a d.f. on R and let/and g be real-valued, continuous functions defined on R. It is assumed that (i) Fn=>F>

(ii) |/ n (#)| < g(x) for all n and all xeR, (iii) fn(x) ->/(#) uniformly on finite intervals, (iv) \g dFn -> \g dF and \g dF is finite. Then

f/ n dF n -> [f&F and

[f&F is finite.

Simple against two-sided alternatives

109

Proof For any c,deR with c < d, one clearly has \\fdF-

\fdF ^ I

\J

J

I f I dJ? 7

I f I dF + I

J(-OO,C]

J(d, OO)

+f

|/|dF+f

|/|dF (5.2)

+ lf f d F - ! IJ (c, d]

J (c,

On account of (ii) and the fact that

|f

fndFn-f

IJ(c,d]

/cLF

(c,d]

J(c,d]

\fn-f\^Fn

•If

fdFn-(

fdF

inequality (5.2) becomes as follows (-oo,c]

W, oo)

-f

l/|cLF+f \f\dF

J (—QOf c]

"f

J (d, oo)

l/n-ZW + lf

J(c,d]

or

fdFn-\ fdF

\J(c,d]

J(c,d]

flrdF.- f flrd^+ f

f l/l^ J ( d , oo) (c,d)

(5.3)

Now by means of (ii)-(iv), it follows that j\f\ dF < oo. Thus, for every e > 0, we may choose c and d to be continuity points of F and such that

f

|/|df<J, f

| oo)

110

A UMP and A UMP U tests

|W W- ff gdF< dFee-

and

J (c, d]

J

(5.4)

b

By (iv) and the Helly-Bray lemma (see Loeve [1], p. 180), one has

fflrd^-f J

gdFn^(gdF-(

J (C,d]

J

gdF. J (C, d]

Therefore by means of the last inequality in (5.4), one has for all sufficiently large n

fi, d]

u

Furthermore, for all sufficiently large n, one also has

f

\fn-f\dFn
and If

fdFn-[

fdF

|

(5.6)

because of (iii) and the Helly-Bray lemma, respectively. By virtue of (5.4)-(5.6) and for all sufficiently large n, inequality (5.3) becomes \jfndFn — jfdF\ < e, as was to be shown.] COROLLARY 5.1 Let / * =fnl(an,bn) and / * =fl(a>b), where an-+a,bn->b and a, b are continuity points oiF in the case they are finite. Then under the assumptions of the foregoing lemma, we have

Proof Since |/*(#)| < |/ n (#)|, so that (ii) is satisfied, it is clear that (5.3) holds true with/ n and/being replaced b y / * and/*, respectively. Also |/*(#)| < |/(#)| and hence (5.4) is valid and so is (5.5). Thus it only remains for us to establish relations (5.6). If a = —oo and b = oo,then/* = / a n d hence the second inequality in (5.6) holds true. Also, for all sufficiently large n, an < c and bn > d, so that / * =fn on (c, d]. Hence the first inequality in (5.6) is also valid and the conclusion follows. Suppose next that a is finite and b = oo. In this case, we assume that a> c, since otherwise the previous argument applies again. Without loss of generality, we also assume that a < d. There is S± > 0 such that

Simple against two-sided alternatives

111

for all sufficiently large n, c < a — S± < an < a + S± < d and a — Sv Sx are continuity points of F. Therefore

f |/*-/*|d^=f J(c,d]

\ft-f*\dFn J(a-Sltd]

= f \ft-f*\Wn+( J /i

\fn-f\dFn, J (a+8u d]

where Ix = (a — S^a + 8^; / * was replaced b y / n above because 6n > d for all sufficiently large n. Since

| / n — / | d J n < e for all sufficiently large n by J (a+Su d]

means of (iii), we need only be concerned with

|/*—/*| d-F^. J ii

In connection with this, we have

f |/*-/*|dF B < f \f*n\dFn+\ \f*\dFn

^ 2 Furthermore,

gdFn->

^ d i ^ by means of (ii).

gdF and by the fact that a is a

continuity point of F, one can choose Sx as before and also such that

gdF < e. It follows that for all sufficiently large n,

f J (c

Next

If

f*6Fn-\

I •/ (c, d]

f*dF = f J (c, d]

fdFn-(

»/ (fib, d]

j J (a, d]

for all sufficiently large n. It follows that the two relations in (5.6) hold true (with - replaced by 5e and e, respectively) and hence the corollary itself. The case that a = — oo and b is finite, as well as the case that both a and b are finite, are treated in an entirely similar way. ]

112

A UMP and A UMP U tests

LEMMA 5.2 Let A* be defined by (1.5) in Chapter 3 when 6 = d0. Then one has

and

$n I A* I -

Proof In the inequality 1 + x ^ exp x, x ^ 0, replace x by |A 0 |A*|, where Ao is as in (1.4) of Chapter 3. Then |1 + R A * | ^ l + iA0|A*| < exp(4A0|A*|), and hence (l + iA0A*)«*S (l + lAo|A*|)«< exp(A0|A*|). It follows that «f0oexp(Ao|A*|) < M(< oo) for all n by Propositions 1.2 and 2.2 in Chapter 3. By setting Fn = JS?(A*|/^ O ), one has then

J

o

; ^ if

for all n

(5.7)

and J Also

Jf

for all n. (5.7')

^(A*|P,,) = Fn * J^(0, F)

(5.8)

by Proposition 2.4 (i) in Chapter 3. By means of (5.7), (5.7') and (5.8), the corollary on p. 184 in Loeve[l] applies and gives

j and

)

J ( l + iAo|a?|)dFn->J(l + iAoH)dar(O,r).

(5.9) (5.9')

Since jxdN(O, T) = 0, relations (5.9) and (5.9') are equivalent to

Simple against two-sided alternatives

113

and respectively, as was to be shown. ] LEMMA

follows

5.3 Let the sequence of tests, {$J*(A*)}, be defined as (I

if A* < a* or A* > 6* ^

7*n or y j w

if A* = a* or A* = &*, respectively,

« < K), (5.10)

VO if a*
= CC

and ^0[A*^*(A*)] = a^ o A*

for all ft (0 < a < 1).

(5.11)

Then a* -> - gj a and 6* -> gj a , where £p is the upper ^)th quantile Proof We have (5.12) Now (5.8) and Lemma 2.1 (i) yield P,o(A* = a * ) ^ O

and

P,o(A* = 6*) -> 0.

(5.13)

Next we assert that a* ^ Mx{ < oo) and ( - oo <) M2 6* for all ft. In fact, suppose that {a*} is not bounded above. Then there exists {a^} c {a*} such that a^ -> oo and since a^ < 6^, one also has 6^ -> oo. But then relations (5.11)-(5.13) imply that

a contradiction. The case in which {&*} is assumed to be unbounded from below is treated symmetrically. Furthermore we claim that both {a*} and {6*} are bounded from both below and above. In fact, suppose that {a*} is not bounded from below. Then there exists {a^} g: {a*} such that

114

A UMP and A UMPU tests

(1% -> — oo. Look at {b^} and suppose first that it is unbounded from above. Then there exists {&*} <= {b^} such that 6* -> oo. Thus a* -» -oo and 6* -> oo. Then relations (5.11)-(5.13) imply a contradiction. Next suppose that {b%} is bounded. Then there exists {&*} c= {6^} such that 6* -» 6( < oo). Thus a* -> — oo and b* ->b(< oo). But by (5.13), one has

= limff

6J, oo)

(5.14) Furthermore the assumptions of Lemma 5.1 are fulfilled with 3 = JS?(A?|P,O), ir = ^(O,r), fn(x)=f(x) and

=x

^(a^) = |#|;

the convergence in (iv) is satisfied because of Lemma 5.2. Then Corollary 5.1 applies in an obvious manner and gives

and

f

adJS?(A*|P0)-> f

J(&J, oo)

°

xdN{O,T).

(5.15)

J(6, oo)

Also

*,§A? = f 0

o;di;+ f ^

J(-oo,o*)

ajd^ + afP^A? = a?)

J « oo)

°

and hence (5.15) and Lemma 5.2 imply that 0. Therefore taking the limits on both sides of the second relation in (5.11) and utilizing the last result obtained above along with Lemma 5.2 and relations (5.14) and (5.15), one has |

x

J (b, oo)

a contradiction. The case that {&*} is assumed to be unbounded from above is treated symmetrically.

Simple against two-sided alternatives

115

Therefore both {a*} and {&*} are bounded. Then there exists {m} c {n} such that a^->a and b^->b. From (5.14), with r replaced by m, it follows then as above that lim*,o[A20*(A*)] = f

xdN(0,T).

(5.16)

J [a, 6]«

Then Lemma 5.2 together with (5.11) and (5.16) implies

L

xdN(0,F) = 0.

' [a, b]'

Hence a = —b. Taking this into consideration and letting n->co through the subsequence {m} in the first equation in (5.11), one obtains

dN(0, F) = 1 - a; thus, a^ -> - gi a and 6^ ->- £i a . J[-b,b]

*

*

Since all convergent subsequences converge to the same limits, it follows that a% -> — £^a and 6* -> gj a , as was to be shown. ] COROLLARY 5.2 Consider the sequence {hn} such that hn->heR and set 6n = do + hnn~b. Then , F). Proof From Proposition 2.4 (ii) in Chapter 3, one has Then the corollary is an immediate consequence of Lemma 2.1 (i), the fact that a* -> — £j a , 6* -> ^ j a and Polya's lemma. J Let An = An(#0) be given by (1.1) for 6 = 60 and define the sequence {(f>n} as follows.

where the sequences {an} and {6n} are chosen, so that «n->-£ia

and

6

n->^

(5.18)

and £p is the upper ^?th quantile of the N(0, F). Then we have the following result. THEOREM

5.1 Under assumptions (A 1-5), (A5'), the 8-2

116

A UMP and A UMPU tests

sequence of tests {n(An)} defined by (5.17) and (5.18) is AUMPU of asymptotic level of significance a for testing Ho: 6 = 60 against A": deo)", where GJ" is given in Definition 5.1. Proof The proof of the theorem is presented in the following three stages. We first show that {n(An)} satisfies the condition of asymptotic unbiasedness; this is done by contradiction. Finally, it is shown that {(An\Pdo) => JV(O, T) by Theorem 4.2 in Chapter 2. Next we show that the condition of asymptotic unbiasedness holds for {^(Aw)}, namely liminf{inf [ge<j>n(kn)\ 6ecu"]} ^ oc.

(5.19)

For contradiction, suppose (5.19) is not true and let the expression on the left-hand side of (5.19) be equal to 8 < oc. Then there exists a subsequence {m} c {n} and a sequence {#m} with 6meo)" JP A (\ \ ^ & (KOCW for all m such that #om9JAm) -> o. (5.20) First suppose that {m^(6m - 60)} is unbounded. Then there exists {r} cz {m} with dr < dQ for all r or dr > d0 for all r such that, respectively, ri(dr-d0)]->-oD or ri(dr-d0)->oo. (5.21) But

br)

(5.22)

and therefore (5.18) together with (5.21) and (A5) or (A5'), respectively, implies that Se 1 which contradicts (5.20).

Simple against two-sided alternatives

117

Next suppose that {mi(dm — 60)} remains bounded. Then there exists {s} c: {m} such that si(ds — 60) -> In. Setting hs = si(6s — 60), we have then 6S = 00 + hss~i with hs -> h. But then {P8y6 } and {PStes} are contiguous by Proposition 6.1 in Chapter 2, so that Theorem 4.6 in the same chapter applies and gives ).

(5.23)

If h = 0, then (5.22) with (r replaced by s) implies by means of (5.18) and (5.23) that &0s$8(A8) -> a which contradicts (5.20); if/&*0, then ) > gjj > a.

Once more we have a contradiction to (5.20). Therefore (5.19) holds true. It remains for us to show that {
(5.24)

The proof is by contradiction. Suppose that (5.24) is not true and let the left-hand side in (5.24) be equal to 88 for some 8 > 0. Then there exists a subsequence {m} c: {n} and a sequence {6m} with dm e to" for all m such that ^mo>m-^yAJ^8«*.

(5.25)

Consider the sequence {mi(dm — d0)} and suppose first that it is unbounded. Then there exists {r} c {m} and {dr} with 6r < 60 for all r or dr > 60 for all r such that, respectively, r i(0 r _0 o )->-oo

or ri(dr-d0)->oo.

(5.26)

In this case, repeating the arguments employed in connection with (5.21) above, we conclude that Se 1, so that (5.25) cannot be true. Next suppose that {m^(6m — #0)} is bounded. Then there exists {s} c {m} such that si(ds — 60) -> h. Setting h8 =

sk(e8-0o),

we have then 6S = d0 + hss~i with hs->h. Once more the sequences {PSt0} and {Ps0 } are contiguous as was mentioned and justified above. Now the test ojn is a function of the Xs. However, from

118

A UMP and A UMPU tests

an asymptotic power point of view, we can replace it by (dn(An), where &Jn(An) = <^o(G;jAn).This issobyTheorem5.1in Chapter 3 according to which (?0sG)s-+O and O.

(5.27)

Thus (5.25) (with m replaced by s) and (5.27) imply that *eBa(ba)-feat8(\)>78

for all s > nv

say. (5.28)

Next for any sequence of tests {^n} defined on R, one has

< 2P6s(As + A*) -> 0,

(5.29)

the convergence to zero being true because of Proposition 2.3 (ii) of Chapter 3. So and, in particular, so that for all sufficiently large s9 s ^ n2, say, one has SQ (os(Af) — $ Q (os(As) > — 8

and $$rf>s(As)— $ $ — 8.

(5.30) Adding up (5.28) and (5.30), and letting n3 = max (nv n2), one has geTD8(k*) - ^,0 8 (A*) > 5* for all 5 ^ n3.

(5.31)

Next, as it was mentioned in the proof of Corollary 5.2, one has that and consequently

J?(Af\P0s)^N(rh,T)

Thus dN(Th, T) - ^J 8 (A*) < 8 for all s ^ n^

say. (5.32)

Simple against two-sided alternatives

119

Now the convergence in (5.19) is still true if {6S} = {0O}, on account of Proposition 2.3 (i). Therefore and hence, by means of this result and also (5.27), one has a.

(5.33)

At this point, look at the probability measure Rs ^ £

= exp [A, A? "B 8 (h 8 )] 9

hg

defined by

exp [Bs(h8)] = S6Qexp (hsA?)

(5.34) (see also (1.6), (1.7) and Theorem 3.1 in Chapter 3). In the family of probability measures defined by (5.34), consider the problem of testing Ho h: hs = 0 against A'^\ hs =f= 0 at level ccs for all sufficiently large s, s ^ n5, say, so that 0 < as < 1. Then the test defined below is a UMPU test. (See, e.g. Lehmann [3], p. 126.) Namely, (1 if A* < as or As* > b_s (a8 < b8) or s(K) = 17i,s 72,s i f A * = «« o r A ? = K respectively, lO if a , < A * < 6 * , (5.35) where the constants as, bs, yx s and y2s are defined by the requirements

^ A ) | i ? 8 > 0 ] = as and

I

^[A*^(A*)|i? s0 ] = « , ^ ( A * | ^ 0 )

for aU s> n&. j (°A >

From (5.34) it follows that Bs0 = P s > v Thus (5.36) becomes and

^[AV(A*)] = a ^ ( A * )

foralls^J

(

*>

Since ag -> a by (5.33), it follows that

f

J(|a?l>gj«,)

so that f

dN(Th, T) -> f J(N>^ a )

di^(rA, T) - f

J(M>£l«.)

diVr(rA, T),

J(W>^a)

dN(Fh, T) < 8 for all s ^ n6, say. (5.37) (537)

120

A UMP and A UMP U tests

Ako

_

f

**&(*?)-

dN(Th,r)->0 J (1*1 >£*«.)

by Corollary 5.3,

so that - f

dN(rA, r) < * for all s ^ n7)

J(\x\>Zi*J

say. (5.38)

Letting now s ^ n8 = max (w4, %6, w7) and adding up (5.32), (5.37) and (5.38), one has * * , f t ( A * ) - ^ , ( A * ) < 3*. Since <^&(A*) = ^QSJ8(A?) = a s and have ^s^f)\Rs>hs]-^s(Af)\RSths]

(5.39)

?8(A*) is UMPU, we

< S for all s > n,, say. (5.40)

The detailed and rigorous justification of (5.40) is given in Lemma 5.4 below in order not to interrupt the proof of the theorem. But

|*, f S.(A*)-*K(\*)Kjl ^ \\Ps>0s-Bs>hs\\->O by Theorem 3.1 in Chapter 3, and therefore
-> 0.

(5.41)

(5.42)

so that hj means of (5.40)-(5.42) and for all s ^ n10, say, one has

>

s

i

s

23. (5.43)

From (5.39) and (5.43) and for all s ^ nlx = max(%,^ ]0 ), we obtain & T T / A * \ & A (\*\ ^ KX However, this contradicts (5.31). The proof of the theorem is completed. ] As in the case of Theorem 3.1 (and also Theorem 3.2), from the proof of Theorem 5.1 it is immediate that, as long as one restricts himself to alternatives 6 for which {n$(d - 0O)} stays bounded, one has

Simple against two-sided alternatives

121

COROLLARY 5.3 Under assumptions (A 1-4), the sequence of tests { 0O. R E M A R K 5.1 In order to see the explicit form of the test in the case of the concrete examples that we have been dealing with, the reader is referred to Section 4. L E M M A 5.4 In connection with the family of probability measures {RS)hs} defined by (5.34), consider the sequence of tests {ws(A*)} such that &0oGJ8(h*) = as -> a (see (5.33)). Then

A*)|^,J < 8 for all sufficiently large s,

(5.44)

where 8 is the one employed in the proof of Theorem 5.1. Proof For each s ^ n5, where n5 appears after relationship (5.34), consider the test aJs(/Af) with <^OQ WS(A*) = as. Then there exists a test cDs(A*) of the form 11 if A* < as or A* > bs (as < bs) &s(^f) = | fi, s or y2t s if A* = ds or A * = t>89 respectively, 10 if a a < A * < 5 a such that and

**O0.(A*) = as ) £[G>s(Af)\Rs>ht] > ^[ffl a (A?)|^ v ]J

( M)

*

where -Rs>^* is defined by (5.34) with hs replaced by hf -> h* This is so by Theorem 2, p. 220 in Ferguson [1]. Now the fact that liminf [inf ($Qo)n\ 6 4= #0)] ^ a, implies that liminf $e*(*)s ^ oc for any sequence {6%} with 6f =# 60 for all s. (5.46) In fact, $e*o)s ^ inf ((oeo)s; 6 == j 60) and hence liminf eo)s; 6 4= ^0)] ^ a.

122

A UMP and A UMPU tests

Let now 0* = 60 + h*s-i. Then by Theorem 5.1, Chapter 3, one has $e*UJs{&8) — $e*G)s-^0. This result together with (5.46) implies in an obvious manner that ws(As) ^ a.

(5.47)

Next, as in (5.29), ?)| ^ 2Pdt(As + &f) -* 0, so that &0*(d8(kf) — &o*(ii>8(k8)->O. This result together with (5.47) implies as above that liminf<^*ws(A*) ^ a. But

K, r ^(A*)-^K(A*)|i?,,, r ]| ^ \\Pet-Bs>ht\\

(5.48) -> 0

by Theorem 3.1, Chapter 3, so that This result together with (5.48) gives (A*)|2J8ffcf] ^ a.

(5.49)

Now inequality (5.49) and the inequality in (5.45) provide us with the following relationship limmfg[a>s(Af)\R8>ht] ^ a. Next so that

|h*]\

< WP^-B^l

(5.50) -> 0,

^ , ( A * ) - ^ 8 ( A * ) | i J 8 f W ] -+ 0.

This result, together with (5.50), gives liminf^cDs(A*) ^ a.

(5.51)

We shall now show that ds -> — ^j a and 5S -> gja. To this end, there exists a subsequence {r} c {5} such that dr->a and Br->b, where — o o ^ a < 6 ^ o o . By taking into consideration the fact that

W | P ) * ^(0 D, ^,dJr(A*) ^ a, Lemma 2.1 (i) and Polya's lemma, one has that where O is the d.f. of iV(0, F). It follows that the equalities a = b = — 00 or a — b = 00 are excluded. Let a = — 00, so that

Simple against two-sided alternatives

123

b = £a. Then by the fact that &(&*\Pef) => N(Th*, T) and considerations similar to the ones employed above, one has that ^•d) r (A*)->l-O(g a -rA*). By choosing A* < 0, we get l_l — oo, and in a similar fashion 6 < oo. We now claim that a = — £j a and 6 = £j a . For contradiction, assume that b < £j a , so that a < — £j a . Then as before, whereas - 0(6 by choosing A* < 0. Thus liminf ^*o)r(A*) < a which contradicts (5.51). Therefore 6 ^ gj a , and b may not be > £^a by a similar argument, so that b = £j a . Similarly a = — £j a : I t follows that all convergent subsequences of {ds} and {5S} converge to - g i a and g ia , respectively, so that a s - > - £ j a and S s ->g ia . Consequently, replacing 6f by ^s = 60 + hss~i, hs -> A, one has

and hence

^[^(Af)|i? Sj ,J-^[^(Af)|i?^J -> 0.

Thus

^[d)s(Af )|^, hs ]-hs ]

< S

for all sufficiently large s, and hence by means of (5.45), ^s(Af)\RSths]-^s(Af)\RSfhs]

< S for all sufficiently large s,

as was to be seen. ]

6 Testing a one-sided hypothesis against one-sided alternatives In Section 3, it was shown that the sequence of tests { # o e0}. In establishing this result, only the material discussed in Chapter 2 was employed. Now by utilizing the theory of exponential approximation developed in Chapter 3, one can show that the before-mentioned sequence of tests is also AUMP for testing the one-sided hypo-

124

A UMP and A UMPU tests

thesis Hx: deaj = {6e@; 6 ^ d0} = o)f [j {d0} against A. The proof

of this fact is along the same lines with the proof of Theorem 5.1. However, we consider it appropriate that an outline of the proof be presented. More precisely, we shall establish the following result. In this, T H E O R E M 6.1 Under assumptions (A 1-5), (A5'), the sequence of tests {^n(A J } defined by (3.2) and (3.3) is AUMP for testing H.x: 6 e To against A: 0 e o) within the class of all sequences of tests {An} for which

limsup[sup(
(6.1)

Proof In the first place, we show that {0w(An)} is in the class under consideration; that is, we show that Iimsup{sup[^# w (A n ); deW]}^ a.

(6.2)

The proof is by contradiction. Suppose that (6.2) is not true. Then there exists a subsequence {m} c \n) and {#m} with 0meo) for all m such that JP ^ > < > > a. (6.3) Look at the sequence {mi(Om — 60)} and suppose first that it is unbounded. Then there exists {r} ^ {m} such that r4(#r — # 0 )->_ oo. At this point we recall that i?(An|P^o) -> iy(0, T) and therefore by Lemma 2.1 (ii), cn -> £a. This was also seen in the proof of Theorem 3.1 in a slightly different context. Then by (A5)' it follows that *erb{\)

= per(K > cr) + 7rPdr(Ar

= cr) ^ P6f(Ar > cr) -> 0

and this contradicts (6.3). Next suppose {m%(dm — 60)} = {hm} remains bounded. Then there exists {s} c {m} such that hs -> h ^ 0. Thus we have 0s = 0O + hss~i with hs -> h. At this point we recall that

so that Se s(ks) -> I dN(Th, T) ^ a s J (x>Ea)

since Th < 0.

Thus we have obtained a contradiction to (6.3) again. It follows that (6.2) is valid.

One-sided against one-sided alternatives

125

We now show the AUMP property of {^n(An)}. This is also done by contradiction. To this end, suppose that for a sequence of tests {o)n} satisfying (6.1), we have lim sup {sup [£e(on - Se 0 n (A n ); 6 e OJ]} = IS > 0.

(6.4)

Then there exists {m} ^ {n} and {6m} with dm£(o for all m such that

* * J

For the case that {mi(6m — 60)} is unbounded, one has by (A 5) that <^0rr(Ar) -> 1 for a subsequence {6r} c= {#m} and this shows that (6.5) and hence (6.4) cannot occur. It remains then for us to arrive at a contradiction to (6.5) for the case that

stays bounded. In this case there exists {s} c {m} such that hs -> h, so that 6S = do + hss~i with hs -> h. Introducing con(An) = ^ o K | A n ) , we obtain as in the proof of Theorem 5.1 that

£0B8(b*)-*0a4>8(bf)>48

for all s> < ,

say.

(6.6)

say,

(6.7)

Again, as in the proof of Theorem 5.1, one has

f

dN(Fh, T) - S'e &(A*) < 8 for all s > nf,

J(

and, by passing to a subsequence of {s} if necessary, a,->a.

*

(6.8)

In the family of probability measures Hshs defined by (5.34), consider now the problem of testing Hlh: hs ^ 0 against Ah: hs > 0 at level ots for each s. Then the test defined by (6.9) and (6.10) below is a UMP test. (See, e.g. Lehmann [3], p. 70.)

rl

if

Af>cs]

= |y.

if A* = c a |

10

otherwise, J

(6.9)

where the constants cs and y s are determined by the requirement =
for all a.

(6.10)

126

A UMP and A UMPU tests

Because of (6.8) one also sees as in the proof of Theorem 5.1 that f

J (*>£,,)

dN(Fh, F) - f

J (x>&

dN(Th, F) < 8 for all a > n j ,

say, (6.11)

and ^,$5a(A*) - I

dN(Tht T) < 8 for all s > raj, say.

(6.12)

From (6.7), (6.11) and (6.12), it follows that Se ^s(Af) — SQ 8(&*) < 35 for all s ^ n* = max (w2, n3, w4). (6.13) By the fact that &(A*) is UMP at level ocs and ^ SJ8(A*) = aa, < 0. (6.14) one has ^[oJs(Af)\RSfhs]-£$s{A£)\BS}hs] From now on, one simply repeats the arguments employed after relationship (5.40) in order to conclude that SQ (os(Af) — $ $ (fis(Af) < 4:8 for all s ^ n*,

say.

However, this contradicts (6.6) and completes the proof of the theorem. ] To Theorem 6.1, there is the following corollary. COROLLARY 6.1 Under assumptions (A 1-4), the sequence of tests {(f>n(An)} defined by (3.2) and (3.3) is asymptotically locally most powerful for testing Hx: 6 e To against A: 6 e co within the class of all sequences of tests {An} satisfying (6.1) when the parameter values are restricted to those 6s in o) for which [ni(d — d0)} remains bounded from above. That is, within the class of sequences of tests just described, {<j)n(An)} satisfies (3.1) when the parameter values are restricted to those 6s in o) for which {ni(d — dQ)} remains bounded from above. For the symmetric case, we have the following results which are established in a way entirely analogous to the one employed for proving Theorem 6.1. Namely, T H E O R E M 6.2 Under assumptions (Al-5), (A5'), the sequence of tests {<j)'n{An)} defined by (3.20) and (3.21) is AUMP for testing H[: 6 e u' = {6 e 0; 6 ^ 60} = co u {60} against A': 6eo)r within the class of all sequences of tests {An} such that (6.15) limsup[sup{SeXn\ Oew')] ^ a.

One-sided against one-sided alternatives

127

COROLLARY 6.2 Under assumptions (A 1-4), the sequence of tests {^(An)} defined by (3.20) and (3.21) is asymptotically locally most powerful for testing H[: deo)' against A': deo)' within the class of all sequences of tests {Xn} satisfying (6.15) when the parameter values are restricted to those 6s in a/ for which {ni(6 — 60)} remains bounded from below. That is, within the class of sequences of tests just described, {^(An)} satisfies (3.1) when the parameter values are restricted to those 6s in to' for which {ni(6 — 6)} remains bounded from below. R E M A R K 6 . 1 For the explicit form of the tests in the case of the concrete examples that we have been dealing with, the reader is referred to Section 4.

Exercises 1. Verify condition (A5)' in connection with Example 1.3. 2. Verify the equivalence asserted in relation (4.4). 3. In connection with Example 4.4, show that an MLE of 0, is a root of the equation 0

= 0,

where

a0 = - - £ -2^-1-* n

j=I

«! = - £ (Xf.! + ZJ) - 1 and a2 = a0. n

j=l

5. Some statistical applications: asymptotic efficiency of estimates

Summary In this chapter, we apply some of the basic results obtained in Chapter 2 to the problem of the asymptotic efficiency of a sequence of estimates of the unknown parameter 6e®. The asymptotic efficiency adopted in this chapter is the one proposed by Wolfowitz [1] (see Definition 1.1). The main results obtained herein are in the nature of establishing a certain upper bound for the limiting probability of concentration of various estimates under consideration. All these results (Theorems 4.1, 5.1 and 5.2), except for one (Theorem 4.2), are established for the case that 0 is an open subset of E. In Sections 6 and 7, we consider the asymptotic efficiency from the classical point of view and show that standard results for both the one-dimensional and multidimensional case (see Theorems 6.1 and 7.1) are obtained either as special cases of, or are closely related to, the main results mentioned above.

1 W-efficiency - preliminaries The classical approach of proving asymptotic efficiency (a.eff.) of estimates has been geared towards showing that an MLE is a.eff. More specifically, under suitable regularity conditions, an MLE, properly normalized, is asymptotically normal with mean zero and variance the inverse of Fisher's information number. One then considers the class of all estimates which, properly normalized, are asymptotically normal with mean zero, and calls the one with the smallest variance (of the limiting normal distribution) an a.eff. estimate, if such an estimate exists. Again, under appropriate conditions, an MLE is shown to be [ 128 ]

W-efficiency

129

a.eff. except possibly on a subset of the parameter space whose Lebesgue measure is zero (see, e.g. LeCam [1], [3] and Bahadur [1]). However, an example has been constructed by Hodges (see LeCam [1]), where on this exceptional set the variance of the limiting normal distribution of an estimate is actually smaller than the corresponding variance of an MLE. Such estimates are known as superefficient estimates. Several legitimate objections have been raised against the classical approach just outlined (see, e.g. Wolfowitz [1]). The existence of superefficient estimates is disturbing enough; but what is entirely unnecessary is to confine ourselves only to those estimates which, properly normalized, are asymptotically normal. Of course, this is necessary if a.eff. is to be judged on the basis of the variance. However, there is no reason that this should be the criterion of a.eff. Arguing along these lines, Wolfowitz [1] was led to introducing an alternative more general and quite reasonable criterion of a.eff. For the formulation of this criterion, some additional notation is required. As usual, X0,Xl9 ...,Xn are n+1 observations from the Markov process under consideration and © is an open subset of R. Let ^f* be a class (to be specified later on) of sequences of estimates (of 6) {Tn} = {Tn(X0,Xl9 ...,X n )}, and let {Vn} =

{Vn(X0,Xv...,Xn)}

be a sequence of estimates not necessarily belonging in <8?*. Also for each n and 0, consider two classes of intervals in R (to be specified later on) of the following form. In(6) = {certain intervals in(6; tv t2); tl9t2 > 0}; 0 e 0 .

(1.1)

Jn(d; T) = {certain intervals jn(d; tvt2,T); tvt2 > 0}; 0 e 0 . (1.2) The presence of T in the definition of Jn(6; T) indicates that the j-intervals also depend on a quantity associated with the sequence {Tn}. D E F I N I T I O N 1.1 Let {Tn}, {Vn} and ^ * be as above and for each n and each 0, let In(0) and Jn(d; T) be defined by (1.1) and (1.2), respectively. Suppose that \\mPe\Vnein(6\ tvt2)] exists for

130

Asymptotic efficiency of estimates

all tl9t2 > 0 and every 6 e 0. Then the sequence of estimates {Vn} is said to be asymptotically efficient in Wolfowitz sense (W-efficient, for short) with respect to the class ^ * if the inequality 1imm-pPe[Tnejn{6; tl9t2,T)] < limPd[Vnein(6; tl9t2)]

(1.3)

holds for all {Tn} in #*, every £1? £2 > 0 and all (or almost all with respect to Lebesgue measure I in R, a.a. [I]) 6 in 0. The optimality of {Vn} consists in that for every tv t2 > 0 and all (a.e. [I]) 6 e 0, asymptotically, Vn is concentrated in in(6; tl912) at least as much as any Tn, such that {T n }e^*, is concentrated in jn(d; tx,t2,T). Actually, Wolfowitz' programme was to show W-efficiency of the MLE in the i.i.d. case. More specifically, what he does is this. He takes 0 to be a finite interval with or without its endpoints and defines the class #*, to be denoted here by ^w, as follows <€w = {VnY, Pe[(ni(Tn-0)

^ x] -> FT(x; 6), a d.f., uniformly in R x ©}.

Of course, the presence of T in the limiting d.f. simply indicates that this limiting d.f. depends o n ^ } . For each 0 e 0 , let lT{6), uT{6) be the 'smallest' and 'largest' median of FT(-; 6) and define / n (0), Jw(0; T) as follows: = [0-*!?&-*, 0 + * a n-*], M a > 0}, Jn(d; T) = {jn(6; tvt2,T);jn(6;

tvt2,T)

where wT and WT are certain specified functions defined on 0 and such that lT{6) ^ wT(6) < WT{0) < uT{d), 0G0. With the above notation and under suitable regularity conditions, Wolfowitz shows that (1.3) holds true (with lim sup replaced by lim) for all but countably many 0s when Vn is equal to the MLE. Prominent among his assumptions is that PdlnHVn - 0) < x] -> O(x; 6) uniformly in R x 0, where *(•; 0) is the d.f. of N{0, l//(0)), 1(6) being Fisher's information number.

W-efficiency

131

Kaufman [1] has generalized Wolfowitz' result for the case that 0 is ^-dimensional. Now the problem of establishing W-efficiency can be split into two parts. First one concentrates on determining an upper bound, B(6; tvt2), say, such that > lnn*vpPe[Tnejn(0;tl9t29T)] ^B(0;tl9t2) forall{T n }e^*,l (1.4) every tvt2 > 0 and all (a.a. [I]), de®, J and then one attempts to find a sequence of estimates {Vn} for which the upper bound is attained, in the sense that lim

Pd[VnGin(d;tvt2)]

-> B(6; tl9t2)

for all tl9t2 > 0 and every 0 e 0 , (1.5)

where in(0; tvt2)eln(d) for a certain specification of In(0). In other words, one would use inequality (1.4) in much the same way that the Cramer-Rao inequality is employed (only in reversed order). However, the optimality of {Vn} is not defined by (1.5) but rather by (1.3), the reason being that (1.3) may be fulfilled for some {Vn} while (1.5) may fail to occur. Schmetterer [1], Roussas [5] and Pfanzagl [1] have modified Wolfowitz' assumptions and have established inequality (1.3) for several specifications of the class <%*. The first two of these authors also established their results in a Markov process framework. Our objective here is to show that assumptions (A 1-4) are sufficient for establishing inequality (1.4) for certain specifications of the classes ^ * and Jn(d; T). Accordingly, we shall not concern ourselves, with the problem of proving (1.5) for some specific {Vn}. 2 Some l e m m a s The lemmas below are needed in the proof of the results in the remainder of this chapter and occasionally elsewhere, too. D E F I N I T I O N 2.1 For each n9 let fn and / be functions defined on E c Bk into B. We say that {fn} converges con9-2

132

Asymptotic efficiency of estimates

tinuously to / in E if for each xeE, fn{xn) ->f{x) whenever xn->x. The following result then holds true. LEMMA 2.1 If {fn} converges continuously t o / i n E, then/is continuous on E.

Proof In order to establish continuity at an arbitrary xeE, it suffices to show that/(# n ) ->f(x) whenever xn -> x. For contradiction, suppose that / is not continuous at x. Then there exists a sequence {x^} such that \f{xn) -f(x)\

> e for some e > 0.

(2.1)

Now from the assumption of continuous convergence it follows that/ ri (x) ->/(#)• (Just take {xn} = {x}.) Then for each fixed n, there exists an integer mn such that

Clearly, the mns can be chosen so that mn < mn, \in < n'. Thus {mn} ^ {n}. Setting ymn = xni we have that ymn -> x and relations (2.1) and (2.2) imply "

\f(ymn)-f(*)\ > c

\fmn(ymn)-f(ymn)\ < ¥.

(2.3)

From (2.3), it follows that \fmn(ymn)-f(x)\ > \e, which contradicts the assumption of the lemma. The proof is completed. ] The concepts of continuous convergence and uniform convergence are related as in the following lemma. L E M M A 2.2 (i) If {fn} converges t o / uniformly on E, then the convergence is continuous, provided/is continuous on E. (ii) If E is compact, then continuous convergence of {fn} to / implies uniform convergence.

Proof (i) Let x,xneE have

for all n be such that xn -> x. Then we

by continuity of/ and uniform convergence of {fn} t o / . (ii) In the first place, / is continuous on E by Lemma 2.1. For contradiction, assume that {fn} does not converge uniformly t o / .

Some lemmas

133

Then there exists a subsequence {m} c= {n} and a sequence {#m} with xmeE for all m, such that \fm(*m) -f(*m)\ > e for some e > 0.

(2.4)

By the compactness of E, there exists a subsequence {xr} c {o;m} such that xr-> xeE. Replacing m by r in (2.4), we have e < |/ f (*,)-/(*,)| < !/,(*,)-/(*)| + |/(*)-/(ar r )|.

(2.5)

But \f(x) —f(xr) | < Je for all sufficiently large r, by the continuity of/. Hence from (2.5), one has |/r(#r) — f(x)\ ^ | e for all sufficiently large r. However, this contradicts the assumption of continuous convergence. The proof is completed.] The lemma stated below generalizes the dominated convergence theorem and its proof can be found in Loeve [1], pp. 162-3 and also Pratt [1]. 2.3 For each n, let hn) gn, Gn and h, g, G be 3&hmeasurable, real-valued functions defined on Rk such that LEMMA

(i) hn{x)->h{x),

gn(x)-+g(x),

Gn(x)->G(x),

(ii) gn(x) < hn(x) ^ Gn(x) for all n and xeRk, and (iii) jgndlk->jgdlk, JGndlk-+ JGdlk with jgdlk and JGdlk finite. Then jhndlk-> jhdlk and jhdlk is finite, where lk is the Lebesgue measure on @tk. A slight variation of the following result is proved in Bahadur L E M M A 2.4 Let {fn} be a sequence of measurable functions defined on an open subset E of Rk into R and letfn(x) ->• 0, xeE and \fn(x)| ^ M(< oo) for all TI and xeE. Then for any sequence {xn} with xn -> 0 there exists a subsequence {m} c {71} and an Zfc-null subset of ^/ (both of which may depend on {xn}) such that for all x e E outside this null set, one has

fjx + xm) -> 0 pointwise. Proof For any xn -> 0, consider the integral r

134

Asymptotic efficiency of estimates (A;)

where 0 is the d.f. of N(0,1) (I is the kxk unit matrix), and perform the transformation x + xn = y. We have then

= J |/ n (* + *«) | (2;r)-*« exp ( - 1 * = J\fn(y)I (2^)-fc/2exp [ - \{y -xj

(y- xj] d/*(y).

On the basis of our assumptions, the conditions of Lemma 2.3 are fulfilled with K(V) = \L(y)\ (2n)-kl*exv[-h(y-xny Gn(y) = and

{y-xn)l

gn(y) = 0,

(2n)-WMexj>[-Uy-xnY(y-Xn)]

h(y) = g(y) = 0, C(y) = (2»r)-*«Jf e x p ( - ^ ' 2 / ) .

Therefore

= J \L(y) I (2TT)-^ exp [ - \{y - x j

so that

(y - xn)] dl\y)

-> 0,

f \fn{x {x + xxnn)\) d®M(x) -> 0.

Therefore the Markov inequality implies that fn(x + xn) -> 0 in O(/e)-measure. Hence there exists a subsequence {m} c {n} and a O(fc)-null set (both of which may depend on {xn}) such that for all x e E outside this null set, one has fm(x + xm) -> ° pointwise. Since O(&) « Z^, one has finally that for all xeE outside the before-mentioned J^-null set, fm(x + xm) -* ° pointwise, as was to be seen. ] The lemma to be presented below follows from Lemma 2.4 above (for k = 1 and E = B) and is discussed in Pfanzagl [1].

Some lemmas

135

L E M M A 2.5 Let {fn} be a sequence of ^-measurable, realvalued functions defined on R and being such that

liminf/n(x) ^ 0, xeR,

and

\fn(x)\ ^ M(< oo)

for all 7i and xeR. Then for any sequence {xn} with #w -> 0 there exists an Z-null set (which may depend on {xn}) outside of which one has 0.

Proof L e t / * = i n f ^ , 0). Then the sequence {/*} satisfies the requirement of Lemma 2.4. Therefore for xn -> 0, there exists a subsequence {m} ^ {n} and an Zfc-null set (both of which may depend on {xn}) outside of which one has fm(x + xm) -> 0 pointwise. From the definition of/*, one has

fn(x + xn) ^ /*(x + xn) for all n and xeR. Therefore ^

= 0,

as was to be shown. ] In closing this section, we would like to point out that Lemmas 2.1, 2.2 and 2.4 are valid in a more general set-up than the one considered here (see Schmetterer [1], Section 1), but their present formulation is sufficient for our purposes.

3 A representation theorem In the present section, we consider a class of sequences of estimates (of 6) defined by (3.1) below such that when each estimate of a given sequence in the class under consideration is properly normalized, the resulting sequence of their distributions converges (weakly) to a probability measure J§?(#), say. Then it is shown (see Theorem 3.1) that <3?{d) assumes a certain representation as the convolution of two specified probability measures. This result is employed in the next section in connection with the asymptotic efficiency of estimates and, of course, is of considerable interest in its own right.

136

Asymptotic efficiency of estimates

For an arbitrary 0 e 0 and any heRk, set dn = 6 + hn~i, so that 6n e 0 for sufficiently large n. Define the class ^ H of sequences of estimates of 0, {Tn}, as follows

, a probability measure},

(3.1)

where J§?(0), in general, depends on {Tn}. The probability measure J§?(0) can be represented as the convolution of two probability measures. This result is due to Hajek [2]. However, the proof to be presented below is based on an idea of Bickel [1]; see also Roussas and Soms [9]. Finally, an alternative proof can be obtained from some general results of LeCam [7]. The precise formulation of the theorem is as follows. THEOREM3.1

Consider the class (SH defined by (3.1). Then

one has that where J2\(0) = JV(O, T-^d)) and j£?2(0) is defined by (3.15) below. Proof Since 0 is kept fixed throughout the proof, we shall omit it from our notation for simplicity. Thus we will write Se, JS?i, JS?2, T, etc. instead of &(0), ^ ( 0 ) , SP^d), T{6)9 etc. Now let A* = AJ(0) be the truncated version of An defined by (1.5) in Chapter 3 and consider the sequence J5f* = SC*(d) = ^[(n*(3;-J5f*, a measure. Then the marginal measures [JS?m*(r m -0)|PJandJ2?(A*|^) of J5f[(m*(Tm-0),A*)|P,] converge (weakly) to the corresponding marginal measures of J§?*. These latter marginal measures are then probability measures since both £f[mi(Tm — 0)\Pe] andoSf (Aj* |i^) converge to probability measures. It follows that JSP* itself is a probability measure. Setting (T, A) for the identity mapping on Rk x Rk, we have then that J2P[(2\ A)|J2?*] = J2?*, so that .

(3.2)

A representation theorem

137

It is shown in Lemma 3.1 below that

- Sdexp [iu'mi(Tm -6) + h'At - \h'YK\ -> 0 (3.3) and gB exp [iu'mi(Tm - 0) + A'A* - \h'Yh] -> g^exp

(iu'T + h'A - \h'Yh).

(3.4)

Therefore by setting {u, h) = & exp (iu'T + ih'A)

(3.5)

and replacing h by zero, we obtain by means of (3.4) that id exp [iu'm$(Tm - 6)] -> S^ exp iu'T = S'# exp iu'T = (u9 0). (3.6) Also from (3.1) and (3.5), it follows that *0mexp[iu'mi(Tm-dm)] Next

-> ^ , 0 ) .

(3.7)

i6m exp [iu'mi(Tm - 0J] = i e exp [iu'mi(Tm -6)- iu'h + A J = exp ( - iu'h) Sd exp [iu'mi(Tm -6) + A J

and this last expression converges to exp ( - iu'h) gg* exp (iu'T + h'A - \h'Yh) by means of (3.3) and (3.4). That is exp ( - iu'h) £
(3.8)

From (3.7) and (3.8), we obtain {u, 0) = exp ( - iu'h) S'#. exp (iu'T + h'A - \h'Yh).

(3.9)

It is shown in Lemma 3.2 below that in (3.9) we may replace h by ih. By doing so, we obtain (u,0) = e^u'hS^ex^(iu'T

+ ih'A + \h'Yh).

(3.10)

By means of (3.5), relation (3.10) may also be written as follows 0(w,O) = (exvu'h)
138

Asymptotic efficiency of estimates

so that

$(u9h) = (u9O)exp(-u'h)exp(-%hTh).

(3.11)

Next, it is easily seen that - u'h - %h'Yh = - \( so that (3.11) becomes (3.12) Setting successively h = 0 and h = — T^u in (3.12), we obtain and

4>{u, 0) = exp ( - iuT-hi)
(3.13)

{u, 0) exp (fyi'T-hi),

(3.14)

respectively. From (3.5) and (3.14), it follows that $(u9 0) exp (tyiT-hi) is a characteristic function (under JS^*), namely, the characteristic function of the random vector T — r - 1 A. Define <2?2 by I

(3.15)

Also exp( — %u'T~-1u) is a characteristic function (under namely, the characteristic function of the random vector r~xA which is distributed as ^ ( O , ^ 1 ) . Set &x = N(0, T"1). Then from relation (3.13) and the composition theorem (see, e.g. Loeve [1], p. 193), it follows that JSP = ££x *J^2> a s w a s ^° ^ e seen. ] We now proceed to establish the lemmas used in the proof of the theorem. L E M M A 3.1 With the notation employed in the theorem, one has that (i) gB exp [iu'mi(Tm -0) + A J - Se exp [iu'mi{Tm - 6) + A'A* and (ii) Se exp [iu'mi(Tm -d) + A'A* - \KTK\ -> £#* exp (iu'T + h' A - \h'Th). Proof (i) For simplicity, set

A representation theorem and

139

Q* = N(-±hTh,hTh).

Then the convergence J?(An\Pe) => J§?(A|Q*), implies that

whereas &0Un = ^g«.exp A = 1. Also the convergence

implies that so that

-2?(A'A* - \h'Yh\Pe) => -2?(A|0*), ^f(Kil^) =>^(exp A|Q*).

Furthermore, ^ exp A'A* -> ^ Q exp A'A = exp |ATA by (3.8) in Chapter 3, where Q = N(0, T). Thus SeVn -> 1, so that Lemma 2.1 in Chapter 3 applies and gives that Un, Vn, n ^ 1, are uniformly integrable. Next Un — Vn -> 0 in /^-probability by arguing as in (3.5) of Chapter 3 and using the fact that An — A'A* -> — \h'Th in P^-probability. Then, by Lemma 2.2 in Chapter 3, it follows that \Un — Vn\,n^ 1, are uniformly integrable and Corollary 2.1 of the same chapter gives that $o\Un — Vn\ -> 0. The proof of (i) is then completed by observing that the left-hand side of (i) is bounded in absolute value by &(\Un — Vn\. (ii) Clearly, it suffices to show that >iu'mi(Tm-d) +ft'A*]-> g&exp (iu'T + *'A).

(3.16)

Set ^4m = (|A'A* | > c). Then one has that \Seexp [m'wifa^ - 6) + A'A*] - ^

f J^w

+1-If

-f

exp (iu'T + A'A)|

* m

f e

) Am

exp [m'mi(ym - 5) + VA* ] dP, (3.17)

As was seen in the proof of (i), ^ e x p (A'A£) -> exp (\h'Th). Also JS?(A'A* |Pe) => JS?(A'A|Q) implies that

140

Asymptotic efficiency of estimates

Therefore Lemma 2.1 in Chapter 3 gives that exp h'A^, for all m, are uniformly integrable. Thus one may choose c sufficiently large, so that f exph'A*dP0<e J Am

and

f expA'AcLSf* JAm = I

expA'AdQ < e. (3.18)

Furthermore

f

f

'AJdJS?*

(3.19)

J(|ft'A|

by the fact that J§?* =>&* and jS?*(i?fc xB) = Q(5) = 0, where B = {keRk; |A'A| = c}. Then relations (3.17)-(3.19) complete the proof of (3.16) and hence that of (ii). ] LEMMA 3.2 With the notation employed in the theorem, consider the expectation $%* exp (iu'T + h'H) as a function of h, call it g(h), where h = (hl9 ...,hk)', h^eR, j = 1,...,k. Then, for j = 1,..., Ii ,g(h) is analytic in the jih coordinate hj when the first j — 1 coordinates hr,r= 1,..., j — 1 are any complex numbers and the last h —j coordinates hr, r = j + 1,..., 1c are any real numbers.

Proof By setting h = (h2,..., hk)', A = (A x ,..., Ak)r and A = (A 2 ,..., Afc)', one has that $2* exp (iu'T + A'A) = £#* exp [(iu'T + A'A) + Ax AJ

= ^ J e x p f m ' T + A'A) S ^ L

i

J!

A representation theorem

141

Now at this point we observe that for n ^ 0, one has = expA'A S "T

^A x |) (3.20) and this last expression is, clearly, Q-integrable and hence J§?*-integrable. Then by the dominated convergence theorem, we get $2+ exp (iu'T + h'A) = £ [i^ [exp (iu'T + h'A) ^ 1 ) h{ which shows that
_ __ foj'

2 exp^T + i^A^^Aj + ^A)-^ wherehx = h^ + ih^K = (h3,..., A7<.)'andA = (A3 ,..., A^)', whereas the last bound on the right-hand side of the same relationship is equal to exp(A11A1 + I / A+ |A2A2|) which is Q-integrable and hence j£?*-integrable. Therefore h2 may be replaced by a complex variable. In a similar fashion each one of the remaining coordinates may be replaced by complex variables and this completes the proof of the lemma. J

4 W-efficiency of estimates: upper bounds via Theorem 3.1 The main purpose of the present section is that of establishing the inequalities in Theorems 4.1 and 4.2 for two specifications of a class of sequences of estimates (of 6) .The first specification, which is due to Schmetterer [1], is given in (4.8) and is restricted to the case that 6 is real-valued. The result associated with it is presented as Theorem 4.1. A certain multiparameter version of this result is presented as Theorem 4.2. The corresponding class

142

Asymptotic efficiency of estimates

of sequences of estimates is given by (4.10) and is the multiparameter analogue of the class (S8 given by (4.8). For the proof of the above-mentioned theorems, one needs an auxiliary result which is formulated as Proposition 4.1. For aeRk with a 4= 0 and veR, consider the half space Ha(v) = {xeRk;a'x

^ v).

Denote by L the d.f. corresponding to a probability measure ££ in Rk, set Qx = N(0, F"1) and let O(#; F"1) be the corresponding d.f. Then it is clear that there exists v*eR such that

f

dL(x) = f

dO(*;r-i) = f

dQv (4.1)

Next, denote by /i and © the probability measure and the d.f., respectively, corresponding to the characteristic function and let (X, Y) be two random vectors distributed SLSQ1X/I. Also let Q2 x /i be an alternative distribution of (X, Y), where Q2 is the measure induced by O ^ + AF^ajF" 1 ), and consider the problem of testing the hypothesis that (X, Y) are distributed as Q1x/i against the alternative that they are distributed as Q2 x [i. Iif(x; F"1) is the density of Q± with respect to the ^-dimensional Lebesgue measure lk, it follows that/(^;F~ 1 ) is also the density oiQ1x/i with respect to lk x JJL. This is so because for any Bl9 B2e&9 one has that

f

x

x Bt).

JB

Similarly, f(x + AF^a;F" 1 ) is the density oiQ2xju, with respect to lk x [i. Then for the hypothesis testing problem above the most powerful test rejects the hypothesis when

which is equivalent to a'x < c* after some simplifications, provided A > 0. Thus the test which rejects the hypothesis in question whenever a'X ^ v* is at least as powerful as the test

Upper bounds via Theorem 3.1

143

which rejects it whenever a'(X + Y) ^ v. These two tests have the same level of significance because f

d[O(x; T-1) x G(y)]

d(Qx x fl) = f

J Ha(v*)

J HO(V*)

=f

dO(a;r-i)=f

J HO(V*)

dL(x),

J Ha(v)

the last equality being true because of (4.1), and f

d(Q1x/i)=

J [a'(x+y)
f

d[*(a?;r-i)

J [a'(x+y)^v]

= f

dL(«) = f

dL(a:).

The second equality from the right-hand side above is true because the characteristic function of the sum X + Y of the independent random vectors X and Y is equal to exp (— |wT~%)
f

f d(Q1xju,)^

I

d(Q1x/^).

(4.2)

But

f

d[O(a: + AT^a; T"1) x G(y)]

d(Q2 x /i) = J

-JT

[ a'(2!+2/-Ar- 'o

=J That is,

f

^

d(Q2x/^)=|

dL(«). di(2).

(4.3)

144

Asymptotic efficiency of estimates

Also f

d(Q2 x pL) = f

JHaiv*)

J (a'x^v*)

=f In the last expression above, we set A =

_± , s > 0. Then

so that

dO^jr- 1 )^ f

f J [o'fe-r-'flXi;*]

Thus

f

dO^T- 1 ).

J (a'^'y*+s)

dO^jT- 1 ).

d(g2x/0=f

(4.4)

On the other hand, for the above choice of A, relation (4.3) becomes as follows f

d(0 a x/O=f

dL(z).

(4.5)

Then relations (4.2), (4.4) and (4.5) give d
f

J (a'x
di(a:)

(5 > 0).

(4.6)

J (a'x^v*+s)

For 5 < 0, we arrive at f

dO^jT" 1 ) ^ f

J (a'x<'y*+s)

dL(x)

(4.6')

J (a'x^v+s)

by considering as alternative distribution of (X, Y) the distribution Q2 x /i, where Q'2 is the measure induced by <$>(x — Ar- 1 ^; F"1), A=

^ - , s < 0. Since also

dQ)(x; F"1) =

dL(x) by

Upper bounds via Theorem 3.1

145

(4.1), considering the two cases that s > 0 and s < 0, relations (4.6) and (4.6') provide easily the following inequality f

dO^jT-1)

dL{x)^{

J Ha(V+s)AHa(v)

(seR).

J Ha(v*+s)AHa(v*)

Thus we have established the following result. 4.1 For aeRk with a 4= O&ndaeR, consider the hyperplane Ha(a) = {xeRk\ a'x ^ ex) and let L(x) and O(x; T"1) be the d.f.s of the (probability) measures J§? (in Rk) and iV(0, F"1), respectively. Furthermore, for any veR, let v* PROPOSITION

be defined by f

dL(x) = f

J Ha(v)

dO^jF" 1 ). Then for any

J Ha(v*)

vfseR, one has that f

dO^jF" 1 ).

dL(s)*sf

J Ha(.V+s)AHa(v)

(4.7)

J Ha(v*+s)AHa(v*)

We are now going to derive the main results in Schmetterer [1] and Roussas [5] as special cases of (4.7). To this end, suppose that 0 is an open subset of R (i.e. k = 1) and define the class ^s of sequences of estimates (of 6) by

<€a = {{Tn}- Pd[ni(Tn -6)<x]-+

FT(x; 0), a d.f.,

continuously in 0 for each fixed xeR and also continuously in R for each fixed 6 e 0}.

(4.8)

From (4.8), we have that for each 0 e 0 ,

Pd[ni(Tn-6)^x]->FT(x;6) continuously in R. Therefore, by Lemma 2.1, FT(-; 6) is continuous and hence the 'smallest' and 'largest' median lT{6) and uT(d), respectively, are such that FT(lT; 6) = FT(uT; 6) = |.Then consider the following definition of the class Jn{6) mentioned in (1.2).

Jn(£>; T) = {jn(6; tv t2, T) ;jn(6; tv tit T) - i + Ma,((9)w-i))<1,«2 > 0}. (4.9) RCO

146

Asymptotic efficiency of estimates

Finally, for each 6 e 0, let B(d; tl9 t2) be defined by B(6;tvt2)

= O[t2cr(e)]-01^(6)1

tl9t2 > 0.

(4.10)

Then one has the following result. T H E O R E M 4.1 Let
limPe[Tnejn(6;tl9t2,T)] ^ B(6 ; tv t2)

for all tvt2> 0 and every 6 e 0.

Proof In the first place, it is clear that VS^^H Next

P6[ni(Tn -6)^

and

FT{-;d) = L(-;d).

lT{6)] -> £ = f

di(x ; 6)

=f i(0)

and similarly

6)^ uT(0)] -> i = f = f

di(x ; 5) dQ[x;
J Hx(0)

Therefore for the present case, if we take v equal to lT{6) and uT{6) in Proposition 4.1, then the corresponding v*s are equal to 0. By also taking s to be equal to — tx and t2, relation (4.7) gives, respectively,

f

dL{x;d) ^ f

dO[x;

and

f

dL(x;d)^ f

J Hdu^+t^AHdu^d)]^

Equivalently, and limPe[uT{d) < ni(Tn-d)

)AH1(0) J H1(0+t22)AH 1

Upper bounds via Theorem 3.1 Hence

147

lim Pe[Tn ejn(6 ; tv t2, T)] fl)-*! < ni(Tn-d)

<

as was to be seen. ] Now we do not assume that k = 1. Then by interpreting the various inequalities below in the coordinatewise sense, we consider the class <€R of sequences of estimates defined as in (4.8), namely VR = {{Tn} ; Pe[nHTn- 6) ^ x] -> FT(x ; 6), a d.f.,

continuously in 0 for each fixed xeRk and also continuously in Rk for each fixed 6 e 0}.

(4.11)

Then for each heRk, it is clear that the d.f. ofnW(Tn — 6), under Pd, converges to a d.f. FT(x; 6, h)i say, continuously in 0 for each fixed xeRk and also continuously in Rk for each fixed 6e&. Furthermore, the d.f. FT( -;6,h) is continuous, so that the ' smallest' and' largest' median lT(0, h) and uT(6, h), respectively, are such that FT(lT; 6) = FT(uT; 6) = J. It is also clear that c €n c ^H. Now working as in the proof of the previous theorem, one obtains the following result. 4.2 Let tSR be obtained by (4.11) and for each heRk, let lT(6, h) and uT(6, h) be as above. Then for each heRk, one has THEOREM

HmP0[-tx + lT{d9h) < nih'(Tn-d)

< t2 + uT(6,h)]

^ OpaO-^^-Ot-^o-^A)] and every

for all ^,«2 > 0

where a2(6,h) = hT

5 W-efficiency of estimates: upper bounds In this section, we establish upper bounds for two additional specifications of the class ^ * for the case that 0 is an open subset of R. For the proof of the relevant theorems, we shall require the relations (5.1) and (5.2) below. In order to formulate them, let

148

Asymptotic efficiency of estimates

6e®, let 0 + heR and set 6n = 6 + hn~i. Then from Theorems 4.3 and 4.5 in Chapter 2, it follows immediately that for xeR,

(6 2)

-

2

where

a\d) = Y(0) = 4<^[01(6>)]

(5.3)

and, recall that, In the sequel, we shall write A^ rather than A(d, dn) for the sake of simplicity. In the definition of the class ^s, Pfanzagl [1] removes the requirement of continuous convergence and assumes instead that the limiting distribution has zero as its median. More precisely, he defines the class ^*, to be denoted here by ^ P , as follows. = TO \Se\n\(Tn-d)\Pe\^5eT^

a probability

measure, such that J£?T>6(( — oo, 0]) ^ | and Thus if 0 G [ZT(5), %»(#)] for all 6> e 0, then ^P 3 ^ . The requirement that 0 be a median for J5fT e for all ^ G 0, is a very natural one, as the following argument shows. Suppose that 0 is not a median for JSfT> e for some # and let lT{6) > 0. Choose a continuity point a; = x(d, T) olSeTtQ such that 0 < x < lT(6). Then | > J5f r ^((-oo,x]) = limP6[ni(Tn-d)

<x]^

]imavpP6[ni{Tn-d)

< 0]

In other words, limsupP^(jP7l ^ ^) < | and in a similar fashion, limsupP^T^ ^ d) <\ if ^T(/9) < 0. Such an asymptotic behaviour of an estimate of 6 perhaps would be judged by many as not being satisfactory. For the proof of the first theorem in this chapter, the following lemma presented in Pfanzagl [1] is required.

W-efficiency: upper bounds

149

L E M M A 5.1 Let {Tn} be a sequence of estimates such that J?[ni(Tn — d)\Pd] => yTid, a probability measure,

and

liminf Pe[ni(Tn -d)>x\>\

for all x < 0

lim MPe[ni(Tn - 0) < x] ^ \

for all x > 0.

Then, under (5.1) and (5.2), one has ^ B(6 ; ^, t2) for all *1? £2 > 0 and a.a. [I] 6 e 0; (5.5) the quantity B(d;tvt2) is given by (4.10) where Jn(6) is defined by (5.6) below. J»(0) = Un(O;t1,tt);jn(0;t1,t,) = (6-^,6

andjn(6;tvt2)eJn(0), + 1^),^,^

> 0}. (5.6)

Proof For each fixed x < 0, define {/n(# ; •)} as follows fn(x;0) = Pd[ni(Tn-d)

>x]-i,de&.

Then, clearly, the conditions of Lemma 2.5 are satisfied. Thus, if for reR we consider the sequence xn(r) = rn~i, it follows that there exists an Z-null set which depends on {x(r)} and perhaps also on x, call it N(x, r), such that lim sup/ n (x ; 6 + rn~i) ^ 0 for all 6 e Nc(x, r). Let RQ stand for the set of rational numbers in R and set Nx= [)[N{x,r);xe(-oo,0)nR0,re(0,oo)nR0]. Then 1(N±) = 0 and lim sup/ n (o:; 6 + rn~i) ^ 0 for all 6eN{, x e (— oo, 0) n Ro and re(0, oo) n Ro, or equivalently, ]imm-pP[ni{Tn-d) > x + r\Pd+rn-k] > | for all deNl,xe(-ao,0)(]

Ro and re(0,oo) n RO- (5.7)

Set s for the left-hand side of (5.7). Then there exists a subsequence {m} c: {n} such that limP[mi(Tm-6)

> » + r|Ptf+rn-»] = 5.

Applying (3.2) with #m = d + rm~i and 5 > 0, we get

150

Asymptotic efficiency of estimates

so that for all sufficiently large m,m ^ ml9 say, one has

< PeJm*(Tm -6)^

x + r]

for all 6 e Ncl9

a e ( - o o , 0 ) ni? 0 andrG(0,oo) (\Bo.

(5.8)

At this point, an ingenious use of the Neyman-Pearson fundamental lemma, due to Wald [1], provides the following inequality P$[mi(Tm-0) > x + r] for all deNl,

xe(-oo,0)

(]Ro and rE(0,oo) n Ro.

(5-9)

Namely, define Cm and Dm by

Then for each m ^ wx, Cm and D m are alternative critical regions for testing the simple hypothesis H: the underlying probability measure is Pm>e against the simple alternative A: the underlying probability measure is Pm e . Since Gm is optimal, by theNeymanPearson fundamental lemma, and its power Pe (Gm) is less than Pdm(Dm), the power of Dm, by (5.8), the level of Cm must also be™less than the level of Dm. That is, Pe{Gm) < Pe(Dm)9 or, equivalently,

t^

j

e) > x+ r]

which is inequality (5.9). In inequality (5.9), we first take the limits as m -> oo and utilize (5.1) and then let S -> 1. One has then

limmvPd[ni(Tn-6) ^ x + r] ^ O(-ra) for all deNl,

xe ( - oo, 0) n Ro and r e (0, oo) n Bo. (5.10) Next, for each fixed x > 0, define {fn(x ; •)} by

W -efficiency: upper bounds

151

Then once again the conditions of Lemma 2.5 are fulfilled and working as above with 6m now defined by dm = 6 — rm~%9 one obtains lim sup Pe[ni{Tn - d) ^ x - r] ^ O( - rcr) for all deNc2 with

l(N2) = 0,xe(0,oo) nR0

and re(0,oo)nR0.

(5.11)

Now, for e > 0, let y = y(e, 6) > 0 be defined by and set 9/ = 7/(e, 6) = 2yor. Then for any xv x2 such that (equivalently, l^cr —x2o*| ^ rj), we clearly have 10(^0-)- 0(x2cr) | ^ e.

(5.12)

Let t > 0 be arbitrary and choose re(t,t + 2y) oRo,

xe(t-r,O) [)R0.

Then |r — ^| ^ 2y, or equivalently, |( — t)or— ( — r)er| ^ ^ and hence relation (5.12) implies -e ^ O(-rcr)-O(-*(r) ^ e.

(5.13)

Thus for deN{, relation (5.10) gives ]imfmpP0[ni{Tn-d)

^ x + r] ^ O(-rcr).

(5.14)

Since also x + r > t by the choice of x, one has Umm-pPd[ni{Tn-d)

^ t] ^ limsn-pP6[ni(Tn-6)

^ x + r]

^ O(-ro-) ^ O(-^o-)-e on account of (5.14) and (5.13). Since this is true for every e > 0, one concludes that d) ^ t] ^ <S)(-t(r)

for all* > 0 and all 6eN\, or equivalently, ]imin£Pe[rA(Tn-O)

< t] ^ O(^cr)

foralH > OandallfleiVJ.

152

Asymptotic efficiency of estimates

Finally, replacing t by t2, one has limmiPd[ni(Tn-d)

< t2] ^ ®(t2cr) for all t2 > 0 and all deN^

(5.15)

For an arbitrary t > 0, choose re(t,t + 2y)nR0

and

xs(0,r-t)

()RO.

c

Then (5.12) holds true again. For 6eN 2, relation (5.11) implies UmsuyP6[ni(Tn-d) ^ x-r] ^ O(-w).

(5.16)

Since also x — r < —tbj the choice of x, one has ]ima\ipPe[ni(Tn-0)

^-t]^

l i m s u p P ^ i ^ - t f ) < x-r] ^ <3)(-rcr) ^ <3>(-to-)-e

on account of (5.16) and (5.13). From this, it follows as above that lim sup P6[ni(Tn-d)

^-t]^

0(-£ 0 and all 6eN%.

Upon replacing t by tv one has ]imm-pP0[nl{Tn-6)

^ -t±] ^ O(-^o-) for all t± > 0 and all 6eNc2.

(5.17)

Now for any tvt2 > 0, let —t[,t2 be continuity points of jSfT ^ such that —1[ < — tl912 > t2. Then liminfP^[-^ < ni(Tn-6) ^ limPe[-t[

< t2]

< ni(Tn-d)

= limP6[n$(Tn-6)

< t'2]-limP6[ni(Tn-d)

= liminfPe[ni(Tn-d)

^ -tfi

< Q-limsupPe[ni(Tn-6)

^ -t[]

and this is bounded above by
< t2] < O(^o-)-O(-^(r).

Letting now —t[-> —tx and 12 -> t2 through continuity points of we get that for all 6ENQ and all tl912 > 0,

&T9O,

P , [ - ^ < ni(Tn-d)

< t2] ^ O(^2o-)-(D(-^o-) = B(6;tl9t2),

as was to be shown. ] Now we can formulate the first main result in this section.

W-efficiency: upper bounds

153

T H E O R E M 5.1 Let ^P, Jn(6 ;tvt2) and B(d ; tv t2) be defined by (5.4), (5.6) and (4.10), respectively. Then one has

lim sup Pe[Tn ejn(d ; tl9t2)] ^ B(d ; tl9t2)

for all tvt2 > 0 and a.a. [I] de@. Proof First we show that the assumptions of Lemma 5.1 are satisfied. Suppose x < 0. Then for each # e 0 , there exists y = y{T, 6) in [x, 0) such that 3?Ti0{{y}) = 0. Thus Pd[ni(Tn-6)

>x\> Pe[ni(Tn-d)

> y]

and hence KminfPe[ni(Tn-d)

^ x] ^ liminfP e [ni(T n -6) ^ y] =

limPe[ni(Tn-d)>y]

= &T,e(\l/><x>)) > &T,e([0><x>)) > h

That is,

lim in£Pe[ni{Tn -d)^x~\>\

for all x < 0,

and in a similar fashion lim inf Pe[ni(Tn - 6) < x] ^ \ for all x > 0. Next we proceed as follows. For e > 0, define y and r\ as in the proof of Lemma 5.1. Let tl912 > 0 be arbitrary and choose sxe {tl9 tx + 2y], s2 G (t2, t2 + 2y], so t h a t Then s1 — t1 ^ 2y,s2 — t2 ^ 2y, or equivalently, s-^cr — t-^cr < 7/, s2cx — t2cr ^ ^/.

From the above choice of sl9 s2, it follows that Pe[Tnejn(6;tvt2)]

^ Pd[Tnejn(6;

svs2)]

and hence limsupP,[r n ej n (0; tl9t2)] < limP6[Tnejn(d; s1}s2)].

(5.18)

By means of (5.5), the right-hand side of (5.18) is bounded by B(6; svs2) a.e. [I]. Thus one has ]imavpPe[Tnejn(0; But

tl9t2)] ^ B(d; svs2)

s1cr — t1cr= \( — t1)o'—( — s1)o'\ ^ y,

a.e. [I].

(5.19)

s2cr — t2(T ^ TJ

154

Asymptotic efficiency of estimates

imply |0(-* l ( r)-<&(-«,er)| < e, |O(« 2 cr)- O(*acr)| < e, and hence -Of-Sjtr) ^ - O ( - ^ ( r ) + e, O(s2
tvt2)] *c B(0; tl9t2) + 2e a.e. [I].

Finally, letting e -> 0, we obtain the desired result. ] In Theorems 4.1, 4.2 and 5.1, it is a basic condition that {S?[(Tn — 6)\Pe]} converges (weakly) to a probability measure. However, this condition is a rather arbitrary restriction. Other criteria for defining a class of estimates could be used as well. Such an alternative criterion used by Pfanzagl [1] is medianunbiasedness. That is, the class ^*, to be denoted here by ^ P ,, is the following one.

^ d) > i for all 6 e 0 and all n}.

(5.20)

Now we can prove the second main result in this section. T H E O R E M 5.2 Let ^ P ,, Jn(6) and B(6; tl9t2) be defined by (5.20), (5.6) and (4.10), respectively. Then one has

limsupPd[Tnejn(6; tvt2)] < B(6; tvt2) for all tv t2 > 0 and all 6 e ©. Proof The proof is based on the same ideas as those used in Theorem 5.1. That is, the utilization of (5.1) and (5.2) and the use of the Neyman-Pearson fundamental lemma. More precisely, let S, h > 0 be arbitrary and set dn = 6 + hn~i. Then the assumption of median-unbiasedness implies that for all n, one has P6n(Tn > dn) > | , or, equivalents, Pen{Tn < 6n) ^ | .

(5.21)

W-efficiency: upper bounds

155

Next, applying (5.2) with x = (l+8)hcr, one has

Therefore, for all sufficiently large n, n ^ %1? say, we have

(5 22)

4

-

Utilizing relations (5.21) and (5.22) and also theNeyman-Pearson fundamental lemma in the same way it was done in the course of the proof of Theorem 5.1, one has that for all sufficiently large n, n ^ n2, say, and all 6e®, Pe{Tn < 6n) < Pe

Letting first n-^co and then 8 -> 0 and replacing h by t2 in the resulting limits, one has limmvP[ni(Tn-d)

< t2] ^ <5>(t2
(t2 > 0).

(5.23)

Next taking dn = 6 — hn~i, In > 0, and x = - Shcr, S > 0, we obtain in a similar way the following result

O(-^cr) ^ l i m i n f P ^ i ^ - f l ) < -tx]

(t± > 0). (5.24)

Therefore, by means of (5.23) and (5.24), one has limsupPd[TnEJn(6; tl9t2)] = limsupP,[-* x < n*(Tn-0) < t2] < ]imfm-pPe[ni{Tn-0) < t2]-liminfP6[ni(Tn-d) = O{t2cr)"O(-tx(r)

< -t±]

= B(6;tl9t2)9

as was to be shown. ] This section is closed with a remark and two examples. R E M A R K 5.1 It is to be pointed out that, as follows from Theorems 4.1, 4.2, 5.1 and 5.2, the variance )]2 plays the same role as Fisher's information number 1(6) under the standard conditions (of pointwise differentiability, etc.). The following example shows that a sequence of estimates {1^} which is optimal with respect to ^ * need not belong to *&*.

156

Asymptotic efficiency of estimates

EXAMPLE

5.1 Refer to Example 3.2 in Chapter 2. Then

^ follows that j= l

J?[ni(S*n-d)\Pe]=>N(0,2d*) (see, e.g. Cramer [1], Sec. 28.3). Thus the upper bound B(6; tl912) given by (4.10) is attained for {Vn} = {8%} in the sense of (1.5). Accordingly, {8%} is optimal with respect to all ^s, ^P and ^ P ,. However, {Sfy^&p*, because Pe{8\ ^ 6) = P0{Xn ^ n) a n ( i therefore Pd{8%/ ^ 6) ^ | and P^(/S| < #) ^ | cannot be simultaneously true. On the other hand, it is clear that Furthermore, { $ | } e ^ s because

/

n

V

(j = 1, ...,n). Next, Y1 = d(Z- 1) with where Tj = {Xj-fi^-d Z being xl so that SeY1 = 0, o-fli = 2^2 and ^ 7 3 = 8^3 (since SZ = 1, cr2Z = 2 and ^ Z 3 = 15). Let F% be the d.f. of

Then the Berry-Esseen theorem (see Loeve [1], p. 288) gives 4c

°

uniformly in e

-

where c is a numerical constant. It follows that

which implies that 2

n-0)\Pe]

=> N(0,26*)

uniformly in d.

This result, together with Lemma 2.2 shows that this last convergence is continuous, as is required in the defining relation (4.8).

W-efficiency: upper bounds

157

5.2 Refer to Example 3.3 in Chapter 2. Here cr\6) = 1, as was seen in Example 1.3. Furthermore, if Vn stands for the sample median, it was seen in Example 4.3 that EXAMPLE

It follows that the upper bound B(6 ; tl912) is attained again for the sequence {Vn}. It is then immediate that {Vn} is optimal within ^P and ^p, and also within <SS (see also the exercise at the end of this chapter). The references Weiss and Wolfowitz [1], [2] and Hajek [3] are of related interest. 6 Asymptotic efficiency of estimates: the classical approach The classical set-up for investigating asymptotic efficiency of estimates is the following. First the class ^ * is specified by V* = {{Tn}; &[ni(Tn-d)\P0]

=> N(0, cr%{6))} for all 6e&.

Next it is shown that, under suitable regularity conditions, one has for all {Tn}eV* <x%{d) > I-\6)

for all (a.a.p])

(0eQ).

(6.1)

Then any sequence of estimates {P^} in ^ * for which Oy(d) = 1^(6) for all 6 e 0 is said to be asymptotically efficient. Under appropriate conditions, a sequence of MLEs enjoys this property. In connection with the classical approach, the reader may wish to consult, e.g. Bahadur [1] and LeCam [2]. Another approach is presented in Rao [2]. In particular, for the Markov case, existence, consistency and asymptotic normality of an MLE is studied in Billingsley [1] and references cited there. A set of conditions for consistency of an MLE different from, the ones used in the reference last cited, are presented in Roussas [3]. These conditions suffice when we restrict ourselves to compact subsets of ©. Also of relevant interest is a paper by Ganssler [1]. Finally, conditions for asymptotic normality of an MLE weaker than those used in Billingsley are given in Roussas [4].

158

Asymptotic efficiency of estimates

What we intend to do in the present section is to show that the classical result (6.1) follows from theorems obtained in Sections 4 and 5 for two important specifications of the class ^ * . In this context, then, classical efficiency is a special case of W-efficiency. Define two classes of sequences of estimates as follows. Vc = {{Tn}; P6[ni(Tn - 0) ^ x] -> d>T(x ; 0)

for all

x e R and 0 e 0, where OT( •; 0) is the d.f. of N(0,cr2T(6))}

(6.2)

and
< x] -> OT(x;d) continuously

in 0 for each xeR, where O T (-; 6) is as above}. (6.3) Then the following theorem holds true. T H E O R E M 6 . 1 With the classes ^c and ^ and (6.3), respectively, one has (i) Ilo-(d) for every 0e®, where a(0) is given by (5.3).

defined by (6.2)

Proof (i) Consider the definition of the class &P in (5.4). In the present case, Or(O;0) = £ for all 0e&. Therefore ^ c c
for all JJ.,^ > Oanda.a. [I]

(0e@). (6.4)

Since <$>T(x\0) — O[xcrT1(0)], relation (6.4) becomes

< d>(£2o')-®(--£10')

for all fx, ^2 > Oanda.a. [Z] (#e0),

(6.5) where we set crT = crT(0). Letting ^ -> oo in (6.5), we obtain the desired result in an obvious manner.

The classical approach

159 <

(ii) Consider the definition of the class €B in (4.8). In the present case, the limiting d.f. is continuous in xeR for each fixed 6e®. Therefore, by Polya's lemma and Lemma 2.2, the convergence in (6.3) is also continuous in R for each fixed 6e&. Hence ^ c
^ O(^o-)-O(-^cr) f o r a l l ^ > 0 and every 6 G0. (6.6) From (6.6), we obtain the desired result as in (i). ] The theorem just proved also provides us with a lower bound for the asymptotic squared error (see Bahadur [1], Proposition 3). Namely, COROLLARY 6.1

One has

(i) limmf[n l/cr*(0) a.e. [I] if{Tn}eVc, and

(ii) liminf \n£e(Tn-dy\ > \\a\d) Proof

for all de® if

Since 7 ^ 0 implies SY = \ [1 - FY(y)] dy, we have Jo

Pd[{Tn-6f > x]dx = ^ Jo

Pe[{Tn-df>x-\d{nx)

= Jo [^

=JorPe o + P6[ni(Tn-d)<

-Jx

160

Asymptotic efficiency of estimates

Since the integrand in the last expression above is non-negative, Fatou's lemma applies and gives by means of (6.2) and (6.3)

x.

=

(6.7)

But cj) I -.2L—\^x \ ^"T/

&(x)2cr2rxdx

=

=—2o'2rl

Jo JO An integration by parts yields

x<S>(x)dx.

J-oo

x
so that

f °° O (-&) Jo

\

dx = \a%.

VT!

Thus (6.7) becomes liminf \nSd{Tn-df\

> cr%.

Then the (i) and (ii) of the corollary follow from (i) and (ii), respectively, of the theorem.] The present chapter is closed with a result referring to the multiparameter case and discussed in Bahadur [1] and Roussas [6]. 7 Classical efficiency of estimates: the multiparameter case In this section, we revert to the case that 0 is a ^-dimensional subset of Rk. The class of estimates to be considered here is defined as follows, where inequalities are to be understood in the coordinate wise sense. Vc. = {{Tn}; Pe[ni(Tn-6)

< z] - • <&*>(* ; GT{6))

continuously in 0 for each z £ Rk, where
(7.1)

The multiparameter case

161

Then by recalling that T(0) = 4<^[^i(0)&(0)], one has the following result. THEOREM7.1

Let ^^ be defined by (7.1). Then the following

is true: CT{0) - r~1(0) is positive semi-definite a.e. [lk]. (7.2) Clearly, the conclusion of the theorem is a direct generalization of the first conclusion in Theorem 6.1. The following important fact is also a consequence of (7.2). COROLLARY

7.1 Let g be a real-valued function defined on

k

R and suppose that ^TT<7(0), j = 1* «-.& exist and are continuous in 0. Let ^c^ g be a class of sequences of estimators {8n} of g(6) defined as follows {{n}l» g ( n ) with Then one has (i) ^{ni[Sn-g(d)]\Pe}^N(0,(r2s(d)) for all 6e®, provided o%(0) > 0, and (ii) o%(e)>gT-H6)g a.e. [I*], where

and

^=(^^).-.^

Proof (i) This is a well-known result and its proof can be found, e.g. in Rao [3], p. 321, (ii). (ii) It follows immediately from (7.2). | For the proof of Theorem 7.1, the following auxiliary result is needed. L E MM A 7.1 For each n, Tn with {Tn} e ^ c » and v e Rk, define the function fn(•; v) on © as follows:

fn(d;v) = Pni6(v'Tn^v'd).

(7.3)

Then/ n ( •; t?) is ^-measurable (recall that ^ is the cr-field of Borel subsets of©). Proof For an arbitrary but fixed 60 e 0 and any A es/n, one has /• J*».»(^) = J -4

162

Asymptotic efficiency of estimates q(X0, ...,Xn;d0,6)

=

^

(see Chapter 2, Section 1). From (A4) (ii), it follows that Pn e(A) is ^-measurable for each n and each fixed A es/n. That is, for each n, Pn (•) is a transition probability (see, e.g. Definition III, 2.1, p. 73 in Neveu [1]). Next,

let B = {{d,G));v'Tn^v'd}. Then it follows that

Be^x^n.

Therefore by setting Zd(o)) = IB(d,o)), it follows by the Tulcea theorem (see, e.g. Neveu [1], p. 74) that 7(0) = f Pnt6(do))Z6(oj) Jn

is ^-measurable. Since, clearly, /. the desired result follows from (7.3). | Proof of Theorem 7.1 The convergence in (7.1) implies that Pe[nhf(Tn-d) ^ 0] -> J and hence fn{6 ; v) -> | . For each n, define on Rk the function/*( •; v) as follows Jn\P>v)-

|0

otherwise.

Then, clearly, 0 ^/*(0;v) < J for all n and # e 0 , and also /*(#;^)~ > 0> 0 e 0 . Thus the assumptions of Lemma 2.4 are satisfied. For heBk, define the sequence {zn(h)} as follows: zn(h) = hn~i. Then, by Lemma 2.4, it follows that there exists a subsequence {m} c {n} and an Zfc-null set of #s (both of which may depend on {zn(h)} through h and also on v) such that fZ(0 + hm~i; V) -> 0 for all d$N(h,v),

(7.5)

where N(h, v) is the exceptional Zfc-null set. By means of (7.4), the convergence in (7.5) is equivalent to

fjd + hm-i; v)->\

for all 6 $ N(h, v);

or, on account of (7.3) and by setting 6m = 6 + hm~i, we have Pe [mh'(Tm-d)>

v'h] -> \

for all 6$N(h, v).

The multiparameter

case

163

Letting RQ be the set of rational numbers in Rk and setting No = u [N(h9v); h,veR\\, we have lk(N0) = 0 and P0Jmfo'(Tm -d)>

v'h] -> |

for every ^^iV^ and all h,ve Rfi. (7.6) Use again the notation Am = A(#, 6m) and let The remaining part of the proof runs parallel to the lines of the corresponding part of the proof of Theorem 5.1 and also the proof of Lemma 5.1. An outline of it is as follows. In the first place we take h as above and =(= 0, so that r 4= 0. Then Theorems 4.3 and 4.5 in Chapter 2 imply in an obvious way that for xeR, ^ m

^

2

O(a;)

(7>7)

^ x\ -> <&{X-T).

(7.8)

From (7.8) and for an arbitrary S > 0, one has (7.9) so that

( 7 - 10 )

1-
Working with 6$N0 and h,veR^y relations (7.6), (7.9) and (7.10) imply that for all sufficiently large m, PgJmW{Tm-d)

> v'h].

(7.11)

At this point one employs the Neyman-Pearson fundamental lemma in a way analogous to the one used in the proof of Lemma 5.1 in order to conclude from (7.11) that Pe[mh'(Tm-d)

> v'h]

for all sufficiently large m.

(7.12)

Letting m->oo in (7.12) and utilizing (7.1) and (7.7), one has ^ l-
v)-*],

164

Asymptotic efficiency of estimates

or

O[(l + *)r] >
(7.13)

Letting 8 -> 0 in (7.13) and replacing r by (AT(0) A)i, it follows that

O[(AT(0) A)*] > O[t/A(^CT(6>

so that, finally (*T(0)A)* ^ v'h(v'CT(d)v)-i

a.e. [Zfc] and for all v, heR%.

Then Corollary 3.1 A gives CT(d) - r~1(^)

is positive semi-definite a.e. [lk],

as was to be shown. ] For {Tn} in ^ c », one also has the following result. P R O P O S I T I O N 7.1 Let <€& be defined by (7.1). Then CT is continuous in © (in any one of the usual norms for matrices).

Proof Clearly, the continuity of CT is equivalent to that of h'CTh for every heRk. Fix an arbitrary heE and set r(d) = T(d,h) = (h'CT(d)h)i. For this h and each xeR, (7.1) gives P0[nW{Tn - 6) ^ x] -> O[s/7(0)] continuously in 0 e ©. Then Lemma 2.1 implies that O[#/T(0)] is continuous on ® as a function of 6 for each xeR. (Here is where the continuous convergence in (7.1) is utilized.) In particular, this is true for x = 1. That is, O[l/r(0)] is a continuous function of 0 on 0. Also z = O(2/) is continuous as a function of y and then so is its inverse y = O~1(^). Since l/r(0) = O-1{O[l/r(6>)]}, the continuity of l/r(0), and hence that of r(6) follows. ] By the proposition just established, Theorem 7.1 has the following corollary. 7.2 Let <€& be defined by (7.1) and suppose that T(6) is continuous. Then COROLLARY

CT(0) — F~1(0) is positive semi-definite for all 6 e 0. In particular, this is true if there exists fiQe&c" such that (7.14)

The multiparameter

case

165

Proof The statement in (7.2) is equivalent to the inequality h'[CT(6)- P" 1 ^)] h ^ 0

a.e. [lk] for every heRk.

(7.15)

Let T(d) be continuous. Then so is F~1(<9). Let 6eN0, the exceptional Zfc-null set. Then there exists {dn} with dnsNcQ for all n such that 6n -> #. Replacing # by 6n in (7.15) and letting n -» oo, we obtain the desired result. If (7.14) holds true then F(#) is continuous by Proposition 7.1, and hence the previous argument applies. ] Since hT(6) h = 4:<^6[hf(/>}(d)]2, it follows that T(d) is continuous if for every heRk, <ee[h'(f>i(0)]2 is continuous. The proposition to be established below also guarantees the continuity of T(d) under (A 1-4) and an additional assumption (A4') to be formulated here. This assumption would be rather easy to check in practice. (A4') There exists S > 0 such that for each d G 0 and each heRk there is a neighbourhood of d, v(d, h), with the property that ASSUMPTION

< M(d,h) (
Proof By Theorem 2.1A (with r = 1), assumption (A4) is equivalent to the convergence q(X0, X±; 6, #*) -> 1 in the first mean [Pe] as 6* -> 6, and this in turn is equivalent to ||PM.-PiJ->0

as

0*-> 6,

(7.16)

by Theorem 1.4 A. It is now claimed that

&[h'jx(0*)\Pe.]=> SClh'faWlPe]

as 0* -> 6.

(7.17)

166

Asymptotic efficiency of estimates

In fact, l e t / be any real-valued, bounded and continuous function defined on i?. One has then

where M(< oo) is a bound for /. However, this last expression tends to zero as 0* -> 6 because of (7.16), (A3) (i) and the dominated convergence theorem. Thus (7.17) holds true. Next let Flf0* and Flfd be the d.f.s of the r.v.s h'fi^d*) and h'^d) under Pxe* and P16) respectively. Then by virtue of (A4'), one has = f |a?|a+*dJ?if^ ^ M(d,h)

for all d*ev(d,h),

(7.18)

while (7.17) implies F

as i,e*^Fi,d 0*->0. (7.19) By relations (7.17) and (7.18), the corollary on p. 184inLoeve [1] applies and gives

(x2 dFh e* -> f x2 dFx e

as

d*-+d,

which is equivalent to

KT{d*) h = goWfad*)?

-> £6[h'x(d)]2

= hT(6)h as d*->6. The proof is completed. ] A brief account of most of the material discussed in this chapter can be found in Roussas [7], Exercise Refer to Example 5.2 and show that {Vn} e ^P, and {Vn} e
6. Multiparameter asymtotically optimal tests

1

Some notation and preliminary results In the present chapter, we consider the problem of testing a simple hypothesis against a composite alternative in the framework described in Section 2 of Chapter 2 and under the assumption that the dimensionality of the parameter space © is greater than one. A sequence of tests having certain asymptotically optimal properties will be derived. In order to be able to formulate the problem and also state the relevant theorems, some preliminary results are needed. These are obtained in the present section and also in Sections 2 and 6. As has been the practice in previous chapters, the dependence of various quantities on a fixed parameter point 60 will not be explicitly indicated. For instance, we shall write An, A*, F, etc. rather than An(#0), A*(#o), F(#o), etc., respectively. The assumption has been made that the symmetric matrix F is positive definite. Therefore there exists a non-singular matrix i f such that M'M = Y (1.1) (see Theorem 4.1A). From (1.1), it immediately follows that (if- 1 )' TM-1 = MT-^M' = I,

(1.2)

where / is the h x k unit matrix. Consider the following transformation of Rk onto itself M(z — 6Q) = V — d0, where v = u + 00 — Md0

and u = Mz. (1.3)

For c > 0, let E(c) be the surface (of an ellipsoid) defined by E{c) = {zeRk; {z-00)'T{z-0Q) [ 167 ]

= c}.

(1.4)

168

Multiparameter optimal tests

Then the transformation (1.3) sends the surface E(c) onto the surface (of a sphere) 8(c), where 8(c) = {zeRk; (z-00)'(z-00)

= c}.

(1.5)

k

For zeR , with z * 0O, set c(z) = (z-60)' T(z-60). Then this quantity is positive, since T is positive definite by assumption. By means of E(c(z)) and S(c(z)), define the function £ as in Wald [2], p. 342. Namely, for any p > 0, define (*)(z,p) by OJ(Z,P) = {UEE(C(Z));\\U-Z\\

< p}

and let o)'{z,p) be the image of the set o)(z,p) under the transformation (1.3). Also denote by A(a>(z,p)) and A(o)'(z,p)) the areas of the sets co{z,p) and a)'(z,p), respectively. Then the function £ is defined as follows

Thus one has the positive-valued function £ defined on Rk — {60} and it can be seen (see also Exercise 1) that its explicit form is -flo) / r(z--g o )]i

r(*0)||

(z + &o)

'

(1-7)

R E M A R K 1.1 The significance of the function £ defined by (1.6) may be seen from the relation

g(z) dA = area of 8{c)9 to be denoted by A(c), (1.8) JE(c)

where E(c) and >S(c) are defined by (1.4) and (1.5), respectively, and the integral in (1.8) is a surface integral (see also Exercise 2). By setting {

f^

(zeE(c))>

(1.9)

one obtains a weight function £(c;-) (integrating to 1) over each one of the surfaces E(c). In the sequel, we will be interested in parameter points 6n of the form dn = d0 + hn-i QieRk). (1.10) Also the As, eventually, will be required to satisfy the condition h'Th = c, c > 0, so that (dn-dQ)' V(dn-60) = en-1. Set En>c = Eicn-1) = {zeRk; (z-Oo)T(z-0Q)

= cn~i),

(1.11)

Notation and preliminary results

169

k

and let

E*{c) = {zeR ; zTz = c}.

(1.12)

The constant c appearing in (1.11) and (1.12) will eventually be chosen to lie in a compact subset K of (0, oo). Then it follows that for all sufficiently large n,n ^ n(K), say, Enc <= 0 for all ceK. Thus, with dn being of the form (1.10), it follows that dneEn>c

if and only if

heE*(c)

En)C^@,ceK,n^n(K).

and

(1.13)

R E M A R K 1.2 Let 0n = do + zn-i. Then from (1.7) it follows that £,{0n) = £*(z), where £* is defined the same way as £ except that 60 is replaced by zero. In particular, if dn is given by (1.10), then £(#n) = £>*(h), where 6n eEn c, so that h e E*(c) (see relations

Closing this section, we should like to mention that all results in the present chapter are derived under the basic assumptions (A 1-4) in Section 2, Chapter 2, although this will not always be stated explicitly. 2 Formulation of some of the main results In the present chapter, the problem we are interested in is that of testing the hypothesis H: 0 = 60, for some fixed parameter point 60, against the alternative A: 6 #= d0. To this end, let J?h = N(rh,T)

(heR"),

(2.1)

and define the set D as follows D = {zeRk\zT-1z^d),

£?0(D) = l-a

(0 < a < 1). (2.2)

Also define the class ^ by and set

f = { C G # ; 0 is closed and convex},

(2.3)

(2.4)

In particular,^set

= {f\ f = ICc, O e f } . ^ = IDc.

(2.5)

Then, clearly, D e f 5 so that ^ G ^ . All tests herein will depend on the random vector A^ and for reasons to be explained in Section 3, we may confine ourselves to tests in J*\

170

Multiparameter optimal tests

For I/TE^, that is i/r = ICc for some Cetf, set fin{6; 0") = fin(d; f) = <^(A m ).

(2.6)

R E M A R K 2.1 As will be seen in the theorems to be stated below, the sequence of tests {^(An)} possesses certain optimal asymptotic properties, where (f) is defined by (2.5). Also with 6n and JS^ defined by (1.10) and (2.1), respectively, it will be shown later (see Lemma 4.1 (i)) that

Thus, if we decide to restrict attention to tests in the class IF of asymptotic level of significance a—which we shall do—we must have <&O(DC) = a. This fact provides the justification for the equation <&0{D) = 1 — a employed in (2.2). Unless otherwise explicitly specified, in all that follows and for each n, we shall consider only parameter points 6n of the form dn = do + hn~i with heE*(c) or he El = E*{cn), so that 6neEntC or 6neEn = EUtCn (see (1.11) and (1.12) for the definition of E^c and E*(c)); here {cn} is a bounded sequence of positive numbers. Furthermore, although, # n -and also in certain instances hnwould be a more appropriate notation, we shall simply write 6 and h when no confusion is possible. It should be borne in mind, however, that 6 = 60 + hn~i, where heE*(c) or heE%, so that 6eEnc or 6eEn, respectively. This will somewhat simplify an already cumbersome notation. The first main result in this chapter is presented in the following theorem. 2.1 Let £n = ^(cn-1 ;•) and En>c be defined by (1.9) and (1.11), respectively. Also let (j) be given by (2.5) and let {i/rn} be any sequence of tests in !F of asymptotic level of significance a. Then, under assumptions (A 1-4), for testing H against A at symptotic level of significance a, one has THEOREM

(2.7)

Formulation of some main results

171

where j3n(0; <j>) and fin{d; ijrn) are defined by (2.6) and K is an arbitrary compact subset of (0, oo). (For a certain uniform (over the tests ijr) version of the result just stated and also its interpretation, the reader is referred to Theorem 4.1.) The second main result herein is the following one. 2.2 Let En,c,,{fn},]Sn(d;<j>) and J3n(6;irn) be as in Theorerfi 2.1. Then under assumptions (A 1-4), for testing H against A at asymptotic level of significance cc, one has (i) l i m [ s u p { s u p [ / ? J 0 ; 0 ) ; 0 e ^ c ] - i n f [/?n(0;0); 6eEn>c]}; ceK] = 0, where K is an arbitrary compact subset of (0, oo); (ii) liminf[inf{inf[/? n (e;^)-/? n ((?;^ n ); deEn>c]}; ceK] > 0 for any tests i/rn as described above and for which fin(0; \Jfn) satisfies (i). The interpretation of the theorem is clear. Part (i) states that the power of the test <j) on the surfaces Enc is asymptotically constant. The second part asserts that, within the class of tests whose power on Enc is asymptotically constant, the test is asymptotically most powerful. The formulation (and proof) of the third main result in the present chapter is deferred to Section 6, since it requires substantial additional notation. THEOREM

3 Restriction to the class of tests & Suppose that we are interested in testing the hypothesis H: 6 = 6Q against the alternative A: 6 #= 60 at asymptotic level of significance a. All tests are to be based on the random vector An and power is to be calculated under Pe , where 6n — do + hn~i. This can be done without loss of generality by Theorem 5.1, Chapter 3. By Theorem 5.2, Chapter 3, one has that for any tests i/rn (not necessarily in ^) and any bounded subset of Rk, B, s u p [ K l M A n ) - * , B f n ( A * ) | ; heB] -> 0, where A* is a suitable truncated version of An defined by (1.5) in Chapter 3. Therefore from an asymptotic point of view, it

172

Multiparameter optimal tests

suffices to base any tests on the random vector A* rather than An. This is so regardless of whether we are interested in pointwise power or average power (see Theorem 2.1 and also Exercise 3). Power is still to be calculated under P0 . Next, by virtue of (1.6) and (1.7), Chapter 3, one has ^ where

y

;],

(3.1)

k

exp Bn(h) = 4 o (exp A'A*) {h e R ).

In the family of probability measure Rn h, the parameter is h and its range is all of Rk. Since for any 6e®, 6 = do + hn~i for some heRk, namely, h = ni(d-60), it follows that 6 = d0 if and only if h = 0. For h = 0, it follows from (3.1) that Rn0 = P0Q. By Corollary 4.1, Chapter 3,

where B is any bounded subset of Rk. Thus for any tests i/rnf one has

Therefore it follows that, from asymptotic power viewpoint, power may be calculated under Rn h rather than P6n. Furthermore, this is true regardless of whether our interest lies in pointwise power or average power in the sense of Theorem 2.1. (See Exercise 4.) In order to summarize: the original testing hypothesis problem H: 6 = 60 against A: 6 4= 60 at asymptotic level of significance a, where tests are to be based on the random vector An and power is to be calculated under Pdn, may be replaced by the equivalent testing hypothesis problem H*:h = 0 against A*:h + 0 at asymptotic level of significance a in connection with the family of probability measures defined by (3.1), where tests are to be based on the random vector A* and power is to be calculated under Rnh. Now we introduce the random vector JZ*, where Z* = P-iA*.

(3.2)

Restriction to class of tests !F

173

Also the following notation is needed:

and

LZh = &(Z*\Rnth)

(KeR*).

(3.4)

The following result holds true. 3.1 (i) Let 3?*^ and JSP**^ be defined by (3.3). Then for every heRk, one has LEMMA

and

[) + A'z], z e 5*.

(ii) Let L%hbe defined by (3.4). Then for every heRk, one has and

[di* fc/dL*0] = exp[-S w (A) + *TZ],zGl2*.

Proof (i) For ^4 e& k and by virtue of (3.3) and (3.1), one has

=f

=f =f as was asserted. (ii) With A as above and by virtue of (3.1), (3.2) and (3.3), one has

exp [ - Bn(h) + h'\*

=f

n€A)

J (Zn

=f =f because i?n>0 = Pg by means of (3.1). Now, since J?(Z*\Rn,0) = LZo,

174

Multiparameter optimal tests

as was asserted. ] From (3.2), it follows that, in testing the hypothesis last described, our tests may be based on the random vector Z* rather than A*. Now for each n, the family of probability densities is of the form (4.21) in the Appendix with T = Rk, C(t) = exp [ - Bn(h)]

and

S = I\

Therefore Corollary 4.2 A (see also Remark 4.1A) applies and we conclude that an arbitrary test i/r^ based on Z* may be replaced by a test i]rn based on Z* of the form (4.10) in the Appendix. Thus for each n, we consider tests ijrn based on Z* such that fl if zed fniz) = I - I (3-5) [0 if seC° far some C t f J where Misgiven by (2.3); the test maybe arbitrary (measurable) on 8Cn, and ^[fn(Zt)\Bnt0]-^ a. Now it would be convenient to avoid arbitrariness of tests ijrn on dCn; for instance, it would be convenient to set i/rn(z) = 0 for zedCn, so that frn(z) = Ice. In order for this modification to be valid, we would have to show that by changing the test \jrn on dCn in any arbitrary (measurable) way, both its asymptotic power and size remain intact. That this is, in fact, the case is the content of Lemma 3.3. In order to be able to prove the lemma, some additional notation and some preliminary results are needed. To this end, let ^ * = {Ce&k; and also set

C is convex},

&n>d = ^ ( A J P J

(0e0).

(3.6) (3.7)

Then by Theorem 4.6, Chapter 2, for O* = do + hnn-b with hn->heRk, SeUte^Seu, where Seh is defined by (2.1). Also P^(A* + A J -> 0 by Proposition 2.3 (ii), Chapter 3. Therefore Se\ei-=>Seh, where JSf**; and J§^ are given by (3.3) and (2.1), respectively.

Restriction to class of tests &

175

On the other hand, \Pe*n-RnthJ -> 0 by Theorem 3.1, Chapter 3, so that Proposition 1.2, Chapter 1, applies and gives t*hn=> Seh, where &Z*hn is given by (3.3). That is, we have =>^

and ^ * * / ^ ^ V

(3-8)

The lemma below shows that these convergences are uniform over the class ^*. More precisely, we have the following result. LEMMA 3.2 Let 0n9 &h, JS?£*n, Sf**h and #* be defined by (1.10), (2.1), (3.3) and (3.6), respectively. Then for any bounded subset B of Rk, one has heB}^0, (i) sup{sup[|J?*, n (C)-^(C)|; OeV*]; and (ii) Bup{8up[|J2?**fc(O)-J2?fc((7)|; Oe«P*]; heB}^ 0.

Proof (i) The proof is by contradiction. Set

and suppose that sup[8n(h); heB]-\>0. Then there is a subsequence {m}^{n} and Ame2? such that $m(hm) -> ^, for some 5 > 0. Equivalently, m

where

5 ,

(3.9)

0* = <90 + Amm-i.

Let {Ar} c {Am} be such that Ar -> teRk. Then one has, by virtue

Thus Theorem 4.3 A applies and gives suptl^X^-JSfJ^ljCG^^O.

(3.10)

On the other hand, we clearly have Bup[|^ r (0)-JS? t (0)|;C'6«'*]->0. Relations (3.10) and (3.11) then imply that sup [|J?V%;(C) -J2V(0)|; C e n -> 0. However, this contradicts (3.9) with m replaced by r.

(3.11)

176

Multiparameter optimal tests

(ii) We have II a?*

a?**

I

\\~z n,dn — ~Z n,h\

= But svp[\\P$n-Rnih\\;heB]^-0 Therefore

\\Pen-RnJ.

by Corollary 4.1, Chapter 3.

aup[||J?* , a - i ^ | | ; * e £ ] - 0.

(3.12)

Clearly, (3.12) implies that Bup{Bup[|jg?*fli,(C)-JS?*^(O)|; Oe**]; fcei?}^ 0. This last convergence together with the first part of the lemma yields the desired conclusion. ] The result just obtained is a strengthening of Lemma 2.1 in Chibisov [1] in that taking the sup over h, we allow h to vary over bounded rather than compact sets. LEMMA 3.3 Let Z * , i * ^ and*'* be defined by (3.2), (3.4) and (3.6), respectively. Then for any bounded subset B ofRk and any sets Cn G **, one has

*uV[Bnth{Z*edCn);

heB] = sup[£* h(dCn); heB] -> 0. k

Proof For any A e^ , one has Z*eA if and only if A* eA, where ^ ^eRk. (3.13) u = Tz,zeA}. Let A0 and A denote the interior and the closure, respectively, of the set A. We then have, by means of (3.3) and (2.1)

L*,h(eon) = &*%(eCn) =

s^Cn)-se**h(Oi)

= l^t%^n)-^h^n)l-[^t%(0°n)-^h(00n)] for any Cn etf*, where 6 equality

Q n

(3.14)

denotes the interior of the set 0n. The

holds because 0^%?* if and only if dne^^ and the boundary of any convex set in Rk has ^-dimensional Lebesgue measure zero and hence JS^-measure zero. It is also well known that, both the closure and the interior of a convex set are also convex.

Restriction to class of tests &

177

Taking the sup of both sides of (3.14) as It varies in B, one obtains the desired result from Lemma 3.2 (ii). ] Returning now to the discussion following the definition of the test i/rn by (3.5), we conclude that we may restrict ourselves to tests i/rn based on Z* and having the following form

and

fn = Iccn for some Cn = V

(3.15)

(3.16)

4 Proof of the first main result For the proof of the first theorem, we shall need some additional notation and also some preliminary results. Set Zn = r - i A , and ThenZneA Set

Lni6 = J?(Zn\Pe)

(4.1) (0e0).

(4.2)

if and only if AneA, where .4 is given by (3.13). Also Zft =

tf(A,r-i)

(he IP).

(4.3)

One then has the following result. LEMMA 4.1 (i) Let JS^, <^* and &nt9 be defined by (2.1), (3.6) and (3.7), respectively. Then

where 6n is given by (1.10) and B is any bounded subset o£Rk. (ii) Let 6n, %>* and B be as above and let Ln e and Lh be defined by (4.2) and (4.3), respectively. Then

Proof (i) The proof is similar to that of Lemma 3.2 (i) and the details are left as an exercise (see Exercise 5). (ii) For Ae0Sk, we have Ln9en(A) = ^n3$n{A), where A is given by (3.13) and Ae%*{Ae4?) if and only if Ae<#*(Ae
(4.4)

178

Multiparameter optimal tests

Therefore

sup [\Ln>l>n(C)-Lh(C)\;

Then taking the sup of both sides of this last relation as h varies in B and utilizing the first part of the lemma, we obtain the desired result. From (4.1), it follows that &neA if and only if ZneA, where A = {ueRk; u = T-^zeA}. Therefore by setting fin(0;A) = Pe(AneA)

(4.5) (Ae@k),

and 0n(d;A) = P6{ZneA)

(4.6) k

wehave J3n(d; A) = f}n{6\ A) (de@,Ae& ). (4.7) We may now proceed with the proof of the first main result. Proof of Theorem 2.1 Throughout the proof of the theorem, we always assume that n > n(K) in addition to any other conditions which may be imposed on n, so that En> c c: @forall c e K. We now proceed with the proof which is by contradiction. To this end, suppose that (2.7) is not true. Then there exists a subsequence {m} c: {n} and {cm} <= K such that Em

~ f fim(0; fm) U « ) M -* *> ^

some S < 0,

(4.8)

where, we recall, that Em stands for EmCm (see (1.11)). By employing the notation in (2.6), this is rewritten as follows

f

Em

J E

or

f J Em E

fim(m)U) J E

By virtue of (4.7), this becomes

f

J Em

P J E

I

6

.

(4.9)

Proof of first main result Now set

ft(h;

179

A) =Lh(A), heR

k

k

(Ae@ ),

(4.10)

where Lh is given by (4.3). Then on account of (4.6) and (4.10), Lemma 4.1 (ii) implies that for arbitrary sets Dm e <%* sup [\L(0m\ Dm) -fth; Dm)\

;heB]^0

k

for any bounded subset B o£R . In particular, s u p f l / U ^ ; Dm)-${M Dm)\;heEt]

-> 0,

(4.11)

where El = E*{cm), and E*{cm) is given by (1.12). At this point, we set (Ae<%k) (4.12) fi(h;A)=fi(mi(Om-do);A)=fi*(em;A) and we recall that, by (1.13), heE^ if and only if 0meEm. The convergence in (4.11) then becomes Bup[|/8fw(0m; Dm)-0*(6m; or

Dm)\;6meEm]

mv[\0Jd;Dm)-/J*(d;DJ\;deEm\-*O.

-> 0, (4.13)

Utilizing (4.13) with Dm replaced by Cm and D successively, we obtain

f Pjfi\0m)UP)M-\ J Em

and

f

fi*(d;Cm)U0)M

+ O (4.14)

J Em

$m(d;$)£je)dA-(

J Em

JE

From (4.9), (4.14) and (4.15), we obtain

f p*(d; J Em

or equivalently,

f /?•(
J Em

where T) denotes the complement of the set t). Thus for all sufficiently large m,m ^ ml9 say, we have

f

JEEmm <

f P*{0; &m) U 0 ) dil + 1 * v Em

(recall that 8 < 0),

180

Multiparameter optimal tests

or by means of (4.12),

fE

J Em

fi(mi(0 ); &mm)) Q Q 0)) AA + + \8 fi(( - 600); \

for all m > mv

JEm

(4.16) Let Am = I

g(z) d^4. (See also (1.8).) Then, on account of (1.9),

jEm

(4.16) becomes

f

JEm

< ff ^ ( m 4 ( ^ ^ ^ 0 ) ; ^ ) ^ ) d ^ + i M m for all m > mx. Em

jE

Set

rai(#

and

(4.17)

- 0O) = h, so that 6 = 60 + Am~i^j j Ae-S*, given by (1.12).

Then by virtue of (4.12) and Remark 1.2, the inequality in (4.17) becomes

f #A;D«) <

ftfcCttPWdA+^fr

for all m^m1?

(4.19)

where the scaling factor Jm results from the transformation in (4.18). It is not hard to show (see Exercise 7) that \Jm\ = m-i^-1*, whereas Am, which is the surface area of the sphere with radius (^m" 1 )* corresponding to Em = Em>Cm in (1.11), is equal to cjkfc-p

as is well known. Therefore

Proof of first main result

181

Now, since cm e K, a compact subset of (0, oo), and by passing to a subsequence if necessary, we may assume that cm -> c ^ 0. First consider the case that c > 0. Then for all sufficiently large m, m ^ m2, say, we have cm ^ |c, so that

. !?. ^ (J^ where

Hence for m ^ m3 = max (m1? m2), (4.19) becomes

f ^(A;D«)g*(*)dil< f fi(h;Ccm)£*(h)dA + 82, (4.20) where

*2 = ** 1 (<0). c

(4.21) c

At this point, we recall that ft(h; C m) = Lh{C m) by (4.10). On the other hand, it is clear from (4.5) that Ac = Ac, whereas

A=A9 as follows in an obvious manner from (3.13) and (4.5). Therefore one has Lh{C%^) = Lh(Cm) and, by (4.4), this is equal to In order to summarize, one has fi(h; &m) = Ln(€°m) = &h{Cem). By (3.7), J?m,eJC°m) = PgJAmeCcm), so that J 4 , eo(Gem) = P,o(Am e C*J and this converges to a. Then Lemma 4.1 (i) gives that

(4.22)

But &Q{Ccm) = L0(Cm), by means of (4.22), and this is also equal to £ 0 (Cy, since Gcm = Ccm as was remarked before. Thus

Now let O e # be such that io(Cc) = a. Then

£0(5!)-A,(C«)^0 and from this it also follows (see Exercise 8) that sup [\Lh(Ci) -Lh(C°)\; heB] -> 0, for any bounded subset B of i2fc.

(4.23)

182

Multiparameter optimal tests

Since E^ remains bounded as m -» oo, it follows that for all sufficiently large m, m ^ m4, say, one has Lh{0%) < Lh(Cc) + e. This, together with (4.10), gives

f fi(h; <%) £*(*)
(m ^ m 4 ),

£*(h) dA is the surface area of the sphere with Jtim

radius c|^ corresponding to E^ = E*(cm) in (1.12). According to the formulae given above,

Combining this inequality with (4.20), we obtain that for m ^ m5 = max (m3, m4),

f fi(h;£)°)£*(h)dA< ff ^(*;

+ *2. (4.24)

From this expression of ^ given above, it follows that {Afy stays bounded. This result, together with (4.21), implies that for m ^ m6, some m6, and some #3 < 0,

f

, (4.25)

JJS

for sufficiently small e. By (4.10),ft(h;A) is the power of the test \]r — IA, based on the random vector Z whose distribution, under h, is Lh = N(h, T-1) (see (4.3)). On account of (4.5), the set JD is given by ^ ={ By taking into consideration (2.2), one has

B = {ueBk;uTu^d}. Applying (4.4) with A = t) and h — 0 and also utilizing (2.2) and f) — T> the fact that we obtain L0(D) = 1 — a. That is,

D = {ueRk; u'Yu ^ d}, LO0) = 1 - a and also

LQ(G) = 1 - a.

Proof of first main result

183

Now for all sufficiently large m, m ^ ra7, say, look at D given above and also E%1 = E*(cm) (see (1.12)), and consider the problem of testing the hypothesis H': h = 0 in connection with the distribution Lh = N(h, F" 1 ). Then we have that both D and E% are of the type required by Theorem 4.4A. Hence relation (4.25) cannot hold true. The desired result is then established. In order to complete the proof of the theorem, we have to show that its conclusion is true if c = 0; that is, if cm -> 0. In this case, by setting hm instead of A in h'Yh = cm, we have that hm->0, or equivalently mi(dm — d0) -> 0. Then repeating the arguments which led us to (3.17) in Chapter 4, we obtain |-f^— P$ || -*" 0. From this and by a simple contradiction argument, we also get sup (||P^m-P^J|; 6meEm) -> 0. Therefore uniformly in i]rm in JF and 6meEm, one has

Hence fim(6m; fm) -> a uniformly in fmeiF and 0meEm, since fim(60] ifrm) -> a. Applying this result for i/rm = 0, we obtain

uniformly in i/rme^r and 6meEm, so that

Em

Thus the left-hand side of (2.7) (with lim inf replaced by lim) is equal to zero. The proof is completed. J From Lemma 4.1 (i), it follows that from asymptotic power viewpoint (both in pointwise and the average power sense), there is no loss by restricting ourselves, for each n, to tests lying in the class ^0 defined by (6.2) below. In this case, the proof of Theorem 2.1 is considerably simpler in that one may deduce the desired contradiction from (4.19). This is so because ft(0; C%) = oc. Also, in this case, one may formulate and prove a uniform version of Theorem 2.1. More precisely, one has the following result. T H E O R E M 4.1 Refer to the testing hypothesis problem considered in Theorem 2.1. Then with the same notation as that

184

Multiparameter optimal tests

employed in Theorem 2.1 and under the same assumptions, one has

liminfLf{inf[f ~\E

Pn(6\
^n(0^)U0)dA;ire^;ceK^==0,

(4.26)

where ^ 0 is given by (6.2) below. Thus, according to the conclusion of the theorem, for every e > 0 and all sufficiently large n9 one has

f fin(d; <j>) UP) M > f J En> c

PJfi 5 f) L

- e

v En> e

simultaneously for all Proof of Theorem 4.1 In the first place, the left-hand side of (4.26) cannot be positive since (j> e ^ 0 . This follows immediately from the definition of <j) by (2.5) and (6.2). Next, suppose that (4.26) is not true and let the left-hand side of it be equal to some 8 < 0. Then there is a subsequence {m} c {n} and {cm} c K such that

infff E

m

J Em

where Em = Emc (see (1.11)). From this it follows that there exists a sequence {^m} of tests in ^ 0 such that

f

J Em

This is the same as relation (4.8) and a repetition of the arguments used in the proof of Theorem 2.1, leads us to (4.19). The desired contradiction then follows as indicated above. ] 5 Proof of the second main result The following inequalities will be useful in the proof of the second theorem. Let {dp jel} and {flpjel} be any collections of bounded real

Proof of second main result

185

numbers and let / be any index set. Then the following inequalities hold (see Exercise 9). | sup (a,; je/)-supOfy; jel)\ ^ sup(|a y -/? y |; jel)

(5.1)

and |inf(a,;je/)-inf(/?,.;je/)| ^ s u p ( | a , - # | ; j e / ) . (5.2) A few more facts will be needed before we proceed with the proof of the theorem. We recall from Chapter 3 that A stands for the identity mapping in Rk. Also Lh = N(h, T"1) by (4.3). Therefore J?(A\Lh) = N(h,T-i) and hence (see, e.g. Rao [3], p. 152) JSPfATAl-Efc) = xl;sw> where

8{h) = h'Th.

(5.3) c

From the definition of A by (4.5), it is immediate that A = Ac. On the other hand, as was mentioned in the proof of Theorem 2.1, where A is given by (3.13). Utilizing these facts with A = &(=]>), where f) is defined by (4.20), relation (4.4) becomes Lh(8°) = &h(D>).

(5.4)

The quantities J§^ and D are defined by (2.1) and (2.2), respectively. From (5.3) and (5.4), one obtains ^h(Dc) = 1 -P[xl;s(h)
/3*(d; Dc) = constant on each En.

We may now start with the proof of the result.

(5.6)

186

Multiparameter optimal tests

Proof of Theorem 2.2 By a familiar contradiction argument, it suffices to show that (i') lim{sup(A(0; 0); deEn]-mi^n(d; 0); deEJ} = 0, (ii') liminf{inf[/?J0; j)-fin(0; i/rn); 6eEn]} > 0, where En = En>Cn (see (1.11)) and {cn} is any sequence with elements in K. (i') By employing (5.6), we have ); deEn]\ ^d; D>); deEn]\ + \w£[fin(6; $); deEn}-mi[($*{d;

D°); 6eEn%

and by means of (5.1) and (5.2), the right-hand side above is bounded above by 2 sup [\fin(d; $) -p*{6; D»)\; 6eEJ = 2sup[1^(5; D>)-0*{d; D*)\;deEn] = 2sup[|/?w(0; !))-/?*((?; D)|; (?6^J. Set (9TO = do + hn~i with Ae.E* = J?*(cn) (see (1.12)). Then on account of (4.6) and (3.7), one has fin(dn; D) = P6n(AneD) =

ST^D).

From (3.13) and (4.5), it follows that

I> = D. Therefore, by virtue of (4.12), (4.10) and (4.4), we have

/?•(
£ |sup[/U0; ^

|; heE*]. Since the expression on the right-hand side above converges to zero by Lemma 4.1 (i), the proof of part (i') is completed, (ii') We first show that liminf{sup[/?n(0; t)-fin{6;

f n ) ; 6eEn]} > 0.

(5.7)

Proof of second main result

187

The proof is by contradiction. Suppose that (5.7) is not true and let the left-hand side be equal to some d < 0. Then there is a subsequence {m} c: {n} such that for all sufficiently large m9 m ^ ml9 say, one has sup[/?m(0; 4>)-PJ6\ f m ) \ 6eEm] < J*. This is equivalent to fim(0'>4>)-fim{O'>&m)
for a11

0eEm

and all m > mv

or ^ and all Hence

6eEm

m ^ mv /?m(^j^l

liminf since f

all

]w

^m(6)dA = l9 and this contradicts (2.7).

%/ Him

We now continue as follows ini[Pn(6;
)];0eEn}

> w£\fin(0\ $KOeEJ + infl = MlfiJO; );0eE ] - s u p Adding and subtracting appropriate quantities, the last expression on the right-hand side above becomes equal to -{sup[/? n (0; 0); 0ei? m ]-inf[/?J0; ); 6eEn]} -{sap[fin(0; fn); deE^-udlfi^d;

tfrj; 6eEn]}

+ {sup[/?n(0; $); deEn]-inf[fin(d; The third of these terms is further written as sup \fiJ0; $); 0eJ and this is bounded below by

i

188

Multiparameter optimal tests

Combining these results, we obtain

-{sup[/?n(0; ); dGEn]-m£[JSn(O; ^); 6eEn]} 6; )-fin(0; f n ) ; deEJ.

(5.8)

Letting now n->co in (5.8), we have that the limit of the first term on the right-hand side is equal to zero by the first part of the theorem, the limit of the second term on the same side is equal to zero, by assumption, and the liminf of the third term on the same side is ^ 0 by (5.7). This establishes (ii') and hence the theorem itself. ] 6 Formulation and proof of the third main result Up to this point we have dealt with tests \{rn in IF defined by (2.4), depending on the random vector A^ and having asymptotic level of significance a. In this section, we are going to further restrict the class !F of tests by introducing another class contained in IF and denoted by J^. The reasons for this restriction are implicit in the definition of the envelope power functions by (6.10) and (6.14) and also Lemma 6.2 below which is needed in the proof of Theorem 6.1. However, in order for the restriction under question to be legitimate, we must show that nothing is lost, asymptotically, in the process neither in terms of power nor in terms of asymptotic level of significance. Set ^ 0 = {Ce&k; C is closed, convex and Jg?0(C) = 1 - a},

(6.1)

where J§?0 is given by (2.1), and define the class ^0 by &0 = {i/r;f = IOc,CeVQ}. Since

J2?(Aw|P,o) = ^

(6.2)

=> tf(o, T) = &0

by Theorem 4.2, Chapter 2, we have that &n,eo(Cn) - ^0(Cn) -> 0 for any sets O n e^*; this is so by Theorem 4.3A. Thus if con = Ice is in «^0, so that J?0(Cn) = 1 — a, the last convergence above gives

Proof of third main result

189

<&n,o (Pn) ~> a- That is, tests in ^ 0 are of asymptotic level of significance a. Then it suffices for us to show that every test ifrn in 3F which is of asymptotic level of significance a, can be replaced, from asymptotic power point of view, by tests o)n in J^. More precisely, it suffices to establish the following result. L E M M A 6.1 For any sequence of tests {i/rn} in IF for which Se ^W(AW) -> a, there is a sequence of tests
0,

(6.3)

where dn = d0 + Aw~4 and £ is any bounded subset of Rk. Proof Since irne!F and <^ o ^JAJ -> a, we have \jrn = /cc for some CneiFand^n>do(Gn) -> l ~ a . Setting A = 0 in Lemma 4.1 (i), we obtain &nt6o{C*)-&Q{Cn) -> 0, so that

WJ-*!-<*•

(6.4)

Thus for all sufficiently large n, n ^ nv say, <2?Q{Cn) > 0- This implies that for w ^ w1? the sets (7n are ^-dimensional because if their dimensionality were less than k, then their ^-dimensional Lebesgue measure would be zero and hence their J§?0-measure zero, a contradiction. Then for each n ^ nl9 consider the following modification of the set Cn. IfJP0(Cn) < 1 — a, enlarge Gn, so that it remains closed and convex and J?0(Cn) = 1 — a. If ^{Cn) > 1 — a, shrink the set Cn until &Q(Cn) = 1 - a . If &0(On) = 1 - a , the set Cn is left intact. Denote the resulting set by Cn 0. Then Cn0 is closed and convex and Thus setting

con = Icc o,

we have G)neJFQ. From (6.4) and {6.5), it follows that 0)->0.

(6.6)

From the process of arriving at Cn> 0, it follows that Cn> 0 c (7n or Cn 0 ^ (7n. Therefore for each heEk, we have — Cn>0)

-J^(C W 0 )

if

C M 0 £C M ,

190

Multiparameter

and

optimal

tests

0 - Cn) h(n,o)-^(Cn)

if

Thus For h = 0, ^h(CnAGnf0)->0 by (6.6). On the other hand, &h<<&0 for every heRk. Thus £eh{CnkCn>0) -> 0. It can be further seen that for any bounded subset B of Bk, one has (see Exercise 10) s u p ^ ^ A ^ o ) ; hGB] -> 0. (6.8) Therefore, (6.7) and (6.8) give mV[\
-> 0.

(6.9)

Now, for any B as described above, one has ^ n w n (AJ|; heB] <

8uV[\J?n>en(Cn)-J?h(Cn)\;heB] + +

mj>[\J?n>en(Cn>0)-J?h(Cni0)\;heB] imV[\h(Cn)-J?h(Cni0)\;heBl

and each one of the terms on the right-hand side above converges to zero on account of Lemma 4.1 (i) and (6.9). The proof of (6.3) is completed and we have the justification for confining ourselves to the class of tests, ^*0. | Before we are able to formulate the third main result, we shall have to introduce a further piece of notation. To this end, let J3n(d; i/r) = <^^(An), as given in (2.6), and suppose that the test where ^o = {GeSSk\ O is closed, convex and J§?0(O) = 1 - a } ; we also recall that J§?0 = iV(0, F). Next, define the envelope power function fin(0; cx) by fin(d; cc) = Sup{/?TO(0; f); fe&0}. Then the third main result in this chapter is as follows.

(6.10)

Proof of third main result

191

T H E O R E M 6.1 Let En c and 0 be defined by (1.11) and (2.5), respectively, and let i/rn be any tests in J^, where ^*0 is given in (6.2). Also let pn(6\
Iimsup[sup{sup[/? n(0; a)-fin(0; 0);

9eEnJ

-mV[j3n(d;a)-fin(6;i/rn);deEntC];ceK}]

^ 0,

(6.11)

where K is an arbitrary compact subset of (0, oo). The interpretation of the theorem is that within the class ^ 0 , the test <j> is asymptotically most stringent on Enc. We recall that in the present framework and for each n, the test <j)n would be said to be most stringent on Enc within the class ^"0, if the quantity sup[/?n(#; ot) —fin(0;ijrn); deEnc] were minimized for For the proof of Theorem 6.1, a couple of auxiliary results will be needed. For their formulation, let us recall once again that J2£ = N(Th, T) and set ; A) = &h(A)

(heRk,Ae&k).

(6.12)

For each n and a sequence of positive numbers {cn}, let (see(1.12))andtransformAto<9G^ = EntCn (see (1.11)) through the transformation 6 = 60 + hn~i. Also set fth; A) = j3(n*(0-00); A) = ft'(6; A).

(6.13)

Next, by means of fi'(6; A), define the envelope power function P\0\ a) as follows P'(0\ a) = sup[/?'(0; C°); OeV0].

(6.14)

The first auxiliary result is given in the following lemma. L E M M A 6.2 The function /?'(#; a) defined by (6.14) remains constant on each En, where En = 2?n Cn (see (1.11)) and {cn} is any bounded sequence of positive numbers.

Proof Clearly, /?'(#; a) = J}(h; a), where ; a) = sup[p(h;Cc);Ce
and

h = ni(d-0O).

192

Multiparameter optimal tests

Thus it suffices to show that /?(A; a) stays constant on each i£*, where E* = E*(cn) (see (1.12)). With M defined by (1.1), consider the transformation t = Mh. Then, by (1.2), E% is transformed into 8* = {teRk; ft = \\t\\* = cn}. Also the class of sets <^0 is transformed into the class of sets <£#, where « ; = { C e # ; C is closed, convex and N0(C) = 1 - a } ,

and

^^(iOf'^Jfrif')

(*eS*)

(6.15)

(6.16)

(see Exercise 11). Therefore, by setting 1

; a) = sup[/?»(*;
(6.18) such that (6.19)

c

Now, clearly, /?°(£2; O ) = Ntz(C ) is equal to the iV(£2,/)-measure of the set (MM'^C0, and by symmetry, this is equal to the N(tv /)-measure of D, where D is the symmetric image of (MM')-XGC with respect to the hyperplane through the origin that is perpendicular to the line segment connecting the points tx and t2. But the N(tv /)-measure of D is equal to Nti((MM') D), and, clearly, (MM')D is the complement of a closed, convex set, Cg, say. Then, by symmetry, one, clearly, has N0(C0) = 1 — a, so that Coe^9 and also J3°(t2; Cc) = ^{tx\ Gc0). Then (6.19) gives /?°(^; oc) < fi°(tx] GCQ). However, this contradicts the definition of fl°(h >a) by (6.17). We reach the same conclusion if the inequality in (6.18) is reversed. Thus the proof of the lemma is completed. ] The second auxiliary result referred to above is the following lemma. This lemma, as well as the one just established, is of some interest in their own right. L E M M A 6.3 Let/? n (0; a) and^'(0; a) be defined by (6.10) and (6.13), respectively. Then for each n, one has

Proof of third main result (i)

193

sup[|/?J0; a)-/?'(
-> 0;

and (ii)

sup[j3n(6; a); 6eEn]-inf

[/?J0; a);deEn]-+

0,

where En = 2?n Cn (see (1.11)) and {cn} is any bounded sequence of positive numbers. Proof (i) In the first place, relation (5.1) justifies the inequality below TO(0;

f);

and this last expression is equal to sup[|^,«(C)-J^(O)|;Oe^0], since and

where A = n*(0-
/?n((9; 0) = P,(A» e 0) = ^m> e(G) fi'(d;C)

= ^(h;G)^^h(G).

That is, with A = »i(0 - d0), \fin(6; a)-p\d;

a)\ < sup[\J?n>e(C)-J?h(C)\;

Hence, with E* = ^*(cw) (see (1.12)), one has sup [|/?n(0; a)-/?'(#; a)|; and the expression on the right-hand side converges to zero by Lemma 4.1 (i). (ii) Letting 6eEn and utilizing Lemma 6.2 and inequalities (5.1) and (5.2), one has |sup[/U0; a); 0 e # J - i n f [ / ? J 0 ; a); 6eEn]\ 0; a); deEJ-/J'(d; a)\ Am(^; a); ^ e ^ J - s u p ^ ' ^ ; a); ^; a); 0eEJ-ia£[fi\0; a); ^ ; a)-/?'((?; a)\;deEn], T3

194

Multiparameter optimal tests

and this last expression tends to zero by part (i). This establishes the lemma. ] We may now proceed with the proof of the third main result herein. Proof of Theorem 6.1 The proof is by contradiction. Suppose that the theorem is not true and let the left-hand side of (6.11) be equal to some 4# > 0. Then there exists a subsequence {m} ci {n} for which 0; a)-fim(6; fm); 6eEm] + 3S (6.20) for all sufficiently large m,m^ mv say. The left-hand side of the inequality above is bounded from above by #; a); 0etfJ-inf[/? m (0; 0); OeEJ

(6.21)

and its right-hand side is bounded from below by inf[/U0; a); deEJ-MtfJid; fm)\ 0etfJ + 3* (6.22) by virtue of (5.2). By means of (6.21) and (6.22), relation (6.20) gives that, for all m ^ ml9

e; 0); 6eEJ > in£[fim(0; a); 6eEm]-m{^m(6;

f J ; deE

or svp[/lm(6; a); 6eEm]-mf[fim(d; a); 6eEm] > inf [fim(0; ^); deEJ-MlfiJd; ftm); OeEJ + 38. (6.23) By Lemma 6.3 (ii), the left-hand side of (6.23) tends to zero and hence it remains less than 8 for all sufficiently large m, m ^ ra2, say. On the other hand, Theorem 2.2 (i) yields wf[fim(d; 0); for all sufficiently large m, m ^ m3, say. On account of these facts, inequality (6.23) becomes then 8 > sup[/U#; 0); deEm]-mf[/1m(d; fm); 6eE

Proof of third main result

195

or inf[/U0; fm); deEm]-8 = inf[/?m(0;

fj-

for all m ^ m4 = max (mlf m2, m3). Hence

Pm(d;fm)-8>fim(d;) for all m ^ m4 and every deEm. Therefore

f ftJB;95)U
fm)Ufl)dA<-8

J Em

for all m ^ m4, and this implies that IJEm

jEm

J

However, this contradicts Theorem 2.1. The desired result follows. ] The following uniform (over the tests ijr) version of Theorem 6.1 is also true. T H E O R E M 6 . 2 Refer to the testing hypothesis problem considered in Theorem 6.1. Then with the same notation as that employed in Theorem 6.1 and under the same assumptions, one has

Iimsup{sup[sup[/?n(0; oc)-/3n(d; 0);

deEnJ

-inf{sup|A(0; *)-fin(O\ ft); eeEnJ; fe^ceK]}

^ 0,

(6.24) where ^ is given by (6.2). Thus, according to the conclusion of the theorem, if S( < 0) is equal to the left-hand side of (6.24), then for every e > 0 and all sufficiently large n, one has

simultaneously for all fte^Q and all ceK. Proof of Theorem 6.2 Suppose that the theorem is not true and let the left-hand side of (6.24) be equal to some 8 > 0. Then 13-2

196

Multiparameter optimal tests

there exists a subsequence {m} c {n} and a sequence {cm} with elements in K, such that, if Em = i/m>Cwi (see (1.11)), then -inf{m-p[fim(6; *)-fim(O; f); Thus for e > 0 and all m ^ m5, say, one has sup[/?m(0; a)-/? m (0; 0);

deEJ-8-e

< inf{sup[/y0; a)-/?ro(0; f ) ; BeEJ; Therefore, for each m ^ m5, there exists a test ^ m e J ^ such that sup[/?m(^; a)-/?m((9; 0); 6 > e ^ J - * - e

< mv\PJfi\ a)-pm{0; fm); deEJ < sup[/?m(^; a)-/? m (^; 0); 0 e # J - * + e , or equivalently, * - e < sup[^m(^; a)-fijfi;

4>); deEJ

-sup[/U0; a)-pjfi\

fm);

6eEm]<8+e,

provided m ^ m5. I t follows that -> *(> 0). However, this result contradicts (6.11). The proof of the theorem is completed. ] 7

Behaviour of the power under non-local alternatives Recall that (j> = IDc, where D is given by (2.2). Also recall that the power of the test 0, based on An = Aw(#0), has been denoted by /3n(0 ;), 6e®. Then the theorems formulated and proved in the previous sections, provide us with some optimal properties of the test (j). However, these properties are local in character, since the alternatives are required to lie close to the hypothesis being tested; actually, they are required to converge to 60 and at a specified rate.

Behaviour of the power

197

The underlying basic assumptions (A 1-4) employed throughout this chapter, do not suffice for establishing optimal properties of the power function at alternatives removed from 60 or not converging to it at the specified rate. This can be done, however, under the following additional condition (A 5). A S S U M P T I O N (A5) Consider a sequence {6n} with dn e © for all n. Then || AW(0O)|| -> oo in Pn>0 -probability whenever

The following result can now be established. 7.1 Under assumptions (Al-5), for testing the hypothesis H: 6 = d0 against the alternative A: 6 #= 60 at asymptotic level of significance a, the test defined by (2.5) possesses the optimal properties mentioned in Theorems 2.1, 2.2, 4.1, 6.1, 6.2 and also has the property that its power converges to 1, i.e. THEOREM

Pn(dn\ ) -> 1, whenever 1 1 ^ ( ^ — ^0)11 ~^ °°Proof The proof is immediate. Since F is positive definite, so is F" 1 . Thus there exists a positive number p such that z'F- 1 * ^ p\\z\\* for all zeRk. Therefore J3n(6n; <j>) = SQ ^ ( A J = Pe (AneDc)

= Pe ( A ^ F - ^ > d).

However, this last quantity is greater than or equal to Pen(p\\An\\* > d)

which converges to 1 by (A5). Thus fin(dn; <j>) -> 1, as was to be seen. J Exercises 1. Verify (1.7). 2. Verify (1.8). 3. Show that if (2.7) in Theorem 2.1 is true, then it is also true if all tests involved are based on the random vector A* rather than An and conversely. 4. For each n and a bounded sequence {cw} of positive numbers, let heRk and set dn = 60 + hn-b. Then heE% if and only if 6eEn, where En = EnCn and E% = E*(cn) are defined

198

Multiparameter optimal tests

by (1.11) and (1.12),respectively. For heE* and #as above, set R n,h = Rn,K-eo)ni = K,0n- Then show that if (2.7) in Theorem 2.1 holds true, it is also true if all tests involved are based on the random vector A* rather than An and power is calculated under R$9en rather than P0n. 5. Provide a detailed proof of Lemma 4.1 (i). 6. Verify (4.4). 7. Justify the passage from (4.17) to (4.19) by means of transformation (4.18) and also the fact that the Jacobian of the transformation is given by | Jm\ = Jm = m~4(Aj~1). 8. Verify (4.23) 9. Establish inequalities (5.1) and (5.2). 10. Verify (6.8). 11. Verify (6.16).

Appendix

In this appendix, we gather together some of the basic theorems used in the body of the monograph. Some of the proofs are supplied and for the omitted ones the reader is referred to appropriate sources. In formulating the theorems herein, we do not strive to present them in their most general form. Simplicity and the needs of the present monograph have been the criteria employed rather than obtaining the most general results possible.

1 Some theorems employed in Chapter 1 Let 8 be a metric space and let S? be the topological Borel (7-field in 8. Then for a probability measure J? defined on £?, a set A e£f is said to be an ££-continuity set \i£?(dA) = 0, where 8A is the boundary of A. T H E O R E M 1.1A Let S b e a metric space and let Sf be the topological Borel cr-field in 8. For n = 1,2, ...,let J2?n, ££ be probability measures defined on Sf. Then J§?n => JSf if and only if Sen{A) -> Se{A) for all £ -continuity sets A in ST. This theorem is contained in Theorem 2.1 in Billingsley [4], pp. 11-14. T H E O R E M 1.2A

Suppose (8,6?) = {Rm9Bm), and for

let £Pn, ££ be probability measures defined on Sf. Then jSfn => JS? if and only if J/dJS^ -> jfdJ? for every real-valued continuous (and bounded) function / defined on 8 and vanishing outside a compact set. Proof One direction of the theorem is immediate from the definition of weak convergence. Thus it suffices to prove the [ 199]

200

Appendix

other one. To this end, let g be a real-valued, bounded and continuous function defined on 8. We wish to show that (1.1) For j = 1,2,..., let 8j be the (open) sphere of radius j centred at the origin. Then 8 — U Sj, and 8$ (the closure ofSj) is a compact subset of Sj+1. Let d be the metric in 8 and for J. c $ ? define d(aU) by d{xA) = M{d{xy). yeA} (L2) Let also the function ^6 be defined by (I if t^O, j ^(^) = | l - ^ if 0 < ^ < 1,1

(o

if t& 1.

(1.3)

J

Then for some 0 < 8 < 1, set

[JM]

(1.4)

Then fa is continuous and vanishes outside the compact set Kj = $7+0. It follows that gfj = g on Sj and gfj = O on ^ = SJ+,.

(1.5)

Next \gfj\ ^ |gr|, integrable with respect to any probability measure, and gfa -> g pointwise, since/;- -> 1 pointwise. Therefore the dominated convergence theorem applies and gives that Jg/icLS?n->JflrdJ2?n and

\gfjd£?->

as j-*oo, UdJSf

for n ^ l (1.6)

as j -> oo.

(1.7)

Hence by means of our assumption, (1.5), (1.6) and (1.7), one has that lim LdJ?n = lim (lim (gfjd
(gfjd^n

= Urn fg/idJS? = fflrdJSf,

Theorems employed in Chapter 1

201

provided lim (lim fgfjdJ?n) n^-co

\./—>oo J

= lim (lim (gfjd^). J

j->co\n—>coj

(1.8) J

For (1.8) to be true, it suffices to show that {gfad^

-> \gd^n

uniformly in n > 1.

(1.9)

Let M be a bound of \g\. We have then, on account of (1.5) and (1-4),

I faf,dJ2P n - \gdJ?n = I f

_ gf^j ^

f jfiSf; ^

.

(1.10)

For e > 0, set e' = ej2M. Then there exists a compact set * - * ( • ) such that #{K)>1_jm (L11) Let jx be the smallest j for which K cz /S^. Then, clearly, Z c Sy c ^ c 5 i + 1 0 > j x ) .

(1.12)

By means of this and (1.4), we have then that

so that

jfh&&* < ^»(^ + i)

and

(» > 1),

jg?(Z) ^ f4dJS?.

(1-13) (1.14)

Taking the limits in (1.13) as w->oo, utilizing the fact that |/;icL£?TC-> J/^dJSf and also taking into consideration (1.11) and (1.14), we obtain 1 - e ' < JS?(Z) ^ jfh&&

< liminf& n

Hence there exists n0 such that JS^(^) > 1 — e', J ^ J x + 1, n ^ ^ 0 ; equivalently,

202

Appendix

From (1.12) and (1.15), one has then

- ( l - e ' ) = e'

(j>j± (1.16)

On account of (1.15) and (1.16), relation (1.10) gives l9n>n0).

(1.17)

Finally, applying (1.6) for n = 1, ...,n0— 1, one can find j2 such that

From this result and (1.17), it follows then that there exists j0 such that i /•

/•

e(j which is what (1.9) asserts. | R E M A R K I . I A This theorem may be established under more general conditions regarding the space 8 than the ones employed herein. T H E O R E M 1.3A Let S be a metric space and let S? be the topological Borel cr-field in 8. For n = 1,2,..., let JS^, J§? be probability measures defined on £f. Then if {^n} is tight, it is relatively compact. If 8 is separable and complete, then the converse is also true. For the proof of this theorem, the reader is referred to Billingsley [4], Theorems 6.1 and 6.2, p. 37.

Let P, Q be probability measures defined on the cr-field si of subsets of Q, and let /, g be densities of P, Q, respectively, relative to a dominating cr-finite measure [i (e.g. ). Then T H E O R E M 1.4A

where

\\P-Q\\ = 2svp[\P(A)-Q(A)\; Aesf].

Theorems employed in Chapter 1

203

Proof For any A estf, one has

c

P(A) — Q(A) —

J

c

c

A

j

fdfi—X gdfi— j

A

f

(f—g)d[i A

f

J A,

J A2 d

=

(f-g) f*-\ (g-f)ty> J Ax

JA%

where A1 = A n (/-gr > 0), A2 = A n (/-gr < 0). Thus |P(^)-Q(^)|=

(f-g)d/i-\ \JA1

Since f-g

[g-f)d[i

(1.18)

jAt

> 0 on ^4X and g-f > 0 on A2, (1.18) is maximised by

maximising

(f—g)d[i and by minimising (g-f)dfi. This J AX JA3 is achieved by taking A = B = (f—g > 0), so that Ax = B, A2= 0 . T h u s

m*x[\P(A)-Q(A)\;Aes/]= In an entirely similar way, one obtains m&x[\P{A)-Q(A)\;Aes/]=

I (f-g)d/i. JB

(1.19)

\ (g-f)dfi,

(1.20)

Jc

where G = (g-f > 0). From (1.19) and (1.20), one has then that

= 2m&x[\P(A)-Q(A)\;Ae<s/l as was to be shown. ] THEOREM 1.5A Let S b e a metric space and let £f be the topological Borel cr-field in S. Then if P,Q are probability measures on £f such that J/dP = jfdQ for every real-valued, bounded and continuous function/ on S, it follows that P, Q are identical. This is Theorem 1.3 in Billingsley [4], p. 9.

204

Appendix

2 Some theorems employed in Chapter 2 In this section, we prove Vitali's theorem which is instrumental in establishing differentiability in quadratic mean, and also the central limit theorem for martingales which is of basic importance in deriving asymptotic distributions. THEOREM 2.1A

(Vitali'stheorem)

Forn = 1,2, ...,letXn, X r

be r.v.s defined on the probability space (£i, si, P). Then Xn -> X if and only if Xn -> X and S | Z n | r -> S\X\r finite (r > 0). For the proof of this theorem, we shall utilize the following lemma due to Scheffe. LEMMA 2 . 1 A

For n= 1,2,..., let Yn, Y be non-negative p

r.v.s defined on the probability space (Q,,s/,P). Then Yn-+ Y and SYn^SY

finite

if and only if Yn -> 7.

Proof One direction of the lemma is immediate. Thus it p

suffices to prove the other one. Now Yn -> Y implies that p

(Y-Yn)+-> 0. Furthermore, the fact that Yn ^ 0 also implies that (Y — Yn)+ < Y. Then the dominated convergence theorem applies and gives that ${Y — Yn)+ -> 0. By assumption, we also Hence £(Y-Yn)--+0. This result, have that g(Y-7n)-+0. along with /(Y-YJ+ -> 0, implies then that €\ Y-7n\ -> 0, as was to be seen. ] Proof of Theorem 2.1 A Since one direction of the theorem is immediate, we concentrate in establishing the other one. To this end, set Yn = \Xn\r% Y = |X| r . Then Lemma 2.1A applies and l

I

givesthatr w -> 7,orequivalently, | X J r - > |Z| r .HenceJ \Xn\rdP are uniformly continuous by the £r-convergence theorem (see p

Loeve [1], p. 163). This result, along with the fact that Xn -> X, r

implies that Xn -> X again by the theorem just cited. The proof of the theorem is completed. | The theorem to be proved below is known as the central limit

Theorems employed in Chapter 2

205

theorem for martingales for reasons to be explained shortly. A proof of this result is presented in Billingsley [1], Theorem 9.1, pp. 52-61 under the assumption that the underlying r.v.s have a finite moment of order 2 + 8 (8 > 0). Subsequently, this assumption was weakened by the same author (see Billingsley [2]), so that 8 may be taken to be equal to zero. However, none of these proofs is particularly easy to follow. An alternative proof of the same theorem is also discussed in Billingsley [4], Theorem 23.1, pp. 206-8, but this proof is based on previous results on weak convergence of probability measures. Finally, an exposition of the proof of the theorem, based only on the standard martingale and ergodic theory theorems, is given by Ibragimov [1]. The proof to be presented herein is based on this last paper. T H E O R E M 2.2A (central limit theorem for martingales) Let {Xn},n = 1,2,... be a stationary and ergodic stochastic process defined on the probability space (Q,stf, P) and such that

SX\ =
^l9s/0

(n > 1), (2.1)

= {0,Q}. Then (2 2)

^A^'V"^^'*

'

REMARK 2.1A

(i) Clearly, £Xn = 0, n ^ 1, so that a2 = ^ Z 2 , n > 1. (ii) For all n^ 1 and Jfc > 1, (f ( X ^ X ^ ) is finite, by the Holder inequality, and = £\_Xn${Xn+h\dn+h_j\ by means of (2.1); i.e. S(XnXn+k)

=0

= O,n,Jc ^ 1, and hence = no*.

Therefore (2.2) is the conclusion of the central limit theorem in its ordinary form. Furthermore, the partial sums

206

Appendix

form a martingale because, by means of (2.1), one has j=i

a.s.

F

/ n

a.s.

I"

In

a.s.

VKI—1

a.s.

/n—1

i

^

\

%

i

-i,

I

Hence the qualification of the theorem as central limit theorem for martingales. (iii) Let Yn = XJcr. Then {Yn}, n ^ 1, is also stationary and ergodic (see Proposition 6.6 and Proposition 6.31 in Breiman [1], pp. 105, 119) and *Y\ = 1, ^(T n |r lf . . . , 7 ^ ) = Oa.s.

(n > 1).

So in the original process, we may assume that a = 1 and we shall do so. (iv) Let {3£n}, n^ 1, be the coordinate process and let P be the distribution of (XVX29...) under P . Then the distribution of (Xv X2,...) under P is the same as the distribution of (J£v 3£2,...) under P, so that

Thus we may assume that (Cl,s/, P) = (JS00, ^ c o , P) and that the original process is the coordinate process, and we shall do so. (v) The stationary and ergodic coordinate process {Xn}, n ^ 1, may be extended to the process {Xn}, n = 0, ± 1,... which is also stationary and ergodic and has the following property *{Xn\Xn_l9Xn_2,...)

= 0 a.s.

(n = 0, ± 1,...)

(2.3)

(see also exercise at the end of this section). That (2.3) holds is seen as follows. Set Xn_l9...)

(n = 0, ±1,...)

(2.4)

Theorems employed in Chapter 2 and

Fn>n_k

207

= <%{Xn,...,Xn_k)

(k*l).

Then for each n, ^(Xnl^)

= Hm
a.s.

(2.5)

(see, e.g. Theorem 5.21 in Breiman [1], p. 92). Now for every A e^rn_ln_k, there exists Be@tk such that A=

{Xn_k,...,Xn^B

and, by stationarity,

= I %k+il(xv... X^B^V •••?^fc)(iJD = I where A1 = (Xv ..., X^B.

But by means of (2.1),

f Xk+xdP=\ J A'

Xk+1dP9

£(Xk+1\s/k)dP = 0, J A'

so that

Xn dP = 0 for every A e ^n^1} n-k.

Finally,

f Xnd^= f ^ ( X J J ^

^

so that jj(Xn\^n_1
= 0 for every

It follows that i(Xn\ 3Fn_Xn_k) = 0 a.s. and therefore from (2.5) one concludes (2.3). Proof of Theorem 2.2 A Set k

(2.6)

Since/„ defined by (2.6) is the characteristic function of Snn~i, by the continuity theorem, it suffices to show that /»(*)- e-* t2 ->0

(teJJ).

(2.7)

Appendix

208 Set

2

n — kt \ ( —IT2) (itZk)] (teR,k = 0,1, ...,n,n > 1).J (2.8) From (2.6) and (2.8), we clearly have that fo(t) = <j>Q{t) = e-4*2, <}>n{t) = 1, fn(t) = /„(«)

(*eR, n > 1).

(2.9) On the other hand, by means of (2.9), one has

Therefore, on account of (2.7), in order to establish the theorem it suffices to show that

(teR).

(2.10)

Since the proof of (2.10) is rather long, we present the intermediate auxiliary results as lemmas. L E M M A 2.2A Forw ^ 1 and k = 0,1, ...,n, let !Fh be defined by (2.4) and let <j>k and \jrk be defined by (2.8). Then for every teR, one has that

t

(\Xx\>n»») n-l

n Proo/ From (2.8) and (2.6), one has = exp ( -

= exp(-

.

(2-11)

Theorems employed in Chapter 2

209

i.e. fk+1{t) - fk(t) = 0fc+1(<) j«f [exp (itZk) exp ( i l l

Now consider the expansion ,x),

\B(m,x)\

(m+iy.

and apply it for m = 1 and m = 2. We get then

and Also That is, one has eix =l+ixwhere

ix2 + R(2, x),

\H(2,x)\ ^ \x\* and also

) \ |iJ(2,a;)| ^ x .) 2

Replacing x by tXk+1n~i in (2.13), we get \tY

/2TT

exj>(itXk+1n-i) = l + ^ J J L ^

+ BJt; Xk+1)

with \Bn(t;Xk+1)\^&^

and also

\Rn(t; Xk+1)\ (2.14)

Next

«? [exp (itZk) exp (UXk+x n~i) \ ^k] a.s.

= exp (HZk) ^[exp 14

(itXk+1n-i)\^k] RCO

210

Appendix

since Zk is ^.-measurable. On the other hand, (2.14) and (2.3) imply that

Therefore «f [exp (itZk) exp (itXk (UZk) ex k

By means of this, (2.12) becomes as follows

(2.15) But

K{

R

!^}|

(2.16) «;Xfc+1)|, whereas by means of (2.14) and stationarity (see, e.g. Proposition 6.6 in Breiman [1], p. 105), one has S\Rn{t;Xk+1)\

=

j^^ni^\Rn(t;Xk+1)\dP

nL

Theorems employed in Chapter 2

211

On account of this, (2.16) becomes as follows

(2.17) In the expansion \

0 < 0(x) < 1,

2

t replace x by — —. Then we obtain

By means of this and the fact that \k(t)\ ^ 1, one has that k=0

k=o -
s

(2.18)

Then summing over k in (2.15) and utilizing (2.17) and (2.18), one obtains (2.11). The proof of the lemma is completed. ] It has been stated that in order to establish the theorem it suffices to show (2.10). By means of Lemma 2.2A, this will be true if the right-hand side of (2.11) converges to zero (as n -> oo). Since the first summand does so by the fact that SX\ < oo, it suffices to show that n-1

n

2<

or by setting (2.19) 14-2

212

Appendix

it suffices to show that

t

0.

(2.20)

n For n ^ 1 and k = 0,1,...,n, define the r.v.s Vkn by Vkn = 0 \^k-hl\^k—n)}

\^'**)

where «^_w is given by (2.4). For each fixed k and as n -> oo, (2.4) gives that «

=

n^k-n-

(2.22)

71=1

Therefore Theorem 5.24 in Breiman [1], p. 93 applies and gives that a.s. 17

^ 17 (1)

Furthermore,

vk = ^ Le V^-^+il^fc^J

=

^"^fc+i

=: A

*

(2.24)

Now, in order to facilitate the proof of (2.20), we establish first the following result. L E M M A 2 . 3 A For each k ^ 0, let Yk and Vk be defined by (2.19) and (2.23), respectively. Then one has that

n-l

> 0. (2.25) Proof From (2.23), one has, in particular, that VOn->Vo. Therefore for e > 0 there exists an integer m = m(e) (> 0) such that -Vo\ < e. (2.26) Om

Next for k > m, and stationarity, one has that

{explitn-i /

-m

\

j = -k+i

\

/

-w

\

S Z, (Fo-Fo) + exp (tt^-i S Xj) I I , ) l o \- F o j=-k+i ) ] . (2.27)

213

Theorems employed in Chapter 2

From (2.23) and (2.22), it follows that Vo is J^-measurable for all n ^ 1. Also, from (2.19), Yo is J^-measurable. Therefore

[exp (itn-^jE^

(70-F0)j

[exp {tor*.3k -i "f lj=-k+l

-V )], x \(V / Om o J the last equality being so because, on account of (2.19) and (2.21),

= vom. Therefore

f X,W-F O)1 / J

e, (2.28)

the last inequality being true because of (2.26). Next, by employing the inequality |e h — 11 < \z\, we obtain (i \

exp

-f \|j==-m

f

y)

__

- ll (7 0 -F 0 )

,.) - ll (Yo-Vo) /

J

Appendix

214 /i

\Y0-V0\dP+j

o

S

(

i—m + l

x(|<|

o

s

(II S j

m+l

and this last expression can be made less than a multiple of e since the first summand goes to zero (as n->oo) because of the finiteness of the integral j\Y0 — Vo\ dP. By means of this result and (2.28), (2.25) follows in an obvious manner.] Now by means of (2.20) and Lemma 2.3 A, in order to prove the theorem it suffices to show that n-l

0,

n or by setting it suffices to show that

=

Vk-l,

(2.29)

n-l

(2.30)

0.

This will be shown by appropriately truncating Wk. More precisely, for M > 0, set _ [ 0 if \Wk\^M] iWk if \Wk\^M Wk{M) = \ ; Wk(M)=\ \Wk if \Wk\ > M.) 10 if \Wk\ > M _ (2.31) From (2.31), it follows that Wk = Wk(M) + Wk(M), so that /2

n-l

n

k=0 n-l

n n n-l

4>k+1(t)
/2n-l

.

(2.32)

Theorems employed in Chapter 2

215

By(2.31),stationarityandthefactthat^|W^| < oo (by (2.29)and (2.24)), one has that

S\Wk{M)\ = [

\Wk\dP=(

J (\Wk\>M)

\W0\dP<e J (\W0\>M)

for sufficiently large M. For such an M,

and therefore from (2.32) it follows that in order to establish (2.30), and hence the theorem, it suffices to show that K = ^ s V a + i W * [ « p (itZk) Wk(M)] -> 0.

(2.33)

The proof of it will be facilitated by the following auxiliary result. L E M M A 2.4A

With An given by (2.33), one has that

{exP (itZ*-i) \jfQ ^)]},

(2-34)

(2Y2

where 0 < dn(t) < 1 and \Rn(t,Xk)\ IV

Proof We have k=0

Now for ak,fik,k= Abel's identity

S a*A = S ^ ( ^

k=l

(2.35) k

1,..., n and sk = 2 A> consider the following fc=i

(

216

Appendix

Applying this identity with <** = &(*) exp (HZ^)

and fik = Wk

(2.35) becomes as follows

fc=i b=i

*=i b=o 1 #w(0 exp {itZn_x). b=o On account of (2.8), one has

(2.36)

%-^<2\

2 — g j exp ,.,_ {itZk_x,) - exp /y M -K( -i + l )-'i -\ j

(

^ — (]c •+• 1 ) £ 2

since Z!e — Zk_1 = Xnn~l as follows from (2.6). By means of this result, (2.36) becomes as follows

= fc=i "S1U=o Ps^t-af)]J (2.37)

Theorems employed in Chapter 2

217

Multiplying both sides of (2.37) by t2jn, taking the expectations and utilizing the definition of An by (2.33), as well as (2.35), we obtain

K =£ (2.38) Now in the expansion ex=

l+xd(x),

0
we replace x by — fij2n and we get 1

- £

m

°<
(2-39)

Also in the expansion eix = l+ix-±x2

+ R(x),

\R(x)\ ^ x2,

we replace x by tXkn~i and obtain erp(i*Xfcn-*) = i + ^ J ^

+

Bn(t9Xk)9

\Rn(t,Xk)\ <

^ (2.40)

From (2.39) and (2.40), we get

)

j

so that

xp (itZk_x) [ | ^ ] [ (

- «f {exp ( i ^ ^ ) exp (itZ^) l^WAM)] ) . b=o Jj

(2.41)

218

Appendix

Next

=
=
Finally, multiplying both sides of (2.44) by -^-(pk+i^) a n d summing over k, as A; goes from 1 to n — 1, and also utilizing (2.28), we obtain (2.34). The proof of the lemma is completed.] Thus on the basis of (2.33), in order to prove the theorem it suffices to show that each one of the four terms on the right-hand side of (2.34) converges to zero (as n -> oo). This is done in the following four lemmas.

Theorems employed in Chapter 2 LEMMA 2.5A

219

Referring to (2.34), one has

l ] -> 0. (2.45) Proo/ From (2.29) and (2.24), it follows that iWk = 0 and 2. On the other hand, from (2.31) and for each k, one has Wk{M)->Wk as if-* oo

and

|WJ.(if)| < |fl£|.

Then the dominated convergence theorem implies that
(2.46)

Next, by stationarity, one has that

=-(fc-i)

J70exp(i*Z0) f _ S,_ T»5(if)+ 4 * |T0 exp

(i«Z

j=-(fc-D

= ^ [ 7 0 exp (itZ0) yJ + *^[F0 exp («Z0) ^ ( l f ) ] , where

yfc = j

S

^(Jf)-#^(if).

(2.47) (2.48)

<4

Therefore multiplying both sides of (2.47) by —%ic+i(t) a n ( i sum#2'2

ming over k, as & goes from 1 to n— 1, one has

/4

?i-

(2.49)

220

Appendix

Now on account of (2.19) and (2.46), £YQ = 1 and &W0{M) -> 0 as M -> oo. Therefore for all sufficiently large M, one has that t^Yo) \£W0(M)\ < e. Thus, in order to show (2.45), it suffices to show that the first summand at the right-hand side of (2.49) converges to zero; i.e.

^sV|r o y fc |->o-

(2.50)

By stationarity and ergodicity, one has that £\yk\-+0

as &->oo,

(2.51)

where yk is given by (2.48) Thus for all sufficiently large Jc,Jc ^ Jco = k(e), say, one has that

*\7k\ <e-

Set

J n—

w l&=i

(2.52)

W

Then for n > Jco+ 1 and by means of (2.52), we have I

It follows that

kQ-l

I

7 S ^Wk\+

n-l

T S ^l7fcl

££n -> 0.

(2.54)

From (2.31), one has that \Wk(M)\ ^ M for all k. Then (2.48) gives that Syk ^ 2M, so that (2.53) implies that in < 2Jf.

(2.55)

Hence (F0>c)

f

(Fo U dP^2M(

Yo dP + cSgn.

) J (ro>c) Thus the fact that «?70 = 1 and (2.54) imply that J(

^0.

(2.56)

Theorems employed in Chapter 2

221

Therefore, on account of (2.53) and (2.56),

This establishes (2.50) and hence the lemma itself. ] LEMMA 2.6A

Referring to (2.34), one has

]} > 0. Proof Applying arguments similar to the ones used in the proof of the previous lemma, one has that

£jexp (itZ^) [ g ^ w ] } = <^{exP (itZo) p0 where yk is given by (2.48). Hence, proceeding as in (2.49), we obtain )

The right-hand side of this last inequality converges to zero by means of (2.46) (for k = 0), (2.53) and (2.54). The proof of the lemma is completed. ] LEMMA 2.7A

Referring to (2.34), one has Bn(t, Xk)\ -> 0.

lj=o

Proof Once again, stationarity yields

= M(exp(i«Z0) R

S

Wt(M)-£W0{M)\ (itZ0) En(t, X,

Rn(t,

222

Appendix

Therefore (2.40) and (2.48) imply that

] Bn(t, Xk)

| XD + WXl) \*W0(M)\ (2.57)

-> 0 as M -> oo by (2.46), it follows that in order to prove (2.57) it suffices to show that ^

0

.

(2.58)

But on account of (2.53),

and

S{^nX\) = f

(6,Z!)dP+ f

«n

(Xl>c)

by means of (2.55). Now the fact that SX\ = 1 and g£n -> 0 by (2.54), implies that S(gnX\) -» 0 and therefore (2.58) by way of (2.59). This completes the proof of the lemma. ] Finally, the following lemma completes the proof of the theorem. L E M M A 2.8A

£

Referring to (2.34), one has

{erp (itZn_x) [ ^ J W ] } -> 0.

Theorems employed in Chapter 2

223

Proof Utilizing stationarity and (2.48), one has

(i*Z0)yJ + n*exp(i*Z0: Hence "'

lM\yn\+t*\*WQ(M)\.

Then (2.46) and (2.52) complete the proof of the lemma and therefore that of the theorem itself. ] Exercise Referring to Remark 2.1 A (v), consider the stationary and ergodic coordinate process {Xn}, n ^ 1, defined on the probability o space (i? 0 0 ,^ 0 0 ^) and set (-WU>, J ^ ) = II ( % ^ ) , where j — — oo

and let {Xn}, ^ = 0, ± 1,..., be the coordinate process defined on (5*00, ^"oo). On ^Loo, define the probability measure P o by ^ xnJ . . , I

W

< xn+k)

(n^O,Jc>

0),

and let Px be defined on ^ ^ ^ by Px = P o x P. Then for the process {Xn}, w = 0, ± 1,..., show that (i) It is stationary,

A

P

(iii) It is ergodic. (See also Proposition 6.5 in Breiman [1], p. 105, and Billingsley [3], p. 206.)

224

Appendix

3 A theorem employed in Chapter 5 T H E O R E M 3 . 1 A Let Cv C2 be kxk positive definite, symmetric matrices. Then Cx - G2X is positive semi-definite if and only if (h'C2h)i ^ v'hKv'C^ for all non-zero h, veRk. The proof of this theorem is based on the following lemma which can be found in Rao [3], p. 48. L E M M A 3.1A

Let A be a positive definite kxk matrix and

let ueRk. Then and the supremum is attained at x* = A~xu. Proof of Theorem 3.1A The proof is by contradiction. Suppose that G1 — C2l is positive semi-definite and that there exist nonzero, h,veRk such that (h'C2h)b < v'h^v'G^)^. Hence v'Cxv < (v'h)*lh'C2h. By Lemma 3.1A, so that v'Gxv < v'C^v, or equivalently, ^(C^ — C^1) v < 0. However, this contradicts the assumption that G1 — G21 is positive semi-definite and therefore (h'C2h)b ^ v'hKv'Qvft for all non-zero h,veRk. Now suppose that (h'C2h)b ^ v'hKv'Qvfi for all non-zero h,veRk and let Cx — C2X be not positive semi-definite. Then there exists 0 4= xeRk such that x'^ — C^x < 0, or equivalently, x'Cxx < x'G2xx. Utilizing our assumption with v = x, we obtain (h'C2h)i ^ x'hftx'Cixft and hence x'Cxx > (xfh)2jhV2h. Thus x'Cxx > sup (by Lemma 3.1A), SO that x'Cxx ^ x'C2xx. However, this contradicts the previously established inequality x'Cxx < x'C2xx. Therefore Gx — G2X is positive semi-definite and the proof of the theorem is completed. ] To this theorem, there is the following immediate corollary.

A theorem employed in Chapter 5

225

COROLLARY 3 . 1 A The conclusion of the theorem remains valid if the inequality in the theorem is true for all non-zero rationals h,veRk.

4

Some theorems employed in Chapter 6

In the present section, as in the previous ones, we gather together some of the theorems used in the various proofs of Chapter 6 for easy reference. T H E O R E M 4 . 1 A If F is a h x k positive definite, symmetric matrix, there exists a non-singular matrix Jf such that M'M = F.

Proof From the assumptions it follows that there exists an orthogonal matrix P such that P'YP = D, where D is a diagonal matrix with (diagonal) elements dj,j = 1, ...,& (the characteristic roots of F). From the orthogonality of P it follows that P " 1 = P'. Therefore PTP = D gives that F = PDP'. Let now Do be the diagonal matrix with (diagonal) elements y/dj (j = 1, ...,&). Then, clearly, D0D0 = D. Therefore F = PDP' = PD0D0P' = PD0D'0P' = (PD0) (PD0)'. Since P , D o are non-singular, so is PD0. Therefore, by setting (PDQY = M, we obtain F = M'M, as was to be seen. ] We now wish to establish Theorem 4.2 A below along with its Corollary 4.2 A. For this purpose, some notation and auxiliary results are needed. The proofs of some of these results are presented herein and the reader is referred to other sources for the omitted proofs. To start with, let T be an open subset of Rk and let V be a ^-dimensional random vector whose distribution Qtv is absolutely continuous with respect to Qt }V (to be shortened to Qo) for some toeT and has a probability density of the standard exponential type, namely, [dQtfVldQ0] = G(t)expt'v 15

(veB\teT).

(4.1) RCO

226

Appendix

We shall assume that if P is any probability measure on T, then the indefinite integral of C(t) is a er-finite measure; i.e. if

fiP(B) = I G(t) dP, then fiP is cr-finite for every P . (4.2) JB In the family (4.1), suppose we are interested in testing the hypothesis H: t = t0 against the alternative A: t 4= ^o o n ^ e basis of the random vector V. Let A = {a0, ax} be the action space, where a0 stands for accepting H and ax for rejecting H, and let $ be a test function defined on Rk. Then if V = v,
The m& B^{t) corresponding to (V) + L(t,a0) [1 - < ^ ( F ) ]

and is easily seen that x)-L{t,a^^{t),

where

(4.3)

^(t) = Stcj>{V) =

In all that follows, the loss function L is taken to be as specified \ ^ below r/, L{tQ9a0) = 0, and for t 4= t0, L(t,a0) = 1, 1^,^) = O.J For such a choice of the loss function L, the corresponding risk, relative to the test 0, becomes as follows

Now for a fixed test ^ (see proof of Theorem 4.2 A for its specification), consider R^(t) as a function of £ and define the following (bounded) loss function

,a) = L(t,a)-B+(t).

Theorems employed in Chapter 6

227

Then for any test (j) the corresponding risk i?*, associated with £*, is given by Bp) = L*(t9a0) + [L*{t9ai)^L*(t9a0)]fl^

(4.6)

according to (4.2), and is easily seen that Bp)=Bf(t)-B+(t).

(4.7)

For the above choice of L, one has then that *o)>

and for

1

t * t0, B%(t) = 1 -

Let W be a probability distribution over 2\ Then the Bayes (or global) risk r(, W), corresponding to the test <j> and the prior distribution W, is given by

j

(4.9)

A Bayes test, with respect to the prior distribution W, is any test which minimizes r(9 W), as (j) varies over the class of all tests. A minimax test (associated with the risk B^(t)) is any test within the class of all tests which minimizes the following quantity The first result in this section is to the effect that a Bayes test with respect to a prior distribution is essentially the indicator of the complement of a closed, convex set. More precisely, one has the following result. L E M M A 4 . 1 A In connection with the family given by (4.1), suppose we are interested in testing the hypothesis H: t = t0 against the alternative A: t 4= t0 on the basis of F. Then any Bayes test <j)Wi with respect to the prior distribution W, is given by

(1 if veCfc]

\ [0 if veCW9)

(4.10)

where Gw is a closed, convex set in Rk. The test <$w may be defined in an arbitrary (measurable) way on the boundary dCw of the set Cw. E5-2

228

Appendix

Proof The lemma is established along the lines of the proof presented by Birnbaum [1]. Let w0 be the weight assigned to {£0}. Then one has, by means of (4.10) and (4.5), that

i,W)=(

B,dW={

B.AW+i

B,

•J, But

(7(0exp^dPfl^(i;)dQ 0 , J where the interchange of the order of integration is valid by Fubini's theorem; also j3^(t0) = J
, W) = (l-wo)+ !Uw0- f It follows that the Bayes risk r(^, W) is minimized by the test cj)w which is defined as follows

if veAx = = LeRk ;2w0< f 0 if = \veRk; 2w0 > \ I JT

$w

C(t)expt'vdw\; )) (4.11) may be arbitrary (but measurable) on the set

As = AZ(W) = [veBk; 2w0 = f C{t)expt'vdw). [ JT J In order for this test to be of the form (4.10), it remains for us to show that the set A2 u Az is a closed, convex set and that Az is its boundary. To this end, set = [veRk; I C(t)expt'vdW I JT

^ 2w0

Theorems employed in Chapter 6

229

and consider the inequality exp[Au1 + (1 - A) u2] < Aexp% + (1-A)exp^ 2

(0 < A < 1).

Let vv v2eCw and set ux = t'vvu2 = t'v2. Then

f

J

= f G(t)exp[M\+(l-A)tfv2]dW JT

^ A f <7(*)exp$'fl1dFr + (l--A) f JT JT < A2wo + (1-A)2w o = 2wo, so that Xvx + (1 — A) v2 e Gw; i.e. Cw is convex. Next, by means of (4.2), one has JT and

JT

G(t) exp t'v d W =

JT

exp t'v d/iw =

JT

exp v't d/iw

expt/tfd/fcrp is continuous (in v) by Theorem 9 on p. 52 in

Lehmann [3]. Therefore the set Gw is also closed. Finally, to show that A3 = dCw. In the first place, 3CW c Az. In fact, if v0 e dCw, then v0 e Gw since Cpp is closed and therefore dCw c (7^. Thus v 0 G-4 2 u^ 3 . However, vo$A2, provided, of course, that A2 #= 0 . This is so, because -42 ^s open, being the inverse image of (0,2w0) under the continuous function

exp v't d/iw, and thereJT fore if v0eA2, there would exist a neighbourhood of vQ lying entirely in A 2. Hence v0eA3. The same conclusion holds if A 2 = 0. This shows that dCw c ^1 3 . Next, we assert that As c ^C^. For a contradiction, suppose that voeAs whereas vo
230

Appendix

with respect to v0 and within the neighbourhood in question. Then for some 0 < A < 1, one has v0 = Av1+(1 — A)v2 and therefore

2wo= I

JT

G(t)expt\dW

^ A f C^exp^dTf+(1-A) f C(t)expt\dW JT

JT

< A2w0 + (1 - A) 2w0 = 2w0,

a contradiction. Thus voeAs and every (sufficiently small) neighbourhood of v0 lies entirely in Az. Furthermore, this is true for all 0 < w0 ^ 1. (For w0 = 0, A2 = A% = 0 and the theorem is trivially true.) However, this cannot be true because

A\{W) = 0 for any probability distribution W in T. This is true for w0 = 1 because then Az is the hyperplane t'ov = log [2/O(£0)]. Thus suppose that 0 < wQ < 1 and let vv v2 be two points in a neighbourhood of v0 lying in As such that vQ, vl9 v2 lie on a line segment, v0 = A.vx+ (1 — A) v2 for some 0 < A < 1 and vl9 v2 lie on the two different halfspaces into which Rk is divided by the hyperplane t'Qv = t'ovo. This last restriction implies that tfov1 4= t'0v2 and therefore

exp t'vQ < A exp tfvx + (1 — A) exp t'v2

Now

(t e T)

and hence

f J T-{U)

G(t) exp t \ dW ^ A f

C(t) exp ^ x d W

J T-{U)

+ (1-A) f

C(t)expt'v2dW,

J T-{t0)

whereas G(t0) exp (^v0) < A(7(^o) exp (t^) + (1 - A) 0(*o) exp (^v8).

Theorems employed in Chapter 6

231

Adding up the last two inequalities, one obtains 2w0 < A2w0 + (1 - A) 2w0 = 2w0,

8b contradiction. The proof of the lemma is completed. ] The following result will also be needed in the sequel. L E M M A 4 . 2 A (weak compactness theorem) Given any sequence of tests { such that

for every real-valued, integrable function g denned on Rk. Proof I t can be found in Lehmann [3], pp. 354-6. ] LEMMA 4 . 3 A If {fim} is a sequence of tests converging weakly to the test (j> (relative to Qo), then

R*Jt)->R*(t)

(teT).

Proof By (4.6),

R$Jt) = L^aJ + iL^aJ-L^a^^Jt),

(4.12)

whereas

fitJt) = JVm(«)d&.p = j^m(v)C(t)expfvdQ0.

(4.13)

Now O(t)expt'v is Q0-integrable and therefore the assumption of weak convergence implies that

J 4>JV) O(t) exp t'v dQ0->U (v) G(t) exp t'v dQ0

By (4.12), (4.13) and (4.14), it follows then that R^Jt) -> R%(t) for every t e T, as was to be seen. ] In Lemma 4.1 A, it was shown that any Bayes test with respect to the prior distribution W was given by (4.10). The next objective in this section is that of showing that the weak limit of a sequence of Bayes tests is again a Bayes test. To this end, the following notions and results will be necessary. Let 8r = 8{09r) = {zeRk; \\z\\ ^ r}

232

Appendix

and for any closed subset, A, of 8r, define its 8-neighbourhood, N8(A), as follows

N8(A) = {zeRk; (z-u)'(z-u) < 8 for some ueA} (8 > 0). For any two closed subsets A and B of 8r, define the numbers St and #2 as follows 8X = inf {8 > 0 ; i c N8(B)}

and

82 = inf {8 > 0; B <= N8(A)}.

Then set d(A,B) = e^ + c^. It can be shown that the function d defined as above on the Cartesian product of the class of all closed subsets of Sr with itself is, in fact, a metric. For the proof of it, the reader is referred to Eggleston [1], p. 60. The distance d(A, B) between A and B is known as the Hausdorff distance. In terms of the metric just defined, one can formulate the following standard result on convex sets. LEMMA 4.4 A (Blaschlce selection theorem) Given any sequence of closed, convex subsets {Gn} of 8r, there exists a subsequence {Gv} and a closed, non-void, convex subset G of 8r such that d(Cv,C)-+0.

Proof It can be found in Eggleston [1], pp. 64-7. ] In the sequel, a generalized version of this lemma, due to Mathes and Truax [1], p. 684, will be needed. Namely, L E M M A 4.5A Let {Om} be a sequence of closed, convex sets not necessarily subsets of a bounded set. Then there is a subsequence {Cv} and a closed, convex set G (which may be empty) such that

d(Cvn8j,Cn8j)-+0

for j = r , r + l , . . . ,

some r > 0.

Proof First suppose that for a sufficiently large r there are infinitely many ms, forming a subsequence {CQo} such that Cqon8rJr 0. For a fixed r as above, consider the sequence of closed, convex subsets {Gqo n Sr} oiSr. Then by Blaschke selection theorem, there exists a subsequence {q^} c: {q0} and a non-empty, closed, Convex subset Gr oi8r such that d(GQl n 8r, Cr) -> 0. Next, consider the sequence {GQi n $r+i}- Then, as above, there exists a subsequence {q2} £ {q^ and a non-empty, closed, convex subset Cr+1 oi8r+l such that d{GQ2 n Sr+1, Gr+1) -> 0. It is easily seen that

Theorems employed in Chapter 6

233

Cr+1 (]Sr = Cr (see also Exercise 1 (i)). By induction, one has that there exists a subsequence {qi+1} ^ {#J and a non-empty, closed, convex subset Cr+i of Sr+i such that d(Gq.+i n Sr+1, Cr+i) -> 0, and that

o»* n £r+*-i

Now consider the sequences {Gq^, i = 0,1,... and let {Gv} be the diagonal sequence. Then, clearly, d(Cv 0 8p Cj) -> 0 for each 00

j = r, r + 1 , . . . . Set G = U Ca. Then G is non-empty, closed and convex (see also Exercise 1 (ii)). Furthermore, since it follows that

Therefore d(Cv n Sj, C 0 8^) -> 0, j = r, r + 1,..., as was to be seen. Next, suppose that for every positive r, Cm f) Sr #= 0 for only finitely many ms. In this case, take ( 7 = 0 . One then has that the lemma is trivially true with {v} = {m}. The proof of the lemma is completed. ] Now the lemma just proved has the following important corollary which is also contained in a result in the reference last cited. Namely, COROLLARY 4 . 1 A Let {m} be a sequence of Bayes tests with respect to some prior distribution W, i.e. tests of the form (4.10), and suppose that this sequence converges weakly to the test <j) (relative to Qo). Then (j) is the indicator of the complement of a closed, convex set in Rk a.s. [Qo] (and hence a.s. [Qt> v] for all teT). However, $ may not correspond to any prior distribution W on the parameter set T.

Proof Let {Cm} be the sequence of closed, convex sets corresponding to {^m} by (4.10). Then, according to Lemma 4.5A, there exists a subsequence {Cv} and a closed, convex set G such that d(Gv n Sj, G 0 Sj) -> 0 for j = r,r+ 1,...,some positive r. There are two cases to consider, namely, G is non-empty and C is empty. First, suppose that C + 0 . Let A be a compact subset of Rk disjoint from C. Then A is closed and bounded and 16

RCO

234

Appendix

there exists a ^-neighbourhood, NS(A), such that Nd(A) (] G = 0 (see also Exercise 2(i)). Choose j sufficiently large, so that N8(A) c= Sj. Then by the fact that d(Cv n ty, C n #,) -> 0, one has that CV(]A= 0, or equivalently, i g O j for all sufficiently large j> (see also Exercise 2 (ii)). Now {v} c {^m} and hence {0,,} converges weakly to 0. That is,

for all real-valued, integrable functions g defined on Rk. Let B be an arbitrary measurable subset of A and take g = IB. Then relation (4.15) gives f 0, d# 0 -» f 0d# o . But £ c ( ? J and 0, = 1 JB

JB

on C^ Thus

f ^ d e o = f dQ0, so that JJ5

JB

f ?idg o = f dg 0 JB

JB

for every measurable subset B of A.It follows that ^ = 1 a.s. [Qo] on A. Now Cc can be covered by a denumerable number of compact sets Aif i = 1,2,... and applying the previous argument to each one of these sets, one concludes that
(I if 4>{z) = \ Y

)

zeCcA \

V

(4.16)

JO if C°\ ' where C is a closed, convex set in Rk. The test may be defined in an arbitrary (but measurable) manner on the boundary dC of G. The tests given by (4.10) are also of the form (4.16). However, in

Theorems employed in Chapter 6

235

the following, we will be interested in tests of the form (4.16) which may not correspond to any prior distribution W on the parameter set. We are now in a position to show that the class of tests of the form (4.16) is an essentially complete class. The following lemma will facilitate the proof of this fact. L E M M A 4.6A Let {tn} be a dense sequence in T with t± = t0 (and distinct terms). Then for each j = 1,2,..., there exists a test (fij of the form (4.10) such that

#&(*<) < 0

(i=l,...,j,i=l,2,...).

(4.17) j

Proof For each J, consider the risk set Cj(R*) in R associated with the risk i?*; i.e. ),i = 1, ...J for some test Then the set Cj(R*) is convex. In fact, by setting for any two tests v u{t). To show that u(t) = R*(t) for some test . But then R$Jt) -> R$(t) by Lemma 4.3 A. Therefore R*rSt) "^ ^ | ( 0 J a s w a s t° be shown. Finally, for each j = 1,2,..., the risk set Cj(R*) is bounded, as is immediate from the boundedness of £*. By taking ^ = c and varying c in [0,1], one sees that the risk set Cj(R*) has points on all sides of the main diagonal, so that the main diagonal intersects Cj(R*). It is then clear that the minimax 16-2

236

Appendix

test <j>p say, corresponds to that point of Cj(R*) on the main diagonal with the smallest coordinates. We shall show now that
xi < dj9 i = l,...,j}.

Then, clearly, C(dj) is a convex set and C(di) and Cj(R*) are disjoint. Then by the separating hyperplane theorem (see, e.g. Theorem 2 in Ferguson [1], p. 73), there is a hyperplane w'z = a such that w'z ^ a for z e C^R*), and

w'z < a for z e C(di)

w'z = a for z = dj(l,..., 1)'.

Now if w = (yv ...,yj)'9 then yi ^ 0,i = 1, ...,j because if yi < 0 for some i, then by letting xi -> — oo in w'z < a and keeping the other coordinates fixed, we would have w'z -> oo which contradicts the fact that w'z < a for zeC(dj). Next, replacing the constant a by a>l\\w\\ = a and also setting w = w/||w||, we have that ^ = Wv -•>yj)t> where yi = yil\\w\\, i = 1,... J , is a probability distribution over the set {tv ...,^-}. So the test (j>p corresponding to the point z^ = d^l,..., 1)' of ^(i?*), has the property that w'zj = a and w'z ^ a for ^6(7^(7?*). Therefore ^ is also Bayes with respect to the distribution w = (yl9 ...,%•)' and hence ^ is of the form (4.10). Furthermore, from (4.7), it follows that R^(t) = 0 for allteT, and so, in particular, i?^(^) = 0, i = 1, . . . , j . Since ^ is a minimax test, it follows that R*fti) < 0, i = 1,..., j , which is what (4.17) asserts. ] We finally have the following result. T H E O R E M 4.2A For testing the hypothesis H:t = t0 against the alternative A: t 4= t0 in the family (4.1), the class of tests of the form (4.16) is essentially complete.

Proof Let ft be a test which does not belong to the class just described (this is the specification of the test i/r which was employed in the definition of the risk R*(t)). We shall show that there exists a test 0 in the class under consideration such that

Theorems employed in Chapter 6

237

Consider the sequence of tests {^} of Lemma 4.6 A. Then, by the weak compactness theorem, the sequence { v] for all t e T, by Corollary 4.1A. Therefore, without loss of generality, we may assume that 0 itself is of the form (4.16). According to Lemma 4.3A, (*= 1,2,...)On the other hand, R*m(t^) ^ 0 for all i and m. Therefore ( i = 1,2,...)-

(4.19)

In particular, R*(t0) ^ 0, or equivalently, /?^(£0) ^ J3^(to), by means of (4.5) and (4.8). This is the first of the inequalities in (4.18). Once more, by means of (4.5) and (4.8), inequality (4.19) ia equivalent to m > m (4.20) (i = 2>8,...). It remains for us to show that (4.20) is true for all £ e Tf t + tQ. But this is an immediate consequence of (4.20), by the fact that the sequence {tn} was chosen to be dense in T, and the continuity of the power function of any test, by Theorem 9 in Lehmann [3], pp. 52-4 (this theorem applies because of our assumptionin (4.2)). The proof is completed. ] In connection with Theorem 4.2 A, it should be mentioned that this result is also stated in Chibisov [1] (see Theorem 1.2, p. 92) but no proof is presented there. Now instead of the family of probability densities in (4.1), suppose that we have the following family [dQfjdQi] = O(t)exp(*'St;)

(veB\ teT),

(4.21)

where S is a positive definite, symmetric matrix. Consider the transformation A = lit, so that £'2 = A'. Then A = Ao if and only if t = t0 (= S~1A0), since S is positive definite. Setting Qs-u = $**> ^ e family (4.21) becomes [dQf*ldQ*] = <7*(A)exp(A'?;)

(veRk, A G T * ) ,

(4.22)

where C*(A) = (7(2-^) and T* = 2T, the image of T under the transformation A = 2£. In connection with the family (4.22), the

238

Appendix

hypothesis testing problem H:t = t0 against A: t 4= t0 becomes H*: A = Ao against A*: A * Ao. By (4.2) and the fact that (4.22) is of the same form as (4.1), it follows that Theorem 4.2 A is still true. Thus we have the following corollary to Theorem 4.2 A. COROLLARY 4.2A For testing the hypothesis H:t = t0 against the alternative A: t #= t0 in the family (4.21), the class of tests of the form (4.16) is essentially complete. R E M A R K 4 . 1 A The significance of Theorem 4.2A (and also Corollary 4.2 A) in the present context is the following. For testing the hypothesis H: t — t0 against the alternative A: t #= t0, if ijr is a test of whatever size but not of the form (4.16), there exists a test of that form with no larger size and no smaller power than that of }[r. Consequently, whatever criterion is proposed in terms of power for specifying a class of tests, it is always possible to restrict ourselves to a subclass of the class of all tests of the form (4.16). The first theorem stated below has been established in R. R. Rao [1] (see Theorem 4.2), where the reader is also referred for its proof. It provides a very useful uniform version of weak convergence of measures. T H E O R E M 4 . 3 A For n = 1,2,..., let ju,n and /JL be measures in Rk such that fi(dC) = 0 for every Cetf, where ^ is the class of all measurable convex sets in Rk. Then / ^ => fi if and only if

In particular, the conclusion is true if /i is absolutely continuous with respect to Lebesgue measure in Rk. We finally formulate the following theorem established by Wald [2] (see Proposition II, pp. 445-8). This theorem may also be shown on the basis of some general results obtained by Lehmann [2]. An explicit proof of it based on invariance considerations has been given by Soms [1]. T H E O R E M 4.4A Let Z be a ^-dimensional random vector distributed as N(d, S) with unknown mean 6 and known nonsingular covariance matrix S. Then for testing the hypothesis

Theorems employed in Chapter 6

239

H: 6 = 60 on the basis of a single observation on each of the components of Z, the critical region given by has uniformly best average power with respect to the surfaces EUt c defined in (1.11) of Chapter 6 and the weight function given in (1.9) of the same chapter. Exercises 1. Referring to the proof of Lemma 4.5 A, show that C^1 f]Sr = Cr;

(i) 00

(ii) G = U Ca is a non-empty, closed and convex set. 2. Referring to the proof of Corollary 4.1A, show that (i) There exists a ^-neighbourhood, N8(A), of A such that NS(A) n C = 0 ; (ii) A <^C% for all sufficiently large v.

Bibliography

[1] [1] [1] [1] [2] [3] [4] [1]

[1] [1] [1] [1] [1] [1] [1]

(referred to in the text) ANDERSON, T. W. An Introduction to Multivariate Analysis. New York (1958). BAHADUR, R. R. On Fisher's bound for asymptotic variances. Ann. Math. Statist. 35 (1964), 1545-52. BICKEL, P. Personal communication. BILLINGSLEY, P. Statistical Inference for Markov Processes. Univ. of Chicago Press (1961). BILLINGSLEY, P. The Lindeberg-Levy theorem for martingales. Proc. Amer. Math. Soc. 12 (1961), 788-92. BILLINGSLEY, P. Ergodic Theory and Information. New York (1965). BILLINGSLEY, P. Convergence of Probability Measures. New York (1968). BIRNBATJM, A. Characterizations of complete classes of tests of some multiparametric hypotheses, with applications to likelihood ratio tests. Ann. Math. Statist. 26 (1955), 21-36. BREIMAN, L. Probability. Reading, Mass. (1968). CHIBISOV, D. M. A theorem on admissible tests and its application to an asymptotic problem of testing hypotheses. Theor. Probability Appl. 12 (1967), 90-103. CRAMER, H. Mathematical Methods of Statistics. Princeton (1946). DOOB, J. L. Stochastic Processes. New York (1953). EGGLESTON, H. G. Convexity. Cambridge University Press (1966). FERGUSON, T. S. Mathematical Statistics. New York (1967). GANSSLER, P. Note on minimum contrast estimates for Markov processes. Mathematisches Institut der Universitdt zu Koln (1969). [ 240 ]

Bibliography

241

[1] HAJEK, J. & SIDAK, Z. Theory of Rank Tests. New York (1967). [2] HAJEK, J. A characterization of limiting distributions of regular estimates. Z. Wahrscheinlichkeitstheorie und Verw. Oebiete, 14 (1970), 323-30. [3] HAJEK, J. Local asymptotic minimax and admissibility in estimation. To appear in the Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability. [1] IBRAGIMOV, I. A. A central limit theorem for a class of dependent random variables. Theor. Probability Appl. 8 (1963), 83-9. [1] JOHNSON, R. A. & ROTJSSAS, G. G. Asymptotically most powerful tests in Markov processes. Ann. Math. Statist. 40 (1969), 1207-15. [2] JOHNSON, R. A. & ROUSSAS, G. G. Asymptotically optimal tests in Markov processes. Ann. Math. Statist. 41 (1970), 918-38. [1] KAUFMAN, S. Asymptotic efficiency of the maximum likelihood estimator. Ann. Inst. Statist. Math. 18 (1966), 155-78. [1] KENDALL, M. G. & STUART, A. The Advanced Theory of Statistics. 2nd ed. New York (1967). [1] KRAFT, C. Some conditions for consistency and uniform consistency of statistical procedures. Univ. California Publ. Statist. 2 (1955), 125-241. [1] LECAM, L. On some asymptotic properties of the maximum likelihood estimates and related Bayes' estimates. Univ. California Publ. Statist. 1 (1953), 277-330. [2] LECAM, L. On the asymptotic theory of estimation and testing hypothesis. Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1 (1956), 129-56. [3] LECAM, L. Les proprietes asymptotiques des solutions de Bayes. Publ. Inst. Statist. Univ. Paris, 7 (1958), 17-35. [4] LECAM, L. Locally asymptotically normal families of distributions. Univ. California Publ. Statist. 3 (1960), 37-98.

242

Bibliography

[5] LECAM, L. Likelihood functions for large numbers of independent observations. In Research Papers in Statistics. (F. N. David, editor.) New York (1966). [6] LCCAM, L. On the assumptions used to prove asymptotic normality of maximum likelihood estimates. Ann. Math. Statist 41 (1970), 802-28. [7] LECAM, L. Limits of experiments and a theorem of J. Hajek. To appear in the Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability. [8] LECAM, L. Theorie asymptotic de la decision Statistique. University of Montreal Press, Montreal (1970). [1] LEHMANN, E. L. Some comments on large sample tests. Proceedings of the First Berkeley Symposium on Mathematical Statistics and Probability (1949), 451-7. [2] LEHMANN, E. L. Optimum invariant tests. Ann. Math. Statist. 30 (1959), 881-4. [3] LEHMANN, E. L. Testing Statistical Hypotheses. New York (1959). [1] LIND, B. & ROUSSAS, G. G. A remark on quadratic mean differentiability. Technical Report no. 272(1911). University of Wisconsin, Madison. [1] LOEVE, M. Probability Theory. 3rd ed. Princeton (1963). [1] MATHES, T. K. & TRUAX, D. R. Tests of composite hypotheses for the multivariate exponential family. Ann. Math. Statist. 38 (1967), 681-97. [1] NEVETJ, J. Mathematical Foundations of the Calculus of Probability. San Francisco (1965). [1] NEYMAN, J. Optimal asymptotic tests of composite statistical hypotheses. In Probability and Statistics. (Ulf Grenander, editor.) New York (1959). [1] PFANZAGL, J. On the asymptotic efficiency of median unbiased estimates. Ann. Math. Statist. 41 (1970), 1550-9. [1] PHILIPPOTT, A. Tests of hypotheses for the logarithmic, logistic and Cauchy distributions. Master's essay (1971). University of Wisconsin, Madison. [1] PRATT, J. W. On interchanging limits and integrals. Ann. Math. Statist. 31 (1960), 74-7.

Bibliography

243

[1] RAO, C. R. Large sample tests of statistical hypotheses concerning several parameters with applications to problems of estimation. Proc. Cambridge Philos. Soc. 44 (1948), 50-7. [2] RAO, C. R. Criterion for estimation in large samples. Sanhhyd Ser. A, 25 (1963), 189-206. [3] RAO, C. R. Linear Statistical Inference and Its Applications. New York (1965). [1] RAO, R. R. Relations between weak and uniform convergence of measures with applications. Ann. Math. Statist. 33 (1962), 659-80. [1] ROUSSAS, G. G. Asymptotic inference in Markov processes. Ann. Math. Statist. 36 (1965), 978-92. [2] ROUSSAS, G. G. Asymptotic inference in Markov processes, II. Bull. Soc. Math. Grece, 6 (1965), 37-50. [3] ROUSSAS, G. G. Extension to Markov processes of a result by A. Wald about the consistency of the maximum likelihood estimate. Z. Wahrscheinlichkeitstherorie und Verw. Gebiete, 4 (1965), 69-73. [4] ROUSSAS, G. G. Asymptotic normality of the maximum likelihood estimate in Markov processes. Metrika, 14 (1968), 62-70. [5] ROUSSAS, G. G. Some applications of the asymptotic distribution of the likelihood functions to the asymptotic efficiency of estimates. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 10 (1968), 252-60. [6] ROUSSAS, G. G. On the concept of contiguity and related theorems. Aarhus Universitet, Preprint Series (1968-9), no. 33. [7] ROUSSAS, G. G. The usage of the concept of contiguity in discussing some statistical problems. Aarhus Universitet, Preprint Series (1968-9), no. 34. [8] ROUSSAS, G. G. & SOMS, A. A remark on some characterizations of contiguity. Technical Report no. 255 (1970). University of Wisconsin, Madison. [9] ROUSSAS, G. G. & SOMS, A. On the exponential approximation of a family of probability measures and a representation theorem of Hajek. Technical Report no. 234 (1971). University of Wisconsin, Madison.

244

Bibliography

[1] SCHMETTERER, L. On the asymptotic efficiency of estimates. In Research Papers in Statistics. (F. N. David, editor.) New York (1966). [1] SOMS, A. Personal communication. [1] WALD, A. Asymptotically most powerful tests of statistical hypotheses. Ann. Math. Statist. 12 (1941), 1-19. [2] WALD, A. Tests of statistical hypotheses concerning several parameters when the number of observations is large. Trans. Amer. Math. Soc. (1943), 426-82. [1] WEISS, L. & WOLFOWITZ, J. Generalized maximum likelihood estimators. Theor. Probability Appl. 11 (1966), 58-81. [2] WEISS, L. & WOLFOWITZ, J. Maximum probability estimators. Ann. Inst. Statist. Math. 19 (1967), 193-206. [1] WOLFOWITZ, J. Asymptotic efficiency of the maximum likelihood estimator. Theor. Probability Appl. 10 (1965), 247-60.

Some other references of related interest

[1] BARTOO, J. B. & PURI, P. S. On optimal asymptotic tests of composite hypotheses. Ann. Math. Statist. 38 (1967), 1845-52. [1] BtiHLER, W. J. & PURI, P. S. On optimal asymptotic tests of composite hypotheses with several constraints. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 5 (1966), 71-88. [1] INAGAKI, NOBUO. On the limiting distribution of a sequence of estimators with uniformity property. Ann. Inst. Statist. Math. 22 (1970), 1-13. [1] LIND, B. Quadratic mean differentiability and some hypotheses testing problems for Markov processes. Ph.D. Thesis (1972), University of Wisconsin, Madison.

Bibliography

245

[2] LIND, B. & ROTTSSAS, G. G. A remark on quadratic mean differentiability. Technical Report no. 272. University of Wisconsin, Madison. Also Ann. Math. Statist. 42 (1971) (Abstract no. 71T-88) and Ann. Math. Statist. 43 (1972). [3] LIND, B. & ROUSSAS, G. G. Cramer-type conditions and quadratic mean differentiability. Technical Report no. 273. University of Wisconsin, Madison. Also Ann. Math. Statist. 42 (1971) (Abstract no. 71T-87). [1] MICHEL, R. & PFANZAGL, J. Asymptotic normality. Mathematisches Institut der Universitdt zu Koln (1970). [1] NOCTURNE, D. Asymptotic efficiency of the maximum likelihood estimators for the parameters of certain stochastic processes. Ph.D Thesis, Technical Report no. 105, Department of Operations Research, Cornell University. [1] PHILIPPOU, A. & ROUSSAS, G. G. Asymptotic distribution of the likelihood function in the independent not identically distributed case. Technical Report no. 291. University of Wisconsin, Madison. Also Ann. Math. Statist. 42 (1971) (Abstract no. 71T-85). [2] PHILIPPOU, A. & ROUSSAS, G. G. Exponential approximation and asymptotically optimal tests in the independent not identically distributed case. Technical Report no. 305. University of Wisconsin, Madison. [1] ROUSSAS, G. G. Multi-parameter asymptotically optimal tests for Markov processes. Aarhus Universitet, Preprint Series (1968-9), no. 42. [1] WEISS, L. & WOLFOWITZ, J. Generalized maximum likelihood estimators in a particular case. Theor. Probability Appl. 13 (1968), 622-7. [2] WEISS, L. & WOLFOWITZ, J. Asymptotically minimax tests of composite hypotheses. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 14 (1969), 161-8. [3] WEISS, L. & WOLFOWITZ, J. Maximum probability estimators and asymptotic sufficiency. Ann. Inst. Statist. Math. 22 (1970), 225-44.

Index

action space, 226 asymptotically locally most powerful Anderson, T W., 94 tests, 104, 105 application of asymptotically locally most powerful unbiased tests, 121, 126, Berry-Esseen theorem, 156 127 composition theorem, 138 dominated convergence theorem, asymptotically most stringent tests, 191 166, 200, 204 asymptotically unbiased tests, 108 ergodic theorem, 55, 61 Fatou's lemma, 160 asymptotically uniformly most powerFubini's theorem, 228 ful tests, 99 Helly-Bray lemma, 75, 110 asymptotically uniformly most powerHelly-Bray theorem, 3, 108 ful unbiased tests, 108 Holder inequality, 59, 70, 205 Kolmogorov inequality, 62 Bahadur, R. R., 129, 133, 157, 159, -Lr-convergence theorem, 76 160 Minkowski inequality, 63, 134 Bayes risk, 227, 228 Neyman-Pearson fundamental Bayes test, 227, 231, 233, 236 lemma, 30, 101, 103, 150, 154, Bickel, P., 136 155, 163 Billingsley, P., 3, 157, 199, 202, 203, 205, 223 normal convergence criterion, 92 Polya's lemma, 74, 88, 89, 92, 97, Birnbaum, A., 228 Blaschke selection theorem, 232 115, 122, 159 Breiman, L., 206, 207, 210, 212, 223 Scheffe's lemma, 204 separating hyperplane theorem, 236 central limit theorem for martingales, Slutsky theorems, 65 205 Tulcea theorem, 162 weak compactness theorem, 136, characterization of contiguity, 8, 11, 17, 31, 32, 33 231, 235 relative compactness, 6 assumptions, 45, 46, 87, 197 tightness, 6 asymptotic distribution, 35, 41, 42, weak convergence, 3, 199 53, 54, 85 Chibisov, D. M., 176, 237 asymptotic efficiency, 128 asymptotic efficiency, classical ap- classical efficiency, the multiparameter case, 160 proach, 157, 158, 160 asymptotic efficiency in Wolfowitz contiguity characterization of, 8, 11, 17, 31, sense (W-efficiency), 128, 130, 32, 33 131, 141, 147, 158 consequences of, 33, 37 asymptotic level of significance, 108, definition of, 7 116, 170, 171, 172, 188, 189, transitivity of, 8, 25 191 continuity set, 199 asymptotic normality of, 52 asymptotic squared error, lower continuous convergence, 131-2 Cramer, H., 156 bound for, 159 [246]

247

Index derivative in q.m., 44, 45, 46, 49 differentiability in q.m., 43, 44 differentially (asymptotically) equivalent, 80 differentially (asymptotically) sufficient, 81 Doob, J. L., 43, 45, 46, 50 double exponential density, 49 Eggleston, H. G., 232 envelope power function, 190, 191 essentially complete class, 234, 236, 238 examples, 9, 11, 47, 48, 49, 50, 87, 88, 90, 92, 106, 107 exponential approximation, 68, 116, 123 exponential family, 42, 53, 67, 68, 69, 83, 84, 116

Z^-norm, 6 loss function, 226 main theorems, proof of, 63 Markov case, 105, 107 Markov processes, 41, 43, 50, 53, 86 Markov property, 60 Mathes, T. K., 232 maximum likelihood estimate, 42, 105, 107, 128, 130, 157 median-unbiasedness, 154 metrically transitive, 50 minimax test, 227, 235-6 most stringent test, 191 multiparameter asymptotically optimal tests, 167 Neveu, J., 162 non-local alternatives, 196

Ferguson, T. S., 121, 236 Fisher's information matrix (number), 42, 128, 130, 155.

Pfanzagl, J., 131, 148, 154 Philippou, A., 41 Pratt, J. W., 133, 134

Ganssler, P., 157 Gaussian process, 50

Rao, C. R., 65, 69, 74, 88, 107, 157, 161, 185, 224 Rao, R. R., 238 relation of absolute continuity and contiguity, 9, 10 continuous and uniform convergence, 133 convergence in i 1 -norm and contiguity, 8, 9, 12 relative compactness and contiguity, 11, 13. 14, 16, 17, 31, 33 relative compactness and tightness, 4, 202 tightness and contiguity, 10, 12, 16, 31, 33 W-efficiency and classical efficiency, 158 relative compactness, 4 risk (global), 226, 227 risk, set, 235 Roussas, G. G., 33, 41, 47, 68, 105, 131, 136, 145, 157, 160, 166

Hajek, J. A., 39, 68, 136, 157 Hausdorff distance, 232 Hodges, J. L., Jr, 129 Ibragimov, I. A., 205 Johnson, R. A., 47, 68, 105 Kaufman, S., 131 Kendall, M. G., 100 Krafft, O., 100 Kraft, C, 39 LeCam, L., 1, 33, 39, 41, 68, 98, 129, 136, 157 Lehmann, E. L., 100, 119, 125, 229, 231, 237, 238 Lind, B., 41 Loeve, M., 3, 42, 60, 62, 63, 65, 74, 75, 76, 88, 92, 108, 110, 112, 133, 138, 156, 166, 204 log-likelihood asymptotic distribution of, 41, 42, 53, 54, 85 asymptotic expansion of, 41, 52, 53, 85

Schmetterer, L., 131, 135, 141, 145 gidak, Z., 39 singular part of, 10 Soms, A., 33, 136, 238

Index

248 truncated version(s) of, 67, 70

stationarity, 55, 63 statistical applications, 2, 33, 35, 54, 84, 85 statistical interpretation, 29 Stuart, A., 100 superefficient estimate, 129

uniformly integrable, 74 uniformly most powerful tests, 106

testing hypothesis (es), 30, 31, 81, 82, 86, 96, 99, 103,107, 123,170,171, 172, 191, 197, 226 tightness, 4 Truax, D. R., 232

Wald, A., 99, 105, 108, 150, 168, 238 weak convergence, 2 weight function, 168 Weiss, L., 157 Wolfowitz, J., 128, 129, 130, 131, 157

Vitali's theorem, 204

Divisors (Cambridge Tracts in Mathematics)

Read more

Convexity (Cambridge Tracts in Mathematics)

Read more

Divisors (Cambridge Tracts in Mathematics)

Read more

Random Variables and Probability Distributions (Cambridge Tracts in Mathematics)

Read more

Sets of Multiples (Cambridge Tracts in Mathematics)

Read more

Consequences of Martin's Axiom (Cambridge Tracts in Mathematics No.84)

Read more

The Lebesgue Integral (Cambridge Tracts in Mathematics)

Read more

The Lévy Laplacian (Cambridge Tracts in Mathematics)

Read more

Hopf Algebras (Cambridge Tracts in Mathematics)

Read more

Simple Noetherian Rings (Cambridge Tracts in Mathematics)

Read more

Quasi-Frobenius Rings (Cambridge Tracts in Mathematics)

Read more

The Lebesgue Integral (Cambridge Tracts in Mathematics)

Read more

Simple Noetherian Rings (Cambridge Tracts in Mathematics)

Read more

Proximity Spaces (Cambridge Tracts in Mathematics 59)

Read more

Forcing Idealized (Cambridge Tracts in Mathematics)

Read more

Totally Positive Matrices (Cambridge Tracts in Mathematics)

Read more

Multiple Forcing (Cambridge Tracts in Mathematics 88)

Read more

Totally Positive Matrices (Cambridge Tracts in Mathematics)

Read more

Forcing Idealized (Cambridge Tracts in Mathematics)

Read more

Asymptotic Expansions (Cambridge Tracts in Mathematics)

Read more

Fixed Point Theorems (Cambridge Tracts in Mathematics)

Read more

Bipartite Graphs and their Applications (Cambridge Tracts in Mathematics)

Read more

Fixed Point Theory and Applications (Cambridge Tracts in Mathematics)

Read more

Fixed Point Theory and Applications (Cambridge Tracts in Mathematics)

Read more

Fixed Point Theory and Applications (Cambridge Tracts in Mathematics)

Read more

Some Applications of Mechanics to Mathematics (Popular Lectures in Mathematics)

Read more

Fourier Integrals in Classical Analysis (Cambridge Tracts in Mathematics)

Read more

Fourier Integrals in Classical Analysis (Cambridge Tracts in Mathematics)

Read more

Jordan Structures in Geometry and Analysis (Cambridge Tracts in Mathematics)

Read more

Convergence of probability measures

Read more

Recommend Documents

Divisors (Cambridge Tracts in Mathematics)

CAMBRIDGE TRACTS IN MATHEMATICS General Editors B. BOLLOBAS, H. HALBERSTAM & C.T.C. WALL 90 Divisors RICHARD R. HA...

Convexity (Cambridge Tracts in Mathematics)

Divisors (Cambridge Tracts in Mathematics)

CAMBRIDGE TRACTS IN MATHEMATICS General Editors B. BOLLOBAS, H. HALBERSTAM & C.T.C. WALL 90 Divisors RICHARD R. HAL...

Random Variables and Probability Distributions (Cambridge Tracts in Mathematics)

Sets of Multiples (Cambridge Tracts in Mathematics)

CAMBRIDGE TRACTS IN MATHEMATICS General Editors B. BOLLOBAS, P. SARNAK, C.T.C. WALL 118 Sets of Multiples Richard R...

Consequences of Martin's Axiom (Cambridge Tracts in Mathematics No.84)

The Lebesgue Integral (Cambridge Tracts in Mathematics)

THE LEBESGUE INTEGRAL BY J. C. BURKILL FellOfD of P�ttrhcnu�, Cambridge CAMBR IDGE AT THE UNIVERS ITY PRESS 1963 ...

The Lévy Laplacian (Cambridge Tracts in Mathematics)

CAMBRIDGE TRACTS IN MATHEMATICS General Editors b. bollobas, w. fulton, a. katok, f. kirwan, p. sarnak, b. simon, b. to...

Hopf Algebras (Cambridge Tracts in Mathematics)

...

Simple Noetherian Rings (Cambridge Tracts in Mathematics)