This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
S}).t (2.49) fE^
J JE 9
For every positive 8 the right side goes to zero as €,0. Therefore, by Theorem 2.4, is a Q-uniformity class. Necessity of (i) also follows immediately from Theorem 2.4. To prove the necessity of (ii), assume that there exist a positive number S and a point x o in S such that sup c j (x o :e)>S
foralle>0.
fE9
This implies the existence of a sequence {x„) of points in S converging to x o and a sequence of functions (f„) c F such that IL(xn)—f.('XO)I>2
(n=1,2,...).
Let Q = Sxo and Q„ = Sx. (n =1, 2, ... ). Clearly, (Q} converges weakly to Q, but
f f.dQ,^ Hence
—
f.dQl =If^(x,^)—f,,(xo)l>2
(n=1,2,...).
is not a Q-uniformity class. Q.E.D.
We have previously remarked that the weak topology on the set Vii' of all probability measures on a separable metric space is metrizable, and the tThe set (x: sup(wj (x : c) : f E f) and, therefore, measurable.
>6) =
u ((x : wj (x : c) > 8):f E i} is open (see Section 11)
Uniformity Classes
17
Prokhorov distance dp metrizes it. Another interesting metrization is provided by the next corollary. For every pair of positive numbers c. d, define a class L(c, d) of Lipschitzian functions on S by L(c,d) = { f : wf (S) < c, I f (x) — f (y)l < dp(x,y) for all x,y ES). (2.50)
Now define the bounded Lipschitzian distance d BL by dBL (Q1 Q2) = sup fdQ, — f fdQ2I fEL(1,1) I
(Q1.Q2E''P). (2.51)
,
COROLLARY 2.8. If S is a separable metric space, then d BL metrizes the weak topology on Proof. By Corollary 2.7, L(l, I) is a Q-uniformity class for every probabil-
ity measure Q on S. Hence if (Q n ) is a sequence *of probability measures converging weakly to Q, then lira dBL (Qn,Q)=0.
(2.52)
n
Conversely, suppose (2.52) holds. We shall show that (Q n ) converges weakly to Q. It follows from (2.52) that lim l f fdQn — f fdQl =0
for every bounded Lipschitzian function f. (2.53)
For, if f (x) — f (y)^ < dp(x,y) for all x,y E S, f fdQ,,— f fdQ=c( f f'dQn
—
f f'dQ),
where c = max {wf (S),d) and f' = f/c E L(1, 1). Let F be any nonempty closed subset of S. We now prove lim Qn (F)
(2.54)
For E >0 define the real-valued function fE on S by ff (x)=^(e
-
'p(x,F))
(x ES),
(2.55)
18
Weak Convergence and Uniformity Classes
where 4, is defined on [0, oo) by
(t) __1
.
1—t if 0 < t < 1, if 1>1. 0
(2.56)
Note that f is, for every positive e, a bounded Lipschitzian function satisfying wf(S)< 1, If,(X) f^(Y)I<) EP(x+F) — - p(Y,F)I —
E
P(x , Y)
(x Y E S ), ,
so that, by (2.53), linm f f^ dQ„ = f f, dQ Since IF
(e>0).
(2.57)
f, for every positive e, linm Q,, (F) < lim f f dQ„ = f f dQ
(e>0). (2.58)
Also, lim f^(x)= IF (x) for all x in S. Hence
lim JJdQ=Q(F).
(2.59)
cj0
By Theorem 1.1 { Q) converges weakly to Q. Finally, it is easy to check that dBL is a distance function on 'P. Q.E.D. Remark. The distance daL may be defined on the class GR, of all finite signed measures on S by letting Q t . Q 2 be finite signed measures in (2.51). The function µ—.d, L (s,0) is a norm on the vector space 9R.. The topology induced by this norm is, in general, weaker than the one induced by the variation norm (1.5). It should also be pointed out that the proof of Corollary 2.8 presupposes metrizability of the weak topology on 9 and merely provides a suitable metric as an alternative to the Prokhorov distance do defined by (1.16)]. This justifies the use of sequences (rather than nets) in the proof.
One can construct many interesting examples of uniformity classes beyond those provided by Corollaries 2.7 and 2.8. We give one example now. The rest of this section will be devoted to another example (Theorem 2.11) of considerable interest from our point of view.
Uniformity Classes
19
Example Let S=R 2 . Let 6! (1) be the class of all Borel-measurable subsets of R 2 each having a boundary contained in some rectifiable curvet of length not exceeding a given positive number 1. We now show that 6D (I) is a Q-uniformity class for every probability measure Q that is absolutely continuous with respect to the Lebesgue measure A 2 on R 2 . Let A E 6 (l) and let 3A C J, where J is a rectifiable curve of length 1. There exist k points z o,z t ,.••,z k -, on J such that (i) z o and z k _ t are the end points of J (may coincide), (ii) k
(L +2)1T(2E)
2
=41rk+87rE 2 [A E (l )].
(2.60)
Let Q be a probability measure that is absolutely continuous with respect to A 2 . Then (2.60) implies (in view of Theorem A.3.1) lim (sup{Q((aA):AE^'i)(1)})=0.
(2.61)
By Corollary 2.6, -D (1) is a Q-uniformity class. We need some preparation before proving the next theorem. Let S be a metric space. Define the Hausdorff distance A between two closed bounded subsets A, B of S by
A(A,B)=inf{ E:c>0 ,AcB`, BCA`}.
(2.62)
The class Q of all closed bounded subsets of S is a metric space with metric A. LEMMA 2.9. Let ')1t be a compact subset of (? . Then for every probability measure Q on S one has lim sup{Q(M`):ME cjo
)=sup{Q(M):MElt).
(2.63)
Proof Let rq be a given positive number. For every M E )lt. there exists a positive number E, t, such that
Q(M`")
,
to rectifiable curve in R 2 is a subset of R 2 of the form {z(t):0G t < 1), where t- z(t) =(x(t),y(t)) is a continuous function of bounded variation on [0,11 into R 2 ; z(0), z(1) are called the endpoints of the curve.
20 Weak Convergence and Uniformity Classes
since M`J.M as 40. By compactness of Olt, there exists a finite collection of sets {M 1 , ..., M,) in OL such that every set in OJlt, is within a A-distance from M for some i, 1
EM;
:1
{
Then sup(Q(M`"):MEOL) <sup{Q(M,`°Mi 2 ):1
Q.E.D. A subset C of Rk is convex if a x, + (1 — a) x2 E C for all x,, x2 E C and all a E [0,1]. A hyperplane H in R' is a set of the form
H={x:=c},
(2.64)
where c is a real number and u is a unit vector of R k ; that is, Ilul) = 1, and ,> denotes euclidean inner product
<
k
=
u,X,,
(2.65)
i=! k
1/2
u?
(lull
[u=(u1,...,uk), x=(x 1 ,...,xk )ER k ].
i^l
A closed half space E is a set of the form E=(x:xER k ,
{cER', uER k , IIull=l]. (2.66)
A hyperplane H= (y : = c) is said to be a supporting hyperplane for a set A at x E A if =c,
AC{z:
(2.67)
that is, if the hyperplane H passes through the point x of A and has A on one side. Note that the sign < in (2.66) and (2.67) may be replaced by >
Uniformity Classes
21
(by merely changing u to — u). It is a well-known fact that if A is a compact convex set, then there exists a supporting hyperplane for A at each x E aA .t LEMMA 2.10. Let A, B be two closed, bounded, convex subsets of R k . Then A(A,B)=A(aA,aB).
(2.68)
Proof. Suppose A(8A,aB) =E. Let E' be any number larger than E. Let There exist xr, x 2 E aA and a E [0,11 such that x = ax I +(1 — a)x 2 , since the intersection of A with any line through x is a closed line segment whose end points (x,,x 2 , for example) lie on 8A. Let y,, y 2 E aB be such that xi —y ; < E' for i= 1, 2. Then letting y = ay, +(l — a)y 2 yields y E B and
x E A.
llx yl! <«Ilx1 yell+(I —
—
—
a)IIx2 — y2fl <E'.
Thus A c B` . Similarly B CA '. Since this is true for every E > E, 0(A, B) < A(aA, aB
).
(2.69)
To prove the opposite inequality, suppose that E > t1(A, B) and let be any positive number. Let x E aA A. Let (z : (1, z> = c) be a supporting hyperplane for A at x. Then the half space H = { z : (l, z> c c + E) contains A. If z E A and 11 z' — z l l< E, then
(l,z'>=(l,z>+(l,z'—z>+IIz'—z!I c+ e. Hence H J A D B. The ball B (x : E + il) intersects R k \ H. This is because the point x +(E +71/2)1 of this ball satisfies (l,X+(E+
2 )l>=( l,x>+(
E+ 2) =C +E+
2
,
and therefore lies in the complement of H. It follows that B (x : E + rl) intersects R k \ B. But, since A C B ` and x E A, B (x : E + rl) certainly intersects B. It follows that B (x : E + rl) intersects aB, so that (aB)"'' D 3A. Similarly (8A)` + ' j B. Therefore A(aA,aB) <E +rl tEggleston II), p. 20.
22
Weak Convergence and Uniformity Classes
for every c > 0(A, B) and every positive rt. Hence 0(aA,aB) <0(A,B).
(2.70)
The inequalities (2.69) and (2.70) together yield (2.68). Q.E.D. THEOREM 1.11. Let C denote the class of all Bore/-measurable convex subsets of R k . Let Q be a probability measure on R k . The class e is a Q-uniformity class if and only if it is a Q-continuity class, that is, if and only if Q(ac)=o
forallCEe.
(2.71)
Proof If e is a Q-uniformity class, then, by relation (2.41) in Corollary 2.6, e is a Q-continuity class. Suppose, conversely, that (2.71) holds. Since the characterization (2.41) of Q-uniformity is in terms of boundaries, and since aC= a (Cl(C)) for every convex set [note that C1(C)= Int(C)u aC and that aC has empty interior for convex C], it follows that C is a Q-uniformity class if and only if e = (C 1(C) : CE C) is. Given ri >0, let r be so chosen that
Q({x:IIxII>r))<2.
(2.72)
Write C', = { C: C E e, CCC1(B (0: r))). By a well-known theorem of Blaschket 3, is compact in the Hausdorff metric A defined by (2.62). Lemma 2.10 shows that the compactness of , is equivalent to the compactness of { aC : C E 13, }. Lemma 2.9 and the hypothesis (2.71) now yield (2.73) lim sup{ Q((8C)`):CEC',}=0. By Corollary 2.6, C, is a Q-uniformity class. Let (Qn ) be a sequence of probability measures weakly converging to Q. Then lim (sup{IQn(C) n
—
Q(C)I:CEC^))
<11im (sup{IQ.(C)—Q(C)^:CE,}) + lim Qn({x:llxll>r))+Q({x:llxll>r)) n =2Q({x:fIxll >r})
(2.74)
Integrals Over Convex Shells
23
Note that the last equality follows from the fact that (x : Ilxil > r) is a Q-continuity set, since its complement is [although, given any probability measure Q and a positive rl, one can always find r such that (x: jlxjl > r) is a Q-continuity set and (2.72) holds]. Since rl is an arbitrary positive number, it follows from (2.74) that is a Q-uniformity class. Consequently, (2 is a Q-uniformity class. Q.E.D.
a
Remark. It follows from the above theorem that if Q is absolutely continuous with respect to Lebesgue measure on R", then is a Q-uniformity class. In particular, is a 4)uniformity class, 4) being the standard normal distribution in R k . We shall obtain a refinement of this last statement in the next section. It is easy to see from the proof of Theorem 2.11 that the class C in its statement may be replaced by any class Ef of Borel-measurable convex sets with the property that
e
a
R,- (CI(A)nCl(B(0:r)):AE($} is compact in the Hausdorff metric for all positive r. In particular, by letting d — {(—oo,x1]X(—oo,x2]X ... X( — oo,xk]:x—(xi,...,Xk)ER k }, we get Polya's result: Let P be a probability measure on R k whose distribution function F, defined by F(x)-P((-oo,x,]x... x(-oo,xk ]) is continuous on Rk. If a sequence converges weakly to P, then
[x-(x......xk)ERk ],
(2.75)
of probability measures {P„} with distribution functions {F
sup(IF„(x)-F(x)I:xER k }-+0
A)
(2.76)
as n-boo. The left side of (2.76) is sometimes called the Kolmogorov distance between P o and P. The converse of this result is also true: if (2.76) holds, then (P„) converges weakly to P. In fact, if (F„} converges to Fat all points of continuity of F, then (P„) converges weakly to P.t
3. INEQUALITIES FOR INTEGRALS OVER CONVEX SHELLS
It is not difficult to check that if C is convex then so are Int(C),C1(C). The convex hull c(B) of a subset B of R k is the intersection of all convex sets containing B. Clearly c(B) is convex. If C is a closed and bounded convex subset of Rk, then it is the convex hull of its boundary; that is,
c(8C)=C. Clearly c(8C) C C. On the other hand, if x E C, x li= 8C, then every line through x intersects aC at two points and x is a convex combination of these two points. Thus c(aC) = C. If C is convex and e >0, then C` is tSee Billingsley [ 1 1, pp. 17-18.
Weak Convergence and Uniformity Classes
24
convex and open, and C -' is convex and closed. The main theorem of this section is the following: THEOREM 3.1. Let g be a nonnegative differentiable function on [0, oo)
such that (i) b= f Ig'(t)It k- 'dt
(ii) lim g(t)=0. r--► oo
Then for every convex subset C of R k and every pair of positive numbers e, p,
C'\C v
(3.1)
g(JI xI1)dx < bak(E+p)
-
where
_ ak
k7l
2^7k/2
k/2
(3.2)
f((k+2)/2) I'(k/2) '
is the surface area of the unit sphere in R k .
COROLLARY 3.2. Let s > 0, k > 1, and f(x)=(21T)-k/2IIxIIsexp j —
11x112 1
(xERk )
Then for all convex subsets C of R k and every pair of positive numbers e, p,
f
C.\C
r((k+s-1 )/2)
c: '/2-+k-1) (2s f(x)dx<2 -,
r(k/2)
((+p). (3.3)
Proof. Here one takes (in Theorem 3.1) g(t)=(217) -k / 2 t'exp(-1 2 /2)
1E[0,00).
Then g,(t)=(2r)-k/2(st$-I— is+ I )exp (— 2 1, 00
( )—k/2
b < 2^r f o
( k+s-2
st
p r j—22 }l di + t k+s) ex J l
Integrals Over Convex Shells =(2.r)-k/2(s,2(k+s-3)/zr(
k+s— 1 )+21c+3_/2r( k+s+ 1
2
=(2^r) - k/2 2 (k+s-3)/2 (2s+k-1) r(
=2 ( s -3) / 2 (2s+k— I)
-k / 2
25
r(
2
k+s 2
-
1
k+s-1 )( 3.4)
which gives (3.3) on substitution in (3.1). Q.E.D. The rest of this section is devoted to the development of the material needed for the proof of Theorem 3.1. For k=1, C` \C -° is contained in the union of two disjoint intervals each of length e+ p. Hence
f
g(IxI)dx 2(e +)( sup g(x)) <2(e+p) f
C`\C ^ -
x>0
'Q
I g'(t)I dt=2(e+p)b
0
=ba 1 (e+p).
(3.5)
For k >. I a more intricate argument is needed. For the rest of the section we assume k> 1. A polyhedron is a closed, bounded, convex set with nonempty interior that is the intersection of a finite number of closed half spaces. If P is a polyhedron, a face of P is a set of the form H n aP, where H is a hyperplane such that H n aP has nonempty interior in H. LEMMA 3.3. Let a polyhedron P be given by P=(x:
(3.6)
where ui 's are distinct unit vectors. Let Li= (x:xEP, =d)
(I < j<m).
(3.7)
Then a P = U L^. If F is a face of P, then F = L^ for some j. Moreover, P= (x : (u^,x> < d for all j for which LL is a face of P }.
(3.8)
Proof. The first assertion is obvious. If F= H n 8P= u(H n L^) is a face of P, then H = (x :
26
Weak Convergence and Uniformity Classes
Lj = Hj n aP. It is clear that the interior of L1 in H^ is (x:=d ),
so that if Lj is not a face of P, then Lj C U L,. This implies 8P C 8Q, r#j
where Q is defined by Q= { x :
Clearly P c Q. It is sufficient to show that aP c aQ implies P= Q. Let x t E Int (P). Assume there exists x 2 E aQ \ aP. Consider the line segment [x 1 ,x 2 1 joining x, and x 2 . This line segment meets aP at x 3 , say. Clearly, [x 1 ,x 2 )CInt(Q) and x 3 x 2 , so that x 3 EInt(Q)naP, which contradicts the fact aP c Q. Q.E.D. LEMMA 3.4. A polyhedron P has a finite number of faces. if F 1 ,.. .,F m are the faces of P, then a P = U F. Moreover, there exist unique unit vectors u^ and constants d3 such that F, = (x : x E P, = d d ), and P C (x : < di ). Also, P then has the representation P=(x:
There are two main steps in the proof of Theorem 3.1. The first one (Lemma 3.9) is to express the surface integral as a derivative of volume in volume integrals. The second step (Lemma 3.10) is to get a uniform bound for the surface integral of a fixed function over the boundary of a polyhedron. The ideas involved here belong naturally to the domain of surface area and surface integrals. In the following paragraphs we develop the material needed for the proofs. The development here is self-contained except for the use of Cauchy's formula, which is stated but not proved. We begin with some notation. Let A k denote the Lebesgue measure on R k normalized by the euclidean distance on R k ; that is, if y 1 , ... ,y k is an tSee Eggleston 111, pp. 29-30.
Integrals Over Convex Shells
27
orthonormal basis and A is a cube with respect to them or A = { I ty ; : a; < t; < b; for all i}, then X.(A)=(b t — a t )• • • (bk — ak ). We also refer to Ak as the k-dimensional Lebesgue measure on R'. On any hyperplane of R' there is a (k — 1)-dimensional Lebesgue measure normalized the same way. We denote this measure as Ak _ 1 and call it the (k— 1)-dimensional Lebesgue measure. For example, if H is a hyperplane, it can be written in the form H=x o +Ry 1 +---+Ryk _ 1 , where x0,y1,...,yk-1EH and y,,•.., yk _ I are (k—I) orthonormal vectors in R'. If f is a function with compact support in R k , then
f
H fdXk-1=
f f(xo+t1Y1+
...
+tk-1Yk-1)dt,...dtk_1.
(3.10)
Next we denote a k _ l as the surface area measure on the unit sphere Sk _ . One can write an explicit formula for 0k- 1 by using Eulerian angles to parametrize points of Sk _ 1 . It then follows that o k _ I (Sk _ 1 )=a k , where a, is given by (3.2). Let P be a polyhedron with faces F, (1 < i < m). Then the surface area of P is defined as Ak-1(F;).
Ak_1(8P)=
(3.11)
;=t
For a bounded Borel-measurable function f on R k we define the surface integral off on a P by
faP fdXk-1= ^, f fd^k-1
(3.12)
and if A is a Borel subset of aP, then the surface area of A is defined by Ak-1(A)=
Ak-1(AnF;).
(3.13)
Remark. Let P be given as P={x:<4., I <j< m}. Then
f
fda k _ 1 =
f fdAk_1. (3.14)
aP I<j<m L,
where LL -(x:xEP,=d1 ). This follows from the fact that A1(L)0 if L^ is not a
face of P.
28
Weak Convergence and Uniformity Classes
CAUCHY'S FORMULA -Let C be a polyhedron. For each unit vector u E R k , let Ak _ I (C: u) denote the X k _ 1 -measure of the orthogonal projection of C on the hyperplane (x: = 0). Then xk-I(aC) = $k_I f Ak-I(C:u)ak-I(du) (3.15) ,
where 13k1 = ak_ 1/(k —1) is the volume (Lebesgue measure in R k- I ) of the unit ball in Rk - I
COROLLARY 1. Let P and Q be polyhedra such that Pc Q. Then xk - I (8P)
(3.16)
Proof. This follows from Cauchy's formula if it is noted that Xk _ I (P : u) < Xk _ I (Q : u) for any u. Q.E.D.
COROLLARY 2. If P is a polyhedron and PC B(0: t), then xk-I(8P)
(3.17)
Proof. The projection of B (0:t) on the hyperplane (y : = 0) is again a ball of radius t in that hyperplane, so that Xk _,(B (0 : t) : u) = t k— I13k _ The estimate now follows from Cauchy's formula. Q.E.D. .
These two corollaries of Cauchy's formula are actually used in the proof of Lemma 3.10. Let P be a polyhedron given by
P=(x:(u1 ,x>
(3.18)
where the unit vectors u^ are assumed distinct. This representation is kept fixed throughout. For each real a define the convex set P a by
Pa ={x:
(3.19)
Then Pa has nonempty interior for a > 0, since PCP, for a >0. In fact, there exists an a 0 <0 such that PQ has nonempty interior if and only if a > a o . Thus P. is a polyhedron for a > a o , if we show that it is bounded. A more precise result is given by the following: LEMMA 3.5. Let xo E Int (P) and suppose that P C B(x o : po). Then P a CB(x o :cpo) for a>0, where c = I + a/d, d = min(d d —<us ,x0 >: 1 <j<m).
Proof. The proof uses the notion of a gauge, or support, function of a convex set. Let C be a closed convex set with nonempty interior and
Integrals Over Convex Shells
29
x 0 EInt(C). Then x-xo FF (x)=inf(a:a>0,x 0 +
a
EC
(3.20)
is called the gauge function of C with respect to x 0 . It is easy to see that if x 0 is an interior point of each of two closed convex sets C I , C2 , then C I c C2 if and only if Fc= (x) < F 1 (X) for all x, where both gauge functions are with respect to x 0 . Returning to P and defining Fp with respect to x 0, an easy calculation shows that [since x o +(x - x o)/a E aP if a = F,(x)] (uj , x - x 0 )
Fp (x) = max
< j < m dj —
FP (x)
U j^ X
—
X 0)
Ima`md+a-
(a>0).
CIearly, Fp (x)> c,FP (x) for all x, where c= min
dj -
<j<m d
+a—
The gauge function of CI(B(x o :p o)) with respect to x 0 is Ilx-x 0 /p 0 , so that P C B (x 0 : p o) implies FP(x)> I llx-x011•
Thus Cl
FF, (x)>—IIx-x 0 11,
which implies that Pa C B (x 0 : p0 /c 1 ). Now d +a-
c -1 = max 1<j<m dj
—
(uj ,x o)
= maxm I+ dj1
Q.E.D.
a a
Cuj,xo))=1+d.
30
Weak Convergence and Uniformity Classes
LEMMA 3.6. Let P be a polyhedron defined by (3.18) and let P. be defined by (3.19) for all a E R'. Then Pa ^P°
if a>0,
PQ CP°
if a<0.
Proof. Suppose a >0 and x E P°• Then there exists y E P such that IIx—y(I
=+ < d + (I < j<m),
+IIuu1I.IIx-YII
so that x E Pa . If a= — b < 0, x E Po , and II y — x II < b, then=+
<4
—
x>
(I<j(m),
implying y E P. Hence x E pa = p -b . Q.E.D. LEMMA 3.7. Let v be a unit vector in R'`, and A = B(x o : p) n { x : a <0, h >0, aER'. Then (k— I)/2
Xk(A) < h r( k+ I ) P
k-i
.
(3.21)
2 Proof. Choose an orthonormal basis v l , ..., vk of R'with v, = v. Let U denote the orthogonal transformation taking v ; to e,, I
Xk (A)=Xk(UA) < hA k _ I(( u E R k- ' : Bull
=h
77
ck—^^/z
r( Q.E.D.
k+l )P 2
k—I
(xo=Ux o ),
Integrals Over
Convex Shells
31
LEMMA 3.8. Let v 1 , v 2 be two linearly independent unit vectors, h > 0, and all a 2 ER'. Let A=B(y o :p)l {x:a ;
<
Then ^(k-2)/2pk-2h2
Ak (A)<
2 1/2
r(k/ 2 )( 1—)
Choose orthonormal vectors pl,p2,...,Pk such that p l = v 1 ,
Proof
P2 = ( 1 —2 ) — 1/2 ( — < v 2, vl>ul + v2)•
Writing y, = (pi , x>, 1 < i < k, Xk(A)=Xk(B(YO:P)r 1 {Y=(YI,...,Yk):al
a 2
(3.22)
where y;=(y; l ,...,yak ) is the image of y o under the map x=(x l ,...,xk ) —>(,..., )=(Yl,...,Yk)=Y, and Y= S=(I,
2 ) 1/2 . Thus
Xk(A) Ak((Y =(Yl ... Yk) : ,
,
(Y3 —Yo3)2 + ... + (Y k — Yok) 2 < p 2 , a 1 < y < a l + h, 8-' (a2 — YYl)
= S — 1112 17(
k-2)/2 k-2
r(k/2) P
Q.E.D. LEMMA 3.9. Let f be a continuous function on R k and let M(a)= f f(x)dx
(aER'),
(3.23)
P,
where P is a polyhedron defined by (3.18), and P, is defined by (3.19). Then
32 Weak Convergence and Uniformity Classes
the function M is differentiable on R' with a continuous derivative given by M'(a)= f fdA k _ I (aER').
(3.24)
aPa
Proof Since Pa is a bounded set for each a, we assume (without loss of generality) that f is bounded on R k . We assume throughout that
O< h < 1.
(3.25)
If there exist ul, , uj ,, where I < j j J2 < m, such that uj , + u=0, then d,+d = >0,
(3.26)
in order that P may not be empty or have an empty interior (as is the case if d,+d2 =0). Similarly, if d,+a+d,+a is negative, Pa+ ,, is empty for sufficiently small h, and M identically vanishes in (— oo, a + h o), say. Thus (3.24) is trivially true, both sides being identically equal to zero. We
therefore assume that a> — /3,
(3.27)
a=min{d,+d=:uj,+u. =0, 1 < j 1 ,j2 <m} J J2
(3.28)
where
if there exist j,,j 2 such that u+u 2 =0, and fl = oo otherwise. Let b be a positive number. By Lemma 3.5 there exists a positive number c, such that
PacB(0:c,)
foralla
(3.29)
The definition of P, gives Pa+h\Pa= U Qli....J., (3.30)
where the union is over all r, 1< d +a for all j ^ (j i ,...,j,)
andd+a<(u1; x>
(3.31)
Then
M(a+h)—M(a)= f
fdx= 2 f fdx,
(3.32)
Integrals Over
Convex Shells
33
where the summation is over all r-tuples (j,'. ,j), I < r < m, as specified above. Let (3.33)
L;(a)={x:xEPa' (u; ,x>=d; +a}.
We shall prove the following inequalities: Z
^k(Qj,.....j,^
6
c 2 h 3 if r 3.34 2, ( )
and f f(x)dx—h f fdA k _,
(I'i<m), (3.35)
L,(a)
where c 2 , c 3 are constants depending only on k, P, and b, and V(h)=sup{if(x)—f(x') :x,x'EB(0:c,),lix—x'
h).
(3.36)
To prove (3.34), note that if any two of the vectors uj. , ... , uj are linearly independent, then (3.34) follows from Lemma 3.8 and (3.29). The only remaining case occurs when r = 2 and uj + uj = 0. In this case Qj .^, 2 c {x : max [ — dj .—a—h,dj ,+a] <
<min[—dj —a,djs +a+h]}. Thus Xk (QJ ,j ,) = 0 unless d.z + a < — d.JI —a,
or,
a < — i (d. + dj:/
(3.37)
But (3.37) is ruled out by (3.27). Hence (3.34) is proved. Now define 2 1 /2 aij = < ui ,uj> , y,=(1—aij)
y=min{y j :y j 0, l
<1,1< m}.
If u; + uj = 0, that is, if y j = 0, then, by (3.27), {x:>d; +a)={x: <—d; —a} c{x:
(3.38)
34 Weak Convergence and Uniformity Classes
so that one may write Q; ={x:
(3.39)
(14 i <m).
In the following discussion we fix the integer i. Let v l ,...,vk be an orthonormal basis such that v 1 =ui . The transformation x=(x 1 ,...,x& )--y =(y 1 ,. .. ,Yk) given by yj =< vj ,x>
(l <j
enables one to write x=y l v l +y', wherey'=Ej_ 2 yj vj . Forj i let u,' be the unit vector defined by Uj
Then, since
_—
Z ) I/Z
uj =aji VI +'Yj; uj .
is orthogonal to u; ,= ajiy I + Yji .
Thus in terms of the new coordinates Qi , respectively, where Qi = l Y=(Y, , ... , Yk):
L1 (a)
become Q , L ( a), ;
;
aj;(YI —d;—a)
•1,
foralljwithyj; O, and di +a
L, (a)= {(di +a,y 2 ,...,yk ) : < u5,y') cj for all j with where
d +a—aj; (d; + a) cj
_ (
Yji
35
Integrals Over Convex Shells
Let Q; (y 1 ) denote the y,-section of Q; . Note that
f_f(x)dx=
Q;
f
L(a)
f
d; + a+h
f(Y1v1+Y')dy2...dyk
dYi f-
d+a
,
Q;(ri)
f((d;+a)v,+y')dy2...dyk.
fdXk-,= J
(3.41)
Q^(d;+a)
Now since Q, (y,) is a section of Q that is the image (under an orthogonal transformation) of Q, c Pa+h c B (0: c,), it follows that ;
If(y^v,+y')
—
f((d,+a)v1+Y')IdY2...dyk
Q, (Y 1)
< V(h)Ak-I(Q; (y1)). c 5 V (h),
(3.42)
say, for all y 1 satisfying d; + a < y, < d; + a + h. Next the symmetric difference Q1 (y 1 )0 Q.(d,+a) is contained in U
c 1 ) fl S (y2 .. .,Y):
{(Y21...,yk): II(Y2,...,Yk)II
,
(j:7j1 .0)
Ia;:l l
I
ii
))
It follows from Lemma 3.7 that Xk - 1(Q; (Yi)AQr (di+a)) < ceh
for a suitable constant c 6 , and so
f .
f( vIYI+y')dy2...dyk
Q;(vi)
< cbl1f h.
—
f
f(v1Y1+Y')dY2...dyx
Q,(d,+a)
(3.43)
The inequality (3.35) now follows from (3.41), (3.42), and (3.43). We thus
have M(a+h)—M(a)—h faP fdA k _ I
Since V (h)- .0 as h-+0, (3.24) is proved. Continuity of this derivative
36 Weak Convergence and Uniformity Classes
follows from that of the function a—i f fdAk _,, which is an immediate L,(a)
consequence of (3.42) and (3.43). Q.E.D. LEMMA 3.10. Let g be a differentiable function on [0, oo) such that lim g(t)=0 I-.00
f
^g'(t)jt k- `dt
(3.44)
faP g(Ilxll)dAk_I < a k •b.
(3.45)
bm
0
00
Then for every polyhedron P
Proof Define the function F on [0, oo) by F(t)=xk _ I (3PnCl(B(0:t))).
(3.46)
Then F is the distribution function of the measure A k - 1 0 h -' induced on [0, oo) by the map x (1x11 on 8P (endowed with the surface area measure A k _ )• Note that F (0) = 0, F(t)=xk _ I (aP)
for all sufficiently large t.
(3.47)
By changing variables and integrating by parts,t
fap g(Ilxll) Xk-.(dx)= f ,
'
g(t)Xk _ 1 oh - '(dt)
= f g(t)dF(t) = lim (F(t)g(t))—F(0)g(0)— f g'(t)F(t)dt _ — f g'(t)F(t)dt, (3.48) from (3.47). Fix 1>0. Let Q be a polyhedron such that Q c C I B (0 : t). tSee Dieudonne [1], Vol. 11, p. 218.
Integrals Over Convex Shells
37
a
Then Q n 8P c (P n Q). Since P n Q is a polyhedron and Ak _, is a measure on the boundary of a polyhedron, it follows that Ak _ I (Q n 3P) k_ I (P n Q)). On the other hand, (3.16) implies that A k _ I (P n Q)) k _,(3Q). Thus by (3.17)
< a (a <x
(a
(3.49)
xk- i(QnaP) <xk- I(aQ)
Choose a countable dense subset (x 1 ,x 2 ,...) of 8B(0:t) and let Q„ be the convex hull of x 1 ,...,x,,. Then Q„TC1(B(O:t)), and (3.49) yields
F(t) =limx k _,(aP n Q„) < a k t k -1 .
( 3.50)
Substituting in (3.48) we get the lemma. Q.E.D. We are now ready to prove Theorem 3.1. Proof (of Theorem 3.1). First suppose that C is bounded, lnt(C) q. Given 6>0, choose x,,...,x„ E aC such that C {x 1 ,...,x„) s. Let P be the convex hull of { x,, ... ,x). ). By taking 6 smaller than the radius of a ball contained in C, we ensure that P has a nonempty interior. Also, clearly,
aC
PC CC
P s,
so that
C` C Pc+s By Lemma 3.6 P
(Pa) CP0 +P
for all p>0, all a E R '. Thus (P_ P )PC P C C, or P_ P C C', so that C`\C-PC(Pa
)t
\P -PC PP +s\P -P
(P>0),
and
P
f
g(Ilxll)dx<
f
g(Ilxll)dx
+s da g(IlxII)Ak-i(dx) =j a ro
f
(P > 0)
38 Weak Convergence and Uniformity Classes
by Lemma 3.9. From Lemma 3.10 we have
fc.\c
p 8(ilxjl)dx <(e+B+p)b a k .
(3.51)
-
Since 8 is arbitrary, we are done. If C is bounded, Int(C)=+, then C °=4 for p>O, and for all 8>0 -
Jc , \c p g(Ilxll)dx= f g(IlxjI)dx< f c6) ,g(llxll)dx c'
a g(Ilxll)dx
G(c+S) bak , (3.52)
^C a ) VC a ) —
since C & has a nonempty interior. Since S is arbitrary, (3.1) is true for all bounded convex sets C. If C is unbounded, look at C, = C n B (0: r) and let r increase to infinity. Since C, 1C and C, - °TC - ° as rToo, and the right side of (3.1) does not depend on the particular convex set, the proof is complete. Q.E.D.
NOTES Section 1.
Detailed accounts of the theory of weak convergence of probability measures may
be found in Billingsley [1] and Parthasarathy [1]. Section 2. A systematic study of uniformity classes was initiated by Rao [3), who proved, in
particular, Corollary 2.7 and Theorem 2.11. The theory was advanced and completed by Billingsley and Topsde [1], who proved the main theorem Theorem 2.4, Corollary 2.6, Lemmas 2.2, 2.3, 2.9, 2.10, and gave the example following Corollary 2.8. These two articles also contain many results and applications not included here. One significant application of the theory is in generalizing and strengthening the classical Gilvenko-Cantelli Theorem. For this, see Rao [3] and Topsde [I]. The useful Lemma 2.1 was proved by Scheffe [1]. The bounded Lipschitzian distance has been studied by Dudley [1], and Corollary 2.8 is due to him. Theorem 2.5, which is a convenient variant of Theorem 2.4, is due to Bhattacharya [3]. Section 3. Rao [1] was the first to prove the existence of a constant c(k), depending only on k, such that 4((8C)') < c(k)c for all convex subsets C of R k . The particular estimation given by Theorem 3.1 is essentially due to von Bahr [3). It is easy to check that for k - I and a nonincreasing g [on (0, oo)], this estimate cannot be improved upon. The proof presented here provides perhaps the first detailed formal derivation of the estimate. The development (in particular, Lemma 3.9) may be of some independent interest.
CHAPTER 2
Fourier Transforms and Expansions of Characteristics Functions
The main mathematical tool used in this monograph is the Fourier transform and its extension, the Fourier—Stielijes transform. The Fourier—Stieltjes transform of a probability measure on R k is better known in probability literature as its characteristic function. In Sections 4 and 5 we present a summary of some basic facts about these transforms. Section 6 introduces moments and cumulants of probability measures on R k and presents some inequalities concerning them. In Section 7 the Cramer—Edgeworth polynomials associated with a set of cumulants are studied. The principal results of Chapter 2 are th'e asymptotic expansions of (derivatives of) characteristic functions of normalized sums of independent random vectors. These expansions, in terms of the Cramer—Edgeworth polynomials, are developed in detail in Sections 8 and 9. In Section 10 some classes of probability measures, which are used as smoothing kernels in Chapters 3 and 4, are introduced.
4. THE FOURIER TRANSFORM
In this section we collect some standard results on Fourier transforms without proofs. Let L'(R'o), I
f jflodx
40
Expansions of Characteristic Functions
with norm I/p
l fII =( f IfIPdx) ,
(4.1)
'
where f gdx denotes the integral of g on R k with respect to Lebesgue measure on R k . As usual, two functions f and g on R k are said to be equivalent if they are equal almost everywhere with respect to Lebesgue measure. The space L 2 (R k ) is a Hilbert space endowed with the inner product < •> 2 (4.2)
2= f f 8 dx.
Here g denotes the complex conjugate of g. For nonnegative integral vectors a=(a 1 ,...,ak ) we write X ° =Xa l • ••Xk
[X=(x^.....xk)ERk],
DDr'...Dk
[I<j
a=(a I ,...,ak )E(Z + ) k ],
l
(4.3)
and say that D°f is the ath derivative of the function f. Also, for each complex-valued function f on R k we define the function f by (xER k ).
f (x)=f(—x)
(4.4)
The function is said to be symmetric if f = f. For f E L'(R k ) the Fourier transform f of f is a complex-valued function on R k defined by (IER k ),
f (t)= fe'<`•">f(x)dx
(4.5)
where <.,> is the usual inner product in Ck : k
= L; lj xj [1=(ti,...,lk), X=(x 1 ,...,Xk )ECk ],
(4.6)
j-1
with c the complex conjugate of the complex number c, giving rise to the norm 1^ • ^1, '/Z
k
IIIII=(2 ItjI ) 2
(IECk).
(4.7)
The Fourier Transform
41
Here 1cl 2 =cc for complex numbers c. We shall sometimes use a different norm I • I in Ck : k
IxI= I Ix^I
(xEC k ).
(4.8)
j-1
THEOREM 4.1. Suppose f E L'(R k ). (i) f is uniformly continuous on R k . (ii) If (t)I < II f II for all t E R k . (iii) (Riemann-Lebesgue Lemma) If(t)I-*0 as lt^I-► oo• (iv) (Fourier Inversion Theorem) If (the equivalence class of) f E L'(R'), then one can recover (the continuous version of) f from f by the formula f(x)=( 21r)
-k ( e -i
Hence
<J.X>f(t)dt
(x ER")
J
J=(2)
J = ( 217 ) -k l•
(v) Let a = (a 1 ,. . . , a k ) be a nonnegative integral vector. Define g by g(x)=xaf(x)
(xERk).
If gEL'(R k ), then D°f exists and g=(—i) 1°1 D °f
(vi) If f E L`(R k ) n L 2 (R k ), then f E L 2 (R k ) and 11 f 112 = ( 21T) -k/2 11 f ^I2•
The last result (vi) says that the map defined by f-*(27r) - "`" 2f regarded as a map on the subset L'(R k )n L 2 (R k ) of L 2 (R k ) into L 2 (R k ), is an isometry. Since L'(R k )n L 2 (R k ) is a dense subset of L 2 (R k ), one can extend the above isometry on all of L 2 (R k ), and f so extended is still called the Fourier transform of f j E L 2 (R k )). The following theorem is then immediate. THEOREM 4.2 (Plancherel Theorem). The map f-^(2?r) - '`" 2 f is a linear isometry on L 2 (R k ), and
-k
2
[f gEL 2 (R k )J.
42
Expansions of Characteristic Functions
For f, g E L'(R IC) one defines the (equivalence class of) function(s) f*g, called the convolution off and g by (f*g)(x)
= f f(x—y)g(y)dy
(xER k ).
(4.9)
It follows from Fubini's theorem that f•gEL'(R k ) and that
III=gI[I< (ifIh IlgtI1
[ffgEL'(Rk)J.
(4.10)
The convolution operation is clearly commutative and associative. The n-fold convolution f " is defined recursively by ,f*'= f,
f*n = f`(n-l) * f
[n> l. fEL'(Rk )].
(4.11)
5. THE FOURIER-STIELTJES TRANSFORM
We shall now extend the concept of the Fourier transform to the set 6_11t of all finite signed measures on the Borel sigma-field of R". Let µ be a finite signed measure. We define the finite signed measure µ" by µ(B)=µ(—B)
(BE3Y').
(5.1)
A finite signed measure t is called symmetric if u = µ. The Fourier—Stieltjes transform µ of a finite signed measure µ is a complex-valued function on RC defined by µ(t)=
f e <' ,
x
>µ(dx)
(:ER', µE ')1L),
(5.2)
where, as usual, the integral is over the whole space R k . If µ is a probability measure, µ is also called the characteristic function of s. Note that if s is absolutely continuous (finite signed measure) with respect to Lebesgue measure, having density (i.e., the Radon—Nikodym derivative) f, then
µ=f.
(5.3)
The convolution µ*v of two finite signed measures is a finite signed measure defined by (µ*p)(B)=
f
(B-x)p(dx)
(BE iYC),
(5.4)
The Fourier-Stielijes Transform
43
where for A C R"`, y E R k , the translate A +y is defined by A+y=(z=u+y:uEA)•
(5.5)
It is clear that convolution is commutative and associative. One defines the n-fold convolution u*" by µ *1 =µ µ*"=µ*("- i) *µ
(n> 1,
e'X).
(5.6)
Let be a signed measure on R k . For any measurable map T on the space (R k , GJ3 k ,µ) into (R 3 ,Jas) one defines the induced signed measure µ ° T -' by (µoT-')(B)=µ(T-'(B))
(B E'3).
(5.7)
THEOREM 5.1 (i) (Uniqueness Theorem). The map µ—*A is one-to-one on 671.. (ii) For u E JR, , µ is uniformly continuous and µ( 0 ) = µ( R k ) , Iµ(t)I
(tERk).
(iii) For µ, v E OiL µ*v=µ $. (iv) If ft E Olt , then µ=µ,
so that µ is symmetric if and only if is real-valued. The symmetrization µ*µ" of is always symmetric since *µ=IZ. (v) If T is an affine transformation on R k into R 5 : Tx = a + Bx, x E R k , then
T- ► (t)=ei<<.a>µ(B't)
(1ERs, IRE
where B' is the adjoint (or transpose) of B.
),
44
Expansions of Characteristic Functions
(vi) If µ E Jll, and f Ix °IIjI(dx)
µ(dx)
I,
... ,
a d,
then D ° µ exists
(t E Rk)
(vii) Ifµ E L 1 (R'`), then is absolutely continuous with respect to Lebesgue measure with a uniformly continuous and bounded density. (viii) (Parseval's Relation). Let µ, v be two finite signed measures on R k . Then
f µ dv = f i dµ. The following theorem (due to Cramer and Levy) gives a useful characterization of weak convergence of probability measures on R'`. THEOREM 5.2 Let (G„ : n> I) be a sequence of probability measures on R k . If the sequence converges weakly to a probability measure G, then {G„) converges pointwise to G. Conversely, if {G„} converges pointwise to a continuous limit h, then there exists a probability measure H such that {G„} converges weakly to H and H = h. It should be noted that if { G„ } converges weakly to G, then {Ô) converges to G not merely pointwise, but uniformly on compact subsets of R k . This is because the class of functions }
(e < r,. >:tEK), ;
where K is a compact subset of R k , is equicontinuous and uniformly bounded; hence, by Corollary 2.7, it is a uniformity class for every probability measure G.
6. MOMENTS, CUMULANTS, AND NORMAL DISTRIBUTION vk ) is a nonnegative Let G be a probability measure on R'. If v = (v 1 v 2 integral vector such that Ix'I is integrable with respect to G, one defines the v-th moment IA, of G by ,
µ,,
= f x'G (dx).
, ... ,
(6.1)
Moments and Cumulants 45
For every nonnegative real s one defines the s-th absolute moment ps of G
by PS=JIIxII'G(dx)
(s>0).
(6.2)
Note that, by Theorem 5.1 (vi), if µ, is finite for some nonnegative integral vector v, then iI°Iµ=(D°G)(0)
(6.3)
and, therefore, one has the Taylor expansions O(i)=l+ ^ µ vi (Iltlls) + o Y
(
IPI<s
( t-> 0),
(6.4)
if ps is finite for some positive integer s. Here v!=v 1 !v 2 !
.
.. Vk !
[V=(V1,....'k
)].
(6.5)
If k=1, = f xJG(dx)
(1=1,2,...).
(6.6)
For a positive real number x, logx denotes the natural logarithm. In other words, the function x.-*logx on (0, oo) is defined as the inverse of the
function x--),exp (x) = 1 + x + x 2 /2! + x 3 /3! + • on R'. For a nonzero complex number z = r exp (i8) (r>0, - it < 8 < ir), we define logz=logr+iO (r>0, -^r<0 it). (6.7) Thus we always take the so-called principal branch of the logarithm. The characteristic function G of a probability measure G on R k is continuous and has a value of one at 0. Hence in a neighborhood of zero one has the Taylor expansion t = ^ Jc^. (it) V +o to gG O— (IIt IIS) ( t- 0 ) ,
(6.8)
IPj<s
if ps is finite for some positive integer s. Here the summation is over tNote that Ixi < I1x11 H , so that I, I
Also, see Corollary 8.3.
Expansions of Characteristic Functions
46
nonnegative integral vectors v, and (see Corollary 8.3) = (D log G )(0).
(6.9)
The coefficient y, is called the v-th cumu/ant of G. A simple calculation gives Xo=0,
X,,=µl,
if lvl=1.
(6.10)
The following formal identity enables one to express a cumulant x in terms of moments by equating coefficients of t' on both sides:
II
' X(v)•=(-1)3+is z^ =i 11 >
(a)
)
J.
(6.11)
/
This is obtained by noting a, log(I+z)=
(-1)° + ' 1 z'
(zEC, lz^
(6.12)
For example, in one dimension X21L2! LI2,
X3 — µs - 3µ2µi+2µi3,
X4=µa -4 µ3µi -3 µZ 2 + 12 µ2µ1 Z-6 µi °, (6.13) Xs=µs -5 µ4 µi — 10 µ3µ2+ 20 µ3µi 2 + 30 µ2 2µI —60 µ2µi 3 + 24 µI 5 .
In general, in R k one may write li..
J
(6.14)
where the summation is over s =1,2,. . . , vi and for each s over all s-tuples of nonnegative integral vectors (v 1 ....,v5 ) and s-tuples of nonnegative integers (j l ,...,j,) satisfying (6.15)
and c(v 1 ,...,v5 ;j,,...,j,) is a constant depending only on (v,,...,v,) and (jl.... is). ,
For a given probability measure G on R" with cumulants {X,, : (vi < s),
Moments and Cumulants 47
we define the polynomial X,.(z) in z = (z ,, ... , z k ) E C k by i
(zECk; s= 1,2,...).
(6.16)
IIS
We point out that for t E R', X,(t) is the sth cumulant of the probability measure on R' induced by the map x-*defined on the probability space (R", J^'Y,G) into (R',). This follows because the function u -*log G (ut) [u E (- a, a) for some a>01 is the logarithm of the characteristic function of this induced probability measure on R' and because [by ( 6. 8 )] "
it
d -S log6(ut)I =s 1 2 X" v) ( -o
I V I- S
Often we find it convenient to state results in terms of random variables and random vectors (although for purposes of this monograph it would be possible to avoid such notions). The with moment µ"(X), the with cumulant y, (X), and the sth absolute moment p(X) of a random vector X in R k are defined to be the corresponding characteristics of the distribution P x of X. Note that for a random vector X with a finite covariance matrix, Cov(TX+a)=Cov(TX)= TCov(X)T', (6.17) where T is an m X k matrix (l < m < k), T' is its transpose, and a E R'". Also, if (X 1 ,...,X„) are n independent random vectors in R' and p 2 (X 1 )
(6.18)
If X is a random vector in R k and has a finite with moment (and cumulant), then for every c E R', N,(cX)=c 1 " pti(X)
(lv >1 ), (6.19)
X,.(cX) = c"""X"(X)
and for all cER', bERc,
x"(cX+b)=X„(cX)=cIX,,(X)
if lvI > 2,
X,,(cX+b)= y. (cX+b)=cµ"(X)+ =cX,,(X)+
if ivl=1.
(6.20)
48 Expansions of Characteristic Functions
It may also be shown, either by direct computation, as in (6.13), or by looking at (6.14) and (6.15), that X,(X)=(X) for Iv1=2,3,
if E(X)=0.
(6.21)
The relations (6.19) and.(6.20) follow easily from definitions; note that in a neighborhood of t=0 logPPx+b (t) = i+logP X (ct).
(6.22)
Since the cumulants y, are invariant under changes of origin for Ivl > 2, they are sometimes called semi-invariants. The effect on y (and µ,) of a change of scale, as given by (6.19), is also quite simple. However the most important property of cumulants from our point of view is the following: If { X 1 ,.. . , X, j are independent random vectors in R k each having a finite with cumulant, then (X1+..• +Xn)=x,(X1)+ • • • +X„(Xn).
(6.23)
This follows from n
logPXl+.,.+X.(t)= 2 logP Xj (t),
(6.24)
js1
which holds in a neighborhood of 1=0. We now derive some inequalities for moments and cumulants of a random vector X. Henceforth, if there is no possibility of confusion, we simply write µ., p S , 7^, for µ,(X), p s(X), L(X), and so forth, respectively. The following basic inequality is stated without proof.t It is used fairly often in this book. LEMMA 6.1 (Generalized Holder Inequality). Let f 1 , 1,< i < m, be measurable functions on a measure space (2,Q,µ) into (R', V) such that If1 1", 1 < i < m, are integrable for some m-tuple of positive reals (p t , ..., p m ) satisfying pj ' + • • • + pm ' =1. Then
f Ifs = fmldµ
-
LEMMA 6.2. Let X be a random vector in R k having a finite sth absolute moment p s for some positive s. If X is not degenerate at 0,; tSee Hardy, Littlewood, and Polya [1] for several different proofs of this important inequality. That is, Prob(X-0)3. 1.
Moments and Cumulants 49
(i) r-->log p, is a convex function on [0, s]. (ii) r-^p /` is nondecreasing on (0,s]. (iii) r_►(p,/p 2 1 / 2 ) 1 / ( r -2) is nondecreasing on )
(2,s] if s>2.
Proof. We may assume that X is not degenerate at 0, so that log p, is defined for 0 < r < s. (i) Let 0 < < 1, 0 5 r,, r 2 s. Then Pa,,+(0 —a)r i =
E (IIXIlar,+(1 —a)r,
) = E (YZ),
where Y=I;XII«r', Z=IIXI )r '. Since E(Y'/a) and E(Z"° - a ) ) are both finite, by Lemma 6.1 (with m=2), E (YZ) c(E (Y l /a)) a (E (Z'/u
p
=
r«
p
^^
-
a,
so that log par, +(i - a)r 2 < a log p, , +(l - a) l og p, 2 . (ii) Since log p o = 0, and (11r) log pr is the slope of the line segment joining (0,logpo ) and (r, log p,), it follows by the convexity of r--*log pr that
the function r-+Iog p; = r logy, is nondecreasing. (iii) If p 2 = 1, then the function r
( -2)
r^logP
=(r-2) -I logPr=(r-2)-)(logy,-logP2) (6.25)
is increasing in (2,s], since the expression on the extreme right of (6.25) is the slope of the line segment joining (2,logp 2 ) and (r,logp,), and r- logp, is convex in [0,s] by (i). In the general case, apply this argument to Y X/pi /2 . Q.E.D. A simple application of Lemma 6.2(ii) yields r
(tai l <mr -I Ila i VV
(a 1 ER', 1
(6.26)
for every r> I and every positive integer m. It also holds, trivially, for r=0.
50
Expansions of Characteristic Functions
LEMMA 6.3. Let X be a random vector in R k having a finite sth absolute moment p S for some positive integer s. Then for nonnegative integral vectors v satisfying Ivl ( s, (6.27)
,LLl < EIX"I < p H ,
and there exists a constant c 1 (v) depending only on v such that (6.28)
Ix.l < c1(v)P1'1. Proof. The inequalities (6.27) follow from the simple inequality
(6.29)
x'I < 11xII''. The inequality (6.28) follows from (6.14) and (6.15) by noting that µV; ... µ s I
. per < p^ PIV I ..
V1
E 1;I,l/IYI "' =p1.
(6.30)
The first inequality in (6.30) follows from (6.27), the second from Lemma 6.2(ii). Q.E.D. Before concluding this section, we mention a few properties of the single most important probability measure on R "—the normal distribution 4,,,, v , whose density 4, n v (with respect to Lebesgue measure) is given by 4)m v (x)=(277) -k/z (DetV) -1/Z exp{
-- <x,V -
^x>}
(xER k ). (6.31)
Of the two parameters, m ER" and V is a symmetric positive-definite k x k matrix. The notation Det V stands for determinant of V, and V' is, of course, the inverse of V. It is well known thatt (D,,,. v (t) = exp { i— 2 )
(t ER").
(6.32)
From this it follows that m is the mean and V is the covariance matrix of For the computation of cumulants X, for II > 2, it is convenient to take m =0 (for It > 2 a change of origin does not affect the cumulants x.). This yields log f o, v (t) = — 2tSee Cramer [4], pp. 118, 119.
(6.33)
The Polynomials P,
51
which shows that X„ = (i,j) element of V if v = ei + ej ,
(6.34)
where e1 is the vector with I for the ith coordinate and 0 for others, < i k. Also, X„=0
if Ip1>2.
(6.35)
Another important property of the normal distribution is
(D m1, V l * ( Dm 2 . V,* ...
* 0
(6.36)
V = 4i m ,,,,
where m=m1+m2+••• +m,
(6.37)
V=V i +V 2 +•••+V,,.
This follows from (6.32) and Theorem 5.1(i), (iii). The normal distribution (D o.r , where I is the k x k identity matrix, is called the standard normal distribution on R k and is denoted by 4'; the density of 4' is denoted by .0. Lastly, if X = (X 1 , ... , X k ) is a random vector with distribution 4',., ^,, then, for every a E R k , a0, the random variable (a, X> has the onedimensional normal distribution with mean and variance = ^k a a i ja r^ , where ij=I a j =(i,j) element of V=cov(X i ,Xj )
(i,j= I....,k).
7. THE POLYNOMIALS P, AND THE SIGNED MEASURES P J Throughout this section v = (v i , ... , v,,) is a nonnegative integral vector in R k . Consider the polynomials
X^ 7,(z)=s!
lvIas P !
(z`=z^
zP
,...
(7.1)
z)
in k variables z 1 , z 2 ,...,z k (real or complex) for a given set of real constants x,. We define the formal polynomials P3 (z: (X„)) in Z 1 ,....Z k by means of the following identity between two formal power series (in the real
variable u). 1+ I P,(z: {X })u s =exp '
X,+2(z) !
u' m
=l+ 1+I ,
m=i
E s + 2)! us .
m! L•^ (s+2)!
s=i
(7.2)
52
Expansions of Characteristic Functions
In other words, Ps (z : {7c}) is the coefficient of u' in the series on the extreme right. Thus
Ps( z : (X"})=
t
V1 f
L
,r,l
(il
X1 1 +2( Z ) Xj2+2(z) ...
Xj„ + 2 ( Z )
(11+2)! (12+ 2 )!
(Jm+2)1 1}
^^
1 0 -0,
1
mt
(s=1,2,...),
(7.3)
where the summation X* is over all m-tuples of positive integers (j 1 ,...,jm ) satisfying M (7.4) ji =1,2,...,s (1G i<m), >ji=s, and S** denotes summation over all m-tuples of nonnegative integral vectors (p 1 ,. . . , vm ) satisfying kI=j; +2
(1
(7.5)
In particular, z
P 1 (z: (X,,})= X 3 i =
-z", I"I' 3
X4( z )
x 32
(
z
)
P2 (z: {X})= 4! + 2!(3!)2
P3(z:})=
x5(z)
5!
+ X4( Z ) X 3( Z ) + X 3( z)
3!4!
(3!)a
(7.6)
LEMMA 7.1. The degree of the polynomial PS (z: {X,}) is 3s, and the smallest order of the terms in the polynomial is s + 2. The coefficients of Ps (z: {x.)) only involve X„'s with M <s+2. Proof. This follows immediately from (7.3). Q.E.D. The notation above has been chosen to suggest that eventually we use cumulants of some probability measure for X„'s in the expression for PP (z: (X"}). The polynomial Pt can be defined in this sense only if the
The Polynomials P, 53
(s + 2)th absolute moment p5+2 is finite. The role of the polynomials P S in the theory of normal approximation is briefly indicated now. More precise results are given in Section 9. Let G be a probability measure on R k with zero mean, positive-definite covariance matrix V, and finite sth absolute moment p, for some integer s>3. Then [see (6.8), (6.16), and (6.23)1 log e"
\ n 1/2) =n
logG ( I) -
s-2
!t t = — i+ E (r+2)! n '/Z+n o(^^ / 2 ii `
,
n
i
)
Lo)
(7.7)
.
(
Thus for any fixed t E R k ,
G"
)
(
=exp(--L)-exp
I
n ^/ 2
S-2
(,12(" n-'/2+o(n-(s-2)/2)
n1/2 s-2
=exp{-i
n - '/ 2 P,(it: {X,})
Vt>}. l+
r=!
x (I +o(n -js-2) / 2 ))
(n-oo),
(7.8)
where, in the evaluation of P,(it: {X„}), one uses the cumulants y of G. Thus, for each t E R k , one has an asymptotic expansion of G" (t /n 1 / 2 ) in powers of n - 1 / 2 , in the sense that the remainder is of smaller order of magnitude than the last term in the expansion. The first term in the asymptotic expansion is the characteristic function of the normal distribution'I ,. The function (of t) that appears as the coefficient of n - '/ 2 in the asymptotic expansion is the Fourier transform of a function that we denote by P,(-0 0. ,,: (y,)). The reason for such a notation is supplied by the following lemma. LEMMA 7.2. The function t-*,(it:{x.})exp{-z)
(IGRk),
(7.9)
is the Fourier transform of the function P,(-40 o v : {X„}) obtained by formally
54 Expansions
of Characteristic Functions
substituting (— 1) IHI D V *O v
for (it) " (7.10)
for each v in the polynomial P r (it: {X,,)). Here 4, is the normal density in R k with mean zero and covariance matrix V. Thus one has the formal identity P,( - 4o: {X,))= P,( —D: {X„})$o.v
(7.11)
where —D=(—D I ,..., —Dk ). Proof. The Fourier transform of 0 o, v is given by [see (6.32)]
— Z}
(tERC).
(7.12)
D qo v(t)=( it)r*o.v(t)
(tER"),
(7.13)
0o,v(t)=exp{
Also
which is obtained by taking the with derivatives with respect to x on both sides of (the Fourier inversion formula) 00,v(x)=(2sr) -k
f
exp{—i}$ o. v (t)dt
(xER k ) ( 7.14)
[or, by Theorem 4.1(iv), (v)]. Q.E.D. We define P,(— (D o, v : {7t,.}) as the finite signed measure on R k whose density is P,(- 4o.v:(X„ }). For any given finite signed measure µ on R", we define u(•), the distribution function of µ, by µ(x)=u((—oo,x])
(xER"),
(7.15)
where (
—
oc,X]=(
—
oo,X1]X .. • X(
—
ci,X k ]
[X= (x1,...,Xk)ERk}. (7.16)
Note that D,... DkP,( — 'Do.v: {X.})(x)= P,( — $o,v: {X,})(x) ,(—D: {X, , }) 40.v(x)
=P,( —D: (X, ))(DI ,
...
DkOo.v)(x)
= D j ... Dk (P,(— D: {Xr))4o.v)(x). (7.17)
The Polynomials P, 55
Thus the distribution function of P,( — (D o, I, : {x,}) is obtained by using the operator P,(— D: (x)) on the normal distribution function (D o, I,(• ). The last equality in (7.17) follows from the fact that the differential operators P,(—D: {X,}) and D I D 2 • • • Dk commute. Let us write down P,(-40: (x,)) explicitly. By (7.6), P I (it : (x}) =
(7.18) -- (it) ° (I ER"),
so that (by Lemma 7.2) P1(
—
$0,v: {x,})(x)
_—
Irl-3
Dr0O.v(x)
k —
6 X(3.0.....0)
—
3
II
v ljxj )
k
+3v 11 k v l'xj
j-I
j-1 k
vkjx. +3v
+... +X0.....0,3) — (
k j -1
k
2 k
2 "12.1.0.....0) — 2I vlxj El^ 2jx! )
k
k
+Zv l2 vlixj+vll j-1
2 k
k +... + X10,....0.1.2)
k
v2Jx1 j-I
k—Ijx ^IVk^xj) (^ I V j
k
+2v k ' k-I 2 v kjxj +v kk 2 Uk-IjxJ j- 1 j-1
Jj
Expansions of Characteristic Functions
56
—
X (1.I.1.0,....0) — (J
x)(±
U2jXl /
I
k
1 v3JxJ /
\
k
k
v 12 U3jx,+U13 2 V2iX^') U 23 UIIXj +...
l-1
JAI
k +X(0.....0,1,1,1)
f-I
I k
k k—IjxJ Vkjxl
Uk-2jXj)(j- U 1 JiI
/
k
+Uk-2.k-1
k UkJx.+Uk-2.k
Uk-1jX.
l -1 l—I
l
k
+Uk—1,k
2
Uk-2jkJ
l- 1
OV(x)
r V -1 =((v')), x=(x l ,...,xk )ER k ]. (7.19)
If one takes V= I, then (7.19) reduces to P1(-4: ( ))(x) =
{
-
—
6["13,0,....0)(
— X^+ 3 X1)+ ... +x((0....,0.3)( — xk+ 3 xk)]
2 [X(2,1,0.....0)(
— x^x2+x2)+ ... + X(0.....0,1.2)( —Xk xk—I +Xk—I)]
[X(1,1,1,0,...,0)(
—x 1 x 2 x 3)
+... +
X(0,...,0,1,1,1)(
Xk xk—I xk-2)]}'O( x )
(x E R k) (7.20)
where is the standard normal density in R k . If k= 1, by letting X^ be the jth cumulant of a probability measure G on R'(j = 1, 2, 3) having zero mean, one gets [using (6.13)] P1(
—
*: {7G.))(x) = 6 µ3 (z3-3x)$(x)
(xER'),
(7.21)
where µ 3 is the third moment of G. Finally, note that whatever the numbers (X,) and the positive definite symmetric matrix V are,
f P,( — $o v: {^))(x)dx=P,( -4 o,: {x,))(R k ) =0,
(7.22)
Approximation of Characteristic Functions
57
for all s > 1. This follows from
for III 1.
f (Dpo o, v )(x)dx=0
(7.23)
The relation (7.23) is a consequence of the fact that 40 o v and all its derivatives are integrable and vanish at infinity.
8. APPROXIMATION OF CHARACTERISTIC FUNCTIONS OF NORMALIZED SUMS OF INDEPENDENT RANDOM VECTORS Let X 1 ,. ..,X ben independent random vectors in R k each with zero mean and a finite third (or fourth) absolute moment. In this section we investigate the rate of convergence of Px,+...+x.)/nt/2 to 1 o.v , where V is the average of the covariance matrices of X 1 ,... ,X. The following form of Taylor's expansion will be us-;ful to us. LEMMA 8.1.t Let f be a complex-valued function defined on an open interval J of the real line, having continuous derivatives f ` of orders r=1,...,s. If x, x+hEJ, then (
)
s—I
f(x+h)=J(x)+
h^ fi 1 (x)+
(s hs l)i
f ' (1–v) s- 'f( s l (x+vh)dv.
–
(8.1) COROLLARY 8.2. For all real numbers u and positive integers s (to)' I J u
exp { iu } — 1 — iu — • • • — (s
—
1)! < —. l(8.2)
Consequently, if G is a probatility measure on R k having a finite sth absolute moment p s for some positive integer s, then i
G(t)
—
I
—
iµi(t)
—...
—
1)! µs -1(t) $ (t) ^ p511tlls s ^ s^ tE ( R k )+ ( 8.3 )
tHardy [fl, p. 327.
58 Expansions of Characteristic Functions
where for r = 1, ... , s, vi µrt r ^ljs(t)= f II s G(dx). (8.4)
A'(t)= f'G(dx)= Irl - r
Proof The inequality (8.2) follows immediately from Lemma 8.1 on taking f (u) = exp (iu) (u E R 1 ) and x = 0, h = u. Inequality (8.3) is obtained on replacing u byin (8.2) and integrating with respect to G(dx). Note that
a:(1)= f II'G(dx) < Iltll'p,. Q.E.D. COROLLARY 8.3. Let f be a complex-valued function defined on an open subset SI of R k , having continuous derivatives Drf for I vl <s (on SZ). If the closed line segment joining x, x + h (E R k ) lies in St, then vl (Dr7)(x)
f(x+h)=J(x)+ 1
+s 2 vl j l (1—u) s-1 (Drf)(x+uh)du,
(8.5)
U
Irl—s
and therefore V
f(x+h) — f(x) — hl
1
<
(D rf)(x) r
Ivll max{^(D7)(x+uh)^:0
Irl-S
Proof. Define the function g on (— c, I + c) by (8.7)
g(u) =f(x + uh),
c being a positive number for which the line segment joining x-eh, x+(l +e)h lies in S2. Using the formula for differentiation of composite functions, one obtains, by induction on r,
g (r)( u ) = vl h'(Drf)(x+uh) Irlsr
(r= 1,.... ․).
(8.8)
Approximation of Characteristic
Functions
59
Hence, by Lemma 8.1, g(1)=g(0)+2 g( r 0) + (
f ' (1—u)s -I g ( : ) (u)du,
(s 1)1
r— 1
which, on substitution from (8.7) and (8.8), yields (8.5). The inequality (8.6) follows immediately from (8.5). Q.E.D. Let X i ,. .. , X„ be n independent random vectors in R' each having a zero mean and a finite sth absolute moment for some s> 2. Assume that the average covariance matrix V, defined by V= n I Cov(Xj )= n 2, Vj ,
(8.9)
l —1 j-1
is nonsingular. Then we define the Liapounov coefficient ii,, by n
n -1 I E(j5 ) l^
II^II—I
n —(s —2)/2 (s > 2).
n s/2
!s „= sup n
1
(8.10)
2 E (2 ) j-1
It is simple to check that 1, ,, is independent of scale. If B is a nonsingular k x k matrix, then BX I ,...,BX„ have the same Liapounov coefficients as X 1 ,...,X, have. If one writes (1<j'cn)
Prj—E(IIXjIIr)
,
n
P
Pr = n
(8.11)
(r>0),
rj
j=1
then it is clear from (8.10) that l,.n n -(: z)/z sup
P, II t CI = PS S
IItII= i fit, Vt>s/2
n
^s12
-(s- 2)/2 (8.12)
where X is the smallest eigenvalue of V. In one dimension (i.e., k= 1)
!S n _P P /2 n
—(s-2)/2
(s > 2).
(8.13)
We also use the following simple inequalities in the proofs of some of the
60 Expansions of Characteristic Functions
theorems below. If V= I, then.. n EI
n' /2l,,nlltIJ .
(8.14)
In the rest of this section we use the notation (8.9)-(8. 11), often without further mention. THEOREM 8.4. Let X 1 ,...,X, be n independent random vectors (with values) in R'` having distributions G 1 ,...,G, respectively. Assume that each Xi has zero mean and a finite third absolute moment. Assume also that the average covariance matrix V is nonsingular. Let B be the symmetric positivedefinite matrix satisfying B2= V '. (8.15) -
Define -
r..2
11
d2
)
r-2 ,
lT)
b„(d)=2-d(da(d)+1)(ls.n)2/3. 6
(8.16)
Then for every d E (0, 2' / 2) and for all t satisfying
Iltll ( d'/',
(8.17)
one has the inequality
n
^G( nBt 1/2 )-exp{-2IItIl2} i-^
<(da(d)+-6L)13,nII1Il3exp{
-
b„(d)IItII 2 }• (8.18)
Proof. Assume first that V= I and, consequently, B =1, where I is the
identity matrix. In the given range (8.17) one has [see (8.14)]
(
)- ll <(2n) - ' E2 n f) <(2n)-'(EI Is)2/3 2/3
<(2n) -
^+ E I< t,X1> 13 '( j o> <(2n) - '(n 3 / 213 IItjI 3 ) 2/3 = 2/3 n 3 IItII 24 d2,
(8.19)
Approximation of Characteristic Functions
61
so that logGj (t/n 1 / 2 ) is defined for 1 < j < n, and [using (8.14)] n
log II Gj (
n
) + 22 I
j^l
_ 2 ^Iog(I—(1—Gj\ j \ -1
n
n
l))+(2n)_'E21
a0
r
+ Gj ( ^ /2 ) l + (2n) - I E2 ]
n
<
n
J
2
°O
2 r-2
(4n2)- ( E < t,Xi> 2) jr-1(^) l
j-1
+6n-3/2EII3
r-2
n
2 ( E I< t,Xj>l 3 ) 4/3+ 6 /3.nll t II 3
< a (d )n -2
j -1
n 4/3
EIl3)
6 13.nII t ll 3
+
(8.20)
Hence, noting that fie"— 11 xleNNN for all complex numbers x, one has n Gj( 1 n /2 ) — exp{ j
—
-
2IItII 2 }
I n
=exp{
—
_ n
i1It1I 2 } exp log IIGj ( j
<(da(d )+
1 ) 13,nlltll 3 exp
1
—1
{ — 2II , II 2
+d(da(d)+ 6 ) 13,3 11 1 11 2 } =(da(d)+6)/3.fIltll 3 exp{
—
b,,(d)11 1 11 2 }.
(8.21)
62
Expansions of Characteristic Functions
This proves the theorem when V= I. In the general case, look at the random vectors B X 1 ,.. . , B X„, whose average covariance matrix is 1, and recall that the Liapounov coefficient is independent of scale. Q.E.D. By going through the proof it is easy to see that if the X D 's in Theorem 8.4 are i.i.d. with common distribution G, then G"( B I exp{
—
1IItI1 2 } <(da(d)+ L ^13,„Illll 3
Xexp { _1'1_d2a(d)_ 6
(8.22)
for all d E (0, 2 1 / 2) and for all t satisfying
(8.23) II 1 II < dl3 • The next two theorems sharpen (8.22) and (8.18), under the additional assumption of finiteness of fourth moments. THEOREM 8.5. Let X be a random vector in Rk with destribution G. Suppose X has zero mean, positive-definite covariance matrix V, and a finite fourth absolute moment p 4 . For all t satisfying
IItII < 1 /4-„t !
(n> 1),
(8.24)
one has the inequality
I
G"l B^ 2 )—exp { — iIIIII Z }( 1 + 6i3n-t/2µ3(1))I
<[(0.1325)n - 1 + 2 l4." ]IItII 4 exp { — Iltlu Z }
+(0.0272)li „IItII 6 exp { — ( 0 . 3835 )IIt1I 2 }
(n> 1), (8.25)
where µ3(t) = E3 , and B is defined by (8.15). Proof. Assume that V= I. In the given range of:
I
G
<(2n)-'IItII2<e. (8.26) ( h)— ll -
Hence
logG" (
n /z )+
1
11 1 11 2 — I i 3n - 11Zµ3(t)
I
=nllog[I—(1—G(n ^Z))l+(2n)-hIItII2-6i3n-3/2µ3(t)
Approximation of Characteristic Functions
=n - Ir - `(1-G( r-z
i n /Z )) r
63
+6( i n /Z ) -1 +( 2 n) ^Iji) 2
6t3n-3/2A3(t) "0
<( 4 n) - ^Iltll ° r - '(sy -Z +iala,"Ilrll 4 r-2
<[(0.1325) n '+i14 "]Iltll ° (8.27) -
Similarly,
I
n log G ( n 1 /Z ) + II t II Z ^ <(0.1325)n - `II t II ° + 6 13."II tll 3 <(0.0663+0.1667)1 3 "11 1 11 3 =( 0 . 233 ) 13 "11 1 11 3 <(0.1165)II 111 2 .
(8.28)
We then have
t(
{ n f2 ) - exp { - illtll 2 }( ] + 6i 3n
-
1/2µ3( l )) 1 3
=exp{
-
211 1 11 2 }^exp{logo"( t +211 1 11 2 ,"( ( + 6n - ` /2µ3(t) 2
<exp( - 211t11 2 )^exp{loge"(
1 n `/Z
1+loge"
+exp{ - 2IItI1 2 }iog&"(
`/2
+ U1111 2 }
(n 1 J2 J
)+iiItI{ 2- 6n -l/Zµ3(i)^ z
r2
<exp{ - 2IItII2} 2^logG"\n`/z)+iIItII2^ exp^^logG"t n'/z)+ II 21 +exp{ - ?IItI1 2 }[( 0 . 1325 )n - `+2 1°,"^Iltli ° <(0.0272)l "IItII 6 exp { -0.383511tf1 2 }
+ {(0.1325)n - `+ 2 1 °,"] II t II 4 exp ( - 1 II t II') .
(8.29)
Expansions of Characteristic Functions
64
Ixl 2el`I in (8.29) [as well as the We have used the inequality Iex — 1— x estimates (8.27), (8.28)]. This proves the theorem for V= I. In the general case look at the random vectors BX I ,...,BX,. Q.E.D. THEOREM 8.6. Let X 1 ,...,X, be n independent random vectors in R k having distributions G 1 ,...,G, respectively. Suppose that each random vector Xi has zero mean and a finite fourth absolute moment. Assume that the average covariance matrix V is nonsingular. Also assume
l
1.
(8.30)
<21,.n'j°,
(8.31)
4
Then for all t satisfying
IltII one has n
3
III G( BI Z )exP{
—
iIIrII Z }( 1 + 6n -112µ3( 1 ))
I -1
(0.175)14,,IItlI 4 exp { — i IItII'} + [(0.018)l ,.111118+3613,, IItll6]exp{—(0.383)IIt1I2}, (8.32) where B is the positive-definite symmetric matrix defined by (8.15), and E
µ3(t)=n-'
J- 1
Proof. We may, as before, take V= I. Note that [by (8.14) and (8.3 1)] i
Iz ( I/Z)-1 <(2n)-IE2<(2n)-1(E. 4)
n
n
1/2
<(2n) -1 (^ E
(8.33)
J-1
in the given range of t. Proceeding as in (8.28), one has to g•^ n G. J —1
- ^ + 2II ! tII i <(0.1325)n2
( 1/2)
n
Z2+ 1 ( Et,X. ^ J^) 6 13.n
Iltll 3
j-1
<(0. 1325 ) 14.nIltll 4 +( 6) 13 ,,.11111 3 <(0. 117 )IftII',
(8.34)
Approximation of Characteristic Functions
65
using (8.30), (8.31) in the last step. Also, as in (8.27),
log
Gj ( In )+ j IItII Z— 6 n -I ^ Zµ3(t) j=I
<[(0.1325)+(1/24)]14. lull ° <( 0 . 175 ) 14.nll 1 II . (8.35) Hence, as in (8.30), n
I-I Gj(__
I
l
/
—
exp{
-
2IIzII 2 }( 1+ 66
n—I ^ Zµ3( 1 ))
I n
<exp(
— zIItII 2 ) exp I logGj ( __^ Z )+iII 1 II Z —1
n
J-1 n
— Y logGj ( ^^ Z )+211 1 11 2 j-1
n
+exp{
— ill 1 II 2 } 1logGj( 1)+ziitti Z
n
j=1
3 T"— I/2(,) —6
<(0.1 7 5)141I 1 II 4 exp { — z II'II Z } + [(0. 1 325) 21;,,11(11 8 + 13 nlltll b j exp ( — ilI'l1 2 +( 0 . 117 )11 1 11 2 ).
(8.36)
Q.E.D. Note that [see (7.6)] if X^ j is the with cumulant of X i and if XY =n-Ili-,x,j, then
t"
P I (it: (X„})=i 3 I IPI-3 v
n
n
=n
n
-I (t3
j 1
l t^^=n v
1^I= 3 j
3
3
(
1
3
31X3.j(0
II-3
=n—I1 2 31 µ3,j( 1 )) — 6 µ3(t) (+ j-1 Iv!-3
66 Expansions of Characteristic Functions
the equality X3 j (t)=tb j (t) being a consequence of the fact that X 3f (t), 53J (1) are, respectively, the third cumulant and third moment of the random variable (t,X1 ), which has zero mean [see (6.21)]. THEOREM 8.7. Let G be a probability measure on R k having zero mean, nonsingular covariance matrix V, and a finite third absolute moment. For all t satisfying (8.37)
11111 <2 h / 2I3. "', one has
G"(
2
B1
Iexp1 — (2 -262 )Iltll 2 }, )<
(8 .38 )
where B=B', B 2 = V Proof. First take V= I. By Taylor expansion, if X has distribution G, G( 1 n )=]-(2n)- 111,1 + 6n-3/2EI
(8.39)
where I0I < 1. Hence, noting that I- (2n) -1 II t II 2 > 0 in the given range oft,
G ( ni ^ 2 )1< 1-(2n)- I II t11 2 + sn-1/3."Iltll' <exp(—(2n) -1 11 1 11 2 +In - '/3.,,IlIII'} <ex p
(1 _ -I i -
2 1 /2 ) - --
1, ) --;;n-Jj -;;
(8.40)
which proves (8.38), when V= 1. For the general case, look at the distribution of BX, where X has distribution G. Q.E.D. Before proving a similar result for non-i.i.d. random vectors, we need a simple lemma. LEMMA 8.8. Let X and Y be two independent random variables (in R') having the same distribution. If this common distribution has mean zero and a finite third absolute moment, then EIX-YI 3 <4E1XI 3 .
(8.41)
Approximation of Characteristic Functions
67
Proof. Since IX
—
YI 3=(X—Y)'IX—YI <(X2-2XY+Y2)(IXI+IYI)
= IXI 3 + IYI 3 + XZIYI + YZIXI — 2XIXIY — 2YIYIX ,
(8.42)
and E(XIXIY)=(E(XIXI))EY=0=E(YIYIX), one has E IX — Yj 3 ' 2 E IX1 3 + 2(E X 2 )E IYI = 2 E IXI 3 + 2(E X 2 )E IXI < 4EIX1 3 .
(8.43)
Q.E.D. THEOREM 8.9. Let X 1 ,...,X, be n independent random vectors in R'` having distributions G,. . . ,G, respectively. Suppose that each X^ has zero mean and a finite third absolute moment. Also, assume that V = n - '2j Coy (Xi ) is nonsingular. Then for every 8 E (0, Z ), n
1G;( .i —
Bt2
) <exp {-31I1112}, (8.44)
for all t satisfying Ilt11 (—&)l,,'.
(8.45)
Here B=B', B 2 = V -' . Proof We may take V=I. In this case, for each j, I < j < n,
z
1 —n-iE(t
Xl \2+3n-3/ZEI(t
n
,X^>I3
C exp { — n -' E ( t, X1 >2 + 3 n - 3/2E I (t, X^>I 3 } , (8.46) since I G^(ut)I 2 is the characteristic function, evaluated at u, of the random variable— , where Y1 is a random vector independent of and having the same distribution as X^; also, by Lemma 8.8,
E[— <',y>] 2 =2E 2 , EI — l 3 <4EI 3 .
(8.47)
68
Expansions of Characteristic Functions
Multiplying both sides of (8.46) over j= 1,...,n, one gets „
2
llG( n112/ exp{
-
IItII 2 +3/s,„Iltll'}
JI
exp{—I1,112+3(i—S)IItII2}
=exp{ — 3s IItII 2 } , (8.48) in the given range (8.45). Q.E.D.
9. ASYMPTOTIC EXPANSIONS OF DERIVATIVES OF CHARACTERISTIC FUNCTIONS We first state and prove some preliminary lemmas. LEMMA 9.1. Let g be a complex-valued function on an open subset 2 of R k having continuous derivatives D"g for 'vj < m, where m is a positive integer. If g has no zero in Sl, then on Stt (D^'g)...
(DP'g)
g where the summation is over all collections of nonnegative integral vectors ($',... /P) satisfying '
f3'+... +13 1 =v,
1 6 j
[I
and the constant c({ $I,...,$3)) depends only on the collection {/3 . ,...,/3i). Proof. The result is obviously true for I vj = 1. Assume that it holds for all v satisfying 1 < m, where m is a positive integer. An immediate computation shows that the result holds for all vectors v + e,, I i < k, where e, is the vector having one in the ith coordinate and zeros elsewhere. Thus the result holds for all v satisfying I < M < m + 1. Q.E.D. LEMMA 9.2 (Cauchy's Estimate). Let f be a complex-valued function on B(a: R)= (z=(z 1 ,...,Z k )EC k : . I lz ; —a ; 1 2 . R 2 )[where a=(a I ,...,a k )EC'` tThis is a local result and is valid whatever the branch of the logarithm chosen.
Expansions of Characteristic Functions
69
and R > 0] given by the power series [absolutely convergent on B(a : R)] J( z ) = I c(v)(z—a)'
[Z—a=(z1—a1....,Zk—ak) .
v
where the summation is over all nonnegative integral vectors v E (Z + )'`. Then (D"f)(a)I
for every r E (0, k -' / 2 R], where M r is defined by
Mr =sup (If(z)I : Iz —a,I =r, l < i < k}. ;
Proof. Since the cube (z:Iz; —a; {
(—Ir
1
Clearly, for all v,
rtc(v)=(27r)-k
(' J[
(8......°k )dO 1 ... dOk . k exp { — i} g r
Noting that the integrand is bounded above by M r , one has I(D"f)(a)I=v!Ic(v)I
Q.E.D. For x,y in R k we define
x
ifxi
(9.1)
We also write x
(9.2)
where the summation X * is over all collections of nonnegatire integers
70
Expansions of Characteristic Functions
(j p : 0< /3 < v) satisfying (here Eli's are integral vectors)
(R:0<$(}
and c((jp :0< /3 < v}) depends only on the collection {jp :03 < v}. Also, H denotes the product over all /3, 0< /3 < v.
Proof
Let e J =(I,0,...,0),...,ek =(0,0,...,0,1). Then D`'(exp{f })=exp{f }(D`f)
(1
Thus the assertion is true for all v with Iv I= 1. Suppose that it is true for all v with II < m. Given v with I vI = m, one has, for all i (I
D" +e, (exp{f))=exp{f)Z*c((jp:03
.Ip(D#.f)'°'-I (D p.+ef)^"(Dsf)'a I { p':je •>0) 1
+exp(f )(Df)Y,*c((jp :0<$3. Then there exists a constant c 2(s) depending only on s such that J(D"log G)(t)I < c2(s)p5
(9.3)
for all t in R k satisfying
G(t)—lI<2
(9.4)
and for all nonnegative integral vectors v satisfying I vI = S. Proof. Note that loge is defined on an open set containing the•set of is satisfying (9.4). Also, if /3 j ,...,/3^ are nonnegative integral vectors satisfying /3 i +... +/3^=v,
Expansions of Characteristic Functions
71
then (D''G)()I
... I(D1 6 )(t)I <(EIX 'i)... (EIX4°ij) '< PIQ.I
.
..
PI $I
PcIs I+... +I s,U/s =P
J
,
(9.5)
by (6.29) and Lemma 6.2(ii). The proof is now completed using Lemma 9.1. Q.E.D.
For the remainder of this section we shall consider n independent random vectors X 1 , ... , A. with respective distributions G i , ... , G. We assume that Xi has mean zero, covariance matrix VV , and a finite sth absolute moment for some integer s > 3. We write n
PrJ =E II XIII ', Pr=n—'
(r>0),
Pr i—I
n
= with cumulant of X^,
Xtl = n -
X„
(9.6)
[PE(z+)k, 0
V^,
V=n ^ -
l -1
unless- otherwise specified. In case V is nonsingular, let B denote the symmetric positive-definite matrix satisfying B 2 =V - '
(9.7)
and write n 1,=fl'
2 EIIBX;Il r (r>0).
(9.8)
J -1
The constants c's that appear below depend only on their arguments. LEMMA 9.5. For every nonnegative integral vector a satisfying 0 < lal <3r, and for r=0, I,...,s-2, I D°Pr( z: {X„})I
(zEC k ). (9.9)
72 Expansions of Characteristic Functions
Proof By (7.3), D"Pr (z: {X,))= 2 _ I 1
m-1
+ ym)1
(v1+.
X
2" l... v 1
j,...,j_
(v1+... +Pm
—
a)!
''
z
+...+r.-a
(9.10)
where I• denotes summation over all m-tuples of positive integers (j1,. .,j,,,) satisfying
(9.11)
j1= r, r-1
and 1" denotes summation over all m-tuples of nonnegative integral vectors (v 1 ,...,vm ) satisfying (7.5). By Lemma 6.3 and averaging, IXr i
...
I < C 1( v 1) = c1(vl)
. .. C
.
i( Vm)Pj i +2
...
PI..+ 2
PZ /
r
...
= c1(v1) .
P1_ +2
.. CI(vm)Pi(j,+2)+••• +(j+2))/2 Pj,+2
(i^+
PJ
..
p2 „+2)/2
. +j..)/(s - 2)
cl(Vm)P^2+m^\ P2/2
..0
(vm)Pz
-r/(s-2)pJ/(s-2).
(9.12)
We have used Lemma 6.2(iii) and the fact that p ; is the r'th absolute moment of n -1 (G I + • • - + G,;), where Gj is the distribution of Xj . On using (9.12) in (9.10), the desired inequality (9.9) is obtained if one notes that
Ilzll m
(O<m'<m<m", zEC k ).
(9.13)
Q.E.D. LEMMA 9.6. Let g be an absolutely convergent power series in z E Ck on B(0: r)= (IIzII
[zEB(0:r)],
(9.14)
where h is a nondecreasing function on [0, oo). Then for every c > 0, J(D rg)(z)l < v!c H " h Ilz11 - "h(( 1 +k l/2c)IIzII)
(9.15)
Expansions of Characteristic Functions
73
for all nonnegative integral vectors v and for all z satisfying (9.16)
IIZII < r (z EC k — (0)). (1 + k'/ 2c)
Proof. By Cauchy's estimate (Lemma 9.2) I(D'g)(Z)I
< v!(clizll) -I ' I sup {I g(z')I : Ilz'II <(1 +k'" 2c)Ilzll } G v!(cllzll)-I"lh((l+k'/2c)IIZII) (9.17) for all z0 satisfying (9.16) if one takes (in Lemma 9.2) a = z, r = c II z II and notes that [for z satisfying (9.16)]
(Z':Iz
—
z,I=c11z11,l
C{Ilz'II
Q.E.D. LEMMA 9.7. Let s be an integer not smaller than 3 and let (y : v E(Z+)k, M < s) be real numbers. Define
X.,=(max{IX,.I"/''
2)
:3
s-3
1
(9.19)
l
c(s,k)= 7 }. r=i M=r+2 v JJJ
Then for every u ER' — {0) and all z satisfying IIZII <(I lc(s,k)iuI /31/(s-2))-1
(9.20)
one has, in the notation of Section 7, s-3 X
s-3 _
r+2( Z)
D exp 2 ( r+2)! r-1 '
ur
— I 'r r-O
—
J -2)— ^ -1
)exp{911zI1 2 ),
(9.21)
74
Expansions of Characteristic Functions
where c'(s, k) depends only on s and k, and D is the ath derivative with respect to z = (z 1 , . . . , z k ), a being any nonnegative integral vector satisfying jaj<s.
Proof Write [see (7.1)] 3-3
g(u:z)=
urXr+2(Z) s-3
r-1 (r+2)!
► —I
v
r
Irl—r+2 P
f(u:z)=exp(g(u:z)},
(9.22)
s-3
h(u:z)=f(u:z)— Y, u'PP (z:( ))
(uER I , zEC").
r=0
By definition of the polynomials P,,
dU
'" h(u : z)I
=0
for m=0,1,...,s-3,
u^0
du'
2
h(u:z)= d ' 2 f(u:z),
when h and f are regarded as functions of the first argument u only. By Corollary 8.3,
S
h(u:z)l< ( I IS 2)I sup{ I da ' z f(a:z)I:0
(ds -2 /dus -2 )f(u:z)
d
f,
(9.23)
is a linear combina-
j.2
ds-2
f(u:z)(du g(u:z)) ... (dus-2 g(u:z))
,
(9.24)
where jl,...,js-2 are nonnegative integers satisfying s-2 2 mjm =s-2.
(9.25)
M-1
If z satisfies J/(s_2) }
Ilzll <(8c(s, k)^uI
-
1
(9.26)
Expansions of Characteristic Functions
75
then, since [by (9.17)] QI'I-2)/(J-2)
IX.I
(9.27)
one has s-3
d m2 r(r— 1)... (r—m+ 1)( m 8(u:z)I = du r=m M=r+2 s-3
Zt )u'-"'
c t (r m k)a Jr J-2)IUIr-m.IIzIlr+2
r=m
s-3
< E c1(r , m , k)(aJ
t/(s-2)
r-m
IuIIIzII) r-mam/(J-2) IIZIl m+2
< c 2 (s,m,k) /3m/(J-2)IlzIIm+2
(1 < m < s-2), (9.28)
so that d (
J.-2
JJ-2
ii
g(u:z)) ... (
d4us-2
g(u:z))
du
< c3(s,k) aJ (IIzIIJ+ 11z11 3(3-2) ).
(9.29)
Also, if z satisfies (9.26), then 3-3
Ig(U:Z)I< 2
2
1 as/(s-2)IIZIIr+2luv
r= 1 1vI-r+2
s-3
<
IIZI1 2 2
1
11
r=I M=r+2 Y '
s-3
<
IIZI1 2 Y.
2
r- I M=r+2
r (IuIIIZII ps t s -2) )
1- (8c(s,k)) - r v
IIZ112 J-3 (gC(S'kI' r=1
I
II
r+2
IIZI1 2 8' ^f(u:z)I<exp{sllzII 2 }.
(9.30)
76
Expansions of Characteristic Functions
Hence, by (9.23), (9.24), (9.29), and (9.30), Ih(u:z)(
118112 (9.31)
for all u 0 and all z satisfying (9.26). One can now use Lemma 9.6 with r given by the right side of (9.26), c=(3k ! / 2 ) - ', to obtain — lal
)D°h(u : z)l <
c4(s , k)I uj s
a!(31k'I/2)
x /3 [(IIIzll) +(311zII)'
-2
(s-2)
$
jexp(9JIz11 2 )
(9.32)
for all z satisfying (9.20), since the right side of (9.20) is smaller than r/(l +k'/ 2c) =;r. Q.E.D. If V= I, then by Lemmas 6.3, 6.2(iii), one has (since p 2 = k) IXYI' /(I ' I -2) <(c1(v)p11) < ci(v,k)p,'
'/(IVI -2) /(s -2)
(3 < Ivy < s).
(9.33)
Taking these x's and u = n -' / 2 in Lemma 9.7, one has LEMMA 9.8. Let s be an integer not smaller than 3. Define (9.34)
c6 (s,k)=(l lc(s,k)max {cj(v,k):3 G 1v <s}) - ' There exists a constant c7 (s, k) such that for all t in R k satisfying
Il t11 < c6(s , k)n `12ps-
U(s-2),
(9.35)
one has, for every nonnegative integral vector a, 0< a! <s, s-3
DQ exp
— Iltll2 + 2
x +2 (tt) n_ /2
E (r+2)! r=1
—exp — 1I21z
' s-3
)J r=0
n-'/ zP,(it: {X,)) C7(s,k)ps
(
I.
111112
(Iltlls",al +I(tll 3(s-2) +lal )exp (— 4
77
Expansions of Characteristic Functions
Proof. The assertion follows from Lemma 9.7, inequality (9.34), and the following:
(i) D a (exp { - I ^ 2 12 } -h (n 1/2: it)) may be expressed as a linear combination of terms like t 2
(DRh(n -112 :it)) D°-13exp{- II2I })
(ii) also
(0<^13
l
D a—s exp
(— II221 <
c'(a
—
1 +IItIlla —R I)exp { — 2 12
fl,k)(l
Here h is the function defined in (9.22). Q.E.D. We are now ready to prove the main theorem of this section. Before stating it, we define d„ = sup { a >0:< a 2 implies G/
l
(
-s) n
- 1 I
))
(9.36) THEOREM 9.9 There exist constants c 8 (s, k), c 9(s, k) such that, for all t in
R k satisfying
Iltll < d, II 1 11 < c 8 (s,k)n'' q, '/(s-2'
(9.37)
one has, for all nonnegative integral vectors a, 0 ( IaI < s, n s-3
DI
Bt 2
) - exp( - il1r11 2 ) Y, n -r/2Pr(iBt:{X,.}) r—o
n /
—c
s -2)/2
[IItIIs —I a I +IIt11 3( 2)+I « I ]exp{
—
elll11 2 )• (9.38)
Proof. First assume that V= B= I. In the given region logarithm of G^(t/n'/ 2) is defined. Write s-3
h^(t)=logd (n .7 )-
- 2n llthI2+
It
(r+2)!) n-(r+2)/2 r—^
n
h(t)=
h/ (t), f-I s-3 n—r/2Xr+ 2,j( it )/( r+2 )
^( t ) — 2bhthI 2 +
n
_
_ n
logGj ( -_f. j ) - h(t).
i
!
r—1
1
(9.39)
Expansions of Characteristic Functions
78
We want to estimate I
n
D
G
t exp {+^(t)}
j=1
= Da[(exp (h(t)} —1)exp (i(t)} ] _ E c lo(a,$)(D'exp {^(t)})[D°
-,
(exp {h(t)} —1)]. (9.40)
0< /3Ga
By relation (9.30),
< 1121 2 + 11812 =Blitll (9.41) 2 if t satisfies (9.37) [see (9.27)]. Also, using (9.33), ( 1 -3 j(D II )(t)I = D O
=
I n'
r=0
^tv M=r+2v
s-3 C^
n—r/2 C` X' .
0,I$I-2? r=max{G
t ► —^
11=Gr+2,(v—/3)!
v>fi
s —t3 ^
<
G r=max(0,((i1-2}
n r/2 cii(v+k)Ps/(5-2)Iltlr+2—l^^ lvl=r+2 v>(3
s-3
(n-1/2c8(s,k)nl/2ps 1/(s-2))
E
/(s-2) g 2-IPI
r— max (0,1#1-2)
(a>0, 1ER' —(0)).
(9.42)
Let j l ,.•.,jr be nonnegative integers and /3,...,/3, nonnegative integral vectors such that jiai=a,
$>0
(1
Expansions of Characteristic Functions
79
By (9.42) one has [using (9.13)] (Dp (t))J1...(DR'p(t))i'I 1 i,(2-IR,I> ,
(two).
(9.43)
It now follows from Lemma 9.3 and inequality (9.30) that
ID'exp {^'(t))I < c14(s , k)(IItII 2-I,8I + IItII IRI )Iexp ('P(t))l < c3o(s , k)(IItlI 2-IQI + IItII I1I )exp { — i 11(11 2 + s IIlII Z } =cia(s,k)(Iltll 2-IQl +lltll' Q ')exp f —IIItII2) (t
0)
(9.44)
if t satisfies (9.37). Next note that for any nonnegative integral vector /3 satisfying 0 I /3 s,
D,6'(D'h^)(0)=0
[0
Hence, applying Corollary 8.3 to g DOhJ , one gets D'Qh (t)I < ;
ItRI
2 sup {I(Dag)(ut)I:0C 1). IR'I-s-I/3 a '
(9.45)
But if I /3'I = s — I /31, then by Lemma 9.4 [note that (9.4) holds for II t II < d„],
j(D O 8)(ut)I =I (D$+R)(ut)I =I (D$+R • log G^)t
nut )I n-s/2
< n —s12c2(s)PsJ,
so that summing over j = 1, ... , n, in (9.45) one has E 1 1 s-lel.(Pi-s-Ifil n a•J
ID ,8h( 1 )I < ASA III
(9.46)
In particular, taking /3=0 in (9.46) yields i h(t)I < cz(s)(^
I$'I=s Q
9.47) l( -,) (.Pz>^i Iltll° < II g n
80
Expansions of Characteristic Functions
for an appropriate choice of c g (s, k). If a - /3 = 0, then [using both inequalities in (9.47)] ID° Q (exp {h(t)) -1)( = (exp {h(t)) -11 < (h(t)Iexp {lh(t)I) -
$ (I Iz }.
(9.48)
If a>/3, then D° - O (exp {h(t)) -1)= D° - " (exp {h(t)}),
(9.49)
which is a linear combination of terms such as (Da'h(t))''... (DRh(t))''exp { h(t)),
where 2; _ 1 j; /3; = a - /3. By (9.46) [and (9.13)1,
(DF'h( t ))i'... (DRh(t))i'l
c ► 6 s k 11 (I Vc s-2)1
n
(s - 2)(EJ, - 1)
,
(s
P [ n /2 - 2)/2
II tII5-2+2±Jl-la-13I
C17( S,k)
ps(IItlls—ja—Ql+ 1(II( sla—PI-2 )
(9.50)
Hence if a> Q, then
ID° a (exp{h(t)) - 1)I -
c's( s, k) ps(Iltll'-IQ-fI n cs-z)/z
+Iltllt+la-BI+2)exp{ Il8 I.
(9.51)
Using (9.44), (9.48), and (9.51) in (9.40), one obtains
D a (n n G t ex l/z) p i=1
Iltll z + s-3 n / J4+z( !t)
2 E -^ .-1
c19(s k)P, ,
(Iltll
Ia l
2
+Iltl l
(r+2)i
l
3+al+z
Iltll2 4 }• (9.52) )exp (-
}
Expansions of Characteristic Functions
81
Now use Lemma 9.8 to complete the proof when V= B= 1. If V r 1. look at random vectors BX 1 ,...,BX n and observe that n
II J=^
Gi \
z l
is the characteristic function of Z n = n - '/ 2 Y. where Y n = B (X, + • • . + X n ). Also, if the (r+2)th cumulant of the random variableis denoted by X,+z,l(t), then the corresponding cumulant of = is Xz(Bt). Q.E.D. The following theorems are easy consequences of Theorem 9.9. THEOREM 9.10. Let G be a probability measure on R k with zero mean, positive-definite covariance matrix V. and finite sth absolute moment for some integer s not smaller than 3. Then there exist two positive constants c 20(s,k), c 21 (s,k) such that for all tin R k satisfying
IItII < c20(s,k)n1/zns
z'
one has, for all nonnegative integral vectors a, 0 al < s, s-3
D ° Gn( ^^z) —ex p{ - 111 , 11 2 } E n—r/2Pr( iB1: (Xv)) r=0
cz,(s,k)71S s lai ncs—z)/z ^Iltll
+Iltll
3(s—z,+iaj ]ex{ — Ill
4 ^.
Here B is the symmetric positive-definite matrix satisfying (9.7),
-s= f
I1BxII-V(dx),
and y,, is the with cumulant of G. Proof Note that if i satisfies 1 1 11 < n'/z^ls 11(s-zt
then Bt, VBt ) t 2 2n = I1I G^ Bi z )—1 < 2n
^z,
Expansions of Characteristic Functions
82
because of the relations fIBxjp 2G(dx)=k<[fit BxIJ5G (dx)]
THEOREM 9.11. Let X 1 ,.. .,X be n independent random vectors in R k with zero means, covariance matrices V i ,...,V. (at least one of which is nonsingular), and finite sth absolute moments for some integer s not smaller than 3. Then there exist two positive constants c 22 (s, k), c 23 (s, k) such that for all t in R k satisfying (s-2)/s
ni/Z
11111 < cZZ(s k)(, 1J( / s-2))
one has for all a, 0 6 IaI < s, E ( ex j . iB!
Da
X^> )) —
Ilt2I `3
r
2_
))
exp — 2 J ^n -1 Pr(i 81 :{X^ r=0
111112 ,ai acs—z>+iQi------j }exp^ , +I{t{I n (z))z [lltll
c23 s ,k s j
where the notation is as in Theorem 9.9. Proof. As in the proof of Theorem 9.9 (see the concluding observations), it is enough to prove this theorem for B= V= I. In this case, for all t satisfying n'/2
Iltll
11u/(52) •j
one has t E
Gilni/z/-11<
> Z IItII ZE(IX II Z ;
2n
<
2n
IItl12(EllXilis)21s 11,112(n',ns)Z/s<2. 2n 2 n
Q.E.D.
83
A Class of Kernels
Before concluding this section, we state another theorem that may be easily proved along the lines of Theorem 9.9, by taking one more term in the Taylor expansion. THEOREM 9.12. Under the hypothesis of Theorem 9.10, D°
/ t 2 s-2 Gn BI Z - )—exp j (Ii 1 2 n-'"2P,(iBt:{X,))
`
ns(2)/2
I r=o
[Iltll
s-ICI
+Iltll
3(s-2)+ l a l]exp
{ _ 1^ 12 t
(9.53)
where S (n)—*0 as n—*oo. In fact, (9.53) holds even for s)2, al 6 s. Remark. Under the hypothesis of Theorem 9.11, one may also prove that if i/2 ) (
1111
2)/
(9.54)
/(,-2)
where A is the largest eigenvalue of V, then R
s-s
D. fJE(exp{<— X/ ))) -exp{-l)•± ro n - '/ 2P,(it:(x.)) ^
j-1
Aldl/ , k) ^2^( sR` < 2
1<: yt>t:-1=I)/2 ,
n ( ' -2) / 2
+
Vt>(3(,-2)+laly9exp
(
- i
(9.55)
For a=0 this follows by replacing Bt by t in Theorem 9.11. The general case follows by induction on al. Note that the derivative D" is with respect to r. Completely analogous modifications hold in the statements of Theorems 9.10 and 9.12.
10. A CLASS OF KERNELS
For a >0 let UI _ a , al denote the probability measure on R' with density
ul-Q,,)(x)= a 2 =0
for —a<x6a for lxl>a.
(10.1)
84
Expansions of Characteristic Functions
is called the uniform distribution on [ – a, a). One has
The measure U1 _
O(_
Q
, a J(t)
=Za f costxdx= 0
a
a
sp a ' (tERl).
(10.2)
The probability measure
To=
(10.3)
U`i_a/2,a/2l
is called the triangular distribution on [ – a, a]. It is easy to show that its density is
Q (1– ad l )
t,(x)=
for IxI
(10.4)
for lxl>a,
=0 and that
at
sin — I a t 2 (tER'). T a (t)=
(105)
2 One can write
c(m) =(f
/Ri
I
sinx I m dx) (m =2,...). -'
X i
(10.6)
For a >0 and integer m > 2 let G,.,,, denote the probability measure on R' with density g,
m
(x)=ac(m)I sin x ax
I
m
(xER').
(10.7)
It follows from (10.2) that for even integers m> 2 sin at _
--
at
m
^ [- a . a
l(t)
(t E R'),
(
10.8)
so that by the Fourier inversion theorem [Theorem 4.l(iv))
d.,.(t)=21Tac(m)u*
=O
if
f tI > ma.
.. a) (t)
(t ER'), (10.9)
A Class of Kernels 85
Let Z I ,...,Zk be independent random variables each with distribution G 112,, 2 , and let Z = (Z,, ... , Z, F ). Then for each d>0 Prob(IIZII>d)
\.
(10.10)
Thus for any a in (0, 1), there exists a constant d depending only on k such that Prob(IIZII> d)< 1—a.
(10.11)
Note that the characteristic function of Z vanishes outside [ — 1,1 ] k . Now let K j denote the distribution of Z/d; then K,((x:lIxll> 1))< 1—a, if t12[—d,d^ k
K 1 (t)=0
(10.12)
One thus has THEOREM 10.1. Let r be any given positive integer and let a E (0, 1). There exists a probability measure K, on Rk such that
(i) K1((x:IIxII> 1))
one has
f I x"I K (dx) < I
oo.
(iii) K 1 (t)=0 for t [ — d, d ) k where d is a constant depending only on a and r. We also need as kernels certain probability measures having compact support and fast-decreasing characteristic functions. To this end we prove THEOREM 10.2. Let u be a real-valued, nonnegative, nonincreasing function on [1, oo) such that
uj t)
dt< oo.
(10.13))
Then for every 1 >0 there exists a probability measure K on R 1 satisfying (i) Support of K[
—
1,l]
(ii) IK(t)I = 0(exp (— Itlu(ItI)))
(ItI—>oo).
86
Expansions of Characteristic Functions
Proof. Define a nonincreasing sequence of nonnegative numbers (a,: r> 1) by eu (ro)
a,=
1
ro
_
eu(r) r
,
r>ro,
where ro is a positive integer chosen to satisfy
00 O° u(r) I a,-eu(ro)+e 2 1. r-1 r r.- rp+ l Let the probability measure K be defined as the infinite convolution K= Ut _ a „ o ,I« UL _ az,Q=J• .. .
Clearly, support of K is a, r-1
r-1
Also,
si na,t r-1
For every positive integer
1
(tER ).
at
K(t) s> ro ,
'
therefore,
S
IK (t)I < 11 ^7 _ tl
< l a!tl' s - `eu(
)It)) • '
Now choose s- [Itlu(Itl)], the integer part of Itlu(ItI). We assume without loss of generality that IiIu(ItI)—'oo
as Itl—+oo.
[For otherwise one may replace u(t) by u 1 (t)= u(1)+(t+ 1) -1 / 2 and note that (10.13) holds for u if and only if it holds for u 1 , and that (ii) holds for u if it holds for u 1 (since u 1 > u).j Thus for sufficiently large Its, one has
A Class of Kernels
87
ro < s < I tl so that lItIU(ItI)J
IK(t)I <( eu(s)Itl ) < Itlu(Itl)
(ltI11(ItI)I
I
<
)
eu(itl)Iti
Remark. The above result is due to Ingham [1], who has also shown that the above theorem provides the best possible rate of growth for the characteristic function of a probability measure with compact support. To be precise, he has shown that if u is a real-valued, nonnegative, nonincreasing function on [1, cc) such that f r[u(r)/:Jdt-oo, then there does not exist any probability measure with compact support whose characteristic function is 0(exp(—Itlu(I:I))) as Itl— ► oo. Note that this also follows from the following result of Wiener and Paley: If fE L 2 (a, oo) then f
J
°O 00
IiogIf (t)II dr
(10.14)
1+r 2
THEOREM 10.3. Given a nonnegative, nonincreasing function u on [1, co) satisfying (10.13), and a nonnegative integer s, there exists a probability measure K on Rk with support contained in the unit ball S= (x: IIxii < 1) such that
ID°K(t)I < c(s,k,u)exp
— k t^lu(It,l) -i
( 10.15 )
for all nonnegative integral vectors a=.(a,,...,a k ) satisfying 0
Define M, = M. Then
M,((x:xER',(xi
M(t)=(M(t)) 3+ '
(tER'),
*See Paley, E. A. C. and Wiener, N. [1], Theorem XII, pp. 16, 17.
88
Expansions of Characteristic Functions
so that for 0 < r < s,
dt'Mt(t)I <(s+ 1)'EIXI'IM(t)I
t-.
<(s+1)'k"'' 2 (s+1) 'IM(t)I
< c 2 (s,k,u)exp ( — Ittu(I1I)), where X denotes a random variable with distribution M. Now define K on R k as the product measure M, X M, x • • • x M t . Then
K({x:xER k ,Iixll<1}) > K({x:x=(x l ,...,xk ),Ix; )
and jD aK(t)I — ^ dt ^ , Mt( 1 1)
1
...
f dtk Mt(tk)I k
It;Iu(It;I)
(t=(tt,...,tk)ERk),
for 0 < Iai < s. Q.E.D. COROLLARY 10.4. For any positive integer s, there exists a probability measure K on Rk satisfying (i) K((x: Ilxll < 1))= 1, (ii)for all nonnegative integral vectors a, 0< Ial <,
(D°K(t),
(tER k ).
Proof In Theorem 10.3 take u(1)=t - " 2 on [1,00) and note > IIt11 t / 2 , for all t-(t I ,...,tk )ER k . Q.E.D. NOTES Sections 4 and 5. The material on the Fourier transform and the Fourier—Stieltjes transform reviewed here is fairly standard and may be found in Cramer [3, 4], Chung [11, Feller 131, Katznelson [11, and Stein and Weiss [11.
A Class of Kernels
89
Section 6. The best reference for the inequalities of this section is Hardy, Littlewood, and
Polya [l). Sections 7-9. The idea of expanding the distribution function F„ of the normalized sum of n independent random variables as E,n - 2P,(-d': {X,)) appears for the first time in Chebyshev (1]; later it was investigated independently by Edgeworth [I]. However the asymptotic expansion of the characteristic function of F. was obtained by Cramer (1, 3) (Chapter VII), who used it to give the first rigorous derivation of the asymptotic expansion of F. under the so-called Cramer's condition (20.1) (see Chapter IV). Theorems 8.4-8.6 are refinements (and extensions to R*) of results of Cramer (3) (Chapter VII); analogs of Theorems 8.7 and 8 .9 for k - I were obtained by Laipounov [2]. There are many such refinements available in the literature, for example, Esseen [I], Gnedenko and Kolmogorov [1], and Petrov [1) in one dimension; Rao [I), Bikjalis (3, 61, and Bhattacharya (3) in multidimension. Theorems 9.9 -9.12 are generalizations of analogous results of Bikjalis [6]. "
Section 10. Kernels such as K i (Theorem 10.1) were used in one dimension by Berry [I] and
Esseen [ I ). Theorem 10.2 is due to Ingham [ I ].
CHAPTER 3
Bounds for Errors of Normal Approximation
Our goal in the present chapter is to estimate j f f dQ„ — J f d for a large class of functions f on R k here Q. is the distribution of the normalized sum of n independent random vectors and cb is the standard normal distribution on R'. For bounded f, the error bound is computed in terms of the average modulus of oscillation of f with respect to 4, or the supremum of this average over all translates of f. This is entirely appropriate in view of the characterization of (D-uniformity classes proved in Chapter 1; also, in most cases of practical importance, these moduli can be estimated efficiently. The main tools used for obtaining these error bounds are expansions of the characteristic function Q. and its derivatives as derived in Chapter 2, together with some smoothing inequalities proved in Section 11. In Section 12 the classical Berry—Esseen bound is obtained with an estimation of the constant involved. Section 13 is devoted to estimations of IJfdQ„ — Jf d 4?j for bounded f under the simplifying assumption that fourth moments are finite; the proofs here are not only simpler than those of the general results of Sections 15-17, but also yield numerical values for constants involved in the bounds. In Section 14 we obtain truncation estimates that enable us to derive the main results of Section 15 on rates of convergence for unbounded fs under the assumption of finiteness of third moments. After a short section showing how to deal with different normalizations of the sum of n independent random vectors, in Section 17 we consider a number of important applications of the theorems in Section 15. A final section deals with rates of convergence under the sole assumption of finiteness of second moments. To facilitate comprehension we briefly sketch the main ideas underlying the rather long route leading to the main results. For the sake of simplicity ;
90
Smoothing Inequalities
91
assume that (X.: n> 1) is a sequence of independent and identically distributed (i.i.d.) random vectors with common distribution Q 1 . Suppose that Q 1 has mean zero, covariance I (identity matrix), and a finite third absolute moment p3 . Let Q. denote the distribution of n - 2(X, + • • • + X„). If Q, has an integrable characteristic function (c.f.) Q 1 , then the c.f. Q. of Q„ is integrable for all n. One can then use Fourier inversion to estimate the density h„ of the signed measure Q„ - (D as "
Ilh.II m sup Ih.(y)I <( 21T) -k IIQR $II1. -
(I)
yER k
The fairly precise estimates of Q„ - ' of Chapter 2 (e.g., Theorem 8.4) yield IIh„II= O(n-' "2). Since such a uniform estimate of h„ cannot be integrated over the unbounded domain R k , one may estimate the variation norm II Q. - 41 11 = II h,, II 1 by estimating the integral of IhI over a sphere S of radius 0(log' / 2 n) as Ilh„II, 0 •vol(S), and the integral over the complement of S by the classical Berry-Esseen theorem and a Chebyshev-type inequality. To avoid the loss in precision arising from the factor vol(S)=0(logk / 2 n), assume that Q, has a finite fourth moment and apply Theorem 8.5, adding one term [n ' 12P,(- b: {x,))] to b and later subtracting the contribution from this term, which is of the order n" 2 . In the general case (i.e., when Q, may not be integrable) we smoothen Q, by convolving it with a smooth kernel KK (a probability measure with an integrable c.f., which for small e assigns most of its mass near zero) and apply the above argument to (Q„ - 4)* Ke , with a proper choice of a (depending on n). The smoothing inequalities of Section 11 enable one to estimate the perturbation due to this convolution by K, and one arrives at Theorems 13.2 and 13.3, which express bounds for jf d (Q„ - ( P)I in terms of the average moduli of oscillation wf or wJ and the range wf (R k ) of f. To estimate Iffd(Q„ -4D)l for unbounded f and at the same time relax the assumption of finiteness of fourth moments (in case of bounded f) we compare the measures IIxll'Q„(dx) and Ilxll' (D(dx) for nonnegative integers r in the same manner in which Q„ and 1 are compared above. Because of the fact that for odd integers r Il xll'Q„(dx) does not have Fourier-Stieltjes transforms as well behaved as those for even r, we replace r by ro, where ro = r + I if r is odd and r0 = r if r is even. As above, we smoothen P,o - llxIi' °(Q„-4)(dx) as p o * KK and apply Lemma 11.6 to the density g of v, o * K,, thus obtaining -
Ilv, o *Kell= IIgll1
c(k)
max 1$I—O,k+i IIo
111,
(2)
where g is the Fourier transform of g. The Fourier-Stieltjes transform of v,o is estimated by Theorems 9.9-9.12 and a sharp estimate of II v a * K, II is
92
Normal Approximation
obtained provided f (Ixll'o +"` +'Q,(dx) is finite (which ensures the existence of D flg for fl = k + 1). To relax this last hypothesis, which is rather restrictive, one resorts to a truncation of the random vectors (X.: n> 1) and applies the above procedure to these truncated vectors. The various lemmas in Section 14 allow one to take care of the perturbation due to truncation. As in the case r0 =0, for final accounting (i.e., to estimate the effect of smoothing by Kc ) one uses the smoothing inequalities of Section 11. The main theorems of Section 15 are obtained in this manner. A further truncation enables one to obtain corresponding analogs when only the finiteness of absolute moments of order 2+8 is assumed for some E, 0 < 8 < 1, thus yielding generalizations and refinements of the classical one-dimensional theorems of Liapounov and Lindeberg.
11. SMOOTHING INEQUALITIES
Lemmas 11.1 and 11.4 show how the difference µ— v between a finite measure and a finite signed measure v is perturbed by convolution with a probability measure Kc that concentrates (for small e) most of its mass near zero. Let f be a real-valued, Borel-measurable function on R k . Recall that in Chapter I we defined the following:
(ACRk),
wf (A)=sup(If(x)—f(y)I:x,yEA} wf (x:e)= wf (B(x:e))
(11.1)
(xER k , a>0).
Also define
MJ (x:e)=sup{f(y):yEB(x:e)), mj (x:e) =inf{ f(y): yEB(x:e))
(11.2)
(xERk, e>0).
Note that
wf (x:e)=M1 (x:e)—mf (x:e)
(xER k , a>0).
(11.3)
The functions M1 (- : e), mf (• : e) are lower and upper semicontinuous, respectively, for every real-valued function f that is bounded on each compact subset of R k . Also, wf(• : e) is lower semicontinuous. These follow from {x: M1 (x:e) >c }=
U {B(x:e): f(x)>c)
(cER5,
z
ml (x:e)=—M_ f (x:e)
(xER k , r >0).
(11.4)
In particular, it follows that Mf (• : r), mf (• : e), wf (• : e) are Borel-measurable
Smoothing Inequalities
93
for every real-valued function f on R k that is bounded on compacts. Recall that the translate fy off by y(E R'") is defined by fy (x) =f(x+y)
(xER k
).
(11.5)
LEMMA 11.1 Let s be a finite measure and v a finite signed measure on R k . Let a be a positive number and K E a probability measure on R k satisfying Ke (B(0:e))= 1.
(11.6)
Then for every real-valued, Bore/-measurable function f on R k that is bounded on compacts, f fd(µ
—v)I
(11.7)
where Y(f :E)= max{ f Mf (•:e)d(tt— v) *KE
, — f mf(•:e)d(µ — v) *Ke }, ,
r(f:2e)=max(f (Mf (•:2e)—f)dv + f (f—tn (•:2e))dv + ), (11.8) provided that M f (- : 2e)I and Im f (- : 2e)I are integrable with respect to and Proof. One has
f Mf(•:e)d(µ —v)*Ke = f [ f MJ(Y+x:e)(IL
Y(f:e)>
=f
B(O:) e
—
v)(dy)]Ke(dx)
[ f Mf(Y +x:e)It(dy) f f(Y)v(dy) — f (Mf(y+x:e)— f(Y))v(dy)]K.(dx) —
> f[ f f(Y) u(dY) f f(Y)v(dY) — f (MJ(y+x:e) — f(Y))v (dy) I KE(dx) > f fd(,u f (M1( . : 2 e) — f)dv . —
B(0:e)
+
—
v) —
+
(11.9)
Normal Approximation
94
Similarly — Y(f:e)<
f
=
mf(•:e)d(µ'v)`KK mf(Y+x: e)IL(cô')
B(O:i){
—f
f(Y)v(d')
+ f (f(Y)—m (y+x:e))v(dy)]K.(dx) l
f fd(µ—v)+ f (f—m (•:2e))dv f
+ .
(11.10)
From (11.9), (11.10), one gets Y(f:e)—T(f:2e)G TY(f:e)
—f
(f—mm(•:2e))dv+
< f fd(µ—v)
+
Q.E.D. COROLLARY 11.2 Under the hypothesis of Lemma 11.1, one has
f fd(µ—v) I
(11.11)
+ .
If, in addition, f is bounded and µ(R k
)= v(R' ),
(11.12)
then
f fd(µ v)L< —
,
1(R k
)II(µ
—
v) *KII+
fw (.:2e)dv f
+ . (11.13)
Proof The inequality (11.11) follows from (11.7) and the relation (11.3). To prove (11.13) note that, in view of (11.12),
f (f
— c)d(µ — v)=
7(f — c:e)= max{
f fd(µ
— v) ,w i- c( - : 2 e)=wf(•: 2 e),
f M _,(.:e)d(µ—v)•K , — f m _ j
K
<sup {If(x)—e{: xER' )II(µ— v)•KcII
f c
(•:e)d(µ—v)*KK
Smoothing Inequalities
95
for all c in R 1 . Letting c=2(sup{f(x): xER k }+inf{f(x): xER k }),
(11.14)
and observing that for this value of c wf _ C (R k )=wf (R k )=sup{f(x)—c: xER k } —inf( f(x)—c: xER k )=2sup{lf(x)—cl: xER k },
one obtains (11.13). Q.E.D. COROLLARY 11.13 For every Bore! subset A of R k one has µ(A) — v(A)
fl( t— v)*KII + v+(A2,\A)
ifs, v, K e are as in Lemma 11.1 and if (11.12) holds. Proof. The inequality (11.15) follows from (11.9) with f= IA if one notes that
f M1_112(': 2 e)d(µ — v)'KK2.
(11.16)
Then for each real-valued, Borel-measurable, bounded function f on R k , one has
I
f fd(tt—v)I<(2a-1)-^y•(f:e)+aT'(f:2E)+(1—a)z•(f:e)^ (11.17)
where
y`(f:e)=sup{y(fy:e):yERk }, T * (f:2e)=sup{7 . (fy :2e ):yER k }.
Proof Let 8=sup{ f fy d(µ—v)I: yER k }
(11.18)
% Normal Approximation
Assume first that 8—sup{ rfy d(µ—v):yER k }.
(11.19)
°
Then, given any positive number ,1, there exists y in R" such that f f,.d(p—v)>6-71.
(11.20)
In this case Y`(J: E) > f Mfy,(• : E) d (,u — P)"K,
=
f
fMfy(y+x:E)(µ—v)(dy)IK.(dx)
B (o: e)
+
f k
J
[
Mf.(y+x:E)(µ — v)(dY)}KK(dx) f
R \B(O:e)
> Jf J,'(y) µ( dy) — f Jy .(y) v ( dy) 8(0: E)
— f (M,(y+x:E)
—
fy o(y))v(dy)I K.(dx)
+1 [ f fy4y+x)µ(dy) R k\$(0:e)
-
f fy.(y+X)Y(dy)
— f (Mjy.(y+x:e)—fy4y+X))v(dy)}K (dx)
I &—— f (Mi,.(y+x:E) — Jyo(y))v + (d1')]K.(dX)
> f
B(O:e)L
>+f f
-8—f(Mfr.(y+x:E)—Jy.(y+X))v+(dy)K^(dX) k
\B(0:e)
8(0: e)
+
[s—n
—
JC Rk\B(0:e)
f (MIy.(y: 2 E) — Jyo(y))v (dY)]K.(dx)
—
S
—
T*(f:E)]KK(dX)
[S—T rt—•(f:2E)]Ke (dx)+[—S—T*(f:E)](1—a) >fB(0:e) _ {8—rt—r*(f:2E)]a— [8+r*(f:E)](1—a) =(2a-1)6—ar*(f:2E)—(1—a)r•(f:e)—art.
(11.21)
Smoothing Inequalities
97
Since rl may be chosen arbitrarily close to zero, Y*(f:e)>(2a — 1)8—aT*(f:2e)—(I — a)T * (f : r),
from which (11.17) follows. If instead of (11.19) one has S= —inf( f f,,d(µ—v):yERk },
°
then, given any ,j>0, find y such that — f f,od(µ—v)>S—q.
Now look at — f,o (instead of f,,o) and note that
M_ 1 (-:e)=—mj (•:e),
f
(fl,—mfy ( •:e )) dv+'
=
f (M-J( . :e)
—
(
(11.22)
— f,))dv + (yERk)•
Proceeding exactly as in (11.21), one obtains Y'(f:e)^ — f m10(.:e)d(IL—v)*K >(2a— 1)6 —aT*(f:2e)—(I —a)T*(f:e)—arl.
Q.E.D. We define the average modulus of oscillation w^(e : µ) of f with respect to a finite measure µ by w^(e: s) = f wj (x:e)u(dx)
(e>0).
(11.23)
Here f is a real-valued, Borel-measurable function on R"`. We also define wf (e : µ) as the supremum of the above average over all translates of f, that
is, wf (e:µ)=sup{wfy (e:IA): yER k )
(e>0).
(11.24)
COROLLARY 11.5 Under the hypothesis of Lemma 11.4, one has
f fd ( ,u -v )) <(2a-1) - '[y*(f:e)+awf (2e :v + )+(1 —a)wf (e:v + )] <(2a-1) '[Y*(f:e)+4 (2e:v + )]. -
(11.25)
98 Normal Approximation
If, in addition, µ(R k)= v(R k ), then ' f fd(µ - v)I <(2a- l)_ [Iwj(R ' )II(µ-v)*K,Il +awf (2e:v + )
+(1-a)w/ (e:v + )] <(2a-1) - '[
,1(Rk)II(µ-v)*KII+4 (2e: v)]. (11.26)
Proof. This corollary follows from Lemma 11.4 exactly as Corollary 11.2 follows from Lemma 11.1. Q.E.D.
The final result of this section relates the L 1 -norm of an integrable function g to the L 1 -norms of certain derivatives of its Fourier transform g. LEMMA 11.6 Let g be a real-valued function in L I (Rk ) satisfying f xll k+l jg(x)^dx
Then there exists a positive constant c(k) depending only on k (and not on g) such that f I Dfig(t)I dt. II gill < c(k) IIll max —U,k+l
(11.28)
Proof We assume that Dflg is integrable with respect to Lebesgue measure for 0< 1/30),
and the 2k quadrants E., one for each vector a =(a 1 , ... , ak ) (in R k ) having zeros or ones as coordinates, by
Ea = (x =(x 1 , ... , xk ): x1 >0 if thejth coordinate of a is zero, xj <0 if the jth coordinate of a is one, I <j< k ). Since the Fourier transform of the function k k+l
-1 , x1 g(x) x-1+( 2 ()°
j_ I
is of the form 7, I$1-0,k+I
c(a,$,k)D' s8,
(xER k )
Berry-Esseen Theorem
99
where c (a, /3, k) depends only on its arguments (and not on g), one has g(x)dx ]
Ilgh 1= I If g(x)dx—f a
_
(t+'xlk+')-'(1+lxlk+l) g(x)dx
(f; a
An E,
J
(Rk\A)nE,
An E,
(Rk\A)nE,
k k+1 X
l+( E (—l) a'xj g(x)dx
I.
j-1 2 ^ lk+l + IX ) ( )
1(1r =
( ✓ An E, - ✓ (R k \A)nj (
k
I
c(a,/3,k)DPg(t))dt dx
X f exp(—i)( 1,01-0,k+l
<(f (l+IxI k+l ) - 'dx)c'(k) max f IDfiI(t)Idt. 101-0.k+1
• M 12. BERRY-ESSEEN THEOREM Let P be a finite measure and Q a finite signed measure on R 1 with distribution functions F and G, respectively, F(x)=P((—oo,x]),
G(x)=Q((—oo,x])
(xER'). (12.1)
Let Ks be a probability measure on R' such that
a=KE((—ESE))>i
(12.2)
for a given e > 0. It follows easily from Lemma 11.4 that sup IF(x)—G(x)I <(2a— 1) - 'sup{I(P—Q)+KK ((—oo,x])I: xER'} xER'
+sup{ IG +(x) — G + (y)I : Ix —yj < 2e } ],
Normal Approximation
100
where G + is the distribution function of Q +. However with a more restricted choice of Q and the kernel K f this inequality can be sharpened. LEMMA 12.1 Let P be a finite measure and Q a finite signed measure on R' with distribution functions F and G, respectively. If Q has a density bounded above in magnitude by m, and if K is a probability measure on R' that is symmetric and satisfies (12.2), then f
sup IF(x)—G(x)j<(2a-1) -1 x€R
sup !(P—Q)•K1 ((—oo,x])I+ame. L ERI
J
S= sup IF(x)—G(x)I= —inf(F(x)—G(x)).
(12.3)
,
Proof First assume that xER'
xERI
Given q > 0, there exists x o such that F(xo)— G(x o )< —S+rl.
Then (P — Q)'KK(( — oo , xo — e]) = ([F(xo — e — y) — G(xo — e — y)]K: (dy)
— ✓ B(o:f)[ F(x
o — e—y)— G (xo—r—y)]K1(dy)
+ f[F(xo — e — y) — G(xo — r — y)]K1(dy) R^\B(O:f)
:f) B(o
[F(xo)]K,(dy)+S(1—a)
f
[F(x o )—G(x 0)+m(e+y)]K1 (dy)+S(1—a)
f
[ —S+7i+m(e+y)]KK (dy)+S(1—a)
B(0:f)
B(o:f)
=(—S+71+me)a+m f B(0:f)
—S(2a— 1)+mae+rla.
yK1(dy)+8(1—a)
101
Berry-Esseen Theorem
Hence S(2a-1)< sup I(P—Q)*KK ((—oo,x])I+mat,
(12.4)
xER'
which proves the lemma if (12.3) holds. If, on the other hand, S= sup (F(x)—G(x)), xER'
then given rl > 0, there exists x o such that F(x o )—G(x o )>S—r^.
Then (P— Q)*Ke ((— oo,x o +e])
B(o:e)
[F(xo+e
—
y)
—
G(xo+e
—
Y)]KE(dY)
+ f[F(x o +e—y)—G(x o +e—y)]KE (dy) R \B(O:e)
>f[F(xo)—G(xo)—(e—Y)m]KK(dy)—S(I —a) B(0:)
(S
—
n)IX
—
am€
—
S(1
—
a)=S(2c
—
1)— amt—a,
so that (2a— l)8 sup (P—Q)*K ((—oo,x])+ame. E
xER
1
Q.E.D. LEMMA 12.2 Let P be a finite measure and Q a finite signed measure on R' with distribution functions F and G, respectively. Assume that
f IxIP(dx)
JIx1IQI(dx)
P(R')=Q(R').
If Q has a density bounded above in magnitude by m, and if K E is a symmetric probability measure on R' satisfying amKe ((—e,€))>,
f JKE(t)Idt
Normal Approximation
102
for some e > 0, then
sup IF(x) — G (x)I <(2a— l) ' -
xER'
x [(2 ,ff) - 'f III -l l(p(t)-Q(t))K^(t)ldt+ame]. (12.5) Proof By Fourier inversion, the density of the signed measure (P - Q)
• Kc is (2ir) - ► f exp(-itx)(P (t)-Q(t))K,(t)dt
(x ER t ).
(12.6)
It is simple to check that the function (12.6) is the derivative of the function (27r) ' f exp{-itx}(-ii) '(P(1)-Q(t))K1 (r)dt -
-
(xER5. (12.7)
Note that Ill - ► Ip(t)_Q(1)1=
lit -'(exp{itx)-1)(P-Q)(dx)I
< f IxIiP- QI(dx) < oo.
By the Riemann-Lebesgue lemma [Theorem 4.l(iii)J, the function (12.7) goes to zero as IxI->oo. Thus
1
(P-Q)`K ((-oo,x]) =(27r) - ' f exp{-itx}(-it) - '(P(t)-Q(t))K,(t)dt,
the constant of integration being zero. The inequality (12.5) now follows from Lemma 12.1. Q.E.D. Remark. 1f, in addition to the assumptions on P and Q in Lemma 12.2, one also assumes that they have integrable Fourier-Stieltjes transforms and that flu
-
'l (t)—Q(t)ldt<eo,
(12.8)
then the above argument yields sup IF(x)- G(x)l <(21r) ' f ltl -
-
IIP(t)-Q(t)ldt.
(12.9)
xERl
We shall use Lemma 12.2 to prove the Berry-Esseen theorem below. The kernel K, is the distribution of eZ/3.25, where Z is a random variable
Berry-Esseen Theorem
103
whose distribution has the density g,, 22 given by x-
x sin 21T '[()
x 2
( x ER').
(12.10)
Recall [see (10.9)] that t K(t)=8i/2.z(3.25)=1- 3.25 e
if j t j <3 E 5,
if ItI>5 .2 .
=0
(12.11)
C
By a careful but straightforward numerical integration one also obtains a- K1
(( —
E,E))=
J
(lxIc3.25)
9,,,22(x)dx> 0 . 79 .
( 12.12)
The following lemma will be useful in estimating the constant in the Berry-Esseen theorem. LEMMA 12.3 Let P be a probability measure on R' with zero mean and variance one. Let F denote the distribution function of P and 4 that of the standard normal distribution on R. Then sup F(x) - 4(x)I0.5416. xER
I
Proof. We first prove the so-called one-sided Chebyshev inequality
F(-x)<(l+x 2 ) - ',
1-F(x)<(l+x 2 ) - '
(x>0). (12.13)
Fix x > 0. For every b > 0, one has 1 +b 2 =
f( - b) 2P(dy) > f
(
(y- b)2P(dy)>(x+b)2F(-x), m,
-
xl
so that g(b)=(l +b 2 )(x+b) -2 > F(-x). The minimum of g in [0, oo) occurs at b = x -', and g(x- 1)=(l+xZ)-1,
104
Normal Approximation
which gives the first inequality in (12.13) [note that for x=0, (12.13) is trivial]. The second is obtained similarly (or, by looking at P). This gives F(-x)-(D(-x) 6(1 +x 2 ) -' -(D(-x)mh(x),
say, for x > 0. The supremum of h in [0, oo) is attained at a point x 0 satisfying or
h'(xo)=0,
xo/ 2x 0 (1+xo) -2 = (27r) - / 2e - 2
A numerical computation yields x 0 =0.2135 and h(x 0)=0.5416, thus proving (12.14)
IF(x)-(D(x)i <0.5416
for all x (0 [note that 11(x)- F (x) < .5 for all x (01. The inequality (12.14) for x > 0 follows similarly (or, by looking at P). Q.E.D. THEOREM 12.4 (Berry-Esseen Theorem) Let X 1 ,... , X„ be n independent random variables each with zero mean and a finite absolute third moment. If n
P2-n -1 2 EX^>0, i=1
then sup IF„(x)-c(x)I<(2.75)/ 3 ,,,,
(12.15)
xER'
is the where Fn is the distribution function of (np 2) - " 2(X 1 + • • . + Xn), the Liapounov ratio standard normal distribution function, and l l3.„°( /2 n -1/Z, P3=n- 2 ' P2 p3). .i -
EIXJ I 3 .
If, in addition to the above hypothesis, X 1 ,.. . , X„ are identically distributed, then sup jF„(x)-4?(x)I <(1.6)l 3 .
(12.16)
xER' Proof To prove (12.15) first assume that P 2 =1. For convenience write
( 12.17 ) In view of Lemma 12.3, sup lF„(x)-(t'(x)+<0.5416, xERI
(12.18)
Berry-Esseen Theorem
105
so that we may assume that 0.5416
1n
2.75
<0.l96.
(12.19)
For, if (12.19) is violated, (12.15) reduces to (12.18). Let P„ denote the distribution of n 1 / 2 (X 1 • • • + X„). Take E=;(3.25)1,,,
(12.20)
and let KE be the distribution specified before the statement of Lemma 12.3.
It follows from Lemma 12.2 and inequality (12.12) that
sup IF,,(x)-4?(x)I <(0.58) - ' (2i) ' xER'
( f
(i (t) — a -12/ 2 )
(1:1 (3/2);-i)
t
x (1—;1„ItI)dt+(277) - '/ 2 (3.25)(0.79);1
(12.21)
Write Pt — nO 1= f{III c(3/2)t ^ ') (
e
-
`'/Z
)
t
—
(1-3lnItIdz 1 +1 2 +1 3 +1 4 +1 5 , (12.22)
where i=
f
^i (t)
—
I
13=
J t 1 / 3
? tt)di
,
2t
f
((l/2)1,1
14 = /
dt
t
( I' i < t; "^ }
12
e -1'/2 )
____ (1-31,,ItI)dt,
t
(t)
l
15 =f
( 1 — il,^Itl)dt. -, = /2
e dt.
(12.23)
Normal Approximation
106
Applying Theorem 8.4 with d= 1 and using (12.19), one gets I 1 <(0.36)1„ f t 2 exp( —0.3779t 2 ) dt <(1.3118)/,,. R'
(12.24)
By Theorem 8.9 (with 8 = 1), f 12 < 1,/f
e_t2/3dt < l, 3 f
=3l„/ 3 exp{ — 3/,^
2/3 }
} ^t^e-`2/3dt
/ =3l„(l^ '/ 3 exp( —;l-
2 3 ))
<(l.9320)1.
(12.25)
Again using Theorem 8.8, this time with 8 = Z, we get J3=
P-(t) (l — 2, 11jtD)di
<321„ f (I11>(1/2)t, '}
It)e-12f6dt e-'2/6d1<3(21.)2 f {IiI>(t/2)t, ')
— !^ 2 =161. exp 24 <(1.06)1,,.
(12.26)
Noting now that Theorem 8.9 was derived from the inequality [see (8.48)] Pn(t)I<exp(— 4-
which holds for ItI <21; 1 , one has z exp^— 2 +
14' < 3 lnf =3!„
<
f exp(-3!„t2(21n '—fit()}dt {i '
„ 3
!h
f
{r
<3info
exp — (l
2 ^
'
1—
I
fit+) dt
'}
exp { —( 3 )u } du / l J
= 212 <(0.3920)/,,.
(12.27)
Berry-Esseen Theorem 107
Finally, 15 =
e_12/2
t
< li/3f
Idt
Itle-12dt
= 21,,'/ 3 exp { - 2
2/3
} (12.28)
<(0.8096)1„. The estimates (12.24)-(12.28) are now used in (12.22) to give
(12.29)
I <(5.5054)1,,. Using this in (12.21) one obtains
sup IF„(x)-4(x)I<(0.58) - '[(27r) - '(5.5054)!„+(2ir) - '/ 2 (3.25)(0.79)31„] xER'
<(2.676)1,,.
(12.30)
This proves (12.15) under the assumption p 2 = 1. In the general case, look at random variables Y^=X^/p2/ 2 , 1 <j < n. We now prove (12.16). Again assume that EX= and < 0.54616
In
=0.3385.
(12.31)
As before, we proceed to estimate I and write
IGI^+Iz+I3,
(12.32)
where (Pn(t)—a—rZ/2
1 = j {ltI2'/ 2! -1 }
)
dt,
t
12 f -^ 1/2
I P^I t) I (1-i1^I'I),
I3= f
Ie
— r2/2 {2'/ 21. ' <1r1 <(3/2)1^ }
1
I(I- 2,1ItI)dt.
(12.33)
108
Normal Approximation
Writing
a=E(expS n
1n 11
b=exp{ —2 ? },
X2 I) , 111
(12.34)
one has
I Pn(t)—e-`2/2I=)a"—b"j=ia—bj•lan-I+an-2b+...
+b"-1).
(12.35)
But, for all t, writing p 3 = E IX 1 1 3 yields
la— bi
<exp{ —
(0.26 42)t 2
n
I.
(12.37)
c = 0.2642,
(12.38)
Hence, writing one has, using (12.36), (12.37) in (12.35), If'n (t)—e -= ^ Z I<( 6nI3/Z
+
)exp(— 8n2
n
n— —r
ct 2 -2n t Z } (12.39)
r-0 J
Now n-1
n—l—r C12_ r t 2 1 r—O
p n
2n = exp { ( 2n — c)1 2 }
exp { — (i — c) l2 n }
ll r=1
f1e-(=-`)`'"dx
` (2n —c)t2111I 0
r =nexpj
` (2n
—c)t 2 ^
1— exp{_ 0_0 12 } Z . (12.40) ! ( 2 c)t
Berry-Esseen Theorem
109
The estimate (12.40) is used in (12.39) to yield — r=/2
n(t)—e
(
Ill
p3
1
1—
I
III
( 6n 1 / 2 + 8n/
2 n )t 2 } x (exp{ —(c— I
—exp{ — " 2n 1 t2}).
(12.41)
Recalling from (12.31) that p 3 n - '/ 2 < 0.3385, so that n) 9, one obtains ligL—c)-11 /RI(6+ 1 exp { — t c
_(2—c)-11„
(2 6 1/2 1(2c— n) -1/2
2n) t2 J (
exp[—n
2n l t21 ) dt
" n 1 ) -1/2 1
1 _ I -1 + 24{( c 2n) 2n —1n 1 ) }
(2ir) 1/z <(1—c) -I 1,, 6 {(2c-9) -1/2
-1}+ 4{(c— 8) 1-2)1
<(1.465)1,,.
(12.42)
Next, proceeding as in the estimation of 1 4 above, one has IZ<2-1/2(1— 23/2)In exp{-3l^ f(2'/2/"-'<,,I,(3/2)/"- ') 3
I ^ilh 1
—Itj)}dt
<(0.0404)1,,.
(12.43)
Finally, 13<(2-1/21„)2(1— 23/2 ) 3 J
^tle-`'/2dt
{2'/z! '
<(0.00002)1,,.
(12.44)
Therefore I <(1.506)1,,,.
(12.45)
and, using (12.45) in (12.21), sup IF,,(x)-41(x)I <(0.58) -1 {(2zr) -1 (1.506)+0.6829 1„ xER'
<(1.6)/,,.
Q.E.D.
(12.46)
)
110
Normal Approximation
Some of the computations above may be sharpened to yield somewhat better bounds in (12.15), (12.16). In Chapter 5 we see that in the i.i.d. case, under the hypothesis of the Berry-Esseen theorem, the finite limit sup IF.(x)-fi(x)l
d(P,P), n-rao lim
(12.47)
xER'
exists, where P is the distribution of X,, and that 3
(12.48) P P3 6V 21r where a 2 = EX2, and the supremum is over all P having mean zero, positive sup °—d(P,(D)= 10 +3 =0.409,
variance and finite third absolute moment. Thus (V +3)/(6V ) is the asympototically best constant for the Berry-Esseen theorem. It is also known that in the general non-i.i.d. case one has (see Notes at the end of this chapter) (12.49) sup I F„ (x) - D(x)I <(0.7975)1 3 ,,. xER'
13. RATES OF CONVERGENCE ASSUMING FINITE FOURTH MOMENTS We begin with a lemma that is used in computing error bounds. LEMMA 13.1 Let X 1 ,...,X n be n independent random vectors with values
in R' having finite third absolute moments p 3.^(1 <j (n) and satisfying n
n '
EX^=0,
-
Cov(X^)=I,
(13.1)
j -1
where I is the k x k identity matrix. As usual, let x,,, j = with cumulant of X^
(I < j < n),
n
x.=n -I T, X,,,.
()v)=3),
(13.2)
j -1 n
p 3 = n - ' I P3j. i-1
Their the variation norm of the signed measure P 1 (-ID: (x,.)) defined in Section 7 satisfies 11P1(-1: {Xr})II
3 / 2 -3k'/ 2 +2) p3. (k (.Z 17
(13.3)
Convergence Assuming Finite Fourth Moments
III
In the special case k = 1, -1/2 ( 4 e -3/2 + 1 )1µ3I II P I( -1: {x})II = ( 2 ir)
< 3 (2 i) -1 ' 2 (4e - 3 / 2 + 1)P3. (13.4)
where n µ3—n-' 2 EXf.
Proof We first prove (13.4). By (7.21), IIPI(—t: {X,})Il = 61 µ3I JRJ Ix 3 -3xlo(x)dx = 3iµ3^^1o,31/2)(3x—x3)$(x)dx+
j 3112 00)(x3-3x)41(x)dx l
-I/2 {((x2-1)e-X2/231/'+[ (1—x2)e-x'/2]31,3) µ3 =(2r)
o
= (2r) ' 2 (4e
312 + 1 )Jµ3]
where ¢ is the standard normal density on R'. For arbitrary k, we use (7.20) to get IIPI( — 'D: (X,} )II < 6 [ IX(3.o....,o)I + .. . + IX(0.....0.3)II f. 1x3-3xlo(x)dx + i [
o)I + P(2,0,1,o.....0)j
.. .
+ IX(o.....0,1,2)1] f l x 2( 1—x I)I$( x 1)$( x 2) dx 1 dx 2 R2
+ IX(0....,0,1,1,1)1]fR ,I x i x 2 x 3141( x 1)4, ( x 2)41( x 3)dx i dx 2dx 3 /
<3( 27r ) -'/2 ( 4 e -3/2+ 1 ) 9,, +(ire)/2)-'0.,2+ i7l
)
3/2
9 ,3,
(13.6)
112
Normal Approximation
where, writing Xi _ (Xi , 1 , ... , Xj k ) and using (6.21) give en, I = IX<0)I + ... + IX(0,...,0,3)1 k
n
=- 1 EX^ i ^ i—t
l
j-1 k
n
N3
0",2 = ("(2,1,0....,0)1 + ... + IX 0....,0 1,2)1 '
n
=
E(X iXj.r )
n-1 1
j-I k
k
(
x.
k
± I Xj.i'I —i-1± I Xj.iI3
i'-1
—1 n
k
i-1
< k 1/2P3 — k — ' 12p3 = k l / 2(1 — )P3,
0n,3 = IX(1,1,I,o,....0)I+ ... +IXCO,...,0.1,1,1)1 n = 1
n —' E j-1
i#i',i'#i",i"#t n
k
3
cn - IEE (GIXj"I j—I
k
k
-3^X;i^ i-1
i'—I
k
IXj,'l - IXj.11 - IXj.il 3 i-I
<(k 3 / 2 — 3k I / 2 + 2 )P3.
Q.E.D.
(13.7)
Convergence Assuming Finite Fourth Moments
113
Let us now introduce a kernel probability measure K on R' whose density i is given by k 11 ga.4(xl)
W(x)=
[X=(XI..... X k )ER k J.
(13.8)
i—I
where a >0, and g 4 is the probability density on R' defined by (10.7). In fact, in this case we may compute the constant in the expression for g a and get 3a sin ay
4
(yERI)'
ga.4(Y)= 2ir( ay )
(13.9)
We shall choose )1/3 a=(4 512 k5/6=2,,7—I/3k5/6. 4
(13.10)
By (10.9), k
K(t)= IT ga4 (t 1 )=0
for I
if t [-4a,4a] k =(1:It 1 1 4a
i-I
[t=(iI,...,rk)E R k ].
(13.11)
Note that
( sin ay l
3ak
K({x: IlxII> 1})^— f [k-'12'x) 1 ay < 3k
4
1 dy
U_4dy= k (ak -) / 2 ) [ak-'/x co )
3
= .
(13.12)
IT
Hence
K({x:
JJxll8.
(13.13)
In the proofs of the theorems below we use some theorems of Section 8, substituting n-(s-2)/2p5 for ls „ (s>2). This is justified in view of (8.12).
THEOREM 13.2 Let X 1 ,.. . , X n be n independent and identically distributed random vectors with values in R k satisfying EX,=0,
Cov(XI)=1,
P4=E11X1114
(13.14)
114
Normal Approximation
Let Q„ denote the distribution of n - '/ 2 (X I + - - • + X,) and let 1 be the standard normal distribution on R k . Then for every real-valued, bounded, Bore/-measurable function f on R k ,
f
fd(Q„ -4 >)
<w1(R k )I (3Cp+aI(k))P3n
-I/2
k 2 +;I -1 / 2 kn - '(logn) -1 / 2 +a 2 (k,P 3 ,P 4 )n - '(logn) /
k (k -2)/2 1 5/2 3 logn 3/2 + zr k P3(n (I'( 2 ) + 3 ( 2 k)
+ 3 wf (27/27T-1/3k4/3p3n-i/2: t),
))-26.1
(13.15)
where wf is defined by (11.24), c o is the universal constant appearing in the Berry-Esseen bound (12.49), and -'/2 (4e -3/2 + 1)+ 3(7re' /2 ) -' k' /2 (1 — k - ') al(k)= 2 (27x)
) 3/2 +a
(
(k 3/2- 3k' /2 +2),
a2(k , P3,P4)= ik k / 2 (k+2)2
k+1
(r(
I
2
)/ (0.14+
+(0.19)(2.3) k k k / 2 (k+2)(k S =(logn) k / 2 J 00
24 )
+4)(r(fl) -' , P2
[rk-Ie-o.264r2+rk-Ie-(I/2)r2 1/2)
+ Ip3n-I/2rk+2e-(l/2)r2]dr.
(13.16)
Proof Let Z denote a random vector whose distribution is K [see (13.8)(13.13)]. By Corollary 11.5 [more specifically, inequality (11.26)],
J fd(Q„ -
-
D)I
Convergence Assuming Finite Fourth Moments
115
for all e > 0. Choose e = 4ak 1/2P3(2n) - I
/2.
(13.18)
Then (2n)"2 if Iltll> (13.19) P3
K1 (t)=0
For every Borel set B and r > 0, write B 1 =BnB(0:r),
B 2 =B\B 1 .
(
13.20)
One has
I (Q
—
'':
(
X1)) *KE(BI)I,
(13.21)
where H„ = Q„ — 4 — n I/2P1( — ( D: ()) and X is the with cumulant of X I (only cumulants with I I = 3 enter into the expression for P I ). By Fourier inversion and (13.19), Hn *K,( B I)I <( 2i )
-k
Ak(B1)
f I H (t)K1(t)I dt <(27r)-kAk(BI)(11+I2), (13.22)
where Ak is Lebesgue measure on R k, and (by Theorem 8.5) I
=f
IH.(i)Idt
(II+II< , 1 / 2 /(2p /2 ))
(
0.14 + ? 4 n 24n
+ <
0.03p3 n
)f 1 ji,114 exp(—
zIItII2)dt
f 11111 6 exp{ —0 . 38 311111 2 ) dl
(Q. 14 + 24n )k(k+2)(2ff)k /2
+
OA7p3 7t k j2 n ( 0.383) k(k+2)(k+4),
(13.23)
116
Normal Approximation
and (by Theorem 8.7) ) IHn (t)I dt / ^/\ 2 P^ / ^) < IIIII <(2n) 1/=^ /P
n
< f
[exp{ —0.264(1111 2 )
(II 1 II>n h/2 /( 2P,1 /2 ))
+( 1 +bn -1/2 P3IItII 3 )exp{
—
iII 1 II 2 }]dt
i/z Lrk—le-0.264r2+rk—Ie_0/2)r2 27rk/2(r(
2
j1/2
n^ /( 2 Pe
+
n-1/2rk+2e-0/2)r=ldr
=21rk/2(r( i.))
(13.24)
Thus I Hn*KK(Bi)l < 2-(k-2)/2(I'(2 )) (k+2)I 0.1 4 + 24n
,
+(0.14)(1.532) -k / 2 (I'( 2 )) (k+2)(k+4)p3n -I r k +2-tk-2)k-'(r(2 ))
-2
(13.25)
snrk.
Also, by Lemma 13.1, n-1/2lPI(—c: (X1.))*Kc (B1)I <1n - " 2 IIPi( — : {X„))II ^2 f 3(217) -1 / 2 (4e -3 / 2 +1)+(1re 1 / 2 ) - 'k'/ 2 (1 —k - ^)
+
=
))
\ TT/ 2 3/2
ai(k)P3n
-
(k3/2-3kh/2+2) lp3n—i/2
'/ 2 ,
J
(13.26)
117
Convergence Assuming Finite Fourth Moments
say. Next, (Qn -4')*Kc(B2)I <max{Qn*K,(B2)+ ^*Kc(B2))
< max (Prob(IIn -' / 2 (X I + • • • + X„) II > 2 ),
f
O(x)dx}+Prob(IIeZII>
JJ)
(IIxII>r/ 2 )
2
)• (13.27)
By the Berry-Esseen theorem, Prob(Iln -1 / 2 (X i + ... +X„))I > 2 ) k
<
Prob(In -'/Z (Xi , + ... +X1)> r
2k'/Z
)
/2 < 2c 0p 3 n - '/ 2 + 4k 3 (21T) 1"2r - ' e -?/sk
$(x)dx <4k3/2(21T)-i/2r-'e-,^/sk {IIxll>r/2)
r
512k°p3
Prob(IIEZII> 2)< 52 3 3 2 irr n
(13.28)
where XJ =(XJ , I ,...,XJ.k ), 1 < j
-'/2+4k3/2(2i')-1/2r- ie-r2/8k
" P
+512 ;/Z 4 3 3/2 .
23
(13.29)
2
Now take r=(8klogn)'/2.
(13.30)
118 Normal Approximation
Then from (13.25), (13.26), and (13.29), (Q,, -
<2tk + 1)
(r(2 ))
k k / 2 (k+2)0-14 -1
/2
+ 24n ) (lo g n)k
+(0.14)(2.3)k(r(2))- k k / 2 (k+2)(k+4)p32n - '(logn) k / 2 +2(k +4)/2k(k- 2)/2(I'(2 )/-2d,, (logn)k /2+ai(k)P3n-I/2, (13.31) I(Qn- 4) *K.(B2)I <2c,P3n-
1
/2+kir- 1 /2(nlog'/2n) -I
+81r -I k $ / 2p3(nlogn) -3 / 2
Now since
1i( Q,,- (D) *K,11 =2sup{i(Q,,-(D) *K,(B)I: BEl k ),
(13.32)
by using (13.31) in (13.17) one obtains the desired inequality (13.15). Q.E.D. One can easily deduce from (13.15) that there exists a constant a 3 (k) (depending only on k) such that
I
1 fd(Q., -( D)I ( wj(R k )a3(k)P4n-
1 /2+3w)
(27/2 -1/3k4/3p3n-1/2: t). (13.33)
We prove a generalization of (13.33) to the non-i.i.d. case. THEOREM 13.3 Let X 1 ,.. .,X, be n independent random vectors with values in R k satisfying (1<1(n),
EXO n
n
-
1
2
Cov(Xj ) =1,
(13.34)
j-1 n
P4 =n -I I EIjXj 11 4 <00. j-1
Let Q„ denote the distribution of n - " 2 (X 1
+ • • • +
X„). Then there exists a
Convergence Assuming Finite Fourth Moments
1 I9
constant a3 (k) depending only on k such that for every real-valued, bounded, Bore/-measurable function f on R' one has f fd(Qn —
t)I <wj(Rc)a3(k)P4n-'/ z+3w.i (27/21-"/3k4/3P3n -1 /2: (13.35)
Proof. We use the same notation as in the preceding proof. Then we have [in place of (13.22) -(13.24)] IHn*K. (B1)I <( 2 Tr)
-k Xk(BJ)(J1
+J 2 ).
(13.36)
Theorem 8.6 yields Ih.(t)Idt
J1mf
(13.37)
(hit<.i nh / 4 /p /4 )
if [see (8.30)] Pan -' < 1,
(13.38)
and Theorem 8.9, with & = Z — 2'/ Z leads to the estimate ,
Jz=
f
'12
(In I/4 /Pi 4 <11 1 11<( 2 n) 1p,]
J (hut >f(n /v.)'
")1
IHn(I)Idt
l 3& 11 1 11 2
_I
}
+ (1 + b pan - 1/1 11 , 11 3 )exp { — II 1 II 2
}
J
dt
(13.39)
Note that we used (13.38) in the last step of (13.39). Now choose r= (8klogn)' / 2 and obtain -1 /2, (13.40) Hn*KK(B1)I
and IHn"K,(B2)I
-
'
+8ir-'k5/2ps(nlogn)-3/2 < ag(k)P4n-i/2. (13.41)
120
Normal Approximation
The rest follows exactly as in the preceding proof. It is simple to check that (13.35) holds with a3 (k)=2 if (13.38) is violated. Thus (13.35) is proved for all cases. Q.E.D. One may compute an explicit error bound, with numerical values for the constants involved, in Theorem 13.3 much the same way as in the i.i.d. case (Theorem 13.2). of all Borel-measurable A significant application is to the class convex subsets of R k . By Corollary 3.2 (with s = 0) sup w+ r (2e:4)= sup 4b((8C ) 2 e)
cEe
cEe c
2 5 ' 2 F(( k + 1)/2) C
(13.42)
r(k/2)
Using this in Theorem 13.3 one obtains sup I Q,, (C)
CE2
—
(C)I < a1o(k)P4n-^/2.
(13.43)
In the i.i.d. case one also has Tim sup V I Q,, (C) — P(C )l " -10° CEC
1) / +1(k)+ 3 1 8 'r - ^ /3k 413 r( k+ 1) 2) IP31 (13.44) where a 1 (k) is given by (13.16). This follows from Theorem 13.2 and the estimate (13.42), provided that one replaces the first estimate in (13.28) by Q,,({x:
Jlxii>z( 8 klogn)" 2 })=o(n -1/2 )
(n-*oo),
(13.45)
which holds (see Corollary 17.12) if P 3 < oo.
14. TRUNCATION
We need some truncation estimates for relaxing the moment condition (namely, finiteness of fourth moments) of the last section to obtain rates of convergence for integrals of unbounded functions and (in Chapters 4 and 5) to derive precise asymptotic expansions. Throughout this section X i ,...,X„ are n independent random vectors with
Truncation 121
values in R k having zero means. We write, as usual, µa j = EX,
X = ath cumulant of X^,
1 p5.j =EjlX ll''
(l < j
n
n
E
Xa=n—' I Xa.j,
A'aj'
(14.1)
l —1 J°I n
[aE(Z+ )k,
p,, = n -I Y, ps,j
s>0].
Define truncated random vectors
X^ Y= 0
1
if IX,II
n 1,2
(14.2)
if I^Xf ll > n
Zj =Y1 —EYJ (Ic j
= EZ,j ,
(14.3)
ps.i = E IIZ1 II5, n
n
ps=n E11ZJ II 3 ,
Xa=n —I 2 X•
—I
j —1 l-1
Also introduce n s
Anj,s
= IIXjII
^n.a =n
—
I
11 Xj 11 >n^^ Z }
(
I An,j.a.
(14.4)
j=I
Finally, write Cov(XX )=((vii )),
V=n-I n
D=n -I Y, Cov(Zj )=((dij (14.5) jal
The constants c's depend only on their arguments.
122
Normal Approximation
LEMMA 14.1 Let p, < oo for some s>2. (i) One has
p,,=EIIY11I3+
(14.6)
(ii) If a is a nonnegative integral vector satisfying I < a <s, then EX — EYl I < n (s lal)/2A "j 5 —
—
IEY^ — EZ^ I < Ial( 21a1 + 1)n1/2An,i.5.
(14.7)
+d11 —v1A 42n -( s -z) / 2 p n.: (1
(14.8)
(iii) One has
(iv) If 2 < s' < s, then
p—EIIZ IIS <2fpJ'j ,
EIIY;II
;
S
ps <2 S p,..
(14.9)
(v) If s' > s, then
EIIY IIt <(En'/2)(3
-s)^
IIX^iis
{IIXj IjGen/ Z )
+(s'—s)/2^
I° < n (f — s ) / Z pf.i (0 < e < 1), IIX^I
{ sn/2< II Xji
Gn2 )
ps..i =EIIZf ll s <2'EIIY II'.
(14.10)
Proof The relations (14.6) follow from definition. To prove the first inequality in (14.7), use the inequality (6.29) to obtain EYj! - I= EX°— l
f
f
X < r 11X.111°I
I {^^Xju>n^^ 2 }
j
l
toxilI>n 1/ )
,-(s-la)/2 fIIX.II s =n -(s- l a l ) / 2 0 nJ,s• {IlXill>n1/2) j
(14.11)
Truncation
123
To derive the second inequality in (14.7), first observe that
xa_YaI_Ixr i...Xk —Y1O". k I = Ix '
Xk I ( Xk Yk k )
+x '• ..Xk 2( xk I Yk I)Yk +... + ( x 1 ' YI ' )Y2 =.•. Yk I
xk
Yk)I ak max (Ixkl ak-I, IYkI
+... +a,max(IxI o1-I, IYII a '
1
N-1 )
)IYZ :... yk (XI — YI)I
< IaI(IIxll lal-I +IIYII I°I-1 )max(lxi — y, : I < i 6 k},
(14.12)
k X=(XI,...,Xk)+Y=(YI,...,Yk)ER ].
Since by (14.11) IYi.r — Z^,I=IEYI
-(:-I)/20
n.•
(1
(14.13)
it follows from (14.12) that
EYE —EZ7I
'EIIY'+EIIZJIIIai-1). (14.14)
Now EIIY^III^I-1= /
IlXJIlal-1
(14.15)
IIEY1II <(EIIY;II 2 ) 1/2 , E I Z^ IIIai-1 < 2 1al -'( E 1IYI II Ial -1 + II EYY II I " I-1 ) C 2laI n tlal- It/z . Hence IEY^ — EZJ I < IaI( 21al + 1)n-(s-IQI)/2An.i..,.
=
124 Normal Approximation
Next we have
4 Idir—vtt1
= n -1 2 I E (XX,1 X1. , — YJ,rY!.,) + E (Y)E (Y1.t)I i- 1 n
l
n,j,s +n
) h/2n -(s-U/2^ nJ.s
l
f I —
2n
-(s-2)/2^ A t.
(14.16)
To prove (iv) note that [using (6.26)]
11Z;II J <(IIY;II+IIEY;II) s < 2s- '(IIY;II S +(EIIY; 11 2) s/2 ) < 2s -'(IIY^II' + E IIYYIi s ) Proof of (v) is clear. Q.E.D. Recall that the norm of a k X k matrix T, denoted IITII, is defined by 11 TIJ =
11 Txll.
sup
(14.17)
xER k .IIxII C I
COROLLARY 14.2 Under the hypothesis of Lemma 14.1, one has I— I<2kn -( s -2) / 2411 t II Z
< 2 kn -(s-z)/2 ps11 1 II 2 (t E R k ),
(14.18)
so that
lI D —VII < 2kn
-(s-2)/2E
2kn _(5_2)/2p3 (14.19)
In particular, if V = I and
ks
(14.20)
then D is nonsingular and
a
IID-III
(14.21)
Truncation
125
Proof. By (14.8) and using (6.26) in the last step, we have k
I— I= i t;tr(d,, i,t-i
—
vu) 2
k
< 2 n -cs-2)/2 p'.:(2 It^I) < 2 kn -cs-2)/2 p"SII 1 II 2 .
(14.22)
Since the matrices D, V are symmetric, (ID — VII= sup II<2kn
-(
II+II <
' -2
l/ 2 A- n ,5 .
(14.23)
Also, if V= I and (14.20) holds, then
(14.24)
(t,Dt>> (ItiI2—aIItIIZ=aIItIIZ
(14.25)
IID — III,
2
and
The last inequality implies that D is nonsingular and that II D - ^ II < i Q.E.D. The next lemma concerns the growth of derivatives of the characteristic function of n -1 / 2 (Z 1 +"• + Z„). LEMMA 14.3 Suppose p s < oo for some s> 3, and that V = 1. Let gi denote the characteristic function of ZZ (1 < j < n). Then if 1/2
II t II < 16p 3 '
(14.26)
one has, for every nonnegative integral vector a,
l g' (n ' ) ^ ^fi (
D
n
+6111II 2 ).
(14.27)
Proof By inequality (8.46) one has
( <exp{ gjln^/2 )1 t
2
—
n - 'E2 +3n -3 / 2EI<,,Z^>I 3 )'. (14.28)
126 Normal Approximation
Observe that exp{ n -I E'(t,Z> 2 — in -3/2E11 3 ) <exp{ n - '(EKt , Zj>I 3 ) z/3— in -3/2EI 1 3 } < supexp{a z — Za 3 } =e 1 / 3 .
(14.29)
a>O
Now let Nr be a subset of N = (1, 2, ..., n) containing r elements. Then in the given range (14.26) one has
11 jEN\N,I
Sj
t
)1
z
n /
2
4
II exp{
—n-IE2+
3n-3/2EII3)
jEN\N,
'exp ^
—<1,D:>+ 3n-3f zj-1 1 EI <1,Zi>`3 }
x 11 exp(n -I E
<exp( —+ 3n -1/zp3II 1 11 3 )e' j3
(14.30)
< exp{ — fit, Dt>+ 311 rll z }e`/ } ,
using (14.9) (with s'=3) and (14.26) in the Iast step. It follows that ) <exp( ( g( J 1 2
-
(14.31)
jEN\N,
which proves the lemma for a=0. To prove it for nonzero a, note that
(DMgj)(
1/z)l=n-1/2IE(Zj,n ,expfi))I
n
i
=n-1/2IE[ZJ m(exp{i)
—1)]I
I
[Zj—(Z 1.1,...,Zj,k)],
(14.32)
using (14.9) in the last step (with s' = 2). For a nonnegative integral vector
Truncation
127
/3 satisfying I /31 > 2, (Dagi)( (- )I < n
-1PI/ZEIZBI < n -IfiI/Z P ,
6 21 #I n -'p2j,
(14.33)
by (6.29) and (14.10). Thus
(DPg^)( ^/2)I
(1
where
c2(a,k)=max(4,21al),
uhf = max{ 1 ,II'll).
(14.35)
By Leibniz' rule for differentiation of the product of n functions, (D*Hj_,g1 )(t/n 1 / 2 ) is the sum of nI"I terms, a typical term being
J
[ f^N, 11 gi\n,2))[D4Igi(112)
_j ) [DPgj(_( ... J,
(14.36)
where N, _ {J1,.. . 'Jr)' 11
(1
Q;=a.
(14.37)
r-i
The number of times the expression (14.36) is repeated among the nIaI summands is given by a1! .. ak l
r
(14.38)
k
II H a, i-1 -I
where a=(a,,...,ak),
$i=(Qri •.• Qik) ,
,
(1
(14.39)
In view of (14.31) and (14.34), the expression (14.36) is bounded in magnitude by exp{ 6 — Z+ blltII 2 } II b1 , BEN,
(14.40)
128
Normal Approximation
where bi=n-IC2(a,k)P2j
IltII (1
(14.41)
< j
Therefore
_ n
(D a 11 9)i)(_/2 ) i I
G 2 c 3 (a,r)exp l 6 — i+ 611t11 2 }' j `(j b I
(14.42)
BEN,
where, for each r, I• denotes summation over all choices of r indices from N=(1,...,n}. Hence
n
_
11 b'! <(^ )-I b;) =(c2(a , k)P2 Iltll )r
sl BEN,
(14.43)
=(c2(a , k)k Iltll )'•
The inequality (14.27) follows on using (14.43) in (14.42). Q.E.D. COROLLARY 14.4 If in addition to the hypothesis of Lemma 14.3 the inequality (14.20) holds, then Da gi)( (
i-I
I z )I^ci(a , k)( 1 +IItfl l°l )exp( — i IItII 2 )
(14.44)
n/
for all t satisfying (14.26). Proof. Use (14.25) in (14.27). Q.E.D.
We also need an estimate for the difference between a polynomial expression in the cumulants of n - '/ 2 (X I + • • • +X„) and the corresponding expression in the cumulants of n - I / 2(Z I + • • • LEMMA 14.5 Suppose that V= I and (14.20) holds for some s > 3. Let v i ,...,v m be nonnegative integral vectors satisfying m
^v,I>3
(1
2 (Iv1 l-2)=r,
(14.45)
i-I
I < r < s — 2. Then Ix,: xi. —x1... x,j < c4(s,k)n-('-r-2)12.,,
for some positive integer r,
' -2) / 2 P,.
(14.46)
Truncation
129
Proof First recall [see (6.14), (6.15)] that for each nonnegative integral vector v, X^ i is a linear combination of terms like µa;,i...µ i
(14.47)
where a,,...,ap are nonnegative integral vectors and r,,...,rr are positive integers satisfying P
r; a; =v
(a ; >0,1
(14.48)
To estimate the difference y, — y i it is therefore enough to estimate 1.... µ
µ« i l
r —' r«i ....µ ,r r «r.J l J ay,J
(14.49)
subject to (14.48) and 3 < I vJ < s. By Lemma 14.1(ii), (s I«I ) /Z0^ (I < II < s). µa.i µail < C5(a)n ✓,s
(14.50)
We may use (14.50) and (14.12) to get µayj...N ^—^^'aiIJ.
..µr^.il
P
<
r.0
l µ .,-iI + I µ '.-lI
( a ) n -(s-da)/20
6
nj,s( 0.J
!=I r^
r.
x µa,j
µ„iµ«,.,,1
...
o4J
'rr
µmil
P
<
c a n
—(—kI)/2
m;/s
(14.51)
^n J,sPs,j '
where c 7 depends on i, a's, and r's, and P
m;=
r; Ia; l ,
r=i
,
—
Ia1I= Iv!
—
Jail (14.52) ,
by (14.48). In the last step of (14.51) we have used the inequalities [see (6.27), Lemma 6.2(ii), and (14.9)] i l/s µaiI < plal,i
(14.53)
130
Normal Approximation
In view of (14.20) one has
11 )11'+A n j
PS.)
<
S
n+ORj.s
(11X;11 4.111)
< nJ/ 2 +n0,,, J < nJ/ 2 + 8k2 (14.54) so that n-(s-larl)/Zp j/J=n-(s-Ia,I)/2P;^VI -Ia,I)/s < n-(s-Iarl)/2n(Irl-la;l)/2 1 + (
8k
< c 8 (a„k)n -( s - I'U/ 2 .
(14.55)
It now follows from (14.51) that Xrj—Xrj
I
and, therefore, averaging over j=1,...,n, ix
— xi
< c 9 (Y,k)Zn -(s-1• I ) / 2 (3 < Ivl < s).
(14.57)
Let now v,, ... , vm be nonnegative integral vectors satisfying (14.45). By (14.57) one has xv, ...
w _^... m C9 ( 1,
k)Q n'5 n -(s-Irrl)/ZlXrr..
. ^i-Ix'ra^...
(14.58)
By Lemma 6.3 (and averaging), Lemma 6.2(iii), and Lemma 14.1(iv) one gets (recall that P 2 - k)
< C10(s k)PI. 1 1 ,
...
p1rr - ^IpjP,.1l
.z)/(=- 2 ) r
...
p1.I
(14.59)
Truncation
131
But because of (14.20), one has n
p, = n -'
p5 , j=I
n
=n '
(f
—
IIXjIIS+O„,i.sl
lll xill
j 1 L
n
{11x:11
< kn ( s -2) / 2 +, s <(1 + 8k)n(s-2)/2.
(14.60)
Using (14.60) in (14.59), one gets ..
IX.,... X;-^
.I < c12(s,k)n (r-II+2)/2 .
(14.61)
Substituting this in (14.58) we obtain (14.46). Q.E.D. The above estimates also lead to LEMMA 14.6 Assume that V=I, and that (14.20) holds for some s>3. Then for every integer r, 0 < r < s — 2, one has n -r/2 V ' ( - 4: {X.})(x) — Pr( - 4o,n: {XP'})(x)I < c r k s 0 n - (5 -2 )/ 2 (1 +lixIl 3 T+ 2
2 xexp{ — 11611 +IIXII}
)
(xERk),
(14.62)
and 2IP,(—$: (X,.})(x+a„) < ci4(r,k, ․ )in,5n
—
-
Pr(
: {X.})(x)I
3.+1 ) C, 2)/2 ( 1 + 11x11 -
xexp(-211x112+ 8
where
—
1Ik /Z }
(xER'`),
(14.63)
)J a„=n - '/ 2 EYE . j—1
(14.64)
Normal Approximation
132
Proof We first obtain an auxiliary estimate. For z E C k write k
g(z)= l -(DetD)-1/2exp^ i 2 z?
-
i-1
i
d11z,z1 1
Iz=(z 1 ,...,Z k )}.
Then k
z?-i drrz1z,.
g(z)I=I(DetD )-1/2exp{i
i=1
1
-1-
i,i'k
III(IIZ11 2 )}
+zlID -1- III(IIzII 2 )exp{IID -1- f II(IIzII 2 )} Z=(Zl,...,zk)EC k ],
(14.66)
where Det denotes determinant and d" is the (i,i') element of D -'. By Corollary 14.2 [replacing t by Bt in (14.18), where B 2 = D -1 ],
- II 1 11 2 1 <2kn-(:-2)'2pn,,IID -I II(IIt11 2 ) <3kn -(s-2)2
p^S11 1 11 2, (14.67)
which implies II D
-1 _ I II < 3 kn -(s-i)/2 „ ^ <;68) . (14.
Also, expanding DetD and making use of (14.8) and (14.20), we get I(DetD) -1 / 2 - l l = (DetD) -1 /'^ I -(DetD ) 1 / 2 I <(Det D ) -1 / 2 t1-(Det D ) 1 j 2 I(1 +(Det D ) 1 / Z) = (DetD) -1 / 2 11- DetDj 4 II D -' ll k / 2 11- DetD l < c15(s,k)L3n-(s-2)/Z
(14.69)
The estimates (14.68), (14.69) are now substituted in (14.66) to yield z _ 14311 1 ^g(z)I
(zECk). (14.70)
Truncation
133
Hence max
I8(z)i
(.-, e& , II: •- zIl
(Ilz11+ 1 ) 2
l
3
J
II 311 2
< c17(s,k)On,5n-(s-2)/2(1 + Ilz11 2 )exp j
+ lizII .
J
l (14.71)
By Cauchy's estimate (Lemma 9.2) one then obtains
I
IDg(z)l
From this follows D' ($ — $o,v)(x)1= I E c19(a)(D a $)(x)(D' -a8)(x) 0
/z(l+IIXII2)expl—
I-
(xER k ).
+11x11} (14.73)
Now recall from Section 7 that [using (7.3) and Lemma 7.2] P,(-0: {x)) r
m^ I
m
`
I ^*
(^**
J^,....Jm
^
vi....,v,,,
^
I• •
m•
(-1)r+zm Ll
P,+...+VT
^I (14.74) J
where X* denotes summation over all m-tuples of positive integers (j l ,...,jm ) satisfying m
j
i
=r,
(14.75)
f=I
and X** denotes summation [for fixed (j l ,...,jm )] over all m-tuples of
134
Normal Approximation
nonnegative integral vectors (v 1 ,.
''m)
1v1 1=j; +2
satisfying
(1
(14.76)
The function P,(-4 o,0 : (c)) is similarly defined by replacing X • by X • and 0 by 4 o.o in (14.74). Next observe that
I XYI...X• D•.+...+.^$(x)_ ... .D•l+...+'.^o,o(x)^ II I ... Xy Dr,+...+P-
(O(x)-0o,a(x))1
+J(X•....X• —X.,...x. )D°^
+...+
'"4,(x)I
(xER k ). (14.77)
By Lemma 14.5 ... Xyj < JXP,... XP_ _ X: 1
c4 ( s ,k) n -( s -r-z) / 2 Q
(14.78)
Also, as in (14.61), one has 1Xy^
...
(14.79)
X^j 4 c21(s,k)n"/2.
Using (14.78), (14.79), and (14.73) in (14.77), we have n(x)l I X•I .. . X. DY^+...+ .4(x )—x , ... X;.D•"+...+ .'Po.
II 6112
+IIxII}
(x E R k ). (14.80)
The inequality (14.62) follows from this and the expression P, given by (14.74).
To prove (14.63), first note that by Lemma 14.1(ii) Ila,ll < n -1/2 I IIEY;II < k 2En,5n-c,-2)/2 <(8k" 2 ) - '. (14.81) I=I From the expression (14.74) and the inequality (14.79) (which also holds if the primes are deleted), we get {})(x+a„)
—
P,(
-
4: {X,.})(x)i
Truncation
135
But D'4 =q"çt,, where q" is a polynomial of degree Iv( with coefficients depending only on v and k, so that ID"4(x+an)—D"-O(X)I
But by (14.12) and (14.81),
I(x+an) °— x°I
(xER k ), (14.84)
and I(x+an ) ° [4(x+an )-4(x)]I
= (x+an) ° -O(x)Ijexp{
—
illx+anII 2 +illxIl z } -1 ^
(x+an) a 14(x)IIanlI(I+11x11)exp{IIa.II(l+11x{1)}
-(,-z)/z
(l+IIxII kN1 )exp{ —
8
k'11
2 }• (14.85)
Relations (14.84), (14.85) are used in (14.83) to yield ID"¢(x+an )
1
—
D"4(x)I
-(5-2)/2
Xexp{ — zIIxII2+
(l+llxII'
81k / z }.
1
)
(14.86)
Finally, (14.86) is used in (14.82) to get (14.62). Q.E.D. A bound for absolute moments of sums of independent random vectors is needed to prove the important Lemma 14.8. This is provided by LEMMA 14.7 Suppose that V=I and (14.20) holds for an integer s)2. Then there exists a constant c27 (r, s, k) such that EIIX,+... for I< r < s.
+X n llr
136
Norma! Approximation
Proof. In view of Lemma 6.2(ii) it is enough to prove the lemma for r = s. For r = s = 2, (14.87) is trivially true. Let us then assume that s > 3. Also by (6.26), z
n
k
l/2
EIIX1+••• +XII'=E E l i-I !
—
/
i
k
E l
s
x1 j ,
i- 1 [Xi=(XJ.i,...,XJ•k)],
(14.88)
so that it is enough to prove the lemma for k = 1. If s is an even integer and k= 1, then by Theorem 9.11, s-3
n-,/zE(X1+... +X.) 5 <
1
n -m/2D sgm( 0) +cza(s) n
-
(: z)/2p,, (14.89) -
m—O
where
r z l g(t)=Pm (it: (7t,))exp( 21 }
(tERI).
(14.90)
By Lemma 9.5 (with k = 1, a = s, z = 0) and inequality (14.60), one has D'Sm(0)1 < c29(m, ․
)P;'/(,-z) < c3o(m, ․)n m/z '
(0<m<s-3).
(14.91)
The lemma is proved for even integers s. Next assume that s is an odd integer, s> 3, and k = 1. Clearly, EIX1+••• +X<EIZ,+••• +Z n l' (14.92)
+EI(X 1 +••. +X„)'—(Z 1 +... +Z„)'I.
By Lemma 14.3 [inequality (14.27) with k= 1, a = s + 1, :=0], EIZ,+••• +Z„I°<(E(Z 1 +••• +Z„): +1 ) < c 31 (s)n'/ z .
s
+1
(14.93)
Truncation
137
Next, EI(X1+••• +X,,)s—(Z1+...
+Z,,)uI
n EI(X 1 + • • +Xj +Z11 +••• +Z n )
<
3
j-1
—(X1+... +Xj—I+Zj+••• +Z,) 5 I n
44
s
1 7. (,) E I( XX — Zj)(Xl +... +Xj—I+Zj+I+... •}•Z ) sm I j=1 m—I n
s
s
+... +Xj—I+Zj+I+... = 2 Y. ( ) E I X7 — Zj IEIXI
+Znls—m.
j-1 m-1
(14.94)
By (6.26) and (14.54) we have E IX 1
+•••
+Xj—I+ZZ+I+...
+Znls—m
62s—m—I (EIXI+...
+Xj+Zj+l+... +Z nl s—m+EIXj Is—m)
<2s—m—lE•IX,+... +Xj+Zj+l.^... +Znls—m +2s-m- ► Bn(s-m)/2.
(14.95)
Writing Tr.j=n-I(EIXII'+... +EIXjI'+EIZj+II r +
...
+EIZ,,I'), (14.96)
one has, using Lemma 14.1 and inequality (14.20), n
T 2•j
EZj +1 EYE +; <EX^ +; ,
(14.97)
j-1 Tz.j — n -1 E^Xi+
... X)
—
E(Xj+I — EZ^+I) —... —(Exn—EZn)]
=n -I [n — E (2— Yj+l) —... —E(Xn—Y^) — (EYj+I)'— ... — (EYn)2]
138 Normal Approximation n
> I-n' I n-(s-2)/20
n
n,j,s
-n-I E n -(s-i)Az
n j,s
j_ j-1 )
I
-
20 n s n —(s-2) / 2 ^4^
7rJ
IXII'+... +EIXjI ' + 2 '(EIXj+I I ,+... +E I X nl ' )]
Look at the random variables
Xi j, l
W = ;
T2 (14.98)
the j+l
T2 j
Applying Theorem 9.11 (with k= 1, a=s-1, t=0) to these random variables, we have n s—m
E ^ W ^
5-1 (s —m) / (s —I)
n
< (E (i -I W1 /
'
I)/2)(3-m)/(s-1) <(c32(s)n(s=C 33 (s,m)n/ 2 (1 <m <s).
(14.99)
In deriving this we have proceeded exactly as in the proof for even integers s. More explicitly, we have used Lemma 9.5, inequalities (14.89), (14.91) with W's replacing X's, rs ,j /-r2j 2 replacing ps , and inequality (14.97) and the modified polynomials Pm corresponding to the cumulants of W., 1 < i < n. Hence EJXI+... +Xj+Zj+I+... +Znls—m
_
T
(j
n s—m
m)/ 2 E)
W,)
(C34 (s,m)n(s-m)/2
(I < m < s).
(14.100)
Using this in (14.95), we get EIXI+... +Xj-1+Zj+1+... +Znls—m
< c 35 (s,rn )n (s
-
m)12
(1 < m < s).
(14.101)
Truncation
139
Also, by Lemma 14.1 and inequality (14.60), n
n
I EX; Z;I< Y. (EIX; —
l-1
—
y;I+IEY;I)
l—1
—
J
_ (
Ox 1>n 1 / 2 ) ;
IX'I+IEYJI)
n
44 1 (n-1/2EX2+n-'/2)=2n1/2, J-1 n
n
2 EIX^—ZZI<2 2 EXj=2n, n
n
I EIXj —Z; I< 2 (EIX,I.^+2mEIX,Im)=(l+2m)npm l- 1 l-1 <(1 +2m)npsrn 2)/(5 - 2)
( (1 + 2m)n (Qn(s-2)/2)(m-2)/(s-2) 8 = C36 (s,
m)n'"/ 2 (3< in < s).
(14.102)
Use (14.101), (14.12) in (14.94) to get EI(X 1 + • • • +Xn)5—(Z1+ • • • +Z„) s l < c37(s)n
2 .
(14.103)
Together with (14.93) this implies EIX,
+
...
+Xnl s < c38(s)ns/2,
(14.104)
completing the proof of the lemma. Q.E.D. Before we state the next lemma, we define On.s(E)=n ' ^, -
IIX^IIs
(e>0).
(14.105)
IIX^i>fn( 1/2) l—1
In this notation (14.106)
Let Qn denote the distribution of n - '/ 2 (X 1 + • • • +X n ) and Q„ that of n
- "/2(y l + ... +Y).
Normal Approximation
140
LEMMA 14.8. Let V = I. If p 6 < co for some s > 0, then there exists a positive constant c 39(s, k) such that
II Qn — Qn II < C 39(s
k)tn,3n -t'-2>/2.
,
(14.107)
Also, there exist two positive constants c4o(s,k), c 41 (s,k) such that whenever .3 )
(14.108)
for some integer s> 2, f II xII'I Qn — Q,','I (dx) < c4,(s,k)On.,n-(s-2)/2
(14.109)
for all rE(0, ․). Proof. Let G^ denote the distribution of n - '/ ZXJ , Gd " that of n 1<j
and
I
n
Q.—Q..1'II = G,*...
H !i—'
*G1— ,*(G 1 —G")*G'+,*... J J
aG,'
I
n
n
<
IIGG — Gd"II= 2 Y. P(II XxtI> n1/2 ) n
IlXjll'=2A-nsn-c:-2>/2, (14.111) <22n - s/ 2 f (IIXjII>n112) i-I
which proves (14.107). Now assume that s is an integer, s>2. Since IIxII' < l + Ilxlls for 0< r < s, it is enough to prove (14.109) for r=0 and r = s. The case r = 0 is precisely (14.107). We therefore need only to prove (14.109) for r=s. One has
f IIxl IQ. Q,'I(dx) 5
—
<
f Ilxll'IG.*
..
.
*GG-i*(G1
—
Gi )*Gi+i* g
...
*G,. I(dx)
n * G,, (du ) <j-21 f (f ►I u+U II sIG^— GJ'I(dv))G,*... *Gi-,**... n
<2s"' 2 (ii
—
Gj"II JII uII 3G,* ...
*Gj-
*G;+I*... * G„' (du )
i-1
+ f iivll 'IG; Gj' I(dv)). —
(14.112)
Truncation
141
Now
f IIvlIJIGj—Gj"I(dv)= f
( [Ix; 1 1> n'lz)
IIn
-i / 2 XJ IIs=n - s/ 2 0 n'j. ,.
(14.113)
Also,
f II uII G1* s
...
*G^-,*Gj+,*... * G„' (du )
=EIIn-1/2(X,+... +X1-^+Y;+,+
...
+Yn)II 3
<2s—'( Elfin — /2(X, + ... +X; +Y;+i +... +Yn)Ii 5
+IIEIIn — " 2 X11_, ) < 22cs—u(E IIn —/2(X, + ... +X1 +Z1+ ,+... +Zn)IIS +IIn -1 / 2 (EY^
+i
+... +EY.)II')+25- 'Eli n -1 / 2 X1 II'. (14.114)
By Lemma 14.1,
I I n "/2(E Y^ + , + ... + E Y1) I IS $
O l =(k2n-c:-2)/20 ) . (14.115)
< ( n -1/2ki/2n -c:-u/2 1
nJJ
J
J
l^^
By inequality (14.54), Elln
-
1/2
X,II s
—
s/2 (n s/2
+
< 1+n —(s-2) / 2 0„ s . (14.116)
We next estimate E IIX, + • • • + Z,, Its by using the preceding lemma. For this assume first that k = 1, and define T,.^ and random variables W as in (14.96), (14.98). Note that ;
IW;IJ=TZ fLIwi >n ) J
5/2
112
f
(/
llx,I>T=i'2&/2}
f I W. I S=T—s/2 2.i f{Iw,I>nhI2) , {IZ,I>*2 /
NIS
(1 < i <.j),
IZ I s /2
)
,
< T2 s/2 f
t IY;I> T,14i2n112— IEY;I }
lye's
+IEY1I 5P(Iv11>T2 2n"/ 2 —IEY I)] 1
(j+l
(14.117)
142
Normal Approximation
Taking c40(s,k)< 1/(8k), one has [by (14.97) and Lemma 14.1) T
EY ; I < n -( s -1) / 2 A
"
J .s
Z 2 >(4) l/2,
n' /2 < n -(s-3)/20 " S < g TZJ 2n'/ 2 —IEY 1 I >n" 2 ,
2n'/2 n'/ 2 )'p (IY;I> 3 ) IEY1I sP(IY;I>T24 2n' /2— IEY;I)<( 8
( nl/2)S ( 2n'/2 3 ) < g PIXrI> flh/2
(
)
I
2 fl 1 /2
)
—J
f
IX;Is > 2n' / 2 /3 )
(jXjj
1
S
{IX j j>2n'/ 2 /3) IXiI,
fIY;^J < f {IY I>r1t 2n'/ 2 —IEY I) ;
;
{IX;I>2n'/i/3}
^X;Is.
(14.118)
Therefore, choosing c, a(s,k)=(,1) , / 2 (l6k) - ', one has n -'
"
IW;I s < T2^S/22a n.^(s) < f {IW , I>"' 12 )
n (r — 2)/2
(14.119)
8
so that Lemma 14.7 may be applied to yield EIX,+••• +X^+Z,+••• +Z,I
$
= TZ j 2E I W, +... + W IS .
EIW J +
+W„I 5
(14.120)
For arbitrary k, apply this estimate coordinatewise to get EIIn — './ 2 (X 1 +... +X^+Z^ + ,+... +Z n )11 3
Truncation
143
The inequalities (14.115), (14.116), and (14.121) now show that the left side of (14.114) is bounded by a constant (depending only on s and k). This fact, together with the estimates (14.113) and (14.111), now gives [on substitution in (14.112)]
f II x II s !Q. — Q,'J(dx) < c44( s,k ) t
s
n.
(14.122)
Q.E.D. 15. MAIN THEOREMS
We are finally ready to prove the two major theorems of this chapter. These provide bounds for I f f dQ n — f f d (Dj over (essentially) all 4continuous functions f that are integrable with respect to Q n under given moment conditions. The bound provided by the first theorem, which is much more useful than the second, cannot in general be improved upon, and the hypothesis in it cannot be relaxed any further (at least if we are to have bounds involving moments alone). The utility of the second theorem is explained later. We continue to use the same notation as in the preceding section. For ease of reference, recall that X 1 ,.. .,X are n independent random vectors with values in R k satisfying n
EXj =O
n I Y, Cov(X1 )=I,
(1<j(n),
-
(15.1)
j -1
I being the k x k identity matrix. The moments and cumulants of XD 's are denoted as in (14.1). In particular, n
p3 =n -1 Y, EIIXj IIs j=
(s>0),
1
n
[vE(Z+)k],
X.=n ' ^. XY.i -
(15.2)
j=1
where is the with cumulant of X. The corresponding quantities for the centered truncated random vectors Zj (1 < j < n), defined by (14.2), are denoted by ps, . Recall also the notation n
D = n -'
Cov (Z^). j-1
(15.3)
144
Normal Approximation
We let Q, Q,,, Q„ denote the distributions of n -1 / 2 7-_ 1 Xj , n -1 / Z E^_ 1 Zj , n -1 / 2 _ 1 Y1 , respectively. As usual, '1" denotes the standard normal distribution on R k , and 4V is the normal distribution with mean a and covariance matrix V. For a real-valued function f on R" define M,(f)= sup (1+IIxIr)-llf(x)I if r>O, xER k
M0(f)= sup If(x)—f(y)I=wj(R" ).
(15.4)
x,yER k
If v is a finite (signed) measure on R k , define a new (signed) measure v, by if r>O,
v,(dx)=(1 +IIxII')'(dx) o = v.
v
(15.5)
As in the preceding section, write n
^n,sl E ) —n-1 f
IIXi 3
^ns=^ns(1).
(15.6)
j-1 01Xj11>cn'/2)
Then define n
inf A* s =O
O n ,,(E)+En
-
'
f
(15.7)
i=1 (jjXill
Note that
Ons
On.a—°On,s(1)
(15.8)
If one takes E = n - '/ 4 in the expression within brackets in (15.7), then one gets n s
n.s(n —1 /4) + n —1 / 4 ps .
(15.9)
Thus if X 1 ,.. .,X,, are the first n terms of a sequence of independent and identically distributed random vectors, then n -(s-2)/2
* ., = o ( n -(s-z)/2)
(n—*oo).
(15.10)
145
Main Theorems
Finally recall the average moduli of oscillation wJ and wJ defined by (11.23), (11.24). THEOREM 15.1. There exist positive constants c (i = 1, 2, 3, 4) depending ;
only on k and s such that if T
(15.11)
h.s^ 3) < c1n(s-2)/2
holds for some integer s > 3, then for every real-valued Bore/-measurable function f satisfying Mr(f)
(15.12)
for some integer r, 0< r < s, one has
J
^
- s-2)/2
+2wg (C4P3n
—
'/2
p*.s : (p+)ra)•
(15.13)
where s-2
_ I n - m/ 2 Pm (-(D:(}), m=0
and _Jr
r°
g(
)
if r is even,
r+1 if r is odd,
(I+jjxjj
ra
)
-
i
f(x)
if r>0,
-
x f(x )
if
(15.14 ) r=0.
Proof The constants c's appearing in the proof do not depend on anything other than r, k, s. We write s-2
m/2
m=0 i//" (B)=1^I(B+a n )
(BE^'ilk),
(15.15)
146
Normal Approximation
where n
an =n -1 / 2 E EYj .
(15.16)
Thus'" is the signed measure induced from iji by the map x- *x — an . Now
f fd(Q,,
—
i )I
—
Q.')I+ f fd(Q,^'
—
^G)I.
(15.17)
If r>0, then (xER k ),
If(x)I <Mr(f)(l+11x11')
(15.18)
so that using Lemma 14.8 one gets (15.19)
f fd (Qn — Q,^' )
Since the left side does not change when f is replaced by f— c, where c is the midpoint of the range of f, (15.19) also holds for r=0. Next
I f fd(Q,,' — *p)I =I f fg,d(Q.' p")I
+Ifa,d( 1P' — ip)l+I f fd('G — p")I , (15.20) where f, denotes the translate off by y; that is,
f
y
(x)=f(x+y)
(xER k ).
(15.21)
By Lemma 14.6,
f fg,d(0'-0)
<Mr (f) f (l+llx+anll')I ' - 4I(dx) <M,(f) f ( 1 +2•llanIIr+2'llxlI')Ip'
-
4I(dx)
Mr(J)I110 '- 7'lI+ 2' llanll ' ll4 '— o11+ 2' f 11x11 ' 14' '- 4'I(dx)J
C6Mr
)Q
(f
..sn-(s-2)/2.
(15.22)
Main Theorems
147
In the last step we also used inequality (14.81). Similarly,
ff.d(P-'P")
-(s-2)/2
(15.23)
Therefore (15.17) reduces to [in view of (15.19), (15.20), (15.22), and
(15.23)]
f fd(Qn
-
4')I
To estimate the second term on the right of (15.24) we introduce a kernel probability measure K on R k satisfying
K((x:llxIj3, f IIxII
K(t)=0
k +s+ 2K(dx)
if 11111 c 9 (tER k ).
(15.25)
One construction of such a probability measure is provided by Theorem 10.1. For c >0 define the probability measure KK by K,(B)=K(E
-
'B)
(BE, E 'B=(E 'X:XEB)). (15.26) -
-
Then one has
Kc ((x:IIx)I<E))>4,
Kc (t)=0
if Iltll> E9 .
(15.27)
Now
f f;
d
(Qn
')I=I f (I+Ilx+a.II To)
-!f
(X+an )
X (I + II x+ anll '°)( Qn — 4 ' )( dx )
1 1 (1 + tx + aIl'°) ' f(X+an)(l+11xII'°)(Q,,
—
+M,°(f) f IIlx+a„II'°_11x11'°I(Q,+j,"I)(dx),
^')(dx)I
148 Normal Approximation
f IIX+a,.II'°
—
Ilxll'°I (Qn+I^F'I)(dx)
+ f ((8k'/2)-.°+'+IIxII'°-I)IiI(dx)
+ f (( 8 k 1/2 ) - '°+i +IIxII'° - ')I^Y'—PI(dx)I < c I00n -(s-2)/2 ,
(15.28)
by Lemmas 14.3, 14.6 and inequalities (14.12), (14.81). Hence [see (15.24), (15.28)]
f fd(
Q —,p)I
)Zn,,n-(s-2)/2
+ f 8(x)( 1 +11x1('°)(Q,, — +G')(dx)I
(15.29)
if it is noted that
M,° (I)=
sup
(l+IIx11'°) -I I.f(x)I
XER k
<M,(f) sup ( 1 +Ilxll')(I+IIxII'°) - '<2M,(f).
(15.30)
XER k
By Corollary 11.5 one has for every c >0 f 8 (x)(l + lI xJI'°)(Q — ')(dx) ;
< 2 M,° (f)II(Q.
—
ip'), *K II+ 2 ;( 2 € : (¢'+),0),
(15.31)
where (Q,'—i').°(dx)=(l + II xll'°)(Q.
-
4 ')(dx), ,
( ,G' + ),° (dx) = ( 1 + IIxII'°)4G' + (dx).
(15.32)
149
Main Theorems
Now choose e = l6c 9p 3 n - I / z .
(15.33)
By Lemma 11.6, (Q,H*P'),-*KcII
where the maximum is over all pairs of nonnegative integral vectots a, /3 satisfying 0
(15.35)
and the integration is with respect to Lebesgue measure on R". Since
K (t)=0 for Iltll>c 9 /e=n 1 / 2 /(16p 3 ), and (
( DPR<)( t )I < (elsllxflIK(dx)
(15.36)
one has
f
I " (Qn — ' ')'
D'KK
=
IDO(Qn—')(t)Idt.
(15.37)
(11:11
Note that [see (14.60) with s=3] a is bounded above by a constant because of (15.11) [whatever the constant c one may choose in (15.11)]. This fact
was used in (15.36). Now
I
— ^G )(t)I dt
II^II fl"2
k+s—I
" D
Q^(t)— 2 n - "'/ 2Pm (it:{X))exp(-2(t,Dt)) dt m=0
A.
k+s—I
+ f D"
m/ZPm(it:{ ))exp{—Z(t,Dt>) di m=s-1
+ fA^ ^(D"Q„)(t)Idt+ fA^ I(D°5P')(t)Idt=1,+1 2 +1 3 +1 4 ,
(15.38)
150 Normal Approximation
where, writing A for the largest eigenvalue of D,
Aga
A-1/z1,
11 1 11< C14( ) ^Ik+s+z
i/z
11th < i }\A,
=
n
EIIBZjhI m (m>0),
,lmmn -I
(15.39)
i-1
B being the symmetric positive-definite matrix satisfying B 2 = D - 1 . By Lemma 14.1(v) (with s'=m) and inequality (14.21) (Corollary 14.2), one has
IIB(I<(;)^/2,
A<;,
11,,, <2m (3) m/2 n
(m—s)/2A* r^
n—(m-2)/271m <2m(3)m/2n—(s-2)/2Qn's
(m>s+1).
(15.40)
Hence, putting m = k + s + 2, we •get n (s-2)/2I/(k+., + 2 )
Aj Iltll
.
An.t
)
I
(15.41) .
If we choose c, a to be the constant c 22 (k + s + 2, k) appearing in Theorem 9.11, then we get [using (15.40)] l i < C 7lk+.s+2n -(k+s)/2 < c 17 n -( s -2) / 2 ^s
(15.42)
For this we have used Theorem 9.11 and the remark after Theorem 9.12. Also, by Lemma 9.5 (with r = m, s = m + 2) and Corollary 14.2, one has [still using (15.40)]
f IDa[n
- '°/ 2Pm
(it: {X;})exp{— I}]Idt
< c 1s n —( s -2) / 2 06 s (s— I <m < k+s— 1),
(15.43)
whereas (14.10) yields n -m /2 Pm+2< 2
m+2 p2= 2m+2
k
(1 <m<s-2).
(15.44)
Main Theorems
151
The inequality (15.43) immediately leads to
12 < c19n - (,- z) / z O*. J .
(15.45)
Next use Corollary 14.4 to get [see (15.39), (15.41)] 1 3= fAw ID a QnI
(
„^-:^,:
)I/(*+2) }( 1 +Ilill l ° I )exp{ — illtll z }dt
j II 1 II>cis l
< c21 n zz
^w,i
n,s I f I t .11 ^,s•
k+ ^ + z (
1 + II t II I°I )
exp { - u II t II z } dt (15.46)
A similar calculation using (15.44) gives 14 - fA ID°5P'I < c 23 n -(,-2) / 2A*,,.
(15.47)
The estimation of the right side of (15.34) is complete, and we have
II(Q, - 4, ')' KI
(15.48)
Finally, by Lemma 14.6, IW8 \
2E : (tP')') — Wg (2E : (4 + ),) < sup J W (x : 2c)I(4,'+),o—(4'+)rjj(' ) ,
yER"
< 2 M,(I)ll(4" ). - (4)Il < C25Mr (f )On,fn
-(:-z)/z.
(15.49)
Now use (15.48), (15.49) to get an estimate for the right side of (15.31) and then use this estimate in (15.29) to complete the proof, noting that M,0(f) < 2 M,(.l), 0 , < Q.E.D. Q.E.D. Remark. In view of relation (7.22), JId(Qw
-
P)- f (I-c)d(Q,^- P) ,
for all c ER'. Hence M,(J) in (15.13) (and in all subsequent theorems involving it) may be
replaced by M, (J)= inf M,(f—c). eER1
(15.50)
Normal Approximation
152
COROLLARY 15.2. If p3 < oo, then for all bounded Borel-measurable functions f one has
J f fd(Q.-)I
k
)p 3 n - '/ 2 +2wf (c aP 3 n -1 / 2 :4?), (15.51)
where c 26 depends only on k. Proof. The corollary follows from the above theorem if one takes r = ro =0, s=3, and notes that in the present case the condition (15.11) is unnecessary. For if p3> 0 ,,.3{i)> c1n 1/2,
then
f fd(Q,
-4i )I
<2w1 (R' )< -- w1(R k )P 3 n -1 / 2
Now (15.13) yields [using (15.8)]
If
fd (Qn
- 0 - n -'/2P,(-,t:
{x,)))I
<wf(R k )n -1/2 A.3+ 2wf ( c4P3 n-1/2 : 0+ In -h/2Pi( -0 : {X,•))I) <w^(Rk)n-1/2p3+2wf(Rk )n -x/2 11 P1( - c:
{))II +2w7
(c4 P 3 n -1
/ 2 :4j).
(15.52) Lemma 13.1 does the rest. Q.E.D. COROLLARY 15.3. If p3 < oo, then for all Bore! sets A one has IQ.(A) -4 D(A)I
sup 40((aA)''+y),
(15.53)
yER h
where 71= c4P3n -
2 (15.54)
Proof. This follows immediately from the preceding corollary if one takes f - IA (the indicator function of A) and recalls that [see (2.40)-(2.43)] wf,, (X: 71) =
'A'\A
- I W=ltaA)'(x)
(xERk, ACR k , 71> 0 ),
w^^(rl:I)= sup f wj (x+y:rl)0(dx)= sup 4((aA)+y). (15.55) yER * yERk
Q.E.D.
Main Theorems
153
Remark. Suppose A is a Borel set such that ''(A) -0. With r—r0 -0, (15.13) then yields IQ.(A)-4(A)I
yE R'`
(15.56)
4 + (( aA) +y), "
where n— c4 p3 n -1 / 2 . If, in addition, A is a set having a small diameter, one may use inequalities (11.25) of Corollary 11.5, rather than inequalities (11.26) as used in (15.31), to get, instead of the term Mo(1A)II(Q.- 4')•K.II (in (15.31)), the expression y•(IA.:c)
where A denotes the Lebesgue measure on R k , and h„ is the density of (Q„-4/)•K1 With a-0 in (15.38), (15.42), (15.45)-(15.47), one obtains .
Ih.(y)I
i-])
/2
(yERk ),
p..r
and therefore
IQ. (A) — C(A)I
-( f
-2)
/ A:,Ak(A')
4 *((aA)"+y),
+2 sup
(15.57)
yERk
with c - c9 p3 n -1 / 2 , rl — c4 p3 n -1 / 2 . We make use of (15.56) and (15.57) in Section 17 (Theorems 17.4, 17.5).
Theorem 15.1 provides sharp bounds for a wide class of functions f. This is seen, for qxample, by comparing it with the more specialized asymptotic expansions in Chapters 4 and 5, or with the general expansions for trigonometric polynomials f as provided by Theorems 9.10-9.12. Section 17 contains several applications. However note that the right side of (15.50) (Corollary 15.2) does not go to zero (as n -'oo) for every bounded II'-continuous f when Q„ is the distribution of n '/ 2 (X 1 + • • • + X) and (X,, : n> 1) is a sequence of independent and identically distributed random vectors with zero means, covariance matrices I, and finite third absolute moments p3 . The reason for this is that the bound is in terms of wf [and wf (R k )] rather than wf . Recall that [see Lemma 1.2(iii), Theorem 1.3(iii)] that f fdQ„---. f fd4 for all bounded c1-continuous f and that a bounded f is (D-continuous if and only if -
lim wf (e:1)=0
{wl ( e:0)=
f
wj (x:e)(P(dx)l.
On the other hand, it is not difficult to construct bounded 4>-continuous functions f such that wj (e does not go to zero with e. For example, let A I , the indicator function of the following Borel subset A of R'. f=
: c)
((,-I)/21
A=U
U r-2
i—i
jxeR 1 :r+? <xr+ r !
(2i+1) r
,
Normal Approximation
154
with [(r— 1)/2] denoting the integer part of (r— 1)/2. It is easy to see that w^A (e:4i)= sup 0((8A)`+y)=1, yERk
W1A (E:4))=4D l (aA) ) <(27l)-1/2 ^ rexp t —
r2 11
'
1
,-
I
J E
(t>0).
It would be ideal if one could replace wj in the bound (15.51) by w . Unfortunately we are unable to do this. The situation is partially salvaged by the next theorem, which provides an effective bound for every bounded 0-continuous f. However a price is paid for greater generality. For "nice" functions and sets there is a loss in precision. On the other hand, apart from its greater generality, this theorem is more useful than Theorem 15.1 for estimating tail probabilities (see Section 17). !
THEOREM 15.4. There exist positive constants c; (i = 1, 2, 3) depending only on k and s such that if A,,,,(i) < cin t
s -2) / 2 (15.58)
holds for some integer s> 3, then for every Bore/-measurable function f satisfying Mr(f)<00
(15.59)
for some integer r, 0 < r < s, one has
f fd(Q„
—
'G)I
+),
(15.60)
where s-2
0 = I n (s 2)/2Pm( -
Y
m-0
-
-
4):
(x.))'
and 'q=ciP3([logn]+ 1)n -1 / z ,
(15.61)
where [logn] is the integer part of logn. The quantities 0 (3) , On.s are defined by (15.6) and (15.7), respectively.
Main Theorems
155
Proof We continue to use the same notation as in the proof of Theorem 15.1. Define , ', ¢" as in (15.15). As in (15.24), one has J fd(QQ - 4i)I
- c ,-2) / z +Iffd(Q '—O')I. ,,
(15.62)
Choose a probability measure K' on R k satisfying K'((x:IIxII<1))=1,
IDCK'(t)I
(15.63)
Such a choice is possible by Corollary 10.4. For E >0 and a positive integer p define K' (B)=K'(e - 'B)
(B E J^ k ), (15.64)
and note that KK. P ({x:llxll
By Corollary 11.2, with µ = Q,,, v = 4', KK = KK , P , and f=4' one has f J^,d(Q^—^'')I
(15.66)
where, by definition of y [see (11.8)] and of M,(J) [see (15.4)] y(f,:p€)<Mr(f)( 1 +IIxII +Ilan ,, + PE) r i(Q, - 4i')'Ki.PI(dx)
<M,.(f) f (C3o+ 2'llxll'+P'E')I(Q, - 41')*KK. P I(dx) < C31Mr(f)[
la+^^ m
ax
IID a (Qn
+p'€' max
Ia+flk
—
II D ° ( —
(15.67)
If r=0, then the first inequality in (15.67) holds because 4'(R k )=V(R k ) =1 [see (7..22) and (11.13)]. The last inequality follows from Lemma 11.6. In view of (15.63), one has
IID° (Q' ^F')'D"K.. P II1 -
I-6I
( ID ° (Q, — i')(t)^exp(
— PIIEt11 112 )
dt.
(15.68)
156 Normal Approximation
Now choose e=c33p3n-1/2,
p=[logn]+1,
(15.69)
and write
f
(15.70)
where, as in (15.38)-(15.47),
I1 <
^,:^
34 k+s-1
f ID a [n - '"/ ZPm (it:{x;})exp{-2(t,Dt)}]Idt
I2- m—s-1
< c n—cs-2)/2Q. n.5
3s
13 < c36n -( s - z ) / 2 A*n,s9
la'= f
I(D ° '')(t)Idt
Rk
,-z>/2 ^.
Is =I(D"Qn)(t)Iexp( — pIIE'II 1/2 )dt { II:II > n" 2 /(16p3) )
(1III>n'/ 2 /<^^))
exp(—plfEtlll/2)di.
(15.71)
The last inequality follows from Corollary 14.4 if one notes that 1 ID°Q„(t)I<EIIn- / 2 (Z1+... +Z,,)II 1 ° 1
<(Elln-1/2(Z1+... +Z.n)II"' I-I/mI, (15.72) ,)
for every even integer m'> Ial, and that for an even integer m', E lI n -1 / 2(Z I + • • • + Z„)II"') may be expressed in terms of ordinary (as opposed to absolute) moments that are estimated by putting t = 0 in
Main Theorems
157
Corollary 14.4. Also,
{IIII>n'/ 2 /(16v,)) xp { —
PIIEtII
1/z
} dt
=E ke xp{—PIItII112}dt (II111>c33/ 16 )
=C 39C
—k ^ 00
u k-I exp{—pu 1 / 2 }du
C33/ 16 2 =2C 391/EP)
kf
2k—I
w
expt
P(c33/16)i/2
exp{—v)dv
—v
(15.73)
}dv.
2
Now choose 2
s 2 2 )
C33 =64 (k+
(15.74)
to get (s 2)/2 < c 42 n IS < c4l(n1/2log2n ) kn -
-
-
-
(3 - 2) / 2
0 ,.
(15.75)
The last inequality in (15.75) derives from [see (14.7), (14.10), and (15.58)] n (s+1)/2
n -1/2c43
n ^I f IIY3II -
2
j-1
n
IIljII
s+
^ s0* . .s
(15.76)
J-1
Hence II D ° (Q^—^^)'DQK^.P^I1
c44(PE)II'In-cs-2)/2pn.s'
(15.77)
which leads to Y(fa :pE )
k+r+l
)n
—(s-2) / 2
^^ s .
(15.78)
158 Normal Approximation
Finally, by Lemma 14.6,
w^w (2pE:(¢') + )=w/ (2pe:0')+
f w/^(x:2pe)^(^'^)+-0+)(dx)
+ f (wf. (x : 2pt) — wf (x : 2p))¢ + (dx) <w1 (2pe:i + )+M,(f) f
(I+(IlxI +IIanII + 2pe) r )IO' — PI(dx)
2 + I.1 Q1(x : pc)((4'") — 4')(dx) <(Jf(2pE:1^if)
+C46 Mr(f)( 1 +(pE) r )n -(5-2)/2A*
(15.79) In the last step we also used the inequality w1 (x:2pc)<2M,(f)(1+(IIxII+2pc) r )
(r>0, xER k ). (15.80)
The proof is now complete by (15.78), (15.79), (15.66), and (15.62) if one writes rl = 2pe. Q.E.D. COROLLARY 15.5 Under the hypothesis of Theorem 15.4, one has
f fd(Q.
-1)I
<e47(s , k)Mr(J )( I+.l
k+r+' )(P3n -1/2 +p,n -cs-2)/2
+wf( 71: b),
)
(15.81)
where, as in (15.61),
71=c p 3 ([logn]+ 1)n" 2 Proof Proof. From the expression (14.74) one has IPm( — $: (X.))(x)l
(xER k , 1 <m<s-2).
(15.82)
Main Theorems
159
For, if I vi I > 3, I°_ 1 (I v ; I - 2) = m, then (recalling that p 2 = k) IX,,. .. X ,r ^ < c49(m k)PI ►,I
.
..
,
pk,I (II-2)/m
.
(15.83)
(1<m<s-2).
Now n
n-m/2Pm+2=n _I
n-m/2( f
+ f
11xj1Im+2
n
=pan- ► /2+psn-(s-2)/2
J i1xills) (15.84)
(1 <m<s-2).
Therefore 11 n -'"/2Pm
(- 4:
(x,) )H < c5 ► (s , k)(Psn -
► /2 + psn
-(s-2)/2)
(I <m<s-2).
(15.85)
Also, using (15.80), (15.82), and (15.84), we get wl(fl' I n-m/2Pm( --b : {)C►,})I) < c52(s , k)( 1 + n r )M,, (f)(Psn -
i/z + pan -(s-2)/2) (1 < m <s-2).
(15.86)
Now (15.81) follows from (15.60) if one uses (15.85) and (15.86), as well as the first inequality in (15.8). Q.E.D. Remark. The stipulation (15,58) [which is the same as (15.11)] is redundant when one has r-0 in Theorem 15.4 or Corollary 15.5. The proof of this assertion in the Corollary 15.5 case is essentially contained in the proof of Corollary 15.2. In the Theorem 15.4 case one needs the additional fact [see (7.22)] P.(-0:(y,))(R k )-0
(1<m<s-2).
(15.87)
160
Normal Approximation
16. NORMALIZATION The main theorems in the last section have been stated for independent random vectors X 1 ,...,X. satisfying (standard normalization) n EXj -0
n ' I Cov(X^)=1
(1 <j< n),
-
(16.1)
j -1
where I is the k x k identity matrix. This was done for the sake of simplicity. It is a relatively minor matter to extend all results of the last section to independent random vectors X 1 ,.. .,X„ satisfying EXj =O
(1 < j
n '
Cov(X^)-. V,
-
j-
(16.2)
1
where V is an arbitrary symmetric, positive-definite matrix. One could also take the mean vectors in (16.2) to be arbitrary. But since this is taken care of merely by changing the integrand f to a translate of it, we assume (16.2) throughout this section. Let Q,, denote the distribution of n - '/ 2 (X I + • • • + X,,). If f is integrable with respect to Qn and (D o. V , then by changing variables x-+ Tx one has f fd(Q, -41o.v)= f f°T ^d(G,-Z),
(16.3)
where T is the symmetric positive-definite matrix satisfying T
2 =V - ',
(16.4)
and G. is the distribution of n - '/ 2(TX 1 + • - - + TX.). Note that the random vectors TX 1 ,..., TX,, satisfy standard normalization, that is, E(TX1)=0
(1 'j< n),
n n - ' Y, Cov(TXj)=1.
(16.5)
j-1
Hence one may use the results of the last section to estimate the right side of (16.3) in terms of the moments - n
•r,-n ' I E^ITXj II' -
(r>0),
(16.6)
j-1
and M,(f°T - ') and w8r or wf, T -1, where (I + IIxIrr") - '(fo T - ')(x) Sr(x)= (foT
- ')(x)
if r>O, if r=0.
(16.7)
Normalization
161
Since T is easily computable, we may leave things at this stage. If one would like the bound to involve moments of X D 's and not those of TX's, then the simple inequality (r>0)
Tr
(16.8)
may be used. Here A is the smallest eigenvalue of V. Note that TII=A -1/2 ,
IIT
-I
II_A'/ 2 ,
(16.9)
where A is the largest eigenvalue of V. We now rewrite (or extend) the results of Section 15 in a series of theorems and corollaries, assuming that the n independent random vectors X 1 ,. . . , X,, satisfy (16.2). THEOREM 16.1. There exist positive constants c ; (i= 1,1,2,3,4) depending only on k and s such that if n (16.10) IITX^II'( 2 /3). 1 / 2 )
holds for some integer s > 3, then for every real-valued, Borel-measurable function f satisfying (16.11) M,(f)
J fd(Q,,—(Do.v)I <M,(f°T-1)(
C2T3n-1/2+C3T.,n—(s-2)/2)
g;( c4T3n-1/2: 4,
+2w
a)
G max (1,A r / 2 }Mr (f)(c 2X -3 / 2 p j n -1 / 2 + c 3 A — s/2
+ 2wwr(c4X -3/2
p3n -1/2: 4 r.).
p, n
_(s_2)/2)
(16.12)
Proof. The first inequality in (16.12) follows from (16.3) and Theorem