Normal Approximation and Asymptotic Expansions (Clasics in Applied Mathmatics)

f jflodx
40

Expansions of Characteristic Functions

with norm I/p

l fII =( f IfIPdx) ,

(4.1)

'

where f gdx denotes the integral of g on R k with respect to Lebesgue measure on R k . As usual, two functions f and g on R k are said to be equivalent if they are equal almost everywhere with respect to Lebesgue measure. The space L 2 (R k ) is a Hilbert space endowed with the inner product < •> 2 (4.2)

2= f f 8 dx.

Here g denotes the complex conjugate of g. For nonnegative integral vectors a=(a 1 ,...,ak ) we write X ° =Xa l • ••Xk

[X=(x^.....xk)ERk],

DDr'...Dk

[I<j
a=(a I ,...,ak )E(Z + ) k ],

l

(4.3)

and say that D°f is the ath derivative of the function f. Also, for each complex-valued function f on R k we define the function f by (xER k ).

f (x)=f(—x)

(4.4)

The function is said to be symmetric if f = f. For f E L'(R k ) the Fourier transform f of f is a complex-valued function on R k defined by (IER k ),

f (t)= fe'<`•">f(x)dx

(4.5)

where <.,> is the usual inner product in Ck : k

= L; lj xj [1=(ti,...,lk), X=(x 1 ,...,Xk )ECk ],

(4.6)

j-1

with c the complex conjugate of the complex number c, giving rise to the norm 1^ • ^1, '/Z

k

IIIII=(2 ItjI ) 2

(IECk).

(4.7)

The Fourier Transform

41

Here 1cl 2 =cc for complex numbers c. We shall sometimes use a different norm I • I in Ck : k

IxI= I Ix^I

(xEC k ).

(4.8)

j-1

THEOREM 4.1. Suppose f E L'(R k ). (i) f is uniformly continuous on R k . (ii) If (t)I < II f II for all t E R k . (iii) (Riemann-Lebesgue Lemma) If(t)I-*0 as lt^I-► oo• (iv) (Fourier Inversion Theorem) If (the equivalence class of) f E L'(R'), then one can recover (the continuous version of) f from f by the formula f(x)=( 21r)

-k ( e -i

Hence

<J.X>f(t)dt

(x ER")

J

J=(2)

J = ( 217 ) -k l•

(v) Let a = (a 1 ,. . . , a k ) be a nonnegative integral vector. Define g by g(x)=xaf(x)

(xERk).

If gEL'(R k ), then D°f exists and g=(—i) 1°1 D °f

(vi) If f E L`(R k ) n L 2 (R k ), then f E L 2 (R k ) and 11 f 112 = ( 21T) -k/2 11 f ^I2•

The last result (vi) says that the map defined by f-*(27r) - "`" 2f regarded as a map on the subset L'(R k )n L 2 (R k ) of L 2 (R k ) into L 2 (R k ), is an isometry. Since L'(R k )n L 2 (R k ) is a dense subset of L 2 (R k ), one can extend the above isometry on all of L 2 (R k ), and f so extended is still called the Fourier transform of f j E L 2 (R k )). The following theorem is then immediate. THEOREM 4.2 (Plancherel Theorem). The map f-^(2?r) - '`" 2 f is a linear isometry on L 2 (R k ), and
-k

2

[f gEL 2 (R k )J.

42

Expansions of Characteristic Functions

For f, g E L'(R IC) one defines the (equivalence class of) function(s) f*g, called the convolution off and g by (f*g)(x)

= f f(x—y)g(y)dy

(xER k ).

(4.9)

It follows from Fubini's theorem that f•gEL'(R k ) and that

III=gI[I< (ifIh IlgtI1

[ffgEL'(Rk)J.

(4.10)

The convolution operation is clearly commutative and associative. The n-fold convolution f " is defined recursively by ,f*'= f,

f*n = f`(n-l) * f

[n> l. fEL'(Rk )].

(4.11)

5. THE FOURIER-STIELTJES TRANSFORM

We shall now extend the concept of the Fourier transform to the set 6_11t of all finite signed measures on the Borel sigma-field of R". Let µ be a finite signed measure. We define the finite signed measure µ" by µ(B)=µ(—B)

(BE3Y').

(5.1)

A finite signed measure t is called symmetric if u = µ. The Fourier—Stieltjes transform µ of a finite signed measure µ is a complex-valued function on RC defined by µ(t)=

f e <' ,

x

>µ(dx)

(:ER', µE ')1L),

(5.2)

where, as usual, the integral is over the whole space R k . If µ is a probability measure, µ is also called the characteristic function of s. Note that if s is absolutely continuous (finite signed measure) with respect to Lebesgue measure, having density (i.e., the Radon—Nikodym derivative) f, then

µ=f.

(5.3)

The convolution µ*v of two finite signed measures is a finite signed measure defined by (µ*p)(B)=

f

(B-x)p(dx)

(BE iYC),

(5.4)

The Fourier-Stielijes Transform

43

where for A C R"`, y E R k , the translate A +y is defined by A+y=(z=u+y:uEA)•

(5.5)

It is clear that convolution is commutative and associative. One defines the n-fold convolution u*" by µ *1 =µ µ*"=µ*("- i) *µ

(n> 1,

e'X).

(5.6)

Let be a signed measure on R k . For any measurable map T on the space (R k , GJ3 k ,µ) into (R 3 ,Jas) one defines the induced signed measure µ ° T -' by (µoT-')(B)=µ(T-'(B))

(B E'3).

(5.7)

THEOREM 5.1 (i) (Uniqueness Theorem). The map µ—*A is one-to-one on 671.. (ii) For u E JR, , µ is uniformly continuous and µ( 0 ) = µ( R k ) , Iµ(t)I
(tERk).

(iii) For µ, v E OiL µ*v=µ $. (iv) If ft E Olt , then µ=µ,

so that µ is symmetric if and only if is real-valued. The symmetrization µ*µ" of is always symmetric since *µ=IZ. (v) If T is an affine transformation on R k into R 5 : Tx = a + Bx, x E R k , then

T- ► (t)=ei<<.a>µ(B't)

(1ERs, IRE

where B' is the adjoint (or transpose) of B.

),

44

Expansions of Characteristic Functions

(vi) If µ E Jll, and f Ix °IIjI(dx)
µ(dx)

I,

... ,

a d,

then D ° µ exists

(t E Rk)

(vii) Ifµ E L 1 (R'`), then is absolutely continuous with respect to Lebesgue measure with a uniformly continuous and bounded density. (viii) (Parseval's Relation). Let µ, v be two finite signed measures on R k . Then

f µ dv = f i dµ. The following theorem (due to Cramer and Levy) gives a useful characterization of weak convergence of probability measures on R'`. THEOREM 5.2 Let (G„ : n> I) be a sequence of probability measures on R k . If the sequence converges weakly to a probability measure G, then {G„) converges pointwise to G. Conversely, if {G„} converges pointwise to a continuous limit h, then there exists a probability measure H such that {G„} converges weakly to H and H = h. It should be noted that if { G„ } converges weakly to G, then {Ô) converges to G not merely pointwise, but uniformly on compact subsets of R k . This is because the class of functions }

(e < r,. >:tEK), ;

where K is a compact subset of R k , is equicontinuous and uniformly bounded; hence, by Corollary 2.7, it is a uniformity class for every probability measure G.

6. MOMENTS, CUMULANTS, AND NORMAL DISTRIBUTION vk ) is a nonnegative Let G be a probability measure on R'. If v = (v 1 v 2 integral vector such that Ix'I is integrable with respect to G, one defines the v-th moment IA, of G by ,

µ,,

= f x'G (dx).

, ... ,

(6.1)

Moments and Cumulants 45

For every nonnegative real s one defines the s-th absolute moment ps of G

by PS=JIIxII'G(dx)

(s>0).

(6.2)

Note that, by Theorem 5.1 (vi), if µ, is finite for some nonnegative integral vector v, then iI°Iµ=(D°G)(0)

(6.3)

and, therefore, one has the Taylor expansions O(i)=l+ ^ µ vi (Iltlls) + o Y

(

IPI<s

( t-> 0),

(6.4)

if ps is finite for some positive integer s. Here v!=v 1 !v 2 !

.

.. Vk !

[V=(V1,....'k

)].

(6.5)

If k=1, = f xJG(dx)

(1=1,2,...).

(6.6)

For a positive real number x, logx denotes the natural logarithm. In other words, the function x.-*logx on (0, oo) is defined as the inverse of the

function x--),exp (x) = 1 + x + x 2 /2! + x 3 /3! + • on R'. For a nonzero complex number z = r exp (i8) (r>0, - it < 8 < ir), we define logz=logr+iO (r>0, -^r<0 it). (6.7) Thus we always take the so-called principal branch of the logarithm. The characteristic function G of a probability measure G on R k is continuous and has a value of one at 0. Hence in a neighborhood of zero one has the Taylor expansion t = ^ Jc^. (it) V +o to gG O— (IIt IIS) ( t- 0 ) ,

(6.8)

IPj<s

if ps is finite for some positive integer s. Here the summation is over tNote that Ixi < I1x11 H , so that I, I

Also, see Corollary 8.3.

Expansions of Characteristic Functions

46

nonnegative integral vectors v, and (see Corollary 8.3) = (D log G )(0).

(6.9)

The coefficient y, is called the v-th cumu/ant of G. A simple calculation gives Xo=0,

X,,=µl,

if lvl=1.

(6.10)

The following formal identity enables one to express a cumulant x in terms of moments by equating coefficients of t' on both sides:

II

' X(v)•=(-1)3+is z^ =i 11 >

(a)

)

J.

(6.11)

/

This is obtained by noting a, log(I+z)=

(-1)° + ' 1 z'

(zEC, lz^
(6.12)

For example, in one dimension X21L2! LI2,

X3 — µs - 3µ2µi+2µi3,

X4=µa -4 µ3µi -3 µZ 2 + 12 µ2µ1 Z-6 µi °, (6.13) Xs=µs -5 µ4 µi — 10 µ3µ2+ 20 µ3µi 2 + 30 µ2 2µI —60 µ2µi 3 + 24 µI 5 .

In general, in R k one may write li..

J

(6.14)

where the summation is over s =1,2,. . . , vi and for each s over all s-tuples of nonnegative integral vectors (v 1 ....,v5 ) and s-tuples of nonnegative integers (j l ,...,j,) satisfying (6.15)

and c(v 1 ,...,v5 ;j,,...,j,) is a constant depending only on (v,,...,v,) and (jl.... is). ,

For a given probability measure G on R" with cumulants {X,, : (vi < s),

Moments and Cumulants 47

we define the polynomial X,.(z) in z = (z ,, ... , z k ) E C k by i

(zECk; s= 1,2,...).

(6.16)

IIS

We point out that for t E R', X,(t) is the sth cumulant of the probability measure on R' induced by the map x-* defined on the probability space (R", J^'Y,G) into (R',). This follows because the function u -*log G (ut) [u E (- a, a) for some a>01 is the logarithm of the characteristic function of this induced probability measure on R' and because [by ( 6. 8 )] "

it

d -S log6(ut)I =s 1 2 X" v) ( -o

I V I- S

Often we find it convenient to state results in terms of random variables and random vectors (although for purposes of this monograph it would be possible to avoid such notions). The with moment µ"(X), the with cumulant y, (X), and the sth absolute moment p(X) of a random vector X in R k are defined to be the corresponding characteristics of the distribution P x of X. Note that for a random vector X with a finite covariance matrix, Cov(TX+a)=Cov(TX)= TCov(X)T', (6.17) where T is an m X k matrix (l < m < k), T' is its transpose, and a E R'". Also, if (X 1 ,...,X„) are n independent random vectors in R' and p 2 (X 1 )
(6.18)

If X is a random vector in R k and has a finite with moment (and cumulant), then for every c E R', N,(cX)=c 1 " pti(X)

(lv >1 ), (6.19)

X,.(cX) = c"""X"(X)

and for all cER', bERc,

x"(cX+b)=X„(cX)=cIX,,(X)

if lvI > 2,

X,,(cX+b)= y. (cX+b)=cµ"(X)+ =cX,,(X)+
if ivl=1.

(6.20)

48 Expansions of Characteristic Functions

It may also be shown, either by direct computation, as in (6.13), or by looking at (6.14) and (6.15), that X,(X)=(X) for Iv1=2,3,

if E(X)=0.

(6.21)

The relations (6.19) and.(6.20) follow easily from definitions; note that in a neighborhood of t=0 logPPx+b (t) = i+logP X (ct).

(6.22)

Since the cumulants y, are invariant under changes of origin for Ivl > 2, they are sometimes called semi-invariants. The effect on y (and µ,) of a change of scale, as given by (6.19), is also quite simple. However the most important property of cumulants from our point of view is the following: If { X 1 ,.. . , X, j are independent random vectors in R k each having a finite with cumulant, then (X1+..• +Xn)=x,(X1)+ • • • +X„(Xn).

(6.23)

This follows from n

logPXl+.,.+X.(t)= 2 logP Xj (t),

(6.24)

js1

which holds in a neighborhood of 1=0. We now derive some inequalities for moments and cumulants of a random vector X. Henceforth, if there is no possibility of confusion, we simply write µ., p S , 7^, for µ,(X), p s(X), L(X), and so forth, respectively. The following basic inequality is stated without proof.t It is used fairly often in this book. LEMMA 6.1 (Generalized Holder Inequality). Let f 1 , 1,< i < m, be measurable functions on a measure space (2,Q,µ) into (R', V) such that If1 1", 1 < i < m, are integrable for some m-tuple of positive reals (p t , ..., p m ) satisfying pj ' + • • • + pm ' =1. Then

f Ifs = fmldµ
-

LEMMA 6.2. Let X be a random vector in R k having a finite sth absolute moment p s for some positive s. If X is not degenerate at 0,; tSee Hardy, Littlewood, and Polya [1] for several different proofs of this important inequality. That is, Prob(X-0)3. 1.

Moments and Cumulants 49

(i) r-->log p, is a convex function on [0, s]. (ii) r-^p /` is nondecreasing on (0,s]. (iii) r_►(p,/p 2 1 / 2 ) 1 / ( r -2) is nondecreasing on )

(2,s] if s>2.

Proof. We may assume that X is not degenerate at 0, so that log p, is defined for 0 < r < s. (i) Let 0 < < 1, 0 5 r,, r 2 s. Then Pa,,+(0 —a)r i =

E (IIXIlar,+(1 —a)r,

) = E (YZ),

where Y=I;XII«r', Z=IIXI )r '. Since E(Y'/a) and E(Z"° - a ) ) are both finite, by Lemma 6.1 (with m=2), E (YZ) c(E (Y l /a)) a (E (Z'/u

p

=

r«

p

^^

-

a,

so that log par, +(i - a)r 2 < a log p, , +(l - a) l og p, 2 . (ii) Since log p o = 0, and (11r) log pr is the slope of the line segment joining (0,logpo ) and (r, log p,), it follows by the convexity of r--*log pr that

the function r-+Iog p; = r logy, is nondecreasing. (iii) If p 2 = 1, then the function r

( -2)

r^logP

=(r-2) -I logPr=(r-2)-)(logy,-logP2) (6.25)

is increasing in (2,s], since the expression on the extreme right of (6.25) is the slope of the line segment joining (2,logp 2 ) and (r,logp,), and r- logp, is convex in [0,s] by (i). In the general case, apply this argument to Y X/pi /2 . Q.E.D. A simple application of Lemma 6.2(ii) yields r

(tai l <mr -I Ila i VV

(a 1 ER', 1
(6.26)

for every r> I and every positive integer m. It also holds, trivially, for r=0.

50

Expansions of Characteristic Functions

LEMMA 6.3. Let X be a random vector in R k having a finite sth absolute moment p S for some positive integer s. Then for nonnegative integral vectors v satisfying Ivl ( s, (6.27)

,LLl < EIX"I < p H ,

and there exists a constant c 1 (v) depending only on v such that (6.28)

Ix.l < c1(v)P1'1. Proof. The inequalities (6.27) follow from the simple inequality

(6.29)

x'I < 11xII''. The inequality (6.28) follows from (6.14) and (6.15) by noting that µV; ... µ s I

. per < p^ PIV I ..

V1

E 1;I,l/IYI "' =p1.

(6.30)

The first inequality in (6.30) follows from (6.27), the second from Lemma 6.2(ii). Q.E.D. Before concluding this section, we mention a few properties of the single most important probability measure on R "—the normal distribution 4,,,, v , whose density 4, n v (with respect to Lebesgue measure) is given by 4)m v (x)=(277) -k/z (DetV) -1/Z exp{

-- <x,V -

^x>}

(xER k ). (6.31)

Of the two parameters, m ER" and V is a symmetric positive-definite k x k matrix. The notation Det V stands for determinant of V, and V' is, of course, the inverse of V. It is well known thatt (D,,,. v (t) = exp { i — 2 )

(t ER").

(6.32)

From this it follows that m is the mean and V is the covariance matrix of For the computation of cumulants X, for II > 2, it is convenient to take m =0 (for It > 2 a change of origin does not affect the cumulants x.). This yields log f o, v (t) = — 2 tSee Cramer [4], pp. 118, 119.

(6.33)

The Polynomials P,

51

which shows that X„ = (i,j) element of V if v = ei + ej ,

(6.34)

where e1 is the vector with I for the ith coordinate and 0 for others, < i k. Also, X„=0

if Ip1>2.

(6.35)

Another important property of the normal distribution is

(D m1, V l * ( Dm 2 . V,* ...

* 0

(6.36)

V = 4i m ,,,,

where m=m1+m2+••• +m,

(6.37)

V=V i +V 2 +•••+V,,.

This follows from (6.32) and Theorem 5.1(i), (iii). The normal distribution (D o.r , where I is the k x k identity matrix, is called the standard normal distribution on R k and is denoted by 4'; the density of 4' is denoted by .0. Lastly, if X = (X 1 , ... , X k ) is a random vector with distribution 4',., ^,, then, for every a E R k , a0, the random variable (a, X> has the onedimensional normal distribution with mean and variance = ^k a a i ja r^ , where ij=I a j =(i,j) element of V=cov(X i ,Xj )

(i,j= I....,k).

7. THE POLYNOMIALS P, AND THE SIGNED MEASURES P J Throughout this section v = (v i , ... , v,,) is a nonnegative integral vector in R k . Consider the polynomials

X^ 7,(z)=s!

lvIas P !

(z`=z^

zP

,...

(7.1)

z)

in k variables z 1 , z 2 ,...,z k (real or complex) for a given set of real constants x,. We define the formal polynomials P3 (z: (X„)) in Z 1 ,....Z k by means of the following identity between two formal power series (in the real

variable u). 1+ I P,(z: {X })u s =exp '

X,+2(z) !

u' m

=l+ 1+I ,

m=i

E s + 2)! us .

m! L•^ (s+2)!

s=i

(7.2)

52

Expansions of Characteristic Functions

In other words, Ps (z : {7c}) is the coefficient of u' in the series on the extreme right. Thus

Ps( z : (X"})=

t



V1 f

L

,r,l

(il

X1 1 +2( Z ) Xj2+2(z) ...

Xj„ + 2 ( Z )

(11+2)! (12+ 2 )!

(Jm+2)1 1}



^^

1 0 -0,

1

mt

(s=1,2,...),

(7.3)

where the summation X* is over all m-tuples of positive integers (j 1 ,...,jm ) satisfying M (7.4) ji =1,2,...,s (1G i<m), >ji=s, and S** denotes summation over all m-tuples of nonnegative integral vectors (p 1 ,. . . , vm ) satisfying kI=j; +2

(1
(7.5)

In particular, z

P 1 (z: (X,,})= X 3 i =

-z", I"I' 3

X4( z )

x 32

(

z

)

P2 (z: {X})= 4! + 2!(3!)2

P3(z:})=

x5(z)

5!

+ X4( Z ) X 3( Z ) + X 3( z)

3!4!

(3!)a

(7.6)

LEMMA 7.1. The degree of the polynomial PS (z: {X,}) is 3s, and the smallest order of the terms in the polynomial is s + 2. The coefficients of Ps (z: {x.)) only involve X„'s with M <s+2. Proof. This follows immediately from (7.3). Q.E.D. The notation above has been chosen to suggest that eventually we use cumulants of some probability measure for X„'s in the expression for PP (z: (X"}). The polynomial Pt can be defined in this sense only if the

The Polynomials P, 53

(s + 2)th absolute moment p5+2 is finite. The role of the polynomials P S in the theory of normal approximation is briefly indicated now. More precise results are given in Section 9. Let G be a probability measure on R k with zero mean, positive-definite covariance matrix V, and finite sth absolute moment p, for some integer s>3. Then [see (6.8), (6.16), and (6.23)1 log e"

\ n 1/2) =n

logG ( I) -

s-2

!t t = — i + E (r+2)! n '/Z+n o(^^ / 2 ii `

,

n

i

)

Lo)

(7.7)

.

(

Thus for any fixed t E R k ,

G"

)

(

=exp(--L)-exp

I

n ^/ 2

S-2

(,12(" n-'/2+o(n-(s-2)/2)

n1/2 s-2

=exp{-i
n - '/ 2 P,(it: {X,})

Vt>}. l+

r=!

x (I +o(n -js-2) / 2 ))

(n-oo),



(7.8)

where, in the evaluation of P,(it: {X„}), one uses the cumulants y of G. Thus, for each t E R k , one has an asymptotic expansion of G" (t /n 1 / 2 ) in powers of n - 1 / 2 , in the sense that the remainder is of smaller order of magnitude than the last term in the expansion. The first term in the asymptotic expansion is the characteristic function of the normal distribution'I ,. The function (of t) that appears as the coefficient of n - '/ 2 in the asymptotic expansion is the Fourier transform of a function that we denote by P,(-0 0. ,,: (y,)). The reason for such a notation is supplied by the following lemma. LEMMA 7.2. The function t-*,(it:{x.})exp{-z)

(IGRk),

(7.9)

is the Fourier transform of the function P,(-40 o v : {X„}) obtained by formally

54 Expansions

of Characteristic Functions

substituting (— 1) IHI D V *O v

for (it) " (7.10)

for each v in the polynomial P r (it: {X,,)). Here 4, is the normal density in R k with mean zero and covariance matrix V. Thus one has the formal identity P,( - 4o: {X,))= P,( —D: {X„})$o.v

(7.11)

where —D=(—D I ,..., —Dk ). Proof. The Fourier transform of 0 o, v is given by [see (6.32)]

— Z}

(tERC).

(7.12)

D qo v(t)=( it)r*o.v(t)

(tER"),

(7.13)

0o,v(t)=exp{

Also

which is obtained by taking the with derivatives with respect to x on both sides of (the Fourier inversion formula) 00,v(x)=(2sr) -k

f

exp{—i}$ o. v (t)dt

(xER k ) ( 7.14)

[or, by Theorem 4.1(iv), (v)]. Q.E.D. We define P,(— (D o, v : {7t,.}) as the finite signed measure on R k whose density is P,(- 4o.v:(X„ }). For any given finite signed measure µ on R", we define u(•), the distribution function of µ, by µ(x)=u((—oo,x])

(xER"),

(7.15)

where (

—

oc,X]=(

—

oo,X1]X .. • X(

—

ci,X k ]

[X= (x1,...,Xk)ERk}. (7.16)

Note that D,... DkP,( — 'Do.v: {X.})(x)= P,( — $o,v: {X,})(x) ,(—D: {X, , }) 40.v(x)

=P,( —D: (X, ))(DI ,

...

DkOo.v)(x)

= D j ... Dk (P,(— D: {Xr))4o.v)(x). (7.17)

The Polynomials P, 55

Thus the distribution function of P,( — (D o, I, : {x,}) is obtained by using the operator P,(— D: (x)) on the normal distribution function (D o, I,(• ). The last equality in (7.17) follows from the fact that the differential operators P,(—D: {X,}) and D I D 2 • • • Dk commute. Let us write down P,(-40: (x,)) explicitly. By (7.6), P I (it : (x}) =

(7.18) -- (it) ° (I ER"),

so that (by Lemma 7.2) P1(

—

$0,v: {x,})(x)

_—

Irl-3

Dr0O.v(x)

k —

6 X(3.0.....0)

—

3

II

v ljxj )

k

+3v 11 k v l'xj

j-I

j-1 k

vkjx. +3v

+... +X0.....0,3) — (

k j -1



k

2 k

2 "12.1.0.....0) — 2I vlxj El^ 2jx! )

k

k

+Zv l2 vlixj+vll j-1

2 k

k +... + X10,....0.1.2)

k

v2Jx1 j-I

k—Ijx ^IVk^xj) (^ I V j

k

+2v k ' k-I 2 v kjxj +v kk 2 Uk-IjxJ j- 1 j-1

Jj

Expansions of Characteristic Functions

56

—

X (1.I.1.0,....0) — (J

x)(±

U2jXl /

I

k



1 v3JxJ /

\

k

k

v 12 U3jx,+U13 2 V2iX^') U 23 UIIXj +...

l-1

JAI

k +X(0.....0,1,1,1)

f-I

I k

k k—IjxJ Vkjxl

Uk-2jXj)(j- U 1 JiI

/

k

+Uk-2.k-1

k UkJx.+Uk-2.k



Uk-1jX.

l -1 l—I

l

k

+Uk—1,k

2

Uk-2jkJ

l- 1

OV(x)

r V -1 =((v')), x=(x l ,...,xk )ER k ]. (7.19)

If one takes V= I, then (7.19) reduces to P1(-4: ( ))(x) =

{

-

—

6["13,0,....0)(

— X^+ 3 X1)+ ... +x((0....,0.3)( — xk+ 3 xk)]

2 [X(2,1,0.....0)(

— x^x2+x2)+ ... + X(0.....0,1.2)( —Xk xk—I +Xk—I)]

[X(1,1,1,0,...,0)(

—x 1 x 2 x 3)

+... +

X(0,...,0,1,1,1)(

Xk xk—I xk-2)]}'O( x )

(x E R k) (7.20)

where is the standard normal density in R k . If k= 1, by letting X^ be the jth cumulant of a probability measure G on R'(j = 1, 2, 3) having zero mean, one gets [using (6.13)] P1(

—

*: {7G.))(x) = 6 µ3 (z3-3x)$(x)

(xER'),

(7.21)

where µ 3 is the third moment of G. Finally, note that whatever the numbers (X,) and the positive definite symmetric matrix V are,

f P,( — $o v: {^))(x)dx=P,( -4 o,: {x,))(R k ) =0,

(7.22)

Approximation of Characteristic Functions

57

for all s > 1. This follows from

for III 1.

f (Dpo o, v )(x)dx=0

(7.23)

The relation (7.23) is a consequence of the fact that 40 o v and all its derivatives are integrable and vanish at infinity.

8. APPROXIMATION OF CHARACTERISTIC FUNCTIONS OF NORMALIZED SUMS OF INDEPENDENT RANDOM VECTORS Let X 1 ,. ..,X ben independent random vectors in R k each with zero mean and a finite third (or fourth) absolute moment. In this section we investigate the rate of convergence of Px,+...+x.)/nt/2 to 1 o.v , where V is the average of the covariance matrices of X 1 ,... ,X. The following form of Taylor's expansion will be us-;ful to us. LEMMA 8.1.t Let f be a complex-valued function defined on an open interval J of the real line, having continuous derivatives f ` of orders r=1,...,s. If x, x+hEJ, then (

)

s—I

f(x+h)=J(x)+

h^ fi 1 (x)+

(s hs l)i

f ' (1–v) s- 'f( s l (x+vh)dv.

–

(8.1) COROLLARY 8.2. For all real numbers u and positive integers s (to)' I J u

exp { iu } — 1 — iu — • • • — (s

—

1)! < —. l(8.2)

Consequently, if G is a probatility measure on R k having a finite sth absolute moment p s for some positive integer s, then i

G(t)

—

I

—

iµi(t)

—...

—

1)! µs -1(t) $ (t) ^ p511tlls s ^ s^ tE ( R k )+ ( 8.3 )

tHardy [fl, p. 327.

58 Expansions of Characteristic Functions

where for r = 1, ... , s, vi µrt r ^ljs(t)= f II s G(dx). (8.4)

A'(t)= f 'G(dx)= Irl - r

Proof The inequality (8.2) follows immediately from Lemma 8.1 on taking f (u) = exp (iu) (u E R 1 ) and x = 0, h = u. Inequality (8.3) is obtained on replacing u by in (8.2) and integrating with respect to G(dx). Note that

a:(1)= f II'G(dx) < Iltll'p,. Q.E.D. COROLLARY 8.3. Let f be a complex-valued function defined on an open subset SI of R k , having continuous derivatives Drf for I vl <s (on SZ). If the closed line segment joining x, x + h (E R k ) lies in St, then vl (Dr7)(x)

f(x+h)=J(x)+ 1
+s 2 vl j l (1—u) s-1 (Drf)(x+uh)du,

(8.5)

U

Irl—s

and therefore V

f(x+h) — f(x) — hl

1
<

(D rf)(x) r



Ivll max{^(D7)(x+uh)^:0
Irl-S

Proof. Define the function g on (— c, I + c) by (8.7)

g(u) =f(x + uh),

c being a positive number for which the line segment joining x-eh, x+(l +e)h lies in S2. Using the formula for differentiation of composite functions, one obtains, by induction on r,

g (r)( u ) = vl h'(Drf)(x+uh) Irlsr

(r= 1,.... ․).

(8.8)

Approximation of Characteristic

Functions

59

Hence, by Lemma 8.1, g(1)=g(0)+2 g( r 0) + (

f ' (1—u)s -I g ( : ) (u)du,

(s 1)1

r— 1

which, on substitution from (8.7) and (8.8), yields (8.5). The inequality (8.6) follows immediately from (8.5). Q.E.D. Let X i ,. .. , X„ be n independent random vectors in R' each having a zero mean and a finite sth absolute moment for some s> 2. Assume that the average covariance matrix V, defined by V= n I Cov(Xj )= n 2, Vj ,

(8.9)

l —1 j-1

is nonsingular. Then we define the Liapounov coefficient ii,, by n

n -1 I E(j 5 ) l^

II^II—I

n —(s —2)/2 (s > 2).

n s/2

!s „= sup n

1

(8.10)

2 E ( 2 ) j-1

It is simple to check that 1, ,, is independent of scale. If B is a nonsingular k x k matrix, then BX I ,...,BX„ have the same Liapounov coefficients as X 1 ,...,X, have. If one writes (1<j'cn)

Prj—E(IIXjIIr)

,

n

P

Pr = n

(8.11)

(r>0),

rj

j=1

then it is clear from (8.10) that l,.n n -(: z)/z sup

P, II t CI = PS S

IItII= i fit, Vt>s/2



n

^s12

-(s- 2)/2 (8.12)

where X is the smallest eigenvalue of V. In one dimension (i.e., k= 1)

!S n _P P /2 n

—(s-2)/2

(s > 2).

(8.13)

We also use the following simple inequalities in the proofs of some of the

60 Expansions of Characteristic Functions

theorems below. If V= I, then.. n EI

n' /2l,,nlltIJ .

(8.14)

In the rest of this section we use the notation (8.9)-(8. 11), often without further mention. THEOREM 8.4. Let X 1 ,...,X, be n independent random vectors (with values) in R'` having distributions G 1 ,...,G, respectively. Assume that each Xi has zero mean and a finite third absolute moment. Assume also that the average covariance matrix V is nonsingular. Let B be the symmetric positivedefinite matrix satisfying B2= V '. (8.15) -

Define -

r..2

11

d2

)

r-2 ,

lT)

b„(d)=2-d(da(d)+1)(ls.n)2/3. 6

(8.16)

Then for every d E (0, 2' / 2) and for all t satisfying

Iltll ( d'/',

(8.17)

one has the inequality

n

^G( nBt 1/2 )-exp{-2IItIl2} i-^

<(da(d)+-6L)13,nII1Il3exp{

-

b„(d)IItII 2 }• (8.18)

Proof. Assume first that V= I and, consequently, B =1, where I is the

identity matrix. In the given range (8.17) one has [see (8.14)]

(

)- ll <(2n) - ' E 2 n f) <(2n)-'(EIIs)2/3 2/3

<(2n) -

^+ E I< t,X1> 13 '( j o> <(2n) - '(n 3 / 213 IItjI 3 ) 2/3 = 2/3 n 3 IItII 24 d2,

(8.19)

Approximation of Characteristic Functions

61

so that logGj (t/n 1 / 2 ) is defined for 1 < j < n, and [using (8.14)] n

log II Gj (

n

) + 22 I

j^l

_ 2 ^Iog(I—(1—Gj\ j \ -1

n



n

l))+(2n)_'E21

a0

r

+ Gj ( ^ /2 ) l + (2n) - I E 2 ]

n

<

n

J

2

°O

2 r-2

(4n2)- ( E < t,Xi> 2) jr-1(^) l

j-1

+6n-3/2EII3

r-2

n

2 ( E I< t,Xj>l 3 ) 4/3+ 6 /3.nll t II 3

< a (d )n -2

j -1

n 4/3

EIl3)


6 13.nII t ll 3

+


(8.20)

Hence, noting that fie"— 11 xleNNN for all complex numbers x, one has n Gj( 1 n /2 ) — exp{ j

—

-

2IItII 2 }

I n

=exp{

—

_ n

i1It1I 2 } exp log IIGj ( j

<(da(d )+

1 ) 13,nlltll 3 exp

1

—1

{ — 2II , II 2

+d(da(d)+ 6 ) 13,3 11 1 11 2 } =(da(d)+6)/3.fIltll 3 exp{

—

b,,(d)11 1 11 2 }.

(8.21)

62

Expansions of Characteristic Functions

This proves the theorem when V= I. In the general case, look at the random vectors B X 1 ,.. . , B X„, whose average covariance matrix is 1, and recall that the Liapounov coefficient is independent of scale. Q.E.D. By going through the proof it is easy to see that if the X D 's in Theorem 8.4 are i.i.d. with common distribution G, then G"( B I exp{

—

1IItI1 2 } <(da(d)+ L ^13,„Illll 3

Xexp { _1'1_d2a(d)_ 6

(8.22)

for all d E (0, 2 1 / 2) and for all t satisfying

(8.23) II 1 II < dl3 • The next two theorems sharpen (8.22) and (8.18), under the additional assumption of finiteness of fourth moments. THEOREM 8.5. Let X be a random vector in Rk with destribution G. Suppose X has zero mean, positive-definite covariance matrix V, and a finite fourth absolute moment p 4 . For all t satisfying

IItII < 1 /4-„t !

(n> 1),

(8.24)

one has the inequality

I

G"l B^ 2 )—exp { — iIIIII Z }( 1 + 6i3n-t/2µ3(1))I

<[(0.1325)n - 1 + 2 l4." ]IItII 4 exp { — Iltlu Z }

+(0.0272)li „IItII 6 exp { — ( 0 . 3835 )IIt1I 2 }

(n> 1), (8.25)

where µ3(t) = E 3 , and B is defined by (8.15). Proof. Assume that V= I. In the given range of:

I

G

<(2n)-'IItII2<e. (8.26) ( h)— ll -

Hence

logG" (

n /z )+

1

11 1 11 2 — I i 3n - 11Zµ3(t)

I

=nllog[I—(1—G(n ^Z))l+(2n)-hIItII2-6i3n-3/2µ3(t)

Approximation of Characteristic Functions

=n - Ir - `(1-G( r-z

i n /Z )) r

63

+6( i n /Z ) -1 +( 2 n) ^Iji) 2

6t3n-3/2A3(t) "0

<( 4 n) - ^Iltll ° r - '(sy -Z +iala,"Ilrll 4 r-2

<[(0.1325) n '+i14 "]Iltll ° (8.27) -

Similarly,

I

n log G ( n 1 /Z ) + II t II Z ^ <(0.1325)n - `II t II ° + 6 13."II tll 3 <(0.0663+0.1667)1 3 "11 1 11 3 =( 0 . 233 ) 13 "11 1 11 3 <(0.1165)II 111 2 .

(8.28)

We then have

t(

{ n f2 ) - exp { - illtll 2 }( ] + 6i 3n

-

1/2µ3( l )) 1 3

=exp{

-

211 1 11 2 }^exp{logo"( t +211 1 11 2 ,"( ( + 6n - ` /2µ3(t) 2

<exp( - 211t11 2 )^exp{loge"(

1 n `/Z

1+loge"

+exp{ - 2IItI1 2 }iog&"(

`/2

+ U1111 2 }

(n 1 J2 J

)+iiItI{ 2- 6n -l/Zµ3(i)^ z

r2

<exp{ - 2IItII2} 2^logG"\n`/z)+iIItII2^ exp^^logG"t n'/z)+ II 21 +exp{ - ?IItI1 2 }[( 0 . 1325 )n - `+2 1°,"^Iltli ° <(0.0272)l "IItII 6 exp { -0.383511tf1 2 }

+ {(0.1325)n - `+ 2 1 °,"] II t II 4 exp ( - 1 II t II') .

(8.29)

Expansions of Characteristic Functions

64

Ixl 2el`I in (8.29) [as well as the We have used the inequality Iex — 1— x estimates (8.27), (8.28)]. This proves the theorem for V= I. In the general case look at the random vectors BX I ,...,BX,. Q.E.D. THEOREM 8.6. Let X 1 ,...,X, be n independent random vectors in R k having distributions G 1 ,...,G, respectively. Suppose that each random vector Xi has zero mean and a finite fourth absolute moment. Assume that the average covariance matrix V is nonsingular. Also assume

l

1.

(8.30)

<21,.n'j°,

(8.31)

4

Then for all t satisfying

IltII one has n

3

III G( BI Z )exP{

—

iIIrII Z }( 1 + 6n -112µ3( 1 ))

I -1

(0.175)14,,IItlI 4 exp { — i IItII'} + [(0.018)l ,.111118+3613,, IItll6]exp{—(0.383)IIt1I2}, (8.32) where B is the positive-definite symmetric matrix defined by (8.15), and E
µ3(t)=n-'

J- 1

Proof. We may, as before, take V= I. Note that [by (8.14) and (8.3 1)] i

Iz ( I/Z)-1 <(2n)-IE2<(2n)-1(E.4)

n

n

1/2

<(2n) -1 (^ E
(8.33)

J-1

in the given range of t. Proceeding as in (8.28), one has to g•^ n G. J —1

- ^ + 2II ! tII i <(0.1325)n2

( 1/2)

n

Z2+ 1 ( Et,X. ^ J^) 6 13.n

Iltll 3

j-1

<(0. 1325 ) 14.nIltll 4 +( 6) 13 ,,.11111 3 <(0. 117 )IftII',

(8.34)

Approximation of Characteristic Functions

65

using (8.30), (8.31) in the last step. Also, as in (8.27),

log

Gj ( In )+ j IItII Z— 6 n -I ^ Zµ3(t) j=I

<[(0.1325)+(1/24)]14. lull ° <( 0 . 175 ) 14.nll 1 II . (8.35) Hence, as in (8.30), n

I-I Gj(__

I

l

/

—

exp{

-

2IIzII 2 }( 1+ 66

n—I ^ Zµ3( 1 ))

I n

<exp(

— zIItII 2 ) exp I logGj ( __^ Z )+iII 1 II Z —1

n

J-1 n

— Y logGj ( ^^ Z )+211 1 11 2 j-1

n

+exp{

— ill 1 II 2 } 1logGj( 1)+ziitti Z

n

j=1

3 T"— I/2(,) —6

<(0.1 7 5)141I 1 II 4 exp { — z II'II Z } + [(0. 1 325) 21;,,11(11 8 + 13 nlltll b j exp ( — ilI'l1 2 +( 0 . 117 )11 1 11 2 ).

(8.36)

Q.E.D. Note that [see (7.6)] if X^ j is the with cumulant of X i and if XY =n-Ili-,x,j, then

t"

P I (it: (X„})=i 3 I IPI-3 v

n

n



=n

n

-I (t3

j 1

l t^^=n v

1^I= 3 j

3

3

(

1

3

31X3.j(0

II-3

=n—I1 2 31 µ3,j( 1 )) — 6 µ3(t) (+ j-1 Iv!-3

66 Expansions of Characteristic Functions

the equality X3 j (t)=tb j (t) being a consequence of the fact that X 3f (t), 53J (1) are, respectively, the third cumulant and third moment of the random variable (t,X1 ), which has zero mean [see (6.21)]. THEOREM 8.7. Let G be a probability measure on R k having zero mean, nonsingular covariance matrix V, and a finite third absolute moment. For all t satisfying (8.37)

11111 <2 h / 2I3. "', one has

G"(

2

B1

Iexp1 — (2 -262 )Iltll 2 }, )<

(8 .38 )

where B=B', B 2 = V Proof. First take V= I. By Taylor expansion, if X has distribution G, G( 1 n )=]-(2n)- 111,1 + 6n-3/2EI
(8.39)

where I0I < 1. Hence, noting that I- (2n) -1 II t II 2 > 0 in the given range oft,

G ( ni ^ 2 )1< 1-(2n)- I II t11 2 + sn-1/3."Iltll' <exp(—(2n) -1 11 1 11 2 +In - '/3.,,IlIII'} <ex p

(1 _ -I i -

2 1 /2 ) - --

1, ) --;;n-Jj -;;

(8.40)

which proves (8.38), when V= 1. For the general case, look at the distribution of BX, where X has distribution G. Q.E.D. Before proving a similar result for non-i.i.d. random vectors, we need a simple lemma. LEMMA 8.8. Let X and Y be two independent random variables (in R') having the same distribution. If this common distribution has mean zero and a finite third absolute moment, then EIX-YI 3 <4E1XI 3 .

(8.41)

Approximation of Characteristic Functions

67

Proof. Since IX

—

YI 3=(X—Y)'IX—YI <(X2-2XY+Y2)(IXI+IYI)

= IXI 3 + IYI 3 + XZIYI + YZIXI — 2XIXIY — 2YIYIX ,

(8.42)

and E(XIXIY)=(E(XIXI))EY=0=E(YIYIX), one has E IX — Yj 3 ' 2 E IX1 3 + 2(E X 2 )E IYI = 2 E IXI 3 + 2(E X 2 )E IXI < 4EIX1 3 .

(8.43)

Q.E.D. THEOREM 8.9. Let X 1 ,...,X, be n independent random vectors in R'` having distributions G,. . . ,G, respectively. Suppose that each X^ has zero mean and a finite third absolute moment. Also, assume that V = n - '2j Coy (Xi ) is nonsingular. Then for every 8 E (0, Z ), n

1G;( .i —

Bt2

) <exp {-31I1112}, (8.44)

for all t satisfying Ilt11 (—&)l,,'.

(8.45)

Here B=B', B 2 = V -' . Proof We may take V=I. In this case, for each j, I < j < n,

z

1 —n-iE(t

Xl \2+3n-3/ZEI(t

n

,X^>I3

C exp { — n -' E ( t, X1 >2 + 3 n - 3/2E I (t, X^>I 3 } , (8.46) since I G^(ut)I 2 is the characteristic function, evaluated at u, of the random variable —, where Y1 is a random vector independent of and having the same distribution as X^; also, by Lemma 8.8,

E[— <',y>] 2 =2E 2 , EI—l 3 <4EI 3 .

(8.47)

68

Expansions of Characteristic Functions

Multiplying both sides of (8.46) over j= 1,...,n, one gets „

2

llG( n112/ exp{

-

IItII 2 +3/s,„Iltll'}

JI

exp{—I1,112+3(i—S)IItII2}

=exp{ — 3s IItII 2 } , (8.48) in the given range (8.45). Q.E.D.

9. ASYMPTOTIC EXPANSIONS OF DERIVATIVES OF CHARACTERISTIC FUNCTIONS We first state and prove some preliminary lemmas. LEMMA 9.1. Let g be a complex-valued function on an open subset 2 of R k having continuous derivatives D"g for 'vj < m, where m is a positive integer. If g has no zero in Sl, then on Stt (D^'g)...

(DP'g)

g where the summation is over all collections of nonnegative integral vectors ($',... /P) satisfying '

f3'+... +13 1 =v,

1 6 j
[I
and the constant c({ $I,...,$3)) depends only on the collection {/3 . ,...,/3i). Proof. The result is obviously true for I vj = 1. Assume that it holds for all v satisfying 1 < m, where m is a positive integer. An immediate computation shows that the result holds for all vectors v + e,, I i < k, where e, is the vector having one in the ith coordinate and zeros elsewhere. Thus the result holds for all v satisfying I < M < m + 1. Q.E.D. LEMMA 9.2 (Cauchy's Estimate). Let f be a complex-valued function on B(a: R)= (z=(z 1 ,...,Z k )EC k : . I lz ; —a ; 1 2 . R 2 )[where a=(a I ,...,a k )EC'` tThis is a local result and is valid whatever the branch of the logarithm chosen.

Expansions of Characteristic Functions

69

and R > 0] given by the power series [absolutely convergent on B(a : R)] J( z ) = I c(v)(z—a)'

[Z—a=(z1—a1....,Zk—ak) .

v

where the summation is over all nonnegative integral vectors v E (Z + )'`. Then (D"f)(a)I
for every r E (0, k -' / 2 R], where M r is defined by

Mr =sup (If(z)I : Iz —a,I =r, l < i < k}. ;

Proof. Since the cube (z:Iz; —a; {
(—Ir
1
Clearly, for all v,

rtc(v)=(27r)-k

(' J[

(8......°k )dO 1 ... dOk . k exp { — i} g r

Noting that the integrand is bounded above by M r , one has I(D"f)(a)I=v!Ic(v)I
Q.E.D. For x,y in R k we define

x
ifxi
(9.1)

We also write x
(9.2)

where the summation X * is over all collections of nonnegatire integers



70

Expansions of Characteristic Functions

(j p : 0< /3 < v) satisfying (here Eli's are integral vectors)

(R:0<$(}

and c((jp :0< /3 < v}) depends only on the collection {jp :0
Proof

Let e J =(I,0,...,0),...,ek =(0,0,...,0,1). Then D`'(exp{f })=exp{f }(D`f)

(1
Thus the assertion is true for all v with Iv I= 1. Suppose that it is true for all v with II < m. Given v with I vI = m, one has, for all i (I
D" +e, (exp{f))=exp{f)Z*c((jp:0
.Ip(D#.f)'°'-I (D p.+ef)^"(Dsf)'a I { p':je •>0) 1

+exp(f )(Df)Y,*c((jp :0<$ 3. Then there exists a constant c 2(s) depending only on s such that J(D"log G)(t)I < c2(s)p5

(9.3)

for all t in R k satisfying

G(t)—lI<2

(9.4)

and for all nonnegative integral vectors v satisfying I vI = S. Proof. Note that loge is defined on an open set containing the•set of is satisfying (9.4). Also, if /3 j ,...,/3^ are nonnegative integral vectors satisfying /3 i +... +/3^=v,

Expansions of Characteristic Functions

71

then (D''G)()I

... I(D1 6 )(t)I <(EIX 'i)... (EIX4°ij) '< PIQ.I

.

..

PI $I

PcIs I+... +I s,U/s =P

J

,

(9.5)

by (6.29) and Lemma 6.2(ii). The proof is now completed using Lemma 9.1. Q.E.D.

For the remainder of this section we shall consider n independent random vectors X 1 , ... , A. with respective distributions G i , ... , G. We assume that Xi has mean zero, covariance matrix VV , and a finite sth absolute moment for some integer s > 3. We write n

PrJ =E II XIII ', Pr=n—'

(r>0),

Pr i—I

n

= with cumulant of X^,

Xtl = n -

X„

(9.6)

[PE(z+)k, 0
V^,

V=n ^ -

l -1

unless- otherwise specified. In case V is nonsingular, let B denote the symmetric positive-definite matrix satisfying B 2 =V - '

(9.7)

and write n 1,=fl'

2 EIIBX;Il r (r>0).

(9.8)

J -1

The constants c's that appear below depend only on their arguments. LEMMA 9.5. For every nonnegative integral vector a satisfying 0 < lal <3r, and for r=0, I,...,s-2, I D°Pr( z: {X„})I
(zEC k ). (9.9)

72 Expansions of Characteristic Functions

Proof By (7.3), D"Pr (z: {X,))= 2 _ I 1

m-1

+ ym)1

(v1+.

X

2" l... v 1

j,...,j_

(v1+... +Pm

—

a)!

''

z

+...+r.-a

(9.10)

where I• denotes summation over all m-tuples of positive integers (j1,. .,j,,,) satisfying

(9.11)

j1= r, r-1

and 1" denotes summation over all m-tuples of nonnegative integral vectors (v 1 ,...,vm ) satisfying (7.5). By Lemma 6.3 and averaging, IXr i

...

I < C 1( v 1) = c1(vl)

. .. C

.

i( Vm)Pj i +2

...

PI..+ 2

PZ /

r


...

= c1(v1) .

P1_ +2

.. CI(vm)Pi(j,+2)+••• +(j+2))/2 Pj,+2

(i^+

PJ

..



p2 „+2)/2

. +j..)/(s - 2)

cl(Vm)P^2+m^\ P2/2

..0

(vm)Pz

-r/(s-2)pJ/(s-2).

(9.12)

We have used Lemma 6.2(iii) and the fact that p ; is the r'th absolute moment of n -1 (G I + • • - + G,;), where Gj is the distribution of Xj . On using (9.12) in (9.10), the desired inequality (9.9) is obtained if one notes that

Ilzll m
(O<m'<m<m", zEC k ).

(9.13)

Q.E.D. LEMMA 9.6. Let g be an absolutely convergent power series in z E Ck on B(0: r)= (IIzII
[zEB(0:r)],

(9.14)

where h is a nondecreasing function on [0, oo). Then for every c > 0, J(D rg)(z)l < v!c H " h Ilz11 - "h(( 1 +k l/2c)IIzII)

(9.15)

Expansions of Characteristic Functions

73

for all nonnegative integral vectors v and for all z satisfying (9.16)

IIZII < r (z EC k — (0)). (1 + k'/ 2c)

Proof. By Cauchy's estimate (Lemma 9.2) I(D'g)(Z)I

< v!(clizll) -I ' I sup {I g(z')I : Ilz'II <(1 +k'" 2c)Ilzll } G v!(cllzll)-I"lh((l+k'/2c)IIZII) (9.17) for all z0 satisfying (9.16) if one takes (in Lemma 9.2) a = z, r = c II z II and notes that [for z satisfying (9.16)]

(Z':Iz

—

z,I=c11z11,l
C{Ilz'II
Q.E.D. LEMMA 9.7. Let s be an integer not smaller than 3 and let (y : v E(Z+)k, M < s) be real numbers. Define

X.,=(max{IX,.I"/''

2)

:3
s-3

1

(9.19)

l

c(s,k)= 7 }. r=i M=r+2 v JJJ

Then for every u ER' — {0) and all z satisfying IIZII <(I lc(s,k)iuI /31/(s-2))-1

(9.20)

one has, in the notation of Section 7, s-3 X

s-3 _

r+2( Z)

D exp 2 ( r+2)! r-1 '

ur

— I 'r r-O

—


J -2)— ^ -1

)exp{911zI1 2 ),

(9.21)

74

Expansions of Characteristic Functions

where c'(s, k) depends only on s and k, and D is the ath derivative with respect to z = (z 1 , . . . , z k ), a being any nonnegative integral vector satisfying jaj<s.

Proof Write [see (7.1)] 3-3

g(u:z)=

urXr+2(Z) s-3

r-1 (r+2)!

► —I

v

r

Irl—r+2 P

f(u:z)=exp(g(u:z)},

(9.22)

s-3

h(u:z)=f(u:z)— Y, u'PP (z:( ))

(uER I , zEC").

r=0

By definition of the polynomials P,,

dU

'" h(u : z)I

=0

for m=0,1,...,s-3,

u^0

du'

2

h(u:z)= d ' 2 f(u:z),

when h and f are regarded as functions of the first argument u only. By Corollary 8.3,

S

h(u:z)l< ( I IS 2)I sup{ I da ' z f(a:z)I:0
(ds -2 /dus -2 )f(u:z)

d

f,

(9.23)

is a linear combina-

j.2

ds-2

f(u:z)(du g(u:z)) ... (dus-2 g(u:z))

,

(9.24)

where jl,...,js-2 are nonnegative integers satisfying s-2 2 mjm =s-2.

(9.25)

M-1

If z satisfies J/(s_2) }

Ilzll <(8c(s, k)^uI

-

1

(9.26)



Expansions of Characteristic Functions

75

then, since [by (9.17)] QI'I-2)/(J-2)

IX.I

(9.27)

one has s-3

d m2 r(r— 1)... (r—m+ 1)( m 8(u:z)I = du r=m M=r+2 s-3

Zt )u'-"'

c t (r m k)a Jr J-2)IUIr-m.IIzIlr+2

r=m

s-3

< E c1(r , m , k)(aJ

t/(s-2)

r-m

IuIIIzII) r-mam/(J-2) IIZIl m+2

< c 2 (s,m,k) /3m/(J-2)IlzIIm+2

(1 < m < s-2), (9.28)

so that d (



J.-2

JJ-2

ii

g(u:z)) ... (

d4us-2

g(u:z))

du

< c3(s,k) aJ (IIzIIJ+ 11z11 3(3-2) ).

(9.29)

Also, if z satisfies (9.26), then 3-3

Ig(U:Z)I< 2

2

1 as/(s-2)IIZIIr+2luv

r= 1 1vI-r+2

s-3

<

IIZI1 2 2

1

11

r=I M=r+2 Y '

s-3

<

IIZI1 2 Y.

2

r- I M=r+2

r (IuIIIZII ps t s -2) )

1- (8c(s,k)) - r v

IIZ112 J-3 (gC(S'kI' r=1

I

II

r+2

IIZI1 2 8' ^f(u:z)I<exp{sllzII 2 }.

(9.30)

76

Expansions of Characteristic Functions

Hence, by (9.23), (9.24), (9.29), and (9.30), Ih(u:z)(
118112 (9.31)

for all u 0 and all z satisfying (9.26). One can now use Lemma 9.6 with r given by the right side of (9.26), c=(3k ! / 2 ) - ', to obtain — lal

)D°h(u : z)l <

c4(s , k)I uj s

a!(31k'I/2)

x /3 [(IIIzll) +(311zII)'

-2

(s-2)

$

jexp(9JIz11 2 )


(9.32)

for all z satisfying (9.20), since the right side of (9.20) is smaller than r/(l +k'/ 2c) =;r. Q.E.D. If V= I, then by Lemmas 6.3, 6.2(iii), one has (since p 2 = k) IXYI' /(I ' I -2) <(c1(v)p11) < ci(v,k)p,'

'/(IVI -2) /(s -2)

(3 < Ivy < s).

(9.33)

Taking these x's and u = n -' / 2 in Lemma 9.7, one has LEMMA 9.8. Let s be an integer not smaller than 3. Define (9.34)

c6 (s,k)=(l lc(s,k)max {cj(v,k):3 G 1v <s}) - ' There exists a constant c7 (s, k) such that for all t in R k satisfying

Il t11 < c6(s , k)n `12ps-

U(s-2),

(9.35)

one has, for every nonnegative integral vector a, 0< a! <s, s-3

DQ exp

— Iltll2 + 2

x +2 (tt) n_ /2

E (r+2)! r=1

—exp — 1I21z

' s-3

)J r=0

n-'/ zP,(it: {X,)) C7(s,k)ps

(

I.

111112

(Iltlls",al +I(tll 3(s-2) +lal )exp (— 4

77

Expansions of Characteristic Functions

Proof. The assertion follows from Lemma 9.7, inequality (9.34), and the following:

(i) D a (exp { - I ^ 2 12 } -h (n 1/2: it)) may be expressed as a linear combination of terms like t 2

(DRh(n -112 :it)) D°-13exp{- II2I })

(ii) also

(0<^13
l

D a—s exp

(— II221 <

c'(a

—

1 +IItIlla —R I)exp { — 2 12

fl,k)(l

Here h is the function defined in (9.22). Q.E.D. We are now ready to prove the main theorem of this section. Before stating it, we define d„ = sup { a >0: < a 2 implies G/

l

(

-s) n

- 1 I
))

(9.36) THEOREM 9.9 There exist constants c 8 (s, k), c 9(s, k) such that, for all t in

R k satisfying

Iltll < d, II 1 11 < c 8 (s,k)n'' q, '/(s-2'

(9.37)

one has, for all nonnegative integral vectors a, 0 ( IaI < s, n s-3

DI

Bt 2

) - exp( - il1r11 2 ) Y, n -r/2Pr(iBt:{X,.}) r—o

n /

—c
s -2)/2

[IItIIs —I a I +IIt11 3( 2)+I « I ]exp{

—

elll11 2 )• (9.38)

Proof. First assume that V= B= I. In the given region logarithm of G^(t/n'/ 2) is defined. Write s-3

h^(t)=logd (n .7 )-

- 2n llthI2+

It

(r+2)!) n-(r+2)/2 r—^

n

h(t)=

h/ (t), f-I s-3 n—r/2Xr+ 2,j( it )/( r+2 )

^( t ) — 2bhthI 2 +

n

_

_ n

logGj ( -_f. j ) - h(t).

i

!

r—1

1

(9.39)

Expansions of Characteristic Functions

78

We want to estimate I

n

D

G

t exp {+^(t)}

j=1

= Da[(exp (h(t)} —1)exp (i(t)} ] _ E c lo(a,$)(D'exp {^(t)})[D°

-,

(exp {h(t)} —1)]. (9.40)

0< /3Ga

By relation (9.30),

< 1121 2 + 11812 =Blitll (9.41) 2 if t satisfies (9.37) [see (9.27)]. Also, using (9.33), ( 1 -3 j(D II )(t)I = D O

=

I n'

r=0

^tv M=r+2v

s-3 C^

n—r/2 C` X' .

0,I$I-2? r=max{G

t ► —^

11=Gr+2,(v—/3)!

v>fi

s —t3 ^

<

G r=max(0,((i1-2}

n r/2 cii(v+k)Ps/(5-2)Iltlr+2—l^^ lvl=r+2 v>(3

s-3

(n-1/2c8(s,k)nl/2ps 1/(s-2))

E

/(s-2) g 2-IPI

r— max (0,1#1-2)


(a>0, 1ER' —(0)).

(9.42)

Let j l ,.•.,jr be nonnegative integers and /3,...,/3, nonnegative integral vectors such that jiai=a,

$>0

(1
Expansions of Characteristic Functions

79

By (9.42) one has [using (9.13)] (Dp (t))J1...(DR'p(t))i'I 1 i,(2-IR,I> ,



(two).

(9.43)

It now follows from Lemma 9.3 and inequality (9.30) that

ID'exp {^'(t))I < c14(s , k)(IItII 2-I,8I + IItII IRI )Iexp ('P(t))l < c3o(s , k)(IItlI 2-IQI + IItII I1I )exp { — i 11(11 2 + s IIlII Z } =cia(s,k)(Iltll 2-IQl +lltll' Q ')exp f —IIItII2) (t

0)

(9.44)

if t satisfies (9.37). Next note that for any nonnegative integral vector /3 satisfying 0 I /3 s,

D,6'(D'h^)(0)=0

[0
Hence, applying Corollary 8.3 to g DOhJ , one gets D'Qh (t)I < ;

ItRI

2 sup {I(Dag)(ut)I:0C 1). IR'I-s-I/3 a '

(9.45)

But if I /3'I = s — I /31, then by Lemma 9.4 [note that (9.4) holds for II t II < d„],

j(D O 8)(ut)I =I (D$+R)(ut)I =I (D$+R • log G^)t

nut )I n-s/2

< n —s12c2(s)PsJ,

so that summing over j = 1, ... , n, in (9.45) one has E 1 1 s-lel.(Pi-s-Ifil n a•J

ID ,8h( 1 )I < ASA III

(9.46)

In particular, taking /3=0 in (9.46) yields i h(t)I < cz(s)(^

I$'I=s Q

9.47) l( -,) (.Pz>^i Iltll° < II g n

80

Expansions of Characteristic Functions

for an appropriate choice of c g (s, k). If a - /3 = 0, then [using both inequalities in (9.47)] ID° Q (exp {h(t)) -1)( = (exp {h(t)) -11 < (h(t)Iexp {lh(t)I) -

$ (I Iz }.
(9.48)

If a>/3, then D° - O (exp {h(t)) -1)= D° - " (exp {h(t)}),

(9.49)

which is a linear combination of terms such as (Da'h(t))''... (DRh(t))''exp { h(t)),

where 2; _ 1 j; /3; = a - /3. By (9.46) [and (9.13)1,

(DF'h( t ))i'... (DRh(t))i'l

c ► 6 s k 11 (I Vc s-2)1

n

(s - 2)(EJ, - 1)

,

(s

P [ n /2 - 2)/2

II tII5-2+2±Jl-la-13I

C17( S,k)

ps(IItlls—ja—Ql+ 1(II( sla—PI-2 )

(9.50)

Hence if a> Q, then

ID° a (exp{h(t)) - 1)I -

c's( s, k) ps(Iltll'-IQ-fI n cs-z)/z

+Iltllt+la-BI+2)exp{ Il8 I.

(9.51)

Using (9.44), (9.48), and (9.51) in (9.40), one obtains

D a (n n G t ex l/z) p i=1

Iltll z + s-3 n / J4+z( !t)

2 E -^ .-1

c19(s k)P, ,

(Iltll

Ia l

2

+Iltl l

(r+2)i

l

3+al+z

Iltll2 4 }• (9.52) )exp (-

}

Expansions of Characteristic Functions

81

Now use Lemma 9.8 to complete the proof when V= B= 1. If V r 1. look at random vectors BX 1 ,...,BX n and observe that n

II J=^

Gi \

z l

is the characteristic function of Z n = n - '/ 2 Y. where Y n = B (X, + • • . + X n ). Also, if the (r+2)th cumulant of the random variable is denoted by X,+z,l(t), then the corresponding cumulant of = is Xz(Bt). Q.E.D. The following theorems are easy consequences of Theorem 9.9. THEOREM 9.10. Let G be a probability measure on R k with zero mean, positive-definite covariance matrix V. and finite sth absolute moment for some integer s not smaller than 3. Then there exist two positive constants c 20(s,k), c 21 (s,k) such that for all tin R k satisfying

IItII < c20(s,k)n1/zns



z'

one has, for all nonnegative integral vectors a, 0 al < s, s-3

D ° Gn( ^^z) —ex p{ - 111 , 11 2 } E n—r/2Pr( iB1: (Xv)) r=0

cz,(s,k)71S s lai ncs—z)/z ^Iltll

+Iltll

3(s—z,+iaj ]ex{ — Ill

4 ^.

Here B is the symmetric positive-definite matrix satisfying (9.7),

-s= f

I1BxII-V(dx),

and y,, is the with cumulant of G. Proof Note that if i satisfies 1 1 11 < n'/z^ls 11(s-zt

then Bt, VBt ) t 2 2n = I1I G^ Bi z )—1 < 2n

^z,

Expansions of Characteristic Functions

82

because of the relations fIBxjp 2G(dx)=k<[fit BxIJ5G (dx)]

THEOREM 9.11. Let X 1 ,.. .,X be n independent random vectors in R k with zero means, covariance matrices V i ,...,V. (at least one of which is nonsingular), and finite sth absolute moments for some integer s not smaller than 3. Then there exist two positive constants c 22 (s, k), c 23 (s, k) such that for all t in R k satisfying (s-2)/s

ni/Z

11111 < cZZ(s k)(, 1J( / s-2))

one has for all a, 0 6 IaI < s, E ( ex j . iB!

Da



X^> )) —

Ilt2I `3



r

2_

))

exp — 2 J ^n -1 Pr(i 81 :{X^ r=0

111112 ,ai acs—z>+iQi------j }exp^ , +I{t{I n (z))z [lltll

c23 s ,k s j

where the notation is as in Theorem 9.9. Proof. As in the proof of Theorem 9.9 (see the concluding observations), it is enough to prove this theorem for B= V= I. In this case, for all t satisfying n'/2

Iltll

11u/(52) •j

one has t E
Gilni/z/-11<

> Z IItII ZE(IX II Z ;

2n

<

2n

IItl12(EllXilis)21s 11,112(n',ns)Z/s<2. 2n 2 n

Q.E.D.

83

A Class of Kernels

Before concluding this section, we state another theorem that may be easily proved along the lines of Theorem 9.9, by taking one more term in the Taylor expansion. THEOREM 9.12. Under the hypothesis of Theorem 9.10, D°

/ t 2 s-2 Gn BI Z - )—exp j (Ii 1 2 n-'"2P,(iBt:{X,))

`

ns(2)/2



I r=o

[Iltll

s-ICI

+Iltll

3(s-2)+ l a l]exp

{ _ 1^ 12 t

(9.53)

where S (n)—*0 as n—*oo. In fact, (9.53) holds even for s)2, al 6 s. Remark. Under the hypothesis of Theorem 9.11, one may also prove that if i/2 ) (

1111
2)/

(9.54)

/(,-2)

where A is the largest eigenvalue of V, then R

s-s

D. fJE(exp{<— X/ ))) -exp{-l)•± ro n - '/ 2P,(it:(x.)) ^

j-1

Aldl/ , k) ^2^( sR` < 2

1<: yt>t:-1=I)/2 ,

n ( ' -2) / 2

+
Vt>(3(,-2)+laly9exp

(

- i
(9.55)

For a=0 this follows by replacing Bt by t in Theorem 9.11. The general case follows by induction on al. Note that the derivative D" is with respect to r. Completely analogous modifications hold in the statements of Theorems 9.10 and 9.12.

10. A CLASS OF KERNELS

For a >0 let UI _ a , al denote the probability measure on R' with density

ul-Q,,)(x)= a 2 =0

for —a<x6a for lxl>a.

(10.1)

84

Expansions of Characteristic Functions

is called the uniform distribution on [ – a, a). One has

The measure U1 _

O(_

Q

, a J(t)

=Za f costxdx= 0

a

a

sp a ' (tERl).

(10.2)

The probability measure

To=

(10.3)

U`i_a/2,a/2l

is called the triangular distribution on [ – a, a]. It is easy to show that its density is

Q (1– ad l )

t,(x)=

for IxI
(10.4)

for lxl>a,

=0 and that

at

sin — I a t 2 (tER'). T a (t)=

(105)

2 One can write

c(m) =(f

/Ri

I

sinx I m dx) (m =2,...). -'

X i

(10.6)

For a >0 and integer m > 2 let G,.,,, denote the probability measure on R' with density g,

m

(x)=ac(m)I sin x ax

I

m

(xER').

(10.7)

It follows from (10.2) that for even integers m> 2 sin at _

--

at

m

^ [- a . a

l(t)

(t E R'),

(

10.8)

so that by the Fourier inversion theorem [Theorem 4.l(iv))

d.,.(t)=21Tac(m)u*

=O

if

f tI > ma.

.. a) (t)

(t ER'), (10.9)

A Class of Kernels 85

Let Z I ,...,Zk be independent random variables each with distribution G 112,, 2 , and let Z = (Z,, ... , Z, F ). Then for each d>0 Prob(IIZII>d)

\.

(10.10)

Thus for any a in (0, 1), there exists a constant d depending only on k such that Prob(IIZII> d)< 1—a.

(10.11)

Note that the characteristic function of Z vanishes outside [ — 1,1 ] k . Now let K j denote the distribution of Z/d; then K,((x:lIxll> 1))< 1—a, if t12[—d,d^ k

K 1 (t)=0

(10.12)

One thus has THEOREM 10.1. Let r be any given positive integer and let a E (0, 1). There exists a probability measure K, on Rk such that

(i) K1((x:IIxII> 1))
one has

f I x"I K (dx) < I

oo.

(iii) K 1 (t)=0 for t [ — d, d ) k where d is a constant depending only on a and r. We also need as kernels certain probability measures having compact support and fast-decreasing characteristic functions. To this end we prove THEOREM 10.2. Let u be a real-valued, nonnegative, nonincreasing function on [1, oo) such that

uj t)

dt< oo.

(10.13))

Then for every 1 >0 there exists a probability measure K on R 1 satisfying (i) Support of K[

—

1,l]

(ii) IK(t)I = 0(exp (— Itlu(ItI)))

(ItI—>oo).

86

Expansions of Characteristic Functions

Proof. Define a nonincreasing sequence of nonnegative numbers (a,: r> 1) by eu (ro)

a,=

1
ro

_

eu(r) r

,

r>ro,

where ro is a positive integer chosen to satisfy

00 O° u(r) I a,-eu(ro)+e 2 1. r-1 r r.- rp+ l Let the probability measure K be defined as the infinite convolution K= Ut _ a „ o ,I« UL _ az,Q=J• .. .

Clearly, support of K is a, r-1

r-1

Also,

si na,t r-1

For every positive integer

1

(tER ).

at

K(t) s> ro ,

'

therefore,

S

IK (t)I < 11 ^7 _ tl

< l a!tl' s - `eu(

)It)) • '

Now choose s- [Itlu(Itl)], the integer part of Itlu(ItI). We assume without loss of generality that IiIu(ItI)—'oo

as Itl—+oo.

[For otherwise one may replace u(t) by u 1 (t)= u(1)+(t+ 1) -1 / 2 and note that (10.13) holds for u if and only if it holds for u 1 , and that (ii) holds for u if it holds for u 1 (since u 1 > u).j Thus for sufficiently large Its, one has



A Class of Kernels

87

ro < s < I tl so that lItIU(ItI)J

IK(t)I <( eu(s)Itl ) < Itlu(Itl)

(ltI11(ItI)I

I

<

)

eu(itl)Iti

Remark. The above result is due to Ingham [1], who has also shown that the above theorem provides the best possible rate of growth for the characteristic function of a probability measure with compact support. To be precise, he has shown that if u is a real-valued, nonnegative, nonincreasing function on [1, cc) such that f r[u(r)/:Jdt-oo, then there does not exist any probability measure with compact support whose characteristic function is 0(exp(—Itlu(I:I))) as Itl— ► oo. Note that this also follows from the following result of Wiener and Paley: If fE L 2 (a, oo) then f

J

°O 00

IiogIf (t)II dr
(10.14)

1+r 2

THEOREM 10.3. Given a nonnegative, nonincreasing function u on [1, co) satisfying (10.13), and a nonnegative integer s, there exists a probability measure K on Rk with support contained in the unit ball S= (x: IIxii < 1) such that

ID°K(t)I < c(s,k,u)exp

— k t^lu(It,l) -i

( 10.15 )

for all nonnegative integral vectors a=.(a,,...,a k ) satisfying 0
Define M, = M. Then

M,((x:xER',(xi
M(t)=(M(t)) 3+ '

(tER'),

*See Paley, E. A. C. and Wiener, N. [1], Theorem XII, pp. 16, 17.

88

Expansions of Characteristic Functions

so that for 0 < r < s,

dt'Mt(t)I <(s+ 1)'EIXI'IM(t)I

t-.

<(s+1)'k"'' 2 (s+1) 'IM(t)I

< c 2 (s,k,u)exp ( — Ittu(I1I)), where X denotes a random variable with distribution M. Now define K on R k as the product measure M, X M, x • • • x M t . Then

K({x:xER k ,Iixll<1}) > K({x:x=(x l ,...,xk ),Ix; )
and jD aK(t)I — ^ dt ^ , Mt( 1 1)

1

...

f dtk Mt(tk)I k


It;Iu(It;I)

(t=(tt,...,tk)ERk),

for 0 < Iai < s. Q.E.D. COROLLARY 10.4. For any positive integer s, there exists a probability measure K on Rk satisfying (i) K((x: Ilxll < 1))= 1, (ii)for all nonnegative integral vectors a, 0< Ial <,

(D°K(t),
(tER k ).

Proof In Theorem 10.3 take u(1)=t - " 2 on [1,00) and note > IIt11 t / 2 , for all t-(t I ,...,tk )ER k . Q.E.D. NOTES Sections 4 and 5. The material on the Fourier transform and the Fourier—Stieltjes transform reviewed here is fairly standard and may be found in Cramer [3, 4], Chung [11, Feller 131, Katznelson [11, and Stein and Weiss [11.

A Class of Kernels

89

Section 6. The best reference for the inequalities of this section is Hardy, Littlewood, and

Polya [l). Sections 7-9. The idea of expanding the distribution function F„ of the normalized sum of n independent random variables as E,n - 2P,(-d': {X,)) appears for the first time in Chebyshev (1]; later it was investigated independently by Edgeworth [I]. However the asymptotic expansion of the characteristic function of F. was obtained by Cramer (1, 3) (Chapter VII), who used it to give the first rigorous derivation of the asymptotic expansion of F. under the so-called Cramer's condition (20.1) (see Chapter IV). Theorems 8.4-8.6 are refinements (and extensions to R*) of results of Cramer (3) (Chapter VII); analogs of Theorems 8.7 and 8 .9 for k - I were obtained by Laipounov [2]. There are many such refinements available in the literature, for example, Esseen [I], Gnedenko and Kolmogorov [1], and Petrov [1) in one dimension; Rao [I), Bikjalis (3, 61, and Bhattacharya (3) in multidimension. Theorems 9.9 -9.12 are generalizations of analogous results of Bikjalis [6]. "

Section 10. Kernels such as K i (Theorem 10.1) were used in one dimension by Berry [I] and

Esseen [ I ). Theorem 10.2 is due to Ingham [ I ].

CHAPTER 3

Bounds for Errors of Normal Approximation

Our goal in the present chapter is to estimate j f f dQ„ — J f d for a large class of functions f on R k here Q. is the distribution of the normalized sum of n independent random vectors and cb is the standard normal distribution on R'. For bounded f, the error bound is computed in terms of the average modulus of oscillation of f with respect to 4, or the supremum of this average over all translates of f. This is entirely appropriate in view of the characterization of (D-uniformity classes proved in Chapter 1; also, in most cases of practical importance, these moduli can be estimated efficiently. The main tools used for obtaining these error bounds are expansions of the characteristic function Q. and its derivatives as derived in Chapter 2, together with some smoothing inequalities proved in Section 11. In Section 12 the classical Berry—Esseen bound is obtained with an estimation of the constant involved. Section 13 is devoted to estimations of IJfdQ„ — Jf d 4?j for bounded f under the simplifying assumption that fourth moments are finite; the proofs here are not only simpler than those of the general results of Sections 15-17, but also yield numerical values for constants involved in the bounds. In Section 14 we obtain truncation estimates that enable us to derive the main results of Section 15 on rates of convergence for unbounded fs under the assumption of finiteness of third moments. After a short section showing how to deal with different normalizations of the sum of n independent random vectors, in Section 17 we consider a number of important applications of the theorems in Section 15. A final section deals with rates of convergence under the sole assumption of finiteness of second moments. To facilitate comprehension we briefly sketch the main ideas underlying the rather long route leading to the main results. For the sake of simplicity ;

90

Smoothing Inequalities

91

assume that (X.: n> 1) is a sequence of independent and identically distributed (i.i.d.) random vectors with common distribution Q 1 . Suppose that Q 1 has mean zero, covariance I (identity matrix), and a finite third absolute moment p3 . Let Q. denote the distribution of n - 2(X, + • • • + X„). If Q, has an integrable characteristic function (c.f.) Q 1 , then the c.f. Q. of Q„ is integrable for all n. One can then use Fourier inversion to estimate the density h„ of the signed measure Q„ - (D as "

Ilh.II m sup Ih.(y)I <( 21T) -k IIQR $II1. -

(I)

yER k

The fairly precise estimates of Q„ - ' of Chapter 2 (e.g., Theorem 8.4) yield IIh„II= O(n-' "2). Since such a uniform estimate of h„ cannot be integrated over the unbounded domain R k , one may estimate the variation norm II Q. - 41 11 = II h,, II 1 by estimating the integral of IhI over a sphere S of radius 0(log' / 2 n) as Ilh„II, 0 •vol(S), and the integral over the complement of S by the classical Berry-Esseen theorem and a Chebyshev-type inequality. To avoid the loss in precision arising from the factor vol(S)=0(logk / 2 n), assume that Q, has a finite fourth moment and apply Theorem 8.5, adding one term [n ' 12P,(- b: {x,))] to b and later subtracting the contribution from this term, which is of the order n" 2 . In the general case (i.e., when Q, may not be integrable) we smoothen Q, by convolving it with a smooth kernel KK (a probability measure with an integrable c.f., which for small e assigns most of its mass near zero) and apply the above argument to (Q„ - 4)* Ke , with a proper choice of a (depending on n). The smoothing inequalities of Section 11 enable one to estimate the perturbation due to this convolution by K, and one arrives at Theorems 13.2 and 13.3, which express bounds for jf d (Q„ - ( P)I in terms of the average moduli of oscillation wf or wJ and the range wf (R k ) of f. To estimate Iffd(Q„ -4D)l for unbounded f and at the same time relax the assumption of finiteness of fourth moments (in case of bounded f) we compare the measures IIxll'Q„(dx) and Ilxll' (D(dx) for nonnegative integers r in the same manner in which Q„ and 1 are compared above. Because of the fact that for odd integers r Il xll'Q„(dx) does not have Fourier-Stieltjes transforms as well behaved as those for even r, we replace r by ro, where ro = r + I if r is odd and r0 = r if r is even. As above, we smoothen P,o - llxIi' °(Q„-4)(dx) as p o * KK and apply Lemma 11.6 to the density g of v, o * K,, thus obtaining -

Ilv, o *Kell= IIgll1

c(k)

max 1$I—O,k+i IIo

111,

(2)

where g is the Fourier transform of g. The Fourier-Stieltjes transform of v,o is estimated by Theorems 9.9-9.12 and a sharp estimate of II v a * K, II is

92

Normal Approximation

obtained provided f (Ixll'o +"` +'Q,(dx) is finite (which ensures the existence of D flg for fl = k + 1). To relax this last hypothesis, which is rather restrictive, one resorts to a truncation of the random vectors (X.: n> 1) and applies the above procedure to these truncated vectors. The various lemmas in Section 14 allow one to take care of the perturbation due to truncation. As in the case r0 =0, for final accounting (i.e., to estimate the effect of smoothing by Kc ) one uses the smoothing inequalities of Section 11. The main theorems of Section 15 are obtained in this manner. A further truncation enables one to obtain corresponding analogs when only the finiteness of absolute moments of order 2+8 is assumed for some E, 0 < 8 < 1, thus yielding generalizations and refinements of the classical one-dimensional theorems of Liapounov and Lindeberg.

11. SMOOTHING INEQUALITIES

Lemmas 11.1 and 11.4 show how the difference µ— v between a finite measure and a finite signed measure v is perturbed by convolution with a probability measure Kc that concentrates (for small e) most of its mass near zero. Let f be a real-valued, Borel-measurable function on R k . Recall that in Chapter I we defined the following:

(ACRk),

wf (A)=sup(If(x)—f(y)I:x,yEA} wf (x:e)= wf (B(x:e))

(11.1)

(xER k , a>0).

Also define

MJ (x:e)=sup{f(y):yEB(x:e)), mj (x:e) =inf{ f(y): yEB(x:e))

(11.2)

(xERk, e>0).

Note that

wf (x:e)=M1 (x:e)—mf (x:e)



(xER k , a>0).

(11.3)

The functions M1 (- : e), mf (• : e) are lower and upper semicontinuous, respectively, for every real-valued function f that is bounded on each compact subset of R k . Also, wf(• : e) is lower semicontinuous. These follow from {x: M1 (x:e) >c }=

U {B(x:e): f(x)>c)

(cER5,

z

ml (x:e)=—M_ f (x:e)

(xER k , r >0).

(11.4)

In particular, it follows that Mf (• : r), mf (• : e), wf (• : e) are Borel-measurable

Smoothing Inequalities

93

for every real-valued function f on R k that is bounded on compacts. Recall that the translate fy off by y(E R'") is defined by fy (x) =f(x+y)

(xER k

).

(11.5)

LEMMA 11.1 Let s be a finite measure and v a finite signed measure on R k . Let a be a positive number and K E a probability measure on R k satisfying Ke (B(0:e))= 1.

(11.6)

Then for every real-valued, Bore/-measurable function f on R k that is bounded on compacts, f fd(µ

—v)I
(11.7)

where Y(f :E)= max{ f Mf (•:e)d(tt— v) *KE

, — f mf(•:e)d(µ — v) *Ke }, ,

r(f:2e)=max(f (Mf (•:2e)—f)dv + f (f—tn (•:2e))dv + ), (11.8) provided that M f (- : 2e)I and Im f (- : 2e)I are integrable with respect to and Proof. One has

f Mf(•:e)d(µ —v)*Ke = f [ f MJ(Y+x:e)(IL

Y(f:e)>

=f

B(O:) e

—

v)(dy)]Ke(dx)

[ f Mf(Y +x:e)It(dy) f f(Y)v(dy) — f (Mf(y+x:e)— f(Y))v(dy)]K.(dx) —

> f[ f f(Y) u(dY) f f(Y)v(dY) — f (MJ(y+x:e) — f(Y))v (dy) I KE(dx) > f fd(,u f (M1( . : 2 e) — f)dv . —

B(0:e)

+

—

v) —

+

(11.9)

Normal Approximation

94

Similarly — Y(f:e)<

f

=

mf(•:e)d(µ'v)`KK mf(Y+x: e)IL(cô')

B(O:i){

—f

f(Y)v(d')

+ f (f(Y)—m (y+x:e))v(dy)]K.(dx) l

f fd(µ—v)+ f (f—m (•:2e))dv f

+ .

(11.10)

From (11.9), (11.10), one gets Y(f:e)—T(f:2e)G TY(f:e)

—f

(f—mm(•:2e))dv+

< f fd(µ—v)
+


Q.E.D. COROLLARY 11.2 Under the hypothesis of Lemma 11.1, one has

f fd(µ—v) I
(11.11)

+ .

If, in addition, f is bounded and µ(R k

)= v(R' ),

(11.12)

then

f fd(µ v)L< —

,

1(R k

)II(µ

—

v) *KII+

fw (.:2e)dv f

+ . (11.13)

Proof The inequality (11.11) follows from (11.7) and the relation (11.3). To prove (11.13) note that, in view of (11.12),

f (f

— c)d(µ — v)=

7(f — c:e)= max{

f fd(µ

— v) ,w i- c( - : 2 e)=wf(•: 2 e),

f M _,(.:e)d(µ—v)•K , — f m _ j

K

<sup {If(x)—e{: xER' )II(µ— v)•KcII

f c

(•:e)d(µ—v)*KK

Smoothing Inequalities

95

for all c in R 1 . Letting c=2(sup{f(x): xER k }+inf{f(x): xER k }),

(11.14)

and observing that for this value of c wf _ C (R k )=wf (R k )=sup{f(x)—c: xER k } —inf( f(x)—c: xER k )=2sup{lf(x)—cl: xER k },

one obtains (11.13). Q.E.D. COROLLARY 11.13 For every Bore! subset A of R k one has µ(A) — v(A)

fl( t— v)*KII + v+(A2,\A)

ifs, v, K e are as in Lemma 11.1 and if (11.12) holds. Proof. The inequality (11.15) follows from (11.9) with f= IA if one notes that

f M1_112(': 2 e)d(µ — v)'KK2.

(11.16)

Then for each real-valued, Borel-measurable, bounded function f on R k , one has

I

f fd(tt—v)I<(2a-1)-^y•(f:e)+aT'(f:2E)+(1—a)z•(f:e)^ (11.17)

where

y`(f:e)=sup{y(fy:e):yERk }, T * (f:2e)=sup{7 . (fy :2e ):yER k }.

Proof Let 8=sup{ f fy d(µ—v)I: yER k }

(11.18)

% Normal Approximation

Assume first that 8—sup{ rfy d(µ—v):yER k }.

(11.19)

°

Then, given any positive number ,1, there exists y in R" such that f f,.d(p—v)>6-71.

(11.20)

In this case Y`(J: E) > f Mfy,(• : E) d (,u — P)"K,

=

f

fMfy(y+x:E)(µ—v)(dy)IK.(dx)

B (o: e)

+

f k

J

[

Mf.(y+x:E)(µ — v)(dY)}KK(dx) f

R \B(O:e)

> Jf J,'(y) µ( dy) — f Jy .(y) v ( dy) 8(0: E)

— f (M,(y+x:E)

—

fy o(y))v(dy)I K.(dx)

+1 [ f fy4y+x)µ(dy) R k\$(0:e)

-

f fy.(y+X)Y(dy)

— f (Mjy.(y+x:e)—fy4y+X))v(dy)}K (dx)

I &—— f (Mi,.(y+x:E) — Jyo(y))v + (d1')]K.(dX)

> f

B(O:e)L

>+f f

-8—f(Mfr.(y+x:E)—Jy.(y+X))v+(dy)K^(dX) k

\B(0:e)

8(0: e)

+

[s—n

—

JC Rk\B(0:e)

f (MIy.(y: 2 E) — Jyo(y))v (dY)]K.(dx)

—

S

—

T*(f:E)]KK(dX)

[S—T rt—•(f:2E)]Ke (dx)+[—S—T*(f:E)](1—a) >fB(0:e) _ {8—rt—r*(f:2E)]a— [8+r*(f:E)](1—a) =(2a-1)6—ar*(f:2E)—(1—a)r•(f:e)—art.

(11.21)

Smoothing Inequalities

97

Since rl may be chosen arbitrarily close to zero, Y*(f:e)>(2a — 1)8—aT*(f:2e)—(I — a)T * (f : r),

from which (11.17) follows. If instead of (11.19) one has S= —inf( f f,,d(µ—v):yERk },

°

then, given any ,j>0, find y such that — f f,od(µ—v)>S—q.

Now look at — f,o (instead of f,,o) and note that

M_ 1 (-:e)=—mj (•:e),

f

(fl,—mfy ( •:e )) dv+'

=

f (M-J( . :e)

—

(

(11.22)

— f,))dv + (yERk)•

Proceeding exactly as in (11.21), one obtains Y'(f:e)^ — f m10(.:e)d(IL—v)*K >(2a— 1)6 —aT*(f:2e)—(I —a)T*(f:e)—arl.

Q.E.D. We define the average modulus of oscillation w^(e : µ) of f with respect to a finite measure µ by w^(e: s) = f wj (x:e)u(dx)

(e>0).

(11.23)

Here f is a real-valued, Borel-measurable function on R"`. We also define wf (e : µ) as the supremum of the above average over all translates of f, that

is, wf (e:µ)=sup{wfy (e:IA): yER k )

(e>0).

(11.24)

COROLLARY 11.5 Under the hypothesis of Lemma 11.4, one has

f fd ( ,u -v )) <(2a-1) - '[y*(f:e)+awf (2e :v + )+(1 —a)wf (e:v + )] <(2a-1) '[Y*(f:e)+4 (2e:v + )]. -

(11.25)

98 Normal Approximation

If, in addition, µ(R k)= v(R k ), then ' f fd(µ - v)I <(2a- l)_ [Iwj(R ' )II(µ-v)*K,Il +awf (2e:v + )

+(1-a)w/ (e:v + )] <(2a-1) - '[

,1(Rk)II(µ-v)*KII+4 (2e: v)]. (11.26)

Proof. This corollary follows from Lemma 11.4 exactly as Corollary 11.2 follows from Lemma 11.1. Q.E.D.

The final result of this section relates the L 1 -norm of an integrable function g to the L 1 -norms of certain derivatives of its Fourier transform g. LEMMA 11.6 Let g be a real-valued function in L I (Rk ) satisfying f xll k+l jg(x)^dx
Then there exists a positive constant c(k) depending only on k (and not on g) such that f I Dfig(t)I dt. II gill < c(k) IIll max —U,k+l

(11.28)

Proof We assume that Dflg is integrable with respect to Lebesgue measure for 0< 1/3 0),

and the 2k quadrants E., one for each vector a =(a 1 , ... , ak ) (in R k ) having zeros or ones as coordinates, by

Ea = (x =(x 1 , ... , xk ): x1 >0 if thejth coordinate of a is zero, xj <0 if the jth coordinate of a is one, I <j< k ). Since the Fourier transform of the function k k+l

-1 , x1 g(x) x-1+( 2 ()°

j_ I

is of the form 7, I$1-0,k+I

c(a,$,k)D' s8,

(xER k )

Berry-Esseen Theorem

99

where c (a, /3, k) depends only on its arguments (and not on g), one has g(x)dx ]

Ilgh 1= I If g(x)dx—f a

_

(t+'xlk+')-'(1+lxlk+l) g(x)dx

(f; a

An E,

J

(Rk\A)nE,

An E,

(Rk\A)nE,

k k+1 X

l+( E (—l) a'xj g(x)dx

I.

j-1 2 ^ lk+l + IX ) ( )

1(1r =

( ✓ An E, - ✓ (R k \A)nj (

k

I

c(a,/3,k)DPg(t))dt dx

X f exp(—i)( 1,01-0,k+l

<(f (l+IxI k+l ) - 'dx)c'(k) max f IDfiI(t)Idt. 101-0.k+1

• M 12. BERRY-ESSEEN THEOREM Let P be a finite measure and Q a finite signed measure on R 1 with distribution functions F and G, respectively, F(x)=P((—oo,x]),

G(x)=Q((—oo,x])

(xER'). (12.1)

Let Ks be a probability measure on R' such that

a=KE((—ESE))>i

(12.2)

for a given e > 0. It follows easily from Lemma 11.4 that sup IF(x)—G(x)I <(2a— 1) - 'sup{I(P—Q)+KK ((—oo,x])I: xER'} xER'

+sup{ IG +(x) — G + (y)I : Ix —yj < 2e } ],

Normal Approximation

100

where G + is the distribution function of Q +. However with a more restricted choice of Q and the kernel K f this inequality can be sharpened. LEMMA 12.1 Let P be a finite measure and Q a finite signed measure on R' with distribution functions F and G, respectively. If Q has a density bounded above in magnitude by m, and if K is a probability measure on R' that is symmetric and satisfies (12.2), then f

sup IF(x)—G(x)j<(2a-1) -1 x€R

sup !(P—Q)•K1 ((—oo,x])I+ame. L ERI

J

S= sup IF(x)—G(x)I= —inf(F(x)—G(x)).

(12.3)

,

Proof First assume that xER'

xERI

Given q > 0, there exists x o such that F(xo)— G(x o )< —S+rl.

Then (P — Q)'KK(( — oo , xo — e]) = ([F(xo — e — y) — G(xo — e — y)]K: (dy)

— ✓ B(o:f)[ F(x

o — e—y)— G (xo—r—y)]K1(dy)

+ f[F(xo — e — y) — G(xo — r — y)]K1(dy) R^\B(O:f)


:f) B(o

[F(xo)]K,(dy)+S(1—a)

f

[F(x o )—G(x 0)+m(e+y)]K1 (dy)+S(1—a)

f

[ —S+7i+m(e+y)]KK (dy)+S(1—a)

B(0:f)

B(o:f)

=(—S+71+me)a+m f B(0:f)

—S(2a— 1)+mae+rla.

yK1(dy)+8(1—a)

101

Berry-Esseen Theorem

Hence S(2a-1)< sup I(P—Q)*KK ((—oo,x])I+mat,

(12.4)

xER'

which proves the lemma if (12.3) holds. If, on the other hand, S= sup (F(x)—G(x)), xER'

then given rl > 0, there exists x o such that F(x o )—G(x o )>S—r^.

Then (P— Q)*Ke ((— oo,x o +e])

B(o:e)

[F(xo+e

—

y)

—

G(xo+e

—

Y)]KE(dY)

+ f[F(x o +e—y)—G(x o +e—y)]KE (dy) R \B(O:e)

>f[F(xo)—G(xo)—(e—Y)m]KK(dy)—S(I —a) B(0:)

(S

—

n)IX

—

am€

—

S(1

—

a)=S(2c

—

1)— amt—a,

so that (2a— l)8 sup (P—Q)*K ((—oo,x])+ame. E

xER

1

Q.E.D. LEMMA 12.2 Let P be a finite measure and Q a finite signed measure on R' with distribution functions F and G, respectively. Assume that

f IxIP(dx)
JIx1IQI(dx)
P(R')=Q(R').

If Q has a density bounded above in magnitude by m, and if K E is a symmetric probability measure on R' satisfying amKe ((—e,€))>,

f JKE(t)Idt
Normal Approximation

102

for some e > 0, then

sup IF(x) — G (x)I <(2a— l) ' -

xER'

x [(2 ,ff) - 'f III -l l(p(t)-Q(t))K^(t)ldt+ame]. (12.5) Proof By Fourier inversion, the density of the signed measure (P - Q)

• Kc is (2ir) - ► f exp(-itx)(P (t)-Q(t))K,(t)dt

(x ER t ).

(12.6)

It is simple to check that the function (12.6) is the derivative of the function (27r) ' f exp{-itx}(-ii) '(P(1)-Q(t))K1 (r)dt -

-

(xER5. (12.7)

Note that Ill - ► Ip(t)_Q(1)1=

lit -'(exp{itx)-1)(P-Q)(dx)I

< f IxIiP- QI(dx) < oo.

By the Riemann-Lebesgue lemma [Theorem 4.l(iii)J, the function (12.7) goes to zero as IxI->oo. Thus

1

(P-Q)`K ((-oo,x]) =(27r) - ' f exp{-itx}(-it) - '(P(t)-Q(t))K,(t)dt,

the constant of integration being zero. The inequality (12.5) now follows from Lemma 12.1. Q.E.D. Remark. 1f, in addition to the assumptions on P and Q in Lemma 12.2, one also assumes that they have integrable Fourier-Stieltjes transforms and that flu

-

'l (t)—Q(t)ldt<eo,

(12.8)

then the above argument yields sup IF(x)- G(x)l <(21r) ' f ltl -

-

IIP(t)-Q(t)ldt.

(12.9)

xERl

We shall use Lemma 12.2 to prove the Berry-Esseen theorem below. The kernel K, is the distribution of eZ/3.25, where Z is a random variable

Berry-Esseen Theorem

103

whose distribution has the density g,, 22 given by x-

x sin 21T '[()

x 2

( x ER').

(12.10)

Recall [see (10.9)] that t K(t)=8i/2.z(3.25)=1- 3.25 e

if j t j <3 E 5,

if ItI>5 .2 .

=0

(12.11)

C

By a careful but straightforward numerical integration one also obtains a- K1

(( —

E,E))=

J

(lxIc3.25)

9,,,22(x)dx> 0 . 79 .

( 12.12)

The following lemma will be useful in estimating the constant in the Berry-Esseen theorem. LEMMA 12.3 Let P be a probability measure on R' with zero mean and variance one. Let F denote the distribution function of P and 4 that of the standard normal distribution on R. Then sup F(x) - 4(x)I0.5416. xER

I

Proof. We first prove the so-called one-sided Chebyshev inequality

F(-x)<(l+x 2 ) - ',

1-F(x)<(l+x 2 ) - '

(x>0). (12.13)

Fix x > 0. For every b > 0, one has 1 +b 2 =

f( - b) 2P(dy) > f

(

(y- b)2P(dy)>(x+b)2F(-x), m,

-

xl

so that g(b)=(l +b 2 )(x+b) -2 > F(-x). The minimum of g in [0, oo) occurs at b = x -', and g(x- 1)=(l+xZ)-1,

104

Normal Approximation

which gives the first inequality in (12.13) [note that for x=0, (12.13) is trivial]. The second is obtained similarly (or, by looking at P). This gives F(-x)-(D(-x) 6(1 +x 2 ) -' -(D(-x)mh(x),

say, for x > 0. The supremum of h in [0, oo) is attained at a point x 0 satisfying or

h'(xo)=0,

xo/ 2x 0 (1+xo) -2 = (27r) - / 2e - 2

A numerical computation yields x 0 =0.2135 and h(x 0)=0.5416, thus proving (12.14)

IF(x)-(D(x)i <0.5416

for all x (0 [note that 11(x)- F (x) < .5 for all x (01. The inequality (12.14) for x > 0 follows similarly (or, by looking at P). Q.E.D. THEOREM 12.4 (Berry-Esseen Theorem) Let X 1 ,... , X„ be n independent random variables each with zero mean and a finite absolute third moment. If n

P2-n -1 2 EX^>0, i=1

then sup IF„(x)-c(x)I<(2.75)/ 3 ,,,,

(12.15)

xER'

is the where Fn is the distribution function of (np 2) - " 2(X 1 + • • . + Xn), the Liapounov ratio standard normal distribution function, and l l3.„°( /2 n -1/Z, P3=n- 2 ' P2 p3). .i -

EIXJ I 3 .

If, in addition to the above hypothesis, X 1 ,.. . , X„ are identically distributed, then sup jF„(x)-4?(x)I <(1.6)l 3 .

(12.16)

xER' Proof To prove (12.15) first assume that P 2 =1. For convenience write

( 12.17 ) In view of Lemma 12.3, sup lF„(x)-(t'(x)+<0.5416, xERI

(12.18)

Berry-Esseen Theorem

105

so that we may assume that 0.5416

1n

2.75

<0.l96.

(12.19)

For, if (12.19) is violated, (12.15) reduces to (12.18). Let P„ denote the distribution of n 1 / 2 (X 1 • • • + X„). Take E=;(3.25)1,,,

(12.20)

and let KE be the distribution specified before the statement of Lemma 12.3.

It follows from Lemma 12.2 and inequality (12.12) that

sup IF,,(x)-4?(x)I <(0.58) - ' (2i) ' xER'

( f

(i (t) — a -12/ 2 )

(1:1 (3/2);-i)

t

x (1—;1„ItI)dt+(277) - '/ 2 (3.25)(0.79);1

(12.21)

Write Pt — nO 1= f{III c(3/2)t ^ ') (

e

-

`'/Z

)

t

—

(1-3lnItIdz 1 +1 2 +1 3 +1 4 +1 5 , (12.22)

where i=

f

^i (t)

—



I

13=

J t 1 / 3
? tt)di

,

2t

f

((l/2)1,1
14 = /

dt

t

( I' i < t; "^ }

12

e -1'/2 )

____ (1-31,,ItI)dt,

t

(t)

l

15 =f

( 1 — il,^Itl)dt. -, = /2

e dt.

(12.23)

Normal Approximation

106

Applying Theorem 8.4 with d= 1 and using (12.19), one gets I 1 <(0.36)1„ f t 2 exp( —0.3779t 2 ) dt <(1.3118)/,,. R'

(12.24)

By Theorem 8.9 (with 8 = 1), f 12 < 1,/f

e_t2/3dt < l, 3 f

=3l„/ 3 exp{ — 3/,^

2/3 }

} ^t^e-`2/3dt

/ =3l„(l^ '/ 3 exp( —;l-

2 3 ))

<(l.9320)1.

(12.25)

Again using Theorem 8.8, this time with 8 = Z, we get J3=

P-(t) (l — 2, 11jtD)di

<321„ f (I11>(1/2)t, '}

It)e-12f6dt e-'2/6d1<3(21.)2 f {IiI>(t/2)t, ')

— !^ 2 =161. exp 24 <(1.06)1,,.

(12.26)

Noting now that Theorem 8.9 was derived from the inequality [see (8.48)] Pn(t)I<exp(— 4-

which holds for ItI <21; 1 , one has z exp^— 2 +

14' < 3 lnf =3!„

<

f exp(-3!„t2(21n '—fit()}dt {i '
„ 3

!h

f

{r

<3info

exp — (l

2 ^

'
1—

I

fit+) dt

'}

exp { —( 3 )u } du / l J

= 212 <(0.3920)/,,.

(12.27)

Berry-Esseen Theorem 107

Finally, 15 =

e_12/2

t

< li/3f

Idt

Itle-12dt

= 21,,'/ 3 exp { - 2

2/3

} (12.28)

<(0.8096)1„. The estimates (12.24)-(12.28) are now used in (12.22) to give

(12.29)

I <(5.5054)1,,. Using this in (12.21) one obtains

sup IF„(x)-4(x)I<(0.58) - '[(27r) - '(5.5054)!„+(2ir) - '/ 2 (3.25)(0.79)31„] xER'

<(2.676)1,,.

(12.30)

This proves (12.15) under the assumption p 2 = 1. In the general case, look at random variables Y^=X^/p2/ 2 , 1 <j < n. We now prove (12.16). Again assume that EX= and < 0.54616

In

=0.3385.

(12.31)

As before, we proceed to estimate I and write

IGI^+Iz+I3,

(12.32)

where (Pn(t)—a—rZ/2

1 = j {ltI2'/ 2! -1 }

)

dt,

t

12 f -^ 1/2

I P^I t) I (1-i1^I'I),

I3= f

Ie

— r2/2 {2'/ 21. ' <1r1 <(3/2)1^ }

1

I(I- 2,1ItI)dt.

(12.33)

108

Normal Approximation

Writing

a=E(expS n

1n 11

b=exp{ —2 ? },

X2 I) , 111

(12.34)

one has

I Pn(t)—e-`2/2I=)a"—b"j=ia—bj•lan-I+an-2b+...

+b"-1).

(12.35)

But, for all t, writing p 3 = E IX 1 1 3 yields

la— bi
<exp{ —

(0.26 42)t 2

n

I.

(12.37)

c = 0.2642,

(12.38)

Hence, writing one has, using (12.36), (12.37) in (12.35), If'n (t)—e -= ^ Z I<( 6nI3/Z

+

)exp(— 8n2

n

n— —r

ct 2 -2n t Z } (12.39)

r-0 J

Now n-1

n—l—r C12_ r t 2 1 r—O

p n

2n = exp { ( 2n — c)1 2 }

exp { — (i — c) l2 n }

ll r=1

f1e-(=-`)`'"dx


` (2n —c)t2111I 0

r =nexpj

` (2n

—c)t 2 ^

1— exp{_ 0_0 12 } Z . (12.40) ! ( 2 c)t

Berry-Esseen Theorem

109

The estimate (12.40) is used in (12.39) to yield — r=/2

n(t)—e

(

Ill

p3

1

1—



I

III

( 6n 1 / 2 + 8n/

2 n )t 2 } x (exp{ —(c— I

—exp{ — " 2n 1 t2}).



(12.41)

Recalling from (12.31) that p 3 n - '/ 2 < 0.3385, so that n) 9, one obtains ligL—c)-11 /RI(6+ 1 exp { — t c

_(2—c)-11„

(2 6 1/2 1(2c— n) -1/2

2n) t2 J (

exp[—n

2n l t21 ) dt

" n 1 ) -1/2 1

1 _ I -1 + 24{( c 2n) 2n —1n 1 ) }

(2ir) 1/z <(1—c) -I 1,, 6 {(2c-9) -1/2

-1}+ 4{(c— 8) 1-2)1

<(1.465)1,,.

(12.42)

Next, proceeding as in the estimation of 1 4 above, one has IZ<2-1/2(1— 23/2)In exp{-3l^ f(2'/2/"-'<,,I,(3/2)/"- ') 3

I ^ilh 1

—Itj)}dt

<(0.0404)1,,.

(12.43)

Finally, 13<(2-1/21„)2(1— 23/2 ) 3 J



^tle-`'/2dt

{2'/z! '
<(0.00002)1,,.

(12.44)

Therefore I <(1.506)1,,,.

(12.45)

and, using (12.45) in (12.21), sup IF,,(x)-41(x)I <(0.58) -1 {(2zr) -1 (1.506)+0.6829 1„ xER'

<(1.6)/,,.

Q.E.D.

(12.46)

)

110

Normal Approximation

Some of the computations above may be sharpened to yield somewhat better bounds in (12.15), (12.16). In Chapter 5 we see that in the i.i.d. case, under the hypothesis of the Berry-Esseen theorem, the finite limit sup IF.(x)-fi(x)l

d(P,P), n-rao lim

(12.47)

xER'

exists, where P is the distribution of X,, and that 3

(12.48) P P3 6V 21r where a 2 = EX2, and the supremum is over all P having mean zero, positive sup °—d(P,(D)= 10 +3 =0.409,

variance and finite third absolute moment. Thus (V +3)/(6V ) is the asympototically best constant for the Berry-Esseen theorem. It is also known that in the general non-i.i.d. case one has (see Notes at the end of this chapter) (12.49) sup I F„ (x) - D(x)I <(0.7975)1 3 ,,. xER'

13. RATES OF CONVERGENCE ASSUMING FINITE FOURTH MOMENTS We begin with a lemma that is used in computing error bounds. LEMMA 13.1 Let X 1 ,...,X n be n independent random vectors with values

in R' having finite third absolute moments p 3.^(1 <j (n) and satisfying n

n '

EX^=0,

-

Cov(X^)=I,

(13.1)

j -1

where I is the k x k identity matrix. As usual, let x,,, j = with cumulant of X^

(I < j < n),

n

x.=n -I T, X,,,.

()v)=3),

(13.2)

j -1 n

p 3 = n - ' I P3j. i-1

Their the variation norm of the signed measure P 1 (-ID: (x,.)) defined in Section 7 satisfies 11P1(-1: {Xr})II
3 / 2 -3k'/ 2 +2) p3. (k (.Z 17

(13.3)

Convergence Assuming Finite Fourth Moments

III

In the special case k = 1, -1/2 ( 4 e -3/2 + 1 )1µ3I II P I( -1: {x})II = ( 2 ir)

< 3 (2 i) -1 ' 2 (4e - 3 / 2 + 1)P3. (13.4)

where n µ3—n-' 2 EXf.

Proof We first prove (13.4). By (7.21), IIPI(—t: {X,})Il = 61 µ3I JRJ Ix 3 -3xlo(x)dx = 3iµ3^^1o,31/2)(3x—x3)$(x)dx+

j 3112 00)(x3-3x)41(x)dx l

-I/2 {((x2-1)e-X2/231/'+[ (1—x2)e-x'/2]31,3) µ3 =(2r)

o

= (2r) ' 2 (4e

312 + 1 )Jµ3]


where ¢ is the standard normal density on R'. For arbitrary k, we use (7.20) to get IIPI( — 'D: (X,} )II < 6 [ IX(3.o....,o)I + .. . + IX(0.....0.3)II f. 1x3-3xlo(x)dx + i [

o)I + P(2,0,1,o.....0)j

.. .

+ IX(o.....0,1,2)1] f l x 2( 1—x I)I$( x 1)$( x 2) dx 1 dx 2 R2

+ IX(0....,0,1,1,1)1]fR ,I x i x 2 x 3141( x 1)4, ( x 2)41( x 3)dx i dx 2dx 3 /

<3( 27r ) -'/2 ( 4 e -3/2+ 1 ) 9,, +(ire)/2)-'0.,2+ i7l

)

3/2

9 ,3,

(13.6)

112

Normal Approximation

where, writing Xi _ (Xi , 1 , ... , Xj k ) and using (6.21) give en, I = IX<0)I + ... + IX(0,...,0,3)1 k

n

=- 1 EX^ i ^ i—t

l

j-1 k

n

N3

0",2 = ("(2,1,0....,0)1 + ... + IX 0....,0 1,2)1 '

n

=

E(X iXj.r )

n-1 1
j-I k

k

(

x.


k

± I Xj.i'I —i-1± I Xj.iI3

i'-1

—1 n

k


i-1

< k 1/2P3 — k — ' 12p3 = k l / 2(1 — )P3,

0n,3 = IX(1,1,I,o,....0)I+ ... +IXCO,...,0.1,1,1)1 n = 1
n —' E j-1

i#i',i'#i",i"#t n

k

3

cn - IEE (GIXj"I j—I



k

k

-3^X;i^ i-1

i'—I

k

IXj,'l - IXj.11 - IXj.il 3 i-I


<(k 3 / 2 — 3k I / 2 + 2 )P3.

Q.E.D.

(13.7)

Convergence Assuming Finite Fourth Moments

113

Let us now introduce a kernel probability measure K on R' whose density i is given by k 11 ga.4(xl)

W(x)=

[X=(XI..... X k )ER k J.

(13.8)

i—I

where a >0, and g 4 is the probability density on R' defined by (10.7). In fact, in this case we may compute the constant in the expression for g a and get 3a sin ay

4

(yERI)'

ga.4(Y)= 2ir( ay )

(13.9)

We shall choose )1/3 a=(4 512 k5/6=2,,7—I/3k5/6. 4

(13.10)

By (10.9), k

K(t)= IT ga4 (t 1 )=0

for I
if t [-4a,4a] k =(1:It 1 1 4a

i-I

[t=(iI,...,rk)E R k ].

(13.11)

Note that

( sin ay l

3ak

K({x: IlxII> 1})^— f [k-'12'x) 1 ay < 3k

4

1 dy

U_4dy= k (ak -) / 2 ) [ak-'/x co )

3

= .

(13.12)

IT

Hence

K({x:

JJxll8.

(13.13)

In the proofs of the theorems below we use some theorems of Section 8, substituting n-(s-2)/2p5 for ls „ (s>2). This is justified in view of (8.12).

THEOREM 13.2 Let X 1 ,.. . , X n be n independent and identically distributed random vectors with values in R k satisfying EX,=0,

Cov(XI)=1,

P4=E11X1114
(13.14)

114

Normal Approximation

Let Q„ denote the distribution of n - '/ 2 (X I + - - • + X,) and let 1 be the standard normal distribution on R k . Then for every real-valued, bounded, Bore/-measurable function f on R k ,

f

fd(Q„ -4 >)

<w1(R k )I (3Cp+aI(k))P3n

-I/2

k 2 +;I -1 / 2 kn - '(logn) -1 / 2 +a 2 (k,P 3 ,P 4 )n - '(logn) /

k (k -2)/2 1 5/2 3 logn 3/2 + zr k P3(n (I'( 2 ) + 3 ( 2 k)

+ 3 wf (27/27T-1/3k4/3p3n-i/2: t),

))-26.1

(13.15)

where wf is defined by (11.24), c o is the universal constant appearing in the Berry-Esseen bound (12.49), and -'/2 (4e -3/2 + 1)+ 3(7re' /2 ) -' k' /2 (1 — k - ') al(k)= 2 (27x)

) 3/2 +a

(

(k 3/2- 3k' /2 +2),

a2(k , P3,P4)= ik k / 2 (k+2)2

k+1

(r(

I

2

)/ (0.14+

+(0.19)(2.3) k k k / 2 (k+2)(k S =(logn) k / 2 J 00

24 )

+4)(r(fl) -' , P2

[rk-Ie-o.264r2+rk-Ie-(I/2)r2 1/2)

+ Ip3n-I/2rk+2e-(l/2)r2]dr.

(13.16)

Proof Let Z denote a random vector whose distribution is K [see (13.8)(13.13)]. By Corollary 11.5 [more specifically, inequality (11.26)],

J fd(Q„ -

-

D)I
Convergence Assuming Finite Fourth Moments

115

for all e > 0. Choose e = 4ak 1/2P3(2n) - I

/2.

(13.18)

Then (2n)"2 if Iltll> (13.19) P3

K1 (t)=0

For every Borel set B and r > 0, write B 1 =BnB(0:r),

B 2 =B\B 1 .

(

13.20)

One has

I (Q


—

'':

(

X1)) *KE(BI)I,

(13.21)

where H„ = Q„ — 4 — n I/2P1( — ( D: ()) and X is the with cumulant of X I (only cumulants with I I = 3 enter into the expression for P I ). By Fourier inversion and (13.19), Hn *K,( B I)I <( 2i )

-k

Ak(B1)

f I H (t)K1(t)I dt <(27r)-kAk(BI)(11+I2), (13.22)

where Ak is Lebesgue measure on R k, and (by Theorem 8.5) I

=f

IH.(i)Idt

(II+II< , 1 / 2 /(2p /2 ))

(

0.14 + ? 4 n 24n

+ <

0.03p3 n

)f 1 ji,114 exp(—

zIItII2)dt

f 11111 6 exp{ —0 . 38 311111 2 ) dl

(Q. 14 + 24n )k(k+2)(2ff)k /2

+

OA7p3 7t k j2 n ( 0.383) k(k+2)(k+4),

(13.23)



116

Normal Approximation

and (by Theorem 8.7) ) IHn (t)I dt / ^/\ 2 P^ / ^) < IIIII <(2n) 1/=^ /P

n

< f

[exp{ —0.264(1111 2 )

(II 1 II>n h/2 /( 2P,1 /2 ))

+( 1 +bn -1/2 P3IItII 3 )exp{

—

iII 1 II 2 }]dt

i/z Lrk—le-0.264r2+rk—Ie_0/2)r2 27rk/2(r(

2

j1/2

n^ /( 2 Pe

+

n-1/2rk+2e-0/2)r=ldr

=21rk/2(r( i.))

(13.24)

Thus I Hn*KK(Bi)l < 2-(k-2)/2(I'(2 )) (k+2)I 0.1 4 + 24n

,

+(0.14)(1.532) -k / 2 (I'( 2 )) (k+2)(k+4)p3n -I r k +2-tk-2)k-'(r(2 ))

-2

(13.25)

snrk.

Also, by Lemma 13.1, n-1/2lPI(—c: (X1.))*Kc (B1)I <1n - " 2 IIPi( — : {X„))II ^2 f 3(217) -1 / 2 (4e -3 / 2 +1)+(1re 1 / 2 ) - 'k'/ 2 (1 —k - ^)

+

=

))



\ TT/ 2 3/2

ai(k)P3n

-

(k3/2-3kh/2+2) lp3n—i/2

'/ 2 ,

J

(13.26)

117

Convergence Assuming Finite Fourth Moments

say. Next, (Qn -4')*Kc(B2)I <max{Qn*K,(B2)+ ^*Kc(B2))

< max (Prob(IIn -' / 2 (X I + • • • + X„) II > 2 ),

f

O(x)dx}+Prob(IIeZII>

JJ)

(IIxII>r/ 2 )

2

)• (13.27)

By the Berry-Esseen theorem, Prob(Iln -1 / 2 (X i + ... +X„))I > 2 ) k

<

Prob(In -'/Z (Xi , + ... +X1)> r

2k'/Z

)

/2 < 2c 0p 3 n - '/ 2 + 4k 3 (21T) 1"2r - ' e -?/sk

$(x)dx <4k3/2(21T)-i/2r-'e-,^/sk {IIxll>r/2)

r

512k°p3

Prob(IIEZII> 2)< 52 3 3 2 irr n

(13.28)

where XJ =(XJ , I ,...,XJ.k ), 1 < j
-'/2+4k3/2(2i')-1/2r- ie-r2/8k

" P

+512 ;/Z 4 3 3/2 .

23

(13.29)

2

Now take r=(8klogn)'/2.

(13.30)

118 Normal Approximation

Then from (13.25), (13.26), and (13.29), (Q,, -

<2tk + 1)

(r(2 ))

k k / 2 (k+2)0-14 -1

/2

+ 24n ) (lo g n)k

+(0.14)(2.3)k(r(2))- k k / 2 (k+2)(k+4)p32n - '(logn) k / 2 +2(k +4)/2k(k- 2)/2(I'(2 )/-2d,, (logn)k /2+ai(k)P3n-I/2, (13.31) I(Qn- 4) *K.(B2)I <2c,P3n-

1

/2+kir- 1 /2(nlog'/2n) -I

+81r -I k $ / 2p3(nlogn) -3 / 2

Now since

1i( Q,,- (D) *K,11 =2sup{i(Q,,-(D) *K,(B)I: BEl k ),

(13.32)

by using (13.31) in (13.17) one obtains the desired inequality (13.15). Q.E.D. One can easily deduce from (13.15) that there exists a constant a 3 (k) (depending only on k) such that

I

1 fd(Q., -( D)I ( wj(R k )a3(k)P4n-

1 /2+3w)

(27/2 -1/3k4/3p3n-1/2: t). (13.33)

We prove a generalization of (13.33) to the non-i.i.d. case. THEOREM 13.3 Let X 1 ,.. .,X, be n independent random vectors with values in R k satisfying (1<1(n),

EXO n

n

-

1

2

Cov(Xj ) =1,

(13.34)

j-1 n

P4 =n -I I EIjXj 11 4 <00. j-1

Let Q„ denote the distribution of n - " 2 (X 1

+ • • • +

X„). Then there exists a

Convergence Assuming Finite Fourth Moments

1 I9

constant a3 (k) depending only on k such that for every real-valued, bounded, Bore/-measurable function f on R' one has f fd(Qn —

t)I <wj(Rc)a3(k)P4n-'/ z+3w.i (27/21-"/3k4/3P3n -1 /2: (13.35)

Proof. We use the same notation as in the preceding proof. Then we have [in place of (13.22) -(13.24)] IHn*K. (B1)I <( 2 Tr)

-k Xk(BJ)(J1

+J 2 ).

(13.36)

Theorem 8.6 yields Ih.(t)Idt
J1mf



(13.37)

(hit<.i nh / 4 /p /4 )

if [see (8.30)] Pan -' < 1,

(13.38)

and Theorem 8.9, with & = Z — 2'/ Z leads to the estimate ,

Jz=

f

'12

(In I/4 /Pi 4 <11 1 11<( 2 n) 1p,]

J (hut >f(n /v.)'

")1

IHn(I)Idt

l 3& 11 1 11 2

_I

}

+ (1 + b pan - 1/1 11 , 11 3 )exp { — II 1 II 2




}

J

dt

(13.39)

Note that we used (13.38) in the last step of (13.39). Now choose r= (8klogn)' / 2 and obtain -1 /2, (13.40) Hn*KK(B1)I
and IHn"K,(B2)I
-

'

+8ir-'k5/2ps(nlogn)-3/2 < ag(k)P4n-i/2. (13.41)

120

Normal Approximation

The rest follows exactly as in the preceding proof. It is simple to check that (13.35) holds with a3 (k)=2 if (13.38) is violated. Thus (13.35) is proved for all cases. Q.E.D. One may compute an explicit error bound, with numerical values for the constants involved, in Theorem 13.3 much the same way as in the i.i.d. case (Theorem 13.2). of all Borel-measurable A significant application is to the class convex subsets of R k . By Corollary 3.2 (with s = 0) sup w+ r (2e:4)= sup 4b((8C ) 2 e)

cEe

cEe c

2 5 ' 2 F(( k + 1)/2) C

(13.42)

r(k/2)

Using this in Theorem 13.3 one obtains sup I Q,, (C)

CE2

—

(C)I < a1o(k)P4n-^/2.

(13.43)

In the i.i.d. case one also has Tim sup V I Q,, (C) — P(C )l " -10° CEC

1) / +1(k)+ 3 1 8 'r - ^ /3k 413 r( k+ 1) 2) IP31 (13.44) where a 1 (k) is given by (13.16). This follows from Theorem 13.2 and the estimate (13.42), provided that one replaces the first estimate in (13.28) by Q,,({x:

Jlxii>z( 8 klogn)" 2 })=o(n -1/2 )

(n-*oo),

(13.45)

which holds (see Corollary 17.12) if P 3 < oo.

14. TRUNCATION

We need some truncation estimates for relaxing the moment condition (namely, finiteness of fourth moments) of the last section to obtain rates of convergence for integrals of unbounded functions and (in Chapters 4 and 5) to derive precise asymptotic expansions. Throughout this section X i ,...,X„ are n independent random vectors with

Truncation 121

values in R k having zero means. We write, as usual, µa j = EX,

X = ath cumulant of X^,

1 p5.j =EjlX ll''

(l < j
n

n

E

Xa=n—' I Xa.j,

A'aj'

(14.1)

l —1 J°I n

[aE(Z+ )k,

p,, = n -I Y, ps,j

s>0].

Define truncated random vectors

X^ Y= 0

1

if IX,II

n 1,2

(14.2)

if I^Xf ll > n

Zj =Y1 —EYJ (Ic j
= EZ,j ,

(14.3)

ps.i = E IIZ1 II5, n

n

ps=n E11ZJ II 3 ,

Xa=n —I 2 X•

—I

j —1 l-1

Also introduce n s

Anj,s

= IIXjII

^n.a =n

—

I

11 Xj 11 >n^^ Z }

(

I An,j.a.

(14.4)

j=I

Finally, write Cov(XX )=((vii )),

V=n-I n

D=n -I Y, Cov(Zj )=((dij (14.5) jal

The constants c's depend only on their arguments.

122

Normal Approximation

LEMMA 14.1 Let p, < oo for some s>2. (i) One has

p,,=EIIY11I3+


(14.6)

(ii) If a is a nonnegative integral vector satisfying I < a <s, then EX — EYl I < n (s lal)/2A "j 5 —

—

IEY^ — EZ^ I < Ial( 21a1 + 1)n1/2An,i.5.

(14.7)

+d11 —v1A 42n -( s -z) / 2 p n.: (1
(14.8)

(iii) One has

(iv) If 2 < s' < s, then

p—EIIZ IIS <2fpJ'j ,

EIIY;II
;

S

ps <2 S p,..

(14.9)

(v) If s' > s, then

EIIY IIt <(En'/2)(3

-s)^

IIX^iis

{IIXj IjGen/ Z )

+(s'—s)/2^

I° < n (f — s ) / Z pf.i (0 < e < 1), IIX^I

{ sn/2< II Xji

Gn2 )

ps..i =EIIZf ll s <2'EIIY II'.

(14.10)

Proof The relations (14.6) follow from definition. To prove the first inequality in (14.7), use the inequality (6.29) to obtain EYj! - I= EX°— l

f

f

X < r 11X.111°I

I {^^Xju>n^^ 2 }

j

l

toxilI>n 1/ )

,-(s-la)/2 fIIX.II s =n -(s- l a l ) / 2 0 nJ,s• {IlXill>n1/2) j

(14.11)

Truncation

123

To derive the second inequality in (14.7), first observe that

xa_YaI_Ixr i...Xk —Y1O". k I = Ix '

Xk I ( Xk Yk k )

+x '• ..Xk 2( xk I Yk I)Yk +... + ( x 1 ' YI ' )Y2 =.•. Yk I


xk

Yk)I ak max (Ixkl ak-I, IYkI

+... +a,max(IxI o1-I, IYII a '

1

N-1 )

)IYZ :... yk (XI — YI)I

< IaI(IIxll lal-I +IIYII I°I-1 )max(lxi — y, : I < i 6 k},

(14.12)

k X=(XI,...,Xk)+Y=(YI,...,Yk)ER ].

Since by (14.11) IYi.r — Z^,I=IEYI
-(:-I)/20

n.•

(1
(14.13)

it follows from (14.12) that

EYE —EZ7I
'EIIY'+EIIZJIIIai-1). (14.14)

Now EIIY^III^I-1= /

IlXJIlal-1
(14.15)

IIEY1II <(EIIY;II 2 ) 1/2 , E I Z^ IIIai-1 < 2 1al -'( E 1IYI II Ial -1 + II EYY II I " I-1 ) C 2laI n tlal- It/z . Hence IEY^ — EZJ I < IaI( 21al + 1)n-(s-IQI)/2An.i..,.



=

124 Normal Approximation

Next we have

4 Idir—vtt1
= n -1 2 I E (XX,1 X1. , — YJ,rY!.,) + E (Y)E (Y1.t)I i- 1 n
l

n,j,s +n

) h/2n -(s-U/2^ nJ.s

l

f I —

2n

-(s-2)/2^ A t.

(14.16)

To prove (iv) note that [using (6.26)]

11Z;II J <(IIY;II+IIEY;II) s < 2s- '(IIY;II S +(EIIY; 11 2) s/2 ) < 2s -'(IIY^II' + E IIYYIi s ) Proof of (v) is clear. Q.E.D. Recall that the norm of a k X k matrix T, denoted IITII, is defined by 11 TIJ =

11 Txll.

sup

(14.17)

xER k .IIxII C I

COROLLARY 14.2 Under the hypothesis of Lemma 14.1, one has I—I<2kn -( s -2) / 2411 t II Z

< 2 kn -(s-z)/2 ps11 1 II 2 (t E R k ),

(14.18)

so that

lI D —VII < 2kn

-(s-2)/2E

2kn _(5_2)/2p3 (14.19)

In particular, if V = I and

ks
(14.20)

then D is nonsingular and

a
IID-III
(14.21)

Truncation

125

Proof. By (14.8) and using (6.26) in the last step, we have k

I—I= i t;tr(d,, i,t-i

—

vu) 2

k

< 2 n -cs-2)/2 p'.:(2 It^I) < 2 kn -cs-2)/2 p"SII 1 II 2 .

(14.22)

Since the matrices D, V are symmetric, (ID — VII= sup II<2kn

-(

II+II <

' -2

l/ 2 A- n ,5 .

(14.23)

Also, if V= I and (14.20) holds, then


(14.24)

(t,Dt>> (ItiI2—aIItIIZ=aIItIIZ

(14.25)

IID — III,

2

and

The last inequality implies that D is nonsingular and that II D - ^ II < i Q.E.D. The next lemma concerns the growth of derivatives of the characteristic function of n -1 / 2 (Z 1 +"• + Z„). LEMMA 14.3 Suppose p s < oo for some s> 3, and that V = 1. Let gi denote the characteristic function of ZZ (1 < j < n). Then if 1/2

II t II < 16p 3 '

(14.26)

one has, for every nonnegative integral vector a,

l g' (n ' ) ^ ^fi (

D

n

+6111II 2 ).

(14.27)

Proof By inequality (8.46) one has

( <exp{ gjln^/2 )1 t

2

—

n - 'E 2 +3n -3 / 2EI<,,Z^>I 3 )'. (14.28)

126 Normal Approximation

Observe that exp{ n -I E'(t,Z> 2 — in -3/2E11 3 ) <exp{ n - '(EKt , Zj>I 3 ) z/3— in -3/2EI1 3 } < supexp{a z — Za 3 } =e 1 / 3 .

(14.29)

a>O

Now let Nr be a subset of N = (1, 2, ..., n) containing r elements. Then in the given range (14.26) one has

11 jEN\N,I

Sj

t

)1

z

n /

2

4



II exp{

—n-IE2+

3n-3/2EII3)

jEN\N,

'exp ^

—<1,D:>+ 3n-3f zj-1 1 EI <1,Zi>`3 }

x 11 exp(n -I E
<exp( — + 3n -1/zp3II 1 11 3 )e' j3

(14.30)

< exp{ — fit, Dt>+ 311 rll z }e`/ } ,

using (14.9) (with s'=3) and (14.26) in the Iast step. It follows that ) <exp( ( g( J 1 2

-


(14.31)

jEN\N,

which proves the lemma for a=0. To prove it for nonzero a, note that

(DMgj)(

1/z)l=n-1/2IE(Zj,n ,expfi))I

n

i

=n-1/2IE[ZJ m(exp{i)

—1)]I

I
[Zj—(Z 1.1,...,Zj,k)],

(14.32)

using (14.9) in the last step (with s' = 2). For a nonnegative integral vector

Truncation

127

/3 satisfying I /31 > 2, (Dagi)( (- )I < n

-1PI/ZEIZBI < n -IfiI/Z P ,

6 21 #I n -'p2j,

(14.33)

by (6.29) and (14.10). Thus

(DPg^)( ^/2)I
(1
where

c2(a,k)=max(4,21al),

uhf = max{ 1 ,II'll).

(14.35)

By Leibniz' rule for differentiation of the product of n functions, (D*Hj_,g1 )(t/n 1 / 2 ) is the sum of nI"I terms, a typical term being

J

[ f^N, 11 gi\n,2))[D4Igi(112)

_j ) [DPgj(_( ... J,

(14.36)

where N, _ {J1,.. . 'Jr)' 1 1

(1
Q;=a.

(14.37)

r-i

The number of times the expression (14.36) is repeated among the nIaI summands is given by a1! .. ak l

r

(14.38)

k

II H a, i-1 -I

where a=(a,,...,ak),

$i=(Qri •.• Qik) ,

,

(1
(14.39)

In view of (14.31) and (14.34), the expression (14.36) is bounded in magnitude by exp{ 6 — Z+ blltII 2 } II b1 , BEN,

(14.40)

128

Normal Approximation

where bi=n-IC2(a,k)P2j

IltII (1

(14.41)

< j
Therefore

_ n

(D a 11 9)i)(_/2 ) i I

G 2 c 3 (a,r)exp l 6 — i+ 611t11 2 }' j `(j b I
(14.42)

BEN,

where, for each r, I• denotes summation over all choices of r indices from N=(1,...,n}. Hence

n

_

11 b'! <(^ )-I b;) =(c2(a , k)P2 Iltll )r

sl BEN,

(14.43)

=(c2(a , k)k Iltll )'•

The inequality (14.27) follows on using (14.43) in (14.42). Q.E.D. COROLLARY 14.4 If in addition to the hypothesis of Lemma 14.3 the inequality (14.20) holds, then Da gi)( (

i-I

I z )I^ci(a , k)( 1 +IItfl l°l )exp( — i IItII 2 )

(14.44)

n/

for all t satisfying (14.26). Proof. Use (14.25) in (14.27). Q.E.D.

We also need an estimate for the difference between a polynomial expression in the cumulants of n - '/ 2 (X I + • • • +X„) and the corresponding expression in the cumulants of n - I / 2(Z I + • • • LEMMA 14.5 Suppose that V= I and (14.20) holds for some s > 3. Let v i ,...,v m be nonnegative integral vectors satisfying m

^v,I>3

(1
2 (Iv1 l-2)=r,

(14.45)

i-I

I < r < s — 2. Then Ix,: xi. —x1... x,j < c4(s,k)n-('-r-2)12.,,

for some positive integer r,


' -2) / 2 P,.

(14.46)

Truncation

129

Proof First recall [see (6.14), (6.15)] that for each nonnegative integral vector v, X^ i is a linear combination of terms like µa;,i...µ i

(14.47)

where a,,...,ap are nonnegative integral vectors and r,,...,rr are positive integers satisfying P

r; a; =v

(a ; >0,1
(14.48)

To estimate the difference y, — y i it is therefore enough to estimate 1.... µ

µ« i l

r —' r«i ....µ ,r r «r.J l J ay,J

(14.49)

subject to (14.48) and 3 < I vJ < s. By Lemma 14.1(ii), (s I«I ) /Z0^ (I < II < s). µa.i µail < C5(a)n ✓,s

(14.50)

We may use (14.50) and (14.12) to get µayj...N ^—^^'aiIJ.

..µr^.il

P

<

r.0

l µ .,-iI + I µ '.-lI

( a ) n -(s-da)/20

6

nj,s( 0.J

!=I r^



r.

x µa,j

µ„iµ«,.,,1

...

o4J

'rr

µmil

P

<

c a n

—(—kI)/2

m;/s

(14.51)

^n J,sPs,j '

where c 7 depends on i, a's, and r's, and P

m;=

r; Ia; l ,

r=i

,

—

Ia1I= Iv!

—

Jail (14.52) ,

by (14.48). In the last step of (14.51) we have used the inequalities [see (6.27), Lemma 6.2(ii), and (14.9)] i l/s µaiI < plal,i
(14.53)

130

Normal Approximation

In view of (14.20) one has

11 )11'+A n j

PS.)

<

S

n+ORj.s

(11X;11 4.111)

< nJ/ 2 +n0,,, J < nJ/ 2 + 8k2 (14.54) so that n-(s-larl)/Zp j/J=n-(s-Ia,I)/2P;^VI -Ia,I)/s < n-(s-Iarl)/2n(Irl-la;l)/2 1 + (

8k

< c 8 (a„k)n -( s - I'U/ 2 .

(14.55)

It now follows from (14.51) that Xrj—Xrj

I
and, therefore, averaging over j=1,...,n, ix

— xi

< c 9 (Y,k)Zn -(s-1• I ) / 2 (3 < Ivl < s).

(14.57)

Let now v,, ... , vm be nonnegative integral vectors satisfying (14.45). By (14.57) one has xv, ...

w _^... m C9 ( 1,

k)Q n'5 n -(s-Irrl)/ZlXrr..

. ^i-Ix'ra^...



(14.58)

By Lemma 6.3 (and averaging), Lemma 6.2(iii), and Lemma 14.1(iv) one gets (recall that P 2 - k)

< C10(s k)PI. 1 1 ,

...

p1rr - ^IpjP,.1l

.z)/(=- 2 ) r
...

p1.I

(14.59)

Truncation

131

But because of (14.20), one has n

p, = n -'

p5 , j=I

n

=n '

(f

—

IIXjIIS+O„,i.sl

lll xill
j 1 L

n


{11x:11
< kn ( s -2) / 2 +, s <(1 + 8k)n(s-2)/2.

(14.60)

Using (14.60) in (14.59), one gets ..

IX.,... X;-^

.I < c12(s,k)n (r-II+2)/2 .

(14.61)

Substituting this in (14.58) we obtain (14.46). Q.E.D. The above estimates also lead to LEMMA 14.6 Assume that V=I, and that (14.20) holds for some s>3. Then for every integer r, 0 < r < s — 2, one has n -r/2 V ' ( - 4: {X.})(x) — Pr( - 4o,n: {XP'})(x)I < c r k s 0 n - (5 -2 )/ 2 (1 +lixIl 3 T+ 2

2 xexp{ — 11611 +IIXII}

)

(xERk),

(14.62)

and 2IP,(—$: (X,.})(x+a„) < ci4(r,k, ․ )in,5n

—

-

Pr(

: {X.})(x)I

3.+1 ) C, 2)/2 ( 1 + 11x11 -

xexp(-211x112+ 8

where

—

1Ik /Z }

(xER'`),

(14.63)

)J a„=n - '/ 2 EYE . j—1

(14.64)

Normal Approximation

132

Proof We first obtain an auxiliary estimate. For z E C k write k

g(z)= l -(DetD)-1/2exp^ i 2 z?

-

i-1

i

d11z,z1 1
Iz=(z 1 ,...,Z k )}.

Then k

z?-i drrz1z,.

g(z)I=I(DetD )-1/2exp{i

i=1

1




-1-

i,i'k

III(IIZ11 2 )}

+zlID -1- III(IIzII 2 )exp{IID -1- f II(IIzII 2 )} Z=(Zl,...,zk)EC k ],

(14.66)

where Det denotes determinant and d" is the (i,i') element of D -'. By Corollary 14.2 [replacing t by Bt in (14.18), where B 2 = D -1 ],

- II 1 11 2 1 <2kn-(:-2)'2pn,,IID -I II(IIt11 2 ) <3kn -(s-2)2

p^S11 1 11 2, (14.67)

which implies II D

-1 _ I II < 3 kn -(s-i)/2 „ ^ <;68) . (14.

Also, expanding DetD and making use of (14.8) and (14.20), we get I(DetD) -1 / 2 - l l = (DetD) -1 /'^ I -(DetD ) 1 / 2 I <(Det D ) -1 / 2 t1-(Det D ) 1 j 2 I(1 +(Det D ) 1 / Z) = (DetD) -1 / 2 11- DetDj 4 II D -' ll k / 2 11- DetD l < c15(s,k)L3n-(s-2)/Z

(14.69)

The estimates (14.68), (14.69) are now substituted in (14.66) to yield z _ 14311 1 ^g(z)I
(zECk). (14.70)

Truncation

133

Hence max

I8(z)i
(.-, e& , II: •- zIl
(Ilz11+ 1 ) 2

l

3

J

II 311 2

< c17(s,k)On,5n-(s-2)/2(1 + Ilz11 2 )exp j

+ lizII .

J

l (14.71)

By Cauchy's estimate (Lemma 9.2) one then obtains

I

IDg(z)l
From this follows D' ($ — $o,v)(x)1= I E c19(a)(D a $)(x)(D' -a8)(x) 0
/z(l+IIXII2)expl—

I-

(xER k ).

+11x11} (14.73)

Now recall from Section 7 that [using (7.3) and Lemma 7.2] P,(-0: {x)) r

m^ I

m

`

I ^*

(^**

J^,....Jm

^

vi....,v,,,

^

I• •

m•

(-1)r+zm Ll

P,+...+VT

^I (14.74) J

where X* denotes summation over all m-tuples of positive integers (j l ,...,jm ) satisfying m

j

i

=r,

(14.75)

f=I

and X** denotes summation [for fixed (j l ,...,jm )] over all m-tuples of



134

Normal Approximation

nonnegative integral vectors (v 1 ,.

''m)

1v1 1=j; +2

satisfying

(1
(14.76)

The function P,(-4 o,0 : (c)) is similarly defined by replacing X • by X • and 0 by 4 o.o in (14.74). Next observe that

I XYI...X• D•.+...+.^$(x)_ ... .D•l+...+'.^o,o(x)^ II I ... Xy Dr,+...+P-

(O(x)-0o,a(x))1

+J(X•....X• —X.,...x. )D°^

+...+

'"4,(x)I

(xER k ). (14.77)

By Lemma 14.5 ... Xyj < JXP,... XP_ _ X: 1

c4 ( s ,k) n -( s -r-z) / 2 Q

(14.78)

Also, as in (14.61), one has 1Xy^

...

(14.79)

X^j 4 c21(s,k)n"/2.

Using (14.78), (14.79), and (14.73) in (14.77), we have n(x)l I X•I .. . X. DY^+...+ .4(x )—x , ... X;.D•"+...+ .'Po.


II 6112

+IIxII}

(x E R k ). (14.80)

The inequality (14.62) follows from this and the expression P, given by (14.74).

To prove (14.63), first note that by Lemma 14.1(ii) Ila,ll < n -1/2 I IIEY;II < k 2En,5n-c,-2)/2 <(8k" 2 ) - '. (14.81) I=I From the expression (14.74) and the inequality (14.79) (which also holds if the primes are deleted), we get {})(x+a„)

—

P,(

-

4: {X,.})(x)i


Truncation

135

But D'4 =q"çt,, where q" is a polynomial of degree Iv( with coefficients depending only on v and k, so that ID"4(x+an)—D"-O(X)I
But by (14.12) and (14.81),

I(x+an) °— x°I


(xER k ), (14.84)

and I(x+an ) ° [4(x+an )-4(x)]I

= (x+an) ° -O(x)Ijexp{

—

illx+anII 2 +illxIl z } -1 ^


(x+an) a 14(x)IIanlI(I+11x11)exp{IIa.II(l+11x{1)}
-(,-z)/z

(l+IIxII kN1 )exp{ —

8

k'11

2 }• (14.85)

Relations (14.84), (14.85) are used in (14.83) to yield ID"¢(x+an )

1

—

D"4(x)I
-(5-2)/2

Xexp{ — zIIxII2+

(l+llxII'

81k / z }.

1

)

(14.86)

Finally, (14.86) is used in (14.82) to get (14.62). Q.E.D. A bound for absolute moments of sums of independent random vectors is needed to prove the important Lemma 14.8. This is provided by LEMMA 14.7 Suppose that V=I and (14.20) holds for an integer s)2. Then there exists a constant c27 (r, s, k) such that EIIX,+... for I< r < s.

+X n llr
136

Norma! Approximation

Proof. In view of Lemma 6.2(ii) it is enough to prove the lemma for r = s. For r = s = 2, (14.87) is trivially true. Let us then assume that s > 3. Also by (6.26), z

n

k

l/2

EIIX1+••• +XII'=E E l i-I !

—

/

i

k




E l

s

x1 j ,

i- 1 [Xi=(XJ.i,...,XJ•k)],

(14.88)

so that it is enough to prove the lemma for k = 1. If s is an even integer and k= 1, then by Theorem 9.11, s-3

n-,/zE(X1+... +X.) 5 <

1

n -m/2D sgm( 0) +cza(s) n

-

(: z)/2p,, (14.89) -

m—O

where

r z l g(t)=Pm (it: (7t,))exp( 21 }

(tERI).

(14.90)

By Lemma 9.5 (with k = 1, a = s, z = 0) and inequality (14.60), one has D'Sm(0)1 < c29(m, ․

)P;'/(,-z) < c3o(m, ․)n m/z '

(0<m<s-3).

(14.91)

The lemma is proved for even integers s. Next assume that s is an odd integer, s> 3, and k = 1. Clearly, EIX1+••• +X<EIZ,+••• +Z n l' (14.92)

+EI(X 1 +••. +X„)'—(Z 1 +... +Z„)'I.

By Lemma 14.3 [inequality (14.27) with k= 1, a = s + 1, :=0], EIZ,+••• +Z„I°<(E(Z 1 +••• +Z„): +1 ) < c 31 (s)n'/ z .

s

+1

(14.93)

Truncation

137

Next, EI(X1+••• +X,,)s—(Z1+...

+Z,,)uI

n EI(X 1 + • • +Xj +Z11 +••• +Z n )

<

3

j-1

—(X1+... +Xj—I+Zj+••• +Z,) 5 I n

44

s

1 7. (,) E I( XX — Zj)(Xl +... +Xj—I+Zj+I+... •}•Z ) sm I j=1 m—I n

s

s

+... +Xj—I+Zj+I+... = 2 Y. ( ) E I X7 — Zj IEIXI

+Znls—m.

j-1 m-1

(14.94)

By (6.26) and (14.54) we have E IX 1

+•••

+Xj—I+ZZ+I+...

+Znls—m

62s—m—I (EIXI+...

+Xj+Zj+l+... +Z nl s—m+EIXj Is—m)

<2s—m—lE•IX,+... +Xj+Zj+l.^... +Znls—m +2s-m- ► Bn(s-m)/2.

(14.95)

Writing Tr.j=n-I(EIXII'+... +EIXjI'+EIZj+II r +

...

+EIZ,,I'), (14.96)

one has, using Lemma 14.1 and inequality (14.20), n

T 2•j
EZj +1 EYE +; <EX^ +; ,

(14.97)

j-1 Tz.j — n -1 E^Xi+

... X)

—

E(Xj+I — EZ^+I) —... —(Exn—EZn)]

=n -I [n — E (2— Yj+l) —... —E(Xn—Y^) — (EYj+I)'— ... — (EYn)2]

138 Normal Approximation n

> I-n' I n-(s-2)/20

n

n,j,s

-n-I E n -(s-i)Az

n j,s

j_ j-1 )

I

-

20 n s n —(s-2) / 2 ^4^

7rJ
IXII'+... +EIXjI ' + 2 '(EIXj+I I ,+... +E I X nl ' )]

Look at the random variables

Xi j, l
W = ;

T2 (14.98)

the j+l
T2 j

Applying Theorem 9.11 (with k= 1, a=s-1, t=0) to these random variables, we have n s—m

E ^ W ^

5-1 (s —m) / (s —I)

n

< (E (i -I W1 /

'

I)/2)(3-m)/(s-1) <(c32(s)n(s=C 33 (s,m)n/ 2 (1 <m <s).

(14.99)

In deriving this we have proceeded exactly as in the proof for even integers s. More explicitly, we have used Lemma 9.5, inequalities (14.89), (14.91) with W's replacing X's, rs ,j /-r2j 2 replacing ps , and inequality (14.97) and the modified polynomials Pm corresponding to the cumulants of W., 1 < i < n. Hence EJXI+... +Xj+Zj+I+... +Znls—m

_

T

(j

n s—m

m)/ 2 E)

W,)

(C34 (s,m)n(s-m)/2



(I < m < s).

(14.100)

Using this in (14.95), we get EIXI+... +Xj-1+Zj+1+... +Znls—m

< c 35 (s,rn )n (s

-

m)12

(1 < m < s).

(14.101)

Truncation

139

Also, by Lemma 14.1 and inequality (14.60), n

n

I EX; Z;I< Y. (EIX; —

l-1

—

y;I+IEY;I)

l—1

—

J

_ (

Ox 1>n 1 / 2 ) ;

IX'I+IEYJI)

n

44 1 (n-1/2EX2+n-'/2)=2n1/2, J-1 n

n

2 EIX^—ZZI<2 2 EXj=2n, n

n

I EIXj —Z; I< 2 (EIX,I.^+2mEIX,Im)=(l+2m)npm l- 1 l-1 <(1 +2m)npsrn 2)/(5 - 2)

( (1 + 2m)n (Qn(s-2)/2)(m-2)/(s-2) 8 = C36 (s,

m)n'"/ 2 (3< in < s).

(14.102)

Use (14.101), (14.12) in (14.94) to get EI(X 1 + • • • +Xn)5—(Z1+ • • • +Z„) s l < c37(s)n

2 .

(14.103)

Together with (14.93) this implies EIX,

+

...

+Xnl s < c38(s)ns/2,

(14.104)

completing the proof of the lemma. Q.E.D. Before we state the next lemma, we define On.s(E)=n ' ^, -

IIX^IIs

(e>0).

(14.105)

IIX^i>fn( 1/2) l—1

In this notation (14.106)

Let Qn denote the distribution of n - '/ 2 (X 1 + • • • +X n ) and Q„ that of n

- "/2(y l + ... +Y).

Normal Approximation

140

LEMMA 14.8. Let V = I. If p 6 < co for some s > 0, then there exists a positive constant c 39(s, k) such that

II Qn — Qn II < C 39(s

k)tn,3n -t'-2>/2.

,

(14.107)

Also, there exist two positive constants c4o(s,k), c 41 (s,k) such that whenever .3 )
(14.108)

for some integer s> 2, f II xII'I Qn — Q,','I (dx) < c4,(s,k)On.,n-(s-2)/2

(14.109)

for all rE(0, ․). Proof. Let G^ denote the distribution of n - '/ ZXJ , Gd " that of n 1<j
and

I

n

Q.—Q..1'II = G,*...

H !i—'

*G1— ,*(G 1 —G")*G'+,*... J J

aG,'

I

n

n

<

IIGG — Gd"II= 2 Y. P(II XxtI> n1/2 ) n

IlXjll'=2A-nsn-c:-2>/2, (14.111) <22n - s/ 2 f (IIXjII>n112) i-I

which proves (14.107). Now assume that s is an integer, s>2. Since IIxII' < l + Ilxlls for 0< r < s, it is enough to prove (14.109) for r=0 and r = s. The case r = 0 is precisely (14.107). We therefore need only to prove (14.109) for r=s. One has

f IIxl IQ. Q,'I(dx) 5

—

<

f Ilxll'IG.*

..

.

*GG-i*(G1

—

Gi )*Gi+i* g

...

*G,. I(dx)

n * G,, (du ) <j-21 f (f ►I u+U II sIG^— GJ'I(dv))G,*... *Gi-,**... n

<2s"' 2 (ii

—

Gj"II JII uII 3G,* ...

*Gj-

*G;+I*... * G„' (du )

i-1

+ f iivll 'IG; Gj' I(dv)). —

(14.112)



Truncation

141

Now

f IIvlIJIGj—Gj"I(dv)= f

( [Ix; 1 1> n'lz)

IIn

-i / 2 XJ IIs=n - s/ 2 0 n'j. ,.

(14.113)

Also,

f II uII G1* s

...

*G^-,*Gj+,*... * G„' (du )

=EIIn-1/2(X,+... +X1-^+Y;+,+

...

+Yn)II 3

<2s—'( Elfin — /2(X, + ... +X; +Y;+i +... +Yn)Ii 5

+IIEIIn — " 2 X11_, ) < 22cs—u(E IIn —/2(X, + ... +X1 +Z1+ ,+... +Zn)IIS +IIn -1 / 2 (EY^

+i

+... +EY.)II')+25- 'Eli n -1 / 2 X1 II'. (14.114)

By Lemma 14.1,

I I n "/2(E Y^ + , + ... + E Y1) I IS $

O l =(k2n-c:-2)/20 ) . (14.115)

< ( n -1/2ki/2n -c:-u/2 1

nJJ

J

J

l^^

By inequality (14.54), Elln

-

1/2

X,II s
—

s/2 (n s/2

+

< 1+n —(s-2) / 2 0„ s . (14.116)

We next estimate E IIX, + • • • + Z,, Its by using the preceding lemma. For this assume first that k = 1, and define T,.^ and random variables W as in (14.96), (14.98). Note that ;

IW;IJ=TZ fLIwi >n ) J

5/2

112

f

(/

llx,I>T=i'2&/2}

f I W. I S=T—s/2 2.i f{Iw,I>nhI2) , {IZ,I>*2 /

NIS

(1 < i <.j),

IZ I s /2

)

,

< T2 s/2 f

t IY;I> T,14i2n112— IEY;I }

lye's

+IEY1I 5P(Iv11>T2 2n"/ 2 —IEY I)] 1

(j+l
(14.117)

142

Normal Approximation

Taking c40(s,k)< 1/(8k), one has [by (14.97) and Lemma 14.1) T

EY ; I < n -( s -1) / 2 A

"

J .s

Z 2 >(4) l/2,

n' /2 < n -(s-3)/20 " S < g TZJ 2n'/ 2 —IEY 1 I >n" 2 ,

2n'/2 n'/ 2 )'p (IY;I> 3 ) IEY1I sP(IY;I>T24 2n' /2— IEY;I)<( 8

( nl/2)S ( 2n'/2 3 ) < g PIXrI> flh/2

(

)

I

2 fl 1 /2

)

—J

f

IX;Is > 2n' / 2 /3 )

(jXjj

1

S

{IX j j>2n'/ 2 /3) IXiI,

fIY;^J < f {IY I>r1t 2n'/ 2 —IEY I) ;

;

{IX;I>2n'/i/3}

^X;Is.

(14.118)

Therefore, choosing c, a(s,k)=(,1) , / 2 (l6k) - ', one has n -'

"

IW;I s < T2^S/22a n.^(s) < f {IW , I>"' 12 )

n (r — 2)/2

(14.119)

8

so that Lemma 14.7 may be applied to yield EIX,+••• +X^+Z,+••• +Z,I

$

= TZ j 2E I W, +... + W IS .

EIW J +

+W„I 5
(14.120)

For arbitrary k, apply this estimate coordinatewise to get EIIn — './ 2 (X 1 +... +X^+Z^ + ,+... +Z n )11 3
Truncation

143

The inequalities (14.115), (14.116), and (14.121) now show that the left side of (14.114) is bounded by a constant (depending only on s and k). This fact, together with the estimates (14.113) and (14.111), now gives [on substitution in (14.112)]

f II x II s !Q. — Q,'J(dx) < c44( s,k ) t

s

n.

(14.122)

Q.E.D. 15. MAIN THEOREMS

We are finally ready to prove the two major theorems of this chapter. These provide bounds for I f f dQ n — f f d (Dj over (essentially) all 4continuous functions f that are integrable with respect to Q n under given moment conditions. The bound provided by the first theorem, which is much more useful than the second, cannot in general be improved upon, and the hypothesis in it cannot be relaxed any further (at least if we are to have bounds involving moments alone). The utility of the second theorem is explained later. We continue to use the same notation as in the preceding section. For ease of reference, recall that X 1 ,.. .,X are n independent random vectors with values in R k satisfying n

EXj =O

n I Y, Cov(X1 )=I,

(1<j(n),

-

(15.1)

j -1

I being the k x k identity matrix. The moments and cumulants of XD 's are denoted as in (14.1). In particular, n

p3 =n -1 Y, EIIXj IIs j=

(s>0),

1

n

[vE(Z+)k],

X.=n ' ^. XY.i -

(15.2)

j=1

where is the with cumulant of X. The corresponding quantities for the centered truncated random vectors Zj (1 < j < n), defined by (14.2), are denoted by ps, . Recall also the notation n

D = n -'

Cov (Z^). j-1

(15.3)

144

Normal Approximation

We let Q, Q,,, Q„ denote the distributions of n -1 / 2 7-_ 1 Xj , n -1 / Z E^_ 1 Zj , n -1 / 2 _ 1 Y1 , respectively. As usual, '1" denotes the standard normal distribution on R k , and 4V is the normal distribution with mean a and covariance matrix V. For a real-valued function f on R" define M,(f)= sup (1+IIxIr)-llf(x)I if r>O, xER k

M0(f)= sup If(x)—f(y)I=wj(R" ).

(15.4)

x,yER k

If v is a finite (signed) measure on R k , define a new (signed) measure v, by if r>O,

v,(dx)=(1 +IIxII')'(dx) o = v.

v

(15.5)

As in the preceding section, write n

^n,sl E ) —n-1 f

IIXi 3

^ns=^ns(1).

(15.6)

j-1 01Xj11>cn'/2)

Then define n

inf A* s =O
O n ,,(E)+En

-

'

f

(15.7)

i=1 (jjXill
Note that

Ons
On.a—°On,s(1)
(15.8)

If one takes E = n - '/ 4 in the expression within brackets in (15.7), then one gets n s

n.s(n —1 /4) + n —1 / 4 ps .

(15.9)

Thus if X 1 ,.. .,X,, are the first n terms of a sequence of independent and identically distributed random vectors, then n -(s-2)/2

* ., = o ( n -(s-z)/2)

(n—*oo).

(15.10)

145

Main Theorems

Finally recall the average moduli of oscillation wJ and wJ defined by (11.23), (11.24). THEOREM 15.1. There exist positive constants c (i = 1, 2, 3, 4) depending ;

only on k and s such that if T

(15.11)

h.s^ 3) < c1n(s-2)/2

holds for some integer s > 3, then for every real-valued Bore/-measurable function f satisfying Mr(f)
(15.12)

for some integer r, 0< r < s, one has

J

^

- s-2)/2


+2wg (C4P3n

—

'/2

p*.s : (p+)ra)•

(15.13)

where s-2

_ I n - m/ 2 Pm (-(D:(}), m=0

and _Jr

r°

g(

)

if r is even,

r+1 if r is odd,

(I+jjxjj

ra

)

-

i

f(x)

if r>0,

-

x f(x )

if

(15.14 ) r=0.

Proof The constants c's appearing in the proof do not depend on anything other than r, k, s. We write s-2

m/2

m=0 i//" (B)=1^I(B+a n )

(BE^'ilk),

(15.15)

146

Normal Approximation

where n

an =n -1 / 2 E EYj .

(15.16)

Thus'" is the signed measure induced from iji by the map x- *x — an . Now

f fd(Q,,

—

i )I
—

Q.')I+ f fd(Q,^'

—

^G)I.

(15.17)

If r>0, then (xER k ),

If(x)I <Mr(f)(l+11x11')

(15.18)

so that using Lemma 14.8 one gets (15.19)

f fd (Qn — Q,^' )

Since the left side does not change when f is replaced by f— c, where c is the midpoint of the range of f, (15.19) also holds for r=0. Next

I f fd(Q,,' — *p)I =I f fg,d(Q.' p")I
+Ifa,d( 1P' — ip)l+I f fd('G — p")I , (15.20) where f, denotes the translate off by y; that is,

f

y

(x)=f(x+y)

(xER k ).

(15.21)

By Lemma 14.6,

f fg,d(0'-0)

<Mr (f) f (l+llx+anll')I ' - 4I(dx) <M,(f) f ( 1 +2•llanIIr+2'llxlI')Ip'

-

4I(dx)

Mr(J)I110 '- 7'lI+ 2' llanll ' ll4 '— o11+ 2' f 11x11 ' 14' '- 4'I(dx)J

C6Mr

)Q

(f

..sn-(s-2)/2.

(15.22)

Main Theorems

147

In the last step we also used inequality (14.81). Similarly,

ff.d(P-'P")
-(s-2)/2

(15.23)

Therefore (15.17) reduces to [in view of (15.19), (15.20), (15.22), and

(15.23)]

f fd(Qn

-

4')I
To estimate the second term on the right of (15.24) we introduce a kernel probability measure K on R k satisfying

K((x:llxIj 3, f IIxII

K(t)=0

k +s+ 2K(dx)
if 11111 c 9 (tER k ).

(15.25)

One construction of such a probability measure is provided by Theorem 10.1. For c >0 define the probability measure KK by K,(B)=K(E

-

'B)

(BE, E 'B=(E 'X:XEB)). (15.26) -

-

Then one has

Kc ((x:IIx)I<E))>4,

Kc (t)=0

if Iltll> E9 .

(15.27)

Now

f f;

d

(Qn

')I=I f (I+Ilx+a.II To)

-!f

(X+an )

X (I + II x+ anll '°)( Qn — 4 ' )( dx )

1 1 (1 + tx + aIl'°) ' f(X+an)(l+11xII'°)(Q,,

—

+M,°(f) f IIlx+a„II'°_11x11'°I(Q,+j,"I)(dx),

^')(dx)I

148 Normal Approximation

f IIX+a,.II'°

—

Ilxll'°I (Qn+I^F'I)(dx)


+ f ((8k'/2)-.°+'+IIxII'°-I)IiI(dx)

+ f (( 8 k 1/2 ) - '°+i +IIxII'° - ')I^Y'—PI(dx)I < c I00n -(s-2)/2 ,

(15.28)

by Lemmas 14.3, 14.6 and inequalities (14.12), (14.81). Hence [see (15.24), (15.28)]

f fd(

Q —,p)I


)Zn,,n-(s-2)/2

+ f 8(x)( 1 +11x1('°)(Q,, — +G')(dx)I

(15.29)

if it is noted that

M,° (I)=

sup

(l+IIx11'°) -I I.f(x)I

XER k

<M,(f) sup ( 1 +Ilxll')(I+IIxII'°) - '<2M,(f).

(15.30)

XER k

By Corollary 11.5 one has for every c >0 f 8 (x)(l + lI xJI'°)(Q — ')(dx) ;

< 2 M,° (f)II(Q.

—

ip'), *K II+ 2 ;( 2 € : (¢'+),0),

(15.31)

where (Q,'—i').°(dx)=(l + II xll'°)(Q.

-

4 ')(dx), ,

( ,G' + ),° (dx) = ( 1 + IIxII'°)4G' + (dx).

(15.32)

149

Main Theorems

Now choose e = l6c 9p 3 n - I / z .

(15.33)

By Lemma 11.6, (Q,H*P'),-*KcII
where the maximum is over all pairs of nonnegative integral vectots a, /3 satisfying 0
(15.35)

and the integration is with respect to Lebesgue measure on R". Since

K (t)=0 for Iltll>c 9 /e=n 1 / 2 /(16p 3 ), and (

( DPR<)( t )I < (elsllxflIK(dx)
(15.36)

one has

f

I " (Qn — ' ')'

D'KK
=

IDO(Qn—')(t)Idt.

(15.37)

(11:11
Note that [see (14.60) with s=3] a is bounded above by a constant because of (15.11) [whatever the constant c one may choose in (15.11)]. This fact

was used in (15.36). Now

I

— ^G )(t)I dt

II^II fl"2

k+s—I


" D

Q^(t)— 2 n - "'/ 2Pm (it:{X))exp(-2(t,Dt)) dt m=0

A.

k+s—I

+ f D"

m/ZPm(it:{ ))exp{—Z(t,Dt>) di m=s-1

+ fA^ ^(D"Q„)(t)Idt+ fA^ I(D°5P')(t)Idt=1,+1 2 +1 3 +1 4 ,

(15.38)

150 Normal Approximation

where, writing A for the largest eigenvalue of D,

Aga

A-1/z1,

11 1 11< C14( ) ^Ik+s+z

i/z

11th < i }\A,

=

n

EIIBZjhI m (m>0),

,lmmn -I

(15.39)

i-1

B being the symmetric positive-definite matrix satisfying B 2 = D - 1 . By Lemma 14.1(v) (with s'=m) and inequality (14.21) (Corollary 14.2), one has

IIB(I<(;)^/2,

A<;,

11,,, <2m (3) m/2 n

(m—s)/2A* r^

n—(m-2)/271m <2m(3)m/2n—(s-2)/2Qn's

(m>s+1).

(15.40)

Hence, putting m = k + s + 2, we •get n (s-2)/2I/(k+., + 2 )

Aj Iltll
.

An.t

)

I

(15.41) .

If we choose c, a to be the constant c 22 (k + s + 2, k) appearing in Theorem 9.11, then we get [using (15.40)] l i < C 7lk+.s+2n -(k+s)/2 < c 17 n -( s -2) / 2 ^s

(15.42)

For this we have used Theorem 9.11 and the remark after Theorem 9.12. Also, by Lemma 9.5 (with r = m, s = m + 2) and Corollary 14.2, one has [still using (15.40)]

f IDa[n

- '°/ 2Pm

(it: {X;})exp{— I}]Idt

< c 1s n —( s -2) / 2 06 s (s— I <m < k+s— 1),

(15.43)

whereas (14.10) yields n -m /2 Pm+2< 2

m+2 p2= 2m+2

k

(1 <m<s-2).

(15.44)

Main Theorems

151

The inequality (15.43) immediately leads to

12 < c19n - (,- z) / z O*. J .

(15.45)

Next use Corollary 14.4 to get [see (15.39), (15.41)] 1 3= fAw ID a QnI


(

„^-:^,:

)I/(*+2) }( 1 +Ilill l ° I )exp{ — illtll z }dt

j II 1 II>cis l

< c21 n zz

^w,i

n,s I f I t .11 ^,s•

k+ ^ + z (

1 + II t II I°I )

exp { - u II t II z } dt (15.46)

A similar calculation using (15.44) gives 14 - fA ID°5P'I < c 23 n -(,-2) / 2A*,,.

(15.47)

The estimation of the right side of (15.34) is complete, and we have

II(Q, - 4, ')' KI
(15.48)

Finally, by Lemma 14.6, IW8 \

2E : (tP')') — Wg (2E : (4 + ),) < sup J W (x : 2c)I(4,'+),o—(4'+)rjj(' ) ,

yER"

< 2 M,(I)ll(4" ). - (4)Il < C25Mr (f )On,fn

-(:-z)/z.

(15.49)

Now use (15.48), (15.49) to get an estimate for the right side of (15.31) and then use this estimate in (15.29) to complete the proof, noting that M,0(f) < 2 M,(.l), 0 , < Q.E.D. Q.E.D. Remark. In view of relation (7.22), JId(Qw

-

P)- f (I-c)d(Q,^- P) ,

for all c ER'. Hence M,(J) in (15.13) (and in all subsequent theorems involving it) may be

replaced by M, (J)= inf M,(f—c). eER1

(15.50)

Normal Approximation

152

COROLLARY 15.2. If p3 < oo, then for all bounded Borel-measurable functions f one has

J f fd(Q.-)I
k

)p 3 n - '/ 2 +2wf (c aP 3 n -1 / 2 :4?), (15.51)

where c 26 depends only on k. Proof. The corollary follows from the above theorem if one takes r = ro =0, s=3, and notes that in the present case the condition (15.11) is unnecessary. For if p3> 0 ,,.3{i)> c1n 1/2,

then

f fd(Q,

-4i )I

<2w1 (R' )< -- w1(R k )P 3 n -1 / 2

Now (15.13) yields [using (15.8)]

If

fd (Qn

- 0 - n -'/2P,(-,t:

{x,)))I

<wf(R k )n -1/2 A.3+ 2wf ( c4P3 n-1/2 : 0+ In -h/2Pi( -0 : {X,•))I) <w^(Rk)n-1/2p3+2wf(Rk )n -x/2 11 P1( - c:

{))II +2w7

(c4 P 3 n -1

/ 2 :4j).

(15.52) Lemma 13.1 does the rest. Q.E.D. COROLLARY 15.3. If p3 < oo, then for all Bore! sets A one has IQ.(A) -4 D(A)I
sup 40((aA)''+y),

(15.53)

yER h

where 71= c4P3n -

2 (15.54)

Proof. This follows immediately from the preceding corollary if one takes f - IA (the indicator function of A) and recalls that [see (2.40)-(2.43)] wf,, (X: 71) =

'A'\A

- I W=ltaA)'(x)

(xERk, ACR k , 71> 0 ),

w^^(rl:I)= sup f wj (x+y:rl)0(dx)= sup 4((aA)+y). (15.55) yER * yERk

Q.E.D.

Main Theorems

153

Remark. Suppose A is a Borel set such that ''(A) -0. With r—r0 -0, (15.13) then yields IQ.(A)-4(A)I
yE R'`

(15.56)

4 + (( aA) +y), "

where n— c4 p3 n -1 / 2 . If, in addition, A is a set having a small diameter, one may use inequalities (11.25) of Corollary 11.5, rather than inequalities (11.26) as used in (15.31), to get, instead of the term Mo(1A)II(Q.- 4')•K.II (in (15.31)), the expression y•(IA.:c)
where A denotes the Lebesgue measure on R k , and h„ is the density of (Q„-4/)•K1 With a-0 in (15.38), (15.42), (15.45)-(15.47), one obtains .

Ih.(y)I
i-])

/2

(yERk ),

p..r

and therefore

IQ. (A) — C(A)I
-( f

-2)

/ A:,Ak(A')

4 *((aA)"+y),

+2 sup

(15.57)

yERk

with c - c9 p3 n -1 / 2 , rl — c4 p3 n -1 / 2 . We make use of (15.56) and (15.57) in Section 17 (Theorems 17.4, 17.5).

Theorem 15.1 provides sharp bounds for a wide class of functions f. This is seen, for qxample, by comparing it with the more specialized asymptotic expansions in Chapters 4 and 5, or with the general expansions for trigonometric polynomials f as provided by Theorems 9.10-9.12. Section 17 contains several applications. However note that the right side of (15.50) (Corollary 15.2) does not go to zero (as n -'oo) for every bounded II'-continuous f when Q„ is the distribution of n '/ 2 (X 1 + • • • + X) and (X,, : n> 1) is a sequence of independent and identically distributed random vectors with zero means, covariance matrices I, and finite third absolute moments p3 . The reason for this is that the bound is in terms of wf [and wf (R k )] rather than wf . Recall that [see Lemma 1.2(iii), Theorem 1.3(iii)] that f fdQ„---. f fd4 for all bounded c1-continuous f and that a bounded f is (D-continuous if and only if -

lim wf (e:1)=0

{wl ( e:0)=

f

wj (x:e)(P(dx)l.

On the other hand, it is not difficult to construct bounded 4>-continuous functions f such that wj (e does not go to zero with e. For example, let A I , the indicator function of the following Borel subset A of R'. f=

: c)

((,-I)/21

A=U

U r-2

i—i

jxeR 1 :r+? <xr+ r !

(2i+1) r

,

Normal Approximation

154

with [(r— 1)/2] denoting the integer part of (r— 1)/2. It is easy to see that w^A (e:4i)= sup 0((8A)`+y)=1, yERk

W1A (E:4))=4D l (aA) ) <(27l)-1/2 ^ rexp t —

r2 11

'

1

,-

I

J E

(t>0).

It would be ideal if one could replace wj in the bound (15.51) by w . Unfortunately we are unable to do this. The situation is partially salvaged by the next theorem, which provides an effective bound for every bounded 0-continuous f. However a price is paid for greater generality. For "nice" functions and sets there is a loss in precision. On the other hand, apart from its greater generality, this theorem is more useful than Theorem 15.1 for estimating tail probabilities (see Section 17). !

THEOREM 15.4. There exist positive constants c; (i = 1, 2, 3) depending only on k and s such that if A,,,,(i) < cin t

s -2) / 2 (15.58)

holds for some integer s> 3, then for every Bore/-measurable function f satisfying Mr(f)<00

(15.59)

for some integer r, 0 < r < s, one has

f fd(Q„

—

'G)I
+),

(15.60)

where s-2

0 = I n (s 2)/2Pm( -

Y

m-0

-

-

4):

(x.))'

and 'q=ciP3([logn]+ 1)n -1 / z ,

(15.61)

where [logn] is the integer part of logn. The quantities 0 (3) , On.s are defined by (15.6) and (15.7), respectively.

Main Theorems

155

Proof We continue to use the same notation as in the proof of Theorem 15.1. Define , ', ¢" as in (15.15). As in (15.24), one has J fd(QQ - 4i)I
- c ,-2) / z +Iffd(Q '—O')I. ,,

(15.62)

Choose a probability measure K' on R k satisfying K'((x:IIxII<1))=1,

IDCK'(t)I
(15.63)

Such a choice is possible by Corollary 10.4. For E >0 and a positive integer p define K' (B)=K'(e - 'B)

(B E J^ k ), (15.64)

and note that KK. P ({x:llxll
By Corollary 11.2, with µ = Q,,, v = 4', KK = KK , P , and f=4' one has f J^,d(Q^—^'')I
(15.66)

where, by definition of y [see (11.8)] and of M,(J) [see (15.4)] y(f,:p€)<Mr(f)( 1 +IIxII +Ilan ,, + PE) r i(Q, - 4i')'Ki.PI(dx)

<M,.(f) f (C3o+ 2'llxll'+P'E')I(Q, - 41')*KK. P I(dx) < C31Mr(f)[

la+^^ m

ax

IID a (Qn

+p'€' max

Ia+flk
—



II D ° ( —

(15.67)

If r=0, then the first inequality in (15.67) holds because 4'(R k )=V(R k ) =1 [see (7..22) and (11.13)]. The last inequality follows from Lemma 11.6. In view of (15.63), one has

IID° (Q' ^F')'D"K.. P II1 -


I-6I

( ID ° (Q, — i')(t)^exp(

— PIIEt11 112 )

dt.

(15.68)

156 Normal Approximation

Now choose e=c33p3n-1/2,

p=[logn]+1,

(15.69)

and write

f

(15.70)



where, as in (15.38)-(15.47),

I1 <

^,:^

34 k+s-1

f ID a [n - '"/ ZPm (it:{x;})exp{-2(t,Dt)}]Idt

I2- m—s-1

< c n—cs-2)/2Q. n.5

3s

13 < c36n -( s - z ) / 2 A*n,s9

la'= f

I(D ° '')(t)Idt
Rk

,-z>/2 ^.

Is =I(D"Qn)(t)Iexp( — pIIE'II 1/2 )dt { II:II > n" 2 /(16p3) )


(1III>n'/ 2 /<^^))

exp(—plfEtlll/2)di.

(15.71)

The last inequality follows from Corollary 14.4 if one notes that 1 ID°Q„(t)I<EIIn- / 2 (Z1+... +Z,,)II 1 ° 1

<(Elln-1/2(Z1+... +Z.n)II"' I-I/mI, (15.72) ,)

for every even integer m'> Ial, and that for an even integer m', E lI n -1 / 2(Z I + • • • + Z„)II"') may be expressed in terms of ordinary (as opposed to absolute) moments that are estimated by putting t = 0 in

Main Theorems

157

Corollary 14.4. Also,

{IIII>n'/ 2 /(16v,)) xp { —

PIIEtII

1/z

} dt

=E ke xp{—PIItII112}dt (II111>c33/ 16 )

=C 39C

—k ^ 00

u k-I exp{—pu 1 / 2 }du

C33/ 16 2 =2C 391/EP)

kf

2k—I

w


expt

P(c33/16)i/2

exp{—v)dv

—v

(15.73)

}dv.

2

Now choose 2

s 2 2 )

C33 =64 (k+

(15.74)

to get (s 2)/2 < c 42 n IS < c4l(n1/2log2n ) kn -

-

-

-

(3 - 2) / 2

0 ,.

(15.75)

The last inequality in (15.75) derives from [see (14.7), (14.10), and (15.58)] n (s+1)/2

n -1/2c43
n ^I f IIY3II -

2

j-1

n


IIljII

s+

^ s0* . .s

(15.76)

J-1

Hence II D ° (Q^—^^)'DQK^.P^I1

c44(PE)II'In-cs-2)/2pn.s'

(15.77)

which leads to Y(fa :pE )
k+r+l

)n

—(s-2) / 2

^^ s .

(15.78)

158 Normal Approximation

Finally, by Lemma 14.6,

w^w (2pE:(¢') + )=w/ (2pe:0')+

f w/^(x:2pe)^(^'^)+-0+)(dx)

+ f (wf. (x : 2pt) — wf (x : 2p))¢ + (dx) <w1 (2pe:i + )+M,(f) f

(I+(IlxI +IIanII + 2pe) r )IO' — PI(dx)

2 + I.1 Q1(x : pc)((4'") — 4')(dx) <(Jf(2pE:1^if)

+C46 Mr(f)( 1 +(pE) r )n -(5-2)/2A*

(15.79) In the last step we also used the inequality w1 (x:2pc)<2M,(f)(1+(IIxII+2pc) r )

(r>0, xER k ). (15.80)

The proof is now complete by (15.78), (15.79), (15.66), and (15.62) if one writes rl = 2pe. Q.E.D. COROLLARY 15.5 Under the hypothesis of Theorem 15.4, one has

f fd(Q.

-1)I

<e47(s , k)Mr(J )( I+.l

k+r+' )(P3n -1/2 +p,n -cs-2)/2

+wf( 71: b),

)

(15.81)

where, as in (15.61),

71=c p 3 ([logn]+ 1)n" 2 Proof Proof. From the expression (14.74) one has IPm( — $: (X.))(x)l
(xER k , 1 <m<s-2).

(15.82)

Main Theorems

159

For, if I vi I > 3, I°_ 1 (I v ; I - 2) = m, then (recalling that p 2 = k) IX,,. .. X ,r ^ < c49(m k)PI ►,I

.

..

,

pk,I (II-2)/m

.


(15.83)

(1<m<s-2).

Now n

n-m/2Pm+2=n _I



n-m/2( f

+ f

11xj1Im+2

n


=pan- ► /2+psn-(s-2)/2

J i1xills) (15.84)

(1 <m<s-2).

Therefore 11 n -'"/2Pm

(- 4:

(x,) )H < c5 ► (s , k)(Psn -

► /2 + psn

-(s-2)/2)

(I <m<s-2).

(15.85)

Also, using (15.80), (15.82), and (15.84), we get wl(fl' I n-m/2Pm( --b : {)C►,})I) < c52(s , k)( 1 + n r )M,, (f)(Psn -

i/z + pan -(s-2)/2) (1 < m <s-2).

(15.86)

Now (15.81) follows from (15.60) if one uses (15.85) and (15.86), as well as the first inequality in (15.8). Q.E.D. Remark. The stipulation (15,58) [which is the same as (15.11)] is redundant when one has r-0 in Theorem 15.4 or Corollary 15.5. The proof of this assertion in the Corollary 15.5 case is essentially contained in the proof of Corollary 15.2. In the Theorem 15.4 case one needs the additional fact [see (7.22)] P.(-0:(y,))(R k )-0

(1<m<s-2).

(15.87)



160

Normal Approximation

16. NORMALIZATION The main theorems in the last section have been stated for independent random vectors X 1 ,...,X. satisfying (standard normalization) n EXj -0

n ' I Cov(X^)=1

(1 <j< n),

-

(16.1)

j -1

where I is the k x k identity matrix. This was done for the sake of simplicity. It is a relatively minor matter to extend all results of the last section to independent random vectors X 1 ,.. .,X„ satisfying EXj =O

(1 < j
n '

Cov(X^)-. V,

-

j-

(16.2)

1

where V is an arbitrary symmetric, positive-definite matrix. One could also take the mean vectors in (16.2) to be arbitrary. But since this is taken care of merely by changing the integrand f to a translate of it, we assume (16.2) throughout this section. Let Q,, denote the distribution of n - '/ 2 (X I + • • • + X,,). If f is integrable with respect to Qn and (D o. V , then by changing variables x-+ Tx one has f fd(Q, -41o.v)= f f°T ^d(G,-Z),

(16.3)

where T is the symmetric positive-definite matrix satisfying T

2 =V - ',

(16.4)

and G. is the distribution of n - '/ 2(TX 1 + • - - + TX.). Note that the random vectors TX 1 ,..., TX,, satisfy standard normalization, that is, E(TX1)=0

(1 'j< n),

n n - ' Y, Cov(TXj)=1.

(16.5)

j-1

Hence one may use the results of the last section to estimate the right side of (16.3) in terms of the moments - n

•r,-n ' I E^ITXj II' -

(r>0),

(16.6)

j-1

and M,(f°T - ') and w8r or wf, T -1, where (I + IIxIrr") - '(fo T - ')(x) Sr(x)= (foT

- ')(x)

if r>O, if r=0.

(16.7)

Normalization

161

Since T is easily computable, we may leave things at this stage. If one would like the bound to involve moments of X D 's and not those of TX's, then the simple inequality (r>0)

Tr
(16.8)

may be used. Here A is the smallest eigenvalue of V. Note that TII=A -1/2 ,

IIT

-I

II_A'/ 2 ,

(16.9)

where A is the largest eigenvalue of V. We now rewrite (or extend) the results of Section 15 in a series of theorems and corollaries, assuming that the n independent random vectors X 1 ,. . . , X,, satisfy (16.2). THEOREM 16.1. There exist positive constants c ; (i= 1,1,2,3,4) depending only on k and s such that if n (16.10) IITX^II'( 2 /3). 1 / 2 )

holds for some integer s > 3, then for every real-valued, Borel-measurable function f satisfying (16.11) M,(f)
J fd(Q,,—(Do.v)I <M,(f°T-1)(

C2T3n-1/2+C3T.,n—(s-2)/2)

g;( c4T3n-1/2: 4,

+2w

a)

G max (1,A r / 2 }Mr (f)(c 2X -3 / 2 p j n -1 / 2 + c 3 A — s/2

+ 2wwr(c4X -3/2

p3n -1/2: 4 r.).

p, n

_(s_2)/2)

(16.12)

Proof. The first inequality in (16.12) follows from (16.3) and Theorem

15.1, as well as the relations (15.82), (15.84), and (15.86). Also, one has M,(f°T ► )= —

sup (l+IIxtI') —I If(T —tx)I

xER 1'

sup (1+IIT-1xIr)—Ilf(T—'x)I)t

sup (1+IIT—IXII')(l+11x11')-1) <( xER k xERk

=M,(f)( sup ( 1 +11T —I xll')(l+11x11') — ') xER k

<M (f)max(1,IIT - 'll'}=M,(f)max{l,A'/ 2 ). '

(16.13)

162

Normal Approximation

The second inequality in (16.12) follows from (16.9) and (16.13). Q.E.D. The constants c's below depend only on s and k.

COROLLARY 16.2. If P3< oo, then for all bounded, Borel-measurable functions f one has

f fd (Qn-0o v)I < cswf(Rk )T3n-1/2 + 2wf r _,(c4 T 3 n - 1/2. D)

< c s w/ (R k )'r 3 n -1 / 2 +2wj (c 4 A h / 2'r 3 n -h / 2 :4,o v ) < c sw1 (R k )A -3/2 p3n -1/2

+2wj ( C4A'/2A - 3/2p3n - '/2:(Do. v)•

(16.14)

Proof The first inequality follows from Theorem 16.1 on taking r = ro = 0, s=3, and recalling Mo(f°T -l )=wf r,_,(R k )=wf(R* ).

(16.15)

To get the second inequality in (16.14) note that wf, T _,(x:e)=suP{I.f(T - ^y)_f(T - 'z)I:y,zEB(x:e)} <sup {If(y')-f(z')I:y',z'EB(T - 'x: IIT - 'IIE))

=wf (T -1x:IIT -1 IIe),

(16.16)

so that c1) sup sup f wi(T—'(x+y):II T-111e)4(dx) yER`

= sup f w (T lx+y :IIT 'IIE)(P(dx) !

-

,

-

y'ER k

= sup fwi(z+y':II T —h IIE)io.v(dz) y'ERk

-wf (II T - 'Il e : t' , v )=wf (A" 2e:Io, v ).

(16.17)

The last inequality in (16.14) now follows from (16.8). Finally, note that

Normalization

163

the condition (16.10) may be replaced by p 3 < oo as in the proof of Corollary 15.2. Q.E.D. COROLLARY 16.3. If P3 < 00, then for all Bore! sets A one has IQ.(A)-4o.v(A)I < c 5 T 3 n -1 / 2 +2

sup

(bo v((aA)n'+y)

yER k


/ 2 p 3 n -i / 2 +2 sup 4i0.v ((aA)''+y), (16.18) yER 4

where 1J=c4T3n-1/2
q,=IIT-'IIr1=A'/1q. (16.19)

Proof. The proof follows from Corollary 16.2 and the relations (15.55). Also note that IA O T -' = I TA . Q.E.D.

Before stating the analog of Theorem 15.4, we note that if we start with the finite signed measure Pm( -40 o,v: {X„}), where Xv is the average of the with cumulants of XD 's and make the transformation x-+Tx on R k , then the induced signed measure is Pm (-(b: {XY )), where j,^, is the average of the with cumulants of TX's, 1 < j < n. To see this, observe that the FourierStieltjes transform of Pm (- 4 o, v : (x)) is Pm (-4 ov :{aG})(t)=Pm (it:{Xv))exp(--L)

(IER k ), (16.20)

and that of the induced signed measure [see Theorem 5.1(v)] is t-+Pm (iTt,(X ))exp{-iIItII2) (tER' ).

(16.21)

Now look at the expression (7.3) to conclude Pm (iTt,(X.))=P.(it:( ))

(tER k ).

(16.22)

THEOREM 16.4. There exist positive constants c 1 , c; (i = 2,3) depending only on k and s such that under the hypothesis of Theorem 16.1 one has s-2

I

fd Qn—m-0 7. n-m/ZP.(-410 v. {X.}) )

< cZ max ( 1, Ar/ 2 )Mr (f)( 1 + rl k+r+ i)X-,/2 pfn -(s-i)/2 I s-2 + +Zif (A'/ 271: (1 n -m/ZPm( - to.v: {X,))) ), m-0

(16.23)

164

Normal Approximation

where ° C31

-

3 /2 P3([logn] + I)n -) /`.

(16.24)

Proof. This is an immediate consequence of Theorem 15.4 and the relation

f fd(Qn

s-2

m=0

n— m/2Pm(— ,DO.V :(x})

=f

l

s-2

(f°T-))d Gn

—

2

m-0

(16.25)

n -m/2Pm( — $:(?(,)) 1 //

which follows from the discussion preceding the statement of the present theorem. One of course also uses (16.8), (16.9), (16.13), and (16.16), as in the proof of Theorem 16.1. Q.E.D. COROLLARY 16.5. Under the hypothesis of Theorem 16.4 one has

f fd(

Qn

- -to,v )I
)

x(A 3 /2P3n - )/ 2+X- 3 /2p,n -(s-2)/2) -

(16.26) where is defined by (16.24). Proof. This follows from Corollary 15.5 if one uses (16.3), (16.8), (16.9), (16.13), and (16.16). Q.E.D. Remark We have already pointed out (see Corollaries 16.2, 16.3) that the stipulation (16.10) is redundant in Theorem 16.1 in the case r-0. This is also true for Theorem 16.4 and Corollary 16.5, and the proof is also essentially the same. Note that (16.10) may be replaced by the slightly more stringent but simpler condition T,

< ctn ' (

-

2) '2

(16.27)

or by (16.28)

17. SOME APPLICATIONS

The present section is devoted to some important applications of the main theorems of Sections 15 and 16. We continue to use the same notation as in Section 16. Thus X 1 ,... ,X,, are n independent random vectors with

Some Applications

165

values in R k satisfying n

(1 < j < n),

EXj =O

n -1 Cov(Xj )= V,

(17.1)

j-1

where V is a symmetric, positive-definite matrix. Also, T is the symmetric positive-definite matrix satisfying (16.4), and we write n

n

pr=n -I EIIXjII",

j

— I

T,

=n -1

E EIITXjII r (r>0).

(17.2)

j-1

Recall that A, A are the smallest and largest eigenvalues of V, respectively, and Qn is the distribution of n -I / 2 (XI+ • • - +X„). We define the class 9 (d: µ) as the class of all Bore! subsets A of R" satisfying sup µ((aA)`+y)<de'

(e>0),

(17.3)

yER*

for a given pair of positive numbers a, d, and a measure µ. THEOREM 17.1. There exist positive constants c 1 and c 2 depending only on k such that if p 3 < oo, then

sup

AE(t';(d:4'o.v)

I Q.(A) -4 0, (A)
< c lX - 3/2 p3n -1 /2 + 2 d (c 2 A I /2A - 3/2 p3n - I /2)°. (17.4) Proof This follows immediately from Corollary 16.3. Q.E.D. COROLLARY 17.2. There exists a constant c 3 depending only on k such that if p 3 < oo, then

sup IQ. (C) -00, (C)I
1 /2
(17.5)

CEe

where

a is the class of all Borel-measurable convex subsets of Rk .

Proof Let Gn be the distribution of n -1 / 2 (TX I + • • • + TX„). Applying Theorem 17.1 to the random vectors TX,,..., TX,, and using Corollary 3.2 [or inequality (13.42)1, one has

sup IQn(C)— D0.v(C)I= sup IG.(C)- 4 (C)I

CEC

CEe


166

Normal Approximation

where d= 25/2

r((k + 1)/2)

(17.7)

F(k/2)

Note that G' is invariant under translation, as well as nonsingular, linear transformations. Q.E.D. COROLLARY 17.3. Let k=2. Denote by 61 (1) the class of all Bore! subsets of R 2 each having a boundary contained in some rectifiable curve of length not exceeding 1. There are absolute constants c 4 , c5 such that if p 3 < oo, then sup JQ.(D)-1Po.v(D)I
DE9(t)

+c5(A- 11 +a-1/2)A'/2A-3/2p3n -1 /2. (17.8) Proof From the estimate (2.60) we get

4i o. V ((8D )`) <(27r) - '( Det V) V 2 (41r1 + 87rc 2 )
(17.9)

(E <._-_2 ),

since Det V > A 2 It is enough to consider E
.

There are Borel subsets A of R k for which sup i((8A)`+y) 0)

(17.10)

yaR k

for some positive constants c6 and a, a> 1. Examples of such sets are affine subspaces of dimensions k' < k — I (and their subsets and complements) and many other manifolds of dimension k'< k —1, for which a = k — k'. Below we assume that V= I merely to avoid notational complexity. THEOREM 17.4. Let V=I. Assume that p, < oo for some integer s)'3. If A is a Borel set satisfying (17.10) for some a> 1, then IQ.(A)

-

1b(A.)I
+c8(P3n -1/2 ) a ( 1 +

I

m-1

n- m /2PM+21og3,n/2n),

(17.11)

Some Applications 167

where c7 depends only on s and k, and cg depends only on s, k, a, and c 6 . In particular, if X 1 ,..., X. are identically distributed, then I QQ(A)

-,

(17.12)

b(A)I = O(n-a/2)

provided that s> a+2. Proof. First assume that 4(A)=0 or 4(A)=1. In this case P.(-4: {X,,)) (A)=0 for all m, 1 <m <s-2, because of (7.22). Hence (15.56) holds, and it remains to show that fl -m/2 IP,n(-^ {x.})(x)Idx (aA)"+y


"'/ 2p

n+2 1og'

- /2

(1 <m <s-2).

n)

This follows from (15.82) and the estimate

f

°

11xHI 3 in$(x)dx < f

(aA) +y

(2alogn) 3 m/ 20(x)dr

(aA)'+y

3rn

/

o(x)dx

•

+ (11X11>(2alogn)1/3)IIxI^

< c 6 (2a logn)3m/2(p3n -1/2)a + n -a/2

We now show that 4(A)=0 or 1. For this first observe that for all y, z E R k (y # z), one has A +y\A +zCCl(AII' - `°I)+y, so that, by (17.10), I4(A +y)-t(A +z)I < c6IIY - zII a .

In other words, the function f(y)-4(A+y) is Lipschitzian of order a> 1. It follows that f is differentiable and has a zero differential on all of R"`. Hence f is a constant on R k ; that is, b(A+y)=O(A)

(17.13)

(yER k ).

Define the measure u on R k by µ(B)=JBIA(x)dx=Xk(BnA)

(BE

),

168 Normal Approximation

where IA is the indicator function of A and Ak is Lebesgue measure on R k . We show that if (17.13) holds, then is translation invariant. This would imply that µ = c11k for some c, 0 < c < oo, which is possible only if IA = 0 (almost everywhere) or IA =1 (almost everywhere), and the proof of the theorem is complete. If B is a bounded Borel set, then for every E >0 there exist c; F_ R', y, ER k , 1 < i < m, say, such that II 'B — = 1 c(x +y) 1
—

µ(B)J=I f'8_ (x)dx — fIB(x)dx f I.(x)dx— f 1,(x)dxI A

A+y

m

< fIs (x)dx— A+y

c;4)(x+y1)dx

E fA+y

m

m

+ 2 fcc¢(x+y,)dx— E f ci4(x+y,)dx i^l '^

+ y

i^l '^

m

+ 2 f c;4(x+y,)dx— f IB (x)dx A

A

<2+0+2=E

for allyER k . Hence µ(B—y)=µ(B) for allyER*. Q.E.D. For the special case (of Theorem 17.4) when A is an affine subspace of dimension k' < k —1, the hypothesis p, < oo may be relaxed. Indeed, one has THEOREM 17.5. There exists a positive number c lo depending only on k such that if the hypothesis of Theorem 17.4 holds with s = 3, then for every affine subspace A (of R'`) of dimension k' < k — 1, one has Q,, (A) < C1o(psn -Z)ck- k >. tSee Wiener [11, Theorem 9, p. 98.



(17.14)

Some Applications 169

Proof An affine subspace A of dimension k' < k has a representation A ={xER k :<11 ,x>=c,,1
and note that n

(1 <j
EUj =O



n

CovUj =1,

j-1

n

n n 7, EIIUj lj 3 < n -1 2 EIIXj II 3 =P 3 .

denote the Let Qn denote the distribution of n -I / 2 ^i_,Uj and let standard normal distribution on R k-k '. Now let c be the vector in Rk-k' whose ith coordinate is c. (1 < i < k — k'). Then Qn(A)=Qn((c)) ID(A)=4)({c})=0, ,

and using (15.57) to Qn , , and {c) (in place of Qn , 4), and A, respectively) for s=3,

Q.( A ) = I Q.( A ) -4 (A)I =I Q,((c))—^((c))I < c l0 n -1 / 2P 3 (P 3 n -t / 2 ) k-k +2 sup >G + ((a {c+y))") k'

c10(n -1/2 P3)

k+1-k '

y E Rk-

+c', o (P 3 n -I / 2 ) k-k ,

where ¢= +n -t / ZP t (— : {X„}), , denoting the average of the with

cumulants of Uj 's. Q.E.D. Remark. It is fairly straightforward to extend the assertion (17.12) to a sequence (X,, : n> 1) of independent random vectors for which n

linminfX.>0,

supn -1 7, EIIX^II'
Here A,,, is the smallest eigenvalue of V. _— n has n -'

n

j_1

_, Cov(X). Note that with T. =

EliT„X1 lr'<x,^ " 2 (n - ' ± EllX^ii'), j -1 j-1

(17.15) one (17.16)

170 Normal Approximation

so that applying (17.11) to random vectors T„X^, I < j< n, one arrives at (17.12), provided that s> a + 2 and (17.15) holds. Similarly, in Corollaries 17.2 and 17.3 the error bounds are O (n —1 / 2) if (17.15) holds with s-3. In the same manner, orders of magnitude of errors for a sequence (X„ : n> I) satisfying (17.15) may be obtained from the remaining theorems and corollaries of this section.

For an application of a different nature, consider a Bbrel set A and define the function f by f(x)=(1+d 3 (0,aA))I4 .(x)

(xER k ),

(17.17)

where

A,_

J

if OERk\A, R k \A if OEA,

A

(17.18)

and d(0, aA) is the euclidean distance of aA from the origin 0. Note that M3 (f)<1

(17.19)

(s>0).

Defining g by (15.14) with r=s, one has, for IlziI <2c
<(1+IIx+Y11:

,

)

-

'If(x+Y+z)-f(x+Y)I

+If(x+y+z)II(l+Ilx+Yll'°) - ► - (l+llx+y+zIl'°) -t ) <(1+ilx+ll) - 'I.f(x+y+z)—f(x+Y)I+c 12 (k, ․)e

<(1 + IIx+y113°)-`(1 +d° (0 , a4 )) 1(aA)(x+Y)+c12(k , ․)c <(1 +(d(O,aA)-2c)'°) - '(1 + d' (0,8A))I (aA) =•(x+y)+c 12 (k, ․)c < c 13 (k, ․)I(aA )''(x+y)+c I2 (k, ․)e,

(17.20)

where c13(k, ․) =

sup (1+(b+c 11 (k, ․))')(l+b'°) ^. -

(17.21)

b>O

Thus in this case w8 (2e: I,o) < c 13 (k, ․) sup 4 ((aA) 2 i+y)+e 14 (k, ․)e,

(17.22)

c14(k, ․ )—c12(k, ․)4',o(R"` ).

(17.23)

yER k

where

Some Applications 171

Specializing to convex sets, we can prove THEOREM 17.6. There exist positive constants c 15 , c 16, c 17 depending only on k and s such that if V = I and ps < c 15 n

( s -2) / 2 (17.24)

for some integer s> 3, then sup ( 1 +d 3 ( 0, aC))IQ.(C) — b(C)I < c16Psn - ' /2 +ci7P:n -t:-2)/2 .

CE@

(17.25)

Proof. Let C be a Borel-measurable convex set. Replacing A by C in (17.17) and using (17.22) in Theorem 15.1 (with r=s), one has (1+d'(U,8C))IQ.(C)-41(C)j

=^ ffd(Q^-0)I s-2

m/2I (' fdp (—: f fd(Q,, — ^)I + I n{X,))I m-1 J c18Pan-(s-2)/2+ c 1 9w (2E : (Y

+ )r°)

s-2

+ 2 n - m/ 2 I f fdpp (-4: (x.))I,

(17.26)

m-1

where e=c 10P 3 n - '/ 2 . Now, by (15.82) and (15.84),

1

(1 + 1I X II^ ° )I Pm ( - -0 : {Xr})(X)I dX 4 C21Pm+2+ n-m/2Pm+2
Hence (17.26) reduces to (I +d 3 (0, aC)))Q,,(C) -4 (C)I
+ C 1$WB (2E : 0to).

(17.28)

After (17.22) and Corollary 3.2 are used in (17.28) the proof is complete.

Q.E.D.



172

Normal Approximation

Theorem 17.6 leads to the so-called global and mean central limit theorems in R'. COROLLARY 17.7. Under the hypothesis of Theorem 17.6 with k = I one has

sup (1+IxI')IF,(x) - 'D( x)I4c23P3n-1/2+c24Pn-(s-2)/2 (17.29) xER'

where F.(•) is the distribution function of n - " 2 (X 1 + • • - + X„) and ID(•) is the standard normal distribution function. It follows that `

I1 F„ — 0 11 P -(

I/P

fR 1F, (x) — ID(x)I P dx

< c25(P , ․ )(c23P3n -1 /2

+ c24P:n -(s-2)/2),

(17.30)

for all p> 1/s. Proof. The inequality (17.29) follows from Theorem 17.6 on taking k=1, C=(—oo,x], xER'. Inequality (17.30) is immediate from this. Q.E.D. Remark. The validity of (17.30) for p>I may be proved without the assumption (17.24). Let P3 < co. Then IF, (x) — $(x)I <(1+IxI') -' c26P3n -1/2 <2(1+x 2 ) -1 c 26 P 3 n -1 / 2 (xER')

(17.31)

if (17.24) holds, that is, if P

3


for a suitable positive number c 27 . However x 2 ^F„ (x) —,D(x)j < x 2 (F„ (x) +@(x))

2g^(dy)+

y

x 2 IF. (x) —4 (x)l _ x21(1— F^ (x)) — (1


Y24(dy)<2

f

(x<0),

(IYI>IxD)

(IYI>IXI) -

4i(x))I

y 2Q^(4Y)+ f

(Irl>x}

IF„(x)-4b(x)t<1

(IYI>x)

Y 24(dr)<2 (x>0), (17.33)

(xER 1 ),

so that the inequality (1+x2)IF„(x)-0(x)I<3

(x ER')

(17.34)

Some Applications 173

holds whenever P 2 = I. Now IF. (x)

-

4b( x)I
-

1/Z (x

ER')

(17.35)

holds whenever P3< if c 28 =max(2c 26 ,3/c 27 }. This is true since (17.35) holds because of (17.31), provided that (17.32) holds. If (17.32) is violated, then (17.35) holds by virtue of (17.34).

The final application of Theorem 15.1 is to Lipschitzian functions. THEOREM 17.8. Let (17.1) hold with V = I. There exist positive constants c29 c30 c31 depending only on r, k, and s such that if Ps < c 29 n (s 2) / 2 (17.36) ,

,

-

for some integer s > 3, then for all f satisfying

Mr(f)
f

fd(Q,, -4 )I
-I/2

+psn -(s-2)/2

)

+ c30d (c3IP3n- I/2)°

(17.38)

If r = 0, (17.36) may be replaced by p 3 < oo . Proof By Theorem 16.1 (with V= 1= T) one has J fd(Q -0 )I
-1/2

+psn -(s-2)/2

)

+ 2; (c33P3 n - 1/2: (Dro ).

(17.39)

Now if r=0, then r0 =0, g = f, and wJ (E : t) Z d(2E)' and we are done by Corollary 15.2. If r>0, then I g(x)

—

g(Y)I

=I(I+IIx11') 'f(x) (l+IlYll') 'f(Y) -

-

-

<(1+IIx11') 'if(x) f(Y)I+If(Y)[(I+Ilxll r ) -'— (I+IIYII -

-

') - ']



'max (IIxII'-',IIYII'-I)

+I x11' -' )(IIxII + 2 E ) r-12 E

< d (2E)"+2 rrMr (f)(1 + IIxII r ) '-' (IIx11' - I + (2E)' - I )E
(IIx y11 <2E).

(17.40)

174 Normal Approximation

Letting E = c33P3n -' /Z , one has c < c 34 by (14.60). Hence (17.38) follows from (17.39) and (17.40). Q.E.D. We now turn to some applications of Theorem 15.4. Specialize Theorem 15.4 to the case r=0, s = 3, f = I4 , where A is a Borel subset of R k. If V= I and p 3 < oo, then one can show that Q,, (A)

—

Z(A) < c35( 1 +r^ k+' )n -I /2A* 3 +*(A"\A),

(17.41)

where, as in (15.61), (17.42)

71—C3P3 ([logn]+1)n-'/2.

The one-sided inequality (17.41) follows if one uses Corollary 11.3 instead of Corollary 11.2 [see (15166)] in the proof of Theorem 15.4. The rest of the proof need not be altered. Let 8„=max

{rl,c35(1

+')n - '/ 2 0

.3 }.

(17.43)

THEOREM 17.9. Let V=I, p 3 < oo. Then the Prokhorov distance d p and the bounded Lipschitzian distance d BL between Q and 4 are estimated by d,, (Q,4)

d8c(Q.,4)
where c 36 depends only on k. Proof. The assertion concerning dBL follows immediately from Theorem 17.8 with r=0 [see (2.51) for the definition of this distance]. As to dp , note that for any two probability measures G,, G 2 on R k one has dp (G i , G 2 ) - inf { e > 0 : G ! (A) < G 2 (A `) + e and G2 (A)
=inf( >0:G 1 (F)< G2 (F`)+e, G 2 (F) < G I (F )+e for all closed F)

(17.45)

since A ` = (closure of A) for all A C R k and all c > 0. Also, d,(G 3 ,G 2 )=inf (t>0: G 1 (F) < G 2 (F

)+c

for all closed F) (17.46)

since, given G 1 (F) < G2 (F`) + e for all closed F and some c > 0, one obtains 1— G 1 (F` )= G 1 (R"`\F`) < G 2 ((R"\F` f)+e
(17.47)

Some Applications 175

for all closed sets F, using

(17.48)

R k \FJ(R"`\F` )(. Now (17.41) and (17.43) yield

Q„(A)
(A E s"). (17.49)

Together with (17.46) this gives the first inequality in (17.44). Q.E.D. Recall that A, 3