Asymptotics: particles, processes and inverse problems. Festschrift for Piet Groeneboom

Institute of Mathematical Statistics LECTURE NOTES–MONOGRAPH SERIES Volume 55

Asymptotics: Particles, Processes and Inverse Problems Festschrift for Piet Groeneboom

Eric A. Cator, Geurt Jongbloed, Cor Kraaikamp, Hendrik P. Lopuha¨ a, Jon A. Wellner, Editors

Institute of Mathematical Statistics Beachwood, Ohio, USA

Institute of Mathematical Statistics Lecture Notes–Monograph Series

Series Editor: R. A. Vitale

The production of the Institute of Mathematical Statistics Lecture Notes–Monograph Series is managed by the IMS Office: Jiayang Sun, Treasurer and Elyse Gustafson, Executive Director.

Library of Congress Control Number: 2007927089 International Standard Book Number (13): 978-0-940600-71-3 International Standard Book Number (10): 0-940600-71-4 International Standard Serial Number: 0749-2170 c 2007 Institute of Mathematical Statistics Copyright
All rights reserved Printed in Lithuania

Contents Preface Eric Cator, Geurt Jongbloed, Cor Kraaikamp, Rik Lopuha¨ a and Jon Wellner . . . . .

v

Curriculum Vitae of Piet Groeneboom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

vii

List of publications of Piet Groeneboom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii List of Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xi

A Kiefer–Wolfowitz theorem for convex densities Fadoua Balabdaoui and Jon A. Wellner . . . . . . . . . . . . . . . . . . . . . . . . . .

1

Model selection for Poisson processes Lucien Birg´ e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

32

Scale space consistency of piecewise constant least squares estimators – another look at the regressogram Leif Boysen, Volkmar Liebscher, Axel Munk and Olaf Wittich . . . . . . . . . . . . . .

65

Confidence bands for convex median curves using sign-tests Lutz D¨ umbgen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

85

Marshall’s lemma for convex density estimation Lutz D¨ umbgen, Kaspar Rufibach and Jon A. Wellner . . . . . . . . . . . . . . . . . . . 101 Escape of mass in zero-range processes with random rates Pablo A. Ferrari and Valentin V. Sisko . . . . . . . . . . . . . . . . . . . . . . . . . . 108 On non-asymptotic bounds for estimation in generalized linear models with highly correlated design Sara A. van de Geer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Better Bell inequalities (passion at a distance) Richard D. Gill . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Asymptotic oracle properties of SCAD-penalized least squares estimators Jian Huang and Huiliang Xie . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 Critical scaling of stochastic epidemic models Steven P. Lalley . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 Additive isotone regression Enno Mammen and Kyusang Yu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 A note on Talagrand’s convex hull concentration inequality David Pollard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 A growth model in multiple dimensions and the height of a random partial order Timo Sepp¨ al¨ ainen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 Empirical processes indexed by estimated functions Aad W. van der Vaart and Jon A. Wellner . . . . . . . . . . . . . . . . . . . . . . . . 234

iii

Preface In September 2006, Piet Groeneboom officially retired as professor of statistics at Delft University of Technology and the Vrije Universiteit in Amsterdam. He did so by delivering his farewell lecture ‘Summa Cogitatio’ ([42] in Piet’s publication list) in the Aula of the university in Delft. To celebrate Piet’s impressive contributions to statistics and probability, the workshop ‘Asymptotics: particles, processes and inverse problems’ was held from July 10 until July 14, 2006, at the Lorentz Center in Leiden. Many leading researchers in the fields of probability and statistics gave talks at this workshop, and it became a memorable event for all who attended, including the organizers and Piet himself. This volume serves as a Festschrift for Piet Groeneboom. It contains papers that were presented at the workshop as well as some other contributions, and it represents the state of the art in the areas in statistics and probability where Piet has been (and still is) most active. Furthermore, a short CV of Piet Groeneboom and a list of his publications are included. Eric Cator Geurt Jongbloed Cor Kraaikamp Rik Lopuha¨ a Delft Institute of Applied Mathematics Faculty of Electrical Engineering, Mathematics and Computer Science Delft University of Technology The Netherlands Jon Wellner Department of Statistics University of Washington, Seattle USA

v

Piet in a characteristic pose. Amsterdam, 2003.

Curriculum Vitae of Piet Groeneboom Born: Citizenship: Dissertation:

September 24, 1941, The Hague The Netherlands Large deviations and asymptotic efficiencies 1979, Vrije Universiteit, Amsterdam. supervisor: J. Oosterhoff.

Professional Career: Mathematical Centre (MC) University of Washington, Seattle University of Amsterdam Delft University of Technology Stanford University Universit´e Paris VI University of Washington, Seattle

Researcher and consultant Visiting assistant professor Professor of statistics Professor of statistics Visiting professor Visiting professor Visiting professor

University of Washington, Seattle Vrije Universiteit Amsterdam Institut Henri Poincar´e, Paris

Affiliate professor Professor of statistics Visiting professor

1973–1984 1979–1981 1984–1988 1988–2006 1990 1994 1998, 1999, 2006 1999– 2000–2006 2001

Miscellanea: Rollo Davidson prize 1985, Cambridge UK. Fellow of the IMS and elected member of ISI. Visitor at MSRI, Berkeley, 1983 and 1991. Three times associate editor of The Annals of Statistics. Invited organizer of a DMV (Deutsche Mathematiker Vereinigung) seminar in G¨ unzburg, Germany, 1990. Invited lecturer at the Ecole d’Et´e de Probabilit´es de Saint-Flour, 1994.

vii

Publications of Piet Groeneboom April 2007 1. Rank tests for independence with best strong exact Bahadur slope (with Y. Lepage and F.H. Ruymgaart), Zeitschrift f¨ ur Wahrscheinlichkeitstheorie und Verwandte Gebiete 36 (1976), 119–127. 2. Bahadur efficiency and probabilities of large deviations (with J. Oosterhoff), Statist. Neerlandica 31 (1977), 1–24. 3. Relevant variables in the advices of elementary school teachers on further education; an analysis of correlational structures (in Dutch, with J. Hoogstraten, G.J. Mellenbergh and J.P.H. van Santen), Tijdschrift voor Onderwijsresearch (Journal for Educational Research) 3 (1978), 262–280. 4. Large deviation theorems for empirical probability measures (with J. Oosterhoff and F.H. Ruymgaart), Ann. Probability 7 (1979), 553–586. 5. Large deviations and asymptotic efficiencies, Mathematical Centre Tract 118 (1980), Mathematical Centre, Amsterdam 6. Large deviations of goodness of fit statistics and linear combinations of order statistics (with G.R. Shorack), Ann. Probability 9 (1981), 971–987. 7. Bahadur efficiency and small-sample efficiency (with J. Oosterhoff), Int. Statist. Rev. 49 (1981), 127–141. 8. The concave majorant of Brownian motion, Ann. Probability 11 (1983), 1016– 1027. 9. Asymptotic normality of statistics based on convex minorants of empirical distribution functions (with R. Pyke), Ann. Probability 11 (1983), 328–345. 10. Estimating a monotone density, in Proceedings of the Conference in honor of Jerzy Neyman and Jack Kiefer, Vol. II (Eds. L.M. Le Cam and R.A. Olshen), Wadsworth, Inc, Belmont, California (1985), 539–555. 11. Some current developments in density estimation, in Mathematics and Computer Science, CWI Monograph 1 (Eds. J.W. de Bakker, M. Hazewinkel, J.K. Lenstra), Elsevier, Amsterdam (1986), 163–192. 12. Asymptotics for incomplete censored observations, Mathematical Institute, University of Amsterdam (1987), Report 87-18. 13. Limit theorems for convex hulls, Probab. Theory Related Fields 79 (1988), 327–368. 14. Brownian motion with a parabolic drift and Airy functions, Probab. Theory Related Fields 81 (1989), 79–109. 15. Discussion on “Age-specific incidence and prevalence, a statistical perspective”, by Niels Keiding in the J. Roy. Statist. Soc. Ser. A. 154 (1991), 371– 412. 16. Information bounds and nonparametric maximum likelihood estimation (with J.A. Wellner), Birkh¨ auser Verlag (1992). 17. Discussion on “Empirical functional and efficient smoothing parameter selection” by P. Hall and I. Johnstone in the J. Roy. Statist. Soc. Ser. B. 54 (1992), 475–530. 18. Isotonic estimators of monotone densities and distribution functions: basic facts (with H.P. Lopuha¨ a), Statist. Neerlandica 47 (1993), 175–183. 19. Flow of the Rhine river near Lobith (in Dutch: “Afvoertoppen bij Lobith”), in Toetsing uitgangspunten rivierdijkversterkingen, Deelrapport 2: Maatgevende belastingen (1993), Ministerie van Verkeer en Waterstaat. 20. Limit theorems for functionals of convex hulls (with A.J. Cabo), Probab. Theory Related Fields 100 (1994), 31–55. viii

ix

21. Nonparametric estimators for interval censoring, in Analysis of Censored Data (Eds. H. L. Koul and J. V. Deshpande), IMS Lecture Notes-Monograph Series 27 (1995), 105–128. 22. Isotonic estimation and rates of convergence in Wicksell’s problem (with G. Jongbloed), Ann. Statist. 23 (1995), 1518–1542. 23. Computer assisted statistics education at Delft University of Technology, (with de P. Jong, D. Tischenko and B. van Zomeren), J. Comput. Graph. Statist. 5 (1996), 386–399. 24. Asymptotically optimal estimation of smooth functionals for interval censoring, part 1 (with R.B. Geskus), Statist. Neerlandica 50 (1996), 69–88. 25. Lectures on inverse problems, in Lectures On Probability and Statistics. Ecole d’Et´e de de Probabilit´es de Saint-Flour XXIV (Ed. P. Bernard), Lecture Notes in Mathematics 1648 (1996), 67–164. Springer Verlag, Berlin. 26. Asymptotically optimal estimation of smooth functionals for interval censoring, part 2 (with R.B. Geskus), Statist. Neerlandica 51 (1997), 201–219. 27. Extreme Value Analysis of North Sea Storm Severity (with C. Elsinghorst, P. Jonathan, L. Smulders and P.H. Taylor), Journal of Offshore Mechanics and Arctic Engineering 120 (1998), 177–184. 28. Asymptotically optimal estimation of smooth functionals for interval censoring, case 2 (with R.B. Geskus), Ann. Statist. 27 (1999), 627–674. 29. Asymptotic normality of the L1 -error of the Grenander estimator (with H.P. Lopuha¨ a and G. Hooghiemstra), Ann. Statist. 27 (1999), 1316–1347. 30. Integrated Brownian motion conditioned to be positive (with G. Jongbloed and J.A. Wellner), Ann. Probability 27 (1999), 1283–1303. 31. A monotonicity property of the power function of multivariate tests (with D.R. Truax), Indag. Math. 11 (2000), 209–218. 32. Computing Chernoff’s distribution (with J.A. Wellner), J. Comput. Graph. Statist. 10 (2001), 388–400. 33. A canonical process for estimation of convex functions: the “invelope” of integrated Brownian motion + t4 (with G. Jongbloed and J.A. Wellner), Ann. Statist. 29 (2001), 1620–1652. 34. Estimation of convex functions: characterizations and asymptotic theory (with G. Jongbloed and J.A. Wellner), Ann. Statist. 29 (2001), 1653–1698. 35. Ulam’s problem and Hammersley’s process, Ann. Probability 29 (2001), 683– 690. 36. Hydrodynamical methods for analyzing longest increasing subsequences, J. Comput. Appl. Math. 142 (2002), 83–105. 37. Kernel-type estimators for the extreme value index (with H.P. Lopuha¨ a and P.-P. de Wolf), Ann. Statist. 31 (2003), 1956–1995. 38. Density estimation in the uniform deconvolution model (with G. Jongbloed), Statist. Neerlandica 57 (2003), 136–157. 39. Hammersley’s process with sources and sinks (with E.A. Cator), Ann. Probability 33 (2005), 879–903. 40. Second class particles and cube root asymptotics for Hammersley’s process (with E.A. Cator), Ann. Probability 34 (2006), 1273–1295. 41. Estimating the upper support point in deconvolution (with L.P. Aarts and G. Jongbloed). To appear in the Scandinavian journal of Statistics, 2007. 42. Summa Cogitatio. To appear in Nieuw Archief voor Wiskunde (magazine of the Royal Dutch Mathematical Association) (2007). 43. Convex hulls of uniform samples from a convex polygon, Conditionally accepted for publication in Probability Theory and Related Fields.

x

44. Current status data with competing risks: Consistency and rates of convergence of the MLE (with M.H. Maathuis and J.A. Wellner). To appear in Ann. Statist. (2007). 45. Current status data with competing risks: Limiting distribution of the MLE (with M.H. Maathuis and J.A. Wellner). To appear in Ann. Statist. (2007). 46. The support reduction algorithm for computing nonparametric function estimates in mixture models (with G. Jongbloed and J.A. Wellner). Submitted.

Contributors to this volume Balabdaoui, F., Universit´e Paris-Dauphine Birg´e, L., Universit´e Paris VI Boysen, L., Universit¨ at G¨ ottingen D¨ umbgen, L., University of Bern Ferrari, P. A., Universidade de S˜ ao Paulo van de Geer, S. A., ETH Z¨ urich Gill, R. D., Leiden University Huang, J., University of Iowa Lalley, S. P., University of Chicago Liebscher, V., Universit¨ at Greifswald Mammen, E., Universit¨ at Mannheim Munk, A., Universit¨ at G¨ ottingen Pollard, D., Yale University Rufibach, K., University of Bern Sepp¨ al¨ ainen, T., University of Wisconsin-Madison Sisko, V. V., Universidade de S˜ ao Paulo van der Vaart, A. W., Vrije Universiteit Amsterdam Wellner, J. A., University of Washington Wittich, O., Technische Universiteit Eindhoven Xie, H., University of Iowa Yu, K., Universit¨ at Mannheim

xi

IMS Lecture Notes–Monograph Series Asymptotics: Particles, Processes and Inverse Problems Vol. 55 (2007) 1–31 c Institute of Mathematical Statistics, 2007

DOI: 10.1214/074921707000000256

arXiv:math/0701179v2 [math.ST] 7 Sep 2007

A Kiefer–Wolfowitz theorem for convex densities Fadoua Balabdaoui1 and Jon A. Wellner2,∗ Universit´ e Paris-Dauphine and University of Washington Abstract: Kiefer and Wolfowitz [Z. Wahrsch. Verw. Gebiete 34 (1976) 73– 85] showed that if F is a strictly curved concave distribution function (corresponding to a strictly monotone density f ), then the Maximum Likelihood bn , which is, in fact, the least concave majorant of the empirical Estimator F distribution function Fn , differs from the empirical distribution function in the uniform norm by no more than a constant times (n−1 log n)2/3 almost surely. We review their result and give an updated version of their proof. We prove a comparable theorem for the class of distribution functions F with conbn vex decreasing densities f , but with the maximum likelihood estimator F en : if X1 , . . . , Xn are sampled of F replaced by the least squares estimator F from a distribution function F with strictly convex density f , then the least en of F and the empirical distribution function Fn differ squares estimator F in the uniform norm by no more than a constant times (n−1 log n)3/5 almost surely. The proofs rely on bounds on the interpolation error for complete spline interpolation due to Hall [J. Approximation Theory 1 (1968) 209–218], Hall and Meyer [J. Approximation Theory 16 (1976) 105–122], building on earlier work by Birkhoff and de Boor [J. Math. Mech. 13 (1964) 827–835]. These results, which are crucial for the developments here, are all nicely summarized and exposited in de Boor [A Practical Guide to Splines (2001) Springer, New York].

1. Introduction: The Monotone Case Suppose that X1 , . . . , Xn are i.i.d. with monotone decreasing density f on (0, ∞). Then the maximum likelihood estimator fbn of f is the well-known Grenander estimator: i.e. the left-derivative of the least concave majorant Fbn of the empirical distribution function Fn . In the context of estimating a decreasing density f so that the corresponding distribution function F is concave, Marshall [17] showed that Fbn satisfies kFbn −F k ≤ √ √ kFn − F k so that we automatically have nkFbn − F k ≤ nkFn − F k = Op (1). Kiefer and Wolfowitz [14] sharpened this by proving the following theorem under strict monotonicity of f (and consequent strict concavity of F ). Let α1 (F ) = inf{t : F (t) = 1}, and write kgk = sup0≤t≤α1 (F ) |g(t)|. Theorem 1.1 (Kiefer–Wolfowitz [14]). If α1 (F ) < ∞, β1 (F ) ≡

inf

0
(−f ′ (t)/f 2 (t)) > 0,

∗ Supported

in part by NSF Grant DMS-05-03822 and by NI-AID Grant 2R01 AI291968-04. de Recherche, en Math´ ematiques de la D´ ecision, Universit´ e Paris-Dauphine, Paris, France, e-mail: [email protected] 2 University of Washington, Department of Statistics, Box 354322, Seattle, Washington 981954322, USA, e-mail: [email protected] AMS 2000 subject classifications: Primary 62G10, 62G20; secondary 62G30. Keywords and phrases: Brownian bridge, convex density, distance, empirical distribution, invelope process, monotone density, optimality theory, shape constraints. 1 Centre

1

2

Balabdaoui and Wellner

γ1 (F ) ≡ sup0
kFbn − Fn k = O((n−1 log n)2/3 )

almost surely.

Although Kiefer and Wolfowitz did not formulate their result in this way, the statement above follows from their proof. Also note that (1) implies that √ nkFbn − Fn k = O(n−1/6 (log n)2/3 ) → 0

almost surely, so that the MLE Fbn and the empirical distribution are asymptotically equivalent under the hypotheses of Theorem 1. Kiefer and Wolfowitz [14] used Theorem 1.1 to show that the MLE Fbn of F in the class of concave distributions is an asymptotically minimax estimator of F . (Also see Kiefer and Wolfowitz [15] for a generalization of the results of Kiefer and Wolfowitz [14] to allow somewhat weaker conditions.) It follows from the rather general theorem of Millar [18] that the empirical distribution function Fn remains asymptotically minimax in a wide range of problems involving shape- constrained families of d.f.’s F . In particular, for the classes Fk of distribution functions corresponding to k-monotone densities, it follows from Millar [18] that the empirical distribution function Fn is asymptotically minimax for estimation of F even in the smaller classes Fk . The interesting question which has not been addressed concerns asymptotic minimaxity of the MLEs within these classes. Our goal in this paper is to make some headway toward answering these questions by giving a partial (and imperfect) analogue of Theorem 1.1 in the case of F2 , the class of distribution functions corresponding to the class of decreasing and convex densities. The MLE and least squares estimators of a density f corresponding to F ∈ F2 have been studied by Groeneboom, Jongbloed and Wellner [11], and those results will provide an important starting point here. In fact, we will not study the MLE, but its natural surrogate, the least squares estimator. This is because of the lack of a complete analogue of Marshall’s lemma for the MLE in the convex case, while we do have such analogues for the least squares estimator; see D¨ umbgen, Rufibach and Wellner [7] and Balabdaoui and Rufibach [1]. One view of the Kiefer–Wolfowitz Theorem 1.1 is that it is driven by the (family of) corresponding local results, as follows: Theorem 1.2 (Local process convergence, monotone case). Suppose that t0 ∈ (0, ∞) is fixed with f (t0 ) > 0 and f ′ (t0 ) < 0, and f and f ′ continuous in a neighborhood of t0 . Then

(2)

n2/3 (Fbn (t0 + n−1/3 t) − Fn (t0 + n−1/3 t)) 1/3  2 2f (t0 ) d {C(at) − (W (at) − a2 t2 )} ⇒ Cb,c (t) − Y1 (t) = −f ′ (t0 )

in (D[−K, K], k · k) for every K > 0 where p Y1 (t) ≡ f (t0 )W (t) + (1/2)f ′ (t0 )t2 ≡ bW (t) − ct2

for W a standard two-sided Brownian motion process starting from 0, Cb,c is the Least Concave Majorant of Y1 , C ≡ C1,1 is the least concave majorant of W (t) − t2 ,
1/3 and a ≡ [f ′ (t0 )]2 /(4f (t0 )) .

A Kiefer–Wolfowitz theorem

3

The (one-dimensional) special case of (2) with t = 0 is due to Wang [26], while the complete result is given by Kulikov and Lopuha¨a [16]. Here the logarithmic term on the right side of (1) reflects the cost of transferring the family of (in distribution) local result to an (almost sure) global result. Here is a heuristic proof of (2); for the complete proof, see Kulikov and Lopuha¨a [16]. For a similar result in the context of monotone regression, see Durot and Tocquet [8], and for a similar theorem in the context of the Wicksell problem studied by Groeneboom and Jongbloed [9], see Wang and Woodroofe [25]. For a related result in the context of estimation of an increasing failure rate, see Wang [24]. Proof of Theorem 1.2. We rewrite the left side of (2) as

(3)

n2/3 {Fbn (t0 + n−1/3 t) − Fn (t0 + n−1/3 t)} = n2/3 {Fbn (t0 + n−1/3 t) − F (t0 ) − n−1/3 f (t0 )t}

− n2/3 {Fn (t0 + n−1/3 t)) − Fn (t0 ) − n−1/3 f (t0 )t} + n2/3 {Fbn (t0 ) − Fn (t0 ) − (Fbn (τ − ) − Fn (τ − ))} 0

0

− n2/3 {Fbn (t0 ) − F (t0 )}

where τ0− is the first point of touch of Fˆn and Fn to the left of t0 . From known local theory for Fˆn and Fn it follows easily that

(4) (5) and (6)

n2/3 {Fn (t0 + n−1/3 t)) − Fn (t0 ) − n−1/3 f (t0 )t} p 1 ⇒ f (t0 )W (t) + f ′ (t0 )t2 ≡ Y1 (t), 2 2/3 b −1/3 n {Fn (t0 + n t) − F (t0 ) − n−1/3 f (t0 )t} ⇒ Cb,c (t) n2/3 {Fbn (t0 ) − F (t0 )} ⇒ Cb,c (0)

where Cb,c is the least concave majorant of Y1 . It remains to handle the third term. But since Fbn (t0 ) − Fbn (τ0− ) = fˆn (t0 )(t0 − τ0− ) by linearity of Fbn on (τ0− , τ0+ ), n2/3 {Fbn (t0 ) − Fn (t0 ) − (Fbn (τ0− ) − Fn (τ0− )} = −n2/3 (Fn (t0 ) − Fn (τ − ) − fˆn (t0 )(t0 − τ − )) 0

0

= −n2/3 (Fn (t0 ) − Fn (τ0− ) − f (t0 )(t0 − τ0− )) + n2/3 (fbn (t0 ) − f (t0 ))(t0 − τ − ) 0

= n2/3 {Fn (t0 + n−1/3 n1/3 (τ0− − t0 )) − Fn (t0 ) − f (t0 )n−1/3 n1/3 (τ0− − t0 )} − n1/3 (fbn (t0 ) − f (t0 ))n1/3 (τ − − t0 ) 0

(1)

(7)

(1)

→d Y1 (τ− ) − Cb,c (0)τ− = Y1 (τ− ) − {Cb,c (0) + Cb,c (0)τ− } + Cb,c (0)

= Y1 (τ− ) − Cb,c (τ− ) + Cb,c (0) = Cb,c (0)

where τ− is the first point of touch of Y1 and Cb,c to the left of 0, and hence Cb,c (τ− ) = Y1 (τ− ). Combining (4), (5), (6) and (7) with (3) it follows that n2/3 {Fbn (t0 + n−1/3 t) − Fn (t0 + n−1/3 t)} ⇒ Cb,c (t) − Y1 (t)

in (D[−K, K], k · k) for each fixed K > 0.

Balabdaoui and Wellner

4

2. The convex case Now suppose that X1 , . . . , Xn are i.i.d. with monotone decreasing and convex density f on (0, ∞). Then the maximum likelihood estimator fbn of f is a piecewise linear, continuous and convex function with at most one change of slope between the order statistics of the data, and, as shown by Groeneboom, Jongbloed and Wellner [11], is characterized by Hn (x, fbn )



≤ 1, x ≥ 0 = 1, if fbn′ (x−) < fbn′ (x+).

where, with K being the class of convex and decreasing and nonnegative functions on [0, ∞), Hn (x, f ) =

Z

[0,x]

2(x − y)/x2 dFn (y), f (y)

(x, f ) ∈ R+ × K.

As shown by Groeneboom, Jongbloed and Wellner [11], the least squares estimator fen of f is also a piecewise linear, continuous, and convex function with at most one change of slope between the order statistics, but is characterized by e n (x) H



≥ Yn (x), x ≥ 0, = Yn (x), if fen′ (x−) < fen′ (x+).

R R R R e n (x) = x y fen (u)dudy ≡ x Fe (y)dy and Yn (x) = x Fn (y)dy. The where H 0 0 0 0R x corresponding estimators Fbn of F and Y are given by Fbn (x) = 0 fbn (y)dy and Rx b n (x) = b H 0 Fn (y)dy respectively. Since pointwise limit theory for both the MLE and the least squares estimators of f are available from Groeneboom, Jongbloed and Wellner [11], we begin by formulating a (family of) local convergence theorems analogous to Theorem 1.2 in the monotone case. These will serve as a guide in formulating appropriate hypotheses in the context of our global theorem. Theorem 2.1 (Local process convergence, convex case). If f (t0 ) > 0, f ′′ (t0 ) > 0, and f (t) and f ′′ (t) are continuous in a neighborhood of t0 , then for b n ) or for (Fn , Hn ) = (Fen , H e n ), (Fn , Hn ) = (Fbn , H 

(8)

 n3/5 (Fn (t0 + n−1/5 t) − Fn (t0 + n−1/5 t)) n4/5 (Hn (t0 + n−1/5 t) − Yn (t0 + n−1/5 t))  (1)  (1) H2 (t) − Y2 (t) ⇒ H2 (t) − Y2 (t)   1/5 f (t0)3 (1) (1) (H2,s (at) − Y2,s (at))   24 ′′ d   f (t0 ) =    1/5 4   f (t ) 0 3 24 ′′ (H2,s (at) − Y2,s (at)) f (t0 )3

in (D[−K, K], k · k) for every K > 0 where Z t p 1 W (s)ds + f ′′ (t0 )t4 Y2 (t) ≡ f (t0 ) 24 0

A Kiefer–Wolfowitz theorem

5

and H2 is the “invelope” process corresponding to Y2 : i.e. H2 satisfies: (a) H2 (t) ≥ R∞ (3) (2) Y2 (t) for all t; (b) −∞ (H2 − Y2 )dH2 = 0; and (c) H2 is convex. Here a=



f ′′ (t0 )2 242 f (t0 )

1/5

,

and H2,s , Y2,s denote the “standard” versions of H2 and Y2 with coefficients 1: i.e. Rt Y2,s (t) = 0 W (s)ds + t4 .

Note that β2 (F ) ≡ inf 0
1 = n3/5 (Fn (t0 + n−1/5 t) − F (t0 ) − n−1/5 f (t0 )t3 ) 6 3/5 −1/5 −1/5 1 − n (Fn (t0 + n t) − Fn (t0 ) − n f (t0 )t3 ) 6 + n3/5 (Fn (t0 ) − Fn (t0 ) − (Fn (τ0− ) − Fn (τ0− )) − n3/5 (Fn (t0 ) − F (t0 )).

Here   (1) 3 −1/5 −1/5 1 f (t0 )t ⇒ H2 (t), Fn (t0 + n t) − F (t0 ) − n n 6   (1) −1/5 −1/5 1 3/5 3 Fn (t0 + n t) − Fn (t0 ) − n n f (t0 )t ⇒ Y2 (t), 6 3/5

(1)

n3/5 (Fn (t0 ) − F (t0 )) ⇒ H2 (0), while

n3/5 (Fn (t0 ) − Fn (t0 ) − (Fn (τ0− ) − Fn (τ0− ))  = n3/5 Fn (t0 + n−1/5 n1/5 (τ0− − t0 )) − Fn (t0 )  1 − n−1/5 f (t0 )(n1/5 (τ0− − t0 ))3 6  3/5 −n Fn (t0 + n−1/5 n1/5 (τ0− − t0 )) − F (t0 )  1 − n−1/5 f (t0 )(n1/5 (τ0− − t0 ))3 6 + n3/5 (Fn (t0 ) − F (t0 )) (1)

(1)

(1)

(1)

→d Y2 (τ− ) − H2 (τ− ) + H2 (0) = H2 (0) (1)

(1)

since Y2 (τ− ) = H2 (τ− ). Combining the pieces yields the claim. The proof for the second coordinate is similar. Now we can formulate our main result. Fix τ < α1 (F ). Our hypotheses are as follows:

Balabdaoui and Wellner

6

R1. F has continuous third derivative F (3) (t) = f ′′ (t) > 0 for t ∈ [0, τ ] and β2 (F, τ ) ≡ inf 0 0 R2. γ˜1 (F, τ ) ≡ sup0
kFen − Fn k ≡ sup |Fen (t) − Fn (t)| = O((n−1 log n)3/5 ), 0≤t≤τ

(10)

e n − Yn k ≡ sup |H e n (t) − Yn (t)| = O((n−1 log n)4/5 ), kH 0≤t≤τ

almost surely.

Note that (9) and (10) imply that (11) (12)

n1/2 kFen − Fn k = O(n−1/10 (log n)3/5 ), e n − Yn k = O(n−3/10 (log n)4/5 ), n1/2 kH

almost surely. To prepare for the proof of Theorem 2.2, fix 0 < τ < α1 (F ) for which the (k) hypotheses of Theorem 2.2 hold. For an integer k ≥ 2 define aj ≡ aj ≡ F −1 ((j/ (k)

k)F (τ )) for j = 1, . . . , k, and set a0 ≡ a0 ≡ α0 (F ) ≡ sup{x : F (x) = 0}. Note (k) (k) that ak = F −1 (F (τ )) = τ for all k ≥ 2. We will often simply write aj for aj , but the dependence of the knots {aj } on k (and the choice of k depending on n) will be crucial for our proofs. We also set ∆j a = aj − aj−1 , and write |a| = max1≤j≤k ∆j a. Let Hn,k be the complete cubic spline interpolant of Yn with knot points given by {aj , j = 0, . . . , k}. Thus Hn,k is piecewise cubic on [aj−1 , aj ], j = 1, . . . , k with two (1) (2) continuous derivatives Hn,k and Hn,k ; see de Boor [5], pages 39–43 and 51–56. We (2)

will choose k = kn ∼ (Cn/ log n)1/5 → ∞ in our arguments. Hn,kn is not necessarily convex, but we will show that it becomes convex on [0, τ ] with high probability as n → ∞, and hence Hn,kn will play a role analogous to the role played by the linear interpolation of Fn in the proofs of Kiefer and Wolfowitz [14]. (We will frequently suppress the dependence of k = kn on n, and write simply k for kn .) Rt Let Y be defined by Y (t) ≡ 0 F (s)ds; thus Y (1) = F , Y (j) = f (j−2) , for j ∈ {2, 3, 4}. We will also need the complete cubic spline interpolant Hkn of Y ; this will play the role of the linear interpolant L = L(k) of F in Kiefer and Wolfowitz [14]. (k) The cubic spline interpolant Hn,k of Yn based on the knot points {aj , j = 0, . . . , k} is completely determined on [0, τ ] by the values of Yn at the knots aj , (1) j = 1, . . . , k together with the values of Yn = Fn at 0 and ak = τ , namely Yn (aj ), (1) (1) j = 1, . . . , J, Yn (0) = Fn (0) = 0, and Yn (τ ); see, e.g., de Boor [5], page 43. As de Boor nicely explains in his Chapter IV, the complete cubic spline interpolant is one case of a family of cubic interpolation methods. Taking de Boor’s function g to be our present function Yn , several different piecewise cubic interpolants of Yn can be described in terms of cubic polynomials Pj on each of the intervals [aj , aj+1 ] where the interpolating function Hn (·; s) is given by Hn (x; s) = Pj (x; s) for x ∈ [aj , aj+1 ],

A Kiefer–Wolfowitz theorem

7

j = 0, . . . , k − 1, and where we require Pj (αj ) = Yn (aj ),

Pj (αj+1 ) = Yn (aj+1 )

Pj′ (aj )

Pj′ (aj+1 ) = sj+1 ,

= sj ,

for j = 0, . . . , k − 1. Here s = (s0 , . . . , sk ) and the sj ’s are free parameters. Different choices of the sj ’s leads to different piecewise cubic functions agreeing with Yn at the knots aj ; all of these different approximating functions Hn (·; s) are continuous and have continuous first derivatives. Of interest to us here are the following particular ways of determining the sj ’s: (1)

• sj = Yn (aj ) = Fn (aj ), j = 0, . . . , k. This gives the piecewise cubic Hermite interpolant of Yn , Hn (·, s) ≡ Hn,Herm . (2) • sj , j = 0, . . . , k chosen so that Hn (·, s) ∈ C 2 [0, τ ]; i.e. so that Hn (·, s) is (1) (1) (1) continuous and s0 = Yn (0) = 0 and sk = Yn (ak ) = Yn (τ ). This gives the complete cubic spline interpolant of Yn , Hn (·, s) ≡ Hn,CS ≡ Hn,k . The complete spline interpolant Hn,CS will play the role for us that the linear interpolant Ln of Fn played in Kiefer and Wolfowitz [14]. As we will see, however, (2) even though the Hermite interpolant Hn,Herm is not in C 2 [0, τ ] (i.e. Hn,Herm is not continuous), the slopes of its piecewise linear second derivative can be given (1) explicitly in terms of Yn and Yn = Fn at the knots, and our proof will proceed by (2) relating the slopes of Hn,Herm to the (more complicated and less explicit) slopes of (2)

(2)

Hn,CS ≡ Hn,kn in order to prove point B in the following outline of our proof. Here is an outline of the proof, paralleling the proof of the K–W theorem. Main steps, proof of (9) distribution function equivalence: A. By the generalization of Marshall’s lemma for the convex density problem (see D¨ umbgen, Rufibach and Wellner [7]), for any function h with convex e e (2) e (1) derivative h′ , kH n − hk ≤ 2kFn − hk where Hn ≡ fn . [This generalization (1) b n of F in F2 corresponding to H b (2) b is not yet available for the MLE H n = fn ; see D¨ umbgen, Rufibach and Wellner [7] for a one-sided result.] (2) B. PF (An ) ≡ PF {Hn,kn is convex on [0, τ ]} ր 1 as n → ∞ if kn ≡ (C0 β2 (F )2 n/ log n)1/5 for some absolute constant C0 . C. On the event An , (1)

(1)

e (1) − Fn k = kH e (1) − H kH n n n,kn + Hn,kn − Fn k (1)

(1)

≤ 2kFn − Hn,kn k + kHn,kn − Fn k

by the generalization of Marshall’s lemma (A) (1)

= 3kFn − Hn,kn k (1)

(1)

(1)

(1)

(1)

= 3kFn − Hn,kn − (F − Hkn ) + F − Hkn k

(1)

≤ 3kFn − Hn,kn − (F − Hkn )k + 3kF − Hkn k ≡ 3Dn + 3En . D. We show that Dn = O((n−1 log n)3/5 ) almost surely via a generalization of the K–W Lemma 2. We also show that En = O((n−1 log n)3/5 )by an analytic (deterministic) argument.

Balabdaoui and Wellner

8

Of course proving step B in this outline involves showing that the slopes of (2) the Hn,kn become ordered with high probability for large n, and this explains our (2)

(2)

(2)

interest in the slopes of both Hn,CS = Hn,kn and Hn,Herm . The assertion (10) of Theorem 2.2 can be proved in a similar way if we replace (1) (1) e (1) e H n , Hn,kn , Hkn , Fn , F by Hn , Hn,kn , Hkn , Yn , Y respectively, and if we replace A by the following recent result of Balabdaoui and Rufibach [1]: ˜ n − Gk ≤ kYn − Gk. A′ . For any function G with convex second derivative g ′′ , kH

Proof of (9) assuming B. First the deterministic term En . As in de Boor [5], page 43, let I4 denote the complete cubic spline interpolation operator, and (as in de Boor [5], page 31, let I2 be the piecewise linear (or “broken line”) interpolation operator. Then by de Boor [5], (20) on page 56, with pn ≡ 1/kn , (1)

En = kF − Hkn k = kY (1) − (I4 Y )(1) k ≤ ≤

1 3 (4) |a| kY k 24

1 γ2 (F, τ )p3n = O((n−1 log n)3/5 ). 24

To handle Dn , let $3 be defined to be the space of all quadratic splines on [0, τ ], and similarly let $2 be the space of all linear splines on [0, τ ]. Then, by de Boor [5], page 56, equation (17), together with (18) on page 36, it follows that with (1)

(1)

Dn = kFn − Hn,kn − (F − Hkn )k = k(Yn − Y )(1) − (I4 (Yn − Y ))(1) k 19 19 ≤ dist((Yn − Y )(1) ; $3 ) ≤ dist((Yn − Y )(1) ; $2 ) 4 4 19 ≤ k(Yn − Y )(1) − I2 [(Yn − Y )(1) ]k 4 19 = k(Fn − F ) − I2 (Fn − F )k 4 19 ≤ ω(Fn − F ; |a|) 4 d 19 −1/2 ω(Un ; pn ) = n 4 p = O(n−1/2 pn log(1/pn )) almost surely = O((n−1 log n)3/5 );

here ω(g; h) ≡ sup{|g(t) − g(s)| : |t − s| ≤ h}, dist(g; S) ≡ min{kg − f k : f ∈ S}, S ⊂ C[0, τ ] and √ Un (t) ≡ n(Gn (t) − t) Pn where Gn (t) = n−1 i=1 1[0,t] (ξi ) is the empirical distribution function of ξ1 , . . . , ξn i.i.d. Uniform(0, 1) random variables. (See de Boor [5], pages xviii, 24, 32, and 34 for definition and use of dist(g; S) and the modulus of continuity ω in conjunction.) Proof of (10) assuming B. By Hall [12] (also see Hall and Meyer [13] for optimality of the constant and de Boor [5], page 55), En ≡ kY − Hkn k ≤

5 5 1 |a|4 kY (4) k ≤ Rγ2 (F ) 4 . 384 384 kn

A Kiefer–Wolfowitz theorem

9

To handle the first term Dn , we note that Yn − Y − (Hn,kn − Hkn ) = (Yn − Y ) − I4 (Yn − Y ) where I4 is the complete spline interpolant, and, on the other hand, for any differentiable function g it follows from de Boor [5], page 45, equation (14), together with (18) on page 36, that 19 19 |a|dist(g ′ , $3 ) ≤ |a|dist(g ′ , $2 ) 8 8 19 19 ≤ |a|kg ′ − I2 gk ≤ |a|ω(g ′ , |a|). 8 8

kg − I4 gk ≤

Applying this to g = Yn − Y , it follows that kYn − Y − (Hn,kn − Hkn )k = k(Yn − Y ) − I4 (Yn − Y )k 19 ≤ |a| ω(Fn − F, |a|) 8 d

= n−1/2 ω(Un , pn ) p Therefore ω(Fn − F ; |a|) = O(n−1/2 pn log(1/pn )) almost surely (just as in the proof of Lemma 2 for the Kiefer-Wolfowitz theorem, see Section 5), we see that the order of Dn is 1/2 n−1/2 p3/2 = O((n−1 log n)4/5 ) n (log(1/pn ))

almost surely

as claimed. Thus the claim (10) is proved if we can verify that B holds. We end this section with a short list of further problems: • It would be of interest to prove a comparable theorem for the MLE Fbn itself rather than Fen . This involves several additional challenges, among which is a complete analogue of Marshall’s lemma. • Are either Fen or Fbn asymptotically minimax for estimating F ∈ F2 ? • We conjecture that similar results hold for k−monotone densities and corresponding distribution functions (k = 1 corresponds to the Kiefer and Wolfowitz monotone density case, while k = 2 corresponds to the convex density case treated here). More concretely, we conjecture that under comparable hypotheses kFn − Fn k = O((n−1 log n)(k+1)/(2k+1) )

almost surely

for Fn = Fen or Fn = Fbn , the least squares estimator or MLE of F ∈ Fk . Some progress on the local theory of the corresponding density estimators is given in Balabdaoui and Wellner [2] and Balabdaoui and Wellner [3]. On the interpolation theory side, the results of Dubeau and Savoie [6] may be useful. • What is the exact order (in probability or expectation) of kFbn − Fn k in the case k = 2? Is it (n−1 log n)3/5 as perhaps suggested by the results of Durot and Tocquet [8] in the case k = 1? (2)

3. Asymptotic convexity of Hn,kn In this section we write C for the complete spline interpolation operator that maps functions g ∈ C 1 [0, τ ] into their complete spline interpolants C[g] (based on the

Balabdaoui and Wellner

10

fixed knot sequence 0 = a0 < a1 . . . < ak = τ ); thus in this section our C is de Boor’s operator I4 . Thus we have Hn,k = C[Yn ],

Hn,k = C[Y ].

It follows from the formula for c4,i in (5) on page 40 of de Boor [5] that the slope (2) of Hn,k on the interval [aj−1 , aj ] is given by 12 Bj ≡ Bj (CS) = (∆j a)3

(1)

(1)

Hn,k (aj−1 ) + Hn,k (aj ) 2

∆j a − ∆j Yn

!

where ∆j f ≡ f (aj ) − f (aj−1 ) for j = 1, . . . , k and any function f on [0, τ ]. In the following we will let H denote the Hermite interpolation operator that (1) maps Yn to Hn : thus Hn,Herm = H[Yn ], Hn,Herm = (H[Yn ])(1) , and so forth. It is important to note that the corresponding slopes of the second derivative of the (2) Hermite interpolant, Hn,Herm = (H[Yn ])(2) on [aj−1 , aj ] are given by the same (1)

(1)

formula as in the last display, but with Hn,k (ai ) replaced by Yn (ai ) = Fn (ai ), i = j − 1, j: (13)

ej ≡ Bj (Herm) = B

12 (∆j a)3



 Fn (aj−1 ) + Fn (aj ) ∆j a − ∆j Yn . 2

ej is expressed explicitly as a function of the data via Fn and Yn , Note that B whereas Bj still involves Hn,k = C[Yn ] and hence also the interpolation operator ej can be shown using only Lemma 3.1 and Lemma 4.5, C. Ordering of the slopes B but (unfortunately) the generalization of Marshall’s lemma does not apply to the (2) Hermite interpolant because the second derivative Hn,Herm is not continuous at the knots. This last formula (13) agrees with the formulas for H and Hn in Groeneboom, Jongbloed and Wellner [10] and Groeneboom, Jongbloed and Wellner [11]; in particular (13) can be viewed as a finite sample analogue of the 3rd derivative of the interpolant H given in Groeneboom, Jongbloed and Wellner [10], page 1631, but based on the fixed knots {aj } rather than random knots determined by the e (2) optimization procedure. Note that the least squares estimator fen = H n can be viewed as the second derivative of either the Hermite interpolant or the complete cubic spline interpolant of Yn since these two interpolants have been forced equal by the optimization procedure which determines the knots as random functions of the data. Set An ≡

n

(2) Hn,kn

o k−1 \ {Bj ≤ Bj+1 } . is convex on [0, τ ] = j=1

To prove B, we want to bound P (Acn ) ≤

k−1 X j=1

P (Bj > Bj+1 ).

A Kiefer–Wolfowitz theorem

11

To prepare for this, we define (C[Yn ])(1) (aj−1 ) + (C[Yn ])(1) (aj ) ∆j a − ∆j Yn , 2 (1) (1) Yn (aj−1 ) + Yn (aj ) ∆j a − ∆j Yn , = 2 (C[Y ])(1) (aj−1 ) + (C[Y ])(1) (aj ) = ∆j a − ∆j Y, 2 Y (1) (aj−1 ) + Y (1) (aj ) = ∆j a − ∆j Y. 2

Tn,j = Rn,j tn,j rn,j

We will frequently suppress the dependence of all of these quantities on n, and ˜j = simply write Tj for Tn,j , Rj for Rn,j , and so forth. Now Bj = 12Tj /(∆j a)3 , B 12Rj /(∆j a)3 , and we can write (14) (15)

Tj − rj = Tj − tj + tj − rj = Rj − rj + {Tj − tj − (Rj − rj )} + tj − rj ≡ Rj − rj + Wj + bj .

We regard Rj −rj as the main random term to be controlled, and view Tj −tj −(Rj − rj ) ≡ Wj and tj − rj ≡ bj as second order terms, the last of which is deterministic. Thus our strategy will be to first develop an appropriate exponential bound for |Rj − rj |, and then by further separate bounds for Wj and bj , derive an exponential bound for |Tj − rj |. For 0 ≤ s < t < ∞, define the family of functions hs,t by hs,t (x) = (x − (s + t)/2)1(s,t] (x). Note that Z t 1 P hs,t = (F (t) + F (s))(t − s) − F (u)du, 2 s Z t 1 Fn (u)du, Pn hs,t = (Fn (t) + Fn (s))(t − s) − 2 s and, furthermore, rj = P haj−1 ,aj ,

Rj = Pn haj−1 ,aj .

Here is a (partial) analogue of Kiefer and Wolfowitz’s Lemma 1. Lemma 3.1. Suppose that γ˜1 (F ) < ∞ and R < ∞. Let hs,t (x) = (x − (s + (k) (k) t)/2)1(s,t] (x), s = aj−1 ≡ aj−1 , and t = aj ≡ aj so that t − s = aj − aj−1 = −1 ∗ ∗ k (1/f (aj )) for some aj ∈ [aj−1 , aj ]. Then if δn → 0 and k ≥ 5˜ γ1 (F )R, P r(|Rj − rj | > δn p3n ) = P r(|Pn − P |(hs,t ) > δn p3n ) ! 3nδn2 f 2 (a∗j )p3n = 2 exp − 1 + pn δn f (a∗j ) ≤ 2 exp(−3nδn2 p3n f 2 (a∗j )(1 + o(1))) where o(1) depends on f (a∗j ), kn , and δn .

Balabdaoui and Wellner

12

Proof. First note that |hs,t | is bounded by (t − s)/2. Thus by Bernstein’s inequality (see e.g. van der Vaart and Wellner [23], page 102),   nx2 /2 P r(|Pn hs,t − P hs,t | > x) ≤ 2 exp − 2 σ + M x/3 for σ 2 ≥ V arF (hs,t (X)), M = (t − s)/2 = 1/(2f (a∗j )k) = [1/(2f (a∗j ))]pn , and x > 0. Note that Z t 2 (x − (t + s)/2)2 dF (x) V ar(hs,t (X)) ≤ Ehs,t (X) = s

≤ f (s)(t − s)3 /12

= f (s)k −3 /(12f 3(a∗j )) = f (s)p3n /(12f 3 (a∗j ))

≤ p3n /(6f 2 (a∗j )) for k ≥ 5˜ γ1 (F )R by Lemma 4.1. Then we obtain P r(|Pn hs,t − P hs,t | > δn p3n ) ≤ 2 exp = 2 exp = 2 exp = 2 exp

nδn2 p6n /2 − 3 pn /(6f (a∗j )2 ) + pn δn p3n /(6f (a∗j )) ! nδn2 f 2 (a∗j )p3n − 1/3 + pn f (a∗j )δn /3 ! 3nδn2 f 2 (a∗j )p3n − 1 + pn δn f (a∗j )
− 3nδn2 p3n f 2 (a∗j )(1 + o(1))

!

where the o(1) term depends on f (t) = f (aj+1 ), pn = 1/kn , and δn . Remark. Note that taking δn = C/kn in Lemma 3.1 yields P r(|Pn hs,t − P hs,t | > Cp4n ) ≤ 2 exp(−3(nC 2 f 2 (a∗j )/kn5 )(1 + o(1))) which seems quite analogous to Lemma 4 of Kiefer and Wolfowitz (1976), but with the power of 3 replaced by 5. The following lemma gives a more complete version of Lemma 3.1 in that it provides an exponential bound for |Tj − rj |. Lemma 3.2. Suppose that the hypotheses of Theorem 2.2 hold: β2 (F, τ ) < ∞, γ2 (F, τ ) < ∞, γ˜1 (F ) < ∞ and R < ∞. Then if δn = Cpn for some constant C and k ≥ {5R ∨ 3}˜ γ1 (F ), ! (100)−1 nδn2 f 2 (a∗j )p3n 3 P r(|Tj − rj | > 3δn pn ) ≤ 6 exp − . 1 + 30−1 pn δn f (a∗j ) Proof. This follows from a combination of Lemma 3.1, Lemma 4.2, and Lemma 4.3. Lemma 4.2 yields |bj | ≡ |tj − rj | ≤ R4 o(1)p4n ≤ δn p3n if n (and hence kn ) is sufficiently large. This implies that P r(|Tj − rj | > 3δn p3n ) ≤ P r(|Tj − tj | > 3δn p3n − |tj − rj |) ≤ P r(|Tj − tj | > 2δn p3n ).

A Kiefer–Wolfowitz theorem

13

In view of the decomposition (15), this yields P r(|Tj − rj | > 3δn p3n ) ≤ P r(|Rj − rj | > δn p3n ) + P r(|Aj | > δn p3n ) ! (100)−1 nδn2 f 2 (a∗j )p3n ≤ 6 exp − 1 + 30−1 pn δn f (a∗j ) by Lemma 3.1, Lemma 4.3, and the fact that 100−1 A A 30 3A = ≤ −1 1 + 30 B 100 30 + B 1+B for A, B > 0. Lemma 3.3. Suppose that β2 ≡ β2 (F, τ ) > 0, γ˜1 ≡ γ˜1 (F, τ ) < ∞ and R ≡ R(f, τ ) < ∞ for some τ < α1 (F ) ≡ inf{t : F (t) = 1}. Let (2)

An ≡ {Hn,kn is convex on [0, τ ]}. Then P (Acn ) ≤ 12kn exp −Kβ22 (F, τ )np5n

(16)




where K −1 = 82 · 1442 · 16 · 200 = 4, 246, 732, 800 ≤ 4.3 · 109 . Proof. Since Acn ≡

k[ n −1 j=1

{Bj > Bj+1 },

it follows that P (Acn ) ≤ =

kX n −1

kX n −1 j=1

+

  P Bj > Bj+1 , Ti − ri ≤ 3δn,j p3n , i = j, j + 1

kX n −1 j=1

≤

mX n −1 j=0

+

P (Bj > Bj+1 )

j=1

  P Bj > Bj+1 , Ti − ri > 3δn,j p3n for i = j or i = j + 1

  P Bj > Bj+1 , Ti − ri ≤ 3δn,j p3n , i = j, j + 1

kX n −1  j=0

(17)

= In + IIn

    3 3 P Tj − rj > 3δn,j pn + P Tj+1 − rj+1 > 3δn,j pn

where we take δn,j =

C(F, τ ) C(F, τ ) δn = pn ≡ ; ∗ ∗ kn f (aj ) f (aj ) f (a∗j )

Balabdaoui and Wellner

14

here a∗j ∈ [aj−1 , aj ] satisfies ∆j a = aj − aj−1 = 1/(kn f (a∗j )), and C(F, τ ) is a constant to be determined. We first bound IIn from above. By Lemma 3.2, we know that !   2 (100)−1nδn,j f 2 (a∗j )p3n 3 P |Tj − rj | > 3δn,j pn ≤ 6 exp − 1 + 30−1 pn δn,j f (a∗j ) 2 where δn,j f 2 (a∗j )p3n = C 2 (F, τ )p5n and

1 1 1 = > 1 + 30−1 pn δn,j f (a∗j ) 1 + 30−1 C(F, τ )p2n 2 when kn > [30−1 C(F, τ )]1/2 . Hence,  
3 (18) P |Tj − rj | > 3δn,j pn ≤ 6 exp −200−1C 2 (F, τ )np5n .

We also have !   2 100−1 nδn,j f 2 (a∗j+1 )p3n 3 P |Tj+1 − rj+1 | > 3δn,j pn ≤ 6 exp − 1 + 30−1 pn δn,j f (a∗j+1 ) where a∗j+1 ∈ [aj , aj+1 ] and aj+1 − aj = ∆j+1 a = 1/(kn f (a∗j+1 ). By Lemma 5.1 we have f (aj )/f (aj+1 ) ≤ 2 if kn ≥ 5˜ γ1 (F, τ )R. But this implies that f (a∗j )/f (a∗j+1 ) ≤ 4 since f (a∗j ) f (aj ) f (aj+1 ) f (a∗j ) = · · f (a∗j+1 ) f (aj ) f (aj+1 ) f (a∗j+1 ) ≤

f (aj−1 ) f (aj ) f (aj+1 ) ≤ 2 · 2 · 1 = 4. f (aj ) f (aj+1 ) f (a∗j+1 )

Hence, we can write 2 δn,j f 2 (a∗j+1 ) =

f 2 (a∗j+1 ) 1 2 1 2 1 C 2 (F, τ ) 2 ≥ C (F, τ ) C (F, τ ) = pn kn2 f 2 (a∗j ) kn2 16 16

and, since f (a∗j+1 )/f (a∗j ) ≤ 1, 1+

30−1 p

1 1 = ∗ −1 1 + 30 C(F, τ )p2n f (a∗j+1 )/f (a∗j ) n δn,j f (aj+1 ) 1 1 > ≥ −1 2 1 + 30 C(F, τ )pn 2

when kn > [30−1 C(F, τ )]1/2 . Thus, we conclude that     200−1 2 5 3 (19) C (F, τ )npn P |Tj+1 − rj+1 | > 3δn,j pn ≤ 6 exp − 16 Combining (18) and (19), we get   200−1 2 5 IIn ≤ 12kn exp − C (F, τ )npn . 16

A Kiefer–Wolfowitz theorem

15

Now we need to handle In . Recall that Bj = 12

Tj , (∆j a)3

Bj+1 = 12

Tj+1 . (∆j+1 a)3

Thus, the event n o Bj > Bj+1 , |Ti − ri | ≤ 3δn,j p3n , i = j, j + 1

is equal to the event o n T Tj+1 j > , |Ti − ri | ≤ 3δn,j p3n , i = j, j + 1 . 3 3 (∆j a) (∆j+1 a) Then, it follows that Tj rj 3δn,j p3n ≤ + 3 3 (∆j a) (∆j a) (∆j a)3 and

Tj+1 rj+1 3δn,j p3n ≥ − , (∆j+1 a)3 (∆j+1 a)3 (∆j+1 a)3

and hence     rj+1 rj+1 3δn,j p3n Tj rj + ≤ − − (∆j a)3 (∆j a)3 (∆j+1 a)3 (∆j+1 a)3 (∆j+1 a)3   3 3 3δn,j pn 3δn,j pn + + 3 (∆j a) (∆j+1 a)3   Tj+1 rj+1 rj + − ≤ 3 3 (∆j a) (∆j+1 a) (∆j+1 a)3   3 3 3δn,j pn 3δn,j pn + . + 3 (∆j a) (∆j+1 a)3 The first term in the right side of the previous inequality is the leading term in (2) the sense that it determines the sign of the difference of the slope of Hn,kn . By Lemma 4.5, we can write rj rj+1 1 1 ′′ − ≤ − f ′′ (a∗∗ (f ∆j a − f ′′j+1 ∆j+1 a). j )∆j a + 3 3 (∆j a) (∆j+1 a) 12 24 j Let a∗j ∈ [aj−1 , aj ] such that ∆j a = pn [f (a∗j )]−1 . Then, we can write 3δn,j p3n 1 1 ′′ 3δn,j p3n + − f ′′ (a∗∗ (f ∆j a − f ′′j+1 ∆j+1 a) j )∆j a + 3 3 (∆j a) (∆j+1 a) 12 24 j 1 1 ′′ ≤ 6δn,j f 3 (a∗j ) − f ′′ (a∗∗ (f j ∆j a − f ′′j+1 ∆j+1 a) j )∆j a + 12 24 ) ( ′′ ∗∗ f (a ) 1 1 ′′ j ′′ = 6f 2 (a∗j ) δn − pn + (f ∆j a − f j+1 ∆j+1 a) 72 f 3 (a∗j ) 144f 2 (a∗j ) j (   ) 1 f ′′ (a∗∗ 1 ∆j+1 a ′′ j ) ′′ 2 ∗ pn = 6f (aj ) δn − pn + f j − f j+1 72 f 3 (a∗j ) 144f 3 (a∗j ) ∆j a ( ! ) ′′ ′′ ∗∗ f ′′j+1 f (a ) f 1 1 ∆ a j+1 j j = 6f 2 (a∗j ) δn − pn pn + − 72 f 3 (a∗j ) 144 f 3 (a∗j ) f ′′j+1 ∆j a

Balabdaoui and Wellner

16

= 6f

≤ 6f

2

2

(a∗j )

(a∗j )

(

(

δn −

3 ∗∗ 1 f ′′ (a∗∗ j ) f (aj ) pn 3 ∗ 72 f 3 (a∗∗ j ) f (aj )

′′ 1 f j+1 + 144 f 3 (a∗j )

′′

fj

f ′′j+1

∆j+1 a − ∆j a

!

′′ 1 f j+1 1 β2 (F, τ ) pn + δn − 72 8 144 f 3 (a∗j )

 1 β2 (F, τ ) = 6f 2 (a∗j ) δn − pn 72 8 ′′ 1 f j+1 + 144 f 3 (a∗j )

′′

pn

) ′′

fj

f ′′j+1

fj ∆j+1 a −1+1− ∆j a f ′′j+1

!

∆j+1 a − ∆j a

pn

!

pn

)

)

where (using arguments similar to those of Lemma 4.2 and taking the bound on ′′ ′′ |f j − f ′′j+1 | to be ǫkf k which is possible by uniform continuity of f ′′ on [0, τ ]) ′′ ′′ f ǫf (τ )3 γ (F, τ ) fj j 2 − 1 ≤ ′′ − 1 ≤ ′′ ′′ f f j+1 f j+1 j+1

√ if kn > max(5˜ γ1 (F, τ )R, ( 2 + 1)R/η) for a given η > 0 and ∆j+1 a ∆j+1 a ≤ − 1 ≤ 8˜ γ1 (F, τ )pn . 1− ∆j a ∆j a Hence

3δn,j p3n 3δn,j p3n 1 1 ′′ + − f ′′ (a∗∗ (f ∆j a − f ′′j+1 ∆j+1 a) j )∆j a + 3 (∆j a) (∆j+1 a)3 12 24 j  1 1 β2 (F, τ ) pn + ǫγ2 (F, τ ) pn ≤ 6f 2 (a∗j ) δn − 72 8 144  8 + γ2 (F, τ )˜ γ1 (F, τ )p2n 144 where we can choose ǫ and pn small enough so that 1 8 1 ǫγ2 (F, τ ) + γ2 (F, τ )˜ γ1 pn ≤ β2 (F, τ ); 144 144 2 · 72 · 8 for example ǫ<

1 β2 (F, τ ) , 16 γ2 (F, τ )

kn = p−1 n > 16 · 8

γ˜1 (F, τ ) . β2 (F, τ )

The above choice yields 3δn,j p3n 3δn,j p3n 1 1 ′′ + − f ′′j ∆j a + (f j ∆j a − f ′′j+1 ∆j+1 a) (∆j a)3 (∆j+1 a)3 12 24   β2 (F, τ ) 2 ∗ pn = 0 ≤ 6f (aj ) δn − 8 · 144

A Kiefer–Wolfowitz theorem

17

by choosing β2 (F, τ ) pn ; 8 · 144 i.e. C(F, τ ) = β2 (F, τ )/(8 · 144). For such a choice, the first term In in (17) is identically equal to 0. δn = C(F, τ )pn =

4. Appendix 1: technical lemmas Lemma 4.1. Under the hypotheses of Theorem 2.2, 1≤

f (aj−1 ) ∆j+1 a ≤ ≤2 f (aj ) ∆j a

uniformly in j if k ≥ 5˜ γ1 R. Proof. Note that for each interval Ij = [aj−1 , aj ] we have  Z ≥ f (aj )∆j a ∗ f (x)dx = f (aj )∆j a pn = ≤ f (aj−1 )∆j a Ij where a∗j ∈ Ij . Thus

pn pn ≤ f (aj ) ≤ ∆j+1 a ∆j a

and

pn pn ≤ f (aj−1 ) ≤ . ∆j a ∆j−1 a

It follows that 1≤

∆j+1 a ∆j+1 a ∆j a f (aj−1 ) ≤ = . f (aj ) ∆j−1 a ∆j a ∆j−1 a

Thus we will establish a bound for ∆j+1 a/∆j a. Note that with c ≡ F (τ ) < 1 j+1 j c) − F −1 ( c) k k c2 −f ′ (ξj+1 ) c 1 + 2 3 = k f (aj ) 2k f (ξj+1 )   c −f ′ (ξj+1 ) f (aj ) c 1 1+ = k f (aj ) 2k f 2 (ξj+1 ) f (ξj+1 )   c 1 c˜ γ1 ≤ 1+ R . k f (aj ) 2k

∆j+1 a = aj+1 − aj = F −1 (

for some ξj+1 ∈ Ij+1 , where ξj+1 ∈ Ij+1 , R < ∞, and γ˜1 < ∞. Similarly, expanding to second order (about aj again!), j−1 j c) ∆j a = aj − aj−1 = F −1 ( c) − F −1 ( k k ′ 2 c f (ξj ) c 1 + 2 3 = k f (aj ) 2k f (ξj )   c f ′ (ξj ) f (aj ) c 1 1+ = k f (aj ) 2k f 2 (ξj ) f (ξj )

Balabdaoui and Wellner

18

  c f ′ (ξj ) c 1 1+ k f (aj ) 2k f 2 (ξj ) since f (aj )/f (ξj ) ≤ 1 and f ′ (ξj ) < 0   c˜ γ1 c 1 1− . ≥ k f (aj ) 2k ≥

where ξj ∈ Ij . Thus it follows that for k = kn so large that γ˜1 /(2k) ≤ 1/2 we have c˜ γ ∆j+1 a 1 + 2k1 R ≤ γ1 ∆j a 1 − c˜ 2k    c˜ γ1 c˜ γ1 R 1+ ≤ 1+ 2k k cγ ˜ c2 γ˜12 = 1 + 1 (R/2 + 1) + R k 2k 2 γ˜1 (R + 1) <1+ k

if k = kn ≥ γ˜1 . The last inequality here follows from γ˜1 γ˜ 2 γ˜1 (R/2 + 1) + 12 R ≤ (R + α) k 2k k if and only if (R/2 + 1) +

γ˜1 R ≤R+α 2k

or, equivalently, if and only if γ˜1 R ≤ R/2 + α − 1, 2k

or

k ≥ γ˜1

R = γ˜1 R + 2(α − 1)

if α = 1. It now follows that 1≤

∆j a ∆j+1 a ∆j+1 a f (aj−1 ) ≤ = ≤2 f (aj ) ∆j−1 a ∆j a ∆j−1 a∆j−1 a

if ∆i+1 a √ ≤ 2 ∆i a for i = j − 1, j. But these inequalities hold if k is so large that 1 + γ˜1 (R+1) ≤ k √ √ or k ≥ 5˜ γ1 R ≥ γ˜1 (R + 1)/( 2 − 1) since R ≥ 1 and 1/( 2 − 1) ≤ 5/2. Lemma 4.2. Under the hypotheses of Theorem 2.2, |tj − rj | = o(1) (∆j a)4 where the o(1) depends only on τ , γ˜1 (F, τ ), and γ2 (F, τ ). Remark. Note that (20)

max |tj − rj | ≤

1≤j≤k

1 4 ′′ 1 1 4 (4) |a| kY k = |a| kf k ≤ Rγ2 (F )p4n . 24 24 24

√

2,

A Kiefer–Wolfowitz theorem

19

This follows since rj − tj =

1  (1) Y (aj−1 ) + Y (1) (aj ) 2

 − ((C[Y ])(1) (aj−1 ) − (C[Y ])(1) (aj )) ∆j a  1 n (1) Y (aj−1 ) − (C[Y ])(1) (aj−1 ) = 2  o + Y (1) (aj ) − (C[Y ])(1) (aj ) ∆j a,

and hence from de Boor [5], (20), page 56, it follows that |rj − tj | ≤

1 3 (4) 1 4 (2) |a| kY k∆j a ≤ |a| kf k, 24 24

and this yields (20). The claim of Lemma 4.2 is stronger because it makes a statement about the differences tj − rj relative to (∆j a)4 ; this is possible because only differences between the derivative of the derivative of Y and the derivative of its interpolant C[Y ] at the knots are involved. Proof. We have rj − tj =

(21)


1 (1) (1) EY (aj−1 ) + EY (aj ) ∆j a, 2

where Eg = g − C[g]. Now, using the result of Problem 2a, Chapter V of de Boor [5] (compare also with the formula (3.52) given in N¨ urnberger [20]), we have (1)

(1)

(1)

δj EY (aj−1 ) + 2EY (aj ) + (1 − δj )EY (aj+1 ) = βj for j = 0, · · · , k − 1, where δj =

aj+1 − aj ∆j+1 a = aj+1 − aj−1 ∆j a + ∆j+1 a

and βj =

δj (−∆j a)3 f ′′ (ξ1,j ) + (1 − δj )(∆j+1 a)3 f ′′ (ξ2,j ) , 24

ξ1,j , ξ2,j ∈ [aj−1 , aj+1 ]. By Problem IV 7(a) in de Boor [5] and the techniques used in Chapter III (see in particular equation (9)), a bound on the maximal value at the knots of the derivative interpolation error can be derived using the following inequality   (1) (1) (1) (22) max |EY (aj )| ≤ max |EY (a0 )|, max |βj |, |EY (ak )| . 0≤j≤k

1≤j≤k−1

(1)

(1)

By definition of the complete cubic spline, EY (a0 ) = EY (ak ) = 0. Thus, we will focus now on getting a sharp bound for max1≤j≤k−1 |βj | under our hypotheses. This will be achieved as follows: • Expanding δj around 1/2: We have δj =

kn−1 [f (a∗j+1 )]−1 aj+1 − aj = −1 , (aj+1 − aj ) + (aj − aj−1 ) kn [f (a∗j+1 )]−1 + kn−1 [f (a∗j )]−1

Balabdaoui and Wellner

20

where a∗j ∈ [aj−1 , aj ] and a∗∗ j ∈ [aj , aj+1 ], and hence δj =

where

f (a∗j+1 ) − f (a∗j ) 1 + 2 2(f (a∗j ) + f (a∗j+1 ))

=

f ′ (a∗∗ 1 j ) (a∗∗ − a∗j ) + ∗ 2 2(f (aj ) + f (a∗j+1 )) j

=

∗ f ′ (a∗∗ a∗∗ 1 1 j ) j − aj ∆j a = + Mj ∆j a + ∗ ∗ 2 2(f (aj ) + f (aj+1 )) aj − aj−1 2

∗ a∗∗ f ′ (a∗∗ j − aj j ) |Mj | = 2(f (a∗j ) + f (a∗j+1 )) aj − aj−1

|f ′ (aj−1 )| aj+1 − aj−1 4f (aj+1 ) aj − aj−1   |f ′ (aj−1 )| f (aj−1 ) aj+1 − aj ≤ +1 4f (aj−1 ) f (aj+1 ) aj − aj−1 √ |f ′ (aj−1 )| γ1 R ≤ 2 · 2 · ( 2 + 1), for kn > 5˜ 4f (aj−1 ) √ |f ′ (aj−1 )| . = ( 2 + 1) f (aj−1 ) ≤

• Approximation of f ′′ (ξ1,j ) and f ′′ (ξ2,j ): Define ǫ1,j and ǫ2,j by ǫ1,j = f ′′ (ξ1,j ) − f ′′ (aj−1 ), and ǫ2,j = f ′′ (ξ2,j ) − f ′′ (aj ). By uniform continuity of f (2) = f ′′ on the compact set [0, τ ], for every ǫ > 0 there exists an η = ηǫ > 0 such that |x − y| < η implies |f ′′ (x) − f ′′ (y)| < ǫ. Fix ǫ > 0 (to be chosen later). We have ξ1,j , ξ2,j ∈ [aj−1 , aj+1 ], where, by the proof of Lemma 4.1, if kn > 5˜ γ1 R, √ 1 ( 2 + 1) ∗ kn f (aj ) √ 1 ≤ ( 2 + 1) kn f (τ ) √ ( 2 + 1)R . ≤ kn   √ Thus, if we choose kn such that kn > max 5˜ γ1 R, ( 2 + 1)/ηR , then aj+1 − aj−1 < η for all j = 1, . . . , k and furthermore n o max |f ′′ (ξ1,j ) − f ′′ (aj−1 )|, |f ′′ (ξ2,j ) − f ′′ (aj−1 )| < ǫ, for j = 1, . . . , k, aj+1 − aj−1 = aj+1 − aj + aj − aj−1 ≤

or, equivalently, max{|ǫ1,j |, |ǫ2,j |} < ǫ, j = 1, . . . , k. • Expanding ∆j+1 a around ∆j a: We have

∆j+1 a = aj+1 − aj = aj − aj−1 + [aj+1 − aj − (aj − aj−1 )]   aj+1 − aj = ∆j a + ∆j a − 1 = ∆j a + ∆j a ǫ3,j aj − aj−1

A Kiefer–Wolfowitz theorem

21

where ǫ3,j = =

f (a∗j ) f (a∗j ) − f (a∗j+1 ) aj+1 − aj −1= − 1 = aj − aj−1 f (a∗j+1 ) f (a∗j+1 ) −f ′ (a∗∗ j ) ∗ (a − a∗j ). ∗ f (aj+1 ) j+1

Thus, |ǫ3,j | ≤

|f ′ (aj−1 )| (aj+1 − aj−1 ) f (aj+1 )

! 1 1 + kn f (a∗j ) kn f (a∗j+1 )  2 |f ′ (aj−1 )| 1 |f ′ (aj−1 )| f (aj−1 ) 1 ≤2 2 =2 2 f (aj+1 ) kn f (aj−1 ) f (aj+1 ) kn 1 |f ′ (aj−1 )| 1 |f ′ (aj−1 )| 1 = 32 2 ≤ 32˜ γ1 . ≤ 2 · 24 2 f (aj−1 ) kn f (aj−1 ) kn kn |f ′ (aj−1 )| = f (aj+1 )

Above, we have used the fact that kn > 5˜ γ1 R to be able to use the inequality f (aj−1 )/f (aj+1 ) < 22 . Now, expansion of βj yields, after straightforward algebra, i h 24βj = − 2Mj f ′′ (aj−1 )(∆j a)4     h i 1 1 3 + ǫ1,j + Mj ∆j a (−∆j a) + ǫ2,j − Mj ∆j a (∆j a)3 2 2  i h 1 − Mj ∆j a (3 + 3ǫ3,j + ǫ23,j )(f ′′ (aj−1 ) + ǫ2,j ) ǫ3,j (∆j a)3 + 2 = T1,j + T2,j + T3,j where √ |f ′ (aj−1 )| ′′ |T1,j | 1 ′′ = 2|M |f (a )(∆ a) ≤ 2( f (aj−1 ) 2 + 1) j j−1 j 3 (∆j a) f (aj−1 ) kn f (a∗j ) √ |f ′ (aj−1 )| ′′ 1 ≤ 4( 2 + 1) 2 f (aj−1 ) f (aj−1 ) kn √ √ 1 1 ≤ 4( 2 + 1)˜ ≤ 4( 2 + 1)˜ γ1 f¯j′′ γ1 γ2 f (τ )3 kn kn √ 1 −1 −3 ≤ 2 ( 2 + 1)˜ γ1 γ2 τ ≡ M1 , kn since f (τ ) ≤ (2τ )−1 by (3.1), page 1669, Groeneboom, Jongbloed and Wellner [11], |T2,j | ≤2 (∆j a)3

! √ γ1 1 2( 2 + 1)˜ ǫ≤2 + 2 kn

! √ 1 2( 2 + 1) ǫ = M2 ǫ, + 2 5R

Balabdaoui and Wellner

22

and |T3,j | ≤ (∆j a)3 ≤ ≤

! √  96˜ γ1 322 γ˜12 1 2( 2 + 1)˜ γ1 1 ′′ + + 3+ (f j + ǫ) 2 kn kn kn2 kn ! √  1 2( 2 + 1) 96 1 322 3+ 2γ2 f (τ )3 + + 2 5R 5R 25R2 kn ! √  96 1 1 322 1 2( 2 + 1) 3+ 2−2 γ2 τ −3 = M3 + + 2 5R 5R 25R2 kn kn

if we choose ǫ < γ2 f (τ )3 = sup0


∆i a ∆j a

3

≤ max

1≤i≤k



f (a∗j ) f (a∗i )

3

≤



f (aj−1 ) f (τ )

where f (aj−1 ) f (aj−1 ) f (aj ) f (ak−1 ) f (aj−1 ) = = · ··· f (τ ) f (ak ) f (aj ) f (aj+1 ) f (ak ) and, for l = 0, . . . , k − 1, f (al ) f (al ) − f (al+1 ) =1+ f (al+1 ) f (al+1 ) −f ′ (a∗l ) (al+1 − al ), a∗l ∈ [al , al+1 ] =1+ f (al+1 ) 1 −f ′ (a∗l ) , a∗∗ =1+ l ∈ [al , al+1 ] ∗∗ f (al+1 )f (al ) kn 1 −f ′ (al ) ≤1+ ∗∗ f (al+1 )f (al ) kn f 2 (al ) 1 −f ′ (al ) ∗∗ 2 f (al ) f (al+1 )f (al ) kn γ˜1 1 ≤1+ , if kn > 5˜ γ1 R. 4 kn

=1+

3

A Kiefer–Wolfowitz theorem

23

Hence,

(24)

3(kn +2−j)  3(kn +2)  γ˜1 1 γ˜1 1 |a|3 ≤ 1+ ≤ 1+ (∆j a)3 4 kn 4 kn  3(kn +2)  6  3kn γ˜1 1 3˜ γ1 1 γ˜1 1 ≤ 1+ 1+ = 1+ 4 kn 4 kn 4 3kn 3kn  3˜ γ1 1 ≤ 2e3˜γ1 /4 ≤2 1+ 4 3kn

if kn ≥ γ˜1 /(4(21/6 −1)) where we used log(1+x) ≤ x for x > 0 in the last inequality. Combining (23) with (24), it follows that if we choose o n √ kn > max 5˜ γ1 R, γ˜1 /(4(21/6 − 1)), ( 2 + 1)/ηR then

  |tj − rj | 1 3˜ γ1 /4 (M1 + M3 ) ≤ 4e + M2 ǫ ∆j a = o(∆j a) (∆j a)3 kn or |tj − rj | = o(1) (∆j a)4 where o(1) is uniform in j. Lemma 4.3. Under the hypotheses of Theorem 2.2, P r |Tj − tj − (Rj − rj )| ≥

δn p3n




(100)−1 nδn2 f 2 (a∗j )p3n ≤ 4 exp − 1 + (1/30)pn δn f (a∗j )

Proof. Write Wj ≡ Tj − tj − (Rj − rj )  (Yn − Y )(1) (aj−1 ) + (Yn − Y )(1) (aj ) =− 2

(C[Yn − Y ])(1) (aj−1 ) + (C[Yn − Y ])(1) (aj ) 2  1  (1) (1) EYn −Y (aj−1 ) + EYn −Y (aj ) ∆j a ≡− 2 −

where



∆j a

Eg(1) (t) ≡ (g − C[g])(1) (t). But for g ∈ C 1 [aj−1 , aj ] with g (1) of bounded variation, ′

g(t) = g(aj−1 ) + g (aj−1 )(t − aj−1 ) + = Pj (t) +

Z

aj

aj−1

gu (t)dg (1) (u)

Z

t

aj−1

(t − u)dg (1) (u)

!

.

Balabdaoui and Wellner

24

where gu (t) ≡ (t−u)+ = (t−u)1[t≥u] . Since C is linear and preserves linear functions Z aj Cgu (t)dg (1) (u), C[g](t) = Pj (t) + aj−1

and this yields Eg (t) =

Z

Eg(1) (t) =

Z

aj

aj−1

and

Egu (t)dg (1) (u)

aj

aj−1

Eg(1) (t)dg (1) (u). u

Applying this second formula to g = Yn − Y yields the relation Z aj (1) Eg(1) (t)d(Fn − F )(u). EYn −Y (t) = u aj−1

R t (1) (1) Now gu is absolutely continuous with gu (t) = 0 gu (s)ds where gu (t) = 1[t≥u] , so by de Boor [5], (17) on page 56 (recalling that our C = I4 of de Boor), k = kgu(1) − (C[gu ])(1) k kEg(1) u

≤ (19/4)dist(gu(1) , $3 ) ≤ (19/4)dist(gu(1) , $2 ) ≤ (19/4)ω(gu(1), |a|) ≤ (19/4) ≤ 5. (1)

Thus the functions (u, t) 7→ Egu (t)∆j a are bounded by a constant multiple of ∆j a, (1) while the functions hj,l (u) = Egu (al )1[aj−1 ,aj ] (u)∆j a, l ∈ {j − 1, j} satisfy Z aj (19/4)2 f (u)du ≤ 52 (∆j a)3 f (aj−1 ) V ar[hj,l (X)] ≤ (∆j a)2 ≤

aj−1 3 2 ∗ 50pn /f (aj )

for k ≥ 5˜ γ1 (F, τ )R as in the proof of Lemma 3.1 in section 3. By applying Bernstein’s inequality much as in the proof of Lemma 3.1 we find that   (1) P r |EYn −Y (al )| > δn p3n ! nδn2 p6n /2 ≤ 2 exp − 50p3n /f (a∗j )2 + pn (5/3)δn p3n /f (a∗j ) ! nδn2 f 2 (a∗j )p3n = 2 exp − 100 + (10/3)pnf (a∗j )δn ! (100)−1 nδn2 f 2 (a∗j )p3n . = 2 exp − 1 + (1/30)pnδn f (a∗j ) Thus it follows that
P r |Wj | > δn p3n   (1) ≤ P r |EYn −Y (aj−1 )| > δn p3n   (1) + P r |EYn −Y (aj )| > δn p3n ! (100)−1 nδn2 f 2 (a∗j )p3n ≤ 4 exp − . 1 + (1/30)pn δn f (a∗j )

A Kiefer–Wolfowitz theorem

25

This completes the proof of the claimed bound. Lemma 4.4. Let R(s, t) be defined by R(s, t) ≡ P hs,t

1 = (F (t) + F (s))(t − s) − 2

Z

t

F (u)du,

s

0 ≤ s ≤ t < ∞.

Then (25)

R(s, t)



≤ ≥

1 ′ 12 f (s)(t 1 ′ 12 f (s)(t

− s)3 + − s)3 +

1 24 1 24

sups≤x≤t f ′′ (x)(t − s)4 inf s≤x≤t f ′′ (x)(t − s)4 .

Remark. It follows from the Hadamard-Hermite inequality that for F concave, R(s, t) ≤ 0 for all s ≤ t; see e.g. Niculescu and Persson [19], pages 50 and 62-63 for an exposition and many interesting extensions and generalizations. Lemmas A4 and A5 give additional information under the added hypotheses that F (2) exists and F (1) is convex. Proof. Since gs (t) ≡ R(s, t) has first three derivatives given by 1 1 d Rs (t) = f (t)(t − s) + (F (t) + F (s) − F (t)) dt 2 2 1 1 t=s = f (t)(t − s) − (F (t) − F (s)) = 0, 2 2 1 1 d2 t=s gs(2) (t) = 2 Rs (t) = f ′ (t)(t − s) + (f (t) − f (t)) = 0, dt 2 2 d3 1 1 gs(3) (t) = 3 Rs (t) = f ′′ (t)(t − s) + f ′ (t), dt 2 2 gs(1) (t) =

we can write R(s, t) as a Taylor expansion with integral form of the remainder: for s < t, 1 R(s, t) = gs (t) = gs (s) + gs′ (s)(t − s) + gs′′ (s)(t − s)2 2! Z 1 t (3) + g (x)(t − x)2 dx 2! s s Z  1 1 t  1 ′′ f (x)(x − s) + f ′ (x) (t − x)2 dx =0+ 2! s 2 2 Z Z t 1 t ′ 1 = f (x)(t − x)2 dx + f ′′ (x)(x − s)(t − x)2 dx 4 s 4 s Z 1 t ′ = {f (s) + f ′′ (x∗ )(x − s)}(t − x)2 dx 4 s Z 1 t ′′ f (x)(x − s)(t − x)2 dx + 4 s Z 1 1 t ′′ ∗ = f ′ (s)(t − s)3 + {f (x ) + f ′′ (x)}(x − s)(t − x)2 dx 12 4 s where |x∗ − x| ≤ |x − s| for each x ∈ [s, t]. Since we find that the inequalities (25) hold.

Rt

s (x

− s)(t − x)2 dx = (t − s)4 /12

Balabdaoui and Wellner

26

Lemma 4.5. Let rn,i ≡ P (hai−1 ,ai ) = R(ai−1 , ai ), i = j, j + 1, f ′′j = inf t∈[aj−1 ,aj ] ′′

f ′′ (t) and f j = supt∈[aj−1 ,aj ] f ′′ (t) . Then there exists a∗j ∈ [aj−1 , aj ] = Ij such that rn,j rn,j+1 1 1 ′′ − ≤ − f ′′ (a∗j )∆j a + (f j ∆j a − f ′′j+1 ∆j+1 a). (∆j a)3 (∆j+1 a)3 12 24 Proof. In view of (25), we have  1 ′ 1 supx∈Ij f ′′ (x)(∆j a)4 ≤ 12 f (aj−1 )(∆j a)3 + 24 rn,j 1 ′ 1 3 ≥ 12 f (aj−1 )(∆j a) + 24 inf x∈Ij f ′′ (x)(∆j a)4 ,  1 ′ 1 supx∈Ij+1 f ′′ (x)(∆j+1 a)4 ≤ 12 f (aj )(∆j+1 a)3 + 24 rn,j+1 1 ′ 1 3 ≥ 12 f (aj )(∆j+1 a) + 24 inf x∈Ij+1 f ′′ (x)(∆j+1 a)4 , and hence rn,j rn,j+1 − (∆j a)3 (∆j+1 a)3 1 1 1 1 ′ f (aj−1 ) + sup f ′′ (x)∆j a − f ′ (aj ) − inf f ′′ (x)∆j+1 a ≤ 12 24 x∈Ij 12 24 x∈Ij+1 1 1 ′′ = − f ′′ (a∗j )∆j a + (f j ∆j a − f ′′j+1 ∆j+1 a), where a∗j ∈ Ij . 12 24 5. Appendix 2: A “modernized” proof of Kiefer and Wolfowitz [14] (k)

Define the following interpolated versions of F and Fn . For k ≥ 1, let aj ≡ aj ≡ F −1 (j/k) for j = 1, . . . , k − 1, and set a0 ≡ α0 (F ) and ak ≡ α1 (F ). Using the notation of de Boor [5], Chapter III, let L(k) = I2 F be the piecewise linear and continuous function on R satisfying (k)

(k)

L(k) (aj ) = F (aj ),

j = 0, . . . , ak .

(k)

Similarly, define Ln ≡ Ln = I2 Fn ; thus (k) L(k) (x) − F (aj )] n (x) = Fn (aj ) + k{Fn (aj+1 ) − Fn (aj )}[L

for aj ≤ x ≤ aj+1 , j = 0, . . . , ak . We will eventually let k = kn and then write pn = 1/kn (so that F (aj+1 ) − F (aj ) = 1/kn = pn ). The following basic lemma due to Marshall [17] plays a key role in the proof. Lemma 5.1 (Marshall [17]). Let Ψ be convex on [0, 1], and let Φ be a continuous real-valued function on [0, 1]. Let Φ(x) = sup{h(x) : h is convex and h(z) ≤ Φ(z) for all z ∈ [0, 1]}. Then sup |Φ(x) − Ψ(x)| ≤ sup |Φ(x) − Ψ(x)|.

0≤x≤1

0≤x≤1

Proof. Note that for all y ∈ [0, 1], either Φ(y) = Φ(y), or y is an interior point of a closed interval I over which Φ is linear. For such an interval, either supx∈I |Φ(x) −

A Kiefer–Wolfowitz theorem

27

Ψ(x)| is attained at an endpoint of I (where Φ = Φ), or it is attained at an interior point, where Ψ < Φ. Since Φ ≤ Φ on [0, 1], it follows that sup |Φ(x) − Ψ(x)| ≤ sup |Φ(x) − Ψ(x)|. x∈I

x∈I

Here is a second proof (due to Robertson, Wright and Dykstra [21], page 329) that does not use continuity of Φ. Let ǫ ≡ kΦ − Ψk∞ . Then Ψ − ǫ is convex, and Ψ(x) − ǫ ≤ Φ(x) for all x. Thus for all x Φ(x) ≥ Φ(x) ≥ Ψ(x) − ǫ, and hence ǫ ≥ Φ(x) − Ψ(x) ≥ Φ(x) − Ψ(x) ≥ −ǫ for all x. This implies the claimed bound. Main steps: A. By Marshall’s lemma, for any concave function h, kFbn − hk ≤ kFn − hk. (k ) B. PF (An ) ≡ PF {Ln n is concave on [0, ∞)} ր 1 as n → ∞ if kn ≡ (C0 β1 (F )× 1/3 n/ log n) for some absolute constant C0 . C. On the event An , it follows from Marshall’s lemma (step A) that n) n) + L(k − Fn k kFbn − Fn k = kFbn − L(k n n (kn ) (kn ) ≤ kFn − Ln k + kLn − Fn k

n) = 2kFn − L(k k n (kn ) = 2kFn − Ln − (F − L(kn ) ) + F − L(kn ) )k

n) ≤ 2kFn − L(k − (F − L(kn ) )k + 2kF − L(kn ) k n ≡ 2(Dn + En ).

D. Dn is handled by a standard “oscillation theorem”; En is handled by an analytic (deterministic) argument. Proof of (1) assuming B holds. Using the notation of de Boor [5], chapter III, we have Fn − F − (Ln − L) = Fn − F − I2 (Fn − F ). But by (18) of de Boor [5], page 36, kg − I2 gk ≤ ω(g; |a|) where ω(g; |a|) is the oscillation modulus of g with maximum comparison distance |a| = maxj ∆aj (and note that de Boor’s proof does not involve continuity of g). Thus it follows immediately that Dn ≡ kFn − F − (Ln − L)k = kFn − F − I2 (Fn − F )k d

≤ ω(Fn − F ; |a|) = n−1/2 ω(Un ; pn )

√ where Un ≡ n(Gn − I) is the empirical process of n i.i.d. Uniform(0, 1) random variables. From Stute’s theorem p (see e.g. Shorack and Wellner [22], Theorem 14.2.1, page 542), lim sup ω(Un ; pn )/ 2pn log(1/pn ) = 1 almost surely if pn → 0, npn → ∞ and log(1/pn )/npn → 0. Thus we conclude that p kFn − F − (Ln − L)k = O(n−1/2 pn log(1/pn )) = O((n−1 log n)2/3 )

Balabdaoui and Wellner

28

almost surely as claimed. To handle En , we use the bound given by de Boor [5], page 31, (2): kg − I2 gk ≤ 8−1 |a|2 kg ′′ k. Applying this to g = F , I2 g = L(k) yields kF − L(k) k = kF − I2 F k ≤

1 2 ′′ |a| kF k 8

1 ≤ γ1 (F )p2n = O((n−1 log n)2/3 ). 8 Combining the results for Dn and En yields the stated conclusion. It remains to show that B holds. To do this we use the following lemma. Lemma 5.2. If pn → 0 and δn → 0, then for the uniform(0, 1) d.f. F = I, 1 P (|Gn (pn ) − pn | ≥ δn pn ) ≤ 2 exp(− npn δn2 (1 + o(1))) 2 where the o(1) term depends only on δn . Proof. From Shorack and Wellner [22], Lemma 10.3.2, page 415,   Gn (t) P (Gn (pn )/pn ≥ λ) ≤ P sup ≥ λ ≤ exp(−npn h(λ)) t pn ≤t≤1 where h(x) = x(log x − 1) + 1. Hence   Gn (pn ) − pn ≥ λ ≤ exp(−npn h(1 + λ)) P pn where h(1 + λ) ∼ λ2 /2 as λ ↓ 0, by Shorack and Wellner [22], (11.1.7), page 44. Similarly, using Shorack and Wellner [22], (10.3.6) on page 416,     1 pn − Gn (pn ) pn ≤ exp(−npn h(1 − λ)) P ≥λ =P ≥ pn Gn (pn ) 1−λ where h(1 − λ) ∼ λ2 /2 as λ ց 0. Thus the conclusion follows with o(1) depending only on δn . Here is the lemma which is used to prove B. Lemma 5.3. If β1 (F ) > 0 and γ1 (F ) < ∞, then for kn large, 1 − P (An ) ≤ 2kn exp(−nβ12 (F )/80kn3 ). Proof. For 1 ≤ j ≤ kn , write Tn,j ≡ Fn (aj ) − Fn (aj−1 ),

∆j a ≡ aj − aj−1 .

(k )

By linearity of Ln n on the sub-intervals [aj−1 , aj ], An =

k\ n −1  j=1

Tn,j Tn,j+1 ≥ ∆j a ∆j+1 a



≡

k\ n −1

Bn,j .

j=1

Suppose that (26)

|Tn,i − 1/kn | ≤ δn /kn ,

i = j, j + 1;

and

∆j+1 a ≥ 1 + 3δn . ∆j a

A Kiefer–Wolfowitz theorem

Then Tn,j ≥

δn 1 − δn 1 − = , kn kn kn

Tn,j+1 ≤

29

1 + δn , kn

and it follows that for δn ≤ 1/3 Tn,j

∆j+1 a 1 − δn 1 + δn 1 − δn (1 + 3δn ) ≥ ≥ Tn,j+1 . ≥ ∆j a kn kn 1 − δn

[1 + 3δ ≥ (1 + δ)/(1 − δ) iff (1 + 2δ − 3δ 2 ) ≥ 1 + δ iff δ − 3δ 2 ≥ 0 iff 1 − 3δ ≥ 0.] Now the ∆ part of (26) holds for 1 ≤ j ≤ kn − 1 provided δn ≤ β1 (F )/6kn < 1/3. Proof: Since f′ d −1 1 d2 −1 F (t) = − 3 (F −1 (t)) F (t) = and −1 2 dt f (F (t)) dt f we can write ∆j+1 a = F

−1

j+1 j 1 1 ( ) − F −1 ( ) = kn−1 + k k f (aj ) 2kn2



−f ′ (ξ) f 3 (ξ)



for some aj ≤ ξ ≤ aj+1 , and ∆j a ≤ kn−1

1 . f (aj )

Combining these two inequalities yields   ∆j+1 a −f ′ (ξ) ≥ 1 + (2kn )−1 f (aj ) ∆j a f 3 (ξ)   1 −f ′ (ξ) 1 ≥ 1+ β1 (F ) ≥1+ 2kn f 2 (ξ) 2kn = 1 + 3δn if δn ≡ β1 (F )/(6kn ). Thus we conclude that k[ n −1

1 − P (An ) = P ( ≤

j=1

kX n −1 j=1

c Bn,j )≤

kX n −1

c P (Bn,j )

j=1

2P (|Tn,j − 1/kn | > δn /kn )

≤ kn 4 exp(−2−1 npn δn2 1 + o(1))) = 4kn exp(−nβ12 (F )/80kn3 ). by using Lemma 5.2 and for kn sufficiently large (so that (1 + o(1)) ≥ 72/80). Putting these results together yields Theorem 1.1. Acknowledgments. The second author owes thanks to Lutz D¨ umbgen and Kaspar Rufibach for the collaboration leading to the analogue of Marshall’s lemma which is crucial for the development here. The second author thanks Piet Groeneboom for many stimulating discussions about estimation under shape constraints, and particularly about estimation of convex densities, over the past fifteen years.

30

Balabdaoui and Wellner

References [1] Balabdaoui, F. and Rufibach, K. (2007). A second marshall inequality in convex estimation. Statist. and Probab. Lett. To appear. [2] Balabdaoui, F. and Wellner, J. A. (2004). Estimation of a k-monotone density, part 1: characterizations, consistency, and minimax lower bounds. Technical report, Department of Statistics, University of Washington. [3] Balabdaoui, F. and Wellner, J. A. (2004). Estimation of a k-monotone density, part 4: limit distribution theory and the spline connection. Technical report, Department of Statistics, University of Washington. [4] Birkhoff, G. and de Boor, C. (1964). Error bounds for spline interpolation. J. Math. Mech. 13 827–835. MR0165294 [5] de Boor, C. (2001). A Practical Guide to Splines. Springer, New York. [6] Dubeau, F. and Savoie, J. (1997). Best error bounds for odd and even degree deficient splines. SIAM J. Numer. Anal. 34 1167–1184. MR1451119 ¨mbgen, L., Rufibach, K. and Wellner, J. A. (2007). Marshall’s lemma [7] Du for convex density estimation. In Asymptotics, Particles, Processes, and Inverse Problems, IMS Lecture Notes Monogr. Ser. Inst. Math. Statist., Beachwood, OH. To appear. [8] Durot, C. and Tocquet, A.-S. (2003). On the distance between the empirical process and its concave majorant in a monotone regression framework. Ann. Inst. H. Poincar´e Probab. Statist. 39 217–240. MR1962134 [9] Groeneboom, P. and Jongbloed, G. (1995). Isotonic estimation and rates of convergence in Wicksell’s problem. Ann. Statist. 23 1518–1542. MR1370294 [10] Groeneboom, P., Jongbloed, G. and Wellner, J. A. (2001). A canonical process for estimation of convex functions: the “invelope” of integrated Brownian motion +t4 . Ann. Statist. 29 1620–1652. MR1891741 [11] Groeneboom, P., Jongbloed, G. and Wellner, J. A. (2001). Estimation of a convex function: characterizations and asymptotic theory. Ann. Statist. 29 1653–1698. MR1891742 [12] Hall, C. A. (1968). On error bounds for spline interpolation. J. Approximation Theory 1 209–218. MR0239324 [13] Hall, C. A. and Meyer, W. W. (1976). Optimal error bounds for cubic spline interpolation. J. Approximation Theory 16 105–122. MR0397247 [14] Kiefer, J. and Wolfowitz, J. (1976). Asymptotically minimax estimation of concave and convex distribution functions. Z. Wahrsch. Verw. Gebiete 34 73–85. MR0397974 [15] Kiefer, J. and Wolfowitz, J. (1977). Asymptotically minimax estimation of concave and convex distribution functions. II. In Statistical Decision Theory and Related Topics. II (Proc. Sympos., Purdue Univ., Lafayette, Ind., 1976) 193–211. Academic Press, New York. MR0443202 ¨, H. P. (2006). The limit process of the dif[16] Kulikov, V. N. and Lopuhaa ference between the empirical distribution function and its concave majorant. Statist. Probab. Lett. 76 1781–1786. MR2274141 [17] Marshall, A. W. (1970). Discussion on Barlow and van Zwet’s paper. In Nonparametric Techniques in Statistical Inference (M. L. Puri, ed.). Proceedings of the First International Symposium on Nonparametric Techniques held at Indiana University, June 1969 174–176. Cambridge University Press, London. [18] Millar, P. W. (1979). Asymptotic minimax theorems for the sample distribution function. Z. Wahrsch. Verw. Gebiete 48 233–252. MR0537670

A Kiefer–Wolfowitz theorem

31

[19] Niculescu, C. P. and Persson, L.-E. (2006). Convex Functions and Their Applications. Springer, New York. MR2178902 ¨rnberger, G. (1989). Approximation by Spline Functions. Springer, [20] Nu Berlin. MR1022194 [21] Robertson, T., Wright, F. T. and Dykstra, R. L. (1988). Order Restricted Statistical Inference. Wiley, Chichester. MR0961262 [22] Shorack, G. R. and Wellner, J. A. (1986). Empirical Processes with Applications to Statistics. Wiley, New York. MR0838963 [23] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York. [24] Wang, J.-L. (1986). Asymptotically minimax estimators for distributions with increasing failure rate. Ann. Statist. 14 1113–1131. MR0856809 [25] Wang, X. and Woodroofe, M. (2007). A Kiefer–Wolfowitz comparison theorem for Wicksell’s problem. Ann. Statist. 35. To appear. [26] Wang, Y. (1994). The limit distribution of the concave majorant of an empirical distribution function. Statist. Probab. Lett. 20 81–84. MR1294808

IMS Lecture Notes–Monograph Series Asymptotic: Particles, Processes and Inverse Problems Vol. 55 (2007) 32–64 c Institute of Mathematical Statistics, 2007
DOI: 10.1214/074921707000000265

Model selection for Poisson processes Lucien Birg´ e1 Universit´ e Paris VI Abstract: Our purpose in this paper is to apply the general methodology for model selection based on T-estimators developed in Birg´e [Ann. Inst. H. Poincar´ e Probab. Statist. 42 (2006) 273–325] to the particular situation of the estimation of the unknown mean measure of a Poisson process. We introduce a Hellinger type distance between finite positive measures to serve as our loss function and we build suitable tests between balls (with respect to this distance) in the set of mean measures. As a consequence of the existence of such tests, given a suitable family of approximating models, we can build T-estimators for the mean measure based on this family of models and analyze their performances. We provide a number of applications to adaptive intensity estimation when the square root of the intensity belongs to various smoothness classes. We also give a method for aggregation of preliminary estimators.

1. Introduction This paper deals with the estimation of the mean measure µ of a Poisson process X on X . More precisely, we develop a theoretical, but quite general method for estimating µ by model selection with applications to adaptive estimation and aggregation of preliminary estimators. The main advantage of the method is its generality. We do not make any assumption on µ apart from the fact that it should be finite and we allow arbitrary countable families of models provided that each model be of finite metric dimension, i.e. is not too large in a suitable sense to be explained below. We do not know of any other estimation method allowing to deal with model selection in such a generality and with as few assumptions. The main drawback of the method is its theoretical nature, effective computation of the estimators being typically computationally too costly for permitting a practical implementation. In order to give a more precise idea of what this paper is about, we need to start by recalling a few well-known facts about Poisson processes that can, for instance, be found in Reiss [29]. 1.1. The basics of Poisson processes Let us denote by Q+ (X ) the cone of finite positive measures on the measurable space (X , E). Given an element µ ∈ Q+ (X ), a Poisson process on X with mean measure µ is a point process X = {X1 , . . . , XN } on X such that N has a Poisson distribution with parameter µ(X ) and, conditionally on N , the Xi are i.i.d. with distribution µ1 = µ/µ(X ). Equivalently, the Poisson process can be viewed as
N a random measure ΛX = i=1 δXi , δx denoting the Dirac measure concentrated at the point x. Then, whatever the partition A1 , . . . , An of X , the n random variables ΛX (Ai ) are independent with Poisson distributions and respective parameters 1 UMR 7599 “Probabilit´ es et mod`eles al´eatoires” Laboratoire de Probabilit´es, boˆıte 188, Universit´ e Paris VI, 4 Place Jussieu, F-75252 Paris Cedex 05, France, e-mail: [email protected] AMS 2000 subject classifications: Primary 62M30, 62G05; secondary 62G10, 41A45, 41A46. Keywords and phrases: adaptive estimation, aggregation, intensity estimation, model selection, Poisson processes, robust tests.

32

Model selection for Poisson processes

33

µ(Ai ) and this property characterizes a Poisson process. We shall denote by Qµ the distribution of a Poisson process with mean measure µ on X . We recall that, for any nonnegative measurable function φ on (X , E), E

(1.1)



N 



φ(Xi ) =

i=1



φ(x) dµ(x) X

and (1.2)

E

N 



φ(Xi ) = exp

i=1



X

 [φ(x) − 1] dµ(x) .

If µ, ν ∈ Q+ (X ) and µ  ν, then Qµ  Qν and (1.3)

N  dµ dQµ (Xi ), (X1 , . . . , XN ) = exp[ν(X ) − µ(X )] dQν dν i=1

with the convention that

0

i=1 (dµ/dν)(Xi )

= 1.

1.2. Introducing our loss function From now on, we assume that we observe a Poisson process X on X with unknown mean measure µ ∈ Q+ (X ) so that µ always denotes the parameter to be estimated. For this, we use estimators µ ˆ(X) with values in Q+ (X ) and measure their µ(X), µ) for q ≥ 1, where H is a suitable performance via the loss function H q (ˆ distance on Q+ (X ). To motivate its introduction, let us recall some known facts. The Hellinger distance h between two probabilities P and Q defined on the same space and their Hellinger affinity ρ are given respectively by     2 1 √ dP − dQ , ρ(P, Q) = dP dQ = 1 − h2 (P, Q), (1.4) h2 (P, Q) = 2 where dP and dQ denote the densities of P and Q with respect to any dominating measure, the result being independent of the choice of such a measure. If X1 , . . . , Xn are i.i.d. with distribution P on X and Q is another distribution, it follows from an exponential inequality that, for all x ∈ R,  n  

     dQ P (Xi ) ≥ 2x ≤ exp n log ρ(P , Q) − x log dP i=1     (1.5) ≤ exp nh2 P , Q − x ,

which provides an upper bound for the errors of likelihood ratio tests. In particular, if µ and µ
are two elements in Q+ (X ) dominated by some measure λ, it follows from (1.3) and (1.2) that the Hellinger affinity ρ(Qµ , Qµ
) between µ and µ
is given by (1.6)

ρ(Qµ , Qµ
) =

 

  dQµ dQµ
dQλ = exp −H 2 (µ, µ
) , dQλ dQλ

L. Birg´ e

34

where (1.7) (1.8)

   1
µ(X ) + µ (X ) − H (µ, µ ) = (dµ/dλ)(dµ
/dλ) 2  2  1  = dµ/dλ − dµ
/dλ . 2 2




Comparing (1.8) with (1.4) indicates that H is merely the generalization of the Hellinger distance h between probabilities to arbitrary finite positive measures and the introduction of H turns Q+ (X ) into a metric space. Moreover, we derive from (1.5) with n = 1 that, when X is a Poisson process with mean measure µ on X ,      dQµ
(1.9) P log (X) ≥ 2x ≤ exp −H 2 (µ, µ
) − x . dQµ

If µ(X ) = µ
(X ) = n, then H 2 (µ, µ
) = nh2 (µ1 , µ
1 ) and (1.9) becomes a perfect analogue of (1.5). The fact that the errors of likelihood ratio tests between two probabilities are controlled by their Hellinger affinity justifies the introduction of the Hellinger distance as the natural loss function for density estimation, as shown by Le Cam [26]. It also motivates the choice of H q as a natural loss function for estimating the mean measure of a Poisson process. For simplicity, we shall first µ(X), µ)]. focus on the quadratic risk E[H 2 (ˆ 1.3. Intensity estimation A case of particular interest occurs when we have at hand a reference positive measure λ on X and we assume that µ  λ with dµ/dλ = s, in which case s is called the intensity (with respect to λ) of the process with mean measure µ. + Denoting by L+ i (λ) the positive part of Li (λ) for i = 1, 2, we observe that s ∈ L1 (λ), √ + + s ∈ L2 (λ) and µ ∈ Qλ = {µt = t · λ, t ∈ L1 (λ)}. The one-to-one correspondence + t → µt between L+ 1 (λ) and Qλ allows us to transfer the distance H to L1 (λ) which gives, by (1.8),

√ √ √    for t, u ∈ L+ (1.10) H(t, u) = H(µt , µu ) = 1/ 2  t − u 1 (λ), 2

where ·2 stands for the norm in L2 (λ). When µ = µs ∈ Qλ it is natural to estimate µ(X), µ) = H(ˆ s(X), s) it by some element µ ˆ(X) = sˆ(X) · λ of Qλ , in which case H(ˆ and our problem can be viewed as a problem of intensity estimation: design an estimator sˆ(X) ∈ L+ 1 (λ) for the unknown intensity s. From now on, given a Poisson process X with mean measure µ, we shall denote by Eµ and Pµ (or Es and Ps when µ = µs ) the expectations of functions of X and probabilities of events depending on X, respectively. 1.4. Model based estimation and model selection It is common practice to try to estimate the intensity s on X by a piecewise constant function, i.e. a histogram estimator sˆ(X) belonging to the set   D   Sm = aj 1lIj , aj ≥ 0 for 1 ≤ j ≤ D   j=1

Model selection for Poisson processes

35

of nonnegative piecewise constant functions with respect to the partition {I1 , . . . , ID } = m of X with λ(Ij ) > 0 for all j. More generally, given a finite family the D-dimensional m = {ϕ1 , . . . , ϕD } of elements of L2 (λ), we may consider  linear √ space S m generated by the ϕj and try to estimate s by some element sˆ(X) ∈ S m . This clearly leads to difficulties since S m is not a subset of L+ 2 (λ), but we shall nevertheless show that it is possible to design an estimator sˆm (X) with the property that     2  √ 2   sm (X), s) ≤ C inf t − s 2 + |m| , (1.11) Es H (ˆ t∈S m

where |m| = D stands for the cardinality of m and √ C is a universal constant. In this approach, S m should be viewed √ as a model for s, which means an approximating set since we never assume that s ∈ S m and the risk bound (1.11) has (up to the constant C) the classical structure of the sum of an approximation term inf t∈S m t− √ 2 s2 and an estimation term |m| corresponding to the number of parameters to be estimated. If we introduce a countable (here countable always means finite or countable) family of models {S m , m ∈ M} of the previous form, we would like to know to what extent it is possible to build a new estimator sˆ(X) such that (1.12)

  s(X), s) ≤ C
inf Es H 2 (ˆ

m∈M



  √ 2 inf t − s2 + |m| ,

t∈S m

for some other constant C
, i.e. to know whether one can design an estimator which realizes, up to some constant, the best compromise between the two components of the risk bound (1.11). The problem of understanding to what extent (1.12) does hold has been treated in many papers using various methods, mostly based on the minimization of some penalized criterion. A special construction based on testing has been introduced in Birg´e [9] and then applied to different stochastic frameworks. We shall show here that this construction also applies to Poisson processes and then derive the numerous consequences of this property. We shall, in particular, be able to prove the following result in Section 3.4.1 below. Theorem 1. Let λ be some positive measure on X and  · 2 denote the norm in L2 (λ). Let {S m }m∈M be a finite or countable family of linear subspaces of L2 (λ) with respective finite dimensions Dm and let {∆m }m∈M be a family of nonnegative weights satisfying (1.13)



exp[−∆m ] ≤ Σ < +∞.

m∈M

Let X be a Poisson process on X with unknown mean measure µ = µs + µ⊥ where ⊥ is orthogonal to λ. One can build an estimator µ ˆ = µ ˆ(X) = s ∈ L+ 1 (λ) and µ sˆ(X) · λ ∈ Qλ satisfying, for all µ ∈ Q+ (X ) and q ≥ 1,   Eµ H q (µ, µ ˆ) ≤ C(q) [1 + Σ]  q  (1.14)   √ µ⊥ (X ) + inf , inf  s − t2 + Dm ∨ ∆m × m∈M

with a constant C(q) depending on q only.

t∈S m

L. Birg´ e

36

When µ = µs ∈ Qλ , (1.14) becomes (1.15)

Es



 H (s, sˆ) ≤ C(q) [1 + Σ] inf q

m∈M



  √   s − t 2 + Dm ∨ ∆m inf

t∈S m

q

.

Typical examples for X and λ are [0, 1]k with the Lebesgue measure or {1; . . . ; n} with the counting measure. In this last case, the n random variables ΛX ({i}) = Ni are independent Poisson variables with respective parameters si = s(i) and observing X is equivalent to observing a set of n independent Poisson variables with varying parameters, a framework which is usually studied under the name of Poisson regression. 1.5. Model selection for Poisson processes, a brief review Although there have been numerous papers devoted to estimation of the mean measure of a Poisson process, only a few, recently, considered the problem of model selection, the key reference being Reynaud-Bouret [30] with extensions to more general processes in Reynaud-Bouret [31]. A major difference with our approach is her use of the L2 (λ)-loss, instead of the Hellinger type loss that we introduce here. It first requires that the unknown mean measure µ be dominated by λ with intensity s and that s ∈ L2 (λ). Moreover, as we shall show in Section 2.3 the use of the L2 -loss typically requires that s ∈ L∞ (λ). This results in rather complicated assumptions but the advantage of this approach is that it is based on penalized projection estimators which can be computed practically while the construction of our estimators is too computationally intensive to be implemented on a computer, as we shall explain below. The same conclusions essentially apply to all other papers dealing with the subject. The approach of Gr´egoire and Nemb´e [21], which extends previous results of Barron and Cover [8] about density estimation to that of intensities, has some similarities with ours. The paper by Kolaczyk and Nowak [25] based on penalized maximum likelihood focuses on Poisson regression. Methods which can also be viewed as cases of model selection are those based on the thresholding of the empirical coefficients with respect to some orthonormal basis. It is known that such a procedure is akin to model selection with models spanned by finite subsets of a basis. They have been considered in Kolaczyk [24], Antoniadis, Besbeas and Sapatinas [1], Antoniadis and Sapatinas [2] and Patil and Wood [28]. 1.6. An overview of the paper We already justified the introduction of our Hellinger type loss-functions by the properties of likelihood ratio tests and we shall explain, in the next section, why the more popular L2 -risk is not suitable for our purposes, at least if we want to deal with possibly unbounded intensities. To show this, we shall design a general tool for getting lower bounds for intensity estimation, which is merely a version of Assouad’s Lemma [3] for Poisson processes. We shall also show that recent results by Rigollet and Tsybakov [32] on aggregation of estimators for density estimation extend straightforwardly to the Poisson case. In Section 3, we briefly recall the general construction of T-estimators introduced in Birg´e [9] and apply it to the specific case of Poisson processes. We also provide an illustration based on nonlinear approximating models. Section 4 is devoted to various applications of our method based on families of linear models. This section essentially relies on results

Model selection for Poisson processes

37

from approximation theory about the approximation of different classes of functions (typically smoothness classes) by finite dimensional linear spaces in L2 . We also indicate how to mix different families of models and introduce an asymptotic point of view which allows to consider convergence rates and to make a parallel with density estimation. In Section 5, we deal with aggregation of estimators with some applications to partition selection for histograms. The final Section 6 is devoted to the proof of the most important technical result in this paper, namely the existence and properties of tests between balls of mean measures. This is the key argument which is required to apply the construction of T-estimators to the problem of estimating the mean measure of a Poisson process. It also has other applications, in particular to the study of Bayesian procedures as done, for instance, in Ghosal, Ghosh and van der Vaart [20] and subsequent work of van der Vaart and coauthors. 2. Estimation with L2 -loss 2.1. From density to intensity estimation A classical approach to density estimation is based on L2 -loss. We assume that the observations X1 , . . . , Xn have a density s1 with respect to some dominating measure λ and that s1 belongs to the Hilbert space L2 (λ) with scalar product ·, · and s − s1 22 ]. norm  · 2 . Given an estimator sˆ(X1 , . . . , Xn ) we define its risk by E[ˆ In this theory, a central role is played by projection estimators as defined by Cencov [14]. Model selection based on projection estimators has been considered by Birg´e and Massart [11]. A more modern treatment can be found in Massart [27]. Thresholding estimators based on wavelet expansions as described in Cohen, DeVore, Kerkyacharian and Picard [15] (see also the many further references therein) can also be viewed as special cases of those. Recently Rigollet and Tsybakov [32] introduced an aggregation method based on projection estimators. Projection estimators have the advantage of simplicity and the drawback or requiring somewhat restrictive assumptions on the density s1 to be estimated, not only that it belongs to L2 but most of the time to L∞ . As shown in Birg´e [10], Section 5.4.1, the fact that s1 belongs to L∞ is essentially a necessary condition to have a control on the L2 -risk of estimators of s1 . As indicated in Baraud and Birg´e [4] Section 4.2, there is a parallel between the estimation of a density s1 from n i.i.d. observations and the estimation of the intensity s = ns1 from a Poisson process. This suggests to adapt the known results from density estimation to intensity estimation for Poisson processes. We shall briefly explain how it works, when the Poisson process X has an intensity s ∈ L∞ (λ) with L∞ -norm s∞ . The starting point is to observe that, given an element ϕ ∈ L2 (λ), a natural 
N estimator of ϕ, s is ϕ(X) = ϕ dΛX = i=1 ϕ(Xi ). It follows from (1.1) that  (2.1) Es [ϕ(X)] = ϕ, s and Vars (ϕ(X)) = ϕ2 s dλ − ϕ, s 2 ≤ s∞ ϕ22 . Given a D-dimensional linear subspace S
of L2 (λ) with an orthonormal basis ϕ1 , . . . , ϕD , we can estimate s by the projection estimator with respect to S
:  N D   ϕj (Xi ) ϕj . sˆ(X) = j=1

i=1

L. Birg´ e

38

It follows from (2.1) that its risk is bounded by   s(X) − s22 ≤ inf
t − s22 + s∞ D. (2.2) Es ˆ t∈S

Note that sˆ(X) is not necessarily an intensity since it may take negative values. This can be fixed: replacing sˆ(X) by its positive part can only reduce the risk since s is nonnegative. 2.2. Aggregation of preliminary estimators

The purpose of this section is to extend some recent results for aggregation of density estimators due to Rigollet and Tsybakov [32] to intensity estimation. The basic tool for aggregation in the context of Poisson processes is the procedure of “thinning” which is the equivalent of sample splitting for i.i.d. observations, see for instance Reiss [29], page 68. Assume that we have at our disposal a Poisson process
N with mean measure µ: ΛX = i=1 δXi and an independent sequence (Yi )i≥1 of i.i.d. Bernoulli variables with parameter p ∈ (0, 1). Then the two random measures
N
N ΛX1 = i=1 Yi δXi and ΛX2 = i=1 (1 − Yi )δXi are two independent Poisson processes with respective mean measures pµ and (1 − p)µ. Now assume that X is a Poisson process with intensity s with respect to λ, that X1 and X2 have been derived from X by thinning and that we have at our disposal a finite family {ˆ sm (X1 ), m ∈ M} of estimators of ps based on the first process and belonging to L2 (λ). They may be projection estimators or others. These estimators span a D-dimensional linear subspace of L2 (λ) with an orthonormal basis ϕ1 , . . . , ϕD , D ≤ |M|. Working conditionally with respect to X1 , we use X2 to build a projection estimator s˜(X2 ) of (1 − p)s belonging to the linear span of the estimators sˆm (X1 ). This is exactly the method used by Rigollet and Tsybakov [32] for density estimation and the proof of their Theorem 2.1 extends straightforwardly to Poisson processes to give Theorem 2. The aggregated estimator s˜ based on the processes X1 and X2 by thinning of X satisfies (2.3)   2         s(X) − (1 − p)s22 ≤ Es  inf ps − θm sˆm (X1 )  + (1 − p)s∞ |M|. Es ˜  θ∈RM  m∈M

2

Setting sˆ(X) = s˜(X)/(1 − p) leads to   Es ˆ s(X) − s22 ≤

# $ s |M| 1 ∞ 2 ps − s ˆ + . inf E (X ) s m 1 2 (1 − p)2 m∈M 1−p

If we start with a finite family {S m , m ∈ M} of finite-dimensional linear subspaces of L2 (λ) with respective dimensions Dm , we may choose for sˆm (X1 ) the projection estimator based on S m with risk bounded by (2.2) # $ 2 Es ˆ sm (X1 ) − ps2 ≤ inf t − ps22 + ps∞ Dm = p2 inf t − s22 + ps∞ Dm . t∈S m

t∈S m

Choosing p = 1/2, we conclude that   # $ 2 s(X) − s2 ≤ inf Es ˆ inf t − s22 + 2s∞ Dm + 2s∞ |M|. m∈M

t∈S m

Model selection for Poisson processes

39

2.3. Lower bounds for intensity estimation It is rather inconvenient to get risk bounds involving the unknown and possibly very large L∞ -norm of s and this problem becomes even more serious if s does not belong to L∞ (λ). It is, unfortunately, impossible to avoid this problem when dealing with the L2 -loss. To show this, let us start with a version of Assouad’s Lemma [3] for Poisson processes. by D = Lemma 1. Let SD = {sδ , δ ∈ D} ⊂ L+ 1 (λ) be a family of intensities indexed
D {0; 1}D and ∆ be the Hamming distance on D given by ∆(δ, δ
) = j=1 |δj − δj
|. Let C be the subset of D × D defined by C = {(δ, δ
) | ∃k, 1 ≤ k ≤ D

with δk = 0, δk
= 1

and

δj = δj
for j = k}.

ˆ Then for any estimator δ(X) with values in D,   #

$ D    1 ˆ  exp −2H 2 (sδ , sδ
)  . (2.4) sup Esδ ∆ δ(X), δ ≥ 4 |C| δ∈D
(δ,δ )∈C

If, moreover, SD ⊂ L ⊂ L+ 1 (λ) and L is endowed with a metric d satisfying d2 (sδ , sδ
) ≥ θ∆(δ, δ
) for all δ, δ
∈ D and some θ > 0, then for any estimator sˆ(X) with values in L,     2  Dθ    1 s(X), s) ≥ exp −2H 2 (sδ , sδ
)  . (2.5) sup Es d (ˆ 16 |C| s∈SD
(δ,δ )∈C

Proof. To get (2.4) it suffices to find a lower bound for D ) ) # $    )ˆ ) ˆ δ = 2−D RB = 2−D Esδ ∆ δ, )δk − δk ) dQsδ , δ∈D

δ∈D

k=1

since the left-hand side of (2.4) is at least as large as the average risk RB . It follows from the proof of Lemma 2 in Birg´e [10] with n = 1 that          RB ≥ 2−D ρ2 Qsδ , Qsδ
. 1 − 1 − ρ2 Qsδ , Qsδ
≥ 2−D−1 (δ,δ
)∈C

(δ,δ
)∈C

Then (2.4) follows from (1.6) since |C| = D2D−1 . Let now sˆ(X) be an estimator ˆ s, sδ ) so that, with values in L and set δ(X) ∈ D to satisfy d(ˆ s, sδˆ) = inf δ∈D d(ˆ s, sδ ). It then follows from our assumptions that whatever δ ∈ D, d(sδˆ, sδ ) ≤ 2d(ˆ #

$     θ  1 ˆ sup Esδ d2 (ˆ s, sδ ) ≥ sup Esδ d2 sδˆ, sδ ≥ sup Esδ ∆ δ(X), δ 4 δ∈D 4 δ∈D δ∈D and (2.5) follows from (2.4).

The simplest application of this lemma corresponds to the case D = 1 which, in its simplest form, dates back to Le Cam [26]. We consider only two intensities s0 and s1 so that θ = d2 (s0 , s1 ) and (2.5) gives, whatever the estimator sˆ(X), (2.6)

   d2 (s0 , s1 )  exp −2H 2 (s0 , s1 ) . s(X), si ) ≥ max Esi d2 (ˆ i=0,1 16

Another typical application of the previous lemma to intensities on [0, 1] uses the following construction of a suitable set SD .

L. Birg´ e

40

Lemma 2. Let D be a positive integer and g be a function on R with support on [0, D−1 ) satisfying  D−1 0 ≤ g(x) ≤ 1 for all x and g 2 (x) dx = a > 0. 0

Set, for 1 ≤ j ≤ D and 0 ≤ x ≤ 1, gj (x) = g(x − D−1 (j − 1)) and, for δ ∈
D D, sδ (x) = a−1 [1 + j=1 (δj − 1/2)gj (x)]. Then sδ − sδ
22 = a−1 ∆(δ, δ
) and H 2 (sδ , sδ
) ≥ ∆(δ, δ
)/8 for all δ, δ
∈ D. Moreover,    (2.7) |C|−1 exp −2H 2 (sδ , sδ
) ≥ exp[−2/7]. (δ,δ
)∈C

Proof. The first equality isclear. Let us then observe that our assumptions on g imply that 1 − g 2 (x)/7 ≤ 1 − g 2 (x)/4 ≤ 1 − g 2 (x)/8, hence, since the functions gj have disjoint supports and are translates of g, −1

2

H (sδ , sδ
) = (2a)

D 

|δj −

δj
|

D  j=1

|δj − δj
|

D −1

0

j=1

= a−1





0

D −1

#

1 + g(x)/2 −



1 − g(x)/2

$2

dx

$ #  1 − 1 − g 2 (x)/4 dx = c∆(δ, δ
),

with 1/8 ≤ c ≤ 1/7. The conclusions follow.

Corollary 1. For each positive integer D and L ≥ 3D/2, one can find a finite set SD of intensities with the following properties: (i) it is a subset of some D-dimensional affine subspace of L2 ([0, 1], dx); (ii) sups∈SD s∞ ≤ L; (iii) for any estimator sˆ(X) with values in L2 ([0, 1], dx) based on a Poisson process X with intensity s,   s − s22 ≥ (DL/24) exp[−2/7]. (2.8) sup Es ˆ s∈SD

Proof. Let apply the construction of Lemma 2 with  us set θ = 2L/3 ≥ D and −1 D/θ 1 l , hence a = θ . This results in the set SD with sδ ∞ ≤ g(x) = [0,1/D) # $  θ 1 + (1/2) D/θ ≤ 3θ/2 = L for all δ ∈ D as required. Moreover sδ − sδ
22 =

θ∆(δ, δ
). Then we use Lemma 1 with d being the distance corresponding to the norm in L2 ([0, 1], dx) and (2.5) together with (2.7) result in (2.8).

This result implies that, if we want to use the squared L2 -norm as a loss function, whatever the choice of our estimator there is no hope to find risk bounds that are independent of the L∞ -norm of the underlying intensity, even if this intensity belongs to a finite-dimensional affine space. This provides an additional motivation for the introduction of loss functions based on the distance H. 3. T-estimators for Poisson processes 3.1. Some notations Throughout this paper, we observe a Poisson process X on X with unknown mean measure µ belonging to the metric space (Q+ (X ), H) and have at hand some ref⊥ erence measure λ on X so that µ = µs + µ⊥ with µs ∈ Qλ , s ∈ L+ 1 (λ) and µ

Model selection for Poisson processes

41

orthogonal to λ. We denote by  · i the norm in Li (λ) for 1 ≤ i ≤ ∞ and by d2 the distance corresponding to the norm  · 2 . We always denote by s the intensity of the part of µ which is dominated by λ and set s1 = s/µs (X ). We also systematically identify Qλ with L+ 1 (λ) via the mapping t → µt , writing t as a shorthand for µt ∈ Qλ . We write H(s, S
) for inf t∈S
H(s, t), a ∨ b and a ∧ b for the maximum and the minimum respectively of a and b, |A| for the cardinality of a finite set A and N = N \ {0} for the set of positive integers. In the sequel C (or C
, C1 , . . .) denote constants that may vary from line to line, the form C(a, b) meaning that C is not a universal constant but depends on some parameters a and b. 3.2. Definition and properties of T-estimators In order to explain our method of estimation and model selection, we need to recall some general results from Birg´e [9] about T-estimators that we shall specialize to the specific framework of this paper. Let (M, d) be some metric space and B(t, r) denote the open ball of center t and radius r in M . Definition 1. A subset S
of the metric space (M, d) is called a D-model with parameters η, D and B
(η, B
, D > 0) if   (3.1) |S
∩ B(t, xη)| ≤ B
exp Dx2 for all x ≥ 2 and t ∈ M.

Note that this implies that S
is at most countable. To estimate the unknown mean measure µ of the Poisson process X, we introduce a finite or countable family {Sm , m ∈ M} of D-models in (Qλ , H) with respective parameters ηm , Dm and B
and assume that (3.2)

for all m ∈ M,

2 Dm ≥ 1/2 and ηm ≥ (84Dm )/5,

and 

(3.3)

m∈M

Then we set S = (3.4)

*

m∈M

 2  exp −ηm /84 = Σ < +∞.

Sm and, for each t ∈ S,

η(t) = inf{ηm | m ∈ M and Sm  t}.

Remark. Note that if we choose for {Sm , m ∈ M} a family of D-models in (Q+ (X ), H), S is countable and therefore dominated by some measure λ that we can always take as our reference measure. This gives an a posteriori justification for the choice of a family of models Sm ⊂ Qλ . Given two distinct points t, u ∈ Qλ we define a test function ψ(X) between t and u as a measurable function from X to {t, u}, ψ(X) = t meaning deciding t and ψ(X) = u meaning deciding u. In order to define a T-estimator, we need a family of test functions ψt,u (X) between distinct points t, u ∈ S with some special properties. The following proposition, to be proved in Section 6 warrants their existence. Proposition 1. Given two distinct points t, u ∈ S there exists a test ψt,u between t and u which satisfies sup {µ∈Q+ (X ) | H(µ,µt )≤H(t,u)/4}

Pµ [ψt,u (X) = u]     ≤ exp − H 2 (t, u) − η 2 (t) + η 2 (u) /4 ,

L. Birg´ e

42

sup {µ∈Q+ (X ) | H(µ,µu )≤H(t,u)/4}

and for all µ ∈ Q+ (X ), (3.5)

Pµ [ψt,u (X) = u] ≤ exp

Pµ [ψt,u (X) = t]     ≤ exp − H 2 (t, u) − η 2 (u) + η 2 (t) /4 ,    16H 2 (µ, µt ) + η 2 (t) − η 2 (u) /4 .

To build a T-estimator, we proceed as follows. We consider a family of tests ψt,u indexed by the two-points subsets {t, u} of S with t = u that satisfy the conclusions of Proposition 1 and we set Rt = {u ∈ S, u = t | ψt,u (X) = u} for each t ∈ S. Then we define the random function DX on S by + ,  if Rt = ∅;  sup H(t, u) u∈Rt DX (t) =  0 if Rt = ∅.

We call T-estimator derived from S and the family of tests ψt,u (X) any measurable s(X)) = minimizer of the function t → DX (t) from S to [0, +∞] so that DX (ˆ inf t∈S DX (t). Such a minimizer need not exist in general but it actually exists under our assumptions. * Theorem 3. Let S = m∈M Sm ⊂ Qλ be a finite or countable family of D-models in (Qλ , H) with respective parameters ηm , Dm and B
satisfying (3.2) and (3.3). Let {ψt,u } be a family of tests indexed by the two-points subsets {t, u} of S with t = u and satisfying the conclusions of Proposition 1. Whatever µ ∈ Q+ (X ), Pµ a.s. there exists at least one T-estimator sˆ = sˆ(X) ∈ S derived fom this family of tests and any of them satisfies, for all s
∈ S,   (3.6) Pµ [H(s
, sˆ) > y] < (B
Σ/7) exp −y 2 /6 for y ≥ 4[H(µ, µs
) ∨ η(s
)]. Setting µ ˆ(X) = sˆ(X) · λ and µ = µs + µ⊥ with µs ∈ Qλ and µ⊥ orthogonal to λ, we also get q    q 
⊥ ˆ(X)) ≤ C(q)[1 + B Σ] inf H(s, Sm ) + ηm + µ (X ) (3.7) Eµ H (µ, µ m∈M

and, for intensity estimation when µ = µs ,   q (3.8) Es H q (s, sˆ(X)) ≤ C(q)[1 + B
Σ] inf {H(s, Sm ) + ηm } . m∈M

Proof. It follows from Theorem 5 in Birg´e [9] with a = 1/4, B = 1, κ = 4 and κ
= 16 that T-estimators do exist, satisfy (3.6) and have a risk which is bounded, for q ≥ 1, by   q  q 
ˆ(X)) ≤ C(q)[1 + B Σ] inf inf H(µ, µt ) ∨ ηm . (3.9) Eµ H (µ, µ m∈M

t∈Sm

In Birg´e [9], the proof of the existence of T-estimators when M is infinite was given only for the case that the tests ψt,u (X) have a special form, namely ψt,u (X) = u when γ(u, X) < γ(t, X) and ψt,u (X) = t when γ(u, X) > γ(t, X) for some suitable function γ. A minor modification of the proof extends the result to the general

Model selection for Poisson processes

43

situation based on the assumption that (3.5) holds. It is indeed enough to use (3.5) to modify the proof of (7.18) of Birg´e [9] in order to get instead Pµ [ ∃ t ∈ S with ψs
,t (X) = 1 and η(t) ≥ y] −→ 0. y→+∞

The existence of sˆ(X) then follows straightforwardly. Since H 2 (µ, µt ) = H 2 (s, t) + µ⊥ (X )/2, (3.7) follows from (3.9). It follows from (3.7) that the problem of estimating µ with T-estimators always reduces to intensity estimation once a reference measure λ has been chosen. A comparison of the risk bounds (3.7) and (3.8) shows that the performance of the estimator sˆ(X) is connected to the choice of the models in L+ 1 (λ), the component µ⊥ (X ) of the risk depending only on λ. We might as well assume that µ⊥ (X ) is known since this would not change anything concerning the performance of the T-estimators for a given λ. This is why we shall essentially focus, in the sequel, on intensity estimation. 3.3. An application to multivariate intensities Let us first illustrate Theorem 3 by an application to the estimation of the unknown intensity s (with respect to the Lebesgue measure λ) of a Poisson process on X = [−1, 1]k . For this, we introduce a family of non-linear models related to neural nets which were popularized in the 90’s by Barron [5, 6] and other authors in view of their nice approximation properties with respect to functions of several variables. These models have already been studied in detail in Sections 3.2.2 and 4.2.2 of Barron, Birg´e and Massart [7] and we shall therefore refer to this paper for their properties. We start with a family of functions φw (x) ∈ L∞ ([−1, 1]k ) indexed by a
parameter w belonging to Rk and satisfying (3.10)

|φw (x) − φw
(x)| ≤ |w − w
|1

for all x ∈ [−1, 1]k ,




where | · |1 denotes the l1 -norm on Rk . Various examples of such families are given in Barron, Birg´e and Massart [7] and one can, for instance, set φw (x) = ψ(a
x − b) with ψ a univariate Lipschitz function, a ∈ Rk , b ∈ R and w = (a, b) ∈ Rk+1 . We set M = (N \ {0, 1})3 and for m = (J, R, B) ∈ M we consider the subset of L∞ ([−1, 1]k ) defined by  )  )  J J   )
Sm = βj φwj (x) )) |βj | ≤ R and |wj |1 ≤ B for 1 ≤ j ≤ J .   ) j=1 j=1

As shown in Lemma 5 of Barron, Birg´e and Massart [7], such a model can be
approximated by a finite subset Tm . More precisely, one can find a subset Tm of Sm J(k
+1)
with cardinality bounded by [2e(2RB + 1)] and such that if u ∈ Sm , there exists some t ∈ Tm such that t − u∞ ≤ 1. Defining Sm as {t2 , t ∈ Tm }, we get the following property:

2 Lemma 3. For m = (J, R, B) ∈ (N \ {0, 1})3 , we set ηm = 42J(k
+ 1) log(RB).
Then Sm is a D-model with parameters ηm , Dm = [J(k + 1)/4] log[2e(2RB + 1)] and 1 in the metric space (L+ 1 (λ), H) and (3.2) and (3.3) are satisfied. Moreover, (λ), for any s ∈ L+ 1 √  √ 2H(s, Sm ) ≤ inf
 s − t2 + 2k/2 . (3.11) t∈Sm

L. Birg´ e

44

Proof. Since |Sm | ≤ |Tm |, to show that Sm is a D-model with the given parameters it is enough to prove, in view of (3.1), that |Tm | ≤ exp[4Dm ], which is clear. That 2 /84 ≥ Dm /5 follows from log[2e(2RB +1)] ≤ 4 log(RB) since RB ≥ 4. Moreover, ηm 2 ≥ 84J log(RB), hence since k
+ 1 ≥ 2, ηm 

m∈M

 2 .2  2      +∞ ηm −J −J  exp − n  ≤ x dx , ≤ 84 3/2 J≥2

n≥2

J≥2


so that (3.3) holds. Let now u ∈ Sm . There exists t ∈ Tm such that t − u √∞ ≤ 1, √ √ √ k/2 2 s − t ≤  s − u + 2 . Then t ∈ S and since  s − t2  2 ≤ hence  2 2 m √  s − t2 , (3.11) follows.

Let now sˆ(X) be a T-estimator derived from the family of D-models {Sm , m ∈ M}. By Theorem 3 and Lemma 3, it satisfies   2 √   2 inf
 s − t2 + 2k + ηm Es H 2 (s, sˆ(X)) ≤ C inf m∈M t∈Sm   2 √ ≤ C(k, k
) inf (3.12) inf
 s − t2 + J log(RB) . m∈M

t∈Sm


with respect to different classes The approximation properties of the models Sm of functions have√been described√in Barron, Birg´e and Massart [7]. They allow to
 s − t2 when s belongs to such classes so that corresponding bound inf t∈Sm risk bounds can be derived from (3.12).

3.4. Model selection based on linear models 3.4.1. Deriving D-models from linear spaces In order to apply Theorem 3 we need to introduce suitable families of D-models Sm in (Qλ , H) with good approximation properties with respect to the unknown s. More precisely, it follows from (3.7) and (1.10) that they should provide approximations √ + of s in L+ 2 (λ). Good approximating sets for elements of L2 (λ) are provided by approximation theory and some recipes to derive D-models from such sets have been given in Section 6 of Birg´e [9]. Most results about approximation of functions in L2 (λ) deal with finite dimensional linear spaces or unions of such spaces and their approximation properties with respect to different classes (typically smoothness classes) of functions. We therefore focus here on such linear subspaces of L2 (λ). To translate their properties in terms of D-models, we shall invoke the following proposition. Proposition 2. Let S be a k-dimensional linear subspace of L2 (λ) and δ > 0. One can find a subset S
of Qλ which is a D-model in the metric space (Qλ , H) with parameters δ, 9k and 1 and such that, for any intensity s ∈ L+ 1 (λ),    √ H(s, S
) ≤ 2.2 inf  s − t2 + δ . t∈S

Proof. Let us denote by BH and B2 the open balls in the metric spaces (L+ 1 (λ), H) and (L2 (λ), d2 ) respectively. It follows from Proposition 8 of Birg´e [9] that one can

Model selection for Poisson processes

45

find a subset T of S which is a D-model of (L2 (λ), d2 ) with parameters δ, k/2 and 1 and such that, whatever u ∈ L2 (λ), d2 (u, T ) ≤ d2 (u, S) + δ. It follows that )

√ ))   ) for r
≥ 2δ and t ∈ L2 (λ). (3.13) )T ∩ B2 t, 3r
2 ) ≤ exp 9k(r
/δ)2

Moreover, if t ∈ T , π(t) = max{t, 0} belongs to L+ 2 (λ) and satisfies d2 (u, π(t)) ≤ d2 (u, t) for any u ∈ L+ 2 (λ). We may therefore apply Proposition √ 12 of Birg´e [9] √with
2 to (M
, d) = (L2 (λ), d2 ), M0 = L+ 2 (λ), λ = 1, ε = 1/10, η = 4 2δ and r = r get a subset S of π(T ) ⊂ L+ 2 (λ) such that



√ √ (3.14) |S ∩ B2 t, r
2 | ≤ |T ∩ B2 t, 3r
2 | ∨ 1 for all t ∈ L2 (λ) and r
≥ 2δ
2 and d2 (u, S) ≤ 3.1d2 (u, T ) for all u ∈ L+ 2 (λ). Setting S = {t · λ, t ∈ S)} ⊂ Qλ and using (1.10), we deduce from (3.13) and (3.14) that   for r
≥ 2δ and µt ∈ Qλ , |S
∩ BH (µt , r
) | ≤ exp 9k(r
/δ)2

hence S
is a D-model in (Qλ , H) with parameters δ, 9k and 1, and

√ √   √   H(s, S
) ≤ 3.1/ 2 d2 s, T < 2.2 d2 s, S + δ .

2 We are now in a position to prove Theorem 1. For each m, let us fix ηm = 84[∆m ∨ (9Dm /5)] and use Proposition 2 to derive from S m a D-model Sm with parameters ηm , Dm = 9Dm and 1 which also satisfies    √ H(s, Sm ) ≤ 2.2 inf  s − t + ηm . t∈S m

2

It follows from the definition of ηm that (3.2) and (3.3) are satisfied so that Theorem 3 applies. The conclusion immediately follows from (3.7). 3.4.2. About the computation of T-estimators We already mentioned that the relevance of T-estimators is mainly of a theoretical nature because of the difficulty of their implementation. Let us give √ here a simple illustrative example based on a single linear approximating space S for s, of dimension k. To try to get a practical implementation, we shall use a simple discretization strategy. The first step is to replace S, that we identify to Rk via the choice of a basis, by θZk . This provides an η-net for Rk with respect to the the Euclidean 2 2 distance,  with η = k(θ/2) . Let us concentrate here on the case of a large value of Γ2 = s dλ in order to have a large number of observations since N has a Poisson distribution with parameter Γ2 . In particular, we shall asume that Γ2 (which plays the role of the number of observations as we shall see in Section 4.6) is much √ larger than k. It is useless, √ in such a case, to use the whole of θZk to approximate s since Γ + η). Of course, Γ is unknown, but when the closest point to s belongs to B(0, √ it is large it can be safely estimated by N in view of the concentration properties 2 of Poisson variables. Let √ N ≥ Γ /2 ≥ 2k. A reasonable √ us therefore assume that approximating set for s is therefore T = B(0, 2N + η) ∩ θZk and since our final + model √ S should√be a subset√of L2 (λ), we can take S = {t ∨ 0, t ∈ T } so that d2 ( s, S) ≤ d2 ( s, T ) ≤ d2 ( s, S + η). It follows from Lemma 5 of Birg´e [9] that - √ .k # √ $k (πe/2)k/2 2 2N + 2η √ |S| ≤ |T | ≤ √ +1
L. Birg´ e

46

 with c = πe/2 ∼ 2.07. This implies that S is a D-model with parameters 2 η, (log K)/4 and 1. In order that (3.2) be satisfied, we need that η ≥ 4.2 √log K. If 2 = 4.2k log(c( N/k + 1)), this inequality holds since η ≥ 2 k, hence we choose η  k K ≤ [c( N/k + 1)] . The number of tests required for building the T-estimator is |S|(|S| − 1) < K 2 . For N of the order of 100 and k as small as 5, K 2 is of the order of 1010 . This toy example illustrates the difficulty of implementing the algorithm. More realistic ones would be much worse. 4. Applications with linear models We now assume that µ = µs = s · λ and focus on the estimation of the intensity s by model selection, starting with linear models in L2 (λ) that possess good √ approximating properties with respect to s. 4.1. Adaptation in Besov spaces It is now well-known that wavelet bases are very good tools for representing smooth functions in L2 ([0, 1]l , dx). In particular, given a suitable wavelet basis {ϕj,k , j ≥ jl −1, k ∈ Λ(j)} with |Λ(−1)| ≤ Γ and 2jl ≤ |Λ(j)|
for all j ≥ 0 any function
∞≤ Γ2 l f ∈ L2 ([0, 1] , dx) can be written as f = j=−1 k∈Λ(j) βj,k ϕj,k . Moreover f α belongs to the Besov space Bp,∞ ([0, 1]l ) if and only if (4.1)

α+ 2l − pl

sup 2j ( j≥0



)



k∈Λ(j)

 p1

α |βj,k |p  = |f |Bp,∞ < +∞,

α and it belongs to Bp,q ([0, 1]l ) with q < +∞ if

  p1 q     j α+ l − l |βj,k |p   = |f |qBp,q < +∞. 2 ( 2 p )  α j≥0

k∈Λ(j)

Many properties of those function spaces are to be found in DeVore and Lorentz [19], DeVore [17] and H¨ ardle, Kerkyacharian, Picard and Tsybakov [22] among other references. As a consequence of Theorem 1, we can derive an adaptation result for the estimation of the intensity of a Poisson process when it belongs to some Besov space on [0, 1]l . Theorem 4. Let X be a Poisson process with unknown intensity s with respect √ to Lebesgue measure on [0, 1]l . Let us assume that s belongs to some Besov √ space α α ([0, 1]l ) for some unknown values of p > 0, α > l(1/p − 1/2)+ and | s|Bp,∞ Bp,∞ given by (4.1). One can build a T-estimator sˆ(X) such that (4.2)

#√ $2l/(2α+l)   α Es H 2 (s, sˆ) ≤ C(α, p, l) | s|Bp,∞ ∨1 .

Proof. We just use Proposition 13 of Birg´e [9] which provides suitable families α Mj (2i ) of linear approximation spaces for functions in Bp,∞ ([0, 1]l ) and use the * * family of linear spaces {S m }m∈M with M = i≥1 j≥0 Mj (2i ) provided by this

Model selection for Poisson processes

47

proposition. Then, for m ∈ Mj (2i ), Dm ≤ c1 (2i ) + c2 (2i )2jl and we choose ∆m = implies that (1.13) holds with Σ < 1. Applying Proposition 13 c3 (2i )2jl +i+j which √ s, r = 2i > α ≥ 2i−1 and q = 2, we derive from Theorem 1 of Birg´e [9] with t = √ α ∨ 1, that, if R = | s|Bp,∞  + ,  Es H 2 (s, sˆ) ≤ C inf C(α, p, l)R2 2−2jα + c4 (α)2jl . j≥0

Choosing for j the smallest integer such that 2j(l+2α) ≥ R2 leads to the result. 4.2. Anisotropic H¨ older spaces Let us recall that a function f defined on [0, 1) belongs to the H¨ older class H(α, R) with α = β + p, p ∈ N, 0 < β ≤ 1 and R > 0 if f has a derivative of order p satisfying |f (p) (x) − f (p) (y)| ≤ R|x − y|β for all x, y ∈ [0, 1). Given two multi-indices α = (α1 , . . . , αk ) and R = (R1 , . . . , Rk ) in (0, +∞)k , we define the anisotropic H¨ older class H(α, R) as the set of functions f on [0, 1)k such that, for each j and each set of k − 1 coordinates x1 , . . . , xj−1 , xj+1 , . . . , xk the univariate function y → f (x1 , . . . , xj−1 , y, xj+1 , . . . , xk ) belongs to H(αj , Rj ). Let now a multi-integer N = (N1 , . . . , Nk ) ∈ (N )k be given. To it corresponds k k the hyperrectangle j=1 [0, Nj−1 ) and the partition IN of [0, 1)k into j=1 Nj translates of this hyperrectangle. Given an integer r ∈ N and m = (N , r) we can define at the linear space S m of piecewise polynomials on the partition IN with degree k most r with respect to each variable. Its dimension is Dm = (r + 1)k j=1 Nj . Setting M = (N )k × N and ∆m = Dm , we get (1.13) with Σ depending only on k as shown in the proof of Proposition 5, page 346 of Barron, Birg´e and Massart [7]. The same proof also implies (see (4.25), page 347) the following approximation lemma. Lemma 4. Let f ∈ H(α, R) with αj = βj + pj , r ≥ max1≤j≤k pj , N = (N1 , . . . , Nk ) ∈ (N )k and m = (N , r). There exists some g ∈ S m such that f − g∞ ≤ C(k, r)

k 

−αj

Rj Nj

.

j=1

We are now in a position to state the following corollary of Theorem 1. Corollary 2. Let X be a Poisson process with unknown intensity s with respect to of linear the Lebesgue measure on [0, 1)k and sˆ be a T-estimator based on the family √ models {S m , m ∈ M} that we have previously defined. Assume that s belongs to the class H(α, R) and set 

α = k −1 If Rj ≥ R

k/(2α+k)

k  j=1

−1

αj−1 

and



R=

k 

j=1

α/k

1/αj 

Rj

for all j, then   2k/(2α+k) Es H 2 (s, sˆ) ≤ C(k, α)R .

.

L. Birg´ e

48

k/(2α+k)

Proof. If αj = βj + pj for 1 ≤ j ≤ k, let us set r = max1≤j≤k pj , η = R and define Nj ∈ N by (Rj /η)1/αj ≤ Nj < (Rj /η)1/αj + 1 so that Nj < 2(Rj /η)1/αj for all j. It follows from Lemma 4 that there exists some t ∈ S m , m = (N , r)
k √ √ −α with  s − t∞ ≤ C1 (k, α) j=1 Rj Nj j , hence  s − t2 ≤ kC1 (k, α)η. It then follows from Theorem 1 that   k $ #   2  k/α Nj  ≤ C3 (k, α) η 2 + R η −k/α . Es H (s, sˆ) ≤ C2 (k, α) η 2 + (r + 1)k j=1

The conclusion follows.

4.3. Intensities with bounded α-variation Let us first recall that a function f defined on some interval J ⊂ R has bounded α-variation on J for some α ∈ (0, 1] if (4.3)

sup i≥1

sup x0 <···<xi

xj ∈J for 0≤j≤i

i 

|f (xj ) − f (xj−1 )|1/α = [Vα (f ; J)]1/α < +∞,

j=1

the classical case of bounded variation corresponding to α = 1. This formulation using the power 1/α (instead of α) implies that an α-H¨olderian function has bounded α-variation over any finite interval J. We want to build a family of linear models which are suitable for estimating √ intensities s with support on some interval J of finite length L and such that s has bounded α-variation on J for some unknown value of α. These models are linear spaces of piecewise constant functions on some finite partitions m of J, namely   D    Sm = t = aj 1lIj when m = {I1 , . . . , ID }.   j=1

We consider for M a special family of partitions m of J derived by dyadic splitting which are in one-to-one correspondence with the family of complete binary trees. They are built according to the following “adaptive” algorithm described in Section 3.3 of DeVore [17]. This algorithm simultaneously grows a complete binary tree and a dyadic partition of J. It starts with a tree reduced to its root which is associated to the interval J. At each step of the algorithm the set of terminal nodes of the current tree is associated to the set of intervals in the current partition. Each step of the algorithm corresponds to choosing one terminal node and adding two sons to it. For the associated partition this means dividing the interval which corresponds to this terminal node into two intervals of equal length which then correspond to the two sons. At some stage the procedure stops and we end with a complete binary tree with D terminal nodes and the associated partition of J into D intervals. We acually take for M the set of all finite partitions m that can be build in that way so that each m corresponds to the complete binary tree with |m| terminal nodes that was used to build the partition. It is known that the number of complete binary trees   withj j + 1 terminal nodes is given by the so-called Catalan numbers (1 + j)−1 2j j ≤ 4 /(1 + j) as explained

Model selection for Poisson processes

49

for instance in Stanley [33], page 172. Setting ∆m = 2|m| leads to    exp[−∆m ] = exp[−2(j + 1)] j≥0 {m∈M | |m|=1+j}

m∈M

≤

(4.4)

 4j exp[−2(j + 1)] j+1

j≥0

= e−2

 (2/e)2j j≥0

j+1

< 1.

* The approximation properties of m∈M S m with respect to functions of bounded α-variation are given by the following proposition the proof of which was kindly communicated to the author by Ron DeVore [18]. Proposition 3. Let f be a function of bounded α-variation on the interval J of finite length L with α-variation Vα (f ; J) given by (4.3). For each j ∈ N, one can find a partition m ∈ M with |m| ≤ c1 (α)2j

(4.5)

inf f − t2 ≤ c2 (α)L1/2 Vα (f ; J)2−jα .

and

t∈S m

with 1 < c1 (α) = (1 − 2−[1/(2α)+1] )(1 − 2−1/(2α) )−1 < 2.21 and   1−2α 1/2 √ 21+2α 1 − 2−[1/(2α)+1] 2 < c2 (α) = < 6.51. 1 − 2−1/(2α) Proof. For any interval I ⊂ J we denote by |I| its length and set V (I) = Vα (f ; I). If m = {I1 ; . . . ; ID } is a partition of J into D intervals, f¯j = |Ij |−1 Ij f (x) dx and
D ¯ f¯ = fj 1lI , then (f − f¯j )1lI ∞ ≤ V (Ij ), hence j=1

(4.6)

j

j

D    f − f¯2 ≤ E(Ij ) 2

with E(I) = |I|V 2 (I).

j=1

In particular (4.5) holds with m = {J} and j = 0. To study the general case we choose some ε > 0 and apply the adaptive algorithm described just before in the following way: at each step we inspect the intervals of the partition and if we find an interval I with E(I) > ε we divide it into two intervals of equal length |I|/2. The algorithm necessarily stops since E(I) ≤ |I|V 2 (J) for all I ⊂ J and this results in some partition m with E(I) ≤ ε for all I ∈ m. It follows from (4.6) that if f¯ is built on this partition, then f − f¯22 ≤ ε|m|. Since the case |m| = 1 has already been considered, we may assume that |m| ≥ 2. Let us denote by D
k the number of intervals in m with length L2−k and set ak = 2−k Dk so that k≥1 ak = 1 (since D0 = 0). If I is an interval of length L2−k , k > 0, it derives from the splitting of an interval I
with length L2−k+1 such that E(I
) > ε, hence, by (4.6), V (I
) > [εL−1 2k−1 ]1/2 and, since the set function V 1/α is subadditive over disjoint intervals, the number of such interval I
is bounded by [V (J)]1/α [εL−1 2k−1 ]−1/(2α) . It follows that Dk ≤ γ2−k/(2α) and ak ≤ γ2−k/(2α)−k with γ = 2[V (J)]1/α [ε/(2L)]−1/(2α) .
k Since |m| = k≥1 2 ak , we can derive a bound on |m| from a maximization of

k −k[1/(2α)+1] . One should k≥1 2 ak under the restrictions k≥1 ak = 1 and ak ≤ γ2 then clearly keep the largest possible indices k with the largest possible values for

L. Birg´ e

50

ak . Let us fix ε so that γ = (1 − 2−[1/(2α)+1] )2j[1/(2α)+1] for some j ≥ 1. Then,
setting ak to its maximal value, we get k≥j γ2−k[1/(2α)+1] = 1, which implies that an upper bound for |m| is |m| ≤



γ2k 2−k[1/(2α)+1] =

k≥j

γ2−j/(2α) 1 − 2−[1/(2α)+1] j = 2 . 1 − 2−1/(2α) 1 − 2−1/(2α)

The corresponding value of ε is 2L(γ/2)−2α V 2 (J) so that 1−2α −j/(2α)   2 f − f¯2 ≤ ε|m| ≤ 2LV 2 (J)22α γ 2 1 − 2−1/(2α)  1−2α 2LV 2 (J)22α 1 − 2−[1/(2α)+1] = 2−2αj . 1 − 2−1/(2α)

These two bounds give (4.5) and we finally use the fact that 0 < α ≤ 1 to bound the two constants. We can then derive from this proposition, (1.15) and our choice of the ∆m that 1 2q  √   s; J 2−jα . Es H q (s, sˆ) ≤ C(q) inf 2j/2 + L1/2 Vα j∈N

An optimization with respect to j ∈ N then leads to the following risk bound.

Corollary 3. Let X be a Poisson process with unknown intensity s with √ respect to the Lebesgue measure on some interval J of length L. We assume that s has finite α-variation equal to V on J, both α and V being unknown. One can build a T-estimator sˆ(X) such that #

$q/(2α+1)   . (4.7) Es H q (s, sˆ) ≤ C(q) L1/2 V ∨ 1 It is not difficult to show, using Assouad’s Lemma, that, up to a constant, this bound is optimal when q = 2.

(λ) be the set of intensities Proposition 4. Let L, α and V be given and S ⊂ L+ 1√ with respect to the Lebesgue measure on [0, L) such that s has α-variation bounded by V . Let sˆ(X) be any estimator based on a Poisson process X with unknown intensity s ∈ S. There exists a universal constant c > 0 (independent of sˆ, L, α and V ) such that #

$2/(2α+1)   sup Es H 2 (s, sˆ) ≥ c L1/2 V ∨ 1 . s∈S

1/2

Proof. If L V < 1, we simply apply (2.6) with s0 = 1l[0,L) and s1 = (1 + L−1/2 )2 1l[0,L) so that 2H 2 (s0 , s1 ) = 1. If L = 1 and V ≥ 1 we fix some positive integer D and define g with support on [0, D−1 ) by   g(x) = x1l[0,(2D)−1 ) (x) + D−1 − x 1l[(2D)−1 ,D−1 ) (x).  1/D Then 0 g 2 (x) dx = (12D3 )−1 and 0 ≤ g(x) ≤ (2D)−1 . If we apply the construction of Lemma 2, we get a family of Lipschitz intensities sδ with values in the interval [12D3 − 3D2 , 12D3 + 3D2 ] ⊂ [9D3 , 15D3 ] and Lipschitz coefficient 6D3 . It follows that if 0 ≤ x < y ≤ 1, ) |s (x) − s (y)| )  ) ) δ δ ) sδ (x) − sδ (y)) ≤ 3/2 6D  2  3  √ 6D ∧ 6D |x − y| ≤ ≤ D [1 ∧ (D|x − y|)] . 3/2 6D

Model selection for Poisson processes

This allows us to bound the α-variation of creasing sequence 0 ≤ x0 < · · · < xi ≤ 1,

51

√ sδ in the following way. For any in-

)1/α i ) i    ) ) 1/(2α) ) sδ (xj ) − sδ (xj−1 )) ≤ D 1l{xj −xj−1 ≥D−1 } ) ) j=1

j=1

+ D3/(2α)

i 

1l{xj −xj−1
j=1

If n =


i

j=1

1l{xj −xj−1 ≥D−1 } ≤ D, then D3/(2α)

i 

1l{xj −xj−1
j=1

≤ D3/(2α) D−1/α (D − n) = D1/(2α) (D − n), √ which shows that the α-variation of sδ is bounded by [D1/(2α) D]α = D(1+2α)/2 . We finally choose for D the largest integer j such that j (1+2α)/2 ≤ V . Then V 2/(1+2α) < 2D and an application of Lemmas 1 and 2 show that   sup Es H 2 (s, sˆ) ≥ 2−8 (2D) exp[−2/7] ≥ 2−8 exp[−2/7]V 2/(1+2α) , s∈SD

which proves our lower bound. The general case L1/2 V ≥ 1 follows from a scaling argument. If X is a Poisson process on [0, L] with intensity s (with respect to the Lebesgue measure), then Y = L−1 X is a Poisson process on [0, 1] with intensity sL to which the previous it follows that H 2 (s, t) = √ results apply. Since sL (y) = Ls(Ly), √ 2 H (sL , tL ) and, if s has α-variation bounded by V , sL has α-variation bounded by L1/2 V . The result for an arbitrary L follows from these remarks. 4.4. Intensities with square roots in weak q -spaces 4.4.1. Approximation based on weak q -spaces As we already mentioned, if s ∈ L+ 1 (λ) is an intensity with √ respect to λ on X , j ≥ 1} of L (λ), be written as and we are given an orthonormal basis {ϕ j 2

√s can  2 2 β ϕ with β = (β ) ∈  =  (N ) and β =  s < +∞. Hence, j j≥1 2 2 2 j≥1 j j j≥1 j √ 2 −2 for all x > 0, |{j ≥ 1 | |βj | ≥ x}| ≤  s2 x , which means that the sequence β belongs to the weak 2 -space w 2. More generally, given a sequence β = (βj )j≥1 converging to zero and aj the rearrangement of the numbers |βj | in nonincreasing order (which means that a1 = supj≥1 |βj |, etc. . . ), we say that β belongs to the weak q -space w q (q > 0) if (4.8)

sup xq |{j ≥ 1 | |βj | ≥ x}| = sup xq |{j ≥ 1 | aj ≥ x}| = |β|qq,w < +∞. x>0

x>0

This implies that aj ≤ |β|q,w j −1/q for j ≥ 1 and the reciprocal actually holds: 1 2 (4.9) |β|q,w = inf y > 0 | aj ≤ yj −1/q for all j ≥ 1 .

Note that, although |θβ|q,w = |θ||β|q,w for θ ∈ R, |β|q,w is not a norm. For convenience, we shall call it the weight of β in w q . By extension, given the basis

L. Birg´ e

52


{ϕj , j ≥ 1}, we shall say that u ∈ L2 (λ) belongs to w q if u = j≥1 βj ϕj and . As a consequence of this control on the size of the coefficients aj , we get β ∈ w q the following useful lemma. Lemma 5. Let β ∈ w q with weight |β|q,w for some q > 0 and (aj )j≥1 be the nonincreasing rearrangement of the numbers |βj |. Then β ∈ p for p > q and for all n ≥ 1,  p q |β|pq,w (n + 1/2)−(p−q)/q . (4.10) aj ≤ p − q j>n Proof. By (4.9) and convexity, 

apj ≤ |β|pq,w

j>n



j −p/q ≤ |β|pq,w

j>n



+∞

x−p/q dx.

n+1/2

As explained in great detail in Kerkyacharian and Picard [23] and Cohen, DeVore, Kerkyacharian and Picard [15], the fact that u ∈ w q for some q < 2 has important consequences for the approximation of u by fonctions in suitable D-dimensional spaces. For
m any finite subset of N , let us define S m as the linear span of {ϕj , j ∈ m}. If u = j≥1 βj ϕj belongs to w q and D is a positive integer, one can find some m with |m| = D and some t ∈ S m such that u − t22 ≤ (2/q − 1)−1 |β|2q,w (D + 1/2)1−2/q .

(4.11)

Indeed, let us take for m the set of indices of the D largest numbers |βj |. It follows from (4.10) that   q |β|2 (D + 1/2)1−2/q . βj2 = a2j ≤ 2 − q q,w j∈m

j>D




Setting t = j∈m βj ϕj gives (4.11) which provides the rate of approximation of * u by functions of the set {m | |m|=D} S m as a decreasing function of D (which is not possible for q = 2). Unfortunately, this involves an infinite family of linear spaces S m of dimension D since the largest coefficients of the sequence β may have arbitrarily large indices. To derive a useful, as well as a practical approximation method for functions in w q -spaces, one has to restrict to those sets m which are subsets of {1, . . . , n} for some given value of n. This is what is done in Kerkyacharian and Picard [23] who show, in their Corollary 3.1, that a suitable thresholding of empirical versions of the coefficients βj for j ∈ {1, . . . , n} leads to estimators that have nice properties. Of course, since this approach ignores the (possibly large) coefficients
with indices bigger than n, an additional condition on β is required to control j>n βj2 . In Kerkyacharian and Picard [23], it takes the form  (4.12) βj2 ≤ A2 n−δ for all n ≥ 1, with A and δ > 0, j>n

while Cohen, DeVore, Kerkyacharian and Picard [15], page 178, use the similar condition BS. Such a condition is always satisfied for functions in Besov spaces α ([0, 1]l ) with p ≤ 2 and α > l(1/p − 1/2). Indeed, if Bp,∞ f=

∞ 



j=−1 k∈Λ(j)

βj,k ϕj,k

Model selection for Poisson processes

53

belongs to such a Besov space, it follows from (4.1) that,  2/p      l l  |βj,k |2 ≤ |βj,k |p  ≤ |f |2Bp,∞ 2−2j (α+ 2 − p ) α j>J k∈Λ(j)

j>J

≤

(4.13)

j>J

k∈Λ(j)

l l 2−2J (α+ 2 − p ) . C|f |2Bp,∞ α

Since the number of coefficients βj,k with j ≤ J is bounded by C
2Jl , after a proper change in the indexing of the coefficients, the corresponding sequence β will satisfy
2 2 −δ β with δ = (2α/l) + 1 − (2/p). j>n j ≤ A n

4.4.2. Model selection for weak q -spaces

It is the very method of thresholding that imposes to fix the value of n as a function of δ or impose the value of δ when n has been chosen in order to get a good performance for the threshold estimators. Model selection is more flexible since it allows to adapt the value of n to the unknown values of A and δ. Let us assume that an orthonormal basis {ϕj , j ≥ 1} for L2 (λ) has been√chosen
and that the Poisson process X has an intensity s with respect to λ so that s = j≥1 βj ϕj with β ∈ 2 . We take for M the set of all subsets m of N such that |m| = 2j for some j ∈ N and choose for S m the linear span of {ϕj , j ∈ m} with dimension Dm = |m|. If |m| = 2j  k and k = inf{i ∈ N | 2i ≥ l for all l ∈ m}, we set ∆m = k + log 22j . Then 

exp[−∆m ] ≤

k k  2

k≥1 j=0

m∈M

2j



k   2 (k + 1) exp[−k], exp −k − log j ≤ 2 k≥1

which allows to apply Theorem 1. Proposition 5. Let sˆ be a T-estimator provided by
1 and based on the √ Theorem previous family of models S m and weights ∆m . If s = j≥1 βj ϕj with β ∈ w q for some q < 2 and (4.12) holds with A ≥ 1 and 0 < δ ≤ 1, the risk of sˆ at s is bounded by #

   q/2 3 2/(1+δ) $ , A Es H 2 (s, sˆ) ≤ C γ 1−q/2 R2 ∨ γ with



q R= 2−q

1/2

|β|q,w

and

γ=δ

−1



   log δ[A ∨ R]2 4 1 . log 2

Proof. Let (aj )j≥1 be the nonincreasing rearrangement of the numbers |βj |, k and j ≤ k be given and m be the set of indices of the 2j largest coefficients among  k {|β1 |, . . . , |β2k |}. Then Dm = 2j and ∆m ≤ k + log 22j . It follows from (4.10) and (4.12) that .    q |β|2 2−j(2/q−1) 1lj
i>2

i>2

√ This shows that one can find t ∈ S m such that  s − t22 ≤ R2 2−j(2/q−1) 1lj
k    2  2 2 −j(2/q−1) 2 −kδ j Es H (s, sˆ) ≤ C inf inf 1lj
L. Birg´ e

54

We recall that C denotes a constant that may change as often as necessary. If j = k, Es [H 2 (s, sˆ)] ≤ C[A2 2−kδ + 2k ] and an optimization with respect to k leads to Es [H 2 (s, sˆ)] ≤ CA2/(1+δ) . For j < k, we notice that ∆m ≤ k +2j [1+log(2k−j )] < 3k2j , so that  1

 2  2 −kδ    A 2 R2 2−j(2/q−1) ∨ k2j ∨ inf (4.14) Es H 2 (s, sˆ) ≤ C inf . k≥1

0≤j
If R2 2−(k−1)(2/q−1) > k2k−1 , we may harmlessly increase k until k = K with ) ) 1 2 1 2 ) ) K = inf i ≥ 1 ) i2i−1 ≥ R2 2−(i−1)(2/q−1) = inf i ≥ 1 ) 2i−1 ≥ Rq i−q/2

and therefore restrict the minimization in (4.14) to k ≥ K. We then choose for j the smallest integer i such that 2i ≥ (R2 /k)q/2 , which leads to 1 2  

 A2 2−kδ ∨ Rq k 1−q/2 ∨ k . Es H 2 (s, sˆ) ≤ C inf k≥K

5 It follows from Lemma 6 below (with a = 1) that, if δA2 ≤ 2, (A2 2−kδ ) k ≥ A2 /2 for all k which does not improve on our previous bound CA2/(1+δ) so that we may assume from now on that δA2 > 2, hence γ > δ −1 . Handling this case in full generality is much more delicate and we shall simplify the minimization problem by replacing A by A = A ∨ R, which amounts to assuming that A ≥ R and leads to Es [H 2 (s, sˆ)] ≤ C inf k≥K f (k) with f (x) = f1 (x) ∨ f2 (x) ∨ x;

2

f1 (x) = A 2−xδ

and f2 (x) = Rq x1−q/2 .

We want to minimize f (x), up to constants. The minimization of f1 (x) ∨ x follows 2 from Lemma 6 with δA > 2. The minimum then takes the form c2 γ > 0.469γ with f1 (γ) = δ −1 < γ hence f (γ) = γ ∨ f2 (γ). To show that inf x f (x) ≥ cf (γ) 2 when δA > 2, we distinguish between two cases. If R2 ≤ γ, f (γ) = γ and we conclude from the fact that inf x f (x) > 0.469γ. If R2 > γ, f2 (x) > x for x ≤ γ, f (γ) = f2 (γ) > γ and the minimum of f (x) is obtained for some x0 < γ. Hence 1 2  2 inf f (x) = inf {f1 (x) ∨ f2 (x)} = Rq inf B2−δx ∨ x1−q/2 with B = A R−q . x

x

x

It follows from Lemma 6 with a = (2 − q)/2 that the result of this minimization depends on the value of 2

2δ 2A δ 4/(2−q) −2q/(2−q) V = A R = 2−q 2−q

A R

2q/(2−q)

2

≥ A δ > 2,

since A ≥ R. Then, inf f (x) ≥ R x

q



(2 − q) log V 3δ

1−q/2

≥R γ

q 1−q/2



(2 − q) log 2 3

> 0.45Rq γ 1−q/2 ,

1−q/2

and we can conclude that, in both cases, inf x f (x) ≥ 0.45f (γ). Let us now fix k such 2 that γ + 1 ≤ k < γ + 2 so that k < 3γ. Then 2k−1 ≥ 2γ = (A δ)1/δ while Rq k −q/2 ≤ (R2 /γ)q/2 ≤ (R2 δ)q/2 . This implies that k ≥ K. Moreover f (k) = k ∨f2 (k) < 3f (γ) which shows that inf k≥K f (k) < 3f (γ) < 6.7 inf x f (x) and justifies this choice of k. Finally Es [H 2 (s, sˆ)] ≤ C[γ ∨ f2 (γ)].

Model selection for Poisson processes

55


p Note that our main assumption, namely that β ∈ w q , implies that j>n aj ≤

R2 n−2/q+1 by (4.10) while (4.12) entails that j>n apj ≤ j>n βjp ≤ A2 n−δ . Since it is only an additional assumption it should not be strictly stronger than the main one, which is the case if A ≤ R and δ ≥ 2/q − 1. It is therefore natural to assume that at least one of these inequalities does not hold. Lemma 6. For positive parameters a, B and θ, we consider on R+ the function f (x) = B2−δx ∨ xa . Let V = a−1 δB 1/a . If V ≤ 2 then inf x f (x) = c1 B with 2−a ≤ c1 < 1. If V > 2, then inf x f (x) = [c2 aδ −1 log V ]a with 2/3 < c2 < 1. Proof. Clearly, the minimum is obtained when x = x0 is the solution of B2−δx = xa . Setting x0 = B 1/a y and taking base 2 logarithms leads to y −1 log2 (y −1 ) = V , hence y < 1. If V ≤ 2, then 1 < y −1 ≤ 2 and the first result follows. If V ≥ 2, the solution takes the form y = zV −1 log2 V with 1 > z > [1 − (log2 V )−1 log2 (log2 V )] > 0.469. 4.4.3. Intensities with bounded variation on [0, 1)2 √ This section, which is devoted to the estimation of an intensity s such that s belongs to the space BV ([0, 1)2 ), owes a lot to discussions with Albert Cohen and Ron DeVore. The approximation results that we use here should be considered as theirs. The definition and properties of the space BV ([0, 1)2 ) of functions with bounded variation on [0, 1)2 are given in Cohen, DeVore, Petrushev and Xu [16] where the reader can also find the missing details. It is known that, with the notations of 1 1 ([0, 1)2 ) ⊂ BV ([0, 1)2 ) ⊂ B1,∞ ([0, 1)2 ). This corSection 4.1 for Besov spaces, B1,1 responds to the situation α = 1, l = 2 and p = 1, therefore α = l(1/p − 1/2), a borderline case which is not covered by the results of Theorem 4. On the other hand, it is proved in Cohen, DeVore, Petrushev and Xu [16], Section 8, that, if a function of BV ([0, 1)2 ) is expanded in the two-dimensional Haar basis, its coefficients belong 2 to the space w 1 . More precisely if f ∈ BV ([0, 1) ) with semi-norm |f |BV and f is expanded in the Haar basis with coefficients βj , then |β|1,w ≤ C|f |BV where |β|1,w is given by (4.8) and C is a universal √ constant. We may therefore use the results of the previous section to estimate s but we √ need an additional assumption to ensure that (4.12) is satisfied. By definition s belongs to L2 ([0, 1)2 , dx) but we shall assume here slightly more, namely that it belongs to Lp ([0, 1)2 , dx) for some p > 2. This is enough to show that (4.12) holds. 2 2 Lemma
∞7. If
f ∈ BV ([0, 1) )∩Lp ([0, 1) , dx) for some p > 2 and has an2 expansion f = k∈Λ(j) βj,k ϕj,k with respect to the Haar basis on [0, 1) , then for j=−1 J ≥ −1,   1 |βj,k |2 ≤ C(p)f p |f |B1,∞ 2−2J(1/2−1/p) . j>J k∈Λ(j)

Proof. It follows from H¨ older inequality that |βj,k | = f, ϕj,k ≤ f p ϕj,k p
with


−1 −1 p = 1−p and by the structure of a wavelet basis, ϕj,k pp
≤ c1 2−j(2−p ) , so that
2 1 2 |βj,k | ≤ c2 f p 2−j(2/p −1) = c2 f p 2−j(1−2/p) . Since
BV ([0, 1) ) ⊂ B1,∞ ([0, 1) ), 1 so that it follows from (4.1) with α = p = 1 and l = 2 that k∈Λ(j) |βj,k | ≤ |f |B1,∞
2 −j(1−2/p) 1 2 for all j ≥ 0. The conclusion follows. k∈Λ(j) |βj,k | ≤ c2 f p |f |B1,∞ Since the number of coefficients βj,k with j ≤ J is bounded by C22J , after a proper reindexing of the coefficients, the corresponding sequence β will satisfy

56

L. Birg´ e

(4.12) with δ = 1/2 − 1/p which shows that it is essential here that p be larger than 2. We finally get the following corollary of Proposition 5 with q = 1. Corollary 4. One can build sˆ with the following properties. Let the √ a T-estimator 2 s ∈ BV ([0, 1) ) ∩ Lp ([0, 1)2 , dx) for some p > 2, so that intensity s be such that √ the expansion √ of s in the Haar basis satisfies (4.12) with δ = 1/2−1/p and A ≥ 1. Let R = | s|BV , then $ #   E H 2 (s, sˆ) ≤ C γ (R2 ∨ γ) ∧ A2/(1+δ)     2 −1 log δ[A ∨ R] ∨1 . with γ = δ log 2 4.5. Mixing families of models We have studied here a few families of approximating models. Many more can be considered and further examples can be found in Reynaud-Bouret [30] or previous papers of the author on model selection such as Barron, Birg´e and Massart [7], Birg´e and Massart [12], Birg´e [9] and Baraud and Birg´e [4]. As indicated in the previous sections, the choice of suitable families of models is driven by results in approximation theory relative to the type of intensity we expect to encounter or, more precisely, to the type of assumptions we make about the unknown function √ s. Different types of assumptions will lead to different choices of approximating models, but it is always possible to combine them. If we have built a few families of linear
models {S m , m ∈ Mj } for 1 ≤ j ≤ J and chosen suitable weights ∆m such that m∈Mj exp[−∆m ] ≤ Σ for all j we may consider the mixed family of models {S m , m ∈ M} with M = ∪Jj=1 Mj and define new weights ∆
m = ∆m + log J for all m ∈ M so that (1.13) still holds with the same value of Σ. It follows from Theorem 1 that the T-estimator based on the mixed family will share the properties of the ones derived from the initial families apart, possibly, for a moderate increase in the risk of order (log J)q/2 . The situation becomes more complex if J is large or even infinite. A detailed discussion of how to mix families of models in general has been given in Birg´e and Massart [12], Section 4.1, which applies with minor modifications to our case. 4.6. Asymptotics and a parallel with density estimation The previous examples lead to somewhat unusual bounds with no number of observations n like for density estimation and no variance size σ 2 as in the case of the estimation of a normal mean. Here, there is no rate of convergence because there is no sequence of experiments, just one with a mean measure µs = s · λ. To get back to more familiar results with rates and asymptotics and recover some classical risk bounds, we may reformulate our problem in a slightly different form which completely parallels the one we use for density estimation. As indicated in  our introduction we may always rewrite the intensity s as s = ns1 with s1 dλ = 1 so that s1 becomes a density and n = µs (X ). We use this notation here, although n need not be an integer, to emphasize the similarity between the estimation of s and density estimation. When n is an integer this also corresponds to observing n
n i.i.d. Poisson processes Xi , 1 ≤ i ≤ n with intensity s1 and set ΛX = i=1 ΛXi . In this case (1.15) can be rewritten in the following way.

Model selection for Poisson processes

57

Corollary 5. Let λ be some positive measure on X , X be a Poisson process with unknown intensity s ∈ L+ 1 (λ), {S m , m ∈ M} be a finite or countable family of linear subspaces of L2 (λ) with respective finite dimensions Dm and let {∆m }m∈M be a family of nonnegative weights satisfying (1.13).  One can build a T-estimator (λ) such that s dλ = n, s1 = n−1 s and all sˆ(X) of s satisfying, for all s ∈ L+ 1 q ≥ 1,  q    #

q $ √ D ∨ ∆ m m ≤ C(q) [1 + Σ] inf . Es n−1/2 H(s, sˆ) inf  s1 − t2 + m∈M t∈S m  n Writtten in this form, our result appears as a complete analogue of √ Theorem 6 of Birg´e [9] about density estimation, the normalized loss function (H/ n)q playing the role of the Hellinger loss hq for densities. We also explained in Birg´e [9], Section 8.3.3, that there is a complete parallel between density estimation and estimation in the white noise model. We can therefore extend this parallel to the estimation of the intensity of a Poisson process. This parallel has also been explained and applied to various examples in Baraud and Birg´e [4], Section 4.2. As an additional consequence, all the families of models that we have introduced in Sections 3.3, 4.2, 4.3 and 4.4 could be used as well for adaptive estimation of densities or in the white noise model and added to the examples given in Birg´e [9]. To recover the familiar rates of convergence that we get when estimating densities which belong to some given function class S, we merely have to assume that s1 (rather than s) belongs to the class S and use the normalized loss function. Let us, for instance, apply this approach to intensities belonging to Besov spaces, assuming √ √ α α ([0, 1]l ) with α > l(1/p − 1/2)+ and that | s1 |Bp,∞ ≤ L with that s1 ∈ Bp,∞ √ √ √ α α L > 0. It follows that s ∈ Bp,∞ ([0, 1]l ) with | s|Bp,∞ ≤ L n. For n large enough, √ √ L n ≥ 1 and Theorem 4 applies, leading to Es [H 2 (s, sˆ)] ≤ C(α, p, l)(L n)2l/(2α+l) . Hence   Es n−1 H 2 (s, sˆ) ≤ C(α, p, l)L2l/(2α+l) n−2α/(2α+l) , which is exactly the result we get for density estimation with n i.i.d. observations. The same argument can be developed for √ the problem we considered in√Sec√ s , rather than s, belongs to H(α, R), then s ∈ tion 4.2. If we assume that 1 √ nR) and the condition R ≥ η of Corollary 2 becomes, after this rescaling, H(α, j √ √ nRj ≥ ( nR)k/(2α+k) which always holds for n large enough. The corresponding normalized risk bound can then be written   2k/(2α+k) −2α/(2α+k) Es n−1 H 2 (s, sˆ) ≤ C(k, α)R n ,

which corresponds to the rate of convergence for this problem in density estimation. Another interesting case is the one considered in Section 4.4.√Let us assume here that instead of putting√the assumptions of Proposition 5 on s we put them√on √ that s satisfies the same assumptions with R replaced by R n s1 . This implies √ and A by A n. Then, for n ≥ n0 (A, R, δ), γ ≤ 2δ −1 log n ≤ nR2 and    1−q/2 Es n−1 H 2 (s, sˆ) ≤ C(q, δ, A, R) n−1 log n .

This result is comparable to the bounds obtained in Corollary 3.1 of Kerkyacharian and Picard [23] but here we do not know the relationship between q and δ. For √ the special situation of s1 ∈ BV ([0, 1)2 ), we get Es [n−1 H 2 (s, sˆ)] ≤ C(q, δ, s1 ) × (n−1 log n)1/2 . One could also translate all other risk bounds in the same way.

L. Birg´ e

58

An alternative asymptotic approach, which has been considered in ReynaudBouret [30], is to assume that X is a Poisson process on Rk with intensity s with respect to the Lebesgue measure on Rk , but which is only observed on [0, T ]k . We therefore estimate s1  l[0,T ]k , letting T go to infinity to get an asymptotic result. We only assume that [0,T ]k s(x) dx is finite for all T > 0, not necessarily that  s(x) dx < +∞. For simplicity, let us consider the case of intensities s on R+ Rk √ with s belonging to the H¨ older class H(α, R). For t an intensity on R+ , we set for 0 ≤ x ≤ 1, tT (x) = T t(T x) so that tT is an intensity on [0, 1] and H(tT , uT ) = √ H(t1l[0,T ] , u1l[0,T ] ). Since sT ∈ H(α, RT α+1/2 ) it follows from Corollary 2 that there is a T-estimator sˆT (X) of sT satisfying 2/(2α+1)

  Es H 2 (sT , sˆT ) ≤ C(α) RT α+1/2 = C(α)T R2/(2α+1) .

  Finally setting sˆ(y) = T −1 sˆT T −1 y for y ∈ [0, T ], we get an estimator sˆ(X) of s1l[0,T ] depending on T with the property that    Es H 2 s1l[0,T ] , sˆ ≤ C(α)T R2/(2α+1)

for all T > 0.

4.7. An illustration with Poisson regression As we mentioned in the introduction, a particular case occurs when X is a finite set that we shall assume here, for simplicity, to be {1; . . . ; 2n }. In this situation, observing X amounts to observing N = 2n independent Poisson variables with respective parameters si = s(i) where s denotes the intensity with respect to the counting If we introduce a family of linear models S m in RN to approxi√ measure. N mate s ∈ R with respect to the Euclidean distance, we simply apply Theorem 1 to get the resulting risk bounds. In this situation, the Hellinger distance between two intensities √ is merely the Euclidean distance between their square roots, up to a factor 1/ 2. As an example, we shall consider linear models spanned by piecewise constant
D functions on X as described in Section 1.4, i.e. S m = { j=1 aj 1lIj } when m = {I1 , . . . , ID } is a partition of X into D = |m| nonvoid intervals. In order to define suitable weights ∆m , we shall distinguish between two types of partitions. First we consider the family MBT of dyadic partitions derived from binary trees and described in Section 4.3. We already know that the choice ∆m = 2|m| is suitable for those partitions and (4.4) applies. Note that these include the regular partitions, k i.e. those for which all intervals Ij have the same size N/|m|  |m| = 2 for  Nand 0 ≤ k ≤ n. For all other partitions, we simply set ∆m = log |m| + 2 log(|m|) so that (1.13) holds with Σ < 3 since the number of possible partitions of X into |m| N −2 intervals is |m|−1 . We omit the details. Denoting by  · 2 the Euclidean norm in N R , we derive from Theorem 1 the following risk bound for T-estimators:   √  2 √ Es  s − sˆ 2    2 √   ≤C inf s − t 2 + |m| inf m∈MBT t∈S m 

  3 2 √ N . inf inf  s − t2 + log(|m|) + log |m| t∈S m m∈M\MBT

Model selection for Poisson processes

59

The performance of the estimator then√depends on the approximation properties √ of the linear spaces S m with respect to s. For instance, if s varies regularly, i.e. √ √ | ≤ R for | si − si−1√ √ all i, one uses a regular partition which belongs to MBT to approximate s. If s has bounded α-variation, as√defined in Section 4.3, one uses dyadic partitions as explained in this section. If s is piecewise constant with  N k . jumps, it belongs to some S m and we get a risk bound of order log(k+1)+log k+1 5. Aggregation of estimators In this section we assume that we have at our disposal a family {ˆ sm , m ∈ M
} of intensity estimators, (T-estimators or others) and that we want to select one of them or combine them in some way in order to get an improved estimator. We already explained in Section 2.3 how to use the procedure of thinning to derive from a Poisson process X with mean measure µ two independent Poisson processes with mean measure µ/2. Since estimating µ/2 is equivalent to estimating µ, we shall assume in this section that we have at our disposal two independent processes X1 and X2 with the same unknown mean measure µs with intensity s to be estimated. We assume that the initial estimators sˆm (X1 ) are all based on the first process and therefore independent of X2 . Proceeding conditionally on the first process, we use the second one to mix the estimators. We shall consider here two different ways of aggregating estimators. The first one is suitable when we want to choose one estimator in a large (possibly infinite) family of estimators and possibly attach to them different prior weights. The second method tries to find the best linear combination from a finite family of estimators √ of s. 5.1. Estimator selection Here we start from a finite or countable family {ˆ sm , m ∈ M} of intensity estimators and a family of weights ∆m ≥ 1/10 satisfying (1.13). Our purpose is to use the sm (X1 ), m ∈ M}. process X2 to find a close to best estimator among the family {ˆ 5.1.1. A general result Considering each estimator sˆm (X1 ) as a model Sm = {ˆ sm (X1 )} with one single 2 = 84∆m . Then Sm is a T-model with parameters ηm , 1/2 and point, we set ηm B
= e−2 , (3.2) and (3.3) hold and Theorem 3 applies. Since each model is reduced to one point, one can find a selection procedure m(X ˆ 2 ) such that the estimator (X ) satisfies the risk bound s˜(X1 , X2 ) = sˆm(X 1 ˆ 2) 1 2 )   2 Es H 2 (s, s˜)) X1 ≤ C[1 + Σ] inf H 2 (s, sˆm (X1 )) + ∆m . m∈M

Integrating with respect to the process X1 gives  +  ,   (5.1) Es H 2 (s, s˜) ≤ C[1 + Σ] inf Es H 2 (s, sˆm ) + ∆m . m∈M

This result completely parallels the one obtained for density estimation in Section 9.1.2 of Birg´e [9].

L. Birg´ e

60

5.1.2. Application to histograms The simplest estimators for the intensity s of a Poisson process X are histograms. Let m be a finite partition m = {I1 , . . . , ID } of X such that λ(Ij ) > 0 for all j. To this partition corresponds
D the linear space of piecewise constant functions on the partition m: S m = { j=1 aj 1lIj }, the projection s¯m of s onto S m and the corre
D  sponding histogram estimator sˆm of s given respectively by s¯m = j=1 ( Ij s dλ) ×
D
N [λ(Ij )]−1 1lIj and sˆm = j=1 Nj [λ(Ij )]−1 1lIj with Nj = i=1 1lIj (Xi ). It is proved in Baraud and Birg´e [4], Lemma 2, that H 2 (s, s¯m ) ≤ 2H 2 (s, S m ). Moreover, one can show an analogue of the risk bound obtained for the case of density estimation in Birg´e and Rozenholc [13], Theorem 1. The proof is identical, replacing h by H, n by 1 and the binomial distribution of N by a Poisson distribution. This leads to the risk bound     Es H 2 (s, sˆm ) ≤ H 2 (s, s¯m ) + D/2 ≤ 2H 2 s, S m + |m|/2.

If we are given an arbitrary family M of partitions of X and a corresponding family of weights {∆m , m ∈ M} satisfying (1.13) and ∆m ≥ |m|/2, we may apply the previous aggregation method which will result in an estimator s˜(X1 , X2 ) = (X1 ) where m(X ˆ sˆm(X 2 ) is a data-selected partition. Finally, ˆ 2)  2 +   ,  (5.2) Es H (s, s˜) ≤ C[1 + Σ] inf H 2 s, S m + ∆m . m∈M

Various choices of partitions and weights have been described in Baraud and Birg´e [4] together with their approximation properties with respect to different classes of functions. Numerous illustrations of applications of (5.2) can therefore be found there. 5.2. Linear aggregation Here we start with a finite family {ˆ si (X1 ), 1 ≤ i ≤ n} of intensity estimators. We choose for M the set of all nonvoid subsets of {1, . . . , n} and to each such subset m, we associate the |m|-dimensional linear subspace S m of L2 (λ) given by      (5.3) Sm = λj sˆj (X1 ) with λj ∈ R for j ∈ m .   j∈m


n n + 2 log(|m|) so that (1.13) holds with Σ = i=1 i−2 . We then set ∆m = log |m| We may therefore apply Theorem 1 to the process X2 and this family of models conditionally to X1 , which results in the bound )   Es H 2 (s, sˆ)) X1 ≤ C[1 + Σ]

   2 √ n   × inf s − t(X1 ) 2 + log + log(|m|) . inf m∈M t∈S m |m| 

Note that the restriction of this bound to subsets m such that |m| = 1 corresponds to a variant of estimator selection and leads, after integration, to    2    2   √ Es H (s, sˆ) ≤ C[1 + Σ] inf inf Es  s − λ sˆi (X1 ) + log n . 1≤i≤n

λ>0

2

This can be viewed as an improved version of (5.1) when we choose equal weights.

Model selection for Poisson processes

61

6. Testing balls in (Q+ (X ), H) 6.1. The construction of robust tests In order to use Theorem 3, we have to find tests ψt,u satisfying the conclusions of Proposition 1. These tests are provided by a straightforward corollary of the following theorem. Theorem 5. Given two elements πc and νc of Q+ (X ) with respective densities dπc and dνc with respect to some dominating measure λ ∈ Q+ (X ) and a number ξ ∈ (0, 1/2), let us define πm and νm in Q+ (X ) by their densities dπm and dνm with respect to λ in the following way:       dπm = ξ dνc + (1 − ξ) dπc and dνm = ξ dπc + (1 − ξ) dνc . Then for all x ∈ R, µ ∈ Q+ (X ) and X a Poisson process with mean measure µ,   

  2 2 dQπm 2 H (µ, νc ) − H (πc , νc ) (X) ≥ 2x ≤ exp −x + (1 − 2ξ) Pµ log dQνm ξ and   

  2 2 dQπm H (µ, πc ) − H 2 (πc , νc ) . Pµ log (X) ≤ 2x ≤ exp x + (1 − 2ξ) dQνm ξ Corollary 6. Let πc and νc be two elements of Q+ (X ), 0 < ξ < 1/2 and   T (X) = log (dQπm /dQνm )(X) − 2x,

with πm and νm given by Theorem 5. Define a test function ψ with values in {πc , νc } by ψ(X) = πc when T (X) > 0, ψ(X) = νc when T (X) < 0 (ψ(X) being arbitrary if T (X) = 0). If X is a Poisson process with mean measure µ, then   Pµ [ψ(X) = πc ] ≤ exp −x − (1 − 2ξ)2 H 2 (πc , νc ) if H(µ, νc ) ≤ ξH(πc , νc ) and

  Pµ [ψ(X) = νc ] ≤ exp x − (1 − 2ξ)2 H 2 (πc , νc )

if H(µ, πc ) ≤ ξH(πc , νc ).

To derive Proposition 1 we simply set πc = µt , νc = µu , ξ = 1/4, x = [η 2 (t) − η (u)]/4 and define ψt,u = ψ in Corollary 6. As to (3.5), it follows from the second bound of Theorem 5. 2

6.2. Proof of Theorem 5 It is based on the following technical lemmas. Lemma 8. Let f , g, f
∈ L+ 2 (λ) and g/f ∞ ≤ K. Denoting by ·, · and  · 2 the scalar product and norm in L2 (λ), we get  (6.1) gf −1 f
2 dλ ≤ Kf − f
22 + 2g, f
− g, f .

L. Birg´ e

62

Proof. Denoting by Q the left-hand side of (6.1) we write    Q = gf −1 (f
− f )2 dλ + 2 gf
dλ − gf dλ, hence the result. Lemma 9. Let µ, π and ν be three mean measures with π  ν and dπ/dν∞ ≤ K 2 and let X be a Poisson process with mean measure µ. Then     dQπ Eµ (X) ≤ exp 2KH 2 (µ, ν) − 2H 2 (π, µ) + H 2 (π, ν) . dQν Proof. By (1.3) and (1.2),  N 6      dπ ν(X ) − π(X ) dQπ Eµ (Xi ) (X) = exp Eµ dQν 2 dν i=1 .    -6 dπ ν(X ) − π(X ) + (x) − 1 dµ(x) = exp 2 dν X    6 ν(X ) − π(X ) dπ = exp − µ(X ) + (x) dµ(x) . 2 dν X Using Lemma 8 and (1.7), we derive that  6    √ dπ (x) dµ(x) ≤ 2KH 2 (µ, ν) + 2 dπdµ − dπdν dν X = 2KH 2 (µ, ν) − 2H 2 (π, µ) + π(X ) + µ(X ) + H 2 (π, ν) − (1/2)[π(X ) + ν(X )]. The conclusion follows. To prove (changing  λ if necessary) that µ  λ and  Theorem 5, we may assume set v = dµ/dλ. We also set tc = dπc /dλ, uc = dνc /dλ, tm = ξuc +(1−ξ)tc and um = ξtc + (1 − ξ)uc . Then πm = t2m · λ and νm = u2m · λ. Note that tc , uc , tm , um + 2 and v belong to L+ 2 (λ) and that for two elements w, z in L2 (λ), w − z2 = 2 2 2 2H (w · λ, z · λ). Since tm /um ∞ ≤ (1 − ξ)/ξ, we may apply Lemma 9 with K = (1 − ξ)/ξ to derive that -  . dQπm 1−ξ tm − um 22 v − um 22 − v − tm 22 + . L = log Eµ (X) ≤ dQνm ξ 2 Using the fact that v − um = v − uc + ξ(uc − tc ),

v − tm = v − uc + (1 − ξ)(uc − tc ),

tm − um = (1 − 2ξ)(tc − uc ) and expending the squared norms, we get, since the scalar products cancel,   (1 − 2ξ)2 1 − 2ξ 2 2 v − uc 2 + ξ(1 − ξ) − (1 − ξ) + L≤ tc − uc 22 , ξ 2

Model selection for Poisson processes

63

which shows that   L ≤ (1 − 2ξ) 2ξ −1 H 2 (µ, νc ) − H 2 (πc , νc ) .

The exponential inequality then implies that      dQπm dQπm −x Pµ log (X) ≥ 2x ≤ e Eµ (X) = exp[−x + L], dQνm dQνm which proves the first error bound. The second one can be proved in the same way.

Acknowledgments Many thanks to Philippe Bougerol for some exchanges about Poisson processes and to Albert Cohen and Ron DeVore for several illuminating discussions on approximation theory, the subtleties of the space BV (R2 ) and adaptive approximation methods. I also would like to thank the participants of the Workshop Asymptotics: particles, processes and inverse problems that was held in July 2006 in Leiden for their many questions which led to various improvements of the paper. References [1] Antoniadis, A., Besbeas, P. and Sapatinas, T. (2001). Wavelet shrinkage for natural exponential families with cubic variance functions. Sankhy¯ a Ser. A 63 309–327. [2] Antoniadis, A. and Sapatinas, T. (2001). Wavelet shrinkage for natural exponential families with quadratic variance functions. Biometrika 88 805–820. [3] Assouad, P. (1983). Deux remarques sur l’estimation. C. R. Acad. Sci. Paris S´er. I Math. 296 1021–1024. ´, L. (2006). Estimating the intensity of a random [4] Baraud, Y. and Birge measure by histogram type estimators. Probab. Theory Related Fields. To appear. Available at arXiv:math.ST/0608663. [5] Barron, A. R. (1993). Universal approximation bounds for superpositions of a sigmoidal function. IEEE Transactions on Information Theory 39 930–945. [6] Barron, A. R. (1994). Approximation and estimation bounds for artificial neural networks. Machine Learning 14 115–133. ´, L. and Massart, P. (1999). Risk bounds for model [7] Barron, A. R., Birge selection via penalization. Probab. Theory Related Fields 113 301–415. [8] Barron, A. R. and Cover, T. M. (1991). Minimum complexity density estimation. IEEE Transactions on Information Theory 37 1034–1054. ´, L. (2006). Model selection via testing: an alternative to (penalized) [9] Birge maximum likelihood estimators. Ann. Inst. H. Poincar´e Probab. Statist. 42 273–325. ´, L. (2006). Statistical estimation with model selection. Indagationes [10] Birge Math. 17 497–537. ´, L. and Massart, P. (1997). From model selection to adaptive esti[11] Birge mation. In Festschrift for Lucien Le Cam: Research Papers in Probability and Statistics (D. Pollard, E. Torgersen and G. Yang, eds.) 55–87. Springer, New York.

64

L. Birg´ e

´, L. and Massart, P. (2001). Gaussian model selection. J. Eur. Math. [12] Birge Soc. 3 203–268. ´, L. and Rozenholc, Y. (2006). How many bins should be put in a [13] Birge regular histogram. ESAIM Probab. Statist. 10 24–45. [14] Cencov, N. N. (1962). Evaluation of an unknown distribution density from observations. Soviet Math. 3 1559–1562. [15] Cohen, A., DeVore, R., Kerkyacharian, G. and Picard, D. (2001). Maximal spaces with given rate of convergence for thresholding algorithms. Appl. Comput. Harmon. Anal. 11 167–191. [16] Cohen, A., DeVore, R., Petrushev, P. and Xu, H. (1999). Nonlinear approximation and the space BV (R2 ). Amer. J. Math. 121 587–628. [17] DeVore, R. A. (1998). Nonlinear approximation. Acta Numerica 7 51–150. [18] DeVore, R. A. (2006). Private communication. [19] DeVore, R. A. and Lorentz, G. G. (1993). Constructive Approximation. Springer, Berlin. [20] Ghosal, S., Ghosh, J. K. and van der Vaart, A. W. (2000). Convergence rates of posterior distributions. Ann. Statist. 28 500–531. ´goire, G. and Nembe ´, J. (2000). Convergence rates for the minimum [21] Gre complexity estimator of counting process intensities. J. Nonparametr. Statist. 12 611–643. ¨rdle, W., Kerkyacharian, G., Picard, D. and Tsybakov, S. (1998). [22] Ha Wavelets, Approximation and Statistical Applications. Lecture Notes in Statist. 129. Springer, New York. [23] Kerkyacharian, G. and Picard, D. (2000). Thresholding algorithms, maxisets and well-concentrated bases. Test 9 283–344. [24] Kolaczyk, E. (1999). Wavelet shrinkage estimation of certain Poisson intensity signals using corrected threshold. Statist. Sinica 9 119–135. [25] Kolaczyk, E. and Nowak, R. (2004). Multiscale likelihood analysis and complexity penalized estimation. Ann. Statist. 32 500–527. [26] Le Cam, L. M. (1973). Convergence of estimates under dimensionality restrictions. Ann. Statist. 1 38–53. [27] Massart, P. (2007). Concentration inequalities and model selection. In Lecture on Probability Theory and Statistics. Ecole d’Et´e de Probabilit´es de SaintFlour XXXIII — 2003 (J. Picard, ed.). Lecture Notes in Math. 1896. Springer, Berlin. [28] Patil, P. N. and Wood, A. T. (2004). A counting process intensity estimation by orthogonal wavelet methods. Bernoulli 10 1–24. [29] Reiss, R.-D. (1993). A Course on Point Processes. Springer, New York. [30] Reynaud-Bouret, P. (2003). Adaptive estimation of the intensity of inhomogeneous Poisson processes via concentration inequalities. Probab. Theory Related Fields 126 103–153. [31] Reynaud-Bouret, P. (2006). Penalized projection estimators of the Aalen multiplicative intensity. Bernoulli 12 633–661. [32] Rigollet, T. and Tsybakov, A. B. (2006). Linear and convex aggregation of density estimators. Available at arXiv:math.ST/0605292 v1. [33] Stanley, R. P. (1999). Enumerative Combinatorics, 2. Cambridge University Press, Cambridge.

IMS Lecture Notes–Monograph Series Asymptotic: Particles, Processes and Inverse Problems Vol. 55 (2007) 65–84 c Institute of Mathematical Statistics, 2007
DOI: 10.1214/074921707000000274

Scale space consistency of piecewise constant least squares estimators – another look at the regressogram Leif Boysen

1,∗

, Volkmar Liebscher2,† , Axel Munk1 and Olaf Wittich3,†

Universit¨ at G¨ ottingen, Universit¨ at Greifswald, Universit¨ at G¨ ottingen and Technische Universiteit Eindhoven Abstract: We study the asymptotic behavior of piecewise constant least squares regression estimates, when the number of partitions of the estimate is penalized. We show that the estimator is consistent in the relevant metric if the signal is in L2 ([0, 1]), the space of c` adl` ag functions equipped with the Skorokhod metric or C([0, 1]) equipped with the supremum metric. Moreover, we consider the family of estimates under a varying smoothing parameter, also called scale space. We prove convergence of the empirical scale space towards its deterministic target.

1. Introduction Initially, the use of piecewise constant functions for regression has been proposed by [25], who called the corresponding reconstruction the regressogram. [25] proposed it as a simple exploratory tool. For a given set of jump locations, the regressogram simply averages the data between two successive jumps. A difficult issue, however, is a proper selection of the location of jumps and its convergence analysis. Approximation by step functions is well examined in approximation theory (see e.g., [7]), and there are several statistical estimation procedures which use locally constant reconstructions. [14] studied the case where the signal is a step function with one jump and showed that in this case the signal can be estimated at the parametric n−1/2 -rate and that the jump location can be estimated at a rate of n−1 . This was generalized by [28] and [29] to step functions with a given a known upper bound for the number of jumps. The locally adaptive regression splines method by [16] and the taut string procedure by [6] use locally constant estimates to reconstruct unknown regression functions, which belong to more general function classes. Both methods reduce the complexity of the reconstruction by minimizing the total variation of the estimator, which in turn leads to a small number of local extreme values. ∗ Supported by Georg Lichtenberg program “Applied Statistics & Empirical Methods” and DFG graduate program 1023 “Identification in Mathematical Models”. † Supported in part by DFG, Sonderforschungsbereich 386 “Statistical Analysis of Discrete Structures”. ‡ Supported by DFG grant “Statistical Inverse Problems under Qualitative Shape Constraints”. 1 Institute for Mathematical Stochastics, Georgia Augusta University Goettingen, Maschmuehlenweg 8-10, D-37073 Goettingen, Germany, e-mail: [email protected]; [email protected] 2 University Greifswald. 3 Technical University Eindhoven. AMS 2000 subject classifications: Primary 62G05, 62G20; secondary 41A10, 41A25. Keywords and phrases: Hard thresholding, nonparametric regression, penalized maximum likelihood, regressogram, scale spaces, Skorokhod topology.

65

66

Boysen et al.

In this work we choose a different approach and define the complexity of the reconstruction by the number of intervals where the reconstruction is constant, or equivalently by the number of jumps of the reconstruction. Compared to the total variation approach, this method obviously captures extreme plateaus more easily but is less robust to outliers. This might be of interest in applications where extreme plateaus are informative, like for example in mass spectroscopy. Throughout the following, we assume a regression model of the type (1)

Yi,n = f¯i,n + ξi,n ,

(i = 1, . . . , n),

where (ξi,n )i=1,...,n is a triangular array of independent zero-mean random variables and f¯i,n is the mean value of a square integrable function f ∈ L2 ([0, 1)) over the interval [(i − 1)/n, i/n] (see e.g. [9]), (2)

f¯i,n = n




i/n

f (u) du. (i−1)/n

This model is well suited for physical applications, where observations of this type are quite common. We consider minimizers Tγ (Yn ) ∈ argmin Hγ (·, Yn ) of the hard thresholding functional n

(3)

Hγ (u, Yn ) = γ · #J(u) +

1 (ui − Yi,n )2 , n i=1

where J(u) = {i : 1 ≤ i ≤ n − 1, ui = ui+1 } is the set of jumps of u. In the following we will call the minimizers of (3) jump penalized least squares estimators or short Jplse. Clearly choosing γ is equivalent to choosing a number of partitions of the Jplse. Figure 1 shows the Jplse for a sample dataset and different choices of the smoothing parameter γ. This paper complements work of the authors on convergence rates of the Jplse. [2] show that given a proper choice of the smoothing parameter γ it is possible to obtain optimal rates for certain classes of approximation spaces under the assumption of subgaussian tails of the error distribution. As special cases the class of piecewise H¨ older continuous functions of order 0 < α ≤ 1 and the class of functions with bounded total variation are obtained. In this paper we show consistency of regressograms constructed by minimizing (3) for arbitrary L2 functions and more general assumptions on the error. If the true function is c` adl` ag, we additionally show consistency in the Skorokhod topology. This is a substantially stronger statement than the L2 convergence and yields consistency of the whole graph of the estimator. In concrete applications the choice of the regularization parameter γ > 0 in (3), which controls the degree of smoothness (which means just the number of jumps) of the estimate Tγ (Yn ), is a delicate and important task. As in kernel regression [18, 23], a screening of the estimates over a larger region can be useful (see [16, 26]). Adapting a viewpoint from computer vision (see [15]), [3, 4] and [17] proposed to consider the family (Tγ (f ))γ>0 , denoted as scale space, as target of inference. This was justified in [4] by the fact that the empirical scale space converges towards that of the actual density or regression function pointwisely and uniformly on compact

Scale space consistency of regressograms

67

Fig 1. The Jplse for different values of γ. The dots represent the noisy observations of some signal f represented by the grey line. The black line shows the estimator, with γ chosen such that the reconstruction has four, six, eight and ten partitions, respectively.

sets. The main motivation for analyzing the scale space is exploration of structures as peaks and valleys in regression and detection of modes in density estimation. Properties of the scale space in kernel smoothing are that structures like modes disappear monotonically for a shrinking resolution level and that the reconstruction changes continuously with respect to the bandwidth. For the Jplse, the family (Tγ (f ))γ>0 behaves quite differently. Notable distinctions are that jumps may not change monotonically and that there are only finitely many possible different estimates. To deal with these features, we consider convergence of the scale space in the space of c` adl` ag functions equipped with the Skorokhod J1 topology. In this setting we deduce (under identifiability assumptions) convergence of the empirical scale space towards its deterministic target. Note that the computation of the empirical scale space is feasible. The family (Tγ (Yn )))γ>0 can be computed in O(n3 ) and the minimizer for one γ in O(n2 ) steps (see [26]). The paper is organized as follows. After introducing some notation in Section 2, we provide in Section 3.1 the consistency results for general functions in the L2 metric. In Section 3.2 we present the results of convergence in the Skorokhod topology. Finally in Section 3.3 convergence results for the scale space are given. The proofs as well as a short introduction to the concept of epi-convergence, which is required in the main part of the proofs, are given in the Appendix. 2. Model assumptions By S([0, 1)) = span{1[s,t) : 0 ≤ s < t ≤ 1} we will denote the space of step functions with a finite but arbitrary number of jumps and by D([0, 1)) the c` adl` ag space of right continuous functions on [0, 1] with left limits and left continuous at 1. Both will be considered as subspaces of L2 ([0, 1)) with the obvious identification of a function with its equivalence class, which is injective for these two spaces. More generally, by D([0, 1), Θ) and D([0, ∞), Θ) we will denote spaces of functions with

Boysen et al.

68

values in a metric space (Θ, ρ), which are right continuous and have left limits.  ·  will denote the norm of L2 ([0, 1)) and the norm on L∞ ([0, 1)) is denoted by  · ∞ . Minimizers of the hard thresholding functionals (3) will be embedded into L2 ([0, 1)) by the map ιn : Rn −→ L2 ([0, 1)), ιn ((u1 , . . . , un )) =

n 

ui 1[(i−1)/n,i/n) .

i=1

Under the regression model (1), this leads to estimates fˆn = ιn (Tγn (Yn )), i.e. fˆn ∈ ιn (argmin Hγn (·, Yn )). Note that, for a functional F we denote by argmin F the whole set of minimizers. Here and in the following (γn )n∈N is a (possibly random) sequence of smoothing parameters. We suppress the dependence of fˆn on γn since this choice will be clear from the context. For the noise, we assume the following condition. (A) For all n ∈ N the random variables (ξi,n )1≤i≤n are independent. Moreover, there exists a sequence (βn )n∈N with n−1 βn → 0 such that (ξi,n + · · · + ξj,n )2 ≤ βn 1≤i≤j≤n j−i+1

(4)

max

P-a.s.,

for almost every n. The behavior of the process (4) is well known for certain classes of i.i.d. subgaussian random variables (see e.g. [22]). If for example ξi,n = ξi ∼ N (0, σ 2 ) for all i = 1, . . . , n and all n, we can choose βn = 2σ 2 log n in Condition (A). The next result shows that (A) is satisfied for a broad class of subgaussian random variables. Lemma 1. Assume the noise satisfies the following generalized subgaussian condition (5)

Eeνξi,n ≤ eαn

ζ

ν2

,

(for all ν ∈ R, n ∈ N, 1 ≤ i ≤ n)

with 0 ≤ ζ < 1 and α > 0. Then there exist a C > 0 such that for βn = Cnζ log n Condition (A) is satisfied. A more common moment condition is given by the following lemma. Lemma 2. Assume the noise satisfies (6)

sup E|ξi,n |2m < ∞, i,n

(for all n ∈ N, 1 ≤ i ≤ n)

for m > 2. Then for all C > 0 and βn = C(n log n)2/m Condition (A) is satisfied. 3. Consistency In order to extend the functional in (3) to L2 ([0, 1)), we define for γ > 0, the functionals Hγ∞ : L2 ([0, 1)) × L2 ([0, 1)) −→ R ∪ ∞:  2 γ · #J (g) + f − g , g ∈ S([0, 1)), ∞ Hγ (g, f ) = ∞, otherwise.

Scale space consistency of regressograms

69

Here J (g) = {t ∈ (0, 1) : g(t−) = g(t+)} is the set of jumps of g ∈ S([0, 1)). For γ = 0, we set H0∞ (g, f ) = f − g for all g ∈ L2 ([0, 1)). The following lemma guarantees the existence of a minimizer. 2

Lemma 3. For any f ∈ L2 ([0, 1)) and all γ ≥ 0 we have argmin Hγ∞ (·, f ) = ∅. In the following we assume that Yn is determined through (1), the noise ξn satisfies (A) and (βn )n∈N is a sequence with βn /n → 0 such that (4) holds. 3.1. Convergence in L2 We start with investigating the asymptotic behavior of the Jplse when the sequence γn converges to a constant γ greater than zero. In this case we do not recover the original function in the limit, but a parsimonious representation at a certain scale of interest determined by γ. Theorem 1. Suppose that f ∈ L2 ([0, 1)) and γ > 0 are such that fγ is a unique minimizer of Hγ∞ (·, f ). Then for any (random) sequence (γn )n∈N ⊂ (0, ∞) with γn → γ P-a.s., we have L2 ([0,1)) fˆn −−−−−→ fγ P-a.s. n→∞

The next theorem states the consistency of the Jplse towards the true signal for γ = 0 under some conditions on the sequence γn . (H) (γn )n∈N satisfies γn → 0 and γn n/βn → ∞ P-a.s.. Theorem 2. Assume f ∈ L2 ([0, 1)) and (γn )n∈N satisfies (H). Then 2

L ([0,1)) fˆn −−−−−→ f, n→∞

P-a.s.

3.2. Convergence in Skorokhod topology As we use c` adl` ag functions for reconstructing the original signal, it is natural to ask, whether it is possible to obtain consistency in the Skorokhod topology. We remember the definition of the Skorokhod metric [12, Section 5 and 6]. Let Λ∞ denote the set of all strictly increasing continuous functions λ : R+ −→ R+ which are onto. We define for f, g ∈ D([0, ∞), Θ) ρ(f (λ(t) ∧ u), g(t)) where L(λ) = sups=t≥0 | log λ(t)−λ(s) |. Similarly, Λ1 is the set of all strictly increast−s ing continuous onto functions λ : [0, 1] −→ [0, 1] with appropriate definition of L. Slightly abusing notation, we set for f, g ∈ D([0, 1), Θ),   ρS (f, g) = inf max(L(λ), sup ρ(f (λ(t)), g(t))) : λ ∈ Λ1 . 0≤t≤1

The topology induced by this metric is called J1 topology. After determining the metric we want to use, we find that in the situation of Theorem 1 we can establish consistency without further assumptions, whereas in the situation of Theorem 2 f has to belong to D([0, 1)).

Boysen et al.

70

Theorem 3.

(i) Under the assumptions of Theorem 1, D([0,1)) fˆn −−−−−→ fγ n→∞

P-a.s.

(ii) If f ∈ D([0, 1)) and (γn )n∈N satisfies (H), then D([0,1)) fˆn −−−−−→ f n→∞

P-a.s.

If f is continuous on [0, 1], then ∞

L ([0,1]) fˆn −−−−−−→ f n→∞

P-a.s.

3.3. Convergence of the scale spaces As mentioned in the introduction, following [4], we now want to study the scale space family (Tγ (f ))γ>0 as target for inference. First we show that the map γ → Tγ (f ) can be chosen piecewise constant with finitely many jumps. Lemma 4. Let f ∈ L2 ([0, 1)). Then there exists a number m(f ) ∈ N ∪ {∞} and a m(f ) decreasing sequence (γm )m=0 ⊂ R ∪ ∞ such that (i) γ0 = ∞, γm(f ) = 0, (ii) for all 1 ≤ i ≤ m(f ) and γ  , γ  ∈ (γi , γi−1 ) we have that argmin Hγ∞ (·, f ) = argmin Hγ∞ (·, f ) , (iii) for all 1 ≤ i ≤ m(f ) − 1 and γi+1 < γ  < γi < γ  < γi−1 we have: argmin Hγ∞i (·, f ) ⊇ argmin Hγ∞ (·, f ) ∪ argmin Hγ∞ (·, f ) , and (iv) for all γ  > γ1 ∞ argmin H∞ (·, f ) = argmin Hγ∞ (·, f ) = {T∞ (f )} .  Here T∞ (f ) is defined by T∞ (f )(x) = f (u) du1[0,1) (x).

Thus we may consider functions τˆn ∈ D([0, ∞), L2 ([0, 1))) with τˆn (ζ) ∈ ιn (argmin H1/ζ (·, Yn )) ,

for all ζ ≥ 0. We will call τˆn the empirical scale space. Similarly, we define the deterministic scale space τ for a given function f , such that (7)

∞ (·, f )), τ (ζ) ∈ argmin H1/ζ

(for all ζ ≥ 0).

The following theorem shows that the empirical scale space converges almost surely to the deterministic scale space. Table 1 and Figure 2 demonstrate this in a finite setting for the blocks signal, introduced by [10]. Theorem 4. Suppose f ∈ L2 ([0, 1)) is such that # argmin Hγ∞ (·, f ) = 1 for all but a countable number of γ > 0 and # argmin Hγ∞ (·, f ) ≤ 2 for all γ ≥ 0. Then τ is uniquely determined by (7). Moreover, τˆn −−−−→ τ n→∞

P-a.s.

holds both in D([0, ∞), D([0, 1))) and D([0, ∞), L2 ([0, 1))).

Scale space consistency of regressograms

71

Fig 2. Comparison of scale spaces. The “Blocks” data of [11] sampled at 64 points (dots) are compared with the different parts of the scale space derived both from the data (black) and the original signal (grey), starting with γ = ∞ and lowering its value from left to right and top to bottom. Note that for the original sampling rate of 2048 the scale spaces are virtually identical.

Boysen et al.

72

Table 1 Comparison of scale spaces. For the “Blocks” data of [10] sampled in 64 points with a signal to noise ratio of 7, the eleven largest γ values (see Lemma 4) for the deterministic signal (bottom) and the noisy signal (top) are compared. The last two values of the bottom row are equal to zero, since there are only nine ways to reconstruct the deterministic signal 852 885

217 249

173 159

148 142

108 100

99.8 99.1

55.9 80.2

46.6 41.3

5.36 38.9

4.62 0

2.29 0

Fig 3. Scale spaces of a sample function (grey line). The black lines show all reconstructions of the sample function for varying γ.

Discussion. The scale space of a penalized estimator with hard thresholding type penalties generally does not have the same nice properties as its counterparts stemming from an l2 - or l1 -type penalty. In our case the function value at some point of the reconstruction does not change continuously or monotonically in the smoothing parameter. Moreover, the set of jumps of a best reconstruction with k partitions is not necessarily contained in the set of jumps of a best reconstruction with k  partitions for k < k , see Figure 3. This leads to increased computational costs, as greedy algorithms in general do not yield an optimal solution. Indeed, one needs only O(n log n) steps to compute the estimate for a given γ if the penalty is of l1 type as in locally adaptive regression splines by [16], compared to O(n2 ) steps for the Jplse. We mention, that penalizing the number of jumps corresponds to an L0 -penalty and is a limiting case of the [20] functional, when the dimension of the signal (image) is d = 1 [27], and results in “hard segmentation” of the data [24].

4. Proofs Some additional notation. Throughout this section, we shorten J(fˆn ) to Jn . We set Sn ([0, 1)) = ιn (Rn ), Bn = σ(Sn ([0, 1))). Observe that ιn (f¯n ) is just the conditional expectation EU0,1 (f |Bn ), denoting the uniform distribution on [0, 1) by U0,1 . Similarly, for any finite J ⊂ (0, 1) define BJ = σ({[a, b) : a, b ∈ J ∪ {0, 1}}) and the partition PJ = {[a, b) : a, b ∈ J ∪ {0, 1}, (a, b) ∩ J = ∅}. For our proofs it is convenient to formulate all minimization procedures on L2 ([0, 1)). Therefore we ¯ defined ˜ γ : L2 ([0, 1)) × L2 ([0, 1)) −→ R, ˜∞ , H introduce the following functionals H γ as  2 2 ˜ γ (g, f ) = γ#J(g) + f − g − f  , if g ∈ Sn ([0, 1)), H ∞, otherwise, ˜ ∞ (g, f ) = H ∞ (g, f ) − f 2 . H γ γ

Scale space consistency of regressograms

73

˜ ∞ has the same minimizers as H ∞ , differing only by a constant. Clearly for each f , H γ γ ˜ γ and Hγ . The following Lemma relates the minimizers of H Lemma 5. For all f ∈ L2 ([0, 1)) and n ∈ N we have u ∈ argmin Hγ (·, f¯n ) if and ˜ γ (·, f ). Similarly, u ∈ argmin Hγ (·, y) for y ∈ Rn if and only if ιn (u) ∈ argmin H n ˜ only if ι (u) ∈ argmin Hγ (·, ιn (y)). Proof. The second assertion follows from the fact that for u, y ∈ Rn ˜ γ (ιn (u), ιn (y)) = Hγ (u, y) − f 2 . H Further, for u ∈ Rn we have ιn (f¯n ) − f, ιn (f¯n ) − ιn (u) = 0 which gives ˜ γ (ιn (u), f ) = γ#J(u) + f − g2 − f 2 H  2  2 = γ#J(u) + ιn (f¯n ) − ιn (u) + f − ιn (f¯n ) − f 2 = Hγ (u, f¯n ) + constf,n what completes the proof. ˜ γ (·, f ) and H ˜ γ∞ (·, f ) for γ > 0 are determined The minimizers g ∈ S([0, 1)) of H by their jump set J(g) through the formula g = EU0,1 (f |BJ(g) ). In the sequel, we abbreviate
µI (f ) = (I)−1

f (u) du

I

to denote the mean of f on some interval I. In addition, we will use the abbreviation fJ := EU0,1 (f |BJ ), such that for any partition PJ of [0, 1) fJ =



µI (f )1I .

I∈PJ

Further, we extend the noise in (1) to L2 ([0, 1)) by ξn = ιn ((ξ1,n , . . . , ξn,n )). 4.1. Technical tools We start by giving estimates on the behavior of (ξn )J =



I∈PJ

µI (ξn )1I .

Lemma 6. Assume (ξi,n )n∈N,1≤i≤n satisfies (A). Then P-almost surely for all intervals I ⊂ [0, 1) and all n ∈ N µI (ξn )2 ≤

βn . n (I)

Proof. For intervals of the type [(i − 1)/n, j/n) with i ≤ j ∈ N the claim is a direct consequence of (4). For general intervals, [(i + p1 )/n, (j − p2 )/n) with p1 , p2 ∈ [0, 1], we have to show that (p1 · ξi,n + ξi+1,n + · · · + ξj−1,n + p2 · ξj,n )2 − βn (p1 + p2 + j − i − 1) ≤ 0. The left expression is convex over [0, 1]2 if it is considered as function in (p1 , p2 ). Hence it attains its maximum in an extreme point of [0, 1]2 .

Boysen et al.

74

Lemma 7. There is a set of P-probability one on which for all sequences (Jn )n∈N of finite sets in (0, 1) the relation limn→∞ βn #Jn /n = 0 implies L2 ([0,1))

(ξn )Jn −−−−−→ 0. n→∞

Proof. By Lemma 6 we find (ξn )Jn 2 =

(8)



(I)µI (ξn )2 ≤

I∈PJn

βn (#Jn + 1), n

This immediately gives the assertion. Now we wish to show that the functionals epi-converge (see section 4.4). To this end we need two more results. Lemma 8. Let (Jn )n∈N be a sequence of closed subsets in (0, 1) which satisfies the relation limn→∞ βn #Jn /n = 0. For (gn )n∈N ⊂ L2 ([0, 1)) with gn − g −−−−→ 0, n→∞ where gn is BJn measurable, we have almost surely 2

2

2

f + ξn − gn  − f + ξn  −−−−→ f − g − f 2 . n→∞

Proof. First observe that 2

2

2

f + ξn − gn  − f + ξn  = gn  − 2f, gn  − 2ξn , gn  2

= gn  − 2f, gn  − 2(ξn )Jn , gn  . Since the sequence (gn )n∈N is bounded we can use Lemma 7 to deduce P-a.s. (ξn )Jn , gn  −−−−→ 0. n→∞

This completes the proof. Before stating the next result, we recall the definition of the Hausdorff metric ρH on the space of closed subsets CL(Θ) of a compact metric space (Θ, ρ). For Θ ⊆ Θ  ϑ we set dist(ϑ, Θ ) = inf{ρ(ϑ, ϑ ) : ϑ ∈ Θ }. Define   max{supx∈A dist(x, B), supy∈B dist(y, A)}, ρH (A, B) = 1,   0,

A, B = ∅, A = B = ∅, A = B = ∅,

With this metric, CL(Θ) is again compact for compact Θ [19, see]. Lemma 9. The map L2 ([0, 1))  g →



#J(g), g ∈ S([0, 1)) ∈ N ∪ {0, ∞} ∞, g ∈ S([0, 1))

is lower semi-continuous, meaning the set {g ∈ S([0, 1)) : #J(g) ≤ N } is closed for all N ∈ N ∪ {0}.

Scale space consistency of regressograms

75

Proof. Suppose that gn − g −−−−→ 0 with #J(gn ) ≤ N < #J(g). Using compactn→∞

ness of the space of closed subsets CL([0, 1]) and turning possibly to a subsequence, we could arrange that J(gn ) ∪ {0, 1} −−−−→ J ∪ {0, 1} for some closed J ⊂ (0, 1), n→∞ where convergence is understood in Hausdorff metric ρH . Since the cardinality is lower semi-continuous with respect to the Hausdorff metric, J must be finite. We conclude for (s, t) ∩ J = ∅ and ε > 0 that (s + ε, t − ε) ∩ J(gn ) = ∅ eventually, i.e. gn is constant on (s + ε, t − ε). Next we observe that gn 1(s+ε,t−ε) converges towards g1(s+ε,t−ε) (in L2 ([0, 1))) what implies that g is constant on (s + ε, t − ε). Since ε > 0 was arbitrary, we derive that g is constant on (s, t). Consequently, g is in S([0, 1)) and J(g) ⊆ J. Using again lower semi-continuity of the cardinality in the space of compact subsets of [0, 1] shows that #J(g) > N ≥ lim sup #J(gn ) ≥ lim inf #J(gn ) ≥ #J ≥ #J(g). n

n

This contradiction completes the proof. ˜ γ as function on L2 ([0, 1)). Now we can state the epi-convergence of H n Lemma 10. For all sequences (γn )n∈N satisfying (H) we have ˜ γ (·, f + ξn ) −−epi ˜ ∞ (·, f ) H −−→ H γ n n→∞

˜ ∞ are considered as functionals on L2 ([0, 1)). ˜γ , H almost surely. Here H γ n Proof. We have to show that on a set with probability one we have ˜ γ (gn , f + ξn ) ≥ H ˜ ∞ (g, f ). (i) If gn −−−−→ g then lim inf n→∞ H γ n n→∞

(ii) For all g ∈ L2 ([0, 1)), there exists a sequence (gn )n∈N ⊂ L2 ([0, 1)), gn −−−−→ g n→∞ ˜ γ (gn , f + ξn ) ≤ H ˜ γ∞ (g, f ). with lim supn→∞ H n

To this end, we fix the set where the assertions of Lemmas 7 and 8 hold simultaneously. Ad 4.1: Without loss of generality, we may assume that Hγn (gn , f +ξn ) converges ˜ ∞ (g, f )/γn the / Sn ([0, 1)) for infinitely many n or #J(gn ) > H in R ∪ ∞. If gn ∈ γ relation 4.1 is trivially fulfilled. Otherwise, we obtain lim sup n→∞

βn βn ˜ ∞ Hγ (g, f ) = 0. #J(gn ) ≤ lim sup n n→∞ nγn

Hence we can apply Lemma 8. Together with Lemma 9 we obtain P-a.s. ˜ γ (gn , f + ξn ) lim inf H n n→∞

2

2

≥ lim inf γn J(gn ) + lim inf (f + ξn − gn  − f + ξn  ) n→∞

n→∞

˜ ∞ (g, f ). ≥ γJ(g) + (f − g − f 2 ) = H γ 2

Ad 4.1: If g ∈ / S([0, 1)) and γ > 0 there is nothing to prove. If γ = 0 and still g∈ / S([0, 1)), choose gn as a best L2 -approximation of g in Sn ([0, 1)) with at most √ 1/ γn jumps. We claim that gn − g → 0 as n → ∞. For that goal, let g˜n,k denote a best approximation of g in {f ∈ Sn ([0, 1)) : #J(f ) ≤ k} and g˜k one in {f ∈ S([0, 1)) : #J(f ) ≤ k}.

Boysen et al.

76

Moreover, for every n, k let Jnk ⊂ (0, 1) be a perturbation of J(˜ gk ), with nJnk ∈ N,  gk ) and ρH (Jnk , J(˜ gk )) ≤ 1/n. Denote gn,k = g˜k ◦ λn,k where λn,k ∈ Λ1 #Jnk = #J(˜ fulfills λn,k (Jnk ) = J(˜ gk ). Since (a, b) → 1[a,b) is continuous in L2 ([0, 1)), we obtain  readily gn,k − g˜k  → 0. This implies for any k ∈ N    gk − g . gn,k − g ≤ lim sup gn,k − g  = ˜ lim sup gn − g ≤ lim sup ˜ n→∞

n→∞

n→∞

Since the right hand side can be made arbitrary small by choosing k, gn converges to g. Then Lemma 8 yields 4.1. If γ > 0 and g ∈ S([0, 1)), gn is chosen as a best approximation of g in Sn ([0, 1)) with at most #J(g) jumps. Finally, in order to obtain 4.1, argue as before.

To deduce consistency with the help of epi-convergence, one needs to show that the minimizers are contained in a compact set. The following lemma will be applied to this end. Lemma 11. Assume (Θ, ρ) is a metric space. A subset A ⊂ D([0, ∞), Θ) is relatively compact if the following two conditions hold (B1) For all t ∈ R+ there is a compact Kt ⊆ Θ such that g(t) ∈ Kt ,

(for all g ∈ A).

(B2) For all T > 0 and all ε > 0 there exists a δ > 0 such that for all g ∈ A there is a step function gε ∈ S([0, T ), Θ) such that sup{ρ(g(t), gε (t)) : t ∈ [0, T )} < ε

and

mpl(gε ) ≥ δ ,

where mpl is the minimum distance between two jumps of f ∈ S([0, T )) mpl(f ) := min{|s − t| : s = t ∈ J(f ) ∪ {0, T }}. A subset A ⊂ D([0, 1), Θ) is relative compact if the following two conditions hold (C1) For all t ∈ [0, 1] there is a compact Kt ⊆ Θ such that g(t) ∈ Kt

(for all g ∈ A).

(C2) For all ε > 0 there exists a δ > 0 such that for all g ∈ A there is a step function gε ∈ S([0, 1), Θ) such that sup{ρ(g(t), gε (t)) : t ∈ [0, 1]} < ε

and

mpl(gε ) ≥ δ .

Proof. We prove only the first assertion, as the proof of the second assertion can be carried out in the same manner. According to [12], Theorem 6.3, it is enough to show that (B2) implies lim sup wg (δ, T ) = 0

δ→0 g∈A

where wg (δ, T ) = inf



max

sup

1≤i≤v s,t∈[ti−1 ,ti )

ρ(g(s), g(t)) : {t1 , . . . , tv−1 } ⊂ (0, T ),  t0 = 0, tv = T, |ti − tj | > δ .

Scale space consistency of regressograms

77

So, fix T > 0, ε > 0 and choose δ from (B2). Then we set for g ∈ A {t0 , . . . , tv } = J(gε )∪{0, T }. Clearly, mpl(gε ) > δ implies |ti −tj | > δ for all i = j. For neighboring ti−1 , ti ∈ J(gε ) ∪ {0, T } and s, t ∈ [ti−1 , ti ) we derive ρ(g(s), g(t)) ≤ ρ(g(s), gε (s)) + ρ(gε (s), gε (t)) + ρ(gε (t), g(t)) < ε + 0 + ε = 2ε. This establishes the above condition and completes the proof. In the context of proving compactness we will also need the following result. Lemma 12. For any f ∈ L2 ([0, 1)) the set {fJ : J ⊂ (0, 1), #J < ∞} is relatively compact in L2 ([0, 1)). Proof. The proof is done in several steps. 1. Since (s, t) → 1[s,t) is continuous,

M  αi 1Ii : |αi | ≤ z, Ii ⊆ [0, 1) interval i=1

is the continuous image of a compact set and hence compact for all M ∈ N and z > 0. 2. If f = 1I for some interval I, we obtain for any J ⊂ [0, 1) that fJ is a linear combination  of at most three different indicator functions. M  M 3. If f = i=1 αi 1Ii is a step function and J arbitrary then fJ = j=1 βj 1Ij holds by 2. for some M  ≤ 3M . Using βj = µIj (f ) ≤

max |αi |

i=1,...,M

as well as 1., we get that {fJ : J ⊂ [0, 1)} is relatively compact for step functions f. 4. Suppose f ∈ L2 ([0, 1)) is arbitrary and ε > 0. We want to show that we can cover {fJ : J ⊂ [0, 1)} by finitely many ε-balls. Fix a step function g such that f − g < ε/2. By the Jensen Inequality for conditional expectations, we get fJ − gJ  < ε/2 for all finite J ⊂ [0, 1). Further, by 3., there are finite sets J1 , . . . , Jp ⊂ [0, 1) with p < ∞ such that minl=1,...,p gJ − gJl  < ε/2 for all finite J ⊂ [0, 1). This implies min fJ − gJi  ≤ min gJ − gJl  + fJ − gJ  < ε

l=1,...,p

l=1,...,p

and the proof is complete. 4.2. Behavior of the partial sum process Proof of Lemma 1. The following Markov inequality is standard for triangular arrays fulfilling condition (A), [21], Section III, §4, and all numbers µi , i = 1, . . . , n: P(|

n 

µi ξi,n | ≥ z) ≤ 2 exp

i=1



 −z 2  4αnζ i µ2i

(for all z ∈ R).

From this, we derive for z 2 > 12α that     P(|ξi,n + · · · + ξj,n | ≥ z j − i + 1 nζ log n) n∈N 1≤i≤j≤n

≤2



n∈N

n2 e−

z 2 log n 4α

=2

 n

n−

z 2 −8α 4α

< ∞.

Boysen et al.

78

Hence, for  > 0 we have with probability one that (ξi,n + · · · + ξj,n )2 ≥ (12 + )αnζ log n 1≤i≤j≤n (j − i + 1) max

only finitely often. For the proof of Lemma 2, we need an auxiliary lemma. Denote by  Dn = (i, j) : 1 ≤ i ≤ j ≤ n such that i = k2l , j = (k + 1)2l  for some l, k ∈ {0, 1, 2, . . .}

the set of all pairs (i, j) which are endpoints of dyadic intervals contained in {1, . . . , n}. Lemma 13. Assume x ∈ Rn such that (9)

max

(i,j)∈Dn

|xi + · · · + xj | √ ≤c j−i+1

for some c > 0. Then √ |xi + · · · + xj | √ ≤ (2 + 2)c . 1≤i≤j≤n j−i+1 max

Proof. Without loss of generality we may assume that n = 2m for some m ∈ N (and add some zeros otherwise). First, we prove by induction on m that (9) implies (10)

√ |x1 + · · · + xj | √ ≤ (1 + 2)c . 1≤j≤n j max

For m = 0 there is nothing to prove. Now assume that the statement is true for m. Let 2m < j ≤ 2m+1 . Note that √ √ j − 2m |x2m +1 + · · · + xj | 2m |x1 + · · · + x2m | |x1 + · · · + xj | √ √ √ √ + . ≤ √ j j j j − 2m 2m Apply the induction hypothesis to the second summand to obtain √ √  √ 2m + (1 + 2) j − 2m |x1 + · · · + xj | √ √ ≤ c. j j For 2m + 1 ≤ j ≤ 2m+1 the √expression on the right hand side is maximal for j = 2m+1 with maximum (1 + 2)c. Hence the statement holds also for m + 1 and we have shown that (9) implies (10). The claim is again proven by induction on m. For m = 0 there is nothing to prove. Assume that the statement is true for m. If i ≤ j ≤ 2m or 2m < i ≤ j ≤ 2m+1 the statement follows by application of the induction hypotheses to (x1 , . . . , x2m ) and (x2m +1 , . . . , x2m+1 ), respectively. Now suppose i < 2m < j. Then √ 2m − i + 1 |xi + · · · + x2m | |xi + · · · + xj | √ √ ≤ √ j−i+1 j−i+1 2m − i + 1 √ m |x2m +1 + · · · + xj | j−2 √ +√ j−i+1 j − 2m

Scale space consistency of regressograms

79

Application of (10) to x = (x2m , x2m −1 , . . . , x1 ) and x ˜ = (x2m +1 , . . . , x2m+1 ) then gives √ √ √ √ √ |xi + · · · + xj | 2m − i + 1 + j − 2m √ √ ≤ (1 + 2)c ≤ 2(1 + 2)c . j−i+1 j−i+1 Proof of Lemma 2. [8] show that for m ≥ 1 and some constant Cm depending on m only  |ξ + · · · + ξ |2m  E|ξi,n |2m + · · · + E|ξj,n |2m i,n j,n ≤ C . E m (j − i + 1)m j−i+1 The Markov inequality then yields for any z > 0 and all 1 ≤ i ≤ j ≤ n  |ξ + · · · + ξ |  Cm sup E|ξi,n |2m i,n j,n i,n √ P . ≥z ≤ z 2m j−i+1

Since there are at most 2n dyadic intervals contained in {1, . . . , n}, we obtain by Lemma 13 for any C > 0 that  |ξ + · · · + ξ |    i,n j,n √ P ≥ C(n log n)1/m j−i+1 n∈N 1≤i≤j≤n     √ |ξi,n + · · · + ξj,n | √ ≤ P ≥ (2 + 2)C(n log n)1/m j−i+1 n∈N (i,j)∈Dn

≤

Cm supi,n E|ξi,n |2m  2n √ < ∞. 2 2m 2m n log2 n (2 + 2) C n∈N

The claim follows by application of the Borel–Cantelli lemma. 4.3. Consistency of the estimator The proofs in this section use the concept of epi-convergence. It is introduced in Appendix. Proof of Lemma 3. For γ = 0 there is nothing to prove. Assume γ > 0 and g ∈ S([0, 1)) with #J(g) > f 2 /γ. This yields Hγ∞ (0, f ) = f 2 < Hγ∞ (g, f ) . Moreover, observe that for g ∈ S([0, 1)) we have Hγ∞ (g, f ) ≥ Hγ∞ (fJ(g) , f ). Thus, it is enough to regard the set {fJ : #J ≤ f 2 /γ}, which is relatively compact in L2 ([0, 1)) by Lemma 12. This proves the existence of a minimizer. Proof of Theorem 1 and Theorem 2. By the reformulation of the minimizers in Lemma 5, Lemma 10 and Theorem 5 (see Appendix) it is enough to prove that almost surely there is a compact set containing  ˜ γ (·, f + ξn ) . argmin H n n∈N

˜ γ (·, f + ξn ) have the form (f + ξn )J for some First note that all fn ∈ argmin H n n ˜ γ (fn , f + ξn ) with H ˜ γ (0, f + ξn ) = 0, we obtain (random) sets Jn . Comparing H n n the a priori estimate 2

2

γn #Jn ≤ (f + ξn )Jn  ≤ 2f 2 + 2 (ξn )Jn  ≤ 2f 2 +

2βn (#Jn + 1) n

Boysen et al.

80

for all n ∈ N. Since γn >

4βn n

eventually, we find P-a.s.

#Jn ≤

2f 2 + γn −

2βn n 2βn n

= O(γn−1 ).

Application of Lemma 7 gives limn→∞ (ξn )Jn = 0 almost surely. Since by Lemma 12, 2 {f  Jn : n ∈ N}˜ is relatively compact in L ([0, 1)), relative compactness of the set n∈N argmin Hγn (·, f + ξn ) follows immediately. This completes the proofs.

Proof of Theorem 3, part (i) . Theorem 1 and Lemma 9 imply lim inf #Jn ≥ #J(fγ ) . n→∞

Suppose lim supn→∞ #Jn ≥ #J(fγ ) + 1. Let fγ,n be an approximation of fγ from Sn ([0, 1)) with the same number of jumps as fγ . Then we could arrange fγ,n −−−−→ n→∞ ˜ γ (fγ,n , f + ξn ) = H ˜ ∞ (fγ , f ). Moreover, we know fγ such that limn→∞ H γ

˜ γ (fˆn , f + ξn ) ≥ γ + H ˜ γ (fγ,n , f + ξn ) ˜ ∞ (fγ , f ) = γ + lim H lim sup H γ n→∞

n→∞

˜ γ (·, f + ξn ) for all n. Therefore, which contradicts that fˆn is a minimizer of H #Jn = #J(fγ ) eventually. Next, chose by compactness a subsequence such that Jn ∪ {0, 1} converges in ρH . Then, by Lemma 9, the limit must be J(fγ ) ∪ {0, 1}. Consequently, the whole sequence (Jn )n∈N converges to J(fγ ) in the Hausdorff metric. Thus eventually, there is a 1-1 correspondence between PJn and PJ(fγ ) such that for each [s, t) ∈ PJ(fγ ) there are [sn , tn ) ∈ PJn with sn −−−−→ s n→∞

and

tn −−−−→ t . n→∞

By Lemma 6 and continuity of (s, t) → 1[s,t) , we find µ[sn ,tn ) (f + ξn ) −−−−→ µ[s,t) (f ) . n→∞

Construct λn ∈ Λ1 linearly interpolating λn (sn ) = s. Then L(λn ) −−−−→ 1 n→∞

as well as fˆn − f ◦ λn ∞ = max |µλ−1 (I) (f + ξn ) − µI (f )| −−−−→ 0 I∈PJ(fγ )

n→∞

which completes the proof. Proof of Theorem 3, part (ii). The proof can be carried out in the same manner as the proof of Theorem 4, part (ii) in [2]. The only difference is, that it is necessary to attend the slightly different rates of the partial sum process (4).

Scale space consistency of regressograms

81

4.4. Convergence of scale spaces Proof of Lemma 4. It is clear, that each g ∈ argmin Hγ∞ (·, f ) is determined by its jump set. Further, if g1 , g2 ∈ S([0, 1)) with #J(g1 ) = #J(g2 ) and f −g1  = f −g2  then g1 is a minimizer of Hγ∞ (·, f ) if and only if g2 is. Since Hγ∞ (0, f ) = f 2 we have that γ ∈ [ν, ∞) implies J(g) ≤ f 2 /ν, for a minimizer g of Hγ∞ (·, f ). Hence on [ν, ∞) we have that min Hγ∞ (·, f ) = min{kγ + ∆k (f ) : k ≤ f 2 /ν} with ∆k (f ) defined by ∆k (f ) := inf{g − f  : g ∈ S([0, 1)), #J(g) ≤ k} . For each ν the map γ → min Hγ∞ (·, f ) is thus a minimum of a finite collection of linear functions with pairwise different slopes on [ν, ∞). If there are different k, k and γ with kγ +hk = k  γ +hk it follows γ = (hk −hk )/(k −k ). From this it follows that there are only finitely many γ where #{k : kλ + ∆k (f ) = min Hγ∞ (·, f )} > 1. Further, argmin Hγ∞ (·, y) is completely determined by the k which realize this minimum. Call those γ, for which different k realize the minimum, changepoints of γ → min Hγ∞ (·, f ). Since the above holds true for each ν > 0, there are only countably many changepoints in [0, ∞). This completes the proof. Proof of Theorem 4. It is easy to see that the assumptions imply J(τ ) = {γm : m = m(x) 1, . . . , m(x)} for the sequence (γm )m=0 ⊂ R ∪ ∞ of Lemma 4. Since the scale space τ is uniquely determined by its jump points, this proves the uniqueness claim. For the proof of the almost sure convergence, note that Theorem 1 and Theorem 3, part (i) show that τˆn (ζ) →n→∞ τ (ζ) if ζ is a point of continuity of τ , i.e. ∞ (·, f ) = 1. Convergence in all continuity points together with relative # argmin H1/ζ compactness of the sequence implies convergence in the Skorokhod topology. Hence, it is enough to show that {ˆ τn : n ∈ N} is relatively compact. To this end, we will use Lemma 11. In the proof of Theorem 1 it was shown, that the sequence (Tγ (Yn ))n∈N is relatively compact in L2 (0, 1). To prove relative compactness in D([0, 1)) we follow the lines of the proof of Theorem 3, part (i). Similarly we find that lim sup #Jn ≤ n→∞

max

∞ (·,f ) g∈argmin H1/ζ

#J(g) .

For each subsequence of (Tγ (Yn ))n∈N , consider the subsequence of corresponding jump sets. By compactness of CL([0, 1]) we choose a converging sub-subsequence and argue as in the proof mentioned above that the corresponding minimizers con∞ (·, f ). Thus we have verified condition (B1). verge to a limit in argmin H1/ζ For the proof of (B2), we will show by contradiction that for all T > 0 we have inf{mpl(ˆ τn |[0,T ] ) : n ∈ N} > 0. This, obviously, would imply (B2). Observe that τˆn jumps in ζ only if there are two jump sets J = J  such that H1/ζ ((Yn )J , Yn ) = H1/ζ ((Yn )J  , Yn ) and H1/ζ ((Yn )J , Yn ) ≤ H1/ζ ((Yn )J  , Yn ) for all J  . If (B2) is not fulfilled for (τn )n∈N , we can switch by compactness to a subsequence τn ), ζn1 < ζn2 and ζn1 −−−−→ ζ, and find sequences (ζ 1 n )n∈N , (ζ 2 n )n∈N with ζn1 , ζn2 ∈ J(ˆ n→∞

ζn2 −−−−→ ζ for some ζ ≥ 0. Choosing again a subsequence, we could assume that the n→∞

Boysen et al.

82

jump sets Jn1 , Jn2 , Jn3 of minimizers fˆnk ∈ ιn (argmin Hγn (·, Yn )) for some sequences γn1 − 1/ζn1 ↓ 0, γn2 ∈ (1/ζn2 , 1/ζn1 ) and γn3 − 1/ζn2 ↑ 0 are constant and (fˆnk )n∈N , k = 1, 2, 3, converge. Further, we know from this choice of γnk and Lemma 4 that #Jn1 > #Jn2 > #Jn3 . This implies (11)

γn1 + γn2 + ιn (Yn ) − fˆn1 2 < γn2 + ιn (Yn ) − fˆn2 2 < ιn (Yn ) − fˆn3 2 .

The same arguments as in Theorem 1 and Theorem 3, part (i) respectively, yield ∞ {limn→∞ fˆnk : k = 1, 2, 3} ⊆ argmin H1/ζ (·, f ). Since (11) holds for all n, the limits ∞ are pairwise different. This contradicts # argmin H1/ζ (·, x) ≤ 2 and proves (B2). Thus {ˆ τn : n ∈ N} is relatively compact in D([0, ∞), D([0, 1))) as well as in D([0, ∞), L2 [0, 1]) and the proof is complete. Appendix: Epi-Convergence Instead of standard techniques from penalized maximum likelihood regression, we use the concept of epi-convergence (see for example [5, 13]). This allows for simple formulation and more structured proofs. The main arguments to derive consistency of estimates which are (approximate) minimizers for a sequence of functionals can briefly be summarized by epi-convergence + compactness + uniqueness a.s. ⇒ strong consistency. We give here the definition of epi-(or Γ-)convergence together with the results from variational analysis which are relevant for the subsequent proofs. Definition 1. Let Fn : Θ −→ R ∪ ∞, n = 1, . . . , ∞ be numerical functions on a epi metric space (Θ, ρ). (Fn )n∈N epi-converges to F∞ (Symbol Fn −−−−→ F∞ ) if n→∞

(i) for all ϑ ∈ Θ, and sequences (ϑn )n∈N with ϑn −−−−→ ϑ n→∞

F∞ (ϑ) ≤ lim inf Fn (ϑn ) n→∞

(ii) for all ϑ ∈ Θ there exists a sequence (ϑn )n∈N with ϑn −−−−→ ϑ such that n→∞

(12)

F∞ (ϑ) ≥ lim sup Fn (ϑn ) n→∞

The main, useful conclusions from epi-convergence are given by the following theorem. epi

Theorem 5 ([1], Theorem 5.3.6). Suppose Fn −−−−→ F∞ . n→∞

(i) For any converging sequence (ϑn )n∈N , ϑn ∈ argmin Fn , it holds necessarily limn→∞ ϑn ∈ argmin F∞ . (ii) If there is a compact set K ⊂ Θ such that ∅ = argmin Fn ⊂ K for large enough n then argmin F∞ = ∅ and dist(ϑn , argmin F∞ ) −−−−→ 0 n→∞

for any sequence (ϑn )n∈N , ϑn ∈ argmin Fn . (iii) If, additionally, argmin F∞ is a singleton {ϑ} then ϑn −−−−→ ϑ n→∞

for any sequence (ϑn )n∈N , ϑn ∈ argmin Fn .

Scale space consistency of regressograms

83

References [1] Beer, G. (1993). Topologies on Closed and Closed Convex Sets. Kluwer Academic Publishers Group, Dordrecht. [2] Boysen, L., Kempe, A., Liebscher, V., Munk, A. and Wittich, O. (2006). Consistencies and rates of convergence of jump-penalized least squares estimators. Submitted. [3] Chaudhuri, P. and Marron, J. S. (1999). SiZer for exploration of structures in curves. J. Amer. Statist. Assoc. 94 807–823. [4] Chaudhuri, P. and Marron, J. S. (2000). Scale space view of curve estimation. Ann. Statist. 28 408–428. [5] Dal Maso, G. (1993). An Introduction to Γ-convergence. Birkh¨ auser, Boston. [6] Davies, P. L. and Kovac, A. (2001). Local extremes, runs, strings and multiresolution. Ann. Statist. 29 1–65. [7] DeVore, R. A. and Lorentz, G. G. (1993). Constructive Approximation. Springer, Berlin. [8] Dharmadhikari, S. W. and Jogdeo, K. (1969). Bounds on moments of certain random variables. Ann. Math. Statist. 40 1506–1509. [9] Donoho, D. L. (1997). CART and best-ortho-basis: A connection. Ann. Statist. 25 1870–1911. [10] Donoho, D. L. and Johnstone, I. M. (1994). Minimax risk over lp -balls for lq -error. Probab. Theory Related Fields 99 277–303. [11] Donoho, D. L. and Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. J. Amer. Statist. Assoc. 90 1200–1224. [12] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes. Wiley, New York. [13] Hess, C. (1996). Epi-convergence of sequences of normal integrands and strong consistency of the maximum likelihood estimator. Ann. Statist. 24 1298–1315. [14] Hinkley, D. V. (1970). Inference about the change-point in a sequence of random variables. Biometrika 57 1–17. [15] Lindeberg, T. (1994). Scale Space Theory in Computer Vision. Kluwer, Boston. [16] Mammen, E. and van de Geer, S. (1997). Locally adaptive regression splines. Ann. Statist. 25 387–413. [17] Marron, J. S. and Chung, S. S. (2001). Presentation of smoothers: the family approach. Comput. Statist. 16 195–207. [18] Marron, J. S. and Wand, M. P. (1992). Exact mean integrated squared error. Ann. Statist. 20 712–736. [19] Matheron, G. (1975). Random Sets and Integral Geometry. Wiley, New York–London–Sydney. [20] Mumford, D. and Shah, J. (1989). Optimal approximations by piecewise smooth functions and associated variational problems. Comm. Pure Appl. Math. 42 577–685. [21] Petrov, V. V. (1975). Sums of Independent Random Variables. Springer, New York. [22] Shao, Q. M. (1995). On a conjecture of R´ev´esz. Proc. Amer. Math. Soc. 123 575–582. [23] Sheather, S. J. and Jones, M. C. (1991). A reliable data-based bandwidth selection method for kernel density estimation. J. Roy. Statist. Soc. Ser. B 53 683–690. [24] Shen, J. (2005). A stochastic-variational model for soft Mumford–Shah seg-

84

Boysen et al.

mentation. IMA Preprint Series 2062. Univ. Minnesota, Minneapolis. [25] Tukey, J. W. (1961). Curves as parameters, and touch estimation. In Proc. 4th Berkeley Sympos. Math. Statist. Probab. I 681–694. Univ. California Press, Berkeley. [26] Winkler, G. and Liebscher, V. (2002). Smoothers for discontinuous signals. J. Nonparametr. Statist. 14 203–222. [27] Winkler, G., Wittich, O., Liebscher, V. and Kempe, A. (2005). Don’t shed tears over breaks. Jahresber. Deutsch. Math.-Verein. 107 57–87. [28] Yao, Y.-C. (1988). Estimating the number of change-points via Schwarz’ criterion. Statist. Probab. Lett. 6 181–189. [29] Yao, Y.-C. and Au, S. T. (1989). Least-squares estimation of a step function. Sankhy¯ a Ser. A 51 370–381.

IMS Lecture Notes–Monograph Series Asymptotic: Particles, Processes and Inverse Problems Vol. 55 (2007) 85–100 c Institute of Mathematical Statistics, 2007
DOI: 10.1214/074921707000000283

Confidence bands for convex median curves using sign-tests Lutz D¨ umbgen1 University of Bern Abstract: Suppose that one observes pairs (x1 , Y1 ), (x2 , Y2 ), . . . , (xn , Yn ), where x1 ≤ x2 ≤ · · · ≤ xn are fixed numbers, and Y1 , Y2 , . . . , Yn are independent random variables with unknown distributions. The only assumption is that Median(Yi ) = f (xi ) for some unknown convex function f . We present a confidence band for this regression function f using suitable multiscale signtests. While the exact computation of this band requires O(n4 ) steps, good approximations can be obtained in O(n2 ) steps. In addition the confidence band is shown to have desirable asymptotic properties as the sample size n tends to infinity.

1. Introduction Suppose that we are given data vectors x, Y ∈ Rn , where x is a fixed vector with components x1 ≤ x2 ≤ · · · ≤ xn , and Y has independent components Yi with unknown distributions. We assume that (1)

Median(Yi ) = f (xi )

¯ where R ¯ denotes the extended real for some unknown convex function f : R → R, line [−∞, ∞]. To be precise, we assume that f (xi ) is some median of Yi . In what ˆ U ˆ ) for f . That means, L ˆ = L(· ˆ | x, Y, α) follows we present a confidence band (L, ¯ such that ˆ =U ˆ (· | x, Y, α) are data-dependent functions from R into R and U 
ˆ ˆ (x) for all x ∈ R ≥ 1 − α ≤ f (x) ≤ U (2) P L(x)

for a given level α ∈ (0, 1). Our confidence sets are based on a multiscale sign-test. A similar method has been applied by D¨ umbgen and Johns [2] to treat the case of isotonic regression functions, and the reader is referred to that paper for further references. The remainder of the present paper is organized as follows: Section 2 contains the explicit definition of our sign-test statistic and provides some critical values. A corresponding ˆ U ˆ ) is described in Section 3. This includes exact algorithms for confidence band (L, ˆ and the lower bound L ˆ whose running time the computation of the upper bound U 4 3 is of order O(n ) and O(n ), respectively. For large data sets these computational complexities are certainly too high. Therefore we present approximate solutions in Section 4 whose running time is of order O(n2 ). In Section 5 we discuss the asymptotic behavior of the width of our confidence band as the sample size n tends 1 Institute of Math. Statistics and Actuarial Science, University of Bern, Switzerland, e-mail: [email protected] AMS 2000 subject classifications: Primary 62G08, 62G15, 62G20; secondary 62G35. Keywords and phrases: computational complexity, convexity, distribution-free, pool-adjacentviolators algorithm, Rademacher variables, signs of residuals.

85

L. D¨ umbgen

86

to infinity. Finally, in Section 6 we illustrate our methods with simulated and real data. Explicit computer code (in MatLab) for the procedures of the present paper as well as of D¨ umbgen and Johns [2] may be downloaded from the author’s homepage. 2. Definition of the test statistic ¯ for f we consider the sign vectors sign(Y − g(x)) Given any candidate g : R → R and sign(g(x) − Y), where g(x) := (g(xi ))ni=1 and ¯ sign(x) := 1{x > 0} − 1{x ≤ 0} for x ∈ R,  n ¯ n. sign(v) := sign(vi ) i=1 for v = (vi )ni=1 ∈ R

This non-symmetric definition of the sign function is necessary in order to deal with possibly non-continuous distributions. Whenever the vector sign(Y − g(x)) or sign(g(x) − Y) contains “too many” ones in some region, the function g is rejected. Our confidence set for f comprises all convex functions g which are not rejected. ˜ Precisely, let To : {−1, 1}n → R be some test statistic such that To (σ) ≤ To (σ) ˜ component-wise. Then we define whenever σ ≤ σ   T (v) := max To (sign(v)), To (sign(−v))

¯ n . Let ξ ∈ {−1, 1}n be a Rademacher vector, i.e. a random vector with for v ∈ R independent components ξi which are uniformly distributed on {−1, 1}. Further let κ = κ(n, α) be the smallest (1 − α)–quantile of T (ξ). Then P (T (Y − f (x)) ≤ κ) ≥ P(T (ξ) ≤ κ) ≥ 1 − α; see D¨ umbgen and Johns [2]. Consequently the set   C(x, Y, α) := convex g : T (Y − g(x)) ≤ κ

contains f with probability at least 1 − α. As for the test statistic To , let ψ be the triangular kernel function given by ψ(x) := max(1 − |x|, 0).

Then we define To (σ) :=

max

d=1,...,
(n+1)/2



max Td,j (σ) − Γ

j=1,...,n


2d − 1  n

,

where Γ(u) := (2 log(e/u))1/2 , n
i − j  σi ψ Td,j (σ) := βd d i=1

with βd :=

d−1


i=1−d

ψ


i 2 −1/2 d

.

Note that Td,j (σ) is measuring whether (σi )j−d
Confidence bands for convex curves

87

Table 1 Critical values κ(n, α) α 0.50 0.10 0.05

100 0.054 0.792 1.035

200 0.124 0.860 1.102

300 0.152 0.867 1.102

Sample size 500 700 0.188 0.216 0.904 0.902 1.135 1.136

n 1000 0.232 0.915 1.152

2000 0.279 0.970 1.229

5000 0.333 0.991 1.231

10000 0.362 1.021 1.246

Γ((2d − 1)/n) is justified by results of D¨ umbgen and Spokoiny [3] about multiscale testing. In fact, Theorem 6.1 of D¨ umbgen and Spokoiny [3] and Donsker’s invariance principle for partial sums of the Rademacher vector ξ together imply that the distribution of T (ξ) converges weakly to a probability distribution on [0, ∞) as n → ∞. Explicit formulae for quantiles of the limiting distribution of T (ξ) are not available. Therefore we list some quantiles of T (ξ) for various values of n and α in Table 1. Each quantile has been estimated in 19999 Monte Carlo simulations. 3. Definition and exact computation of a band ˜ U ˜ ) via In principle one could define a confidence band (L,   ˜ := inf g ∈ C(x, Y, α) L   = inf convex g : To (sign(Y − g(x)) ≤ κ, To (sign(g(x) − Y) ≤ κ ,   ˜ := sup g ∈ C(x, Y, α) U   = sup convex g : To (sign(Y − g(x)) ≤ κ, To (sign(g(x) − Y) ≤ κ .

Throughout this paper maxima or minima of functions are defined pointwise. Un˜ U ˜ ) is far from trivial. Therefore we fortunately, the explicit computation of (L, ˆ U ˆ ) in two steps. Our upper modify the latter definition and compute a band (L, boundary is given by   ˆ := max convex g : To (sign(g(x) − Y) ≤ κ . U

˜ and Thus we just drop the constraint To (sign(Y − g(x))) ≤ κ in the definition of U ˆ ≥U ˜ . With U ˆ at hand, our lower boundary is defined as obtain U

 ˆ := min convex g : g ≤ U ˆ , To (sign(Y − g(x))) ≤ κ . L

˜ with Here we replace the constraint To (sign(g(x) − Y)) ≤ κ in the definition of L ˆ and obtain L ˆ ≤ L. ˜ In what follows we concentrate the weaker constrint g ≤ U ˆ = ˆ = (L ˆ ˆ i )∞ = L(x) and U on the computation of the corresponding vectors L i=1 n ˆ ˆ (Ui )i=1 = U (x). ˆ 3.1. Computation of U ˆ . To determine U ˆ it suffices to consider the class A simplified expression for U G consisting of the following convex functions gj,k : For 1 ≤ j < k ≤ n with xj < xk define Yk − Yj (x − xj ), gj,k (x) := Yj + xk − xj

L. D¨ umbgen

88

describing a straight line connecting the data points (xj , Yj ) and (xk , Yk ). Moreover, for j, k ∈ {1, . . . , n} let

Then

  ∞ if Yk if g0,k (x) :=  −∞ if   −∞ if Yj if gj,n+1 (x) :=  ∞ if

x < xk , x = xk , x > xk . x < xj , x = xj , x > xj .

  ˆ = max g : g ∈ G, To (sign(g(x) − Y)) ≤ κ . U

(3)

For let g be any convex function such that To (sign(g(x) − Y)) ≤ κ. Let g˜ be the largest convex function such that g˜(xi ) ≤ Yi for all indices i with g(xi ) ≤ Yi . This function g˜ is closely related to the convex hull of all data points (xi , Yi ) with g (x) − Y)) = To (sign(g(x) − Y)). Let g(xi ) ≤ Yi . Obviously, g˜ ≥ g and To (sign(˜ ω(1) < · · · < ω(m) be indices such that xω(1) < · · · < xω(m) and 

     (x, g˜(x)) : x ∈ R ∩ (xi , Yi ) : 1 ≤ i ≤ n = (xω() , Yω() ) : 1 ≤  ≤ m .

With ω(0) := 0 and ω(m + 1) := n + 1 one may write g˜ as the maximum of the functions gω(−1),ω() , 1 ≤  ≤ m + 1, all of which satisfy the inequality To (sign(gω(−1),ω() (x) − Y)) ≤ To (sign(˜ g (x) − Y)) ≤ κ. Figure 1 illustrates these considerations. Computational complexity. As we shall explain in Section 3.3, the computation of To (sign(g(x) − Y)) for one single candidate function g ∈ G requires O(n2 ) ˆ with the vector steps. In case of To (sign(g(x) − Y)) ≤ κ we have to replace U n  ˆi ) in another O(n) steps. Consequently, since G contains at most max(g(xi ), U i=1 ˆ requires O(n4 ) steps. n(n − 1)/2 + 2n = O(n2 ) functions, the computation of U

Fig 1. A function g and its associated function g˜.

Confidence bands for convex curves

89

ˆ 3.2. Computation of L ˆ is nontrivial, i.e. that U ˆi = U ˆ (xi ) < ∞ for some From now on we assume that U value xi . Moreover, letting xmin and xmax be the smallest and largest such value, ˆ (x))) ≤ κ. Otherwise the we assume that xmin < xmax . Finally let To (sign(Y − U confidence set C(x, Y, α) would be empty, meaning that convexity of the median function is not plausible. ˆ Simplified formulae for L. Similarly as in the previous section, one may replace the set of all convex funcˆ ). First of all let h be any convex function such tions with a finite subset H = H(U ˆ that h ≤ U and To (sign(Y − h(x))) ≤ κ. For any real number t let z := h(t). Now ˜≤U ˜ = z. Obviously ˜=h ˜ t,z be the largest convex function such that h ˆ and h(t) let h ˜ ˜ h ≥ h, whence To (sign(Y − h(x))) ≤ κ. Consequently, (4)



˜ t,z (x))) ≤ κ . ˆ L(t) = inf z ∈ R : To (sign(Y − h

˜ t,z . Note that h ˜ t,z is given by the convex hull Figure 2 illustrates the definition of h ˆ , i.e. the set of all pairs (x, y) ∈ R2 such of the point (t, z) and the epigraph of U ˆ (x) ≤ y. that U ˆ For that Starting from equation (4) we derive a computable expression for L. purpose we define tangent parameters as follows: Let J be the set of all indices ˆ (xj ) ≥ Yj . For j ∈ J define j ∈ {1, . . . , n} such that U  

−∞ if xj ≤ xmin , ˆ (xi ) Yj − U  max else, xi <xj xj − xi  if xj ≤ xmin , xj  l ˆ Yj − U (xi ) aj := else,  arg max xj − xi xi <xj slj :=

˜ t,z of two points (t, z). Fig 2. The extremal function h

L. D¨ umbgen

90

 

∞ if xj ≥ xmax , ˆ (xk ) − Yj U  min else, xk >xj xk − xj  if xj ≥ xmax , xj  r ˆ U (xk ) − Yj aj := else.  arg min xk − xj xk >xj srj :=

With these parameters we define auxiliary tangent functions  ˆ (x) U if x < alj , l hj (x) := Yj + slj (x − xj ) if x ≥ alj ,  Yj + srj (x − xj ) if x ≤ arj , hrj (x) := ˆ (x) U if x > arj .

Figure 3 depicts these functions hlj and hrj . Note that  

ˆ , h(xj ) ≤ Yj if  max h(x) : h convex, h ≤ U

 hlj (x) = ˆ , h(xj ) ≥ Yj if  min h(x) : h convex, h ≤ U  

ˆ , h(xj ) ≥ Yj if  min h(x) : h convex, h ≤ U

 hrj (x) = ˆ , h(xj ) ≤ Yj if  max h(x) : h convex, h ≤ U

x ≤ xj , x ≥ xj , x ≤ xj , x ≥ xj ,

In particular, hlj (xj ) = hrj (xj ) = Yj . In addition we define hl0 (x) := hrn+1 (x) := −∞. Then we set hj,k := max(hlj , hrk )

and H := {hj,k : j ∈ {0} ∪ J , k ∈ J ∪ {n + 1}} .

This class H consists of at most (n + 1)2 functions, and elementary considerations show that   ˆ = min h ∈ H : To (sign(Y − h(x))) ≤ κ . (5) L

Fig 3. The tangent functions hlj and hrk .

Confidence bands for convex curves

91

w Computational complexity. Note first that any pair (aw j , sj ) may be computed ˆ in O(n) steps. Consequently, before starting with L we may compute all tangent ˆ may be computed in parameters in time O(n2 ). Then Equation (5) implies that L O(n4 ) steps. However, this can be improved considerably. The reason is, roughly saying, that for fixed j, one can determine the smallest function hrk such that To (sign(Y − hj,k (x))) ≤ κ in O(n2 ) steps, as explained in the subsequent section. ˆ in O(n3 ) steps. Hence a proper implementation lets us compute L

3.3. An auxiliary routine In this section we show that the value of To (σ) can be computed in O(n2 ) steps. More generally, we consider n–dimensional sign vectors σ (0) , σ (1) , . . . , σ (q) such that for 1 ≤  ≤ q the vectors σ (−1) and σ () differ exactly in one component, say, (−1)

σω()

()

σω() = −1

= 1 and

for some index ω() ∈ {1, . . . , n}. Thus σ (0) ≥ σ (1) ≥ · · · ≥ σ (q) component-wise. In particular, To (σ () ) is non-increasing in . It is possible to determine the number


  ∗ := min  ∈ {0, . . . , q} : To (σ () ) ≤ κ ∪ {∞}

in O(n2 ) steps as follows: Algorithm. We use three vector variables S, S (0) and S (1) plus two scalar variables  and d. While running the algorithm the variable S contains the current vector σ () , while n
S (0) = Si , S (1) =




j=1

i∈[j−d+1,j+d−1]



(d − |j − i|)Si

i∈[j−d+1,j+d−1]

n

j=1

.

Initialisation.  ← 0,

d ← 1 and

Induction step. Check whether (6)

max

i=1,...,n

(1) Si

≤

d−1


(d − i)2

i=1−d

 S  S (0) ← σ (0) . (1)  S 1/2 

Γ((2d − 1)/n) + κ



  = ((2d2 + 1)d/3)1/2 Γ((2d − 1)/n) + κ .

• If (6) is fulfilled and d < (n + 1)/2 , then

d ← d + 1,  (0)  Si + Si+d−1 for i < d,  (0) (0) Si ← Si + Si+1−d + Si+d−1 for d ≤ i ≤ n + 1 − d,   (0) Si + Si+1−d for i > n + 1 − d,

S (1) ← S (1) + S (0) .

L. D¨ umbgen

92

• If (6) is fulfilled and d = (n + 1)/2 , then ∗ ← . • If (6) is violated and  < q, then  ←  + 1, Sω() ← −1, (0)

Si

(1) Si

(0)

← Si ←

(1) Si

− 2 and − 2(d − |i − ω()|) for ω() − d < i < ω() + d.

• If Condition (6) is violated but  = q, then To (σ (q) ) > κ, and ∗ ← ∞. As for the running time of this algorithm, note that each induction step requires O(n) operations. Since either d or  increases each time by one, the algorithm terminates after at most n + q + 1 ≤ 2n + 1 induction steps. Together with O(n) operations for the initialisation we end up with total running time O(n2 ). 4. Approximate solutions ˆ . Recall that the exact computation of U ˆ involves testing Approximation of U whether a straight line given by a function g(·) and touching one or two data points (xi , Yi ) satisfies the inequality To (sign(g(x) − Y)) ≤ κ. The idea of our approximation is to restrict our attention to straight lines whose slope belongs to a given finite set. Step 1. At first we consider the straight lines g0,k instroduced in section 3.1, all having slope −∞. Let ω(1), . . . , ω(n) be a list of {1, . . . , n} such that g0,ω(1) ≤ · · · ≤ g0,ω(n) . In other words, for 1 <  ≤ n either xω(−1) < xω() , or xω(−1) = xω() and Yω(−1) ≤ Yω() . With the auxiliary procedure of Section 3.3 we can determine the the smallest number ∗ such that To (sign(g0,ω(∗ ) (x) − Y)) ≤ κ in O(n2 ) steps. We ˆ (x) < ∞}. . Note that xω( ) is equal to xmin = min{x : U write G0 := g 0,ω(∗ )

∗

Step 2. For any given slope s ∈ R let a(s) be the largest real number such that the sign vector  n σ(s) := sign(Yi − a(s) − sxi ) i=1

satisfies the inequality To (σ(s)) ≤ κ. This number can also be determined in time O(n2 ). This time we have to generate and use a list ω(1), . . . , ω(n) of {1, 2, . . . , n} such that Yω() − sxω() is non-increasing in . Now we determine the numbers a(s1 ), . . . , a(sM −1 ) for given slopes s1 < · · · < sM −1 . Then we define G (x) := a(s ) + s x for 1 ≤  < M. Step 3. Finally we determine the largest function GM among the degenerate linear functions g1,n+1 , . . . , gn,n+1 such that To (sign(GM (x) − Y)) ≤ κ. This is ˆ (x) < ∞}. analogous to Step 1 and yields the number xmax = max{x : U Step 4. By means of this list of finitely many straight lines G0 , G1 , . . . , GM one ˆ . In fact, one could even ˆ∗ := max(G0 , G1 , . . . , GM ) for U obtains the lower bound U ˜ ˜  (xi ) ≤ Yi whenever replace G with the largest convex function G such that G

Confidence bands for convex curves

93

G (xi ) ≤ Yi . Each of these functions can be computed via a suitable variant of the pool-adjacent-violators algorithm in O(n) steps; see Robertson et al. [6]. ˆ , for 1 ≤  ≤ M let H be the ˆ ∗ for U Step 5. To obtain an upper bound U smallest concave function such that H (xi ) ≥ Yi whenever max(G−1 (xi ), G (xi )) ≥ Yi . Again H may be determined via the pool-adjacent-violators algorithm. Then elementary considerations show that 
ˆ ≤ U ˆ ∗ := max U ˆ∗ , H1 , H2 , . . . , HM . U

All in all, these five steps require O(M n2 ) steps. By visual inpection of these two ˆ ∗ one may opt for a refined grid of slopes or use U ˆ ∗ as a surrogate ˆ∗ and U curves U ˆ. for U ˆ Recall that the exact computation amounts to fixing any Approximation of L. function hlj und finding the smallest function hrk such that To (sign(Y−hj,k (x))) ≤ κ. Now approximations may be obtained by picking only a subset of the potential indices j. In addition, one may fix some functions hrk and look for the smallest hlj satisfying the constraint To (sign(Y − hj,k (x))) ≤ κ. Again this leads to approximaˆ ∗ for L ˆ such that L ˆ∗ ≤ L ˆ≤L ˆ∗. ˆ ∗ and L tions L 5. Asymptotic properties In this section we consider a triangular array of observations xi = xn,i and Yi = Yn,i . ˆ U ˆ ) will be shown to have certain consistency properties, Our confidence band (L, provided that f satisfies some smoothness condition, and that the following two requirements are met for some constants −∞ < a < b < ∞: (A1) Let Mn denote the empirical distribution of the design points xn,i . That means, Mn (B) := n−1 #{i : xn,i ∈ B} for B ⊂ R. There is a constant c > 0 such that Mn [an , bn ] ≥ c lim inf n→∞ b n − an whenever a ≤ an < bn ≤ b and lim inf n→∞ log(bn − an )/ log n > −1. (A2) All variables Yi = Yn,i with xn,i ∈ [a, b] satisfy the following inequalities:  1 + H(r) P(Yi < µi + r) for any r > 0, ≥ P(Yi > µi − r) 2 where H is some fixed function on [0, ∞] such that lim

r→0+

H(r) > 0. r

These conditions (A1) and (A2) are satisfied in various standard models, as pointed out by D¨ umbgen and Johns [2]. Theorem 1. Suppose that assumptions (A1) and (A2) hold. (a) Let f be linear on [a, b]. Then for arbitrary a < a < b < b,  +  ˆ  sup f (x) − L(x)  x∈[a,b]  + = Op (n−1/2 ). ˆ (x) − f (x)  sup U  x∈[a ,b ]

L. D¨ umbgen

94

(b) Let f be H¨ older continuous on [a, b] with exponent β ∈ (1, 2]. That means, f is differentiable on [a, b] such that for some constant L > 0 and arbitrary x, y ∈ [a, b], |f  (x) − f  (y)| ≤ L|x − y|β−1 . 1/(2β+1)

Then for ρn := log(n + 1)/n and δn := ρn ,   + ˆ  sup f (x) − L(x)    x∈[a,b]  + = Op ρβ/(2β+1) . n ˆ  U (x) − f (x)  sup x∈[a+δn ,b−δn ]

Part (a) of this theorem explains the empirical findings in Section 6 that the ˆ U ˆ ) performs particularly well in regions where the regression function f band (L, is linear. ˆ . Note that Proof of Theorem 1, step I. At first we prove the assertions about U ˆ for arbitrary t, z ∈ R with z ≤ U (t) there exist parameters µ, ν ∈ R such that z = µ + νt and n
i − j  ψ sign(µ + νxi − Yi ) d i=1

2d − 1   +κ for any (d, j) ∈ Tn ; ≤ βd−1 Γ n

Sd,j (µ, ν) := (7)

here Tn denotes the set of all pairs (d, j) of integers d > 0, j ∈   [d, n+1−d]. Therefore it is crucial to have good simultaneous upper bounds for Sd,j (µ, ν) − Σd,j (µ, ν), where n
2d − 1   Σd,j (µ, ν) := ESd,j (µ, ν) = 2P(Yi < µ + νxi ) − 1 . ψ n i=1  One may write Sd,j (µ, ν) = gd,j,µ,ν dΨn with the random measure Ψn :=

n

δi ⊗ δxi ⊗ δYi

i=1

and the function (i, x, y) → gd,j,µ,ν (i, x, y) := ψ


i − j  d

sign(µ + νx − y) ∈ [−1, 1]

on R3 . The family of all these functions gd,j,µ,ν is easily shown to be a Vapnik-Cervonenkis subgraph class in the sense of van der Vaart and Wellner [7]. Moreover, Ψn is a sum of n stochastically independent random probability measures. Thus wellknown results from empirical process theory (cf. Pollard [5]) imply that for arbitrary η ≥ 0,     Sd,j (µ, ν) − Σd,j (µ, ν) ≥ n1/2 η P sup (d,j)∈Tn , µ,ν∈R

(8)

≤ C exp(−η 2 /C),     P sup Sd,j (µ, ν) − Σd,j (Y, µ, ν) ≥ d1/2 η for some (d, j) ∈ Tn µ,ν∈R

(9)

≤ C exp(2 log n − η 2 /C),

Confidence bands for convex curves

95

where C ≥ 1 is a universal constant. Consequently, for any fixed α > 0 there is a constant C˜ > 0 such that the following inequalities are satisfied simultaneously for arbitrary (d, j) ∈ Tn and (µ, ν) ∈ R2 with probability at least 1 − α : (10)

  Sd,j (µ, ν) − Σd,j (µ, ν) ≤



˜ 1/2 , Cn 1/2 ˜ Cd log(n + 1)1/2 .

˜ In what follows we assume (10) for some fixed C. ˆ Proof of part (a) for U . Suppose that f is linear on [a, b], and let [a , b ] ⊂ (a, b). ˆ , the maximum of U ˆ − f over [a , b ] is attained at a or b . We By convexity of U ˆ (a ) ≥ f (a ) + n for some n > 0. Then consider the first case and assume that U there exist µ, ν ∈ R satisfying (7) such that µ + νa = f (a ) + n and ν ≤ f  (a ). In particular, µ + νx − f (x) ≥ n for all x ∈ [a, a ]. Now we pick a pair (dn , jn ) ∈ Tn with dn as large as possible such that 

xjn −dn +1 , xjn +dn −1



⊂ [a, a ].

Assumption (A1) implies that dn ≥ (c/2 + o(1))n. Now jn +d n −1

Σdn ,jn (µ, ν) ≥

i=jn −dn +1

ψ


i − j  n H( n ) = dn H( n ) dn

by assumption (A2). Combining this inequality with (7) and (10) yields (11)



2d − 1   n ˜ 1/2 . βd−1 Γ + κ ≥ dn H( n ) − Cn n n

But βd−1 = 3−1/2 (2d − 1)1/2 + O(d−1/2 ), and x → x1/2 Γ(x) is non-decreasing on (0, 1]. Hence (11) implies that


2d − 1    n −1/2 1/2 ˜ 1/2 H( n ) ≤ d−1 + o(1))(2d − 1) (3 Γ + κ + Cn n n n  −1/2  1/2 ˜ (3 ≤ d−1 n + o(1))(Γ(1) + κ) + C n = O(n−1/2 ).

Consequently, n = O(n−1/2 ). ˆ . Now suppose that f  is H¨older-continuous on [a, b] with Proof of part (b) for U ˆ (x) ≥ f (x) + n for some exponent β − 1 ∈ (0, 1] and constant L > 0. Let U x ∈ [a + δn , b − δn ] and n > 0. Then there are numbers µ, ν ∈ R satisfying (7) such that µ + νx = f (x) + n . Let (dn , jn ) ∈ Tn with dn as large as possible such that either   xjn −dn +1 , xjn +dn −1 ⊂ [x, x + δn ], f  (x) ≤ ν and or

f  (x) ≥ ν

and

Assumption (A1) implies that

  xjn −dn +1 , xjn +dn −1 ⊂ [x − δn , x].

dn ≥ (c/2 + o(1))δn n.

L. D¨ umbgen

96

Moreover, for any i ∈ {jn − dn + 1, jn + dn − 1},  xi µ + νxi − f (xi ) = n + (ν − f  (t)) dt xxi (f  (x) − f  (t)) dt ≥ n + x

≥ n − L



δn

sβ−1 ds

0

= n − O(δnβ ), so that

  Σdn ,jn (µ, ν) ≥ dn H ( n − O(δnβ ))+ .

Combining this inequality with (7) and (10) yields



2d − 1     n −1 ˜ −1/2 log(n + 1)1/2 H ( n − O(δnβ ))+ ≤ d−1 + κ + Cd n βdn Γ n n −1 1/2 ˜ −1/2 log(n + 1)1/2 ≤ d−1 log(n)1/2 + κ) + Cd n βdn (2 n = O(δnβ ).

(12)

 β/(2β+1)  This entails that n has to be of order O(δnβ ) = O ρn .

ˆ For that purpose we Proof of Theorem 1, step II. Now we turn our attention to L. change the definition of Sd,j (·, ·) and Σd,j (·, ·) as follows: Let Un be a fixed convex function to be specified later. Then for (t, z) ∈ R2 we define hn,t,z to be the largest convex function h such that h ≤ Un and h(t) ≤ z. This definition is similar to the ˜ t,z in Section 3.2. Indeed, if U ˆ ≤ Un and L(t) ˆ ≤ z, then definition of h n
i − j  sign(Yi − hn,t,z (xi )) ψ d i=1

2d − 1   ≤ βd−1 Γ +κ for any (d, j) ∈ Tn . n

Sd,j (t, z) := (13) Here we set

n
2d − 1   2P(Yi > hn,t,z (xi )) − 1 . ψ Σd,j (t, z) := ESd,j (t, z) = n i=1

˜ Again we may and do assume that (10) is true for some constant C. ˆ Proof of part (a) for L. Suppose that f is linear on [a, b]. We define Un (x) := f (x) + γn−1/2 + 1{x ∈ [a , b ]}∞ with constants γ > 0 and a < a < b < b. Since ˆ ≤ Un ) tends to one as γ → ∞, we may assume that U ˆ ≤ Un . lim inf n→∞ P(U −1/2 ˆ . A simple Suppose that L(t) ≤ z := f (t) − 2 n for some t ∈ [a, b] and n ≥ γn geometrical consideration shows that hn,t,z ≤ f − n on an interval [a , b ] ⊂ [a, b] of length b −a ≥ (b −a )/3. If we pick (dn , jn ) ∈ Tn with dn as large as possible such that [xjn −dn +1 , xjn +dn −1 ] ⊂ [a , b ], then dn ≥ (c(b − a )/6 + o(1))n. Moreover, (13) and (10) entail (11), whence n = O(n−1/2 ). ˆ Now suppose that f  is H¨older-continuous on [a, b] with Proof of part (b) for L. exponent β − 1 ∈ (0, 1] and constant L > 0. Here we define Un (x) := f (x) + γδnβ + ˆ ≤ Un . 1{x ∈ [a + δn , b − δn ]}∞ with a constant γ > 0, and we assume that U

Confidence bands for convex curves

97

ˆ Suppose that L(t) ≤ z := f (t) − 2 n for some t ∈ [a, b] and n > 0. If t ≤ b − 2δn , then hn,t,z (t + λδn ) ≤ z + λ(Un (t + δn ) − z) = f (t) − 2 n + λ(f (t + δn ) − f (t) + 2 n + γδnβ )  δn = f (t) − 2(1 − λ) n + λ f  (t + s) ds + λγδnβ 0

for 0 ≤ λ ≤ 1. Thus

f (t + λδn ) − hn,t,z (t + λδn )  δn = 2(1 − λ) n + λ (f  (t + λs) − f  (t + s)) ds − λγδnβ ≥ 2(1 − λ) n − λ ≥ n − O(δnβ )



0

δn

L(1 − λ)β−1 sβ−1 ds − λγδnβ

0

uniformly for 0 ≤ λ ≤ 1/2. Analogous arguments apply in the case t ≥ a + 2δn . Consequently there is an interval [an , bn ] ⊂ [a, b] of length δn /2 such that f −hn,t,z ≥

n − O(δnβ ), provided that a + 2δn ≤ b − 2δn . Again we choose (dn , jn ) ∈ Tn with dn as large as possible such that [xjn −dn +1 , xjn +dn −1 ] ⊂ [an , bn ]. Then dn ≥ (c/4 + β/(2β+1) ). o(1))δn n, and (13) and (10) lead to (12). Thus n = O(δnβ ) = O(ρn 6. Numerical examples ˆ U ˆ ) defined in Section 3 with some At first we illustrate the confidence band (L, simulated data. Precisely, we generated Yi = f (xi ) + σ i with xi := (i − 1/2)/n, n = 500 and  −12(x − 1/3) if x ≤ 1/3, f (x) := (27/2)(x − 1/3)2 if x ≥ 1/3. Moreover, σ = 1/2, and the random errors 1 , . . . , n have been simulated from a student distribution with five degrees of freedom. Figure 4 depicts these data ˆ U ˆ ) and f itself. Note together with the corresponding 95%–confidence band (L, that the width of the band is smallest near the center of the interval (0, 1/3) on which f is linear. This is in accordance with part (a) of Theorem 1. Secondly we applied our procedure to a dataset containing the income xi and the expenditure Yi for food in the year 1973 for n = 7125 households in Great Britain (Family Expenditure Survey 1968–1983). This dataset has also been analyzed by H¨ ardle and Marron [4]. They computed simultaneous confidence intervals for E(Yi ) = f˜(xi ) by means of kernel estimators and bootstrap methods. Figure 5 depicts the data. In order to enhance the main portion, the axes have been chosen such that 72 outlying observations are excluded from the display. Figure 6 shows a 95%–confidence band for the isotonic median function f , as described by D¨ umbgen and Johns [2]. Figure 7 shows a 95%–confidence band for the concave median function f , as described in the present paper. Note that the latter band has substantially smaller width than the former one. This is in accordance with our theoretical results about rates of convergence.

98

L. D¨ umbgen

ˆ U ˆ ), where n = 500. Fig 4. Simulated data and 95%–confidence band (L,

Fig 5. Income-expenditure data.

Confidence bands for convex curves

Fig 6. 95%–confidence band for isotonic median function.

Fig 7. 95%–confidence band for concave median function.

99

100

L. D¨ umbgen

Acknowledgments. The author is indebted to Geurt Jongbloed for constructive comments. Many thanks also to Wolfgang H¨ ardle (Humboldt Unversity Berlin) for providing the family expenditure data. References [1] Davies, P. L. (1995). Data features. Statist. Neerlandica 49 185–245. ¨mbgen, L. and Johns, R. B. (2004). Confidence bands for isotonic median [2] Du curves using sign tests. J. Comput. Graph. Statist. 13 519–533. ¨mbgen, L. and Spokoiny, V. G. (2001). Multiscale testing of qualitative [3] Du hypotheses. Ann. Statist. 29 124–152. ¨rdle, W. and Marron, J. S. (1991). Bootstrap simultaneous error bars [4] Ha for nonparametric regression. Ann. Statist. 19 778–796. [5] Pollard, D. (1990). Empirical Processes: Theory and Applications. IMS, Hayward, CA. [6] Robertson, T., Wright, F. T. and Dykstra, R. L. (1988). Order Restricted Statistical Inference. Wiley, New York. [7] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes with Applications to Statistics. Springer, New York.

IMS Lecture Notes–Monograph Series Asymptotics: Particles, Processes and Inverse Problems Vol. 55 (2007) 101–107 c Institute of Mathematical Statistics, 2007
DOI: 10.1214/074921707000000292

Marshall’s lemma for convex density estimation Lutz D¨ umbgen1 , Kaspar Rufibach1 and Jon A. Wellner2 University of Bern and University of Washington Abstract: Marshall’s [Nonparametric Techniques in Statistical Inference √ (1970) 174–176] lemma is an analytical result which implies n–consistency of the distribution function corresponding to the Grenander [Skand. Aktuarietidskr. 39 (1956) 125–153] estimator of a non-decreasing probability density. The present paper derives analogous results for the setting of convex densities on [0, ∞).

1. Introduction Let F be the empirical distribution function of independent random variables X1 , X2 , . . . , Xn with distribution function F and density f on the halfline [0, ∞). Various shape restrictions on f enable consistent nonparametric estimation of it without any tuning parameters (e.g. bandwidths for kernel estimators). The oldest and most famous example is the Grenander estimator fˆ of f under the assumption that f is non-increasing. Denoting the family of all such densities by F, the Grenander estimator may be viewed as the maximum likelihood estimator,
  fˆ = argmax log h dF : h ∈ F , or as a least squares estimator,
 ˆ f = argmin

∞

h(x) dx − 2 2

0



 h dF : h ∈ F ;

 cf. Robertson et al. [5]. Note that if F had a square-integrable  ∞ density F , 2then the preceding argmin would be identical with the minimizer of 0 (h − F )(x) dx over all non-increasing probability densities h on [0, ∞). A nice property of fˆ is that the corresponding distribution function Fˆ ,  r Fˆ (r) := fˆ(x) dx, 0

√ is automatically n–consistent. More precisely, since Fˆ is the least concave majorant of F, it follows from Marshall’s [4] lemma that Fˆ − F ∞ ≤ F − F ∞ .

A more refined asymptotic analysis of Fˆ − F has been provided by Kiefer and Wolfowitz [3]. 1 Institute of Math. Statistics and Actuarial Science, University of Bern, Switzerland, e-mail: [email protected]; [email protected] 2 Dept. of Statistics, University of Washington, Seattle, USA, e-mail: [email protected] AMS 2000 subject classifications: 62G05, 62G07, 62G20. Keywords and phrases: empirical distribution function, inequality, least squares, maximum likelihood, shape constraint, supremum norm.

101

L. D¨ umbgen, K. Rufibach and J. A. Wellner

102

2. Convex densities Now we switch to the estimation of a convex probability density f on [0, ∞). As pointed out by Groeneboom et al. [2], the nonparametric maximum likelihood estimator fˆml and the least squares estimator fˆls are both well-defined and unique, but they are not identical in general. Let K denote the convex cone of all convex and integrable functions g on [0, ∞). (All functions within K are necessarily nonnegative and non-increasing.) Then  ∞   h(x) dx , fˆml = argmax log h dF − h∈K 0   ∞  fˆls = argmin h(x)2 dx − 2 h dF . h∈K

0

Both estimators have the following property: Proposition 1. Let fˆ be either fˆml or fˆls . Then fˆ is piecewise linear with • at most one knot in each of the intervals (X(i) , X(i+1) ), 1 ≤ i < n, • no knot at any observation Xi , and • precisely one knot within (X(n) , ∞). The estimators fˆml , fˆls and their distribution functions Fˆml , Fˆls are completely characterized by Proposition 1 and the next proposition. Proposition 2. Let ∆ be any function on [0, ∞) such that fˆml + t∆ ∈ K for some t > 0. Then   ∆ dF ≤ ∆(x) dx. fˆml Similarly, let ∆ be any function on [0, ∞) such that fˆls + t∆ ∈ K for some t > 0. Then   ∆ dF ≤ ∆ dFˆls .

In what follows we derive two inequalities relating Fˆ − F and F − F , where Fˆ stands for Fˆml or Fˆls : Theorem 1. (1) (2)

3 1 inf (F − F ) − sup (F − F ), 2 [0,∞) 2 [0,∞)     Fˆls − F  ≤ 2 F − F  . ∞ ∞

inf (Fˆml − F ) ≥

[0,∞)

Both results rely on the following lemma:

Lemma 1. Let F, Fˆ be continuous functions on a compact interval [a, b], and let F be a bounded, measurable function on [a, b]. Suppose that the following additional assumptions are satisfied: (3) (4) (5) (6)

Fˆ (a) = F(a) and Fˆ (b) = F(b), Fˆ has a linear derivative on (a, b), F has a convex derivative on (a, b),  b  b ˆ F (y) dy ≤ F(y) dy for all r ∈ [a, b]. r

r

Marshall’s lemma for convex densities

Then

103

3 1 sup (F − F ) − (F − F )(b). 2 [a,b] 2

sup (Fˆ − F ) ≤ [a,b]

If condition (6) is replaced with 

(7)



r

Fˆ (x) dx ≥ a

r

F(x) dx

for all r ∈ [a, b],

a

then

3 1 inf (F − F ) − (F − F )(a). 2 [a,b] 2

inf (Fˆ − F ) ≥

[a,b]

The constants 3/2 and 1/2 are sharp. For let [a, b] = [0, 1] and define  x2 − c for x ≥
, F (x) := (x/
)(
2 − c) for x ≤
, Fˆ (x) := 0, F(x) := 1{0 < x < 1}(x2 − 1/3) for some constant c ≥ 1 and some small number
∈ (0, 1/2]. One easily verifies conditions (3)–(6). Moreover, sup (Fˆ − F ) = c −
2 ,

sup (F − F ) = c − 1/3 and

[0,1]

[0,1]

(F − F )(1) = c − 1.

Hence the upper bound (3/2) sup(F − F ) − (1/2)(F − F )(1) equals sup(Fˆ − F ) +
2 for any c ≥ 1. Note the discontinuity of F at 0 and 1. However, by suitable approximation of F with continuous functions one can easily show that the constants remain optimal even under the additional constraint of F being continuous. Proof of Lemma 1. We define G := Fˆ − F with derivative g := G on (a, b). It follows from (3) that max G = max (F − F ) ≤ {a,b}

{a,b}

3 1 sup (F − F ) − (F − F )(b). 2 [a,b] 2

Therefore it suffices to consider the case that G attains its maximum at some point r ∈ (a, b). In particular, g(r) = 0. We introduce an auxiliary linear function g¯ on [r, b] such that g¯(r) = 0 and 



b

g¯(y) dy = r

b

g(y) dy = G(b) − G(r).

r

Note that g is concave on (a, b) by (4)–(5). Hence there exists a number yo ∈ (r, b) such that  ≥ 0 on [r, yo ], g − g¯ ≤ 0 on [yo , b). This entails that  y  b (g − g¯)(u) du = − (g − g¯)(u) du ≥ 0 for any y ∈ [r, b]. r

y

L. D¨ umbgen, K. Rufibach and J. A. Wellner

104

Consequently, G(y) = G(r) + ≥ G(r) + = G(r) +





y

g(u) du r y

g¯(u) du r

(y − r)2 [G(b) − G(r)], (b − r)2

so that 

 G(b) − G(r) b (y − r)2 dy (b − r)2 r 2

1 = (b − r) G(r) + G(b) 3 3 2

1 = (b − r) G(r) + (F − F )(b) . 3 3

b

G(y) dy ≥ (b − r)G(r) + r

On the other hand, by assumption (6),  b  b G(y) dy ≤ (F − F )(y) dy ≤ (b − r) sup (F − F ). r

[a,b]

r

This entails that G(r) ≤

3 1 sup (F − F ) − (F − F )(b). 2 [a,b] 2

If (6) is replaced with (7), then note first that 3 1 min (F − F ) − (F − F )(a). 2 {a,b} 2

min G = min (F − F ) ≥

{a,b}

{a,b}

Therefore it suffices to consider the case that G attains its minimum at some point r ∈ (a, b). Now we consider a linear function g¯ on [a, r] such that g¯(r) = 0 and  r  r g¯(x) dx = g(x) dx = G(r) − G(a). a

a

Here concavity of g on (a, b) entails that  r  x (g − g¯)(u) du = − (g − g¯)(u) du ≤ 0 for any x ∈ [a, r], a

x

so that G(x) = G(r) − ≤ G(r) −



r

xr

g(u) du g¯(u) du

x

= G(r) − Consequently, 

r a

(r − x)2 [G(r) − G(a)]. (r − a)2

 G(r) − G(a) r G(x) dx ≤ (r − a)G(r) − (r − x)2 dx (r − a)2 a 2

1 = (r − a) G(r) + (F − F )(a) , 3 3

Marshall’s lemma for convex densities

whereas



r

G(x) dx ≥ a



105

r

(F − F )(x) dx ≥ (r − a) inf (F − F ), [a,b]

a

by assumption (7). This leads to G(r) ≥

3 1 inf (F − F ) − (F − F )(a). 2 [a,b] 2




Proof of Theorem 1. Let 0 =: t0 < t1 < · · · < tm be the knots of fˆ, including the origin. In what follows we derive conditions (3)–(5) and (6/7) of Lemma 1 for any interval [a, b] = [tk , tk+1 ] with 0 ≤ k < m. For the reader’s convenience we rely entirely on Proposition 2. In case of the least squares estimator, similar inequalities and arguments may be found in Groeneboom et al. [2]. Let 0 <
< min1≤i≤m (ti − ti−1 )/2. For a fixed k ∈ {1, . . . , m} we define ∆1 to be continuous and piecewise linear with knots tk−1 −
(if k > 1), tk−1 , tk and / (tk−1 −
, tk +
) and tk +
. Namely, let ∆1 (x) = 0 for x ∈   fˆml (x) if fˆ = fˆml ∆1 (x) := for x ∈ [tk−1 , tk ]. 1 if fˆ = fˆls This function ∆1 satisfies the requirements of Proposition 2. Letting
 0, the function ∆1 (x) converges pointwise to  1{tk−1 ≤ x ≤ tk }fˆml (x) if fˆ = fˆml , 1{tk−1 ≤ x ≤ tk } if fˆ = fˆls , and the latter proposition yields the inequality F(tk ) − F(tk−1 ) ≤ Fˆ (tk ) − Fˆ (tk−1 ). Similarly let ∆2 be continuous and piecewise linear with knots at tk−1 , tk−1 +
, / (tk−1 , tk ) and tk −
and tk . Precisely, let ∆2 (x) := 0 for x ∈   −fˆml (x) if fˆ = fˆml ∆2 (x) := for x ∈ [tk−1 +
, tk −
]. −1 if fˆ = fˆls The limit of ∆2 (x) as
 0 equals  −1{tk−1 < x < tk }fˆml (x) if −1{tk−1 < x < tk } if

fˆ = fˆml , fˆ = fˆls ,

and it follows from Proposition 2 that F(tk ) − F(tk−1 ) ≥ Fˆ (tk ) − Fˆ (tk−1 ). This shows that F(tk )−F(tk−1 ) = Fˆ (tk )− Fˆ (tk−1 ) for k = 1, . . . , m. Since Fˆ (0) = 0, one can rewrite this as (8)

F(tk ) = Fˆ (tk )

for k = 0, 1, . . . , m.

Now we consider first the maximum likelihood estimator fˆml . For 0 ≤ k < m / (tk −
, r), let ∆ be linear on [tk −
, tk ], and r ∈ (tk , tk+1 ] let ∆(x) := 0 for x ∈

L. D¨ umbgen, K. Rufibach and J. A. Wellner

106

and let ∆(x) := (r − x)fˆml (x) for x ∈ [tk , r]. One easily verifies, that this function ∆ satisfies the conditions of Proposition 2, too, and with
 0 we obtain the inequality  r  r (r − x) F(dx) ≤ (r − x) Fˆ (dx). tk

tk

Integration by parts (or Fubini’s theorem) shows that the latter inequality is equivalent to  r  r (F(x) − F(tk )) dx ≤ (Fˆ (x) − Fˆ (tk )) dx. tk

tk

Since F(tk ) = Fˆ (tk ), we end up with 



r

F(x) dx ≤

tk

r

Fˆ (x) dx

for k = 0, 1, . . . , m − 1 and r ∈ (tk , tk+1 ].

tk

Hence we may apply Lemma 1 and obtain (1). Finally, let us consider the least squares estimator fˆls . For 0 ≤ k < m and / (tk −
, r), let ∆ be linear on [tk −
, tk ] as well r ∈ (tk , tk+1 ] let ∆(x) := 0 for x ∈ as on [tk , r] with ∆(tk ) := r − tk . Then applying Proposition 2 and letting
 0 yields  r  r (r − x) F(dx) ≤ (r − x) Fˆ (dx), tk

so that 



r

F(x) dx ≤

tk

tk

r

Fˆ (x) dx

for k = 0, 1, . . . , m − 1 and r ∈ (tk , tk+1 ].

tk

Thus it follows from Lemma 1 that   3 1 inf (F − F ) − sup (F − F ) ≥ −2 F − F ∞ . 2 [0,∞) 2 [0,∞)

inf (Fˆ − F ) ≥

[0,∞)

Alternatively, for 1 ≤ k ≤ m and r ∈ [tk−1 , tk ) let ∆(x) := 0 for x ∈ / (r, tk +
), let ∆ be linear on [r, tk ] as well as on [tk , tk +
] with ∆(tk ) := −(tk − r). Then applying Proposition 2 and letting
 0 yields 

tk

(tk − x) F(dx) ≥

r



tk

(tk − x) Fˆ (dx),

r

so that 

tk

F(x) dx ≥

r



r

Fˆ (x) dx

for k = 1, 2, . . . , m and r ∈ [tk−1 , tk ).

tk

Hence it follows from Lemma 1 that sup (Fˆ − F ) ≤ [0,∞)

  3 1 sup (F − F ) − inf (F − F ) ≤ 2 F − F ∞ . 2 [0,∞) 2 [0,∞)

Acknowledgment. The authors are grateful to Geurt Jongbloed for constructive comments and careful reading.

Marshall’s lemma for convex densities

107

References [1] Grenander, U. (1956). On the theory of mortality measurement, part II. Skand. Aktuarietidskr. 39 125–153. [2] Groeneboom, P., Jongbloed, G. and Wellner, J. A. (2001). Estimation of a convex function: Characterization and asymptotic theory. Ann. Statist. 29 1653–1698. [3] Kiefer, J. and Wolfowitz, J. (1976). Asymptotically minimax estimation of concave and convex distribution functions. Z. Wahrsch. Verw. Gebiete 34 73–85. [4] Marshall, A. W. (1970). Discussion of Barlow and van Zwet’s paper. In Nonparametric Techniques in Statistical Inference. Proceedings of the First International Symposium on Nonparametric Techniques held at Indiana University, June, 1969 (M. L. Puri, ed.) 174–176. Cambridge University Press, London. [5] Robertson, T., Wright, F. T. and Dykstra, R. L. (1988). Order Restricted Statistical Inference. Wiley, New York.

IMS Lecture Notes–Monograph Series Asymptotics: Particles, Processes and Inverse Problems Vol. 55 (2007) 108–120 c Institute of Mathematical Statistics, 2007
DOI: 10.1214/074921707000000300

Escape of mass in zero-range processes with random rates Pablo A. Ferrari

1,∗

and Valentin V. Sisko

2,†

Universidade de S˜ ao Paulo Abstract: We consider zero-range processes in Zd with site dependent jump rates. The rate for a particle jump from site x to y in Zd is given by λx g(k)p(y− x), where p(·) is a probability in Zd , g(k) is a bounded nondecreasing function of the number k of particles in x and λ = {λx } is a collection of i.i.d. random variables with values in (c, 1], for some c > 0. For almost every realization of the environment λ the zero-range process has product invariant measures {νλ,v : 0 ≤ v ≤ c} parametrized by v, the average total jump rate from any given site. The density of a measure, defined by the asymptotic average number of particles per site, is an increasing function of v. There exists a product invariant measure νλ,c , with maximal density. Let µ be a probability measure concentrating mass on configurations whose number of particles at site x grows less than exponentially with x. Denoting by Sλ (t) the semigroup of the process, we prove that all weak limits of {µSλ (t), t ≥ 0} as t → ∞ are dominated, in the natural partial order, by νλ,c . In particular, if µ dominates νλ,c , then µSλ (t) converges to νλ,c . The result is particularly striking when the maximal density is finite and the initial measure has a density above the maximal.

1. Introduction In the zero-range process there are a finite number of particles at each site of Zd . At a rate depending monotonically on the number of particles at the site, one of the particles jumps to another site chosen independently with a transition probability function. The rate at which particles leave any site is bounded. When the rate at each site x is multiplied by a random variable λx chosen at time zero independently of the process, the system may show a phase transition in the density. For almost every realization of the environment λ the zero-range process has product invariant measures {νλ,v : 0 ≤ v ≤ c} parametrized by v, the average total jump rate from any given site. The density of a measure is the asymptotic number of particles per site (when this exists). For each v ≤ c the invariant measure νλ,v has density ρ(v), which is an increasing function of v. Our main result is to start the system with a measure concentrating mass in configurations not growing too fast (see (3) below) and show that the distribution of the process as time goes to infinity is dominated by the maximal measure νλ,c . This is particularly interesting when ρ(c) < ∞ and the initial density of µ is strictly bigger than ρ(c). In this case we say that there is ∗ Supported

in part by FAPESP. by FAPESP (2003/00847–1) and CNPq (152510/2006–0). 1 Departamento de Estat´ istica, Instituto de Matem´ atica e Estat´istica, Universidade de S˜ ao Paulo, Caixa Postal 66281, CEP 05311–970 S˜ ao Paulo, SP, Brazil, e-mail: [email protected], url: www.ime.usp.br/~pablo 2 IMPA, Estrada Dona Castorina 110, CEP 22460-320 Rio de Janeiro, Brasil, e-mail: [email protected], url: www.ime.usp.br/~valentin AMS 2000 subject classifications: 60K35, 82C22. Keywords and phrases: random environment, zero-range process. † Supported

108

Zero range processes with random rates

109

an “escape of mass”. When the initial distribution dominates the maximal invariant measure, the process converges to the maximal invariant measure. The zero-range process appeared first as a network of queues when Jackson [13] showed that the product measures are invariant for the process in a finite number of sites. Spitzer [22] introduced the process in a countable number of sites as a model of infinite particle system with interactions. The existence of the process has been proved by Holley [12] and Liggett [17, 19]. We use Harris [11] direct probabilistic construction which permits the particles to be distinguishable, so one can follow the behavior of any particular particle. Using Liggett’s [18] approach, Andjel [1] gave a description of the set of invariant measures for the zero-range process in some cases. Bal´azs, Rassoul-Agha, Sepp¨al¨ ainen, and Sethuraman [4] studied the case of rates bounded by an exponential function of k in a one dimensional asymmetric model. The study of conservative interacting particle systems in random environment was proposed simultaneously by Benjamini, Ferrari and Landim [5] and Evans [7], who observed the existence of phase transition in these models; see also Krug and Ferrari [15]. Benjamini, Ferrari and Landim [5], Krug and Sepp¨ al¨ ainen [20] and Koukkous [14] investigated the hydrodynamic behavior of conservative processes in random environments; Landim [16] and Bahadoran [3] considered the same problem for non-homogeneous asymmetric attractive processes; Gielis, Koukkous and Landim [9] deduced the equilibrium fluctuations of a symmetric zero-range process in a random environment; Andjel, Ferrari, Guiol and Landim [2] proved the convergence to the maximal invariant measure for a one-dimensional totally asymmetric nearest-neighbor zero-range process with random rates. This phenomenon is studied by Sepp¨ al¨ ainen, Grigorescu and Kang [10] in one dimension. Evans and Hanney [8] have recently published a review paper on the zero-range process which includes many references to the mathematical physics literature. Section 2 includes definitions, results and at the end a summary of the contents of the other sections. 2. Results We study the zero-range process with site dependent jump rates. Let N = {0, 1, d 2, . . . } and give N the discrete topology. It would seem natural to take X = NZ for the state space, but for topological reasons, let us begin by setting N = N ∪ {∞}. Zd

We give N the topology of one point compactification and take X = N with the product topology for the state space. The set X is compact. We associate with X the Borel σ-field. The product topology on X is metrizable. For x = (x1 , . . . , xd ) ∈ Zd , denote the sup-norm of x by x = max |xi |. i=1,...,d

Let γ : N → [0, 2] be such that γ(0) = 2, γ(n) = 1/n, n = 1, 2, . . . , and γ(∞) = 0. For instance, the metric d(η, ξ) =




x∈Zd

2

 1   γ(η(x)) − γ(ξ(x)) x

110

P. A. Ferrari and V. V. Sisko

is compatible with the product topology on X . The set X is a complete separable metric space. Fix 0 < c < 1 and consider a collection λ = {λx }x∈Zd taking values in (c, 1] such d that c = inf x∈Zd λx . We call λ the environment. Let p : Z → [0, 1] be a probability d on Z : x∈Zd p(x) = 1. We assume that the range of p is bounded by some M > 0: p(x) = 0 if x > M . Moreover, suppose that the random walk with transition function p(x, y) = p(y − x) is irreducible. Let g : N → [0, 1] be a nondecreasing continuous function with 0 = g(0) < g(1) and g(∞) = lim g(k) = 1. The zero-range process in the environment λ is a Markov process informally described as follows. Initially distribute particles on the lattice Zd , then if there are k particles at site x, at rate λx g(k)p(y − x) a particle jumps from x to y. In Section 5 we recall the construction of a process ηt with this behavior as a function a la Harris. Let {Sλ (t), t ≥ 0} be the semigroup of a Poisson process in Zd × R, ` associated to this process, that is, Sλ (t)f (η) = E[f (ηt ) | η0 = η]. where E is expectation and ηt = ηλt is the process with fixed environment λ. The corresponding generator Lλ , defined by Lλ f (η) =

 d  Sλ (t)f (η) , dt t=0 Zd

acts on cylinder continuous functions f : N → R as follows:

λx p(y − x) g(η(x)) [f (η x,y ) − f (η)]. (Lλ f )(η) = x∈Zd y∈Zd

where η x,y = η − δx + δy and δz ∈ X is the configuration with just one particle at z and no particles elsewhere; addition of configurations is performed componentwise. We set ∞ ± 1 = ∞. The natural state space for this Markov process is X rather than X . From the construction ` a la Harris it is possible to see that if the standard Markov process whose semigroup is given by Sλ (t) is started in X , then it never leaves X : if µ(X ) = 1, then µSλ (t)(X ) = 1 for any t. For each v ∈ [0, c] and environment λ, denote νλ,v the product measure with marginals (1)

νλ,v {ξ : ξ(x) = k} =

(v/λx )k 1 , Z(v/λx ) g(k)!

where we use the notation g(k)! = g(1) · · · g(k) and g(0)! = 1; (2)

Z(u) =


uk g(k)!

k≥0

is the normalizing constant. These measures are invariant for the process [1, 13, 22]. In some cases it is known that all invariant measures (concentrated on X ) are convex combinations of measures in {νλ,v : 0 ≤ v ≤ c} (see [1, 2]). To define the standard partial order for probability measures on X let η ≤ ξ if η(x) ≤ ξ(x) for all x ∈ Zd . A real valued function f defined on X is increasing

Zero range processes with random rates

111

if η ≤ ξ implies that   f (η) ≤ f (ξ). If µ and ν are two probability measures on X , µ ≤ ν if f dµ ≤ f dν for all increasing continuous functions f . In this case we say that ν dominates µ. This is equivalent to the existence of a probability measure ν¯ on X × X with marginals µ and ν such that ν¯{(η, ξ) : η ≤ ξ} = 1, (coupling); see Theorem 2.4 of Chapter II in [19]. Since X is compact, any sequence of probability measures on X is tight, and therefore, has a weakly convergent subsequence. Our main theorem holds for measures µ on X giving total mass to configurations for which the number of particles in x increases less than exponentially with x. That is, measures satisfying (3)

∞


n=1

e−βn




η(x) < ∞

µ-a.s. for all β > 0.

x:x=n

The product measure νλ,v obviously satisfies (3). We consider random rates λ = {λx }x∈Zd , a collection of independent identically distributed random variables in (c, 1]. Call P and E the probability and expectation induced by these variables. Assume that for any ε > 0, P(λ0 ∈ (c, c + ε)) > 0. Theorem 1. Let µ be a probability measure on X satisfying (3). Then P-a.s. (i) Every weak limit of µSλ (t) as t tends to infinity is dominated by νλ,c . (ii) If νλ,c ≤ µ then µSλ (t) converges to νλ,c as t goes to infinity. The result is better understood using the notion of density of particles. Recall that lim g(k) = 1 and notice that the function Z : [0, 1) → [0, ∞) defined in (2) is analytic. Let R : [0, 1) → [0, ∞) be the strictly increasing function defined by R(u) =

Z  (u) 1
uk =u . k Z(u) g(k)! Z(u) k≥0

It is easy to see that R is onto [0, ∞). Under the measure νλ,v the expected number of particles (density) at site x is (4)

νλ,v [η(x)] = R(v/λx ),

and the expected value of the jump rate is νλ,v [λx g(η(x))] = v. Since v/λx < 1, for any v ∈ [0, c] and x, (5)

νλ,c [η(x)] = lim R(v/λx ) < ∞. v→c

Since the rate distribution is translation invariant, taking the average with respect to the rates, the mean number of particles per site is  ρ(v) := P(dλ0 )R(v/λ0 ). For v ∈ [0, c), ρ(v) < ∞. Depending on the distribution of λ0 , two cases are possible: ρ(c) < ∞ and ρ(c) = ∞. Since R(u) is a nondecreasing nonnegative function,

(6)

ρ(c) = lim ρ(v), vc

P. A. Ferrari and V. V. Sisko

112

The equation also holds when ρ(c)= ∞. For v ∈ [0, c], denote mv := P(dλ) νλ,v the measure obtained by first Pchoosing an environment λ and then choosing a configuration η with νλ,v . Under this law {η(x)}x∈Zd are independent identically distributed random variables with average number of particles per site given by mv [η(0)] = ρ(v). By the strong law of large numbers, (7)


1 η(x) = ρ(v) d n→∞ (2n + 1) lim

mv -a.s.

x≤n

Thus, P-a.s., the limit (7) holds νλ,v -a.s.; it also holds when ρ(v) = ∞. For η ∈ X , the lower asymptotic density of η is defined by (8)

D(η) := lim inf n→∞


1 η(x), d (2n + 1) x≤n

and the upper asymptotic density of η is defined by (9)

D(η) := lim sup n→∞


1 η(x). d (2n + 1) x≤n

Take some probability measure µ satisfying (3) and some environment λ. Let µ ˜ be a weak limit of µSλ (t) along a convergent subsequence. Then Theorem 1 (i) implies (10)

D(η) ≤ ρ(c)

µ ˜ -a.s.

Suppose that ρ(c) < ∞ and µ concentrates mass on configurations with lower asymptotic density strictly bigger than ρ(c), that is, (11)

D(η) > ρ(c) µ-a.s.

Inequality (10) says that weak limits of µSλ (t) are concentrated on configurations with the upper asymptotic density of η not greater than ρ(c). This behavior is remarkable as the process is conservative, i.e., the total number of particles is conserved, but in the above limit there is an “escape of mass”. Heuristically, a fraction of the particles get stacked at further and further sites with lower and lower rates. Sketch of proof. The proof is based on the study of a family of zero-range processes indexed with α > 0; we call them the α-truncated process. The αtruncated process behaves as the original process but at all times there are infinitely many particles in sites x with λ(x) ≤ c + α. The measure νλα is invariant for the process. Let the measure µα be the law of a configuration chosen with µ modified by putting infinitely many particles in sites x with λ(x) ≤ c + α and leaving the other sites unchanged. We use the fact that there is a density of sites with infinitely many particles to show that the α-truncated process starting with µα for µ satisfying (3) converges weakly to νλα . We prove the convergence using coupling arguments. Two α-truncated processes starting respectively with µα and the invariant law νλα are jointly realized using the so called “basic coupling” [19] which amounts to use the same Poisson processes to construct both marginals. The coupling induces first and second class particles, the last represent the discrepancies between both marginals.

Zero range processes with random rates

113

A key element of the proof is the study of the motion of a single tagged second class particle in the α-truncated process. The skeleton of the trajectory of each particle is a simple random walk with jump probabilities p(·) absorbed at sites x with λ(x) ≤ c + α. The interaction with the other particles and with the environment λ governs the waiting times between jumps but does not affect the skeleton of the motion. We show that with probability one (a) only a finite number of second class particles will visit any fixed site x: particles starting sufficiently far away will be absorbed before arriving to x and (b) the finite number of particles hitting x will be eventually absorbed. The weak convergence and the uniqueness of the invariant measure for the α-process is a consequence of this result. The α-process dominates stochastically the original process (which corresponds to α = 0) when both start with the same configuration. Since νλα converges to the maximal invariant measure as α → 0, this will conclude the proof. In Section 3 we introduce the α-truncated process, and state the two main results which lead to the proof of Theorem 1: the ergodicity of the α-truncated process and the fact that it dominates the original process. In the same section we prove Theorem 1. In Section 4 we prove results for the random walk absorbed at sites x with λx ≤ c + α, and in Section 5 we graphically construct the process, introduce the relevant couplings and prove the ergodicity and domination results. 3. The α-truncated process We introduce a family of zero-range process with infinite number of particles at sites with sufficiently slow rates. Let α > 0, cα = c + α and λα = {λα x }x∈Zd the truncation given by  cα if λx ≤ cα , λα x = λx if λx > cα . For each α ≥ 0 consider a X -valued zero-range process ηtα in the environment λ . We call it the α-truncated process or just the truncated process when α is clear from the context. When α = 0 we have the original process: ηt0 = ηt . Partition Zd = Λ(λ, α) ∪ Λc (λ, α) with α

Λ(λ, α) = {x ∈ Zd : λx > c + α} and

Λc (λ, α) = {x ∈ Zd : λx ≤ c + α}.

We impose that ηtα (x) = ∞ for all t for all x ∈ Λc (λ, α). The truncated process ηtα is defined in the same way as ηt from Section 2 with the following differences. Particles jump as before to Λc (λ, α), but since there are infinitely many particles in Λc (λ, α), the rate of jump from x ∈ Λc (λ, α) to y is (c + α)g(∞)p(y − x). Since the number of particles in x is always infinity, this jumps can be interpreted as creation of particles in y. Hence the process ηtα can be thought of as evolving in X α := NΛ(λ,α) with boundary conditions “infinitely many particles at sites in Λc (λ, α)”. α α Let Lα λ be the generator of the α-truncated process ηt and {Sλ (t), t ≥ 0} be the α a la Harris in semigroup associated to the generator Lλ . We construct this process ` Section 5. We consider measures on configurations of the processes ηt and ηtα as measures on X . The product measure νλα with marginals  k 1 (cα /λα  x) if x ∈ Λ(λ, α), α α α νλ {ξ : ξ(x) = k} = Z(c /λx ) g(k)!  1{k = ∞} if x ∈ Λc (λ, α),

P. A. Ferrari and V. V. Sisko

114

is invariant for the process ηtα . Since cα → c and λα (x) → λ(x) as α goes to zero, lim νλα = νλ,c

(12)

weakly.

α→0

Let T α : X → X be the truncation operator defined by (13)

 η(x) T α η(x) = ∞

if λx > c + α, if λx ≤ c + α.

The operator T α induces an operator on measures that we also call T α . Define µα := T α µ. We clearly have µ ≤ µα .

(14)

This domination is preserved by the respective processes: Lemma 1. Let α > 0 and t ≥ 0. Then µSλ (t) ≤ µα Sλα (t). The truncated process converges to the invariant measure: Proposition 1. Let µ be a probability measure on X satisfying (3). Then for any α > 0, lim µα Sλα (t) = νλα

(15)

t→∞

P-a.s.

We prove Lemma 1 and Proposition 1 in Section 5. Proof of Theorem 1. For any α > 0, Lemma 1 and Proposition 1 imply lim sup µSλ (t) ≤ lim sup µα Sλα (t) = νλα . t→∞

t→∞

Item (i) follows by taking α → 0 and applying (12). To prove item (ii), take µ such that νλ,c ≤ µ. In the same way as in the proof of Lemma 1, it is easy to see that the semigroup Sλ (t), acting on measures, preserves the ordering: νλ,c Sλ (t) ≤ µSλ (t) for any t. Since νλ,c is invariant, νλ,c = νλ,c Sλ (t). Therefore, by item (i), νλ,c = lim sup νλ,c Sλ (t) ≤ lim sup µSλ (t) ≤ νλ,c . t→∞

t→∞

Our task is to prove Proposition 1. The point is that the skeleton of each particle is just a discrete-time random walk with absorption at the sites where λx ≤ c + α. Since there is a positive density of those sites, only a finite number of particles will arrive at any fixed finite region. On the other hand, the absorbing sites create new particles. We couple the process with initial measure µα with the process with initial invariant measure νλα in such a way that new particles are created at the same time in the same sites to both processes. New created particles jump together at both marginals. We show that as time goes to infinity, in both processes only new particles will be present in any finite region.

Zero range processes with random rates

115

4. Family of independent random walks Fix η such that the inequality in (3) holds. Fix α > 0. Since η and α are fixed, we omit them in the notation when it is possible. For example, Λc (λ) := Λc (λ, α). For each x ∈ Zd , enumerate the η(x) particles at site x in some way and let

 ζ = ζn (x, i) : x ∈ Zd , i ∈ N ∩ [1, η(x)]

be a family of independent discrete-time random walks with starting points ζ0 (x, i) = x, x ∈ Zd , i ∈ N ∩ [1, η(x)] and transitions governed by p(·). We use the notation P and E for the law and expectation induced by ζ. Recall P and E are the law and expectation induced by the environment λ. By P × P denote the product measure with marginals P and P. For each (x, i) and for each subset A of Zd , denote τ (x, i; A) = min{n ≥ 0 : ζn (x, i) ∈ A} the first time the walk hits the set A (this could be ∞). Let us prove that if we consider the random walks in time [0, τ (x, i; Λc (λ))] only a finite number of walks visit the origin and the number of visits of the origin by each of the walks is finite. More formally, by N (λ, ζ) denote the last time any walk visits the origin before entering in Λc (λ): {m : m ∈ [0, τ (x, y; Λc (λ))] and ζm (x, i) = 0}. N (λ, ζ) = sup x

i

Proposition 2. (16)

(P × P){(λ, ζ) : N (λ, ζ) < ∞} = 1.

Proof. Denote θ = P(λ0 ≤ c + α). If α is small enough, then 0 < θ < 1. Call Ex,i the subset of Zd visited by the walk ζn (x, i) in the time interval [0, τ (x, i; Λc (λ))] and denote Cx,i,N = {(λ, ζ) : |Ex,i | ≥ N } where N ≥ 0 and |Ex,i | is the number of elements in the set Ex,i . Since each site of Ex,i has probability θ to be in the set Λc (λ), (17)

(P × P)(Cx,i,N ) ≤ (1 − θ)N → 0

as N → ∞.

By hypothesis the random walk with transitions governed by p(·) is irreducible, hence it cannot be confined to a finite region. This implies that the number of new sites visited by time n goes to infinity as n increases. This and (17) implies that



 τ (x, i; Λc (λ)) < ∞ (18) (P × P) = 1. (x,i):i≤η(x)

Define

 Dx,i = (λ, ζ) : τ (x, i; {0}) < τ (x, i; Λc (λ)) .

Since the range of the random walk is M < ∞, we see that the random walk ζn (x, i) visits at least (the integer part of) x/M different sites before it reaches the origin. Therefore, (19)

(P × P)(Dx,i ) ≤ (1 − θ)x/M .

P. A. Ferrari and V. V. Sisko

116

Thus




(P × P)(Dx,i ) ≤




(1 − θ)k/M

k

(x,i):i≤η(x)




η(x) < ∞

x:x=k

because we assumed η satisfies (3). Borel-Cantelli then implies that with (P × P) probability one only a finite number of events Dx,i happen. Thus, if we consider the random walks in time [0, τ (x, i; Λc (λ))], then only a finite number of walks visit the origin, and by (18), each walk visits the origin a finite number of times. 5. Construction and coupling Zd

We construct ` a la Harris a Markov process ηt on X = N corresponding to the above description. Let (Nx,y , x, y ∈ Zd ) be a collection of independent Poisson process such that Nx,y has intensity p(y − x). If a Poisson event s belongs to a Poisson process Nx,y , then we say that the event has the origin x and the end y. To tune the rate with the environment and the number of particles, we associate to each Poisson event s ∈ ∪x,y Nx,y a random variable U (s), uniform in [0, 1], independent of the Poisson processes and independent of the other uniform variables. Since the probability that any two Poisson events from ∪x,y Nx,y happen at the same time is zero, all the Poisson events can be indexed by their times, in other words, they can be ordered by their time of occurrence. The evolution of the process ηt = ηλt in the environment λ is given by the following (deterministic) rule: if the Poisson process Nx,y has an event at time s and (20)

U (s) < λx g(ηs− (x)),

then one particle is moved from x to y at that time. Since g(0) = 0, if no particle is in x, then the Poisson event produces no effect in the process in this case. Using that p is finite range, a percolation argument shows that, for h sufficiently small, Zd can be partitioned in finite (random) subsets with the following property: all Poisson events in the interval [0, h] have the origin and the end in the same subset. Since there is a finite number of Poisson events in time interval [0, h] in each of the subsets, the Poisson events can be well ordered by their time of occurrence and the value of ηh for each subset can be obtained with the rule (20) proceeding from the first event to the last in each subset. Starting at ηh , we repeat the construction in the interval [h, 2h] and so on. Thus, for any t, the process ηt is well defined as a function of the Poisson processes and the uniform random variables. The α-truncated process ηtα in the same environment λ is also realized as a function of the Poisson processes and uniform variables with a similar rule: if the Poisson process Nx,y has an event at time s and (21)

α U (s) < λα x g(ηs− (x)),

then one particle is moved from x to y at that time. Rules (20) and (21) induce a natural coupling between the processes ηt and ηtα . This is the key of the proof of Lemma 1. We use the notation P and E for the probability and expectation induced by the Poisson processes and corresponding uniform associated random variables. Notice that this alea does not depend on λ.

Zero range processes with random rates

117

Proof of Lemma 1. Fix a configuration η0 and an environment λ and let η0α (x) = η0 (x) if x ∈ Λ(λ, α) and η0α (x) = ∞ if x ∈ Λc (λ, α). Let (ηt , ηtα ) be the coupling obtained by constructing each marginal as a function of the Poisson processes (Nx,y , x, y ∈ Zd ) and uniform random variables (U (s), s ∈ ∪x,y Nx,y ) following rules (20) and (21). It suffices to show that each jump keeps the initial order. Consider the jump associated to a Poisson event at time s ∈ Nx,y with uniform variable U (s). There are two possibilities: (1) If x ∈ Λ(λ, α), then λx = λα x . Since the function g(·) is monotone and the random variable U (s) is the same for both marginals, the order is kept. (2) If x ∈ Λc (λ, α), then λx < λα x . In this case a ηs− (x) particle jumps from x to y if α α (x) particle jumps from x to y if U (s) < λα U (s) < λx g(ηs− (x)) and a ηs− x g(ηs− (x)). α α α Hence, if ηs− (x) ≤ ηs− (x) and ηs− (y) ≤ ηs− (y), then ηs (y) ≤ ηs (y). On the other hand, ηs (x) ≤ ηsα (x) = ∞. To prove Proposition 1, we need the following result. It helps to prove that the second class particles do not stop forever at some place: eventually every such particle either move or coalesce. Lemma 2. Fix an environment λ and consider the stationary process (ηtα , t ∈ R) with time-marginal distribution νλα and fix x ∈ Λ(λ, α). Then ηtα (x) = 0 infinitely often with probability one: (22)

lim inf ηtα (x) = 0. t→∞

Proof. Consider the discrete time stationary process (ηnα (x), n ∈ N) —this is just the process (ηt (x), t ∈ R) observed at integer times. It is sufficient to show (23)

lim inf ηnα (x) = 0 n→∞

with probability one. A theorem of Poincar´e (Chapter IV in [21] or Theorem 3.4 of Chapter 6 in [6]) implies that for every k ∈ N,   P ηnα (x) = k infinitely often in n | η0α (x) = k = 1.

Returning for a moment to the continuous time process ηtα , if at time t site x has at least one particle, then one of the particles at x will jump with probability bounded below by g(1)λx /(1 + λx ) > 0, this is the probability the exponential jump time of x is smaller than the  jump-times of particles from the other sites to x, whose rate is bounded by g(∞) y p(y, x) = 1. Fix k ∈ N. By the same reasoning, for any m, α (x) = k, then there is a positive probability to be visiting 0 at time m + 1 if ηm independently of previous visits and uniformly in the configuration outside x at time m. Since these are independent attempts, Borel–Cantelli implies   P ηnα (x) = 0 infinitely often in n | η0α (x) = k = 1. This implies (23).

Proof of Proposition 1. In an environment λ, consider the coupling process of two versions of the process ηtα obtained by using the same family  of Poisson processes (Nx,y : x, y ∈ Zd ) and uniform random variables (U (s), s ∈ x,y Nx,y ). By {S¯λα (t) : t ≥ 0} denote the semigroup of the process and by P the probability associated to the process.

P. A. Ferrari and V. V. Sisko

118

Since νλα is invariant for ηtα , it is enough to show that, for any α > 0, any µ satisfying (3), any λ (P-a.s.), and for any x ∈ Λ(λ, α), lim (µα × νλα ) S¯λα (t) {(ξ, η) : ξ(x) = η(x)} = 0.

(24)

t→∞

In coupling terms, (24) reads     (25) lim µα (dξ) νλα (dη) P ξt (x) = ηt (x) | (ξ0 , η0 ) = (ξ, η) = 0, t→∞

where we have denoted ξt the first coordinate of the coupled processes and ηt the second. Therefore, to prove the proposition it is enough to prove that, for any α > 0, any µ satisfying (3), any λ (P-a.s.), any ξ 0 (µ-a.s.), any η 0 (νλα -a.s.), and for any x ∈ Λ(λ, α),   (26) lim P ξt (x) = ηt (x) | (ξ0 , η0 ) = (ξ 0 , η 0 ) = 0. t→∞

Without loss of generality we assume x = 0 and α small enough such that 0 ∈ Λ(λ, α). Fix α, λ, ξ 0 and η 0 . The configurations ξ 0 and η 0 are in principle not ordered: there are sites y ∈ Λ(λ, α) such that (ξ 0 (y) − η 0 (y))+ > 0 and sites z ∈ Λ(λ, α) such that (ξ 0 (z) − η 0 (z))− > 0. We say that we have ξη-discrepancies in the first case and ηξ-discrepancies in the second one. Denote ξ¯t (z) := min{ξt (z), ηt (z)} the number of coupled particles at site z at ¯ time t. The ξ-particles move as regular zero-range particles; they are usually called first class particles. There is at most one type of discrepancy at each site at any time. Discrepancies of both types move as second class particles, i.e., ξη-discrepancy jumps from y to z with rate (27)

¯ λα y p(z − y)[g(ξ(y)) − g(ξ(y))]

and ηξ-discrepancy jumps from y to z with rate (28)

¯ λα y p(z − y)[g(η(y)) − g(ξ(y))]

that is, second class particles jump with the difference rate. For instance, in the case g(k) ≡ 1, the second class particles jump only when there are no coupled particles in the site. If a ξη-discrepancy jumps to a site z occupied by at least one ηξ-discrepancy, then the ξη-discrepancy and one of the ηξ-discrepancies at z coalesce into a coupled ¯ ξ-particle in z. Analogously, for the case when a ηξ-discrepancy jumps to a site z occupied by at least one ξη-discrepancy. The coupled particle behaves from this moment on as a first class particle. If a discrepancy of any type jumps to a site z with infinite number of particles, that is, z ∈ Λc (λ, α), then the discrepancy ¯ Therefore, disappears. All particles in sites x ∈ Λc (λ, α) are first class ξ-particles. c any particle that jump from any site x ∈ Λ (λ, α) is a first class particle. At time zero there are |ξ 0 (y) − η 0 (y)| discrepancies at site y. To the ith discrepancy at site y at time zero, that is, discrepancy (y, i), we associate the random walk ζn (y, i) from the model of Section 4. Since the interaction with the other particles and the environment λ governs the waiting times between jumps but does not affect the skeleton of the discrepancy motion until coalescence or absorbing time, it is possible to couple the skeleton of the discrepancy (y, i) with the random walk ζn (y, i) in such a way that they perform the same jumps together until (a) the coalescence of the discrepancy with another

Zero range processes with random rates

119

discrepancy of different type or (b) the absorption of the discrepancy at some site of Λc (λ). In any case, the number of discrete jumps is at most τ (y, i; Λc (λ)). Therefore, the full trajectory of discrepancy (y, i) is shorter (visits not more sites and has not more number of visits to each site) than the trajectory of the random walk ζn (y, i) in the time interval [0, τ (y, i; Λc (λ))]. Thus, Proposition 2 implies that only a finite number of discrepancies visit x and the number of visits of site x by each of the discrepancies is finite. Lemma 2 implies that there are no η-particles at x infinitely often. Therefore, there are no ηξ-discrepancies at x infinitely often. This means that every ηξ-discrepancy that at some moment is at x will eventually jump out or coalesce. It follows that after some random time there is no ηξ-discrepancies in x forever. Moreover, if at time t site x ∈ Λ(λ, α) has no η-particles, then a ξη-discrepancy at x will jump with probability bounded below by g(1)λx /(1 + λx ) > 0. Therefore, using Lemma 2, we see that after some random time there is no ξη-discrepancies in x forever. Acknowledgments. We thank Enrique Andjel and James Martin for discussions. Part of this paper was written when the first author was participating of the program Principles of the Dynamics of Non-Equilibrium Systems in the Isaac Newton Institute for Mathematical Sciences visiting Newton Institute, in May-June 2006. This paper was partially supported by FAPESP and CNPq. References [1] Andjel, E. D. (1982). Invariant measures for the zero range processes. Ann. Probab. 10 525–547. MR659526 [2] Andjel, E. D., Ferrari, P. A., Guiol, H. and Landim, C. (2000). Convergence to the maximal invariant measure for a zero-range process with random rates. Stochastic Process. Appl. 90 67–81. MR1787125 [3] Bahadoran, C. (1998). Hydrodynamical limit for spatially heterogeneous simple exclusion processes. Probab. Theory Related Fields 110 287–331. MR1616563 ´zs, M., Rassoul-Agha, F., Seppa ¨ la ¨inen, T. and Sethuraman, S. [4] Bala (2007). Existence of the zero range process and a deposition model with superlinear growth rates. To appear. math/0511287. [5] Benjamini, I., Ferrari, P. A. and Landim, C. (1996). Asymmetric conservative processes with random rates. Stochastic Process. Appl. 61 181–204. MR1386172 [6] Durrett, R. (1996). Probability: Theory and Examples. Duxbury Press, Belmont, CA. MR1609153 [7] Evans, M. R. (1996). Bose-Einstein condensation in disordered exclusion models and relation to traffic flow. Europhys. Lett. 36 13–18. [8] Evans, M. R. and Hanney, T. (2005). Nonequilibrium statistical mechanics of the zero-range process and related models. J. Phys. A 38 R195–R240. MR2145800 [9] Gielis, G., Koukkous, A. and Landim, C. (1998). Equilibrium fluctuations for zero range processes in random environment. Stochastic Process. Appl. 77 187–205. MR1649004 ¨ la ¨inen, T. (2004). Behavior dom[10] Grigorescu, I., Kang, M. and Seppa inated by slow particles in a disordered asymmetric exclusion process. Ann. Appl. Probab. 14 1577–1602. MR2071435

120

P. A. Ferrari and V. V. Sisko

[11] Harris, T. E. (1972). Nearest-neighbor Markov interaction processes on multidimensional lattices. Advances in Math. 9 66–89. MR0307392 [12] Holley, R. (1970). A class of interactions in an infinite particle system. Advances in Math. 5 291–309. MR0268960 [13] Jackson, J. R. (1957). Networks of waiting lines. Operations Res. 5 518–521. MR0093061 [14] Koukkous, A. (1999). Hydrodynamic behavior of symmetric zero-range processes with random rates. Stochastic Process. Appl. 84 297–312. MR1719270 [15] Krug, J. and Ferrari, P. A. (1996). Phase transitions in driven diffusive systems with random rates. J. Phys. A: Math. Gen. 29 1465–1471. [16] Landim, C. (1996). Hydrodynamical limit for space inhomogeneous onedimensional totally asymmetric zero-range processes. Ann. Probab. 24 599– 638. MR1404522 [17] Liggett, T. M. (1972). Existence theorems for infinite particle systems. Trans. Amer. Math. Soc. 165 471–481. MR0309218 [18] Liggett, T. M. (1973). An infinite particle system with zero range interactions. Ann. Probab. 1 240–253. MR0381039 [19] Liggett, T. M. (1985). Interacting Particle Systems. Springer, New York. MR776231 ¨ la ¨inen, T. and Krug, J. (1999). Hydrodynamics and platoon forma[20] Seppa tion for a totally asymmetric exclusion model with particlewise disorder. J. Statist. Phys. 95 525–567. MR1700871 [21] Shiryaev, A. N. (1996). Probability. Springer, New York. MR1368405 [22] Spitzer, F. (1970). Interaction of Markov processes. Advances in Math. 5 246–290. MR0268959

IMS Lecture Notes–Monograph Series Asymptotics: Particles, Processes and Inverse Problems Vol. 55 (2007) 121–134 c Institute of Mathematical Statistics, 2007
DOI: 10.1214/074921707000000319

On non-asymptotic bounds for estimation in generalized linear models with highly correlated design Sara A. van de Geer1 ETH Z¨ urich Abstract: We study a high-dimensional generalized linear model and penalized empirical risk minimization with
1 penalty. Our aim is to provide a non-trivial illustration that non-asymptotic bounds for the estimator can be obtained without relying on the chaining technique and/or the peeling device.

1. Introduction We study an increment bound for the empirical process, indexed by linear combinations of highly correlated base functions. We use direct arguments, instead of the chaining technique. We moreover obtain bounds for an M-estimation problem inserting a convexity argument instead of the peeling device. Combining the two results leads to non-asymptotic bounds with explicit constants. Let us motivate our approach. In M-estimation, some empirical average indexed by a parameter is minimized. It is often also called empirical risk minimization. To study the theoretical properties of the thus obtained estimator, the theory of empirical processes has been a successful tool. Indeed, empirical process theory studies the convergence of averages to expectations, uniformly over some parameter set. Some of the techniques involved are the chaining technique (see e.g. [13]), in order to relate increments of the empirical process to the entropy of parameter space, and the peeling device (a terminology from [10]) which goes back to [1], which allows one to handle weighted empirical processes. Also the concentration inequalities (see e.g. [9]), which consider the concentration of the supremum of the empirical process around its mean, are extremely useful in M-estimation problems. A more recent trend is to derive non-asymptotic bounds for M-estimators. The papers [6] and [4] provide concentration inequalities with economical constants. This leads to good non-asymptotic bounds in certain cases [7]. Generally however, both the chaining technique and the peeling device may lead to large constants in the bounds. For an example, see the remark following (5). Our aim in this paper is simply to avoid the chaining technique and the peeling device. Our results should primarily be seen as non-trivial illustration that both techniques may be dispensable, leaving possible improvements for future research. In particular, we will at this stage not try to optimize the constants, i.e. we will make some arbitrary choices. Moreover, as we shall see, our bound for the increment involves an additional log-factor, log m, where m is the number of base functions (see below). 1 Seminar f¨ ur Statistik, ETH Z¨ urich, LEO D11, 8092 Z¨ urich, Switzerland, e-mail: [email protected] AMS 2000 subject classification: 62G08. Keywords and phrases: convex hull, convex loss, covering number, non-asymptotic bound, penalized M-estimation.

121

S. A. van de Geer

122

The M-estimation problem we consider is for a high-dimensional generalized linear model. Let Y ∈ Y ⊂ R be a real-valued (response) variable and x be a covariate with values in some space X . Let
 m  fθ (·) := θk ψk (·), θ ∈ Θ k=1

be a (subset of a) linear space of functions on X . We let Θ be a convex subset of Rm , possibly Θ = Rm . The functions {ψk }m k=1 form a given system of real-valued base functions on X . The number of base functions, m, is allowed to be large. However, we do have the situation m ≤ n in mind (as we will consider the case of fixed design). Let γf : X × Y → R be some loss function, and let {(xi , Yi )}ni=1 be observations in X × Y. We consider the estimator with 1 penalty 
n  2 2(1−s) 1 2−s (1) θˆn := arg min γf (xi , Yi ) + λn I 2−s (θ) , θ∈Θ n i=1 θ where (2)

I(θ) :=

m 

|θk |

k=1

denotes the 1 norm of the vector θ ∈ Rm . The smoothing parameter λn controls the amount of complexity regularization, and the parameter s (0 < s ≤ 1) is governed by the choice of the base functions (see Assumption B below). Note that for a properly chosen constant C depending on λn and s, we have for any I > 0,   2 2(1−s) C 2−s 2−s λn I = min λI + 2(1−s) . λ λ s 2

2(1−s)

In other words, the penalty λn2−s I 2−s (θ) can be seen as the usual Lasso penalty λI(θ) with an additional penalty on λ. The choice of the latter is such that adaption to small values of I(θn∗ ) is achieved. Here, θn∗ is the target, defined in (3) below. The loss function γf is assumed to be convex and Lipschitz (see Assumption L below). Examples are the loss functions used in quantile regression, logistic regression, etc. The quadratic loss function γf (x, y) = (y − f (x))2 can be studied as well without additional technical problems. The bounds then depend on the tail behavior of the errors. The covariates x1 , . . . , xn are assumed to be fixed, i.e., we consider the case of fixed design. For γ : X × Y → R, use the notation n

P γ := Our target function θn∗ is defined as (3)

1 Eγ(xi , Yi ). n i=1

θn∗ := arg min P γfθ . θ∈Θ

∗ are When the target is sparse, i.e., when only a few of the coefficients θn,k ˆ nonzero, it makes sense to try to prove that also the estimator θn is sparse. Nonasymptotic bounds for this case (albeit with random design) are studied in [12]. It

Non-asymptotic bounds for GLM

123

is assumed there that the base functions {ψk } have design matrix with eigenvalues bounded away from zero (or at least that the base functions corresponding to the non-zero coefficients in θn∗ have this property). In the present paper, the base functions are allowed to be highly correlated. We will consider the case where they form a VC class, or more generally, have -covering number which is polynomial in 1/. This means that a certain smoothness is imposed a priori, and that sparseness is less an issue. We use the following notation. The empirical distribution based on the sample {(xi , Yi )}ni=1 is denoted by Pn , and the empirical distribution of the covariates {xi }ni=1 is written as Qn . The L2 (Qn ) norm is written as  · n . Moreover,  · ∞ denotes the sup norm (which in our case may be understood as f ∞ = max1≤i≤n |f (xi )|, for a function f on X ). We impose four basic assumptions: Assumptions L, M, A and B. Assumption L. The loss function γf is of the form γf (x, y) = γ(f (x), y), where γ(·, y) is convex for all y ∈ Y. Moreover, it satisfies the Lipschitz property |γ(fθ (x), y) − γ(fθ˜(x), y)| ≤ |fθ (x) − fθ˜(x)|, ∀ (x, y) ∈ X × Y, ∀ θ, θ˜ ∈ Θ. Assumption M. There exists a non-decreasing function σ(·), such that all M > 0 and all all θ ∈ Θ with fθ − fθn∗ ∞ ≤ M , one has P (γfθ − γfθ∗ ) ≥ fθ − fθn∗ 2n /σ 2 (M ). n

Assumption M thus assumes quadratic margin behavior. In [12], more general margin behavior is allowed, and the choice of the smoothing parameter does not depend on the margin behavior. However, in the setup of the present paper, the choice of the smoothing parameter does depend on the margin behavior. Assumption A. It holds that ψk ∞ ≤ 1, 1 ≤ k ≤ m. Assumption B. For some constant A ≥ 1, and for V = 2/s − 2, it holds that N (, Ψ) ≤ A−V , ∀  > 0. Here N (, Ψ) denotes the -covering number of (Ψ,  · n ), with Ψ := {ψk }m k=1 . The paper is organized as follows. Section 2 presents a bound for the increments of the empirical process. Section 3 takes such a bound for granted and presents a non-asymptotic bound for fθˆn − fθn∗ n and I(θˆn ). The two sections can be read independently. In particular, any improvement of the bound obtained in Section 2 can be directly inserted in the result of Section 3. The proofs, which are perhaps the most interesting part of the paper, are given in Section 4. 2. Increments of the empirical process indexed by a subset of a linear space Let ε1 , . . . , εn be i.i.d. random variables, taking values ±1 each with probability 1/2. Such a sequence is called a Rademacher sequence. Consider for  > 0 and M > 0, the quantity   n 1     sup fθ (xi )εi . Z,M :=   fθ n ≤, I(θ)≤M  n i=1

S. A. van de Geer

124

We need a bound for the mean EZ,M , because this quantity will occur in the concentration inequality (Theorem 4.1). In [12], the following trivial bound is used:   n  1    EZ,M ≤ M E max εi ψk (xi ) . 1≤k≤m n i=1

On the right hand side, one now has the mean of finitely many functions, which is easily handled (see for example Lemma 4.1). However, when the base functions ψk are highly correlated, this bound is too rough. We need therefore to proceed differently. m m Let conv(Ψ) = {fθ = k=1 θk ψk : θk ≥ 0, k=1 θk = 1} be the convex hull of Ψ. Recall that s = 2/(2+V ), where V is from Assumption B. From e.g. [10], Lemma 3.2, it can be derived that for some constant C, and for all  > 0,   n   1 1  max (4) E f (xi )εi  ≤ Cs √ . n f ∈conv(Ψ),f n ≤ n i=1 The result follows from the chaining technique, and applying the entropy bound (5)

log N (, conv(Ψ)) ≤ A0 −2(1−s) ,  > 0,

which is derived in [2]. Here, A0 is a constant depending on V and A. Remark. It may be verified that the constant C in (4) is then at least proportional to 1/s, i.e., it is large when s is small. Our aim is now to obtain a bound from direct calculations. Pollard ([8]) presents the bound 1 log N (, conv(Ψ)) ≤ A1 −2(1−s) log ,  > 0,  where A1 is another constant depending on V and A. In other words, Pollard’s bound has an additional log-factor. On the other hand, we found Pollard’s proof a good starting point in our attempt to derive the increments directly, without chaining. This is one of the reasons why our direct bound below has an additional log m factor. Thus, our result should primarily be seen as illustration that direct calculations are possible. Theorem 2.1. For  ≥ 16/m, and m ≥ 4, we have  

n   √ 1 log(6m)   s max . f (xi )εi  ≤ 20 1 + 2A E  f ∈conv(Ψ),f n ≤ n n i=1

m

Clearly the set { k=1 θk ψk : I(θ) ≤ 1} is the convex hull of {±ψk }m k=1 . Using a renormalization argument, one arrives at the following corollary Corollary 2.1. We have for /M > 8/m and m ≥ 2

√ log(12m) 1−s s . EZ,M ≤ 20 1 + 4AM  n Invoking symmetrization, contraction and concentration inequalities (see Section 4), we establish the following lemma. We present it in a form convenient for application in the proof of Theorem 3.1.

Non-asymptotic bounds for GLM

125

Lemma 2.1. Define for  > 0, M > 0, and /M > 8/m, m ≥ 2, Z,M :=

sup ∗ )≤M fθ −fθ∗ n ≤, I(θ−θn

|(Pn − P )(γfθ − γfθ∗ )|. n

n

Let λn,0

√ log(12m) . := 80 1 + 4A n

Then it holds for all σ > 0, that    2 n2 P Z,M ≥ λn,0 s M 1−s + ≤ exp − . 27σ 2 2 × (27σ 2 )2 3. A non-asymptotic bound for the estimator The following theorem presents bounds along the lines of results in [10], [11] and [3], but it is stated in a non-asymptotic form. It moreover formulates explicitly the dependence on the expected increments of the empirical process. Theorem 3.1. Define for  > 0 and M > 0, Z,M :=

sup ∗ )≤M fθ −fθ∗ n ≤, I(θ−θn

|(Pn − P )(γfθ − γfθ∗ )|. n

n

Let λn,0 be such that for all 8/m ≤ /M ≤ 1, we have EZ,M ≤ λn,0 s M 1−s .

(6) Let c ≥ 3 be some constant. Define

Mn := 2 2(1−s) (27)− 2(1−s) c 1−s I(θn∗ ), 2−s

s

1

σn2 := σ 2 (Mn ), and n :=

√

2

1

1−s

2−s 54σn2−s c 2−s λn,0 I 2−s (θn∗ ) ∨ 27σn2 λn,0 . 1

Assume that (7)

1≤



27 2

2−s − 2(1−s)

1

c 1−s

1 σn2 λn,0

I(θn∗ ) ≤

m 2−s 8

Then for λn := cσns λn,0 , with probability at least 2 2(1−s) 2  2−s 2−s nλn,0 c I 2−s (θn∗ ) 1 − exp − , 4(1−s) 27σn 2−s

we have that fθˆn − fθn∗ n ≤ n and

I(θˆn − θn∗ ) ≤ Mn .

.

S. A. van de Geer

126

Let us formulate the asymptotic implication of Theorem 3.1 in a corollary. For positive sequences {an } and {bn }, we use the notation an b n , when 0 < lim inf n→∞

an an ≤ lim sup < ∞. bn n→∞ bn

The corollary yields e.g. the rate n n−1/3 for the case where the penalty represents the total variation of a function f on {x1 , . . . , xn } ⊂ R (in which case s = 1/2). Corollary 3.1. Suppose that A and s do not depend on n,√and that I(θn∗ ) 1 and σ 2 (Mn ) 1 for all Mn 1. By (4), we may take λn 1/ n, in which case, with probability 1 − exp[−dn ], it holds that fθˆn − fθn∗ n ≤ n , and I(θˆn − θn∗ ) ≤ Mn , with 1 1−s n n− 2(2−s) , Mn 1, dn n2n n 2−s . 4. Proofs 4.1. Preliminaries Theorem 4.1 (Concentration theorem [6]). Let Z1 , . . . , Zn be independent random variables with values in some space Z and let Γ be a class of real-valued functions on Z, satisfying ai,γ ≤ γ(Zi ) ≤ bi,γ , for some real numbers ai,γ and bi,γ and for all 1 ≤ i ≤ n and γ ∈ Γ. Define L2 := sup

n  (bi,γ − ai,γ )2 /n,

γ∈Γ i=1

and

  n 1     (γ(Zi ) − Eγ(Zi )) . Z := sup    n γ∈Γ i=1

Then for any positive z,

 nz 2 P(Z ≥ EZ + z) ≤ exp − 2 . 2L

The Concentration theorem involves the expectation of the supremum of the empirical process. We derive bounds for it using symmetrization and contraction. Let us recall these techniques here. Theorem 4.2 (Symmetrization theorem [13]). Let Z1 , . . . , Zn be independent random variables with values in Z, and let ε1 , . . . , εn be a Rademacher sequence independent of Z1 , . . . , Zn . Let Γ be a class of real-valued functions on Z. Then  n  n             εi γ(Zi ) . E sup  {γ(Zi ) − Eγ(Zi )} ≤ 2E sup      γ∈Γ γ∈Γ i=1

i=1

Non-asymptotic bounds for GLM

127

Theorem 4.3 (Contraction theorem [5]). Let z1 , . . . , zn be non-random elements of some space Z and let F be a class of real-valued functions on Z. Consider Lipschitz functions γi : R → R, i.e. s)| ≤ |s − s˜|, ∀ s, s˜ ∈ R. |γi (s) − γi (˜ Let ε1 , . . . , εn be a Rademacher sequence. Then for any function f ∗ : Z → R, we have       n n         ∗ ∗ εi {γi (f (zi )) − γi (f (zi ))} ≤ 2E sup  εi (f (zi ) − f (zi )) . E sup    f ∈F  f ∈F  i=1

i=1

We now consider the case where Γ is a finite set of functions.

Lemma 4.1. Let Z1 , . . . , Zn be independent Z-valued random variables, and γ1 , . . . , γm be real-valued functions on Z, satisfying ai,k ≤ γk (Zi ) ≤ bi,k , for some real numbers ai,k and bi,k and for all 1 ≤ i ≤ n and 1 ≤ k ≤ m. Define n  L := max (bi,k − ai,k )2 /n, 2

1≤k≤m

Then

i=1

 

n 1   log(3m)   E max  . {γk (Zi ) − Eγk (Zi )} ≤ 2L  1≤k≤m  n n i=1 

Proof. The proof uses standard arguments, as treated in e.g. [13]. Let us write for 1 ≤ k ≤ m,  n  1 γ¯k := γk (Zi ) − Eγk (Zi ) . n i=1 By Hoeffding’s inequality, for all z ≥ 0

 nz 2 P (|¯ γk | ≥ z) ≤ 2 exp − 2 . 2L Hence,



 ∞    n 4L2 2 log t dt γ¯ = 1 + P |¯ γk | ≥ E exp 4L2 k n 1 ≤1+2



1

∞

1 dt = 3. t2

Thus   4 2 2L E max |¯ γk | = √ E max log exp γ ¯ 1≤k≤m 1≤k≤m 4L2 k n 

   4 2 2L log(3m) . ≤√ log E max exp γ¯k ≤ 2L 2 1≤k≤m 4L n n 



S. A. van de Geer

128

4.2. Proofs of the results in Section 2 Proof of Theorem 2.1. Let us define, for k = 1, . . . , m, n

ξk := We have

1 ψk (xi )εi . n i=1

n

m

 1 fθ (xi )εi = θk ξk . n i=1 k=1

Partition {1, . . . , m} into N := N ( , Ψ) sets Vj , j = 1, . . . , N , such that s

ψk − ψl n ≤ 2s , ∀ k, l ∈ Vj . We can write n N   1 fθ (xi )εi = αj pj,k ξk , n i=1 j=1 k∈Vj

where αj = αj (θ) :=



θk , pj,k = pj,k (θ) :=

k∈Vj

Set for j = 1, . . . , N , nj = nj (α) := 1 +

θk . αj

αj

. 2(1−s)

Choose πt,j = πt,j (θ), t = 1, . . . , nj , j = 1, . . . , N independent random variables, independent of ε1 , . . . , εn , with distribution P(πt,j = k) = pj,k , k ∈ Vj , j = 1, . . . , N. nj nj ψπt,j /nj and ξ¯j = ξ¯j (θ) := i=1 ξπt,j /nj . Let ψ¯j = ψ¯j (θ) := i=1 ¯ ¯ N We will choose a realization {(ψj∗ , ξj∗ ) = (ψj∗ (θ), ξj∗ (θ))}N j=1 of {(ψj , ξj )}j=1 den pending on {εi }i=1 , satisfying appropriate conditions (namely, (9) and (10) below). We may then write  N  m  m N            ∗ ∗ θk ξk  ≤  αj ξj  +  θk ξk − αj ξj .        k=1

j=1

k=1

j=1

Consider now

N 

αj ξj∗ .

j=1

N

Let AN := { i=1 αj = 1, αj ≥ 0}. Endow AN with the 1 metric. The -covering number D() of AN satisfies the bound  N 4 . D() ≤ 

N  Let A  be a maximal -covering set of A . For all α ∈ A there is an α ∈ A such N  that j=1 |αj − αj | ≤ .

Non-asymptotic bounds for GLM

129

We now write         m N m N N                ∗ ∗  ∗ θk ξk  ≤  (αj − αj )ξj  +  θk ξk − αj ξj  +  αj ξj           k=1

j=1

j=1

k=1

j=1

:= i(θ) + ii(θ) + iii(θ).

Let Π be the set of possible values of the vector {πt,j : t = 1, . . . , nj , j = 1, . . . , N }, as θ varies. Clearly, i(θ) ≤  max max|ξ¯j |, j

Π

where we take the maximum over all possible realizations of {ξ¯j }N j=1 over all θ. For each t and j, πt,j takes its values in {1, . . . , m}, that is, it takes at most m values. We have N 

nj ≤ N +

j=1

N 

αj 2(1−s) 

j=1

≤ A−sV +

m

k=1 θk 2(1−s) 

= (1 + A)−2(1−s) ≤ K + 1. where K is the integer K := (1 + A)2(1−s) . N The number of integer sequences {nj }N j=1 with j=1 nj ≤ K + 1 is equal to   −2(1−s) N +K +2 ≤ 2N +K+2 ≤ 4 × 2(1+2A) . K +1 So the cardinality |Π| of Π satisfies −2(1−s)

|Π| ≤ 4 × 2(1+2A)

−2(1−s)

× m(1+A)

−2(1−s)

≤ (2m)(1+2A)

,

since A ≥ 1 and m ≥ 4. ≤ 1 for all ψ ∈ Ψ, we know that for any convex combination Now, since ψ∞ 2 ¯2 ¯ p ξ , one has E| k k k k pk ξk | ≤ 1/n. Hence Eξj ≤ 1/n for any fixed ξj and thus, by Lemma 4.1,



√ √ log(6m) log(6m) −(1−s) s ¯ = 2 1 + 2A . (8) E max max |ξj | ≤ 2 1 + 2A j Π n n We now turn to ii(θ). By construction, for i = 1, . . . , n, t = 1, . . . , nj , j = 1, . . . , N ,  pj,k ψk (xi ) := gj (xi ) Eψπt,j (xi ) = k∈Vj

and hence E(ψπt,j (xi ) − gj (xi ))2 ≤ max (ψk (xi ) − ψl (xi ))2 . k,l∈Vj

Thus E(ψ¯j (xi ) − gj (xi ))2 ≤ max (ψk (xi ) − ψl (xi ))2 /nj , k,l∈Vj

S. A. van de Geer

130

and so Eψ¯j − gj 2n ≤ max ψk − ψl 2n /nj ≤ (2s )2 /nj = 42s /nj . k,l∈Vj

Therefore  2 N N      ¯ E αj (ψj − gj ) = αj2 Eψ¯j − gj 2n   j=1

j=1

n

≤ 42s

N  αj2 j=1

nj

≤ 42s

N  αj2 2(1−s) j=1

αj

≤ 42 .

Let Eε denote conditional expectation given {εi }ni=1 . Again, by construction  Eε ξπt,j = pj,k ξk := ej = ej (θ), k∈Vj

and hence Eε (ξπt,j − ej )2 ≤ max (ξk − ξl )2 . k,l∈Vj

Thus Eε (ξ¯j − ej )2 ≤ max (ξk − ξl )2 /nj . k,l∈Vj

So we obtain N  N N     |ξk − ξl |   ¯ Eε  αj (ξj − ej ) ≤ αj Eε |ξ¯j − ej | ≤ αj max √   k,l∈Vj nj j=1 j=1 j=1

N N   αj 1−s √ 1−s max |ξk − ξl | = αj  max |ξk − ξl | √ k,l∈Vj k,l∈Vj α j j=1 j=1 √ √ ≤ N 1−s max max |ξk − ξl | ≤ A max max |ξk − ξl |.

≤

j

k,l∈Vj

j

k,l∈Vj

It follows that, given {εi }ni=1 , there exists a realization {(ψj∗ , ξj∗ ) = (ψj∗ (θ), ξj∗ (θ))}N j=1 of {(ψ¯j , ξ¯j )}N j=1 such that 

(9)

N 

αj (ψj∗ − gj )2n ≤ 4

j=1

as well as (10)

  N   √   ∗ αj (ξj − ej ) ≤ 2 A max max |ξk − ξl |.  j k,l∈Vj   j=1

Thus we have

√ ii(θ) ≤ 2 A max max |ξk − ξl |. j

k,l∈Vj

Since E|ξk − ξl |2 ≤ 22 /n for all k, l ∈ Vj and all j, we have by Lemma 4.1,

√ s log(6m) √ . (11) 2 AE max max |ξk − ξl | ≤ 6 A j k,l∈Vj n

Non-asymptotic bounds for GLM

131

Finally, consider iii(θ). We know that   N     αj gj  ≤ . fθ n =    j=1

n

Moreover, we have shown in (9) that   N     ∗ αj (ψj − gj ) ≤ 4.    j=1

Also

  N N      ∗ |αj − αj |ψj∗ n ≤ ,  (αj − αj )ψj  ≤   j=1

since

ψj∗ ∞

n

j=1

n

≤ 1 for all j . Thus

      N N N            ∗  ∗ ∗ αj ψj  ≤  (αj − αj )ψj  +  αj (ψj − gj ) + fθ n ≤ 6.        j=1

n

j=1

j=1

n

The total number of functions of the form

n

N

j=1

αj ξj∗ is bounded by

 A−2(1−s)  N −2(1−s) 4 4 × |Π| ≤ × (2m)(1+2A)   −2(1−s)

≤ (2m)(1+2A)

,

since we assume  ≥ 16/m, and A ≥ 1. Hence, by Lemma 4.1, (12)

max | E max  α ∈A

Π

N 

αj ξj∗ |

√ ≤ 12 1 + 2As

j=1



log(6m) . n

We conclude from (8), (11), and (12), that   N     αj (θ)ej (θ) E max  θ  j=1



√ s log(6m) √ √ log(6m) log(6m) s s ≤ 2 1 + 2A + 6 A + 12 1 + 2A n n n

√ log(6m) ≤ 20 1 + 2As . n Proof of Lemma 2.1. Let   n 1     sup γfθ (xi , Yi )εi  Z,M :=   fθ n ≤, I(θ)≤M  n i=1 denote the symmetrized process. Clearly, {fθ = ˜ := {±ψk }m . Moreover, we have convex hull of Ψ k=1

m

˜ ≤ 2N (, Ψ). N (, Ψ)

k=1 θk ψk

: I(θ) = 1} is the

S. A. van de Geer

132

˜ and use a rescaling argument, to see that Now, apply Theorem 2.1, to Ψ,

√ log(12m) s 1−s EZ,M ≤ 20 1 + 4A M . n Then from Theorem 4.2 and Theorem 4.3, we know that EZ,M ≤ 4EZ,M . The result now follows by applying Theorem 4.1. 4.3. Proofs of the results in Section 3 The proof of Theorem 3.1 depends on the following simple convexity trick. Lemma 4.2. Let  > 0 and M > 0. Define f˜n = tfˆn + (1 − t)fn∗ with t := (1 + fˆn − fn∗ n / + I(fˆn − fn∗ )/M )−1 , and with fˆn := fθˆn and fn∗ := fθn∗ . When it holds that f˜n − fn∗ n ≤ then

 M , and I(f˜n − fn∗ ) ≤ , 3 3

fˆn − fn∗ n ≤ , and I(fˆn − fn∗ ) ≤ M.

Proof. We have

f˜n − fn∗ = t(fˆn − fn∗ ),

so f˜n − fn∗ n ≤ /3 implies fˆn − fn∗ n ≤

  = (1 + fˆn − fn∗ n / + I(fˆn − fn∗ )/M ) . 3t 3

So then (13)

  + I(fˆn − fn∗ ). 2 2M

fˆn − fn∗ n ≤

Similarly, I(f˜n − fn∗ ) ≤ M/3 implies (14)

I(fˆn − fn∗ ) ≤

M ˆ M + fn − fn∗ n . 2 2

Inserting (14) into (13) gives fˆn − fn∗ n ≤

3 1 ˆ + fn − fn∗ n , 4 4

i.e., fˆn − fn∗ n ≤ . Similarly, Inserting (13) into (14) gives I(fˆn − fn∗ ) ≤ M . Proof of Theorem 3.1. Note first that, by the definition of of Mn , n and λn , it holds that (15)

λn,0 sn Mn1−s =

2n , 27σn2

Non-asymptotic bounds for GLM

133

and also 2(1−s)

2

(27) 2−s c− 2−s λn2−s Mn 2−s = s

(16) Define

2

2n . 27σn2

θ˜n = tθˆn + (1 − t)θn∗ ,

where

t := (1 + fθˆn − fθn∗ n /n + I(fθˆn − fθn∗ )/Mn )−1 .

We know that by convexity, and since θˆn minimizes the penalized empirical risk, we have 2

2(1−s)

Pn γfθ˜n + λn2−s I 2−s (θ˜n )     2 2 2(1−s) 2(1−s) 2−s 2−s ∗ ˆ 2−s 2−s ≤ t Pn γfθˆn + λn I (θn ) + (1 − t) Pn γfθ∗ + λn I (θn ) n

2 2−s

≤ Pn γfθ∗ + λn n

I

2(1−s) 2−s

(θn∗ ).

This can be rewritten as 2

P (γfθ˜n − γfθn ∗ ) + λn2−s I

2(1−s) 2−s

2

(θ˜n ) ≤ −(Pn − P )(γfθ˜n − γfθ∗ ) + λn2−s I n

2(1−s) 2−s

(θn∗ ).

Since I(fθ˜n − fθn∗ ) ≤ Mn , and ψk ∞ ≤ 1 (by Assumption A), we have that fθ˜n − fθn∗ ∞ ≤ Mn . Hence, by Assumption M, P (γfθ˜n − γfθn ∗ ) ≥ fθ˜n − fθn∗ 2n /σn2 . We thus obtain 2 fθ˜n − fθn∗ 2n 2(1−s) + λn2−s I 2−s (θ˜n − θn∗ ) 2 σn 2 2 f ˜ − fθ∗ 2n 2(1−s) 2(1−s) ≤ θn 2 n + λn2−s I 2−s (θ˜n ) + λn2−s I 2−s (θn∗ ) σn 2

≤ −(Pn − P )(γfθ˜n − γfθ∗ ) + 2λn2−s I

2(1−s) 2−s

n

(θn∗ ).

Now, fθ˜n − fθn∗ n ≤ n and I(θ˜n − θn∗ ) ≤ Mn . Moreover n /Mn ≤ 1 and in view of (7), n /Mn ≥ 8/m. Therefore, we have by (6) and Theorem 4.1, with probability at least  n2n 1 − exp − , 2 × (27σn2 )2 that 2 fθ˜n − fθn∗ 2n 2(1−s) + λn2−s I 2−s (θ˜n − θn∗ ) 2 σn 2

≤ λn,0 sn Mn1−s + 2λn2−s I

2(1−s) 2−s

(θn∗ ) + 2

2n 27σn2 2(1−s)

≤ λn,0 sn Mn1−s + (27) 2−s c− 2−s λn2−s Mn 2−s + s

=

1 2  , 9σn2 n

2

2n 27σn2

134

S. A. van de Geer

where in the last step, we invoked (15) and (16). It follows that n fθ˜n − fθn∗ n ≤ , 3 and also that   2(1−s) 2 2(1−s) Mn 2−s 2n − 2−s ∗ ˜ 2−s I (θn − θn ) ≤ λn ≤ , 9σn2 3 since c ≥ 3. To conclude the proof, apply Lemma 4.2.

References [1] Alexander, K. S. (1985). Rates of growth for weighted empirical processes. Proc. Berkeley Conf. in Honor of Jerzy Neyman and Jack Kiefer 2 475–493. University of California Press, Berkeley. [2] Ball, K. and Pajor, A. (1990). The entropy of convex bodies with “few” extreme points. Geometry of Banach Spaces (Strobl., 1989) 25–32. London Math. Soc. Lecture Note Ser. 158. Cambridge Univ. Press. [3] Blanchard, G., Lugosi, G. and Vayatis, N. (2003). On the rate of convergence of regularized boosting classifiers. J. Machine L. Research 4 861–894. [4] Bousquet, O. (2002). A Bennett concentration inequality and its application to suprema of empirical processes. C. R. Acad. Sci. Paris 334 495–500. [5] Ledoux, M. and Talagrand, M. (1991). Probability in Banach Spaces: Isoperimetry and Processes. Springer, New York. [6] Massart, P. (2000). About the constants in Talagrand’s concentration inequalities for empirical processes. Ann. Probab. 28 863–884. [7] Massart, P. (2000). Some applications of concentration inequalities to statistics. Ann. Fac. Sci. Toulouse 9 245–303. [8] Pollard, D. (1990). Empirical Processes: Theory and Applications. IMS, Hayward, CA. [9] Talagrand, M. (1995). Concentration of measure and isoperimetric inequalities in product spaces. Publ. Math. de l’I.H.E.S. 81 73–205. [10] van de Geer, S. (2000). Empirical Processes in M-Estimation. Cambridge Univ. Press. [11] van de Geer, S. (2002). M-estimation using penalties or sieves. J. Statist. Planning Inf. 108 55–69. [12] van de Geer, S. (2006). High-dimensional generalized linear models and the Lasso. Research Report 133, Seminar f¨ ur Statistik, ETH Z¨ urich Ann. Statist. To appear. [13] van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York.

IMS Lecture Notes–Monograph Series Asymptotics: Particles, Processes and Inverse Problems Vol. 55 (2007) 135–148 c Institute of Mathematical Statistics, 2007

DOI: 10.1214/074921707000000328

arXiv:math/0610115v2 [math.ST] 6 Sep 2007

Better Bell inequalities (passion at a distance) Richard D. Gill 1,∗,† Mathematical Institute, Leiden University and EURANDOM, NWO Abstract: I explain so-called quantum nonlocality experiments and discuss how to optimize them. Statistical tools from missing data maximum likelihood are crucial. New results are given on CGLMP, CH and ladder inequalities. Open problems are also discussed.

1. The name of the game QM vs. LR. Bell’s [5] theorem states that quantum physics (aka quantum mechanics, QM) is incompatible with classical physics. His proof exhibits a pattern of correlations, predicted in a certain situation by quantum physics, which is forbidden by any physical theory having a certain basic (and formerly uncontroversial) property called local realism (LR). Under LR, correlations must satisfy a Bell inequality, which however under QM can be violated. Local realism = locality + realism, is closely connected to causality; a precise mathematical formulation will follow later. As we will see then, a further basic (and also uncontroversial) assumption called freedom needs to be made as well. For the time being I offer the following explanatory remarks. Let us agree that the task of physics is to provide a causal explanation (or if you prefer, description) of reality. Events have causes (realism); cause and effect are constrained by time and space (locality). Realism has been taken for granted in physics since Aristotle; together with locality it has been a permanent feature and criterion of basic sanity till Einstein and others began to uncover disquieting features of quantum physics, see Einstein, Podolsky and Rosen [11], referred to hereafter as EPR. For some, John Bell’s theorem is a reason to argue that quantum physics must dramatically break down at some (laboratory accessible) level. For Bohr it would merely have confirmed the Copenhagen view that there is no underlying classical reality behind quantum physics, no Aristotelian/Cartesian/rationalist explanation of the random outcomes of quantum measurements. For others, it is a powerful incentive to deliver experimental proof that Nature herself violates local realism. ∗ This

paper is dedicated to my friend Piet Groeneboom on the occasion of his 65th birthday. I started the research during my previous affiliation at the Mathematical Institute, Utrecht University. I acknowledge financial support from the European Community project RESQ, contract IST-2001-37559. The paper is based on work in progress joint with Toni Acin, Marco Barbieri, Wim van Dam, Nicolas Gisin, Peter Gr¨ unwald, Jan-˚ Ake Larsson, Philipp Pluch, Stefan Zohren, ˙ and Marek Zukowski. Last but not least, Piet’s programming assistance was vital. Lang zal hij leven, in de gloria! † NWO is the Dutch national Science Foundation. 1 Mathematical Institute, Snellius Bldg, University of Leiden, Niels Bohrweg 1, 2333 CA Leiden, Netherlands, e-mail: [email protected]; url: http://www.math.leidenuniv.nl/∼gill AMS 2000 subject classifications: Primary 60G42, 62M07; secondary 81P68. Keywords and phrases: latent variables, missing data, quantum non-classicality, so-called quantum non-locality. 135

136

Richard D. Gill

By communis opinio, the splendid experiment of Aspect, Dalibard, and Grangier [3] settled the matter in favour of quantum physics. However, insiders have long known that that experiment has major shortcomings which imply that the matter is not settled at all. Twenty-five years later these shortcomings have still not been overcome, despite a continuing and intense effort and much progress; see Gill [14, 15], Santos [25]. I can report that certain experimenters think that a definitive successful experiment might well be achieved within ten years. A competition seems to be on to do it first. We will see. Bell-type experiments. We are going to study the sets of all possible joint probability distributions of the outcomes of a Bell-type experiment, under two sets of assumptions, corresponding respectively to local realism and to quantum physics. Bell’s theorem can be reformulated as saying that the set of LR probability laws is strictly contained in the QM set. But what is a Bell-type experiment? That is not so difficult to explain. Here is a description of a p × q × r Bell experiment, where p, q and r are fixed integers all at least equal to 2. The experiment involves a diabolical source, Lucifer, and a number p of players or parties, usually called Alice, Bob, and so on. Lucifer sends a package to Alice and each of her friends by FedEx. After the packges have been handed over by Lucifer to FedEx, but before each party’s package is delivered at his or her laboratory, each of the parties commits him or herself to using one particular tool or measurement-device out of some fixed set of toolboxes with which to open their packages. Suppose each party can choose one out of q tools; each party’s tools are labelled from 1 to q. There is no connection between different party’s tools (and it is just for simplicity that we suppose each party has the same number). The q tools of each party are conventionally called measurements or settings. When the packages arrive, each of the parties opens their own package with the measurement setting that they have chosen. What happens precisely now is left to the reader’s imagination; but we suppose that the possible outcomes for each of the parties can all be classified into one of r different outcome categories, labelled from 0 to r − 1. Again, there is not necessarily any connection between the outcome category labelled x of different measurements for the same or different parties. Given that Alice chose setting a, Bob b, and so on, there is some joint probability p(x, y, . . . |a, b, . . . ) that Alice will then observe outcome x, Bob y, . . . . We suppose that the parties chose their settings a, b, . . . , at random from some joint distribution with probabilties π(a, b, . . . ); a, b, . . . = 1, . . . , q. Altogether, one run of the whole experiment has outcome (a, b, . . . ; x, y, . . . ) with probability p(a, b, . . . ; x, y, . . . ) = π(a, b, . . . )p(x, y, . . . |a, b, . . . ). If the different party’s settings are independent, then each party would in practice generate their own setting in their own laboratory according to its marginal distribution. In general however we need a trusted, independent, referee, who we will call Piet, who generates the settings of all parties simultaneously and makes sure that each one receives their own setting in separate, sealed envelopes. One can (and should) also consider “unbalanced” experiments with possibly different numbers of measurements per party, different numbers of outcomes per party’s measurement. Moreover, more complicated multi-stage measurement strategies are sometimes considered. We stick here to the basic “balanced” designs, just for ease of exposition. The classical polytope. Local realism and freedom can be taken mean the following:

Better Bell inequalities

137

Measurements which were not done also have outcomes; actual and potential measurement outcomes are independent of the measurement settings actually used by all the parties.

The outcomes of measurements which were not actually done are obviously counterfactual. I am not claiming the actual existence in physical reality of these outcomes, whatever that might be supposed to mean (see EPR for one possible definition). I am supposing that a mathematical model for the experiment does allow the existence of such variables. To argue this point, consider a computer simulation of the Bell experiment in which Lucifer’s packages are put together on a classical computer, using randomization if necessary, while what goes on in each party’s laboratory is also simulated on a computer. The package that is sent to each party can therefore be represented by a random number. What happens in each party’s lab is the result of inputting the message from Lucifer, and the setting from Piet the referee, into another computer program which might also make use of random number generation. There can be any kind of dependence between the random numbers used in Lucifer’s, Alice’s, Bob’s . . . computers. But without loss of generality all this randomization might as well be done at Lucifer’s computer; Alice’s computer merely evaluates some function of the message from Lucifer, and the setting from Piet. We see that the outcomes are now simultaneously defined of every measurement which each party might choose, simply by considering all possible arguments to their computers programs. The assumption of freedom is simply that Piet’s settings are independent of Lucifer’s random numbers. Now, given Lucifer’s randomization, everything that happens is completely deterministic: the outcome of each possible measurement of each party is fixed. For ease of notation, consider briefly a two party experiment. Let X1 , . . . , Xq and Y1 , . . . , Yq denote the counterfactual outcomes of each of Alice’s and Bob’s possible q measurements (taking values in {0, . . . , r − 1}. We may think of these in statistical terms as missing data, in physical terms as so-called hidden variables. Denote by A and B Alice’s and Bob’s random settings, each taking values in {1, . . . , q}. The actual outcomes observed by Alice and Bob are therefore X = XA and Y = YB . The data coming from one run of the experiment, A, B, X, Y , has joint probability distribution with mass function p(a, b; x, y) = π(a, b, . . . )p(x, y, |a, b) = π(a, b) Pr(Xa = x, Yb = y). Now the joint probability distribution of the Xa and Yb can be arbitrary, but in any case it is a mixture of all possible degenerate distributions of these variables. Consequently, for fixed setting distribution π, the joint distribution of A, B, X, Y is also a mixture of the possible distributions corresponding to degenerate (deterministic) hidden variables. Since there are only finitely many degenerate distributions when p, q and r are all fixed, we see that Under local realism and freedom, the joint probability laws of the observable data lie in a convex polytope, whose vertices correspond to degenerate hidden variables.

We call this polytope the classical polytope. The quantum body. Introductions to quantum statistics can be found in Gill [13], Barndorff-Nielsen et al. [4]. The bible of quantum information, Nielsen and Chuang [22], is a splendid resource and has introductory material for beginners to the field whether coming from physics, computer science or mathematics. The basic rule for computation of a probability distribution in quantum mechanics is called Born’s law: take the squared lengths of the projections of the state vector

138

Richard D. Gill

into a collection of orthogonal subspaces corresponding to the different possible outcomes. For ease of notation, consider a two-party experiment. Take two complex Hilbert spaces H and K. Take a unit vector |ψi in H ⊗ K. For each a, let Lax , x = 0, . . . , r − 1, denote orthogonal closed subspaces of H, together spanning all of H. Similarly, let Myb denote the elements of q collections of decompositions of K into orthogonal subspaces. Finally, define p(x, y|a, b) = kΠLax ⊗ ΠMyb |ψik2 , where Π denotes orthogonal projection into a closed subspace. The reader should verify (basically by Pythagoras’ theorem), that this does define a collection of joint probability distributions of X and Y , indexed by (a, b). As before we take p(a, b, . . . ; x, y, . . . ) = π(a, b, . . . )p(x, y, . . . |a, b, . . . ). The following fact is not trivial: The collection of all possible quantum probability laws of A, B, X, Y (for fixed setting distribution π) forms a closed convex body containing the local polytope.

Beyond the 2 × 2 × 2 case very little indeed is known about this convex body. The no-signalling polytope. The two convex bodies so far defined are forced to live in a lower dimensional affine P subspace, by the basic normalization properties of probability distributions: x,y p(a, b; x, y) = π(a, b) for all a, b. Moreover, probabilities are necessarily nonnegative, so this restricts us further to some convex polytope. However, physics (locality) implies another collection of equality constraints, putting us into a still P smaller affine subspace. These constraints are called the no-signalling constraints: y p(a, b; x, y) should be independent of b for each a and x, and vice versa. It is easy to check that both the local realist probability laws, and the quantum probability laws, satisfy no-signalling. Quantum mechanics is certainly a local theory as far as manifest (as opposed to hidden) variables are concerned. The set of probability laws satisfying no-signalling is therefore another convex polytope in a low dimensional affine subspace; it contains the quantum body, which in turn contains the classical polytope.

Bell and Tsirelson inequalities. “Interesting” faces of the classical polypope, i.e., faces which do not correspond to the positivity constraints, generate (generalized) Bell inequalities, that is, linear combinations of the joint probabilities of the observable variables which reach a maximum value at the face. Similarly, “interesting” supporting hyperplanes to the quantum body correspond to (generalized) Tsirelson inequalities. These latter inequalities can be recast as inequalities concerning expectation values of certain observables called Bell operators. The original Bell (more precisely, CHSH – Clauser, Horne, Shimony and Holt [6]) and Cirel’son [8] inequalities concern the 2 × 2 × 2 case. However we will proceed by proving Bell’s theorem – the quantum body is strictly larger than the local polytope – in the 3 × 2 × 2 case for which a rather elegant proof is available due to Greenberger, Horne and Zeilinger [17]. By the way, the subtitle “passion at a distance” is a phrase coined by Abner Shimony and it expresses that though there is no action at a distance (no manifest non-locality), still quantum physics seems to allow the physical system at Alice’s site to have some feeling for what is going on far away at Bob’s. Rather like the oracles of antiquity, no-one can make any sense of what the oracle is saying till it is too late . . . . But one can use these non-classical correlations, as the physicists like to call them, to enable Alice and her friends to succeed at certain collaborative tasks, in which Lucifer is their ally while Piet is their adversary, with larger probability

Better Bell inequalities

139

than is possible under any possible classical-like physics. The following example should inspire the reader to imagine such a task. GHZ paradox. We consider a now famous 3 × 2 × 2 example due to Greenberger, Horne and Zeillinger [17]. We use this example partly for fun, partly to exemplify the computation of Bell probability laws under quantum mechanics and under local realism. Firstly, under local realism, one can introduce hidden variables X1 , X2 , Y1 , Y2 , Z1 , Z2 , standing for the counterfactual outcomes of Alice, Bob and Claudia’s measurements when assigned settings 1 or 2 by Piet. These variables are binary, and we may as well denote their possible outcomes by ±1. Now note that (X1 Y2 Z2 ).(X2 Y1 Z2 ).(X2 Y2 Z1 ) = (X1 Y1 Z1 ). Thus, if the setting patterns (1, 2, 2), (2, 1, 2) and (2, 2, 1) always result in X, Y and Z with XY Z = +1, it will also be the case the setting pattern (1, 1, 1) always results in X, Y and Z with XY Z = +1. Next define the 2 × 2 matrices     0 1 1 0 σ1 = , σ2 = . 1 0 0 −1 One easily checks that σ1 σ2 = −σ2 σ1 , (anticommutation), σ12 = σ22 = 1, the 2 × 2 identity matrix. Since σ1 and σ2 are both Hermitean, it follows that they have real eigenvalues, which by the properties given above, must be ±1. Now define matrices X1 = σ1 ⊗ 1 ⊗ 1, X2 = σ2 ⊗ 1 ⊗ 1, Y1 = 1 ⊗ σ1 ⊗ 1, Y2 = 1 ⊗ σ2 ⊗ 1, Z1 = 1 ⊗ 1 ⊗ σ1 , Z2 = 1 ⊗ 1 ⊗ σ2 . It is now easy to check that (X1 Y2 Z2 ).(X2 Y1 Z2 ).(X2 Y2 Z1 ) = −(X1 Y1 Z1 ), and that (X1 Y2 Z2 ), (X2 Y1 Z2 ), (X2 Y2 Z1 ) and (X1 Y1 Z1 ) commute with one another. Since these four 8 × 8 Hermitean matrices commute they can be simultaneously diagonalized. Some further elementary considerations lead one to conclude the existence of a simultaneous eigenvector |ψi of all four, with eigenvalues +1, +1, +1, −1 respectively. We take this to be the state |ψi, with the three Hilbert spaces all equal to C2 . We take the two orthogonal subspaces for the 1 and 2 measurements of Alice, Bob, and Claudia all to be the two eigenspaces of σ1 and σ2 respectively. This generates quantum probabilties such that the setting patterns (1, 2, 2), (2, 1, 2) and (2, 2, 1) always result in X, Y and Z with XY Z = +1, while the setting pattern (1, 1, 1) always results in X, Y and Z with XY Z = −1. Thus we have shown that a vector of quantum probabilities exists, which cannot possibly occur under local realism. Since the classical polytope is closed, the corresponding quantum law must be strictly outside the classical polytope. It therefore violates a generalized Bell inequality corresponding to some face of the classical polytope, outside of which it must lie. It is left as an exercise to the reader to generate the corresponding “GHZ inequality.” GHZ experiment. This brings me to the point of the paper: how should one design good Bell experiments; and what is the connection of all this physics with mathematical statistics? Indeed there are many connections – as already alluded to, the hidden variables of a local realist theory are simply the missing data of a nonparametric missing data problem.

140

Richard D. Gill

In the laboratory one creates the state |ψi, replacing Lucifer by a source of entangled photons, and the measurement devices of Alice and Bob by assemblages of polarization filters, beam splitters and photodetectors implementing hereby the measurements corresponding to the subspaces Lxa , etc. One also settles on a joint setting probability π. One repeats the experiment many times, hoping to indeed observe a quantum probability law lying outside the classical polytope, i.e., violating a Bell inequality. The famous Aspect et al. [3] experiment implemented this program in the 2 × 2 × 2 case, violating the so-called CHSH inequality (which we will describe later) by a large number of standard deviations. What is being done here is statistical hypothesis testing, where the null hypotheses is local realism, the alternative is quantum mechanics; the alternative being true by design of the experimenter and validity of quantum mechanics. Dirk Bouwmeester recently carried out the GHZ experiment; the results are exciting enough to be published in Nature (Pan et al. [23]). He claimed in a newspaper interview that this experiment is of a rather special type: only a finite number of repetitions are necessary since the experiment exhibits events which are impossible under classical physics, but certain under quantum mechanics. However please note that the events which are certain or impossible, are only certain or impossible conditional on some other events being certain. Since the experiment is not perfect, Bouwmeester did observe some “wrong” outcome patterns, thereby destroying by his own logic the conclusion of his experiment. Fortunately his data does statistically significantly violate the accompanying GHZ inequality and publication in Nature was justified! The point is: all these experiments are statistical in nature; they do not prove for sure that local realism is false; they only give statistical evidence for this proposition; evidence which does become overwhelming if N , the number of repetitions, is large enough. How to compare different experiments. Because of the dramatic zero-one nature of the GHZ experiment, it is felt by many physicists to be much stronger or better than experiments of the original 2 × 2 × 2 CHSH type (still to be elucidated!) The original aim of the research described here was to supply objective and quantitative evaluation of such claims. Now the geometric picture above naturally leads one to prefer an experiment where the distance from the quantum physical reality is as far as possible from the nearest local realistic or classical description. Much research has been done by physicists focussing on the corresponding Euclidean distance. However, it is not so clear what this distance means operationally, and whether it is comparable over experiments of different types. Moreover the Euclidean distance is altered by taking different setting distributions π (though physicists usually only consider the uniform distribution). It is true that Euclidean distance is closely related to noise resistance, a kind of robustness to experimental imperfection. As one mixes the quantum probability distribution more and more with completely random, uniform outomes, corresponding to pure noise in the photodetectors, the quantum probability distribution shrinks towards the center of the classical polytope, at some point passing through one of its faces. The amount of noise which can be allowed while still admitting violation of local realism is directly related to Euclidean distance, in our picture. Van unwald [10] however propose to use relative entropy, D(q : PDam, Gill and Gr¨ p) = abxy q(abxy) log2 (q(abxy)/p(abxy)), where q now stands for the “true” probability distribution under some quantum description of reality, and p stands for a local realist probability distribution. Their program is to evaluate supq inf p D(q : p)

Better Bell inequalities

141

where the supremum is taken over parameters at the disposal of the experimenter (the quantum state |ψi, the measurement projectors, the setting distribution π; while the infimum is taken over probability distributions of outcomes given settings allowed by local realism (thus q and p in supremum and infimum actually stand for something different from the probability laws q and p lying in the quantum body and classical polytope respectively; hopefully this abuse of notation may be excused. They argue that this relative entropy gives direct information about the number of trials of the experiment required to give a desired level of confidence in the conclusion of the experiment. Two experiements which differ by a factor 2 are such that the one with the smaller divergence needs to be repeated twice as often as the other in order to give an equally convincing rejection of local realism. Moreover, optimizing over different sets of quantum parameters leads to various measures of “strength of non-locality.” For instance, one can ask what is the best experiment based on a given entangled state |ψi? Experiments of different format can be compared with one another, possibly discounting the relative entropies according to the numbers of quantum systems involved in the different experiments in the obvious way (typically, a p party experiment involves generation of p particles at a time, so a four party experiment should be downweighted by a factor 2 when comparing with a two party experiment). We will give some examples later. Finally, that paper showed how the interior infimum is basically the computation of a nonparametric maximum likelihood estimator in a missing data problem. Various algorithms from statistics can be succesfully applied here, in numerical rather than analytical experimentation; and progams developed by Piet Groeneboom (see Groeneboom et al. [18]) played a vital role in obtaining the results which we are now going to display. 2. CHSH and CGLMP The 2×2×2 case is particularly simple and well researched. In a later section, I want to compare the corresponding two particle CHSH experiment with the three particle GHZ. In another section I will discuss properties of 2 × 2 × d experiments, which form a natural generalization of CHSH and have received much attention both by theorists and experimenters in recent years. We will see that many open problems exist here and some remarkable conjectures can be posed. Preparatory to that, I will therefore now describe the so-called CGLPM inequality, the generalization from 2 × 2 × 2 to 2 × 2 × d of CHSH. For the 2×2×d case an important step was made by Collins, Gisin, Linden, Massar and Popescu [9], in the discovery of a generalized Bell inequality (i.e., interesting face of the classical polytope), together with a quantum state and measurements which violated the inequality. The original specification of the inequality is rather complex, and its derivation also took two closely printed pages. Here I offer a new and extremely short derivation of an equivalent inequality, found very recently by Stefan Zohren, which further simplifyies an already very simple version of my own. Proof of equivalence with the original CGLMP is tedious! P Recall that a Bell inequality is the face of a classical polytope of the form abxy cabxy p(abxy) ≤ C. Now since we are only concerned with probability distributions within the no-signalling polytope, the probabilities p(abxy) necessarily satisfy a large number of equality constraints (normalization, no-signalling), which allows one to rewrite the Bell inequality in many different forms; sometimes remarkably different. A canonical form can be obtained by removing, by appropriate

142

Richard D. Gill

substitutions, all p(abxy) with x and y equal to one particular value from the set of possible outcomes, e.g., outcome 0, and involving also the marginals p(ax) and p(by) with x and y non zero. This is not necessarily the “nicest” form of an inequality. However, in the canonical form the constant C does disappear (becomes equal to 0). To return to CGLMP: consider four random variables X1 , X2 , Y1 , Y2 . Note that X1 < Y2 and Y2 < X2 and X2 < Y1 implies X1 < Y1 . Consequently, X < 1 ≥ Y1 implies X1 ≥ Y2 or Y2 ≥ X2 or X2 ≥ Y1 , and this gives us Pr(X1 ≥ Y1 ) ≤ Pr(X1 ≥ Y2 ) + Pr(Y2 ≥ X2 ) + Pr(X2 ≥ Y1 ). This is a CGLMP inequality, when we further demand that all four variables take values in {0, . . . , d − 1}. The case d = 2 gives the CHSH inequality (though also in an unfamiliar form). CGLMP describe a state and quantum measurements which generate probabilities, which violate this inequality. Take Alice √ Hilbert space each to be P and Bob’s d-dimensional. Consider the states |ψi = d−1 x=0 |xxi/ d, where |xxi = |xi ⊗ |xi, and |xi for x = 0, . . . , d − 1 is an orthonormal basis of Cd . Alice and Bob’s settings 1, 2 are taken to correspond to angles α1 = 0, α2 = π/4, and β1 = π/8, β2 = −π/8. When Alice or Bob receives setting a or b, each applies the diagonal unitary operation with diagonal elements exp(ixθ/d), x = 0, . . . , d − 1, to their part of the quantum system, where θ stands for their own angle (setting). Next Alice applies the quantum Fourier transform Q to her part, and Bob its inverse (and adjoint) Q∗ ; Qxy = exp(ixy/d), Q∗xy = exp(−ixy/d). Finally Alice and Bob “measure in the computational basis”, i.e., projecting onto the one-dimensional subspaces corresponding to the bases |xi, |yi. Applying a unitary U and then measuring the projector ΠM is of course the same as measuring the projector ΠU ∗ M ; with a view to implementation in the laboratory it is very convenient to see the different measurements as actually “the same measurement” applied after different unitary transformations of each party’s state have been applied. In quantum optics these operations might correspond to use of various crystals, applying an electomagnetic field across a light pulse, and so on. That these choices gives a violation of a CGLMP inequality follows from some computation and we desperately need to understand what is going on here, as will become more obvious in a later section when I describe conjectures concerning CGLMP and these measurements. 3. Comparing some classical experiments: GHZ vs CHSH First of all, let me briefly report some results from van Dam et al. [10] concerning the comparison of CHSH and GHZ. It is conjectured, and supported numerically, but not yet proved, that the best 2 × 2 × 2 experiment in the sense of Kullback-Leibler divergence is the CGLMP experiment with d = 2 described in the last section, and usually known as the CHSH experiment. The setting probabilities should be uniform, the state is maximally entangled, the measurements are those implemented by Aspect et al. It turns out that D is equal to 0.0423.... For GHZ, which is can be conjectured to be the best 3 × 2 × 2 experiment, one finds D = 0.400, with setting probabilities uniform over the four setting patterns involved in the derivation of the paradox; zero on the other. So this experiment is apparently almost 10 times better. By the way, D = 1 would be the strength of the experiment when one repeatedly throws a coin which always comes up heads, in order to disprove the theory that

Better Bell inequalities

143

Pr(heads) = 1/2. So GHZ is less than half as good as an experiment in which one compares probabilities 1 and 1/2; let alone comparable to an experiment comparing impossible with certain outcomes! However in practice the GHZ experiment is not performed exactly in optimal fashion. To begin with, in order to produce each triple of photons, Bouwmeester generated two maximally entangled pairs of photons, measured the polarization of one of the four, and accepted the remaining set of three when the measured polarization was favourable, which occurs in half of the times. Since we need two pairs of photons for each triple, and discard the result half the times, the figure of merit should be divided by four. Next, the optimal setting probabilities is uniform over half of the eight possible combinations. In practice one generates settings at random at each measurement station, so that half of the combinations are actually useless. This means we have to halve again, resulting in a figure of merit for GHZ which is barely better than CHSH, and very far from the “infinity” which would correspond to an all or nothing experiment. Actually things are even worse since the pairs of photon pairs are generated at random times and one has to be quite lucky to have two pairs generated close enough in time to one another that one has four photons to start with. Then there are the inevitable losses which further degrade the experiment . . . (more on this later). Bouwmeester needs to carry on measuring for hours in order to achieve what can be done with CHSH in minutes. Which is not to say that his experiment is not a splendid acheivement! 4. CGLMP as # outcomes goes to infinity In Acin, Gill and Gisin [2] a start is made with studying optimal 2 × 2 × r experiments, and some remarkable findings were made, though almost all conclusions depend on numerics, and even on numerics depending on conjectures. Let me first describe one rather fundamental conjecture whose truth would take us a long way in understanding what is going on. In general nothing is known about the geometry of the classical polytope. An impossible open problem is to somehow classify all interesting faces. It is not even known if, in general, all faces which are not trivial (i.e., correspond to nonnegativity constraints) are “interesting” in the sense of being violable by quantum mechanics. As the numbers grow, the number and type of faces grow explosively, and exhausitive enumeration has only been done for very small numbers. Clearly there are many many symmetries — the labelling of parties, measurements and outcomes is completely arbitrary. Moreover, there are three ways in which inequalities for smaller experiments remain inequalities for larger. Firstly, by merging categories in the larger experiment one obtains a smaller one, and the Bell inequalities for the smaller can be lifted to the larger. Next, by simply omitting measurements one can lift Bell inequalities for smaller experiments to larger. Finally, by conditioning on a particular outcome of a particular measurement of a particular party, one reduces a larger experiment to one with less parties, and conversely can lift a smaller inequality to a larger. With the understanding that interesting faces for smaller polytopes can be lifted to interesting faces of larger in three different ways, the following conjecture seems highly plausible: All the faces of the 2 × 2 × r polytope are boring (nonnegativity) or interesting CGLMP, or lifted CGLMP, inequalities.

144

Richard D. Gill

This is certainly true for r = 2, 3, 4 and 5 but beyond this there is only numerical evidence: numerical search for optimal experiments using the maximallly entangled state |ψi has only uncovered the CGLMP measurements, violating the CGLMP inequality. Moreover this is true both using Euclidean and relative entropy distances. The next, stunning, finding is that the best state for these experiments P is not the maximally entangled state at all! Rather, it is a state of the form x cx |xxi where the so-called Schmidt coefficients cx are symmetric around x = (r−1)/2, first decreasing and then increasing. This “U-shape” become more and more pronounced as r increases. Moreover the shape is found for both figures of merit, though it is a different state for the two cases (even less entangled for divergence than for Euclidean, i.e., less entangled for statistical strength than for noise resistance). Rather thorough numerical search takes us up to about r = 20 and has been replicated by various researchers. Taking as a conjecture a) that all faces are CGLMP, b) that the best measurements are also CGLMP and the state is U -shaped, we only need to optimize over the Schmidt coeffficients cx . Numerically one can quite easily get up to about r = 1000 in this way. However with some tricks one can go to r = 10 000 or even 100 000. Note that we are solving supq inf p D(q : p) where the infimum is over the local realist polytope, the supremum is just over the cj . Now a solution must also be a stationary point for both optimizations. Differentiating with respect to the classical parameters, and recalling the form of D, one finds that one must have P qabxy /ˆ pabxy )(pabxy − pˆabxy ) = 0 for classical probabilities p on the face of abxy (ˆ the classical polytope passing through the solution pˆ. But this face is a CGLMP inequality! Hence the coefficients, qˆabxy /ˆ pabxy are the coefficients involved in this inequality, i.e., up to some normalization constants they P are already known! Howqabxy /ˆ pabxy ) ever, the quantity we want to optimize, D itself, is abxy qabxy log2 (ˆ and this is optimal over q at q = qˆ (i.e., this the accompanying Tsirelson inequality, or supporting hyperplane to the quantum body at the optimum). Since the terms in the logarithm are known (up to a normalization constant) we just have to optimize the mean of an almost known Bell operator over the state. This is a largest eigenvalue problem, numerically easy up to very very large d. All this raises the question what happens when r → ∞. In particular, can one attain the largest conceivable violation of CGLMP, namely when the probability on the left is 1 and the three on the right are all 0, with infinite dimensional Hilbert spaces, and if so, are the corresponding state and measurements interesting and feasible experimentally? Strongly positive evidence and further conjectures are given in Zohren and Gill [27]. Some recent numerical results on r = 3 and 4 are given by Navascues et al. [21]. We think of this conjectured “perfect passion at a distance” as the optimal solution of a variant of the infamous game of Polish Poker (played in Russian bars between a Polish traveller and local Russian drinkers with the inevitable outcome that the Pole always gets the Roubles...). Now, Alice and Bob are playing together, against Piet. Piet chooses (completely randomly) a “setting” a = 1, 2 for Alice, and b = 1, 2 for Bob. Alice doesn’t know Bob’s setting and vice versa. Alice and Bob must now, separately, each think of a number. Denote Alice’s number by xa , Bob’s by yb . Alice and Bob’s aim is to attain x1 < y2 (if Piet calls “1; 2”), and y2 < x2 (if Piet calls “2; 2”), and x2 < y1 (if ...), and y1 < x1 (if ...). If they choose their numbers by any classical means, e.g., with classical dice, they must fail at least a quarter of the times. However, with quantum dice (i.e., with the help of a couple of bundles of photons, donated to each of them in advance by Lucifer) they can

Better Bell inequalities

145

succeed with probability arbitrarily close to certainty, by taking measurements with enough outcomes. At least, according to Zohren and Gill’s conjecture... There remains the question: why are the CGLMP measurements optimal for the CGLMP inequality? Where do these angles come from, what has this to do with QFT? There are some ideas about this and the problem seems ripe to be cracked. 5. Ladder proofs Is the CHSH experiment the best possible experiment with two maximally entangled qubits? This seemed a very good conjecture till quite recently. However the conjecture certainly needs modification now, as I will explain. There has been some interest recently in so-called ladder proofs of Bell’s theorem. These appear to allow one to use less entangled states and get better experiments, though that dream is shown to be fallacious when one uses statistical strength as a figure of merit rather than a criterion connected to “probability zero under LR, but positive under QM” (conditional on certain other probabilities equal to zero). Exactly as for GHZ, the size of this positive probability is not very important, the experiment is about violating an inequality, not about showing that some probability is positive when it should be zero. Let me explain the ladder idea. Consider the inequality Pr(X1 ≥ Y1 ) ≤ Pr(X1 ≥ Y2 ) + Pr(Y2 ≥ X2 ) + Pr(X2 ≥ Y1 ). Now add to this the same inequality for another pair of hidden variables: Pr(X2 ≥ Y2 ) ≤ Pr(X2 ≥ Y3 ) + Pr(Y3 ≥ X3 ) + Pr(X3 ≥ Y2 ). The intermediate “horizontal” 2—2 term cancels and we are left only with cross terms 1—2 and 2—3, and “end” terms 1—1 and 3—3. With a ladder built from adding four inequalities involving X1 to X5 and Y1 to Y5 , out of the 25 possible comparisons, only the two end horizontal terms and eight crossing terms survive, 10 out of the total. Numerical optimization of D for longer and longer ladders, shows that actually the optimal state is always the maximally entangled state. Moreover, much to my surprise, the best D is obtained with the ladder of X1 to X5 and Y1 to Y5 , and it is much better than the original CHSH! However, it has a uniform distribution over 10 out of 25 combinations. If one would implement the same experiment with the uniform distribution over all 25, it becomes worse that CHSH. So the new conjecture is that CHSH is the optimal 2 × 2 × 2 experiment with uncorrelated settings. These findings come from new unpublished work with Marco Barbieri; we are thinking of actually doing this experiment. 6. CH for Bell In a CHSH experiment an annoying feature is that some photons are not registered at all. This means that there are really three outcomes of each measurement, with a third outcome “no photon”; however, the outcome “no photon, no photon” is not observed at all. One has a random sample size from the conditional distribution given that there is an event in at least one of the two laboratories of Alice and Bob. It is better to realise that the original, complete sample size is actually also random, and typically Poisson, hence the observed counts of the various events are

146

Richard D. Gill

all Poisson. But can we create useful Bell inequalities for this situation? The answer is yes, using the possibility of reparametrization of inequalities using the equality constraints. In a 2×2×3 experiment one can rewrite any Bell inequality as an inequality involving only the pabxy with one of x or y not zero, as well as the marginal probabilities pax , pby with x and y nonzero. The constant term in the inequality becomes 0. So one gets a linear inequality involving only observed, Poisson distributed, random variables. “Poisson statistics” allows one to supply a valid standard error even though the “total sample size” was unknown. Applying this technique in the 2 × 2 × 2 case gives a known inequality, the Clauser-Horne (CH) inequality, useful when one has binary outcomes but one of the two outcomes is not observable at all; i.e., the outcomes are “detector click” and “no detector click.” How to find a good inequality for 2 × 2 × 3? I simply add a certain probability of “no event”, independent on both sides of the experiment, to the quantum probabilities belonging to the classical CHSH set-up. Next I solve the problem inf p D(q : p) using Piet Groeneboom’s programs. I observe the values of q/ˆ p which define the face of the local polytope closest to q. I rewrite the inequality in its classical form. The result is a new inequality (not quite new: Stefano Pironio informs me it is known to N. Gisin and others) which takes account of “no event” and which is linear in the observed counts. The linearity means that the inequality can be studied using martingale techniques to show that the experiment is “insured” against time dependence and time trends, as long as the settings are chosen randomly; cf. Gill [14, 15]. It turns out to be essentially equivalent to some rather non-linear inequalities developed by Jan˚ Ake Larsson, see Larsson and Gill [20], which were till now the only known way to deal with “non-events.” We intend to pursue this development in the near future combining treatment of the detection, coincidence and memory loopholes (Gill [16] and Larsson and Gill [20]). 7. Conclusions I did not yet mention that studying the boundary of the 2×2×2 quantum body and some different generalizations led Tsirelson into some deep mathematics and connections with fundamental questions involving Grothendieck’s mysterious constant, see Cirel’son [8], Tsirelson [26] (the same person . . . ), Reeds [24], and Fishburn and Reeds [12]. Bell experiments offer a rich field involving many statistical ideas, beautiful mathematics, and offering deep exciting challenges. Moreover it is a hot topic in quantum information and quantum optics. Much remains to be done. One remains wondering why nature is like this? There are two ways nature uses to generate probabilities: one is to take a line segment of length one and cut it in two. The different experiments found by cutting it at different places are compatible with one another; one sample space will do (the unit interval). The other way of nature is to take a line segment of length one, and let it be the hypothenuse of a right angled triangle. Now the squares of the other two sides are probabilities adding to one. The different experiments are not compatible with one another (at least, in dimension three or more, according to the Kochen–Specker theorem). According to quantum mechanics and Bell’s theorem, the world is completely different from how it has been thought for two thousand years of Western science. As Vovk and Shafer recently argued, Kolmogorov was one of the first to take the radical step of associating the little omega of a probability space with the outcome

Better Bell inequalities

147

and not the hidden cause. Before then, all probability in physics could be traced back to uncertainty in initial conditions. Going back far enough, one could invoke symmetry to reduce the situation to “equally likely elementary outcomes.” Or more subtly, sufficient chaoticity ensures that mixed up distributions are invariant under symmetries and hence uniform. At this stage, frequentists and Bayesians use the same probabilities and get the same answers, even if they interpret their probabilities differently. According to Bell’s theorem, the randomness of quantum mechanics is truly ontological and not epistemological: it cannot be traced back to ignorance but is “for real.” It is curious that the quantum physics community is currently falling under the thrall of Bayesian ideas even though their science should be telling them that the probabilities are objective. Of course, one can mix subjective uncertainties with objective quantum probabilities, but to my mind this is dissolving the baby in the bathwater, not an attractive thing to do. Still, why is nature like this, why are the probabilities what they are? My rough feeling is as follows. Reality is discrete. Hence nature cannot be continuous. However we do observe symmetries under continuous groups (rotations, shifts); the only way to accomodate this is to make nature random, and to have the probabiltiy distributions continuous, or even covariant, with the groups. Current research in the foundations of quantum mechanics (e.g., by Inge Helland) points to the conclusions that symmetry forces the shape of the probabilities (and even forces the complex Hilbert space); just as in the Aristotelian case, but at a much deeper level, probabilities are objectively fixed by symmetries. References [1] Acin, A., Gisin, N. and Toner, B. (2006). Grothendieck’s constant and local models for noisy entangled quantum states. Phys. Rev. A 73 062105 (5 pp.). arxiv:quant-ph/0606138. MR2244753 [2] Acin, A., Gill, R. D. and Gisin, N. (2005). Optimal Bell tests do not require maximally entangled states. Phys. Rev. Lett. 95 210402 (4 pp.). arxiv:quant-ph/0506225. [3] Aspect, A., Dalibard, J. and Roger, G. (1982). Experimental test of Bell’s inequalities using time-varying analysers. Phys. Rev. Lett. 49 1804–1807. MR0687359 [4] Barndorff-Nielsen, O. E., Gill, R. D. and Jupp, P. E. (2003). On quantum statistical inference (with discussion). J. R. Statist. Soc. B 65 775– 816. arxiv:quant-ph/0307191. MR2017871 [5] Bell, J. S. (1964). On the Einstein Podolsky Rosen paradox. Physics 1 195– 200. [6] Clauser, J. F., Horne, M. A., Shimony, A. and Holt, R. A. (1969). Proposed experiment to test local hidden-variable theories. Phys. Rev. Lett. 23 880–884. [7] Clauser, J. F. and Horne, M. A. (1974). Experimental consequences of objective local theories. Phys. Rev. D 10 526–35. [8] Cirel’son, B. S. (1980). Quantum generalizations of Bell’s inequality. Lett. Math. Phys. 4 93–100. MR0577178 [9] Collins, D., Gisin, N., Linden, N., Massar, S. and Popescu, S. (2002). Bell inequalities for arbitrarily high dimensional systems. Phys. Rev. Lett. 88 040404 (4 pp.). arxiv:quant-ph/0106024. MR1884489

148

Richard D. Gill

¨nwald, P. D. (2005). The statisti[10] van Dam, W., Gill, R. D. and Gru cal strength of nonlocality proofs. IEEE Trans. Inf. Theory 51 2812–2835. arxiv:quant-ph/0307125. MR2236249 [11] Einstein, A., Podolsky, B. and Rosen, N. (1935). Can quantummechanical description of physical reality be considered complete? Phys. Rev. 47 777–780. [12] Fishburn, P. C. and Reeds, J. A. (1994). Bell inequalities, Grothendieck’s constant, and root two. SIAM J. Discr. Math. 7 48–56. MR1259009 [13] Gill, R. D. (2001). Teleportation into quantum statistics. J. Korean Statist. Soc. 30 291–325. arxiv:math.ST/0405572. MR1892211 [14] Gill, R. D. (2003a). Time, finite statistics, and Bell’s fifth position. In Foundations of Probability and Physics 2 (V¨ axj¨ o, 2002). Math. Model. Phys. Eng. Cogn. Sci. 5 179–206. V¨ axj¨ o Univ. Press, V¨ axj¨ o. arxiv:quant-ph/0301059. MR2039718 [15] Gill, R. D. (2003b). Accardi contra Bell (cum mundi): The impossible coupling. In Mathematical Statistics and Applications: Festschrift for Constance van Eeden (M. Moore, S. Froda, and C. L´eger, eds.). IMS Lecture Notes – Monographs 42 133–154. Institute of Mathematical Statistics, Beachwood, OH. arxiv:quant-ph/0110137. MR2138290 [16] Gill, R. D. (2005). The chaotic chameleon. In Quantum Probability and Infinite Dimensional Analysis: from Foundations to Applications (M. Sch¨ urmann and U. Franz, eds.) QP–PQ: Quantum Probability and White Noise Analysis 18 269–276. World Scientific, Singapore. arxiv:quant-ph/0307217. MR2212455 [17] Greenberger, D. M., Horne, M. and Zeilinger, A. (1989). Going beyond Bell’s theorem. In Bell’s Theorem, Quantum Theory, and Conceptions of the Universe, (M. Kafatos, ed.) 73–76. Kluwer, Dordrecht. [18] Groeneboom, P., Jongbloed, G. and Wellner, J. A. (2003). Vertex direction algorithms for computing nonparametric function estimates in mixture models. arxiv:math.ST/0405511. [19] Hardy, L. (1993). Nonlocality for two particles without inequalities for almost all entangled states. Phys. Rev. Lett. 71 1665–1668. MR1234454 [20] Larsson, J.-˚ A. and Gill, R. D. (2004). Bell’s inequality and the coincidence-time loophole. Europhys. Lett. 67 707–713. arxiv:quant-ph/0312035. MR2172249 [21] Navascues, M., Pironio, S. and Acin, A. (2006). Bounding the set of quantum correlations. arxiv:quant-ph/0607119. [22] Nielsen, M. A. and Chuang, I. L. (2000). Quantum Computation and Quantum Information. Cambridge University Press, New York. MR1796805 [23] Pan, J. W., Bouwmeester, D., Daniell, M., Weinfurter, H. and Zeilinger, A. (2000). Experimental test of quantum nonlocality in threephoton Greenberger–Horne–Zeilinger entanglement. Nature 403 (6769) 515– 519. [24] Reeds, J. A. (1991). A new lower bound on the real Grothendieck constant. Available at http://www.dtc.umn.edu/∼reedsj/bound2.dvi. [25] Santos, E. (2005). Bell’s theorem and the experiments: Increasing empirical support to local realism. Studies In History and Philosophy of Modern Physics 36 544–565. arxiv:quant-ph/0410193. MR2175810 [26] Tsirelson, B. S. (1993). Some results and problems on quantum Belltype inequalities. Hadronic Journal Supplement 8 329–345. Available at http://www.tau.ac.il/∼tsirel/download/hadron.html. MR1254597 [27] Zohren, S. and Gill, R. D. (2006). On the maximal violation of the

Better Bell inequalities

149

CGLMP inequality for infinite dimensional states. Phys. Rev. Lett. To appear. arxiv:quant-ph/0612020.

IMS Lecture Notes–Monograph Series Asymptotics: Particles, Processes and Inverse Problems Vol. 55 (2007) 149–166 c Institute of Mathematical Statistics, 2007
DOI: 10.1214/074921707000000337

Asymptotic oracle properties of SCAD-penalized least squares estimators Jian Huang1 and Huiliang Xie University of Iowa Abstract: We study the asymptotic properties of the SCAD-penalized least squares estimator in sparse, high-dimensional, linear regression models when the number of covariates may increase with the sample size. We are particularly interested in the use of this estimator for simultaneous variable selection and estimation. We show that under appropriate conditions, the SCAD-penalized least squares estimator is consistent for variable selection and that the estimators of nonzero coefficients have the same asymptotic distribution as they would have if the zero coefficients were known in advance. Simulation studies indicate that this estimator performs well in terms of variable selection and estimation.

1. Introduction Consider a linear regression model Y = β0 + X
β + ε, where β is a p × 1 vector of regression coefficients associated with X. We are interested in estimating β when p → ∞ as the sample size n → ∞ and when β is sparse, in the sense that many of its elements are zero. This is motivated from biomedical studies investigating the relationship between a phenotype of interest and genomic covariates such as microarray data. In many cases, it is reasonable to assume a sparse model, because the number of important covariates is usually relatively small, although the total number of covariates can be large. We use the SCAD method to achieve variable selection and estimation of β simultaneously. The SCAD method is proposed by Fan and Li [1] in a general parametric framework for variable selection and efficient estimation. This method uses a specially designed penalty function, the smoothly clipped absolute deviation (hence the name SCAD). Compared to the classical variable selection methods such as subset selection, the SCAD has two advantages. First, the variable selection with SCAD is continuous and hence more stable than the subset selection, which is a discrete and non-continuous process. Second, the SCAD is computationally feasible for high-dimensional data. In contrast, computation in subset selection is combinatorial and not feasible when p is large. In addition to the SCAD method, several other penalized methods have also been proposed to achieve variable selection and estimation simultaneously. Examples include the bridge penalty (Frank and Friedman [3]), LASSO (Tibshirani [11]), and the Elastic-Net (Enet) penalty (Zou and Hastie [14]), among others. 1 Department of Statistics and Actuarial Science, 241 SH, University of Iowa, Iowa City, Iowa 52246, USA, e-mail: [email protected] AMS 2000 subject classifications: Primary 62J07; secondary 62E20. Keywords and phrases: asymptotic normality, high-dimensional data, oracle property, penalized regression, variable selection.

149

150

J. Huang and H. Xie

Fan and Li [1] and Fan and Peng [2] studied asymptotic properties of SCAD penalized likelihood methods. Their results are concerned with local maximizers of the penalized likelihood, but not the maximum penalized estimators. These results do not imply existence of an estimator with the properties of the local maximizer without auxiliary information about the true parameter value. Therefore, they are not applicable to the SCAD-penalized maximum likelihood estimators, nor the SCAD-penalized estimator. Knight and Fu [7] studied the asymptotic distributions of bridge estimators when the number of covariates is fixed. Huang, Horowitz and Ma [4] studied the bridge estimators with a divergent number of covariates in a linear regression model. They showed that the bridge estimators have an oracle property under appropriate conditions if the bridge index is strictly between 0 and 1. Several earlier studies have investigated the properties of regression estimators with a divergent number of covariates. See, for example, Huber [5] and Portnoy [9, 10]. Portnoy proved consistency and asymptotic normality of a class of M-estimators of regression parameters under appropriate conditions. However, he did not consider penalized regression or selection of variables in sparse models. In this paper, we study the asymptotic properties of the SCAD-penalized least squares estimator, abbreviated as LS-SCAD estimator henceforth. We show that the LS-SCAD estimator can correctly select the nonzero coefficients with probability converging to one and that the estimators of the nonzero coefficients are asymptotically normal with the same means and covariances as they would have if the zero coefficients were known in advance. Thus, the LS-SCAD estimators have an oracle property in the sense of Fan and Li [1] and Fan and Peng [2]. In other words, this estimator is asymptotically as efficient as the ideal estimator assisted by an oracle who knows which coefficients are nonzero and which are zero. The rest of this article is organized as follows. In Section 2, we define the LSSCAD estimator. The main results for the LS-SCAD estimator are given in Section 3, including the consistency and oracle properties. Section 4 describes an algorithm for computing the LS-SCAD estimator and the criterion to choose the penalty parameter. Section 5 offers simulation studies that illustrate the finite sample behavior of this estimator. Some concluding remarks are given in Section 6. The proofs are relegated to the Appendix. 2. Penalized regression with the SCAD penalty Let (Xi , Yi ), i = 1, . . . , n be n observations satisfying Yi = β0 + X
i β + εi ,

i = 1, . . . , n,

where Yi ∈ R is a response variable, Xi is a pn × 1 covariate vector and εi has mean 0 and variance σ 2 . Here the superscripts are used to make it explicit that both the covariates and parameters may change with n. For simplicity, we assume β0 = 0. Otherwise we can center the covariates and responses first. In sparse models, the pn covariates can be classified into two categories: the important ones whose corresponding coefficients are nonzero and the trivial ones whose coefficients are zero. For convenience of notation, we write β = (β
1 , β
2 )
, where β
1 = (β1 , . . . , βkn ) and β
2 = (0, . . . , 0). Here kn (≤ pn ) is the number of nontrivial covariates. Let mn = pn − kn be the number of zero coefficients. Let

LS-SCAD estimator

151

Y = (Y1 , . . . , Yn )
and let X = (Xij , 1 ≤ i ≤ n, 1 ≤ j ≤ pn ) be the n × pn design matrix. According to the partition of β, write X = (X1 , X2 ), where X1 and X2 are n × kn and n × mn matrices, respectively. Given a > 2 and λ > 0, the SCAD penalty at θ is  |θ| ≤ λ,  λ|θ|, λ < |θ| ≤ aλ, pλ (θ; a) = −(θ2 − 2aλ|θ| + λ2 )/[2(a − 1)],  |θ| > aλ. (a + 1)λ2 /2, More insight into it can be gained through its first derivative:  |θ| ≤ λ,  sgn(θ)λ, λ < |θ| ≤ aλ, p
λ (θ; a) = sgn(θ)(aλ − |θ|)/(a − 1),  0, |θ| > aλ.

The SCAD penalty is continuously differentiable on (−∞, 0) ∪ (0, ∞), but not differentiable at 0. Its derivative vanishes outside [−aλ, aλ]. As a consequence, SCAD penalized regression can produce sparse solutions and unbiased estimates for large coefficients. More detailed discussions of this penalty can be found in Fan and Li (2001). The penalized least squares objective function for estimating β with the SCAD penalty is (1)

Qn (b; λn , a) = Y − Xb + n 2

pn 

pλn (bj ; a),

j=1

where  ·  is the L2 norm. Given penalty parameters λn and a, the LS-SCAD estimator of β is  ≡ β(λ  n ; a) = arg min Qn (b; λn , a). β n

 = (β 
, β 
)
the way we partition β into β and β . We write β n 1 2 1n 2n 3. Asymptotic properties of the LS-SCAD estimator

In this section we state the results on the asymptotic properties of the LS-SCAD estimator. Results for the case of fixed design are slightly different from those for the case of random design. We state them separately. For convenience, the main assumptions required for conclusions in this section are listed here. (A0) through (A4) are for fixed covariates. Let ρn,1 be the smallest eigenvalue of n−1 X
X. πn,kn and ωn,mn are the largest eigenvalues of n−1 X
1 X1 and n−1 X
2 X2 , respectively. Let X
i1 = (Xi1 , . . . , Xikn ) and X
i2 = (Xi,kn +1 , . . . , Xipn ). (A0) (a) εi ’s are i.i.d with mean 0 and variance σ 2 ; (b) For any √ j ∈ {1, . . . , pn }, X·j 2 = n. √ (A1) (a) limn→∞ kn λn / ρn,1 = 0; √ √ (b) limn→∞ pn / nρn,1 = 0. √  √ (A2) (a) limn→∞ kn λn / ρn,1 min1≤j≤kn |βj | = 0;   √ √ (b) limn→∞ pn / nρn,1 min1≤j≤kn |βj | = 0;  (c) limn→∞ pn /n/ρn,1 = 0.  √ (A3) limn→∞ max(πn,kn , ω n,mn )pn /( nρn,1 λn ) = 0. n (A4) limn→∞ max1≤i≤n X
i1 ( i=1 Xi1 X
i1 )−1 Xi1 = 0.

J. Huang and H. Xie

152

For random covariates, we require conditions (B0) through (B3). Suppose (X
i , εi )’s are independent and identically distributed as (X
, ε) = (X1 , . . . , Xpn , ε). Analogous to the fixed design case, ρ1 denotes the smallest eigenvalue of E[XX
]. Also πkn and ωmn are the largest eigenvalues of E[Xi1 X
i1 ] and E[Xi2 X
i2 ], respectively. (B0) (X
i , εi ) = (Xi1 , . . . , Xipn , εi ), i = 1, . . . , n are i.i.d. with (a) E[Xij ] = 0, Var(Xij ) = 1; (b) E[ε|X] = 0, Var(ε|X) = σ 2 . (B1) (a) limn→∞ p2n /(nρ21 ) = 0; (b) limn→∞ kn λ2n /ρ1 = 0. √ √ (B2) (a) limn→∞ p√ n /( nρ1 min1≤j≤kn |βj |) = 0; √ (b) limn→∞ λn kn /( ρ1 min1≤j≤kn |βj |) = 0. (B3)  max(πkn , ωmn )pn √ = 0. lim n→∞ nρ1 λn Theorem 1 (Consistency in the fixed design setting). Under (A0)–(A1), P

 − β → 0 β n

as n → ∞.

A similar result holds for the random design case. Theorem 2 (Consistency in the random design setting). Suppose that there exists an absolute constant M4 such that for all n, max1≤j≤pn E[Xj 4 ] ≤ M4 < ∞. Then under (B0)–(B1), P

 − β → 0 β n

as n → ∞.

For consistency, λn has to be kept small so that the SCAD penalty would not introduce any bias asymptotically. Note that in both design settings, the restriction on the penalty parameter λn does not involve mn , the number of trivial covariates. This is shared by the Lq (0 < q < 1)-penalized estimators in Huang, Horowitz and Ma [4]. However, unlike the bridge estimators, no upper bound requirement is imposed on the components of β 1 , since the derivative of the SCAD penalty vanishes beyond a certain interval while that of the Lq penalty does not. In the fixed design case, (A1.b) is needed for model identifiability, as required in the classical regression. For the random design case, a stricter requirement on pn is entailed by the need of the convergence of n−1 X
X to E[XX
] in the Frobenius norm. The next two theorems state that the LS-SCAD estimator is consistent for variable selection. Theorem 3 (Variable selection in the fixed design setting). Under (A0)–  = 0m with probability tending to 1. (A3), β 2n

n

Theorem 4 (Variable selection in the random design setting). Suppose there exists an absolute constant M such that max1≤j≤pn |Xj | ≤ M < ∞. Then  = 0m with probability tending to 1. under (B0)–(B3), β 2n n

(A2.a) and (A2.b) are identical to (A1.a) and (A1.b), respectively, provided that lim inf min |βj | > 0. n→∞ 1≤j≤kn

(B2) has a requirement for min1≤j≤kn |βj | similar to (A2). (A3) concerns the largest eigenvalues of n−1 X
1 X1 and n−1 X
2 X2 . Due to the standardization of covariates, πn,kn ≤ kn and ωn,mn ≤ mn .

LS-SCAD estimator

153

So (A3) is implied by pn = 0. lim √ nρn,1 λn

n→∞

Likewise, (B3) can be replaced with pn lim √ = 0. nρ1 λn

n→∞

Both (A3) and (B3) require λn not to converge too fast to 0 in order for the estimator to be able to “discover” the trivial covariates. It may be of concern if there are λn ’s that simultaneously satisfy (A1)–(A3) (in the random design setting (B1)– (B3)) under certain conditions. When lim inf ρn,1 > 0 and lim inf n→∞ min1≤j≤kn |βj | > 0, it can be checked that there exists λn that meets both (A2) and (A3) as long as pn = o(n1/3 ). If we further know either that kn is fixed, or that the largest eigenvalue of n−1 X
X is bounded from above, as is assumed in Fan and Peng [2], pn = o(n1/2 ) is sufficient. When both of these are true, pn = o(n) is adequate for the existence of such λn ’s. Similar conclusions hold for the random design case except that pn = o(n1/2 ) is indispensable there. The advantage of the SCAD penalty is that once the trivial covariates have been correctly picked out, regression with or without the SCAD penalty will make no  is asymptotically difference to the nontrivial covariates. So it is expected that β 1n normally distributed. Let {An , n = 1, 2, . . .} be a sequence of matrices of dimension d × kn with full row rank. Theorem 5 (Asymptotic normality in the fixed design setting). Under (A0)–(A4), √ −1/2 D  −β )→ n Σn An (β N (0d , Id ), 1n 1 n where Σn = σ 2 An ( i=1 Xi1 X
i1 /n)−1 A
n .

Theorem 6 (Asymptotic normality in the random design setting). Suppose that there exists an absolute constant M such that max1≤j≤pn Xj  ≤ M < ∞ and a σ4 such that E[ε4 |X11 ] ≤ σ4 < ∞ for all n. Then under (B0)–(B3), n−1/2 Σ−1/2 An E −1/2 [Xi1 X
i1 ] n

n  i=1

where Σn = σ 2 An A
n .

 − β ) → N (0d , Id ), Xi1 X
i1 (β 1n 1 D

For the random design the assumptions for asymptotic normality are no more than those for variable selection. While for the fixed design, a Lindeberg-Feller condition (A4) is needed in addition to (A0)–(A3).

4. Computation We use the algorithm of Hunter and Li [6] to compute the LS-SCAD estimator for a given λn and a. This algorithm approximates a nonconvex target function with a convex function locally at each iteration step. We also describe the steps to compute the approximate standard error of the estimator.

J. Huang and H. Xie

154

4.1. Computation of the LS-SCAD estimator Given λn and a the target function to be minimized is Qn (b; λn , a) =



(Yi − X
i b)2 + n

i=1

pn 

pλn (bj ; a).

j=1

Hunter and Li [6] proposes to minimize its approximation Qn,ξ (b; λn , a) =

n 

(Yi − X
i b)2 + n

i=1

=

n 

pn  j=1

(Yi −

X
i b)2

i=1

+n

pn  j=1

pλn ,ξ (bj ; a)

pλn (bj ; a) − ξ



|bj |

0

p
λn (t; a) dt ξ+t



Around b(k) = (b(k),1 , . . . , b(k),pn )
, it can be approximated by Sk,ξ (b; λn , a) =

n 

(Yi − X
i b)2

i=1

 pn  p
λn (|b(k),j |+; a) 2 2 (bj − b(k),j ) , +n pλn ,ξ (b(k),j ; a) + 2(ξ + |b(k),j |) j=1

where ξ is a very small perturbation to prevent any component of the estimate from getting stuck at 0. Therefore the one-step estimator starting from b(k) is b(k+1) = (X
X + nDξ (b(k) ; λn , a))−1 X
Y, where Dξ (b(k) ; λn , a) is the diagonal matrix whose diagonal elements are 12 p
λn × (|b(k),j |+; a)/(ξ + |b(k),j |), j = 1, . . . , pn . Given the tolerance τ , convergence is claimed when    ∂Qn,ξ (b)  τ    ∂bj  < 2 , ∀j = 1, . . . , pn . And finally the bj ’s that satisfy    ∂Qn,ξ (b) ∂Qn (b)  nξp
λn (|bj |+; a) τ  = > −  ∂bj  ∂bj ξ + |bj | 2

 , the least squares estimator. are set to 0. A good starting point would be b(0) = β LS The perturbation ξ should be kept small so that difference between Qn,ξ (·) and Qn (·) is negligible. Hunter and Li [6] suggests using ξ=

τ min{|b(0),j | : b(0),j = 0}. 2nλn

4.2. Standard errors The standard errors for the nonzero coefficient estimates can be obtained via the approximation   ; λ, a) ∂Sξ (β ∂Sξ (β 1 ; λn , a) ∂ 2 Sξ (β 1 ; λn , a)   1n ≈ + β 1n − β 1 .
 ∂β 1 ∂β 1 ∂β 1 ∂β 1n

LS-SCAD estimator

155

So  −β ≈− β 1n 1 ≈−





Since

∂ 2 Sξ (β 1 ; λn , a) ∂β 1 ∂β
1

−1

 ; λn , a) ∂ 2 Sξ (β 1n  ∂β 
∂β 1n

1n

∂Sξ (β 1 ; λn , a) ∂β 1

−1

 ; λn , a) ∂Sξ (β 1n .  ∂β 1n

 ; λn , a) β p
(|β |; a) ∂Sξ (β 1n  + n j λn j = −2X
·j Y + 2X
·j X1 β 1n ∂ βj ξ + |βj |   n  βj p
λn (|βj |; a)
 = , −2Xij Yi + 2Xij Xi1 β 1n + ξ + |βj | i=1
2

n 

Uij (ξ; λn , a),

i=1

letting Uij = Uij (ξ; λn , a), we have, for j, l = 1, . . . , kn ,

  −1/2 ∂Sξ (β 1n ; λn , a) −1/2 ∂Sξ (β 1n ; λn , a) ,n Cov n ∂ βj ∂ βl ≈

n n n  4 4  Uij Uil − 2 Uij Uil . n i=1 n i=1 i=1

Let C = (Cjl , j, l = 1, . . . , kn ), where Cjl =

n n n  1 1  Uij Uil − 2 Uij Uil . n i=1 n i=1 i=1

The variance-covariance matrix of the estimates can be approximated by
 ) ≡ n(X
X1 + nDξ (β  ; λn , a))−1 C (X
X1 + nDξ (β  ; λn , a))−1 . Cov(β 1n 1n 1n 1 1

4.3. Selection of λn

The above computational algorithm is for the case when λn and a are specified. In data analysis, they can be selected by minimizing the generalized cross validation score, which is defined to be GCV(λn , a) = where

 2 /n Y − X1 β 1n , (1 − p(λn , a)/n)2

 −1 
 X
1 p(λn , a) = tr X1 X1 X1 + nD0 (β 1n ; λn , a)

 ; λn , a) is a submatrix of the is the number of effective parameters and D0 (β 1n  ; λn , a) with ξ = 0. By submatrix, we mean the diagodiagonal matrix Dξ (β n  ; λn , a) only contains the elements corresponding to the nontrivial nal of D0 (β 1n

J. Huang and H. Xie

156

 Note that here X1 also only includes the columns of which the components in β.  are non-vanishing. corresponding elements of β n The requirement that a > 2 is implied by the SCAD penalty function. Simulation suggests that the generalized cross validation score does not change much with a given λ. So to improve computing efficiency, we fix a = 3.7, as suggested by Fan and Li [1]. 5. Simulation studies In this section we illustrate the LS-SCAD estimator’s finite sample properties with a simulated example. We simulate covariates Xi , i = 1, . . . , n from the multivariate normal distributions with mean 0 and Cov(Xij , Xil ) = ρ|j−l| , 1 ≤ j, l ≤ p, The response Yi is computed as Yi =

p 

Xij βj + εi ,

i = 1, . . . , n.

j=1

where βj = j, 1 ≤ j ≤ 4, βj = 0, 5 ≤ j ≤ p, and εi ’s are sampled from N (0, 1). For each (n, p, ρ) ∈ {(100, 10), (500, 40)} × {0, 0.2, 0.5, 0.8}, we generated N = 400 data sets and use the algorithm in Section 4 to compute the LS-SCAD estimator. We set the tolerance τ described in Section 4.1 at 10−5 . For comparison we also apply the ordinary least square (LS) method, the ordinary least square method with model selection based on AIC (abbreviated as AIC), and the ordinary least squares assuming that βj = 0 for j ≥ 5 are known beforehand (ORA). Note that this last estimator (ORA) is not feasible in a real data analysis setting. We use it here as a benchmark in the comparisons. The results are summarized in Tables 1 and 2. Columns 4 through 7 in Table 1 are the biases of the estimates of βj , j = 1, . . . , 4 respectively. In the parentheses following each of them are the standard deviations of these estimates. Column 8 (K) lists the numbers of estimates of βj , 5 ≤ j ≤ p that are 0, averaged over 400  For LS, an estimate is set replications, and their modes are given in Column 9 (K). −5 −5 to be 0 if it lies within [−10 , 10 ]. In Table 1, we see that the LS-SCAD estimates of the nontrivial coefficients have biases and standard errors comparable to the ORA estimates. This is in line with Theorems 5 and 6. The average numbers of nonzero estimates for βj (j > 4), K, with respect to LS-SCAD are close to p, the true number of nonzero coefficients among βj (j > 4). As the true number of trivial covariates increases, the LS-SCAD estimator may be able to discover more trivial ones than AIC. However, there is more variability in the number of trivial covariates discovered via LS-SCAD than that via AIC. Table 2 gives the averages of the estimated standard errors of βj , 1 ≤ j ≤ 4 using the SCAD method over the 400 replications. They are obtained based on the approach described in Section 4.2. They are slightly smaller than the sampling standard deviations of βj , 1 ≤ j ≤ 4, which are given in parentheses in the rows for LS-SCAD. Suppose for a data set the estimate of β via one of these four approaches is  then the average model error (AME) regarding this approach is computed as β, n  − β)]2 . Box plots for these AME’s are given in Figure 1. n−1 i=1 [X
i (β n

LS-SCAD estimator

157

Table 1 Simulation example 1, comparison of estimators (n, p) ρ (100, 10) 0

Estimator LS ORA AIC

β1 β2 β3 β4 K .0007 (.1112) −.0034 (.0979) −.0064 (.1127) −.0024 (.1091) 0 .0008 (.1074) −.0054 (.0936) −.0057 (.1072) −.0007 (.1040) 6 .0007 (.1083) −.0026 (.1033) −.0060 (.1156) −.0019 (.1181) 4.91

 K 0 6 5

SCAD 0.2 LS ORA AIC

−.0006 −.0003 −.0005 −.0002

(.1094) (.1051) (.1010) (.1031)

−.0037 −.0028 −.0031 −.0024

(.0950) −.0058 (.1094) −.0014 (.1060) (.1068) .0093 (.1157) .0037 (.1103) (.1035) .0107 (.1131) .0020 (.1035) (.1063) .0107 (.1150) .0021 (.1079)

4.62 0 6 4.95

5 0 6 5

SCAD 0.5 LS ORA AIC

−.0025 .0000 −.0002 −.0003

(.1035) (.1177) (.1129) (.1162)

−.0026 −.0007 −.0072 −.0064

(.1046) (.1353) (.1317) (.1338)

.0104 .0010 .0115 .0114

(.1141) (.1438) (.1393) (.1413)

.0024 .0006 .0022 .0017

(.1066) (.1360) (.1171) (.1294)

4.64 0 6 4.91

5 0 6 5

SCAD 0.8 LS ORA AIC

.0035 −.0005 −.0039 −.0021

(.1115) (.1916) (.1835) (.1857)

−.0219 −.0229 −.0196 −.0209

(.1404) (.2293) (.2197) (.2235)

.0135 .0059 .0070 .0063

(.1481) (.2319) (.2250) (.2289)

.0006 .0060 .0092 .0013

(.1293) (.2200) (.1787) (.2072)

4.78 0 6 4.85

5 0 6 6

SCAD LS ORA AIC

−.0038 .0021 .0027 .0023

(.1868) (.0466) (.0446) (.0460)

−.0197 −.0000 −.0005 −.0003

(.2249) .0062 (.2280) .0032 (.0475) −.0010 (.0466) .0014 (.0453) −.0003 (.0448) .0011 (.0465) −.0004 (.0453) .0016

SCAD 0.2 LS ORA AIC

.0027 .0018 .0003 .0024

(.0447) −.0004 (.0454) −.0004 (.0450) (.0478) .0003 (.0478) −.0014 (.0487) (.0522) −.0000 (.0465) −.0010 (.0517) (.0473) .0002 (.0471) −.0014 (.0475)

SCAD 0.5 LS ORA AIC

.0028 .0024 .0027 .0031

(.0461) (.0542) (.0526) (.0537)

SCAD 0.8 LS ORA AIC SCAD

.0025 .0014 .0010 .0020 .0014

(.0528) .0017 (.0587) (.0788) −.0012 (.1014) (.0761) .0017 (.0954) (.0776) .0003 (.0996) (.0773) .0018 (.0982)

(500, 40) 0

.0002 .0001 .0017 .0007

.0013 .0005 .0009 .0018

(.2024) 4.87 6 (.0439) 0 0 (.0426) 36 36 (.0433) 29.91 30 (.0429) 32.22 35 (.0437) 0 0 (.0458) 36 36 (.0436) 29.87 30

(.0460) −.0011 (.0475) .0006 (.0433) (.0617) .0050 (.0608) −.0048 (.0563) (.0581) .0033 (.0597) −.0030 (.0488) (.0603) .0037 (.0605) −.0038 (.0526) .0034 .0090 .0060 .0066 .0059

(.0601) (.1000) (.0983) (.0995) (.0990)

−.0037 −.0077 −.0044 −.0071 −.0050

(.0494) (.0943) (.0704) (.0862) (.0790)

32.20 35 0 0 36 36 29.87 32 31.855 0 36 29.56 29.38

35 0 36 30 35

Table 2 Simulated example, standard error estimate (n, p) ρ 1 ) se(β 2 ) se(β 3 ) se(β 4 ) se(β

0 .0983 .0980 .0996 .0988

(100, 10) 0.2 0.5 .1005 .1139 .1028 .1276 .1027 .1278 .1006 .1150

0.8 .1624 .2080 .2086 .1727

0 .0442 .0443 .0442 .0441

(500, 40) 0.2 0.5 .0444 .0512 .0447 .0571 .0445 .0573 .0444 .0512

0.8 .0735 .0940 .0940 .0764

The LS estimator definitely has the worst performance in terms of AME. This becomes more obvious as the number of trivial predictors increases. LS-SCAD outperforms AIC in this respect and is comparable to ORA. But it is also seen that the AME’s of LS-SCAD tend to be more diffuse as ρ increases. This is also the result of more spread-out estimates of the number of trivial covariates. 6. Concluding remarks In this paper, we have studied the asymptotic properties of the LS-SCAD estimator when the number of covariates and regression coefficients increases to infinity as

158

J. Huang and H. Xie

Fig 1. Box plots of the average model errors for four estimators: AIC, LS, ORA, and LS-SCAD. In the top four panels, (n, p, ρ) = (100, 10, 0), (100, 10, 0.2), (100, 10, 0.5), (100, 10, 0.8); and in the bottom four panels, (n, p, ρ) = (500, 40, 0), (500, 40, 0.2), (500, 40, 0.5), (500, 40, 0.8), where n is the sample size, p is the number of covariates, and ρ is the correlation coefficient used in generating the covariate values.

n → ∞. We have shown that this estimator can correctly identify zero coefficients with probability converging to one and that the estimators of nonzero coefficients are asymptotically normal and oracle efficient. Our results were obtained under the assumption that the number of parameters is smaller than the sample size. They are not applicable when the number of parameters is greater than the sample size, which arises in microarray gene expression studies. In general, the condition that p < n is needed for identification of the regression parameter and consistent variable selection. To achieve consistent variable selection in the “large p, small n” case, certain conditions are required for the design matrix. For example, Huang et al. [4]

LS-SCAD estimator

159

showed that, under a partial orthogonality assumption in which the covariates of the zero coefficients are uncorrelated or only weakly correlated with the covariates of nonzero coefficients, then the univariate bridge estimators are consistent for variable selection under appropriate conditions. This result also holds for the univariate LS-SCAD estimator. Indeed, under the partial orthogonality condition, it can be shown that the simple univariate regression estimator can be used to consistently distinguish between nonzero and zero coefficients. Finally, we note that our results are only valid for a fixed sequence of penalty parameters λn . It is an interesting and difficult problem to show that the asymptotic oracle property also holds for λn determined by cross validation. Appendix We now give the proofs of the results stated in Section 3.  , it is necessary that Qn (β  ) ≤ Qn (β). Proof of Theorem 1. By the definition of β n n It follows that kn       − β)2 − 2ε
X(β  − β) + n  0 ≥ X(β ( β ; a) − p (β ; a) p λn j λn j n n j=1

   − β)2 − 2ε
X(β  − β) − 2−1 n(a + 1)kn λ2 ≥ X(β n n n 
1/2 
−1/2
2   = [X X] (β n − β) − [X X] Xε − ε
X[X
X]−1 X
ε − 2−1 n(a + 1)kn λ2n .

By the Cr -inequality (Lo´eve [8], page 155),   
1/2  − β)2 ≤ 2[X
X]1/2 (β  − β) − [X
X]−1/2 X
ε2 + 2ε
X[X
X]−1 X
ε [X X] (β n n ≤ 4ε
X[X
X]−1 X
ε + n(a + 1)kn λ2n .

In the fixed design,   ε
X[X(n)
X]−1 X
ε = E ε
X[X(n)
X]−1 X
ε OP (1) = σ 2 tr(X[X
X]−1 X
)OP (1) = pn OP (1).

Since


1/2  − β)2 ≥ nρn,1 β  − β2 , [X X] (β n n

√  √  pn kn λn + = oP (1). √ √ nρn,1 ρn,1 n (n) (n) Proof of Theorem 2. Let A(n) = (Ajk )j,k=1,...,pn with Ajk = n−1 i=1 Xij Xik − E[Xij Xik ]. Let ρ1 (A(n) ) and ρpn (A(n) ) be the smallest and largest of the eigenvalues of A(n) , respectively. Then by Theorem 4.1 in Wang and Jia [13], we have

 − β = OP β n

ρ1 (A(n) ) ≤ ρn,1 − ρ1 ≤ ρpn (A). By the Cauchy inequality and the properties of eigenvalues of symmetric matrices, max(|ρ1 (A(n) )|, |ρpn (A(n) )|) ≤ A(n) .

J. Huang and H. Xie

160

When (B1.a) holds, A(n)  = oP (ρ1 ) = oP (1), as is seen for any ξ > 0, P (A(n) 2 ≥ ξρ1 2 ) ≤

EA(n) 2 p2 ≤ n2 2 ξρ1 ξρ1

sup 1≤j,k≤pn

(n)

Var(Ajk ) ≤

p2n M4 . nξρ1 2

Since ρ1 > 0 holds for all n, n−1 X
X is invertible with probability tending to 1. Following the argument for the fixed design case, with probability tending to 1, 
1/2  − β)2 ≤ 4ε
X[X
X]−1 X
ε − n(a + 1)kn λ2 . [X X] (β n n

In the random design setting,

  1 2

−1
 (n) 2 E ε X[X X] X εA  < ρ1 2

  1 2 2
−1
 (n) 2 = σ E tr(X[X X] X )A  < ρ1 2 = σ 2 pn .

The rest of the argument remains the same as for the fixed design case and leads to √ √  pn kn λn  + √ β n − β = OP √ = oP (1). nρ1 ρ1 Lemma 1 (Convergency rate in the fixed design setting). Under (A0)–(A2),   − β = OP ( pn /n/ρn,1 ). β n Proof. In the proof of consistency, we have  − β = OP (un ), β n

where un = λn

For any L1 , provided that b − β ≤ 2L1 un ,



kn /ρn,1 +



pn /(nρn,1 ).

min |bj | ≥ min |βj | − 2L1 un .

1≤j≤kn

1≤j≤kn

If (A2) holds, then for n sufficiently large, un / min1≤j≤kn |βj | < 2−L1 −1 . It follows that min |bj | ≥ min |βj |/2, 1≤j≤kn

1≤j≤kn

which further implies than min1≤j≤kn |bj | > aλn for n sufficiently large (assume lim inf n→∞ kn > 0). Let {hn } be a sequence converging to 0. As in the proof of of Theorem 3.2.5 ¯ of Van der Vaart and Wellner [12], decompose Rpn \{0pn } into shells {Sn,l , l ∈ Z} where Sn,l = {b : 2l−1 hn ≤ b − β < 2l hn }. For b ∈ Sn,l such that 2l hn ≤ 2L1 un , Qn (b) − Qn (β) = (b − β)
X
X(b − β) − 2ε
X(b − β) pn pn   pλn (bj ; a) − n pλn (βj ; a) +n j=1





j=1


= (b − β) X X(b − β) − 2ε X(b − β)
In1 + In2 ,

and In1 ≥ nρn,1 b − β2 ≥ 22(l−1) h2n nρn,1 .

LS-SCAD estimator

161

Thus    − β ≥ 2L hn P β n     ∈ Sn,l P β ≤ o(1) + n l>L

2l hn ≤2L1 un



≤ o(1) +

P

l>L 2l hn ≤2L1 un



≤ o(1) +



 inf Qn (b) ≤ Qn (β)

b∈Sn,l

P



sup ε
X(b − β) ≥ 22l−3 h2n nρn,1

b∈Sn,l

l>L,



2l−1 hn ≤2L1 un

E| supb∈Sn,l ε
X(b − β)|



≤ o(1) +

l>L, 2l−1 hn ≤2L1 un

≤ o(1) +

22l−3 h2n nρn,1

 2l hn E 1/2 [ε
X2 ]

22l−3 h2n nρn,1   2l nσ 2 pn ≤ o(1) + , 22l−3 hn nρn,1 l>L

l>L

  − β = OP ( pn /n/ρn,1 ). from which we see β n

Lemma 2 (Convergence  rate in the random design setting). Under (B0)–  (B2), β n − β = OP ( pn /n/ρ1 ).

Proof. Deduction is similar to that of Lemma 1. However, since X is a random matrix in this case, extra details are needed in the following part. Let A(n) = n (n) (n) (Ajk )j,k=1,...,pn with Ajk = n1 i=1 Xij Xik − E[Xj Xk ]. We have    − β ≥ 2L hn P β n     ∈ Sn,l , A(n)  ≤ ρ1 /2 + o(1) P β ≤ n l>L

2l hn ≤2L1 un



≤

l>L 2l hn ≤2L1 un

≤

P



inf Qn (b) ≤ Qn (β), A

b∈Sn,l

     2l hn E 1/2 ε
X2 A ≤ ρ1 /2 l>L

(n)

22l−4 h2n nρ1



 ≤ ρ1 /2 + o(1)

+ o(1).

  − β = OP ( pn /n/ρ1 ). The first inequality follows from (B1.a). This leads to β n  − β ≤ λn with probability tending to 1 Proof of Theorem 3. By Lemma 1, β n under (A3). Consider the partial derivatives of Qn (β + v). For j = kn + 1, . . . , pn ,

J. Huang and H. Xie

162

if |vj | ≤ λn , n  ∂ Qn (β + v) =2 Xij (εi − X
i v) + nλn sgn(vj ) ∂ vj i=1

=2

n 

Xij εi − 2

i=1

n 

Xij X
i1 v1 − 2

i=1

n 

Xij X
i2 v2 + nλn sgn(vj )

i=1


IIn1,j + IIn2,j + IIn3,j + IIn4,j . Examine the first three terms one by one.

E[

max

kn +1≤j≤pn



|IIn1,j |] ≤ E 1/2 



pn 

2  = 2√nmn σ, IIn1,j

j=kn +1

  n    
max |IIn2,j | = 2 max  Xij Xi1 v1   kn +1≤j≤pn kn +1≤j≤pn  i=1  ≤ 2v1  max (X·j )
X1 X
1 X·j kn +1≤j≤pn

≤ 2v1 

max

kn +1≤j≤pn

max

kn +1≤j≤pn


X·j ρ1/2 max (X1 X1 )


= 2v1  max X·j  ρ1/2 max (X1 X1 ) kn +1≤j≤pn √ = 2n πn,kn v1 , n  |IIn3,j | = 2 max | Xij X
i2 v2 | kn +1≤j≤pn

i=1


≤ 2v1 X·j ρ1/2 max (X2 X2 ) √ = 2n ωn,mn v2 .

Following the above argument we have 

P

When (A3) holds, Therefore  ! P

!

kn +1≤j≤pn

≤



{|IIn1,j | > |IIn4,j | − |IIn2,j | − |IIn3,j |}

nλn − 2n

√

√ 2 nmn σ2 . √ πn,kn v1  + ωn,mn v2 

 √ √ nλn / mn → ∞. Under (A1)–(A2), v = OP ( pn /n/ρn,1 ).

kn +1≤j≤pn



{|IIn1,j | > |IIn4,j | − |IIn2,j | − |IIn3,j |} → 0 as n → ∞.

This indicates that with probability tending to 1, for all j = kn + 1, . . . , pn , the sign is the same as vj , provided that |vj | < λn , which further implies that of ∂ Qn∂(β+v) vj  = 0m ) = 1. lim P (β 2n n

n→∞

LS-SCAD estimator

163

Proof of Theorem 4. Follow the argument in the proof of Theorem 3. Note that in the random design setting, under (B1.a),  n     
max |IIn2,j | = 2 max  Xij Xi1 v1   kn +1≤j≤pn kn +1≤j≤pn  i=1  ≤ 2v1  max (X·j )
X1 X
1 X·j kn +1≤j≤pn


max X·j ρ1/2 max (X1 X1 )  √ ≤ 2v1  nM ρmax (X
1 X1 )  √ ≤ 2M nv1  n [ρmax (E[X1 X
1 ]) + A11 ]  ≤ 2nv1  πkn + E 1/2 A11 2 OP (1) $ 1/2 M kn = 2nv1  πkn + OP (ρ1 ) 4 √ ρ1 n √ ≤ 4nv1  πkn OP (1)

≤ 2v1 

kn +1≤j≤pn

for sufficiently large n. Similarly max

kn +1≤j≤pn

√ |IIn3,j | ≤ 4nv2  ωmn OP (1).

The rest of the argument is identical to that in the fixed design case and thus omitted here. Proof of Theorem 5. During the course of proving Lemma 1, we have under (A0)–    − β = OP (λn kn /ρn,1 + pn /(nρn,1 )). Under (A2), this implies that (A1), β n  − β  = oP ( min |β |). β 1n 1 j 1≤j≤kn

Also from (A2), λn = o(min1≤j≤kn |β j |). Therefore, with probability tending to 1, all the βj (1 ≤ j ≤ kn ) are bounded away from [−aλn , aλn ] and so the partial  derivatives exist. At the same time, β 2n = 0mn with probability tending to 1. Thus with probability tending to 1, the stationarity condition holds for the first kn  necessarily satisfies the equation components. That is, β 1n n  i=1

 )Xi1 = 0, (Yi − X
i1 β 1n

i.e.

n  i=1

So the random vector being considered √

εi Xi1 =

n  i=1

 − β ). Xi1 X
i1 (β 1n 1

 −β ) nΣ−1/2 An (β 1n 1 n n √  −1 = n Σ−1/2 An (X
1 X1 ) Xi1 εi n

Zn


i=1


n−1/2

n  i=1

(n)

Ri εi ,

J. Huang and H. Xie

164 −1/2

where Ri = Σn An (n−1 X
1 X1 )−1 Xi1 . The equality holds with probability tend√ (n) ing to 1. max1≤i≤n Ri / n → 0 is implied by (A4), as can be seen from (n)

 −1 An n−1 X
1 X1 Xi1  √ n   −1/2  −1/2  −1
Xi1  ≤n  n X1 X1  −1/2
−1  −1/2  · ρ1/2 n−1 X
1 X1 An Σn An n−1 X
1 X1 max     −1/2   −2 −1/2 −1/2 σ Xi1  ρ1/2 Σ Σ Σ = n−1/2  n−1 X
1 X1 n max n n %

n −1 & &  Xi1 X
i1 Xi1 . = 'σ −2 X
i1 −1/2

(n) Ri  Σn √ = n

i=1

Therefore for any ξ > 0,

 √ 1   (n) 2 (n) E Ri εi  1{Ri εi  > nξ} n i=1 n

 √ 1  (n)
(n)  2 (n) = Ri Ri E εi 1{Ri εi  > nξ} n i=1

 n √ 1  (n)
(n) (n) 2 ≤ R Ri E εi 1{|εi | > nξ/ max Ri } 1≤i≤n n i=1 i n

n

=

1  (n)
(n) R Ri o(1) n i=1 i

n  −1 1 
 −1
−1
−1 Xi1 n X1 X1 An Σn An n−1 X
1 X1 Xi1 o(1) n i=1 ( ) n   −1
−1
−1  −1
−1 Xi1 X
i1 = tr n X1 X1 An Σ n An n X 1 X 1 o(1) n i=1 * −1
−1 + An Σn An o(1) = tr n−1 X
1 X1  

n −1    1 = tr Σ−1 Xi1 X
i1 A
n o(1) n An   n

=

i=1

= o(1)d.

So

D

Zn → N (0d , Id ). follows from the Lindeberg-Feller central limit theorem and Var(Zn ) = Id . Proof of Theorem 6. The vector being considered n  1  −β ) √ Σ−1/2 An E −1/2 [Xi1 X
i1 ] Xi1 X
i1 (β 1n 1 n n i=1 n  1 −1/2
= √ Σ−1/2 A E [X X ] εi Xi1 n i1 i1 n n i=1

LS-SCAD estimator

165

−1/2

with probability tending to 1. Let Zni = √1n Σn An E −1/2 [Xi1 X
i1 ]Xi1 εi , i = 1, . . . , n. {Zni , n = 1, 2, . . . , i = 1, . . . , n} form a triangular array and within each row, they are i.i.d random vectors. First,

n    −1/2
Var Zni = Var Σ−1/2 A E [X X ]X ε = Id . n i1 i1 11 n i=1

Second, under (B1.a), n  i=1

0 / 0 / E Zni 2 1{Zni >ξ} = nE Zn1 2 1{Zn1 >ξ}

≤ nE 1/2 [Zn1 4 ]P 1/2 (Zn1  > ξ) = o(1),

since E 1/2 [Zn1 4 ] = E 1/2 [(Z
n1 Zn1 )2 ]

2  1 1/2 4 
− 1

−1 − 12
2 = E ε X11 E [X11 X11 ]An Σn An E [X11 X11 ]X11 n   1 1/2 2 −1 ≤ σ4 ρmax (A
n Σ−1 [X11 X
11 ])E 1/2 (X
11 X11 ) n An ) ρmax (E n

 2  1 1/2 (n)
−1
−1 1/2 X11 X11 ≤ σ4 ρmax (Σn An An ) ρ1 E n   2  kn   1 1/2 −2 −1 1/2  2 = σ4 σ ρ1 E X1j   n  j=1    kn =O , nρ1 and P

1/2

√ d E 1/2 (Z
n1 Zn1 ) =√ , (Zn1  > ξ) ≤ ξ nξ

by the Lindeberg–Feller central limit theorem we have  − β ) → N (0d , Id ). An E −1/2 [Xi1 X
i1 ]X
1 X1 (β n−1/2 Σ−1/2 1n 1 n D

Acknowledgments. JH is honored and delighted to have the opportunity to contribute to this monograph in celebration of Professor Piet Groeneboom’s 65th birthday and his contributions to mathematics and statistics. The authors thank the editors and an anonymous referee for their constructive comments which led to significant improvement of this article. References [1] Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348–1360. [2] Fan, J. and Peng, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters. Ann. Statist. 32 928–961. [3] Frank, I. E. and Friedman, J. H. (1993). A statistical view of some chemometrics regression tools (with discussion). Technometrics 35 109–148.

166

J. Huang and H. Xie

[4] Huang, J., Horowitz, J. L. and Ma, S. G. (2006). Asymptotic properties of bridge estimators in sparse high-dimensional regression models. Technical Report # 360, Department of Statistics and Actuarial Science, University of Iowa. [5] Huber, P. J. (1981). Robust Statistics. Wiley, New York. [6] Hunter, D. R. and Li, R. (2005). Variable selection using MM algorithms. Ann. Statist. 33 1617–1642. [7] Knight, K. and Fu, W. J. (2000). Asymptotics for lasso-type estimators. Ann. Statist. 28 1356–1378. `ve, M. (1963). Probability Theory. Van Nostrand, Princeton. [8] Loe [9] Portnoy, S. (1984). Asymptotic behavior of M estimators of p regression parameters when p2 /n is large: I. Consistency. Ann. Statist. 12 1298–1309. [10] Portnoy, S. (1985). Asymptotic behavior of M estimators of p regression parameters when p2 /n is large: II. Normal approximation. Ann. Statist. 13 1403–1417. [11] Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. J. Roy. Statist. Soc. Ser. B 58 267–288. [12] Van der Vaart, A. W. and Wellner, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York. [13] Wang, S. and Jia, Z. (1993). Inequalities in Matrix Theory. Anhui Education Press. [14] Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. J. Roy. Statist. Soc. Ser. B 67 301–320.

IMS Lecture Notes–Monograph Series Asymptotics: Particles, Processes and Inverse Problems Vol. 55 (2007) 167–178 c Institute of Mathematical Statistics, 2007
DOI: 10.1214/074921707000000346

Critical scaling of stochastic epidemic models∗ Steven P. Lalley1 University of Chicago

To Piet Groeneboom, on the occasion of his 39th birthday. Abstract: In the simple mean-field SIS and SIR epidemic models, infection is transmitted from infectious to susceptible members of a finite population by independent p−coin tosses. Spatial variants of these models are proposed, in which finite populations of size N are situated at the sites of a lattice and infectious contacts are limited to individuals at neighboring sites. Scaling laws for both the mean-field and spatial models are given when the infection parameter p is such that the epidemics are critical. It is shown that in all cases there is a critical threshold for the numbers initially infected: below the threshold, the epidemic evolves in essentially the same manner as its branching envelope, but at the threshold evolves like a branching process with a sizedependent drift.

1. Stochastic epidemic models 1.1. Mean-field models The simplest and most thoroughly studied stochastic models of epidemics are meanfield models, in which all individuals of a finite population interact in the same manner. In these models, a contagious disease is transmitted among individuals of a homogeneous population of size N . In the simple SIS epidemic, individuals are at any time either infected or susceptible; infected individuals remain infected for one unit of time and then become susceptible. In the simple SIR epidemic (more commonly known as the Reed-Frost model), individuals are either infected, susceptible, or recovered ; infected individuals remain infected for one unit of time, after which they recover and acquire permanent immunity from future infection. In both models, the mechanism by which infection occurs is random: At each time, for any pair (i, s) of an infected and a susceptible individual, the disease is transmitted from i to s with probability p = pN . These transmission events are mutually independent. Thus, in both the SIR and the SIS model, the number Jt+1 = JtN of infected individuals at time t + 1 is given by (1)

Jt+1 =

St


ξs ,

s=1

∗ Supported

by NSF Grant DMS-04-05102. of Chicago, Department of Statistics, 5734 S. University Avenue, Eckhart 118, Chicago, Illinois 60637, USA, e-mail: [email protected] AMS 2000 subject classifications: 60K30, 60H30, 60K35. Keywords and phrases: stochastic epidemic model, spatial epidemic, Feller diffusion, branching random walk, Dawson-Watanabe process, critical scaling. 1 University

167

168

S. P. Lalley

where St = StN is the number of susceptibles at time t and the random variables ξs are, conditional on the history of the epidemic to time t, independent, identically distributed Bernoulli-1 − (1 − p)Jt . In the SIR model, (2)

Rt+1 = Rt + Jt

and

St+1 = St − Jt+1 , where Rt is the number of recovered individuals at time t, while in the SIS model, (3)

St+1 = St + Jt − Jt+1 .

In either model, the epidemic ends at the first time T when JT = 0. The most basic and interesting questions concerning these models have to do with the duration T  and size t≤T Jt of the epidemic and their dependence on the infection parameter pN and the initial conditions. 1.2. Spatial SIR and SIS epidemics In the simple SIS and SIR epidemics, no allowance is made for geographic or social stratifications of the population, nor for variability in susceptibility or degree of contagiousness. Following are descriptions of simple stochastic models that incorporate a geographic stratification of a population. We shall call these the (spatial) SIS −d and SIR−d epidemics, with d denoting the spatial dimension. Assume that at each lattice point x ∈ Zd is a homogeneous population of Nx individuals, each of whom may at any time be either susceptible or infected, or (in the SIR variants) recovered. These populations may be thought of as “villages”. As in the mean-field models, infected individuals remain contagious for one unit of time, after which they recover with immunity from future infection (in the SIR variants) or once again become susceptible (in the SIS models). At each time t = 0, 1, 2, . . . , for each pair (ix , sy ) of an infected individual located at x and a susceptible individual at y, the disease spreads from ix to sy with probability α(x, y). The simple Reed-Frost and stochastic logistic epidemics described in section 1.1 terminate with probability one, regardless of the value of the infection parameter p, because the population is finite. For the spatial SIS and SIR models this is no longer necessarily the case: If x∈Zd Nx = ∞ then, depending on the value of the parameter p and the dimension d, the epidemic may persist forever with positive probability. (For instance, if Nx = 1 for all x and α(x, y) = p for nearest neighbor pairs x, y but α(x, y) = 0 otherwise, then the SIS −d epidemic is just oriented percolation on Zd+1 , which is known to survive with positive probability if p exceeds a critical value pc < 1 [6].) Obvious questions of interest center on how the epidemic spreads through space, and in cases where it eventually dies out, how far it spreads. The figure below shows a simulation of an SIS-1 epidemic with village size Nx = 20224 and infection parameter 1/60672. At time 0 there were 2048 infected individuals at site 0; all other individuals were healthy. The epidemic lasted 713 generations (only the first 450 are shown). 1.3. Epidemic models and random graphs All of the models described above have equivalent descriptions as structured random graphs, that is, percolation processes. Consider for definiteness the simple SIR

Critical Scaling of Stochastic Epidemic Models

169

Fig 1.

(Reed-Frost) epidemic. In this model, no individual may be infected more than once; furthermore, for any pair x, y of individuals, there will be at most one opportunity for infection to pass from x to y or from y to x during the course of the epidemic. Thus, one could simulate the epidemic by first tossing a p−coin for every pair x, y, drawing an edge between x and y for each coin toss resulting in a Head, and then using the resulting (Erd¨ os-Renyi) random graph determined by these edges to determine the course of infection in the epidemic. In detail: If Y0 is the set of infected individuals at time 0, then the set Y1 of individuals infected at time 1 consists of all x ∈ / Y0 that are connected to individuals in Y0 , and for any subsequent time n, / ∪j≤n Yj who are the set Yn+1 of individuals infected at time n + 1 consists of all x ∈ connected to individuals in Yn . Note that the set of individuals ultimately infected during the course of the epidemic is the union of those connected components of the random graph containing at least one vertex in Y0 . Similar random graph descriptions may be given for the simple SIS and the spatial SIS and SIR epidemic models. 1.4. Branching envelopes of epidemics For each of the stochastic epidemic models discussed above there is an associated branching process that serves, in a certain sense, as a “tangent” to the epidemic. We shall refer to this branching process as the branching envelope of the epidemic. The branching envelopes of the simple mean-field epidemics are ordinary GaltonWatson processes; the envelopes of the spatial epidemics are branching random walks. There is a natural coupling of each epidemic with its branching envelope in which the set of infected individuals in the epidemic is at each time (and in the spatial models, at each location) dominated by the corresponding set of individuals in the branching envelope.

170

S. P. Lalley

Fig 2.

Following is a detailed description of the natural coupling of the simple SIS epidemic with its branching envelope. The branching envelope is a Galton-Watson process Zn with offspring distribution Binomial-(N, p), where p is the infection parameter of the epidemic, and whose initial generation Z0 coincides with the set of individuals who are infected at time 0. Particles in the Galton-Watson process are marked red or blue: red particles represent infected individuals in the coupled epidemic, while blue offspring of red parents represent attempted infections that are not allowed because the attempt is made on an individual who is not susceptible, or has already been infected by another contagious individual. Colors are assigned as follows: (1) Offspring of blue particles are always blue. (2) Each red particle reproduces by tossing a p−coin N times, once for each individual i in the population. Each Head counts as an offspring, and each represents an attempted infection. If several red particles attempt to infect the same individual i, exactly one of these is marked as a success (red), and the others are marked as failures (blue). Also, if an attempt is made to infect an individual who is not susceptible, the corresponding particle is colored blue. Clearly, the collection of all particles (red and blue) evolves as a Galton-Watson process, while the collection of red particles evolves as the infected set in the SIS epidemic. See the figure below for a typical evolution of the coupling in a population of size N = 80, 000 with p = 1/80000 and 200 individuals initially infected. 2. Critical behavior: mean-field case When studying the behavior of the simple SIR and SIS epidemics in large populations, it is natural to consider the scaling p = pN = λN /N for the infection parameter p. In this scaling, λ = λN is the mean of the offspring distribution in the branching envelope. If λ < 1 then the epidemic will end quickly, even if a large number of individuals are initially infected. On the other hand, if λ > 1 then with positive probability (approximately one minus the extinction probability for the associated branching envelope), even if only one individual is initially infected, the epidemic will be large, with a positive limiting fraction of the population eventually being infected. The large-N behavior of the size of the SIR epidemic in this case is

Critical Scaling of Stochastic Epidemic Models

171

well understood: see for example [12] and [14]. 2.1. Critical scaling: size of the epidemic The behavior of both the SIS and SIR epidemics is more interesting in the critical case λN ≈ 1. When the set of individuals initially infected is sufficiently small relative to the population size, the epidemic can be expected to evolve in much the same manner as a critical Galton-Watson process with Poisson-1 offspring distribution. However, when the size of the initially infected set passes a certain critical threshold, then the epidemic will begin to deviate substantially from the branching envelope. For the SIR case, the critical threshold was (implicitly) shown by [11] and [1] (see also [12]) to be at N 1/3 , and that the critical scaling window is of width N −4/3 : Theorem 1 ([11], [1]). Assume that pN = 1/N + a/N 4/3 + o(n−4/3 ), and that the J0N /N 1/3 → b as the popunumber J0N of initially infected individuals is such that  lation size N → ∞. Then as N → ∞, the size UN := t Jt obeys the asymptotic law (4)

D

UN /N 2/3 −→ Tb

where Tb is the first passage time to the level b by Wt + t2 /2 + at, and Wt is a standard Wiener process. The distribution of the first passage time Tb can be given in closed form: See [11], also [8], [13]. For the critical SIS epidemic, the critical threshold is at N 1/2 , and the critical scaling window is of width N −3/2 : Theorem 2 ([4]). Assume that pN = 1/N + a/N 3/2 + o(n−3/2 ), and that the N 1/2 as N → ∞. Then the initial number of infected individuals  satisfies J0 ∼ bN total number of infections UN := t Jt during the course of the epidemic obeys (5)

D

UN /N −→ τ (b − a; −a)

where τ (x; y) is the time of first passage to y by a standard Ornstein-Uhlenbeck process started at x. 2.2. Critical scaling: time evolution of the epidemic For both the SIR and SIS epidemics, if the number of individuals initially infected is much below the critical threshold then the evolution of the epidemic will not differ noticeably from that of its branching envelope. It was observed by [7] (and proved by [9]) that a (near-) critical Galton-Watson process initiated by a large number M of individuals behaves, after appropriate rescaling, approximately as a Feller diffusion: In particular, if ZnM is the size of the nth generation of a Galton-Watson with Z0M ∼ bM with offspring distribution Poisson(1 + a/M )then as M → ∞, (6)

D

M Z[M t] /M −→ Yt

where Yt satisfies the stochastic differential equation  dYt = aYt dt + Yt dWt , (7) Y0 = b.

S. P. Lalley

172

What happens at the critical threshold, in both the SIR and SIS epidemics, is that the deviation from the branching envelope exhibits itself as a size-dependent drift in the limiting diffusion: N Theorem 3 ([4]). Let J N (n) = J[n] be the number infected in the nth generation of a simple SIS epidemic in a population of size N . Then under the hypotheses of Theorem 2, √ √ D (8) J N ( N t)/ N −→ Yt

where Y0 = b and Yt obeys the stochastic differential equation  (9) dYt = (aYt − Yt2 ) dt + Yt dWt

Note that the diffusion (9) has an entrance boundary at ∞, so that it is possible with initial condition Y0 = 0. When the SIS to define a version Yt of the process √ N initially infected, the number JtN infected epidemic is begun with J0N √ √ will rapidly drop (over the first ε N generations) until reaching a level of order N , and then evolve as predicted by (8). The following figure depicts a typical evolution in a population of size N = 80, 000, with infection parameter p = 1/N and I0 = 10, 000 initially infected. N N Theorem 4 ([4]). Let J N (n) = J[n] and RN (n) = R[n] be the numbers of infected and recovered individuals in the nth generation of a simple SIR epidemic in a population of size N . Then under the hypotheses of Theorem 1,    −1/3 N 1/3  J(t) J (N t) N D −→ (10) R(t) N −2/3 RN (N 1/3 t)

where J0 = b, R0 = 0, and (11)

dJ(t) = (aJ(t) − J(t)R(t)) dt + dR(t) = J(t) dt.



J(t) dWt ,

Theorems 1–2 can be deduced from Theorems 3–4 by simple time-change arguments (see [4]).

Fig 3.

Critical Scaling of Stochastic Epidemic Models

173

2.3. Critical scaling: heuristics

The critical thresholds for the SIS −d and SIR−d epidemics can be guessed by simple comparison arguments using the standard couplings of the epidemics with their branching envelopes. Consider first the critical SIS epidemic in a population of size N . Recall (Section 1.4) that the branching envelope is a critical Galton-Watson process whose offspring distribution is Binomial-(N, 1/N ). The particles of this Galton-Watson process are marked red or blue, in such a way that in each generation the number of red particles coincides with the number of infected individuals in the SIS epidemic. Offspring of blue particles are always blue, but offspring of red particles may be either red or blue; the blue offspring of red parents in each generation represent attempted infections that are suppressed. Assume that initially there are N α infected individuals and thus also N α individuals in the zeroth generation of the branching envelope. By Feller’s theorem, we may expect that the extinction time of the branching envelope will be on the order N α , and that in each generation up to (shortly before) extinction the branching process will have order N α individuals. If α is small enough that the SIS epidemic obeys the same rough asymptotics (that is, stays alive for O(N α ) generations and has O(N α ) infected individuals in each generation), then the number of blue offspring of red parents in each generation will be on the order N ×(N 2α /N 2 ) (because for each of the N individuals of the population, the chance of a double infection is about N 2α /N 2 ). Since the duration of the epidemic will be of the same rough order of magnitude as the size of the infected set in each generation, there should be at most O(1) blue offspring of red parents in any generation (if there were more, the red population would die out long before the blue). Thus, the critical threshold must be at N 1/2 . A similar argument applies for the SIR epidemic. The branching envelope of the critical SIR is once again a critical Galton-Watson process with offspring distribution Binomial-(N, 1/N ), with constituent particles again labeled red or blue, red particles representing infected individuals in the epidemic. The rule by which red particles reproduce is as follows: Each red particle tosses a p−coin N times once for each individual i in the population. Each Head counts as an offspring, and represents an attempted infection. However, if a Head occurs on a toss at individual i where i was infected in an earlier generation, then the Head results in a blue offspring. Similarly, if more than one red particle tosses a Head at an individual i which has not been infected earlier, then one of these is labeled red and the excess are all labeled blue. Assume that initially there are N α infected individuals. As before, we may expect that the extinction time of the branching envelope will be on the order N α , and that in each generation up to extinction the branching process will have order N α individuals. If α is small enough, the extinction time and the size of the red population will also be O(N α ). Consequently, the size of the recovered population will be (for all but the first few generations) on order N 2α . Thus, in each generation, the number of blue offspring of red parents will be on order (N 2α /N )×N α (because the chance that a recovered individual is chosen for attempted infection by an infected individual is O(N α /N )). Therefore, by similar reasoning as in the SIS case, the critical threshold is at N 1/3 , as this is where the the number of blue offspring of red parents in each generation is O(1).

174

S. P. Lalley

3. Critical behavior: SIS -1 and SIR-1 Spatial epidemics Consider now the spatial SIS -d and SIR-d epidemic models on the d-dimensional integer lattice Zd . Assume that the village size Nx = N is the same for all sites x ∈ Zd , and that the infection probabilities α(x, y) are nearest neighbor homogeneous, and uniform, that is,  p = pN , if |x − y| ≤ 1; (12) α(x, y) = 0, otherwise. 3.1. Scaling limits of branching random walks The branching envelope of a spatial SIS −d or SIR−d epidemic is a nearest neighbor branching random walk on the integer lattice Zd . This evolves as follows: Any particle located at site x at time t lives for one unit of time and then reproduces, placing random numbers ξy of offspring at the sites y such that |y − x| ≤ 1. The random variables ξy are mutually independent, with Binomial-(N, pN ) distributions. The analogue for branching random walks of Feller’s theorem for Galton-Watson processes is Watanabe’s theorem. This asserts that, after suitable rescaling, as the particle density increases, critical branching random walks converge to a limit, the Dawson-Watanabe process, also known as super Brownian motion. A precise statement follows: Consider a sequence of branching random walks, indexed by M = 1, 2, . . . , with offspring distribution Binomial-(N, pM ) as above, and (13)

pM = pN,M =

a 1 − . (2d + 1)N NM

(Note: N may depend on M .) The rescaled measure-valued process √ XtM associated with the M th branching random walk puts mass 1/M at location x/ M and time t for each particle of the branching random walk that is located at site x at time [M t]. (Note: The branching random walk is a discrete-time process, but the associated measure-valued process runs in continuous time.) Watanabe’s theorem ([15]). Assume that the initial values X0M converge weakly (as finite Borel measures on Rd ) to a limit measure X0 . Then under the hypothesis (13) the measure-valued processes XtM converge in law as M → ∞ to a limit Xt : (14)

XtM =⇒ Xt .

The limit process is the Dawson-Watanabe process with killing rate a and initial value X0 . (The term killing rate is used because the process can be obtained from the “standard” Dawson-Watanabe process (a = 0) by elimination of mass at constant rate a.) The Dawson-Watanabe process Xt with killing rate a can be characterized by a martingale property: For each test function φ ∈ Cc2 (Rd ),  t  σ t Xs , ∆φ ds + a Xs , ϕ ds (15) Xt , φ − X0 , φ − 2 0 0 is a martingale. Here σ 2 = 2d/(2d + 1) is the variance of the random walk kernel naturally associated with the branching random walks. It is known [10] that in dimension d = 1 the random measure Xt is for each t absolutely continuous relative to Lebesgue measure, and the Radon-Nikodym derivative X(t, x) is jointly continuous in t, x (for t > 0). In dimensions d ≥ 2 the measure Xt is almost surely singular, and is supported by a Borel set of Hausdorff dimension 2 [3].

Critical Scaling of Stochastic Epidemic Models

175

3.2. Spatial SIS-1 and SIR-1 epidemics: critical scaling As in the mean-field case, there are critical thresholds for the SIS -1 and SIR-1 epidemics at which they begin to deviate noticeably from their branching envelopes. These are at N 2/3 and N 2/5 , respectively: Theorem 5 ([5]). Fix α > 0, and let XtN be the discrete-time measure-valued process obtained from an SIS−1 or an SIR−1 epidemic on a one-dimensional grid of size-N villages by attaching mass 1/N α to the point (t, x/N α/2 ) for each infected individual at site x at time [tN α ]. Assume that X0N converges weakly to a limit measure X0 as the village size N → ∞. Then as N → ∞, D

N X[N α t] −→ Xt ,

(16)

where Xt is a measure-valued process with initial value X0 whose law depends on the value of α and the type of epidemic (SIS or SIR) as follows: (a) SIS: If α < 23 then Xt is a Dawson-Watanabe process with variance σ 2 . (b) SIS: If α = 23 then Xt is a Dawson-Watanabe process with variance σ 2 and killing rate (17)

θ(x, t) = X(x, t)/2.

(c) SIR: If α < 25 then Xt is a Dawson-Watanabe process with variance σ 2 . (d) SIR: If α = 25 then Xt is a Dawson-Watanabe process with variance σ 2 and killing rate  t X(x, s) ds. (18) θ(x, t) = X(x, t) 0

The Dawson-Watanabe process with variance σ 2 and (continuous, adapted) killing rate θ(t, x, ω) is characterized [2] by a martingale problem similar to (15) above: for each test function φ ∈ Cc2 (R),  t  σ t Xs , ∆φ ds + Xs , θϕ ds (19) Xt , φ − X0 , φ − 2 0 0 is a martingale. The law of this process is mutually absolutely continuous relative to that of the Dawson-Watanabe process with no killing, and there is an explicit formula for the Radon-Nikodym derivative – see [2]. 3.3. Critical scaling for spatial epidemics: heuristics Arguments similar to those given above for the mean-field SIS and SIR epidemics can be used to guess the critical thresholds for the spatial SIS-1 and SIR-1 epidemics. For the spatial epidemics, the associated branching envelopes are branching random walks. In the standard couplings, particles of the branching envelope are labeled either red or blue, with the red particles representing infected individuals in the epidemics. As in the mean-field cases, offspring of blue particles are always blue, but offspring of red particles may be either red or blue; blue offspring of red parents represent attempted infections that are suppressed. These may be viewed as an attrition of the red population (since blue particles created by red parents are potential red offspring that are not realized!).

176

S. P. Lalley

Consider first the SIS-1 epidemic. Assume that initially there are N α particles, distributed (say) uniformly among the N α/2 sites nearest the origin. Then by Feller’s limit theorem (recall that the total population size in a branching random walk is a Galton-Watson process), the branching envelope can be expected to survive for OP (N α ) generations, and at any time prior to extinction the population will have OP (N α ) members. These will be distributed among the sites at distance OP (N α/2 ) from the origin, and therefore in dimension d = 1 there should be about OP (N α/2 ) particles per site. Consequently, for the SIS −1 epidemic, the rate of attrition per site per generation should be OP (N α−1 ), and so the total attrition rate per generation should be OP (N 3α/2−1 ). If α = 2/3, then the total attrition rate per generation will be OP (1), just enough so that the total attrition through the duration of the branching random walk envelope will be on the same order of magnitude as the population size N α . For the SIR−1 epidemic there is a similar heuristic calculation. As for the SIS −1 epidemic, the branching envelope will survive for OP (N α ) generations, and up to the time of extinction the population should have OP (N α ) individuals, about OP (N α/2 ) per site. Therefore, through N α generations, about N α ×N α/2 numbers j will be retired, and so the attrition rate per site per generation should be OP (N α/2 × N 3α/2 ), making the total attrition rate per generation OP (N 5α/2 ). Hence, if α = 2/5 then the total attrition per generation should be OP (1), just enough so that the total attrition through the duration of the branching random walk envelope will be on the same order of magnitude as the population size. 3.4. Critical scaling in dimensions d ≥ 2 In higher dimensions, the critical behavior of the SIS -d and SIR-d epidemics appears to be considerably different. We expect that there will be no analogous threshold effect, in particular, we expect that the epidemic will behave in the same manner as the branching envelope up to the point where the infected set is a positive fraction of the total population. This is because in dimensions d ≥ 2, the particles of a critical branching random walk quickly spread out, so that (after a short initial period) there are only OP (1) (in dimensions d ≥ 3) or OP (log N ) (in dimension d = 2) particles per site. (With N particles initially, a critical branching random walk typically lives √ O(N ) generations, and particles are distributed among the sites at distance O( N ) from the origin; in dimensions d ≥ 2, there are O(N d/2 ) such sites, enough to accomodate the O(N ) particles of the branching random walk without crowding.) Consequently, the rate at which “multiple” infections are attempted (that is, attempts by more than one contagious individual to simultaneously infect the same susceptible) is only of order OP (1/N ) (or, in dimension d = 2, order OP (log N/N )). The interesting questions regarding the evolution of critical epidemics in dimensions d ≥ 2 center on the initial stages, in the relatively small amount of time (order o(N ) generations) in which the particles spread out from their initial sites. These will be discussed in the forthcoming University of Chicago Ph. D. dissertation of Xinghua Zheng. 3.5. Weak convergence of densities There is an obvious gap in the heuristic argument of Section 3.3 above: Even if the total number of infected individuals is, as expected, on the order N α , and even if

Critical Scaling of Stochastic Epidemic Models

177

these are concentrated in the sites at distance on the order N α/2 from the origin, it is by no means obvious that these will distribute themselves uniformly (or at least locally uniformly) among these sites. The key step in filling this gap in the argument is to show that the particles of the branching envelope distribute themselves more or less uniformly on scales smaller than N α/2 . Consider, as in Section 3.1, a sequence of branching random walks, indexed by M = 1, 2, . . . , with offspring distribution Binomial-(N, pM ) as above, and pM given by (13). Let YtM (x) be the number of particles at site x at time [t], and let X M (t, x) be the continuous function of t ≥ 0 and x ∈ R obtained by linear interpolation from the values √ √ YM t ( M x) M √ for M t ∈ Z+ and M x ∈ Z. (20) X (t, x) = M Theorem 6 ([5]). Assume that d = 1. Assume also that the initial con√ particle √ figuration is such that all particles are located in an interval [−κ M , κ M ] and such that the initial particle density satisfies (21)

X M (0, x) =⇒ X(0, x)

as M → ∞ for some continuous function X(0, x) with support [−κ, κ]. Then as M → ∞, (22)

X M (t, x) =⇒ X(t, x),

where X(t, x) is the density function of the Dawson-Watanabe process with killing rate a and initial value X(0, x). The convergence is relative to the topology of uniform convergence on compacts in the space C(R+ × R) of continuous functions. Since the measure-valued processes associated with the densities X M (t, x) are known to converge to the Dawson-Watanabe process, by Watanabe’s theorem, to prove Theorem 6 it suffices to establish tightness. This is done by a somewhat technical application of the Kolmogorov-Chentsov tightness criterion, based on a careful estimation of moments. See [5] for details. It is also possible to show that convergence of the particle density processes holds in Theorem 5. References [1] Aldous, D. (1997). Brownian excursions, critical random graphs and the multiplicative coalescent. Ann. Probab. 25 812–854. [2] Dawson, D. A. (1978). Geostochastic calculus. Canad. J. Statist. 6 143–168. [3] Dawson, D. A. and Hochberg, K. J. (1979). The carrying dimension of a stochastic measure diffusion. Ann. Probab. 7 693–703. [4] Dolgoarshinnykh, R. and Lalley, S. P. (2006). Critical scaling for the sis stochastic epidemic. J. Appl. Probab. 43 892–898. [5] Dolgoarshinnykh, R. and Lalley, S. P. (2006). Spatial epidemics: Critical behavior. Preprint. [6] Durrett, R. (1984). Oriented percolation in two dimensions. Ann. Probab. 12 999–1040. [7] Feller, W. (1939). Die Grundlagen der Volterraschen Theorie des Kampfes ums Dasein in wahrscheinlichkeitstheoretischer Behandlung. Acta Bioth. Ser. A 5 11–40.

178

S. P. Lalley

[8] Groeneboom, P. (1989). Brownian motion with a parabolic drift and Airy functions. Probab. Theory Related Fields 81 79–109. ˇ ˇina, M. (1969). On Feller’s branching diffusion processes. Casopis [9] Jir Pˇest. Mat. 94 84–90. 107. [10] Konno, N. and Shiga, T. (1988). Stochastic partial differential equations for some measure-valued diffusions. Probab. Theory Related Fields 79 201–225. ¨ f, A. (1998). The final size of a nearly critical epidemic, and the [11] Martin-Lo first passage time of a Wiener process to a parabolic barrier. J. Appl. Probab. 35 671–682. [12] Nagaev, A. V. and Starcev, A. N. (1970). Asymptotic analysis of a certain stochastic model of an epidemic. Teor. Verojatnost. i Primenen 15 97–105. [13] Salminen, P. (1988). On the first hitting time and the last exit time for a Brownian motion to/from a moving boundary. Adv. in Appl. Probab. 20 411– 426. [14] Sellke, T. (1983). On the asymptotic distribution of the size of a stochastic epidemic. J. Appl. Probab. 20 390–394. [15] Watanabe, S. (1968). A limit theorem of branching processes and continuous state branching processes. J. Math. Kyoto Univ. 8 141–167.

IMS Lecture Notes–Monograph Series Asymptotics: Particles, Processes and Inverse Problems Vol. 55 (2007) 179–195 c Institute of Mathematical Statistics, 2007
DOI: 10.1214/074921707000000355

Additive isotone regression Enno Mammen1,∗ and Kyusang Yu1,∗ Universit¨ at Mannheim

This paper is dedicated to Piet Groeneboom on the occasion of his 65th birthday Abstract: This paper is about optimal estimation of the additive components of a nonparametric, additive isotone regression model. It is shown that asymptotically up to first order, each additive component can be estimated as well as it could be by a least squares estimator if the other components were known. The algorithm for the calculation of the estimator uses backfitting. Convergence of the algorithm is shown. Finite sample properties are also compared through simulation experiments.

1. Introduction In this paper we discuss nonparametric additive monotone regression models. We discuss a backfitting estimator that is based on iterative application of the pool adjacent violator algorithm to the additive components of the model. Our main result states the following oracle property. Asymptotically up to first order, each additive component is estimated as well as it would be (by a least squares estimator) if the other components were known. This goes beyond the classical finding that the estimator achieves the same rate of convergence, independently of the number of additive components. The result states that the asymptotic distribution of the estimator does not depend on the number of components. We have two motivations for considering this model. First of all we think that this is a useful model for some applications. For a discussion of isotonic additive regression from a more applied point, see also Bacchetti [1], Morton-Jones et al. [32] and De Boer, Besten and Ter Braak [7]. But our main motivation comes from statistical theory. We think that the study of nonparametric models with several nonparametric components is not fully understood. The oracle property that is stated in this paper for additive isotone models has been shown for smoothing estimators in some other nonparametric models. This property is expected to hold if the estimation of the different nonparametric components is based on local smoothing where the localization takes place in different scales. An example are additive models of smooth functions where each localization takes place with respect to another covariate. In Mammen, Linton and Nielsen [28] the oracle property has been verified for the local linear smooth backfitting estimator. As local linear estimators, also the isotonic least squares is a local smoother. The estimator is a local average of the response variable but in contrast to local linear estimators the local neighborhood is chosen ∗ Research of this paper was supported by the Deutsche Forschungsgemeinschaft project MA 1026/7-3 in the framework of priority program SPP-1114. 1 Department of Economics, Universit¨ at Mannheim, L 7, 3–5, 68131 Mannheim, Germany, email: [email protected]; [email protected] AMS 2000 subject classifications: 62G07, 62G20. Keywords and phrases: isotone regression, additive regression, oracle property, pool adjacent violator algorithm, backfitting.

179

180

E. Mammen and K. Yu

by the data. This data adaptive choice is automatically done by the least squares minimization. This understanding of isotonic least squares as a local smoother was our basic motivation to conjecture that for isotonic least squares the oracle property should hold as for local linear smooth backfitting. It may be conjectured that the oracle property holds for a much larger class of models. In Horowitz, Klemela and Mammen [19] a general approach was introduced for applying one-dimensional nonparametric smoothers to an additive model. The procedure consists of two steps. In the first step, a fit to the additive model is constructed by using the projection approach of Mammen, Linton and Nielsen [28]. This preliminary estimator uses an undersmoothing bandwidth, so its bias terms are of asymptotically negligible higher order. In a second step, a one-dimensional smoother operates on the fitted values of the preliminary estimator. For the resulting estimator the oracle property was shown: This two step estimator is asymptotically equivalent to the estimator obtained by applying the one-dimensional smoother to a nonparametric regression model that only contains one component. It was conjectured that this result also holds in more general models where several nonparametric components enter into the model. Basically, a proof could be based on this two step procedures. The conjecture has been verified in Horowitz and Mammen [20, 22] for generalized additive models with known and with unknown link function. The study of the oracle property goes beyond the classical analysis of rates of convergence. Rates of convergence of nonparametric estimators depend on the entropy of the nonparametric function class. If several nonparametric functions enter into the model the entropy is the sum of the entropies of the classes of the components. This implies that the resulting rate coincides with the rate of a model that only contains one nonparametric component. Thus, rate optimality can be shown for a large class of models with several nonparametric components by use of empirical process theory, see e.g. van de Geer [39]. Rate optimality for additive models was first shown in Stone [38]. This property was the basic motivation for using additive models. In contrast to a full dimensional model it allows estimation with the same rate of convergence as a one-dimensional model and avoids for this reason the curse of dimensionality. On the other hand it is a very flexible model that covers many features of the data nonparametrically. For a general class of nonparametric models with several components rate optimality is shown in Horowitz and Mammen [21]. The estimator of this paper is based on backfitting. There is now a good understanding of backfitting methods for additive models. For a detailed discussion of the basic statistical ideas see Hastie and Tibshirani [18]. The basic asymptotic theory is given in Opsomer and Ruppert [34] and Opsomer [35] for the classical backfitting and in Mammen, Linton and Nielsen [28] for the smooth backfitting. Bandwidth choice and practical implementations are discussed in Mammen and Park [29, 30] and Nielsen and Sperlich [33]. The basic difference between smooth backfitting and backfitting lies in the fact that smooth backfitting is based on a smoothed least squares criterion whereas in the classical backfitting smoothing takes place only for the updated component. The full smoothing of the smooth backfitting algorithm stabilizes the numerical and the statistical performance of the estimator. In particular this is the case for degenerated designs and for the case of many covariates as was shown in simulations by Nielsen and Sperlich [33]. In this paper we use backfitting without any smoothing. For this reason isotone additive least squares will have similar problems as classical backfitting and these problems will be even more severe because no smoothing is used at all. Smooth backfitting methods for generalized additive models were introduced in Yu, Park and Mammen [42]. Haag

Additive isotone regression

181

[15] discusses smooth backfitting for nonparametric additive diffusion models. Tests based on smooth backfitting have been considered in Haag [16] and Mammen and Sperlich [31]. Backfitting tests have been proposed in Fan and Jiang [10]. Additive regression is an example of a nonparametric model where the nonparametric function is given as a solution of an integral equation. This has been outlined in Linton and Mammen [24] and Carrasco, Florens and Renault [6] where also other examples of statistical integral equations are given. Examples are additive models where the additive components are linked as in Linton and Mammen [25] and regression models with dependent errors where an optimal transformation leads to an additive model, see Linton and Mammen [26]. The representation of estimation in additive models as solving an empirical integral equation can also be used to understand why the oracle property holds. In this paper we verify the oracle property for additive models of isotone functions. It is shown that each additive component can be estimated with the same asymptotic accuracy as if the other components would be known. We compare the performance of a least squares backfitting estimator with a least squares isotone estimator in the oracle model where only one additive component is unknown. The backfitting estimator is based on iterative applications of isotone least squares to each additive component. Our main theoretical result is that the differences between these two estimators are of second order. This result will be given in the next section. The numerical performance of the isotone backfitting algorithm and its numerical convergence will be discussed in Section 3. Simulations for the comparison of the isotone backfitting estimator with the oracle estimator are presented in Section 4. The proofs are deferred to the Appendix.

2. Asymptotics for additive isotone regression We suppose that we have i.i.d. random vectors (Y 1 , X11 , . . . , Xd1 ), . . . , (Y n , X1n , . . . , Xdn ) and we consider the regression model (1)

E(Y i |X1i , . . . , Xdi ) = c + m1 (X1i ) + · · · + md (Xdi )

where mj (·)’s are monotone functions. Without loss of generality we suppose that all functions are monotone increasing. We also assume that the covariables take values in a compact interval, [0, 1], say. For identifiability we add the normalizing condition


(2)

1

mj (xj ) dxj = 0.

0

The least squares estimator for the regression model (1) is given as minimizer of (3)

n  (Y i − c − µ1 (X1i ) − · · · − µd (Xdi ))2 i=1

with respect to monotone increasing functions µ1 , . . . , µd and a constant c that 1  1, . . . , m  d and  c. fulfill 0 µj (xj ) dxj = 0. The resulting estimators are denoted as m OR  j that make use We will compare the estimators m  j with oracle estimators m of the knowledge of ml for l
= j. The oracle estimator m  OR is given as minimizer j

E. Mammen and K. Yu

182

of n 

(Y i − c − µj (X1i ) −

i=1



ml (Xli ))2

l=j

=

n 

(mj (Xji ) + εi − c − µj (X1i ))2

i=1

with respect to a monotone increasing function µj and a constant c that fulfill 1 µ (xj )dxj = 0. The resulting estimators are denoted as m  OR and  cOR . j 0 j In the case d = 1, this gives the isotonic least squares estimator proposed by Brunk [4] which is given by (i)

(4)

m  1 (X1 ) = max min s≤i t≥i

(1)

(n)

t 

Y (j) /(t − s + 1)

j=s

where X1 , . . . , X1 are the order statistics of X 1 , . . . , X n and Y (j) is the obser(j) vation at the observed point X1 . Properties and simple computing algorithms are discussed e.g. in Barlow et al. [2] and Robertson, Wright, and Dykstra [36]. A fast way to calculate the estimator is to use the Pool Adjacent Violator Algorithm (PAVA). In the next section we discuss a backfitting algorithm for d > 1 that is based on iterative use of PAVA. We now state a result for the asymptotic performance of m  j . We use the following assumptions. To have an economic notation, in the assumptions and in the proofs we denote different constants by the same symbol C. (A1) The functions m1 , . . . , md are differentiable and their derivatives are bounded on [0, 1]. The functions are strictly monotone, in particular for G(δ) = inf |u−v|≥δ,1≤j≤d |mj (v) − mj (u)| it holds G(δ) ≥ Cδ γ for constants C, γ > 0 for all δ > 0. (A2) The d-dimensional vector X i = (X1i , . . . , Xdi ) has compact support [0, 1]d . The density p of X i is bounded away from zero and infinity on [0, 1]d and it is continuous. The tuples (X i , Y i ) are i.i.d. For j, k = 1, . . . , d the density pXk ,Xj of (Xki , Xji ) fulfills the following Lipschitz condition for constants C, ρ > 0 sup

0≤uj ,uk ,vk ≤1

|pXk ,Xj (uk , uj ) − pXk ,Xj (vk , uj )| ≤ C|uk − vk |ρ .

(A3) Given X i the error variables εi = Y i − c − m1 (X1i ) − · · · − md (Xdi ) have conditional zero mean and subexponential tails, i.e. for some γ > 0 and C  > 0, it holds that     a.s. E exp(γ|εi |)X i < C  The conditional variance of εi given X i = x is denoted by σ 2 (x). The conditional variance of εi given X1i = u1 is denoted by σ12 (u1 ). We assume that σ12 is continuous at x1 .

These are weak smoothness conditions. We need (A3) to apply results from empirical process theory. Condition (A1) excludes the case that a function mj has flat parts. This is done for the following reason. Isotonic least squares regression produces piecewise constant estimators where for every piece the estimator is equal to the sample average of the piece. If the function is strictly monotone the pieces

Additive isotone regression

183

shrink to 0, a.s. If the function has flat parts these averages do not localize at the flat parts. But in our proof we make essential use of a localization argument. We conjecture that our oracle result that is stated below also holds for the case that there are flat parts. But we do not pursue to check this here. It is also of minor interest because at flat parts the monotone least squares estimator is of − mj = oP (n−1/3 ). Thus the oracle result m j −m  OR = oP (n−1/3 ) then order m  OR j j j only implies that m  j − mj = oP (n−1/3 ). In particular, it does not imply that m OR and m  j have the same asymptotic distribution limit.  1 for For d = 1 the asymptotics for m  1 are well known. Note that the estimator m for d > 1 that is based on isotonizing d = 1 coincides with the oracle estimator m  OR 1 Y i − c − m2 (X2i ) − · · · − md (Xdi ) = m1 (X1i ) + εi in the order of the values of X1i (i = 1, . . . , n). For the oracle model (or for the model (1) with d = 1) the following asymptotic result holds under (A1)–(A3): For all x1 ∈ (0, 1) it holds that −1/3 m  OR ). 1 (x1 ) − m1 (x1 ) = OP (n

Furthermore at points x1 ∈ (0, 1) with m1 (x1 ) > 0, the normalized estimator n1/3

[2p1 (x1 )]1/3 [m  OR (x1 ) − m1 (x1 )] σ1 (x1 )2/3 m1 (x1 )1/3 1

converges in distribution to the slope of the greatest convex minorant of W (t)+ t2 , where W is a two-sided Brownian motion. Here, p1 is the density of X1i . The greatest convex minorant of a function f is defined as the greatest convex function g with g ≤ f , pointwise. This result can be found, e.g. in Wright [41] and Leurgans [23]. Compare also Mammen [27]. For further results on the asymptotic law of m  OR 1 (x1 ) − m1 (x1 ), compare also Groeneboom [12, 13].  OR We now state our main result about the asymptotic equivalence of m  j and m j . Theorem 1. Make the assumptions (A1)–(A3). Then it holds for c large enough that sup n−1/3 ≤xj ≤1−n−1/3

−1/3 |m  j (xj ) − m  OR ), j (xj )| = oP (n

−2/9 sup |m  j (xj ) − m  OR (log n)c ) j (xj )| = oP (n

0≤xj ≤1

The proof of Theorem 1 can be found in the Appendix. Theorem 1 and the above mentioned result on m  OR immediately implies the following corollary. 1

Corollary 1. Make the assumptions (A1)–(A3). For x1 ∈ (0, 1) with m1 (x1 ) > 0 it holds that [2p1 (x1 )]1/3 n1/3 [m  1 (x1 ) − m1 (x1 )] σ1 (x1 )2/3 m1 (x1 )1/3

converges in distribution to the slope of the greatest convex minorant of W (t) + t2 , where W is a two-sided Brownian motion. 3. Algorithms for additive isotone regression The one-dimensional isotonic least squares estimator can be regarded as a projection of the observed vector (Y (1) , . . . , Y (n) ) onto the convex cone of isotonic vectors in Rn

E. Mammen and K. Yu

184

n with respect to the scalar product f, g ≡ i=1 f (i) g (i) where f ≡ (f (1) , . . . , f (n) ) and g ≡ (g (1) , . . . , g (n) ) ∈ Rn . Equivalently, we can regard it as a projection of a right continuous simple function with values (Y (1) , . . . , Y (n) ) onto the convex cone (i) of right continuous simple monotone functions which can have jumps only at X1 ’s. The projection is with respect to the L2 norm defined by the empirical measure, Pn (y, x1 ) which gives mass 1/n at each observations (Y i , X1i ). Other monotone functions m with m(X (i) ) = g (i) would also solve the least square minimization. Now, we consider the optimization problem (3). Without loss of generality, we drop the constant. Let Hj , j = 1, . . . , d be the sets of isotonic vectors of length n or right continuous monotone simple functions which have jumps only at Xji ’s with respect to the ordered Xj ’s. It is well known that these sets are convex cones. Then, our optimization problem can be written as follows: (5)

min

g∈H1 +···+Hd

n  (Y i − g i )2 . i=1

We can rewrite (5) as an optimization problem over a product sets H1 × · · · × Hd . Say g = (g1 , . . . , gd ) ∈ H1 × · · · × Hd where gj ∈ Hj for j = 1, . . . , d. Then the minimization problem (5) can be represented as minimizing a function over a cartesian product of sets, i.e., (6)

min

g∈H1 ×···×Hd

F (Y, g).

n Here, F (Y, g) = i=1 (Y i − g1i − · · · − gdi )2 . A classical way to solve an optimization problem over product sets is a cyclic iterative procedure where at each step we minimize F with respect to one gj ∈ Hj [r] while keeping the other gk ∈ Hk , j
= k fixed. That is to generate sequences gj , [r]

[r]

[r]

r = 1, 2, . . . , j = 1, . . . , d, recursively such that gj minimizes F (y, g1 , . . . , gj−1 , u, [r−1] [r−1] gj+1 , . . . , gd )

over u ∈ Hj . This procedure for (6) entails the well known backfitting procedure with isotonic regressions on Xj , Π(·|Hj ) which is given as 

[r] [r] [r] [r−1] [r−1]  (7) gj = Π Y − g1 − · · · − gj−1 − gj+1 − · · · − gd  Hj , [0]

r = 1, 2, . . . , j = 1, . . . , d, with initial values gj = 0 where Y = (Y 1 , . . . , Y n ). For i,[r] i,[r] i,[r] a more precise description, we introduce a notation Y = Y i −g −· · ·−g − j

i,[r−1]

i,[r−1]

1

j−1

i,[r]

gj+1 − · · · − gd where gk is the value of gk at Xki after the r-th iteration, i,[r] i.e. Y is the residual at the j-th cycle in the r-th iteration. Then, we have j

i,[r]

gj

= max min s≤i t≥i

t  =s

(),[r] Yj /(t − s + 1).

(),[r] Here, Yj is the residual at the k-th cycle in the r-th iteration corresponding to ()

the Xj . Let g ∗ be the projection of Y onto H1 + · · · + Hd , i.e., the minimizer of the problem (5).   [r] [r−1] , converges to g ∗ Theorem 2. The sequence, g(r,j) ≡ 1≤k≤j gk + j≤k≤d gk as r → ∞ for j = 1, . . . , d. Moreover if the problem (6) has a solution that is unique

Additive isotone regression

185

up to an additive constant, say g∗ = (g1∗ , . . . , gd∗ ), the sequences gj converge to a vector with constant entries as r → ∞ for j = 1, . . . , d. [r]

In general, g∗ is not unique. Let g = (g1 , . . . , gd ) be a solution of (6). Suppose, e.g. that there exists a tuple of non constant vectors (f1 , . . . , fd ) such that f1i +· · ·+fdi = 0 for i = 1, . . . , n and gj + fj are monotone. Then, one does not have the unique d d solution for (6) since j=1 gji = j=1 (gji + fji ) and gj + fj are monotone. This phenomenon is similar to ’concurvity’, introduced in Buja et al. (1989). One simple example for non-uniqueness is the case that elements of X are ordered in the same way, i.e., Xjp ≤ Xjq ⇔ Xkp ≤ Xkq for any (p, q) and (j, k). For example when d = 2, if g solves (5), then (αg, (1 − α)g) for any α ∈ [0, 1] solve (6). Other examples occur if elements of X are ordered in the same way for a subregion. 4. Simulations In this section, we present some simulation results for the finite sample performance. These numerical experiments are done by R on windows. We iterate 1000 times for each setting. For each iteration, we draw random samples from the following model Y = m1 (X1 ) + m2 (X2 ) + ,

(8)

where (X1 , X2 ) has truncated bivariate normal distribution and  ∼ N (0, 0.52 ). In Table 1 and 2, we present the empirical MISE (mean integrated squared error) of the backfitting estimator and the oracle estimator. We also report the ratio (B/O), MISE of the backfitting estimator to MISE of the oracle estimator. For Table 1, we set m1 (x) = x3 and m2 (x) = sin(πx/2). The results in Table 1 show that the backfitting estimator and the oracle estimator have a very similar finite sample performance. See that the ratio (B/O) is near to one in most cases and converges to one as sample size grows. We observe that when two covariates have strong negative correlation, the backfitting estimator has bigger MISE than the oracle estimator but the ratio (B/O) goes down to one as sample size grows. Figure 1 shows typical curves from the backfitting and oracle estimators for m1 . We show the estimators that achieve 25%, 50% and 75% quantiles of the L2 -distance Table 1 Comparison between the backfitting and the oracle estimator: Model (8) with m1 (x) = x3 , m2 (x) = sin(πx/2), sample size 200, 400, 800 and different values of ρ for covariate distribution m1 n 200

400

800

ρ 0 0.5 −0.5 0.9 −0.9 0 0.5 −0.5 0.9 −0.9 0 0.5 −0.5 0.9 −0.9

Backfitting 0.01325 0.01312 0.01375 0.01345 0.01894 0.00824 0.00825 0.00831 0.00846 0.10509 0.00512 0.00502 0.00509 0.00523 0.00603

Oracle 0.01347 0.01350 0.01345 0.01275 0.01309 0.00839 0.00845 0.00830 0.00814 0.00805 0.00525 0.00513 0.00513 0.00500 0.00498

m2 B/O 0.984 0.972 1.022 1.055 1.447 0.982 0.977 1.001 1.040 1.305 0.976 0.977 0.994 1.046 1.211

Backfitting 0.01793 0.01817 0.01797 0.01815 0.02363 0.01068 0.01070 0.01081 0.01092 0.01311 0.00654 0.00646 0.00660 0.00667 0.00757

Oracle 0.01635 0.01674 0.01609 0.01601 0.01633 0.01000 0.01001 0.00997 0.00997 0.00992 0.00621 0.00614 0.00620 0.00611 0.00612

B/O 1.096 1.086 1.117 1.134 1.447 1.068 1.063 1.084 1.095 1.321 1.053 1.052 1.066 1.091 1.220

E. Mammen and K. Yu

186

between the backfitting and the oracle estimator for m1 (x). We observe that the backfitting and the oracle estimator produce almost identical curves. Table 2 reports simulation results for the case that one component function is not smooth. Here, m1 (x) = x, |x| > 0.5; 0.5, 0 ≤ x ≤ 0.5; −0.5, −0.5 ≤ x < 0 and m2 (x) = sin(πx/2). Even in this case the backfitting estimator shows a quite good performance. Thus the oracle property of additive isotonic least square regression Table 2 Comparison between the backfitting and the oracle estimator: Model (8) with m1 (x) = x, |x| > 0.5; 0.5, 0 ≤ x ≤ 0.5; −0.5, −0.5 ≤ x < 0, m2 (x) = sin(πx/2), sample size 200, 400, 800 and different values of ρ for covariate distribution m1 n 200

400

800

ρ 0 0.5 −0.5 0.9 −0.9 0 0.5 −0.5 0.9 −0.9 0 0.5 −0.5 0.9 −0.9

Backfitting 0.01684 0.01686 0.01726 0.01793 0.02269 0.01016 0.00987 0.01010 0.01000 0.01192 0.00576 0.00578 0.00588 0.00598 0.00695

Oracle 0.01548 0.01541 0.01541 0.01554 0.01554 0.00950 0.00944 0.00944 0.00897 0.00897 0.00552 0.00555 0.00555 0.00538 0.00538

m2 B/O 1.088 1.094 1.120 1.154 1.460 1.071 1.046 1.070 1.115 1.330 1.044 1.041 1.059 1.110 1.291

Backfitting 0.01805 0.01756 0.01806 0.01852 0.02374 0.01094 0.01088 0.01084 0.01105 0.01308 0.00657 0.00651 0.00657 0.00670 0.00772

Oracle 0.01635 0.01604 0.01609 0.01628 0.01633 0.01014 0.01025 0.00998 0.00996 0.00996 0.00622 0.00617 0.00614 0.00616 0.00612

B/O 1.104 1.095 1.123 1.138 1.454 1.079 1.062 1.086 1.109 1.314 1.056 1.055 1.071 1.088 1.262

Fig 1. The real lines, dashed lines and dotted lines show the true curve, backfitting estimates and oracle estimates, respectively. The left, center and right panels represent fitted curves for the data sets that produce 25%, 50% and 75% quantiles for the distance between the backfitting and the oracle estimator in Monte Carlo simulations with ρ = 0.5 and 200 observations.

Additive isotone regression

187

is well supported by our asymptotic theory and by the simulations.

Appendix: Proofs A.1. Proof of Theorem 1 The proof of Theorem 1 is divided into several lemmas. Lemma 3. For j = 1, . . . , d it holds that sup n−2/9 ≤uj ≤1−n−2/9

|m  j (uj ) − mj (uj )| = OP [(log n)n−2/9 ].

Proof. Because εi has subexponential tails (see (A3)) we get that sup1≤i≤n |εi | =  j (uj )| = OP (log n). We now OP (log n). This implies that max1≤j≤d sup0≤uj ≤1 |m consider the regression problem Y i /(log n) = c/(log n) + m1 (X1i )/(log n) + . . . + md (Xdi )/(log n) + εi /(log n).

Now, in this model the least squares estimators of the additive components are bounded and therefore we can use the entropy bound for bounded monotone functions (see e.g. (2.6) in van de Geer [39] or Theorem 2.7.5 in van der Vaart and Wellner [40]). This gives by application of empirical process theory for least squares estimators, see Theorem 9.2 in van de Geer [39] that n

2 1  m  1 (X1i ) − m1 (X1i ) + . . . + m  d (Xdi ) − md (Xdi ) = OP [(log n)2 n−2/3 ]. n i=1

And, using Lemma 5.15 in van de Geer [39], this rate for the empirical norm can be replaced by the L2 norm:
2 [m  1 (u1 ) − m1 (u1 ) + . . . + m  d (ud ) − md (ud )] p(u) du = OP [(log n)2 n−2/3 ].

Because p is bounded from below (see (A2)) this implies
2 [m  1 (u1 ) − m1 (u1 ) + . . . + m  d (ud ) − md (ud )] du = OP [(log n)2 n−2/3 ].

Because of our norming assumption (2) for m  j and mj the left hand side of the last equality is equal to

2 2  d (ud ) − md (ud )] dud . [m  1 (u1 ) − m1 (u1 )] du1 + . . . + [m This gives (9)

max

1≤j≤d




[m  j (uj ) − mj (uj )] duj = OP [(log n)2 n−2/3 ]. 2

We now use the fact that for j = 1, . . . , d the derivatives mj are bounded. This gives together with the last bound the statement of Lemma 3.

E. Mammen and K. Yu

188

We now define localized estimators m  OR  j,loc . They are defined as m  OR j j,loc and m 1/γ and m  j but now the sum of squares runs only over indices i with xj − (log n) ×  OR minimizes n−2/(9γ) cn ≤ Xji ≤ xj + (log n)1/γ n−2/(9γ) cn , i.e. m j,loc 

i:|Xji −xj |≤(log n)1/γ n−2/(9γ) cn

and m  j,loc minimizes



i:|Xji −xj |≤(log n)1/γ n−2/(9γ) cn



i 2 mj (Xji ) + εi − m  OR j,loc (Xj )



Y i −

 l=j

2

m  l (Xli ) − m  j,loc (Xji ) .

Here cn is a sequence with cn → ∞ slowly enough (see below). We now argue that

(10)

m  j,loc (xj ) = m  j (xj ) for j = 1, . . . , d and 0 ≤ xj ≤ 1 with probability tending to 1.

This follows from Lemma 3, the fact that mj fulfills (A1) and the representation (compare (4)): (11)

(12)

m  j (xj ) = max

min

xj ≤v≤1

0≤u≤xj

m  j,loc (xj ) =



i:u≤Xji ≤v

Yji

#{i : u ≤ Xji ≤ v}

max

xj −(log n)1/γ n−2/(9γ) cn ≤u≤xj

,



i:u≤Xji ≤v

min

xj ≤v≤xj +(log n)1/γ n−2/(9γ) cn

Yji

#{i : u ≤ Xji ≤ v}

 with Yji = Y i − l=j m  l (Xli ). Here #A denotes the number of elements of a set A. Proceeding as in classical discussions of the case d = 1 one gets: (13)

m  OR  OR j,loc (xj ) = m j (xj ) for j = 1, . . . , d and 0 ≤ xj ≤ 1 with probability tending to 1.

We now consider the functions   j (uj , xj ) = n−1 M Yji − n−1 Yji , i:Xji ≤uj

jOR (uj , xj ) = n−1 M Mj (uj , xj ) = n−1



i:Xji ≤uj



i:Xji ≤uj

i:Xji ≤xj



mj (Xji ) + εi − n−1 mj (Xji ) + εi ,

mj (Xji ) − n−1



i:Xji ≤xj

mj (Xji ).

i:Xji ≤xj

For xj − (log n)1/γ n−2/(9γ) cn ≤ uj ≤ xj + (log n)1/γ n−2/(9γ) cn we consider the j (uj , xj ), M OR (uj , xj ) or Mj (uj , xj ), functions that map #{i : Xji ≤ uj } onto M j OR respectively. Then we get m  j,loc (xj ), m  j,loc (xj ) and mj (xj ) as the slopes of the greatest convex minorants of these functions at uj = xj . We now show the following lemma.

Additive isotone regression

189

Lemma 4. For α > 0 there exists a β > 0 such that uniformly for 1 ≤ l, j ≤ d, 0 ≤ xj ≤ 1 and xj − (log n)1/γ n−2/(9γ) cn ≤ uj ≤ xj + (log n)1/γ n−2/(9γ) cn (14) (15)

OR (uj , xj ) − Mj (uj , xj ) M j

= OP ({|uj − xj | + n−α }1/2 n−1/2 (log n)β ), j (uj , xj ) − M OR (uj , xj ) M j  
    [m  l (ul ) − ml (ul )] pXl |Xj (ul |Xji ) dul n−1  − =− l=j

(16)

i:Xji ≤uj

i:Xji ≤xj

+ OP ({|uj − xj | + n−α }2/3 n−13/27 (log n)β ), 
  −1   n [m  l (ul ) − ml (ul )] pXl |Xj (ul |Xji ) dul − 

i:Xji ≤uj

i:Xji ≤xj



= n−1 #{i : Xji ≤ uj } − #{i : Xji ≤ xj }
× [m  l (ul ) − ml (ul )] pXl |Xj (ul |xj ) dul

+ OP ({|uj − xj | + n−1 }n−2ρ/(9γ) (log n)β ).

Proof. Claim (14) is a standard result on partial sums. Claim (16) directly follows from (A2). For a proof of claim (15) we use the following result: For a constant C suppose that ∆ is a difference of monotone functions on [0, 1] with uniform bound supz |∆(z)| ≤ C and that Z 1 , . . . , Z k is a triangular array of independent random variables with values in [0, 1]. Then it holds uniformly over all functions ∆ k 

∆(Z i ) − E[∆(Z i )] = OP (k 2/3 ),

i=1

see e.g. van de Geer [39]. This result can be extended to l 

∆(Z i ) − E[∆(Z i )] = OP (k 2/3 ),

i=1

uniformly for l ≤ k and for ∆ a difference of monotone functions with uniform bound supz |∆(z)| ≤ C. More strongly, one can show an exponential inequality for the left hand side. This implies that up to an additional log-factor the same rate applies if such an expansion is used for a polynomially growing number of settings with different choices of k, Z i and ∆. We apply this result, conditionally given Xj1 , . . . , Xjn , with Z i = Xli and ∆(u) =  l (u) − ml (u)]/(n−2/9 log n). The last factor is justified I[n−2/9 ≤ u ≤ 1 − n−2/9 ][m by the statement of Lemma 3. This will be done for different choices of k ≥ n1−α . Furthermore, we apply this result with Z i = Xli and ∆(u) = {I[0 ≤ u < n−2/9 ] +  l (u) − ml (u)]/(log n) and k ≥ n1−α . This implies claim I[1 − n−2/9 < u ≤ 1]}[m (15). We now show that Lemma 4 implies the following lemma.

Lemma 5. Uniformly for 1 ≤ j ≤ d and n−1/3 ≤ xj ≤ 1 − n−1/3 it holds that 
OR (17) m  j (xj ) = m  j (xj )− [m  l (ul ) − ml (ul )] pXl |Xj (ul |xj )dul +oP (n−1/3 ) l=j

E. Mammen and K. Yu

190

and that with a constant c > 0 uniformly for 1 ≤ j ≤ d and 0 ≤ xj ≤ n−1/3 or 1 − n−1/3 ≤ xj ≤ 1 
m  j (xj ) = m  OR (x ) − [m  l (ul ) − ml (ul )] pXl |Xj (ul |xj )dul j j l=j

(18)

+ oP (n−2/9 (log n)c ).

Proof. For a proof of (17) we use that for n−1/3 ≤ xj ≤ 1 − n−1/3 (19) (20) (21) where

m −  j (xj ) ≤ m + j (xj ) ≤ m j (xj ) with probability tending to 1,

m  OR,− (xj ) ≤ m  OR  OR,+ (xj ) with probability tending to 1, j (xj ) ≤ m j j sup m  OR,+ (xj ) − m  OR,− (xj ) = oP (n−1/3 ), j j

0≤xj ≤1

m − j (xj )

=

m + j (xj ) =

m  OR,− (xj ) = j m  OR,+ (xj ) j

=

max

xj −en ≤u≤xj −dn

max

xj −en ≤u≤xj

max

max

xj ≤v≤xj +en

min

xj +dn ≤v≤xj +en

xj −en ≤u≤xj −dn

xj −en ≤u≤xj

min

min

xj ≤v≤xj +en

min

xj +dn ≤v≤xj +en



i:u≤Xji ≤v

Yji

#{i : u ≤ Xji ≤ v}  Y i i i:u≤Xj ≤v

#{i : u ≤ 

Xji

i:u≤Xji ≤v



j

≤ v}

,

,

mj (Xji ) + εi

#{i : u ≤ Xji ≤ v}

i:u≤Xji ≤v

mj (Xji ) + εi

#{i : u ≤ Xji ≤ v}

, ,

compare (11) and (12). Here, en = (log n)1/γ n−2/(9γ) cn and dn is chosen as dn = n−δ with 1/3 < δ < 4/9. Claims (19) and (20) follow immediately from the definitions of the considered quantities and (10) and (13). Claim (21) can be established by using well known properties of the isotone least squares estimator. Now, (15),(16),(19) and (20) imply that uniformly for 1 ≤ j ≤ d and n−1/3 ≤ xj ≤ 1 − n−1/3 
OR,± ± m  j (xj ) = m j (xj ) − [m  l (ul ) − ml (ul )] pXl |Xj (ul |xj )dul + oP (n−1/3 ). l=j

This shows claim (17). For the proof of (18) one checks this claim separately for n−7/9 (log n)−1 ≤ xj ≤ n−1/3 or 1 − n−1/3 ≤ xj ≤ 1 − n−7/9 (log n)−1 (case 1) and for 0 ≤ xj ≤ n−7/9 (log n)−1 or 1 − n−7/9 (log n)−1 ≤ xj ≤ 1 (case 2). The proof for Case 1 is similar to the proof of (17). For the proof in Case 2 one considers the set Ij,n = {i : 0 ≤ Xji ≤ n−7/9 (log n)−1 }. It can be easily checked that with probability tending to 1 it holds that n−2/9 ≤ Xli ≤ 1 − n−2/9 . Therefore it holds  l (Xli ) − ml (Xli )| = OP [(log n)n−2/9 ], see Lemma 3. Therefore for that supi∈Ij,n |m 0 ≤ xj ≤ n−7/9 (log n)−1 the estimators m  j (xj ) and m  OR j (xj ) are local averages of −2/9 ]. Thus also the difference of quantities that differ by terms of order OP [(log n)n −2/9  OR (x ) is of order O [(log n)n ]. This shows (18) for Case 2. m  j (xj ) and m j P j We now show that Lemma 5 implies the statement of the theorem.

Additive isotone regression

191

Proof of Theorem 1. We rewrite equations (17) and (18) as (22)

m  =m  OR + H(m  − m) + ∆,

where m,  m  OR and ∆ are tuples of functions m  j, m  OR or ∆j , respectively. For ∆j j it holds that sup

(23)

n−1/3 ≤xj ≤1−n−1/3

(24)

|∆j (xj )| = oP (n−1/3 ),

sup |∆j (xj )| = oP (n−2/9 (log n)c ).

0≤xj ≤1

Furthermore, H is the linear integral operator that corresponds to the linear map in (17) and (18). For function tuples  f we denote by N f the normalized function tuple with (N f )j (xj ) = fj (xj ) − fj (uj )duj and we introduce the pseudo norms
f 22 = [f1 (x1 ) + . . . + fd (xd )]2 p(x) dx, f ∞ = max

sup |fj (xj )|.

1≤j≤d 0≤xj ≤1

Here pj is the marginal density of Xji and p is the joint density of X i . We make use of the following properties of H. On the subspace F0 = {f : f = N f } the operator H has bounded operator norm: (25)

sup f ∈F0 ,f 2 =1

Hf 2 = O(1).

For the maximal eigenvalue λmax of H, it holds that (26)

λmax < 1.

Claim (25) follows directly from the boundedness of p. Claim (26) can be seen as follows. Compare also with Yu, Park and Mammen [42]. A simple calculation gives

2 2 (27) (m1 (u1 ) + · · · + md (ud )) p(u) du = m 2 = mT (I − H)m(u)p(u)du. Let λ be an eigenvalue of H and mλ be an eigen(function)vector corresponding to λ. With (27), we have

2 T mλ 2 = mλ (I − H)mλ (u)p(u)du = (1 − λ) mTλ mλ (u)p(u)du. Thus, the factor 1 − λ must be strictly positive, i.e. λ < 1. This implies I − H is invertible and hence we get that N (m  − m) = (I − H)−1 N (m  OR − m) + (I − H)−1 N ∆.

Here we used that because of (22)

N (m  − m) = N (m  OR − m) − N H(m  − m) + N ∆

= N (m  OR − m) − HN (m  − m) + N ∆.

E. Mammen and K. Yu

192

We now use (I − H)−1 = I + H + H(I − H)−1 H, (I − H)−1 = I + H(I − H)−1 . This gives  OR − m) N (m  − m) = N (m  OR − m) + N ∆ + HN (m −1 OR + H(I − H) HN (m  − m) + H(I − H)−1 ∆.

We now use that (28) (29)

 OR − m) ∞ = oP (n−1/3 ), HN (m  OR − m) 2 ≤ HN (m sup Hf ∞ = O(1).

f ∈F0 ,f ∞ =1

Claim (28) follows because m  OR is a local average of the data, compare also Groeneboom [12], Groeneboom, Lopuhaa and Hooghiemstra [14] and Durot [8]. Claim (29) follows by a simple application of the Cauchy Schwarz inequality, compare also (85) in Mammen, Linton and Nielsen [28]. This implies that

Thus,

N (m  − m) − N (m  OR − m) − N ∆ ∞ = oP (n−1/3 ). sup n−1/3 ≤xj ≤1−n−1/3

|N (m  −m  OR )j (xj )| = oP (n−1/3 ),

sup |N (m  −m  OR )j (xj )| = oP (n−2/9 (log n)c )

0≤xj ≤1

This implies the statement of Theorem 1. A.2. Proof of Theorem 2

For a given closed convex cone K, we call K ∗ ≡ {f : f, g ≤ 0 for all g ∈ K} the dual cone of K. It is clear that K ∗ is also a convex cone and K ∗∗ = K. It is pointed out in Barlow and Brunk [3] that if P is a projection onto K then I − P is a projection onto K ∗ where I is the identity operator. Let Pj be a projection onto Hj then Pj∗ ≡ I − Pj is a projection onto Hj∗ . The backfitting procedure (7) to solve the minimization problem (5) corresponds in the dual problem to an algorithm introduced in Dykstra [9]. See also Gaffke and Mathar [11]. We now explain this relation. Let Hj , j = 1, . . . , d, be sets of monotone vectors in Rn with respect to the orders of Xj and Pj = Π(·|Hj ). Denote the residuals in algorithm (7) after the k-th cycle in the r-th iteration with h(r,k) . Then, we have h(1,1) = Y − g1 = P1∗ Y, [1]

h(1,2) = Y − g1 − g2 = P1∗ Y − P2 P1∗ Y = P2∗ P1∗ Y, .. . [1]

(30)

[1]

h(1,d) = Y − g1 − · · · − gd = Pd∗ · · · P1∗ Y; [1]

[1]

Additive isotone regression

193

h(r,1) = Y − g1 − g2 − · · · − gd = P1∗ (Y − g2 .. . [r] [r] [r−1] [r−1] h(r,k) = Y − g1 − · · · − gk − gk+1 − · · · − gd [r]

[r−1]

[r−1]

[r−1]

[r−1]

− · · · − gd

),

= Pk∗ (Y − g1 − · · · − gk−1 − gk+1 − · · · − gd ), .. . [r] [r] [r] [r] (31) h(r,d) = Y − g1 − · · · − gd = Pd∗ (Y − g1 − · · · − gd−1 ). [r]

[r]

[r−1]

[r−1]

[r]

With the notation Ir,k ≡ −gk for the incremental changes at the k-th cycle in the r-th iteration, equations (30) and (31) form a Dykstra algorithm to solve the following optimization problem: (32)

min ∗

h∈H1 ∩···∩Hd∗

n 

(Y i − hi )2 .

i=1

Denote the solutions of (32) with h∗ . Theorem 3.1 of Dykstra [9] shows that h(r,j) converges to h∗ as r → ∞ for j = 1, . . . , d. From the dual property, it is well known g ∗ = Y − h∗ and also it is clear that g(r,j) = Y − h(r,j) for j = 1, . . . , d. Since h(r,j) converges to h∗ , g(r,j) converge to g ∗ as r → ∞ for j = 1, . . . , d. The [r] convergence of gj follows from Lemma 4.9 of Han [17]. References [1] Bacchetti, P. (1989). Additive isotonic models. J. Amer. Statist. Assoc. 84 289–294. [2] Barlow, R. E., Bartholomew, D. J., Bremner, J. M. and Brunk, H. D. (1972). Statistical Inference under Order Restrictions. Wiley, New York. [3] Barlow, R. E. and Brunk, H. D. (1972). The isotonic regression problem and its dual. J. Amer. Statist. Assoc. 67 140–147. [4] Brunk, H. D. (1958). On the estimation of parameters restricted by inequalities. Ann. Math. Statist. 29 437–454. [5] Buja, A., Hastie, T. and Tibshirani, R. (1989). Linear smoothers and additive models. Ann. Statist. 17 454–510. [6] Carrasco, M., Florens, J.-P. and Renault, E. (2006). Linear inverse problems in structural econometrics: Estimation based on spectral decomposition and regularization. In Handbook of Econometrics (J. Heckman and E. Leamer, eds.) 6. North Holland. [7] De Boer, W. J., Besten, P. J. and Ter Braak, C. F. (2002). Statistical analysis of sediment toxicity by additive monotone regression splines. Ecotoxicology 11 435–50. [8] Durot, C. (2002). Sharp asymptotics for isotonic regression. Probab. Theory Related Fields 122 222–240. [9] Dykstra, R. L. (1983). An algorithm for restricted least squares regression. J. Amer. Statist. Assoc. 78 837–842. [10] Fan, J. and Jiang, J. (2005). Nonparametric inference for additive models. J. Amer. Statist. Assoc. 100 890–907. [11] Gaffke, N. and Mathar, R. (1989). A cyclic projection algorithm via duality. Metrika 36 29–54.

194

E. Mammen and K. Yu

[12] Groeneboom, P. (1985). Estimating a monotone density. In Proceedings of the Berkeley Conference in Honor of Jerzy Neuman and Jack Kiefer (L. M. LeCam and R. A. Olshen, eds.) 2 539–555. Wadsworth, Belmont, CA. [13] Groeneboom, P. (1989). Brownian motions with a parabolic drift and airy functions. Probab. Theory and Related Fields 81 79–109. [14] Groeneboom, P., Lopuhaa, H. P. and Hooghiemstra, G. (1999). Asymptotic normality of the L1 -error of the Grenander estimator. Ann. Statist. 27 1316–1347. [15] Haag, B. (2006). Nonparametric estimation of additive multivariate diffusion processes. Working paper. [16] Haag, B. (2006). Nonparametric regression tests based on additive model estimation. Working paper. [17] Han, S.-P. (1988). A successive projection method. Mathematical Programming 40 1–14. [18] Hastie, T. J. and Tibshirani, R. J. (1990). Generalized Additive Models. Chapman and Hall, London. [19] Horowitz, J., Klemela, J. and Mammen, E. (2006). Optimal estimation in additive regression models. Bernoulli 12 271–298. [20] Horowitz, J. and Mammen, E. (2004). Nonparametric estimation of an additive model with a link function. Ann. Statist. 32 2412–2443. [21] Horowitz, J. and Mammen, E. (2007). Rate-optimal estimation for a general class of nonparametric regression models. Ann. Statist. To appear. [22] Horowitz, J. and Mammen, E. (2006). Nonparametric estimation of an additive model with an unknown link function. Working paper. [23] Leurgans, S. (1982). Asymptotic distributions of slope-of-greatest-convexminorant estimators. Ann. Statist. 10 287–296. [24] Linton, O. B. and Mammen, E. (2003). Nonparametric smoothing methods for a class of non-standard curve estimation problems. In Recent Advances and Trends in Nonparametric Statistics (M.G. Akritas and D. N. Politis, eds.). Elsevier, Amsterdam. [25] Linton, O. and Mammen, E. (2005). Estimating semiparametric ARCH (∞) models by kernel smoothing methods. Econometrika 73 771–836. [26] Linton, O. and Mammen, E. (2007). Nonparametric transformation to white noise. Econometrics. To appear. [27] Mammen, E. (1991). Nonparametric regression under qualitative smoothness assumptions. Ann. Statist. 19 741–759. [28] Mammen, E., Linton, O. B. and Nielsen, J. P. (1999). The existence and asymptotic properties of a backfitting projection algorithm under weak conditions. Ann. Statist. 27 1443–1490. [29] Mammen, E. and Park, B. U. (2005). Bandwidth selection for smooth backfitting in additive models. Ann. Statist. 33 1260–1294. [30] Mammen, E. and Park, B. U. (2006). A simple smooth backfitting method for additive models. Ann. Statist. 34 2252–2271. [31] Mammen, E. and Sperlich, S. (2006). Additivity Tests Based on Smooth Backfitting. Working paper. [32] Morton-Jones, T., Diggle, P., Parker, L., Dickinson, H. O. and Binks, K. (2000). Additive isotonic regression models in epidemiology. Stat. Med. 19 849–59. [33] Nielsen, J. P. and Sperlich, S. (2005). Smooth backfitting in practice. J. Roy. Statist. Soc. Ser. B 67 43–61. [34] Opsomer, J. D. and Ruppert, D. (1997). Fitting a bivariate additive model

Additive isotone regression

195

by local polynomial regression. Ann. Statist. 25 185–211. [35] Opsomer, J. D. (2000). Asymptotic properties of backfitting estimators. J. Multivariate Analysis 73 166–179. [36] Robertson, T., Wright, F. and Dykstra, R. (1988). Order Restricted Statistical Inference. Wiley, New York. ¨rdle, W. (1999). Integration and [37] Sperlich, S., Linton, O. B. and Ha Backfitting methods in additive models: Finite sample properties and comparison. Test 8 419–458. [38] Stone, C. J. (1985). Additive regression and other nonparametric models. Ann. Statist. 13 689–705. [39] van de Geer, S. (2000). Empirical Processes in M-Estimation. Cambridge University Press. [40] van der Vaart, A. and Wellner, J. (1996). Weak Convergence and Empirical Processes. Springer, New York. [41] Wright, F. T. (1981). The asymptotic behaviour of monotone regression estimates. Ann. Statist. 9 443–448. [42] Yu, Kyusang, Park, B. U. and Mammen, E. (2007). Smooth backfitting in generalized additive models. Ann. Statist. To appear.

IMS Lecture Notes–Monograph Series Asymptotics: Particles, Processes and Inverse Problems Vol. 55 (2007) 196–203 c Institute of Mathematical Statistics, 2007
DOI: 10.1214/074921707000000364

A note on Talagrand’s convex hull concentration inequality David Pollard1 Yale University Abstract: The paper reexamines an argument by Talagrand that leads to a remarkable exponential tail bound for the concentration of probability near a set. The main novelty is the replacement of a mysterious calculus inequality by an application of Jensen’s inequality.

1. Introduction Let X be a set equipped with a sigma-field A. For each vector w = (w1 , . . . , wn ) in Rn+ , the weighted Hamming distance between two vectors x = (x1 , . . . , xn ) and y = (y1 , . . . , yn ), in X n is defined as 
1 if xi
= yi dw (x, y) := wi hi (x, y) where hi (x, y) = i≤n 0 otherwise. For a subset A of X n and x ∈ X n , the distances dw (x, A) and D(x, A) are defined by dw (x) := inf{y ∈ A : dw (x, y)} and D(x, A) := supw∈W dw (X, A), where the supremum is taken over all weights in the set  W :=

2

(w1 , . . . , wn ) : wi ≥ 0 for each i and |w| :=




i≤n

wi2



≤1 .

Talagrand ([10], Section 4.1) proved a remarkable concentration inequality for random elements X = (X1 , . . . , Xn ) of X n with independent coordinates and subsets A ∈ An : (1)

P{X ∈ A}P{D(X, A) ≥ t} ≤ exp(−t2 /4)

for all t ≥ 0.

As Talagrand showed, this inequality has many applications to problems in combinatorial optimization and other areas. See [12], Chapter 6 of [7] and Section 4 of [6], for further examples. There has been a strong push in the literature to establish concentration and deviation inequalities by “more intuitive” methods, such as those based on the tensorization, as in [1, 3–5]. I suspect the search for alternative approaches has been driven by the miraculous roles played by some elementary inequalities in Talagrand’s proofs. 1 Statistics Department, Yale University, Box 208290 Yale Station, New Haven, CT 06520-8290, USA, e-mail: [email protected]; url: http://www.stat.yale.edu/~pollard/ AMS 2000 subject classifications: Primary 62E20; secondary 60F05, 62G08, 62G20. Keywords and phrases: Concentration of measure, convex hull, convexity.

196

Talagrand’s concentration inequality

197

Talagrand [10] used an induction on n to establish his result. He invoked a slightly mysterious inequality in the inductive step, inf u

0≤θ≤1

−θ

exp



(1 − θ)2 4



≤2−u

for 0 < u < 1,

which he borrowed from [2] – see Talagrand’s discussion following his Lemma 4.1.3 for an explanation of how those authors generalized the earlier result from [8]. There is similar mystery in Talagrand’s Lemma 4.2.1, which (in my notation) asserts that   1+c−u sup u−θ/c exp ψc (1 − θ) ≤ c 0≤θ≤1

for 0 < u < 1,

where ψc is defined in equation (6) below. (My ψc (u) equals Talagrand’s ξ(α, u) with α = 1/c.) Talagrand [9] had used this inequality to further generalize the result of [2], giving the concentration inequality listed as Theorem 4.2.4 in [10]. It was my attempts to understand how he arrived at his ξ(α, u) function that led me to the concavity argument that I present in the note. It is my purpose to modify Talagrand’s proof so that the inductive step becomes a simple application of the H¨ older inequality (essentially as in the original proof) and the Jensen inequality. Most of my methods are minor variations on the methods in the papers just cited; my only claim of originality is for the recognition that the mysterious inequalities can be replaced by more familiar appeals to concavity. See also the Remarks at the end of this Section. The distance D(x, A) has another representation, as a minimization over a convex subset of [0, 1]. Write h(x, y) for the point of {0, 1}n with ith coordinate hi (x, y). For each fixed x, the function h(x, ·) maps A onto a subset h(x, A) := {h(x, y) : y ∈ A} of {0, 1}n . The convex hull, co (h(x, A)), of h(x, A) in [0, 1]n is compact, and D(x, A) = inf{|ξ| : ξ ∈ co (h(x, A))}.  Each point ξ of co (h(x, A)) can be written as h(x, y) ν(dy) for a ν in the set P(A) of all probability measures for which ν(A) = 1. That is, ξi = ν{y ∈ A : yi
= xi }. Thus
 2 ν{y ∈ A : yi
= xi } . (2) D(x, A)2 = inf i≤n

ν∈P(A)

Talagrand actually proved inequality (1) by showing that (3)

P{X ∈ A}P exp

1

2 4 D(X, A)



≤ 1.

He also established an even stronger result, in which the D(X, A)2 /4 in (3) is replaced by a more complicated distance function. For each convex, increasing function ψ with ψ(0) = 0 = ψ  (0) define (4)

Fψ (x, A) :=

inf

ν∈P(A)




i≤n

  ψ ν{y ∈ A : yi
= xi } ,

For each c > 0 ([10], Section 4.2) showed that (5)

  (P{X ∈ A})c P exp Fψc (X, A) ≤ 1,

D. Pollard

198

where

(6)

   (1 − θ) + c ψc (θ) := c−1 (1 − θ) log(1 − θ) − (1 − θ + c) log 1+c   k 2 k−1
Rc + Rc + · · · + Rc θ 1 . = with Rc := k≥2 k (k − 1) c+1 θ2 ≥ . 2 + 2c

As you will see in Section 3, this strange function is actually the largest solution to a differential inequality, ψ  (1 − θ) ≤ 1/(θ2 + θc)

for 0 < θ < 1.

Inequality (5) improves on (3) because D(x, A)2 /4 ≤ Fψ1 (x, A). Following the lead of [10], Section 4.4, we can ask for general conditions on the convex ψ under which an analog of (5) holds with some other decreasing function of P{X ∈ A} as an upper bound. The following slight modification of Talagrand’s theorems gives a sufficient condition in a form that serves to emphasize the role played by Jensen’s inequlity. Theorem 1. Suppose γ is a decreasing function with γ(0) = ∞ and ψ is a convex function. Define Gψ (η, θ) := ψ(1 − θ) + θη and Gψ (η) := inf 0≤θ≤1 Gψ (η, θ) for η ∈ R+ . Suppose (i) r → exp(Gψ (γ(r) − γ(r0 ))) is concave on [0, r0 ], for each r0 ≤ 1 (ii) (1 − p)eψ(1) + p ≤ eγ(p) for 0 ≤ p ≤ 1. Then P exp(Fψ (X, A)) ≤ exp(γ(P{X ∈ A})) for every A ∈ An and every random element X of X n with independent components. The next lemma, a more general version of which is proved in Section 3, leads to a simple sufficient condition for the concavity assumption (i) of Theorem 1 to hold. Lemma 2 (Concavity lemma). Suppose ψ : [0, 1] → R+ is convex and increasing, with ψ(0) = 0 = ψ  (0) and ψ  (θ) > 0 for 0 < θ < 1. Suppose ξ : [0, r0 ] → R+ ∪ {∞} is continuous and twice differentiable on (0, r0 ). Suppose also that there exists some finite constant c for which ξ  (r) ≤ cξ  (r)2 for 0 < r < r0 . If ψ  (1 − θ) ≤ 1/(θ2 + θc)

for 0 < θ < 1

then the function r → exp(Gψ (ξ(r))) is concave on [0, r0 ]. Lemma 2 will be applied with ξ(r) = γ(r) − γ(r0 ) for 0 ≤ r ≤ r0 . As shown in Section 3, the conditions of the Lemma hold for ψ(θ) = θ2 /4 with γ(r) = log(1/r) and also for the ψc from (6) with γ(r) = c−1 log(1/r). Remarks. (i) If γ(0) were finite, the inequality asserted by Theorem 1 could not hold for all nonempty A and all X. For example, if each Xi had a nonatomic distribution and A were a singleton set we would have Fψ (X, A) = nψ(1) almost surely. The quantity P exp(Fψ (X, A)) would exceed exp(γ(0)) for large enough n. It it to avoid this difficulty that we need γ(0) = ∞.

Talagrand’s concentration inequality

199

(ii) Assumption (ii) of the Theorem, which is essentially an assumption that the asserted inequality holds for n = 1, is easy to check if γ is a convex function with γ(1) ≥ 0. For then the function B(p) := exp(γ(p)) is convex with B(1) ≥ 1 and B  (1) = γ  (1)eγ(1) . We have B(p) ≥ (1 − p)eψ(1) + p

for all p in [0, 1]

if B  (1) ≤ 1 − eψ(1) . (iii) A helpful referee has noted that both my specific examples are already covered by Talagrand’s results. He (or she) asked whether there are other (ψ, γ) pairs that lead to other useful concentration inequalities. A good question, but I do not yet have any convincing examples. Actually, I had originally thought that my methods would extend to the limiting case where c tends to zero, leading to an answer to the question posed on page 128 of [10]. Unfortunately my proof ran afoul of the requirement γ(0) = ∞. I suspect more progress might be made by replacing the strong assumption on ψ  from Lemma 2 by something closer to the sufficient conditions presented in Section 3. 2. Proof of Theorem 1 Argue by induction on n. As a way of keeping the notation straight, replace the subscript on Fψ (x, B) by an n when the argument B is a subset of X n . Also, work with the product measure Q = ⊗i≤n Qi for the distribution of X and Q−n = ⊗i
For n = 1 and B ∈ A we have F1 (x, B) = ψ(1){x ∈ / B} + 0{x ∈ B} so that Q1 exp(F1 (x, B)) ≤ (1 − p)eψ(1) + p, where p = Q1 B. Assumption (ii) then gives the desired exp(γ(p)) bound. Now suppose that n > 1 and that the inductive hypothesis is valid for dimensions strictly smaller than n. Write Q as Q−n ⊗ Qn . To simplify notation, write w for x−n := (x1 , . . . , xn−1 ) and z for xn . Define the cross section Az := {w ∈ X n−1 : (w, z) ∈ A} and write Rz for Q−n Az . Define r0 := supz∈X Rz . Notice that r0 ≥ Qzn Rz = QA.

D. Pollard

200

The key to the proof is a recursive bound for Fn : for each x = (w, z) with Az
= ∅, each m with Am
= ∅, and all θ = 1 − θ¯ ∈ [0, 1], (7)

¯ + Fn (x, A) ≤ θFn−1 (w, Az ) + θF ¯ n−1 (w, Am ). ψ(θ)

To establish inequality (7), suppose µz is a probability measure concentrated on Az and µm is a probability measure concentrated on Am . For a θ in [0, 1], ¯ m ⊗ δm , a probability measure concentrated on the subset define ν = θµz ⊗ δz + θµ (Az × {z}) ∪ (Am × {m}) of A. Notice that, for i < n, ¯ m {w ∈ Am : yi
= xi } ν{y ∈ A : yi
= xi } = θµz {w ∈ Az : yi
= xi } + θµ so that, by convexity of ψ,   ¯ ψ ν{yi
= xi } ≤ θψ(µz {w ∈ Az : yi
= xi }) + θψ(µ m {w ∈ Am : yi
= xi });

and (remembering that xn = z),

 θ¯ if z
= m
xn } = ν{y ∈ A : yn = 0 otherwise

¯ ≤ θ.

Thus ¯ +θ Fn (x, A) ≤ ψ(θ)




i

  ψ µz {yi
= xi } + θ¯

i
  ψ µm {yi
= xi } .

The two sums over the first n − 1 coordinates are like those that appear in the definitions of Fn−1 (w, Az ) and Fn−1 (w, Az ). Indeed, taking an infimum over all µz ∈ P(Az ) and µm ∈ P(Am ) we get the expression on the right-hand side of (7). Take exponentials of both sides of (7) then integrate out with respect to Q−n ¯ over the w component. For 0 < θ < 1 invoke the H¨ older inquality, Q−n U θ V θ ≤ θ¯ θ (Q−n U ) (Q−n V ) , with U = exp(Fn−1 (w, Az )) and V = exp(Fn−1 (w, Am )), for a fixed m. For each z with Az
= ∅ we get   Q−n exp Fn ((w, z), A) (8)  θ    ¯ ¯  ≤ eψ(θ) Q−n exp Fn−1 (w, Az ) Q−n exp Fn−1 (w, Am ) θ . The inequality also hold in the extreme cases where θ = 0 or θ = 1, by continuity. The inductive hypothesis bounds the last product by     ¯ + θγ(Rz ) + θγ(R ¯ exp ψ(θ) m ) = exp γ(Rm ) + G(γ(Rz ) − γ(Rm ), θ) .

The exponent is a decreasing function of Rm . Take an infimum over m, to replace γ(Rm ) by γ(r0 ). Then take an infimum over θ to get     Q−n exp Fn ((w, z), A) ≤ exp γ(r0 ) + G(ξ(Rz )) (9) where ξ(r) := γ(Rz ) − γ(r0 ) for 0 ≤ r ≤ r0 .

If the cross section Az is empty, the set P(Az ) is empty. The argument leading from (7) to (9) still works if we fix θ equal to zero throughout, giving the bound     Qw if Az = ∅. −n exp Fn (x, A) ≤ exp γ(r0 ) + ψ(1)

Thus the inequality (9) also holds with Rz = 0 when Az = ∅, because ξ(0) = γ(0) − γ(r0 ) = ∞ and G(∞) = ψ(1).

Talagrand’s concentration inequality

201

By Assumption (i), the function r → exp(G(ξ(r))) is concave on [0, r0 ]. Integrate both sides of (9) with respect to Qn to average out over the z variable. Then invoke Jensen’s inequality and the fact that Qn Rz = QA, to deduce that      Q exp Fn (x, A) ≤ exp γ(r0 ) + G γ(QA) − γ(r0 ) .

Finally, use the inequality G(η) ≤ η to bound the last expression by exp(γ(QA)), thereby completing the inductive step. Remark. Note that it is important to integrate with respect to Qn before using the bound on G: the upper bound exp(−γ(Rz )) is a convex function of Rz , not concave. 3. Proof of the concavity lemma I will establish a more detailed set of results than asserted by Lemma 2. Invoke  the monotonicity and continuity of ψ  to define g(η) as the solution to ψ  1 − g(η) = η if 0 ≤ η < ψ  (1) and g(η) = 0 if ψ  (1) ≤ η. Then the following assertions are true. (I drop the ψ subscripts for notational simplicity.) (i)

   ψ 1 − g(η) + ηg(η) G(η) = ψ(1)

for 0 ≤ η < ψ  (1) for ψ  (1) ≤ η

(ii) G is increasing and concave, with a continuous, decreasing first derivative g. In particular, G(0) = 0 and G (0) = g(0) = 1. (iii) G (η) = g  (η) = −[ψ  (1 − g(η))]−1 for 0 < η < ψ  (1). (iv) G(η) ≤ η for all η ∈ R+ . (v) Suppose ξ : J → R+ is a convex function defined on a subinterval J of the real line, with ξ 
= 0 on the interior of J. Suppose 1 ≥ g(ξr )2 + g(ξr )ξ  (r)/ξ  (r)2 , − ξr )

ψ  (1

for all r in the interior of J for which ξr := ξ(r) ∈ (0, 1). Then r → exp(G(ξ(r))) is a concave function on J. Proof of (i) through (iv). The fact that G is concave and increasing follows from its definition as an infimum of increasing linear functions of η. (It would also follow from the fact that G (η) = g(η), which is nonnegative and decreasing.) Replacement of the infimum over 0 ≤ θ ≤ 1 by the value at θ = 1 gives the inequality G(η) ≤ η. If η ≥ ψ  (1), the derivative −ψ  (1 − θ) + η is nonnegative on [0, 1], which ensures that the infimum is achieved at θ = 1. If 0 < η < ψ  (1), the infimum is achieved at the zero of the derivative, θ = g(η). Differentiation of the defining equality ψ  (1 − g(η)) = η then gives the expression for g  (η). Similarly   G (η) = −ψ  1 − g(η) g  (η) + ηg  (η) + g(η) = g(η).

The infimum that defines G(0) is achieved at g(0) = 1, which gives G(0) = ψ(0) = 0. Continuity of g at 0 then gives G (0) = g(0) = 1.

D. Pollard

202

Proof of (v). Note that the function L(r) := exp(G(ξ(r))) is continuous on J and takes the value eψ(1) for all r at which ξ(r) ≥ ψ  (1). The second derivative L (r) exists except possibly at points r for which ξ(r) = ψ  (1). In particular, L (r) = 0 when ξ(r) > ψ  (1) and   L (r) = g  (ξr )(ξr )2 + g(ξr )ξr + g(ξr )2 (ξr )2 L(r) for 0 < ξr < ψ  (1). From (iii) and the positivity of L, the last expression is ≤ 0 if and only if −

(ξr )2 + g(ξr )ξr + g(ξr )2 (ξr )2 ≤ 0. − g(ξr ))

ψ  (1

Divide through by (ξr )2 then rearrange to get the asserted inequality for ψ  . Lemma 2 follows as a special case of (i) through (iv). Special cases. If supr ξ  (r)/ξ  (r)2 ≤ c, with c a positive constant, the inequality from part (v) will certainly hold if ψ  (1 − θ) ≤ (θ2 + cθ)−1

(10)

for all 0 < θ < 1.

This differential inequality can be solved, subject to the constraints 0 = ψ(0) = ψ  (0), by two integrations. Indeed, 

ψ (1 − θ) =



1



ψ (1 − t) dt ≤

θ



1

θ

   θ+c dt −1 =c − log θ + log t2 + ct 1+c

and, with ψc defined by (6), ψ(1 − θ) =



1



ψ (1 − t) dt ≤ c

θ

−1



1

− log t + log θ



t+c 1+c



dt = ψc (1 − θ).

Note that ψc (1 − θ) is the solution to the differential equation ψc (1 − θ) =

1 θ2 + cθ

for all 0 < θ < 1, with ψc (0) = ψc (0) = 0.

It is the largest solution to (10). References [1] Boucheron, S., Lugosi, G. and Massart, P. (2000). A sharp concentration inequality with applications. Random Structures Algorithms 16 277–292. [2] Johnson, W. B. and Schechtman, G. (1991). Remarks on Talagrand’s deviation inequality for Rademacher functions. Lecture Notes in Math. 1470 72–77. [3] Ledoux, M. (1996). On Talagrand’s deviation inequalities for product measures. ESAIM Probab. Statist. 1 63–87. [4] Lugosi, G. (2003). Concentration-of-measure inequalities. Notes from the Summer School on Machine Learning, Australian National University. Available at http://www.econ.upf.es/˜lugosi/. [5] Massart, P. (2003). Saint-Flour Lecture Notes 2003: Concentration Inequalities and Model Selection. Available at http://www.math.u-psud.fr/˜massart/.

Talagrand’s concentration inequality

203

[6] McDiarmid, C. (1998). Concentration. In Probabilistic Methods for Algorithmic Discrete Mathematics (M. Habib, C. McDiarmid, J. Ramirez-Alfonsen and B. Reed, eds.) 195–248. Springer, Berlin. [7] Steele, J. M. (1997). Probability Theory and Combinatorial Optimization. SIAM, Philadelphia, PA. [8] Talagrand, M. (1988). An isoperimetric theorem on the cube and the Kintchine-Kahane inequalities. Proc. Amer. Math. Soc. 104 905–909. [9] Talagrand, M. (1991). A new isoperimetric inequality for product measure and the tails of sums of independent random variables. Geom. Funct. Anal. 1 211–223. [10] Talagrand, M. (1995). Concentration of measure and isoperimetric inequalities in product spaces. Publ. Math. de l’I.H.E.S. 81 73–205. [11] Talagrand, M. (1996a). New concentration inequalities in product spaces. Invent. Math. 126 505–563. [12] Talagrand, M. (1996b). A new look at independence. Ann. Probab. 24 1–34.

IMS Lecture Notes–Monograph Series Asymptotics: Particles, Processes and Inverse Problems Vol. 55 (2007) 204–233 c Institute of Mathematical Statistics, 2007
DOI: 10.1214/074921707000000373

A growth model in multiple dimensions and the height of a random partial order Timo Sepp¨ al¨ ainen1,∗ University of Wisconsin-Madison Abstract: We introduce a model of a randomly growing interface in multidimensional Euclidean space. The growth model incorporates a random order model as an ingredient of its graphical construction, in a way that replicates the connection between the planar increasing sequences model and the one-dimensional Hammersley process. We prove a hydrodynamic limit for the height process, and a limit which says that certain perturbations of the random surface follow the characteristics of the macroscopic equation. By virtue of the space-time Poissonian construction, we know the macroscopic velocity function explicitly up to a constant factor.

1. Introduction We introduce a model of a randomly growing interface, whose construction involves the height of a random partial order. The interface is defined by a height function on d-dimensional Euclidean space, and the related model of random order is in d + 1 dimensional space-time. Our goal is to emulate in higher dimensions the fruitful relationship between the one-dimensional Hammersley process and the model of increasing sequences among planar Poisson points. The connection between Hammersley’s process and increasing sequences was suggested in Hammersley’s paper [15], first utilized by Aldous and Diaconis [1], and subsequently in papers [21, 22, 25, 27]. A review of the use of Hammersley’s process to study increasing sequences appeared in [14], and of the wider mathematical context in [2]. The study of higher dimensional random orders was started by Winkler [30]. The interface process we introduce is defined through a graphical representation which utilizes a homogeneous space-time Poisson point process, and in particular the heights of the partial orders among the Poisson points in space-time rectangles. This definition suggests a natural infinitesimal description, which we verify in a sense. After defining the process, we prove a hydrodynamic limit for the height function. This proceeds in a familiar way, by the path level variational formulation. The deterministic limiting height is the solution of a Hamilton-Jacobi equation given by a Hopf-Lax formula. Next we use this process to prove a limit that in a way generalizes the law of large numbers of a second class particle in one-dimensional systems. In interacting particle systems, a second class particle is the location X(t) of the unique discrepancy between two coupled systems that initially differ by exactly one particle (see [16], part III). This makes sense for example for Hammersley’s process and exclusion type processes. If the particle system lives in one-dimensional space, we can ∗ Research

partially supported by NSF Grants DMS-01-26775 and DMS-04-02231. Department, University of Wisconsin-Madison, Madison, WI 53706-1388, USA e-mail: [email protected] AMS 2000 subject classifications: Primary 60K35; secondary 82C22. Keywords and phrases: characteristics, growth model, hydrodynamic limit, increasing sequences, random order, second-class particle. 1 Mathematics

204

Growth model

205

also look at the system as the height function of an interface, so that the occupation variables of the particle system define the increments of the height function. In terms of the height functions, we start by coupling two systems σ and ζ so that initially ζ = σ to the left of X(0), and ζ = σ + 1 to the right of X(0). Then at all times the point X(t) is the boundary of the set {x : ζ(x, t) = σ(x, t) + 1}. This last idea generalizes naturally to the multidimensional interface model. We couple two height processes σ and ζ that satisfy σ ≤ ζ ≤ σ + 1 at all times, and prove that the boundary of the random set {x : ζ(x, t) = σ(x, t) + 1} follows the characteristics of the macroscopic equation. Laws of large numbers for height functions of asymmetric interface models of the general type considered here have been earlier studied in a handful of papers. A hydrodynamic limit for ballistic deposition was proved in [24], and for models that generalize the exclusion process in [19, 20]. Articles [19, 24] deal with totally asymmetric models, and utilize the path-level variational formulation that generalizes from one-dimensional systems [23]. Article [20] introduces a different approach for partially asymmetric systems. These results are existence results only. In other words convergence to a limiting evolution is shown but nothing explicit about the limit is known, except that it is defined by a Hamilton-Jacobi equation. For partially asymmetric systems in more than one dimension it is presently not even known if the limit is deterministic. Our motivation for introducing a new model is to have a system for which better results could be proved. An advantage over earlier results is that here we can write down explicitly the macroscopic velocity function up to a constant factor. This is because the process is constructed through a homogeneous space-time Poisson process, so we can simultaneously scale space and time. This is not possible for a lattice model. With an (almost) explicit velocity we can calculate macroscopic profiles, for example see what profiles with shocks and rarefaction fans look like. In the earlier cases at best we know that the velocity function is convex (or concave), but whether the velocity is C 1 or strictly convex is a hard open question. Here this question is resolved immediately. Hammersley’s process has been a fruitful model for studying large scale behavior of one-dimensional asymmetric systems, by virtue of its connection with the increasing sequence model. For example, by a combination of the path-level variational construction and the Baik-Deift-Johansson estimates [3], one can presently prove the sharpest out-of-equilibrium fluctuation results for this process [25, 27]. The model introduced in the present paper has a similar connection with a simple combinatorial model, so it may not be too unrealistic to expect some benefit from this in the future. Before the arrival of the powerful combinatorial and analytic approach pioneered in [3], Hammersley’s process was used as a tool for investigating the increasing sequences model. This approach was successful in finding the value of the limiting constant [1, 21] and in large deviations [22]. The proofs relied on explicit knowledge of invariant distributions of Hammersley’s process. A similar motivation is possible for us too, and this time the object of interest would be the height of the random partial order. But currently we have no explicit knowledge of steady states of the process introduced here, so we cannot use the process to identify the limiting constant for the random order model. Recently Cator and Groeneboom [8] developed an approach to the one-dimensional Hammersley process that captures the correct order t1/3 of the current fluctuations across a characteristic. The argument utilizes precise equilibrium calculations and a time reversal that connects maximizing increasing paths with trajectories

T. Sepp¨ al¨ ainen

206

of second class particles. In [4] this method is adapted to the totally asymmetric exclusion process. Whether the idea can be applied in multidimensional settings remains to be seen. Organization of the paper. We begin by reminding the reader of the random partial order model, and then proceed to define the process and state the limit theorems. Proofs follow. A technical appendix at the end of the paper shows that the process has a natural state space that is a complete, separable metric space. 2. The height of a random partial order Fix an integer ν ≥ 2. Coordinatewise partial orders on Rν are defined for points x = (x1 , . . . , xν ) and y = (y1 , . . . , yν ) by (1)

x ≤ y iff xi ≤ yi for 1 ≤ j ≤ ν, and x < y iff xi < yi for 1 ≤ j ≤ ν.

We use interval notation to denote rectangles: (a, b] = {x ∈ Rν : a < x ≤ b} for a < b in Rν , and similarly for [a, b] and the other types of intervals. Consider a homogeneous rate 1 Poisson point process in Rν . A sequence of Poisson points pk , 1 ≤ k ≤ m, is increasing if p1 < p2 < · · · < pm in the coordinatewise sense. For a < b in Rν , let H(a, b) denote the maximal number of Poisson points on an increasing sequence contained in the set (a, b]. Let 1 = (1, 1, . . . , 1) ∈ Rν . (This is the only vector we will denote by a boldface.) Kingman’s subadditive ergodic theorem and simple moment bounds imply the existence of constants cν such that (2)

lim

n→∞

1 H(0, n1) = cν n

a.s.

Presently the only known value is c2 = 2, first proved by Vershik and Kerov [29] and Logan and Shepp [17]. The case ν = 2 is the same as the problem of the longest increasing subsequence of a random permutation, see [2] for a review. Bollob´ as and Winkler [7] proved that cν → e as ν → ∞. The general case is called the ν-dimensional random partial order, and H(0, n1) is the height of the random partial order. The study of random partial orders was initiated by Winkler [30]. Here is an alternative construction of the random partial order on a fixed (rather than Poisson) number of elements. From the k! linear orders on a set of k elements, choose ν orders ≺1 , ≺2 , . . . , ≺ν uniformly at random with replacement. Define the random order ≺ as the intersection, namely x ≺ y iff x ≺j y for j = 1, . . . , ν. The height of the random order is the maximal size of a linearly ordered subset. Conditioned on the number k of Poisson points in the cube (0, n1], H(0, n1) has the same distribution as the height of the random order ≺. Let us also point out that by the spatial scaling of the Poisson point process, for any b = (b1 , . . . , bν ) > 0 in Rd , (3)

lim

n→∞

1 H(0, nb) = cν (b1 b2 b3 · · · bν )1/ν n

a.s.

3. The interface process Fix a spatial dimension d ≥ 2. Appropriately interpreted, everything we say is true in d = 1 also, but does not offer anything significantly new. We describe the evolution of a random, integer-valued height function σ = (σ(x))x∈Rd . Height values ±∞ are permitted, so the range of the height function is Z∗ = Z ∪ {±∞}.

Growth model

207

The state space of the process is the space Σ of functions σ : Rd → Z∗ that satisfy conditions (i)–(iii): (4)

(i) Monotonicity: x ≤ y in Rd implies σ(x) ≤ σ(y).

The partial order x ≤ y on Rd is the coordinatewise one defined in Section 2. (ii) Discontinuities restricted to a locally finite, countable collection of coordinate hyperplanes: for each bounded cube [−q1, q1] ⊆ Rd , there are finite partitions i −q = s0i < s1i < · · · < sm =q i

along each coordinate direction (1 ≤ i ≤ d), such that any discontinuity point of σ in [−q1, q1] lies on one of the hyperplanes {x ∈ [−q1, q1] : xi = ski }, 1 ≤ i ≤ d and 0 ≤ k ≤ mi . At discontinuities σ is continuous from above: σ(y) → σ(x) as y → x so that y ≥ x in Rd . Since σ is Z∗ -valued, this is the same as saying that σ is constant on the left closed, right open rectangles (5)

[sk , sk+1 ) ≡

d


[ski i , ski i +1 ) ,

k = (k1 , k2 , . . . , kd ) ∈

i=1

d


{0, 1, 2, . . . , mi − 1},

i=1

determined by the partitions {ski : 0 ≤ k ≤ mi }, 1 ≤ i ≤ d. (iii) A decay condition “at −∞”:   (6) for every b ∈ Rd , lim sup |y|−d/(d+1) σ(y) : y ≤ b, |y| ≥ M = −∞. ∞ ∞ M →∞

The role of the (arbitrary) point b in condition (6) is to confine y so that as the limit is taken, all coordinates of y remain bounded above and at least one of them diverges to −∞. Hence we can think of this as “y → −∞” in Rd . The ∞ norm on Rd is |y|∞ = max1≤i≤d |yi |. We can give Σ a complete, separable metric. Start with a natural Skorohod metric suggested by condition (ii). On bounded rectangles, this has been considered earlier by Bickel and Wichura [5], among others. This metric is then augmented with sufficient control of the left tail so that convergence in this metric preserves (6). The Borel σ-field under this metric is generated by the coordinate projections σ → σ(x). These matters are discussed in a technical appendix at the end of the paper. Assume given an initial height function σ ∈ Σ. To construct the dynamics, assume also given a space-time Poisson point process on Rd × (0, ∞). We define the process σ(t) = {σ(x, t) : x ∈ Rd } for times t ∈ [0, ∞) by (7)

σ(x, t) = sup {σ(y) + H((y, 0), (x, t))}. y:y≤x

The random variable H((y, 0), (x, t)) is the maximal number of Poisson points on an increasing sequence in the space-time rectangle ((y, 0), (x, t)] = {(η, s) ∈ Rd × (0, t] : yi < ηi ≤ xi (1 ≤ i ≤ d)}, as defined in Section 2. One can prove that, for almost every realization of the Poisson point process, the supremum in (7) is achieved at some y, and σ(t) ∈ Σ for all t > 0. In particular, if initially σ(x) is finite then σ(x, t) remains finite for all 0 ≤ t < ∞. And if σ(x) = ±∞, then σ(x, t) = σ(x) for all 0 ≤ t < ∞. This defines a Markov process on the path space D([0, ∞), Σ).

T. Sepp¨ al¨ ainen

208

The local effect of the dynamical rule (7) is the following. Suppose (y, t) ∈ Rd × (0, ∞) is a Poisson point, and the state at time t− is σ. Then at time t the state changes to σ y defined by  σ(x) + 1, if x ≥ y and σ(x) = σ(y), y (8) σ (x) = σ(x), for all other x ∈ Rd . We can express the dynamics succinctly like this: Independently at all x ∈ Rd , σ(x) jumps to σ(x) + 1 at rate dx (d-dimensional volume element). When a jump at x happens, the height function σ is updated to σ + 1 on the set {w ∈ Rd : w ≥ x, σ(w) = σ(x)} to preserve the monotonicity property (4). It also follows that if σ(y) = ±∞ then σ y = σ. We express this in generator language as follows. Suppose φ is a bounded measurable function on Σ, and supported on a compact cube K ⊆ Rd . By this we mean that φ is a measurable function of the coordinates (σ(x))x∈K . Define the generator L by  (9) Lφ(σ) = [φ(σ y ) − φ(σ)]dy. Rd

The next theorem verifies that L gives the infinitesimal description of the process in one basic sense. Theorem 3.1. For bounded measurable functions φ on Σ, σ ∈ Σ, and t > 0,  t E σ [Lφ(σ(s))]ds. (10) E σ [φ(σ(t))] − φ(σ) = 0

σ

E denotes expectation under the path measure P σ of the process defined by (7) and started from state σ. 4. Hydrodynamic limit for the height process Let u0 : Rd → R be a nondecreasing locally Lipschitz continuous function, such that for any b ∈ Rd ,   (11) lim sup |y|−d/(d+1) u (y) : y ≤ b, |y| ≥ M = −∞. 0 ∞ ∞ M →∞

The function u0 represents the initial macroscopic height function. Assume that on some probability space we have a sequence of random initial height functions {σn (y, 0) : y ∈ Rd }, indexed by n. Each σn (· , 0) is a.s. an element of the state space Σ. The sequence satisfies a law of large numbers: (12)

for every y ∈ Rd , n−1 σn (ny, 0) → u0 (y) as n → ∞, a.s.

Additionally there is the following uniform bound on the decay at −∞: (13)

for every fixed b ∈ Rd and C > 0, with probability 1 there exist finite, possibly random, M, N > 0 such that, if n ≥ N , y ≤ b and |y|∞ ≥ M , d/(d+1) then σn (ny, 0) ≤ −Cn|y|∞ .

Augment the probability space of the initial σn (·, 0) by a space-time Poisson point process, and define the processes σn (x, t) by (7). For x = (x1 , . . . , xd ) ≥ 0 in Rd , define g(x) = cd+1 (x1 x2 x3 · · · xd )1/(d+1) .

Growth model

209

The constant cd+1 is the one from (2), and it comes from the partial order among Poisson points in d + 1 dimensional space-time rectangles. Define a function u(x, t) on Rd × [0, ∞) by u(x, 0) = u0 (x) and for t > 0, (14)

u(x, t) = sup {u0 (y) + tg((x − y)/t)}. y:y≤x

The function u is nondecreasing in x, increasing in t, and locally Lipschitz in Rd × (0, ∞). Theorem 4.1. Suppose u0 is a locally Lipschitz function on Rd that satisfies (11). Define u(x, t) through (14). Assume that the initial random interfaces {σn (y, 0)} satisfy (12) and (13). Then for all (x, t) ∈ Rd × [0, ∞), (15)

lim n−1 σn (nx, nt) = u(x, t)

n→∞

a.s.

By the monotonicity of the random height and the continuity of the limiting function, the limit (15) holds simultaneously for all (x, t) outside a single exceptional null event. Extend g to an u.s.c. concave function on all of Rd by setting g ≡ −∞ outside [0, ∞)d . Define the constant (16)

κd =



cd+1 d+1

d+1

.

The concave conjugate of g is g ∗ (ρ) = inf x {x · ρ − g(x)}, ρ ∈ Rd . Let f = −g ∗ . Then f (ρ) = ∞ for ρ ∈ / (0, ∞)d , and (17)

f (ρ) = κd (ρ1 ρ2 · · · ρd )−1

for ρ > 0 in Rd .

The Hopf-Lax formula (14) implies that u solves the Hamilton-Jacobi equation (see [10]) (18)

∂t u − f (∇u) = 0 ,

u|t=0 = u0 .

In other words, f (∇u) is the upward velocity of the interface, determined by the local slope. The most basic case of the hydrodynamic limit starts with σ(y, 0) = 0 for y ≥ 0 and σ(y, 0) = −∞ otherwise. Then σ(x, t) = H((0, 0), (x, t)) for x ≥ 0 and −∞ otherwise. The limit is u(x, t) = tg(x/t). 5. The defect boundary limit Our objective is to generalize the notion of a second class particle from the onedimensional context. The particle interpretation does not make sense now. But a second class particle also represents a defect in an interface, and is sometimes called a ‘defect tracer.’ This point of view we adopt. Given an initial height function σ(y, 0), perturb it by increasing the height to σ(y, 0) + 1 for points y in some set A(0). The boundary of the set A(0) corresponds to a second class particle, so we call it the defect boundary. How does the perturbation set A(·) evolve in time? To describe the behavior of this set under hydrodynamic scaling, we need to look at how the Hamilton-Jacobi equation (18) carries information in time.

T. Sepp¨ al¨ ainen

210

For (x, t) ∈ Rd × (0, ∞), let I(x, t) be the set of maximizers in (14): (19)

I(x, t) = {y ∈ Rd : y ≤ x, u(x, t) = u0 (y) + tg((x − y)/t)}.

Continuity and hypothesis (11) guarantee that I(x, t) is a nonempty compact set. It turns out that these three statements (i)–(iii) are equivalent for a point (x, t): (i) the gradient ∇u in the x-variable exists at (x, t), (ii) u is differentiable at (x, t), and (iii) I(x, t) is a singleton. We call a point (x, t) with t > 0 a shock if I(x, t) has more than one point. For y ∈ Rd let W (y, t) be the set of points x ∈ Rd for which y is a maximizer in the Hopf-Lax formula (14) at time t: (20)

W (y, t) = {x ∈ Rd : x ≥ y, u(x, t) = u0 (y) + tg((x − y)/t)},

and for any subset B ⊆ Rd , (21)

W (B, t) =



W (y, t).

y∈B

Given a closed set B ⊆ Rd , let (22)

X(B, t) = W (B, t) ∩ W ( B c , t).

W (B, t) and W ( B c , t) are both closed sets. We can characterize x ∈ X(B, t) as follows: if (x, t) is not a shock then the unique maximizer {y} = I(x, t) in (14) lies on the boundary of B, while if (x, t) is a shock then I(x, t) intersects both B and Bc. If dimension d = 1 and B = [a, ∞) ⊆ R, an infinite interval, then X(B, t) is precisely the set of points x for which there exists a forward characteristic x(·) such that x(0) = a and x(t) = x. By a forward characteristic we mean a Filippov solution of dx/dt = f  (∇u(x, t)) [9, 18]. A corresponding characterization of X(B, t) in multiple dimensions does not seem to exist at the moment. The open ε-neighborhood of a set B ⊆ Rd is denoted by (23)

B (ε) = {x : d(x, y) < ε for some y ∈ B}.

The distance d(x, y) can be the standard Euclidean distance or another equivalent metric, it makes no difference. Let us write B (−ε) for the set of x ∈ B that are at least distance ε > 0 away from the boundary:

c (24) B (−ε) = {x ∈ B : d(x, y) ≥ ε for all y ∈ / B} = (B c )(ε) .

The topological boundary of a closed set B is bdB = B ∩ B c . Suppose two height processes σ(t) and ζ(t) are coupled through the space-time Poisson point process. This means that on some probability space are defined the initial height functions σ(y, 0) and ζ(y, 0), and a space-time Poisson point process which defines all the random variables H((y, 0), (x, t)). Process σ(x, t) is defined by (7), and process ζ(x, t) by the same formula with σ replaced by ζ, but with the same realization of the variables H((y, 0), (x, t)). If initially σ ≤ ζ ≤ σ + h for some constant h, then the evolution preserves these inequalities. We can follow the evolution of the “defect set” A(t), defined as A(t) = {x : ζ(x, t) = σ(x, t) + h} for t ≥ 0. This type of a setting we now study in the hydrodynamic context. In the

Growth model

211

introduction we only discussed the case h = 1, but the proof works for general finite h. Now precise assumptions. On some probability space are defined two sequences of initial height functions σn (y, 0) and ζn (y, 0). The {σn (y, 0)} satisfy the hypotheses (12) and (13) of Theorem 4.1. For some fixed positive integer h, (25)

σn (y, 0) ≤ ζn (y, 0) ≤ σn (y, 0) + h for all n and y ∈ Rd .

Construct the processes σn (t) and ζn (t) with the same realizations of the space-time Poisson point process. Then (26)

σn (x, t) ≤ ζn (x, t) ≤ σn (x, t) + h

for all n and (x, t).

In particular, ζn and σn satisfy the same hydrodynamic limit. Let (27)

An (t) = {x ∈ Rd : ζn (x, t) = σn (x, t) + h}.

Our objective is to follow the evolution of the set An (t) and its boundary bd{An (t)}. We need an initial assumption at time t = 0. Fix a deterministic closed set B ⊆ Rd . We assume that for large n, n−1 An (0) approximates B locally, in the following sense: almost surely, for every compact K ⊆ Rd and ε > 0, (28)

 B (−ε) ∩ K ⊆ n−1 An (0) ∩ K ⊆ B (ε) ∩ K

for all large enough n.

Theorem 5.1. Let again u0 satisfy (11) and the processes σn satisfy (12) and (13) at time zero. Fix a positive integer h and a closed set B ⊆ Rd . Assume that the processes σn are coupled with processes ζn through a common space-time Poisson point process so that (26) holds. Define An (t) by (27) and assume An (0) satisfies (28). If W (B, t) = ∅, then almost surely, for every compact K ⊆ Rd , An (nt) ∩ nK = ∅ for all large enough n. Suppose W (B, t) = ∅. Then almost surely, for every compact K ⊆ Rd and ε > 0, (29)

bd {n−1 An (nt)} ∩ K ⊆ X(B, t)(ε) ∩ K

for all large enough n.

In addition, suppose no point of W (B c , t) is an interior point of W (B, t). Then almost surely, for every compact K ⊆ Rd and ε > 0, (30)

 W (B, t)(−ε) ∩ K ⊆ n−1 An (nt) ∩ K ⊆ W (B, t)(ε) ∩ K for all large enough n.

The additional hypothesis for (30), that no point of W (B c , t) is an interior point of W (B, t), prevents B and B c from becoming too entangled at later times. For example, it prohibits the existence of a point y ∈ bd B such that W (y, t) has nonempty interior (“a rarefaction fan with interior”).

T. Sepp¨ al¨ ainen

212

6. Examples and technical comments 6.1. Second class particle analogy Consider a one-dimensional Hammersley process z(t) = (zi (t))i∈Z with labeled particle locations · · · ≤ z−1 (t) ≤ z0 (t) ≤ z1 (t) ≤ · · · on R. In terms of labeled particles, the infinitesimal jump rule is this: zi jumps to the left at exponential rate zi −zi−1 , and when it jumps, its new position zi is chosen uniformly at random from the interval (zi−1 , zi ). The height function is defined by σ(x, t) = sup{i : zi (t) ≤ x} for x ∈ R. Now consider another Hammersley process z˜(t) constructed with the same realization of the space-time Poisson point process as z(t). Assume that at time 0, z˜(0) has exactly the same particle locations as z(0), plus h additional particles. Then at all later times z˜(t) will have h particles more than z(t), and relative to the z(t)-process, these extra particles behave like second class particles. Suppose the labeling of the particles is such that z˜i (t) = zi (t) to the left of all the second class particles. Let X1 (t) ≤ · · · ≤ Xh (t) be the locations of the second class particles. Then the height functions satisfy σ

(x, t) = σ(x, t) for x < X1 (t), and σ

(x, t) = σ(x, t) + h for x ≥ Xh (t). So in this one-dimensional second class particle picture, the set A(t) is the interval [Xh (t), ∞). It has been proved, in the context of one-dimensional asymmetric exclusion, K-exclusion and zero-range processes, that in the hydrodynamic limit a second-class particle converges to a characteristic or shock of the macroscopic p.d.e. [12, 18, 26]. Despite this analogy, good properties of the one-dimensional situation are readily lost as we move to higher dimensions. For example, we can begin with a set A(0) that is monotone in the sense that x ∈ A(0) implies y ∈ A(0) for all y ≥ x. But this property can be immediately lost: Suppose a jump happens at w such that ζ(w, 0) = σ(w, 0) but the set V = {x ≥ w : σ(x, 0) = σ(w, 0)} intersects A(0) = {x : ζ(x, 0) = σ(x, 0) + 1}. Then after this event ζ = σ on V , and cutting V away from A(0) may have broken its monotonicity. 6.2. Examples of the limit in Theorem 5.1 We consider here the simplest macroscopic profiles for which we can explicitly calculate the evolution W (B, t) of a set B, and thereby we know the limit of n−1 An (nt) in Theorem 5.1. These are the flat profile with constant slope, and the cases of shocks and rarefaction fans that have two different slopes. Recall the slope-dependent velocity f (ρ) = κd (ρ1 ρ2 ρ3 · · · ρd )−1 for ρ ∈ (0, ∞)d , where κd is the (unknown) constant defined by (2) and (16). For the second class particle in one-dimensional asymmetric exclusion, these cases were studied in [12, 13]. Flat profile. Fix a vector ρ ∈ (0, ∞)d , and consider the initial profile u0 (x) = ρ · x. Then u(x, t) = ρ · x + tf (ρ), for each (x, t) there is a unique maximizer y(x, t) = x + t∇f (ρ) in the Hopf-Lax formula, and consequently for any set B, W (B, t) = −t∇f (ρ) + B. Shock profile. Fix two vectors λ, ρ ∈ (0, ∞)d , and let (31)

u0 (x) =



ρ · x, (ρ − λ) · x ≥ 0, λ · x, (ρ − λ) · x ≤ 0.

Growth model

213

Then at later times we have  ρ · x + tf (ρ), (ρ − λ) · x ≥ t(f (λ) − f (ρ)), u(x, t) = λ · x + tf (λ), (ρ − λ) · x ≤ t(f (λ) − f (ρ)). The Hopf-Lax formula is maximized by  x + t∇f (ρ), if (ρ − λ) · x ≥ t(f (λ) − f (ρ)), y= x + t∇f (λ), if (ρ − λ) · x ≤ t(f (λ) − f (ρ)). In particular, points (x, t) on the hyperplane (ρ − λ) · x = t(f (λ) − f (ρ)) are shocks, and for them both alternatives above are maximizers. In the forward evolution, W (y, t) is either a singleton or empty:  y − t∇f (ρ),       if (ρ − λ) · y ≥ t(f (λ) − f (ρ)) + t(ρ − λ) · ∇f (ρ),       ∅, if t(f (λ) − f (ρ)) + t(ρ − λ) · ∇f (λ) < (ρ − λ) · y W (y, t) =  < t(f (λ) − f (ρ)) + t(ρ − λ) · ∇f (ρ),       y − t∇f (λ),      if (ρ − λ) · y ≤ t(f (λ) − f (ρ)) + t(ρ − λ) · ∇f (λ). In this situation Theorem 5.1 is valid for all sets B. Rarefaction fan profile. Fix two vectors λ, ρ ∈ (0, ∞)d , and let  λ · x, (ρ − λ) · x ≥ 0, u0 (x) = ρ · x, (ρ − λ) · x ≤ 0.

For (x, t) such that −t(ρ − λ) · ∇f (ρ) < (ρ − λ) · x < −t(ρ − λ) · ∇f (λ) there exists a unique s = s(x, t) ∈ (0, 1) such that (ρ − λ) · x = −t(ρ − λ) · ∇f (sλ + (1 − s)ρ). Then at later times the profile can be expressed as  ρ · x + tf (ρ), if (ρ − λ) · x ≤ −t(ρ − λ) · ∇f (ρ),    (sλ + (1 − s)ρ) · x + tf (sλ + (1 − s)ρ), if u(x, t) = −t(ρ − λ) · ∇f (ρ) < (ρ − λ) · x < −t(ρ − λ) · ∇f (λ),    λ · x + tf (λ), if (ρ − λ) · x ≥ −t(ρ − λ) · ∇f (λ).

The forward evolution manifests the rarefaction fan: points y on the hyperplane (ρ − λ) · y = 0 have W (y, t) given by a curve, while for other points y W (y, t) is a singleton:    y − t∇f (ρ), if (ρ − λ) · y < 0, W (y, t) = {y − t∇f (sλ + (1 − s)ρ) : 0 ≤ s ≤ 1}, if (ρ − λ) · y = 0,   y − t∇f (λ), if (ρ − λ) · y > 0. In Theorem 5.1, consider the half-space B = {x : (ρ − λ) · x ≥ 0}. Then

X(B, t) = {x : −t(ρ − λ) · ∇f (ρ) ≤ (ρ − λ) · x ≤ −t(ρ − λ) · ∇f (λ)},

214

T. Sepp¨ al¨ ainen

the “rarefaction strip” in space. Statement (30) is not valid for B, because the interior of X(B, t) lies in the interiors of both W (B, t) and W ( B c , t). Statement (29) is valid, and says that the boundary of n−1 An (nt) is locally contained in any neighborhood of X(B, t). In the corresponding one-dimensional setting, Ferrari and Kipnis [13] proved that on the macroscopic scale, the second class particle is uniformly distributed in the rarefaction fan. Their proof depended on explicit calculations with Bernoulli distributions, so presently we cannot approach such precise knowledge of bd{n−1 An (nt)}. 6.3. Some random initial conditions We give here some natural examples of random initial conditions for Theorems 4.1 and 5.1 for the case d = 2. We construct these examples from space-time evolutions of one-dimensional Hammersley’s process. The space-time coordinates (y, t) of the 1-dimensional process will equal the 2-dimensional spatial coordinates x = (x1 , x2 ) of a height function. Flat profiles. In one dimension, Aldous and Diaconis [1] denoted the Hammersley process by N (y, t). The function y → N (y, t) (y ∈ R) can be regarded as the counting function of a point process on R. Homogeneous Poisson point processes are invariant for this process. To construct all flat initial profiles u0 (x) = ρ · x on R2 , we need two parameters that can be adjusted. The rate µ of the spatial equilibrium of N (y, t) gives one parameter. Another parameter τ is the jump rate, in other words the rate of the space-time Poisson point process in the graphical construction of N (y, t). Let now N (y, t) be a process in equilibrium, defined for −∞ < t < ∞, normalized so that N (0, 0) = 0, with jump rate τ , and so that the spatial distribution at each fixed time is a homogeneous Poisson process at rate µ. Then the process of particles jumping past a fixed point in space is Poisson at rate τ /µ [1], Lemma 8. Consequently EN (y, t) = µy + (τ /µ)t. This way we can construct a random initial profile whose mean is a given flat initial profile: given ρ = (ρ1 , ρ2 ) ∈ (0, ∞)2 , take an equilibrium process {N (y, t) : y ∈ R, t ∈ R} with µ = ρ1 and τ = ρ1 ρ2 , and define the initial height function for x = (x1 , x2 ) ∈ R2 by σ((x1 , x2 ), 0) = N (x1 , x2 ). Shock profiles. Next we construct a class of initial shock profiles. Suppose ρ = (ρ1 , ρ2 ) and λ = (λ1 , λ2 ) satisfy ρ > λ and ρ1 /ρ2 < λ1 /λ2 . Start by constructing the equilibrium Hammersley system {N (y, t) : y ∈ R, t ∈ R} with spatial density µ = λ1 and jump rate τ = λ1 λ2 . Set a = (ρ1 − λ1 )/(ρ2 − λ2 ) > 0. Stop each Hammersley particle the first time it hits the space-time line t = −ay, and “erase” the entire evolution of N (y, t) above this line. The assumption ρ1 /ρ2 < λ1 /λ2 guarantees that each particle eventually hits this line. Now we have constructed the slope-λ height function σ((x1 , x2 ), 0) = N (x1 , x2 ) below the line (ρ − λ) · x = 0 ⇐⇒ x2 = −ax1 . (Slope-λ in the sense that Eσ(x, 0) = λ · x.) To continue the construction, put a rate τ  = ρ1 ρ2 space-time Poisson point process above the line t = −ay in the space-time picture of the 1-dim Hammersley process. Let the Hammersley particles evolve from their stopped locations on the line t = −ay, according to the usual graphical construction [1] of the process, using the rate τ  space-time Poisson points. The construction is well defined, because given any finite time T , N (y, T ) is already constructed for y ≤ −T /a, and for y > −T /a the particle trajectories can be constructed one at a time from left to right, starting with the leftmost particle stopped at a point (y, −ay) for y > −T /a.

Growth model

215

One can check that defining σ((x1 , x2 ), 0) = N (x1 , x2 ) for x2 > −ax1 gives the slope-ρ height function above the line (ρ − λ) · x = 0. Now we have a random initial height function σ(x, 0) with mean Eσ(x, 0) = u0 (x) as in (31). Finally, we describe a way to define initial configurations for the coupled processes ζ and σ in the context of this shock example. We shall do it so that the set {x : ζ(x, 0) = σ(x, 0) + 1} lies inside B = {x : x2 ≥ −ax1 }, and approximates it closely. Let ζ(x, 0) be the height function defined above in terms of the N (y, t) constructed in two steps, first below and then above the line t = −ay. Let zk (t) be the trajectories of the labeled Hammersley particles. These trajectories are the level curves of ζ(x, 0), namely ζ((x1 , x2 ), 0) ≥ k iff zk (x2 ) ≤ x1 . The construction performed above has the property that each zk (t) crosses the line t = −ay exactly once (the particles were stopped upon first hitting this line, and then continued entirely above the line). Define new trajectories zk (t) as follows: zk (t) = zk (t) below the line t = −ay. From the line t = −ay the trajectory zk (t) proceeds vertically upward (in the tdirection) until it hits the trajectory of zk+1 (t). From that point onwards zk (t) follows the trajectory of zk+1 (t). This is done for all k. Let N  (y, t) be the counting function defined by N  (y, t) = sup{k : zk (t) ≤ y}. And then set σ((x1 , x2 ), 0) = N  (x1 , x2 ) The initial height functions σ(x, 0) and ζ(x, 0) thus defined have these properties: σ(x, 0) = ζ(x, 0) for x2 ≤ −ax1 . For any point (x1 , x2 ) such that x2 > −ax1 and some particle trajectory zk (t) passes between (x1 , −ax1 ) and (x1 , x2 ), ζ(x, 0) = σ(x, 0) + 1. This construction satisfies the hypotheses of Theorem 5.1. 6.4. Some properties of the multidimensional Hamilton-Jacobi equation Let u(x, t) be the viscosity solution of the equation ut = f (∇u), defined by the Hopf-Lax formula (14). By assumption, the initial profile u0 is locally Lipschitz and satisfies the decay estimate (11). Hypothesis (11) is tailored to this particular velocity function, and needs to be changed if f is changed. Part (b) of this lemma will be needed in the proof of Thm. 5.1.  Lemma 6.1. (a) For any compact K ⊆ Rd , x∈K I(x, t) is compact. (b) W (B, t) is closed for any closed set B ⊆ Rd . Proof. (a) By (11), as y → −∞ for y ≤ x, u0 (y) + tg((x − y)/t) tends to −∞ uniformly over x in a bounded set. Also, the condition inside (19) is preserved by limits because all the functions are continuous. (b) If W (B, t)  xj → x, then by (a) any sequence of maximizers yj ∈ I(xj , t) ∩ B has a convergent subsequence.

The association of I(x, t) to x is not as well-behaved as in one dimension. For example, not only is there no monotonicity, but a simple example can have x1 < x2 with maximizers yi ∈ I(xi , t) such that y2 < y1 . The local Lipschitz condition on u0 guarantees that each y ∈ I(x, t) satisfies y < x (i.e. strict inequality for all coordinates). Properties that are not hard to check include the following. Part (a) of the lemma implies that u(x, t) is locally Lipschitz on Rd ×(0, ∞). Lipschitz continuity does not necessarily hold down to t = 0, but continuity does. u is differentiable at (x, t) iff I(x, t) is a singleton {y}, and then ∇u(x, t) = ∇g((x−y)/t). Also, ∇u is continuous on the set where it is defined because whenever (xn , tn ) → (x, t) and yn ∈ I(xn , tn ), the sequence {yn } is bounded and all limit points lie in I(x, t).

T. Sepp¨ al¨ ainen

216

A converse question is when W (y, t) has more than one point. As in one dimension, one can give a criterion based on the regularity of u0 at y. The subdifferential D− u0 (x) and superdifferential D+ u0 (x) of u0 at x are defined by   u0 (y) − u0 (x) − q · (y − x) D− u0 (x) = q ∈ Rd : lim inf ≥0 y→x y − x and

  u0 (y) − u0 (x) − p · (y − x) d D u0 (x) = p ∈ R : lim sup ≤0 . y − x y→x +

It is a fact that both D± u0 (x) are nonempty iff u0 is differentiable at x, and then D± u0 (x) = {∇u0 (x)}.   One can check that W (y, t) ⊆ y − t∇f D+ u0 (y) . Consequently if D− u0 (y) is nonempty, W (y, t) cannot have more than 1 point. Another fact from onedimensional systems that also holds in multiple dimensions is that if we restart the evolution at time s > 0, then all forward sets W (y, t) are empty or singletons. ˜(x, s) In other words, if u ˜ is a solution with initial profile u ˜0 , and we define u0 (x) = u and u(x, t) = u ˜(x, s+t), then D− u0 (y) is never empty. This is because ∇g((x−y)/s) u(·, s)}(x) for every y that maximizes the Hopf-Lax formula for u ˜(x, s). lies in D− {˜ 7. Proof of the generator relation In this section we prove Theorem 3.1. Throughout the proofs we use the abbreviation x! = x1 x2 x3 · · · xd for a point x = (x1 , . . . , xd ) ∈ Rd . We make the following definition related to the dynamics of the process. For a height function σ ∈ Σ and a point x ∈ Rd , let  {y ∈ Rd : y ≤ x, σ(y) = σ(x)}, if σ(x) is finite, (32) Sx (σ) = ∅, if σ(x) = ±∞. Sx (σ) is the set in space where a Poisson point must arrive in the next instant in order to increase the height value at x. Consequently the Lebesgue measure (volume) |Sx (σ)| is the instantaneous rate at which the height σ(x) jumps up by 1. Since values σ(x) = ±∞ are not changed by the dynamics, it is sensible to set Sx (σ) empty in this case. For a set K in Rd we define  Sx (σ), (33) SK (σ) = x∈K

the set in space where an instantaneous Poisson arrival would change the function σ in the set K. We begin with a simple estimate. Lemma 7.1. Let x > 0 in Rd , t > 0, and k a positive integer. Then P {H((0, 0), (x, t)) ≥ k} ≤

(x!t)k ≤ e−k(d+1) (k!)(d+1)

where the second inequality is valid if k ≥ e2 (x!t)1/(d+1) . Note that above (0, 0) means the space-time point (0, 0) ∈ Rd × [0, ∞).

Growth model

217

Proof. Let γ = x!t = x1 x2 x3 · · · xd · t > 0 be the volume of the space-time rectangle (0, x]×(0, t]. k uniform points in this (d+1)-dimensional rectangle form an increasing chain with probability (k!)−d . Thus  e−γ γ j  j  P {H((0, 0), (x, t)) ≥ k} ≤ (k!)−d = γ k (k!)−(d+1) j! k j:j≥k

≤ γ k (k/e)−k(d+1) ≤ e−k(d+1)

if k ≥ e2 γ 1/(d+1) . We need to make a number of definitions that enable us to control the height functions σ ∈ Σ. For b ∈ Rd and h ∈ Z, let y b,h (σ) be the maximal point y ≤ b such that the rectangle [y, b] contains the set {x ≤ b : σ(x) ≥ h}, with y b,h (σ) = b if this set is empty. Note that if y b,h (σ) = b then there exists x ≤ b such that σ(x) ≥ h and |b − x|∞ = |b − y b,h (σ)|∞ . Throughout this section we consider compact cubes of the type K = [−q1, q1] ⊆ Rd for a fixed number q > 0. When the context is clear we may abbreviate y h = y q1,h (σ). Define λk (σ) =

sup

−∞
(q1 − y h )! · (k − h)−(d+1) .

Property (6) of the state space guarantees that λk (σ) < ∞. The minimal and maximal finite height values in K are defined by I(K, σ) = min{σ(x) : x ∈ K, −∞ < σ(x) < ∞} and J(K, σ) = max{σ(x) : x ∈ K, −∞ < σ(x) < ∞}. If σ = ±∞ on all of K we interpret I(K, σ) = ∞ = −J(K, σ). Otherwise these quantities are finite because σ can take only finitely many values in K. If σ is finite on all of K then by monotonicity I(K, σ) = σ(−q1) and J(K, σ) = σ(q1). Set  1/2 J(K,σ)+1  λ2k (σ) . (34) ψK (σ) =  k=I(K,σ)+1

If σ = ±∞ on all of K then ψK (σ) = 0. The next two lemmas are preliminary and illustrate how ψK (σ) appears as a bound. Lemma 7.2. For a cube K = [−q1, q1] and σ ∈ Σ, (35)

|SK (σ)| ≤ 2d+1 ψK (σ).

Proof. If σ = ±∞ on all of K then both sides of (35) are zero by the definitions. Suppose I(K, σ) is finite (this is the complementary case). If x ∈ K and y ∈ Sx (σ), then y ≤ x ≤ q1 and σ(y) = σ(x) ≥ I(K, σ), and consequently y ∈ [y I(K,σ) , q1]. This is true for an arbitrary point y ∈ SK (σ). Since y I(K,σ)−1 ≤ y I(K,σ) , we can weaken the conclusion to SK (σ) ⊆ [y I(K,σ)−1 , q1] to get |SK (σ)| ≤ (q1 − y I(K,σ)−1 )! =

2d+1 (q1 − y I(K,σ)−1 )! (I(K, σ) + 1 − (I(K, σ) − 1))d+1

≤ 2d+1 λI(K,σ)+1 (σ) ≤ 2d+1 ψK (σ).

T. Sepp¨ al¨ ainen

218

Lemma 7.3. Define the event G = {there exist x ∈ K and y ∈ Rd such that y < x, −∞ < σ(y) ≤ σ(x) − 1 < ∞, and H((y, 0), (x, t)) ≥ σ(x) + 1 − σ(y)}.

(36)

Then for 0 < t < 1/(2ed+1 ψK (σ)), 2 P σ (G) ≤ 2e2(d+1) ψK (σ)t2 .

Proof. Let the index k run through the finite values of σ(x)+1 in K, and h represent σ(y). Then J(K,σ)+1

P σ (G) ≤





P {H((y h , 0), (q1, t)) ≥ k − h}.

k=I(K,σ)+1 h≤k−2

By Lemma 7.1 and the inequality j! ≥ (j/e)j , for a fixed k the inner sum becomes  ((q1 − y h )!t)k−h  (λk (σ)t)j j (d+1)j ≤ ≤ 2(ed+1 λk (σ)t)2 . d+1 ((k − h)!) (j!)d+1 j≥2

h≤k−2

The assumption on t was used to sum the geometric series. Now sum over k. We get the first bound on the evolution. Lemma 7.4. Let φ be a bounded measurable function on Σ, supported on a compact cube K = [−q1, q1] ⊆ Rd . This means that φ(σ) depends on σ only through (σ(x))x∈K . Then there is a finite constant C = C( φ∞ ) such that, for all σ ∈ Σ and t > 0, the quantity ∆t (σ) = E σ [φ(σ(t))] − φ(σ) − tLφ(σ) 2 satisfies the bound |∆t (σ)| ≤ Ct2 ψK (σ), provided 0 ≤ t < 1/(2ed+1 ψK (σ)).

Proof. We may assume that σ is not ±∞ on all of K. For otherwise from the definitions E σ [φ(σ(t))] = φ(σ) and Lφ(σ) = 0, and the lemma is trivially satisfied. Observe that on the complement Gc of the event defined in (36), σ(x, t) =

sup

{σ(y) + H((y, 0), (x, t))}

y∈Sx (σ)∪{x}

for all x ∈ K. (The singleton {x} is added to Sx (σ) only to accommodate those x for which σ(x) = ±∞ and Sx (σ) was defined to be empty.) Consequently on the event Gc the value φ(σ(t)) is determined by σ and the Poisson points in the space-time region SK (σ) × (0, t]. Let Dj be the event that SK (σ) × (0, t] contains j space-time Poisson points, and D2 = (D0 ∪ D1 )c the event that this set contains at least 2 Poisson points. On the event D1 , let Y ∈ Rd denote the space coordinate of the unique Poisson point, uniformly distributed on SK (σ). Then   E σ [φ(σ(t))] = φ(σ) · P σ (Gc ∩ D0 ) + E[φ(σ Y ) · 1Gc ∩D1 ] + O P σ (G) + P σ (D2 )   2 (σ) + |SK (σ)|2 . = φ(σ) + tLφ(σ) + t2 · O ψK

Growth model

219

To get the second equality above, use P σ (Dj ) = (j!)−1 (t|SK (σ)|)j exp(−t|SK (σ)|), Lemma 7.3 for bounding P σ (G), and hide the constant from Lemma 7.3 and φ∞ in the O-terms. Proof of the lemma is completed by (35). We insert here an intermediate bound on the height H. It is a consequence of Lemma 7.1 and a discretization of space. Lemma 7.5. Fix t > 0, α ∈ (0, 1/2), and β > e2 t1/(d+1) . Then there are finite positive constants θ0 , C1 and C2 such that, for any θ ≥ θ0 ,  P there exist y < x such that |y|∞ ≥ θ, |x − y|∞ ≥ α|y|∞ , (37) d/(d+1) and H((y, 0), (x, t)) ≥ β|x − y|∞ ≤ C1 exp(−C2 θd/(d+1) ). For positive m, define (38)

σ m (x, t) ≡

sup y≤x |y|∞ ≤m



σ(y) + H((y, 0), (x, t))

Corollary 7.6. Fix a compact cube K ⊆ Rd , 0 < T < ∞, and initial state σ ∈ Σ. Then there exists a finite random variable M such that, almost surely, σ(x, t) = σ M (x, t) for (x, t) ∈ K × [0, T ]. Proof. If σ(x) = ±∞ then y = x is the only maximizer needed in the variational formula (7). Thus we may assume that I(K, σ) is finite. Fix α ∈ (0, 1/2) and β > e2 T 1/(d+1) . By the boundedness of K, (37), and BorelCantelli there is a finite random M such that H((y, 0), (x, T )) ≤ β|x − y|d/(d+1) ∞ whenever x ∈ K and |y|∞ ≥ M . Increase M further so that M ≥ 1 + |x| for all −d/(d+1) x ∈ K, and σ(y) ≤ −(2β + |I(K, σ)| + 1)|y|∞ for all y such that y ≤ x for some x ∈ K and |y|∞ ≥ M . Now suppose y ≤ x, x ∈ K, σ(x) is finite, and |y|∞ ≥ M . Then + β|x − y|d/(d+1) σ(y) + H((y, 0), (x, t)) ≤ −(2β + |I(K, σ)| + 1)|y|−d/(d+1) ∞ ∞ ≤ −|I(K, σ)| − 1 ≤ σ(x) − 1 ≤ σ(x, t) − 1. We see that y cannot participate in the supremum in (7) for any (x, t) ∈ K × [0, T ]. To derive the generator formula we need to control the error in Lemma 7.4 uniformly over time, in the form ∆τ (σ(s)) with 0 ≤ s ≤ t and a small τ > 0. For a fixed k, λk (σ(s)) is nondecreasing in s because each coordinate of y h decreases over time. For q > 0 and k ∈ Z introduce the function (39)

Ψq,k (σ) = sup  x≤q1

|x|d∞ 1 ∨ {k − σ(x)}

A calculation that begins with

λk (σ) ≤ q d /2 +

sup h≤k−2 y q1,h =q1

d+1 .

2d |y q1,h |d∞ (k − h)d+1

T. Sepp¨ al¨ ainen

220

shows that λk (σ) ≤ q d /2 + 2d Ψq,k (σ). Interface heights σ(x, s) never decrease with time, and Ψq,k (σ) is nonincreasing in k but nondecreasing in σ. Therefore we can bound as follows, uniformly over s ∈ [0, t]: J(K,σ(s)+1 2 ψK (σ(s)) =



λ2k (σ(s))

k=I(K,σ(s))+1

(40)

  ≤ J(K, σ(t)) − I(K, σ(0)) + 1 ·

max

I(K,σ(0))+1≤k≤J(K,σ(t))+1

λ2k (σ(s))

 2   ≤ J(K, σ(t)) − I(K, σ(0)) + 1 (q 2d + 1) + 24d Ψ4q,I(K,σ(0)) σ(t) .

Above we used the inequality c(a + b)2 ≤ 2ca2 + c2 + b4 for a, b, c ≥ 0. The next lemma implies that the moments E σ [Ψpq,I(K,σ) (σ(t))] are finite for all p < ∞. Lemma 7.7. Let σ be an element of the state space Σ. Fix t > 0 and a point q1 ∈ Rd+ . Then there exists a finite number v0 (σ) such that, for v ≥ v0 (σ),    (41) P σ Ψq,I(K,σ) σ(t) > v ≤ C1 exp(−C2 v 1/(d+1) ),

where the finite positive constants C1 , C2 are the same as in Lemma 7.5 above. Proof. Choose α, β so that (37) is valid. Let β1 = 2β + β(2α)d/(d+1) + 2.

Fix v0 = v0 (σ) > 0 so that these requirements are met: v0 ≥ 1 + |I(K, σ)|d+1 , and for all y ≤ x ≤ q1 such that |x|d∞ ≥ v0 , σ(x) ≤ −β1 |x|d/(d+1) ∞

and

|y|∞ ≥ |x|∞ ≥ θ0 .

Here θ0 is the constant that appeared in Lemma 7.5, and we used property (6) of the state space Σ. Let v ≥ v0 . We shall show that the event on the left-hand side of (41) is contained in the event in (37) with θ = v 1/d . Suppose the event in (41) happens, so that some x ≤ q1 satisfies (42)

> I(K, σ) − σ(x, t) v −1/(d+1) |x|d(d+1) ∞

and |x|d∞ ≥ v.

Note that the above inequality forces σ(x, t) > −∞, while the earlier requirement on v0 forces σ(x) < ∞, and thereby also σ(x, t) < ∞. Find a maximizer y ≤ x so that σ(x, t) = σ(y) + H((y, 0), (x, t)). Regarding the location of y, we have two cases two consider. Case 1. y ∈ [x − 2α|x|∞ 1, x]. Let y  = x − 2α|x|∞ 1. Then |x − y  |∞ ≥ α|y  |∞ by virtue of α ∈ (0, 1/2). Also y  ≤ x so the choices made above imply |y  |∞ ≥ |x|∞ ≥ v 1/d . H((y  , 0), (x, t)) ≥ H((y, 0), (x, t)) = σ(x, t) − σ(y) > I(K, σ) − v −1/(d+1) |x|d/(d+1) − σ(x) ∞ ≥ (β1 − 2)|x|d/(d+1) ≥ β(2α)d/(d+1) |x|d/(d+1) ∞ ∞ = β|x − y  |d/(d+1) . ∞

Growth model

221

In addition to (42), we used −v −1/(d+1) ≥ −1, −σ(y) ≥ −σ(x) ≥ β1 |x|∞ 1/(d+1) d/(d+1) I(K, σ) ≥ −v0 ≥ −|x|∞ . Case 2. y ∈ / [x − 2α|x|∞ 1, x]. This implies |x − y|∞ ≥ α|y|∞ .

d/(d+1)

, and

H((y, 0), (x, t)) = σ(x, t) − σ(y) > I(K, σ) − v −1/(d+1) |x|d/(d+1) − σ(y) ∞ ≥ −|x|d/(d+1) − v −1/(d+1) |x|d/(d+1) + β1 |y|d/(d+1) ∞ ∞ ∞ ≥ 2β|y|d/(d+1) ≥ β|x − y|d/(d+1) . ∞ ∞ We conclude that the event in (41) lies inside the event in (37) with θ = v 1/d , as long as v ≥ v0 , and the inequality in (41) follows from (37). Corollary 7.8. Let K be a compact cube, ε > 0, and 0 < t < ∞. Then there exists a deterministic compact cube L such that  P σ SK (σ(s)) ⊆ L for all s ∈ [0, t] ≥ 1 − ε. Proof. For 0 ≤ s ≤ t, x ∈ SK (σ(s)) implies that I(K, σ(s)) is finite, x ≤ q1 and σ(x, s) ≥ I(K, σ(s)). Consequently     |x|d∞ ≤ Ψq,I(K,σ(s)) σ(s) ≤ Ψq,I(K,σ) σ(t) . Thus  given ε,we can  choose L = [−m1, m1] with m picked by Lemma 7.7 so that P σ Ψq,I(K,σ) σ(t) > md < ε. We are ready for the last stage of the proof of Theorem 3.1.

Proposition 7.9. Let φ be a bounded measurable function on Σ supported on the compact cube K = [−q1, q1] of Rd , and σ ∈ Σ. Then (43)

E σ [φ(σ(t))] − φ(σ) =



t

E σ [Lφ(σ(s))]ds. 0

Proof. Pick a small τ > 0 so that t = mτ for an integer m, and denote the partition by sj = jτ . By the Markov property,   m−1   E σ(sj ) [φ(σ(τ ))] − φ(σ(sj ))  E σ [φ(σ(t))] − φ(σ) = E σ  j=0

  σ =E

(44)

0

t m−1  j=0



1(sj ,sj+1 ] (s)Lφ(σ(sj+1 ))ds

  m−1    + τ Lφ(σ) − E σ [Lφ(σ(t))] + E σ  ∆τ (σ(sj )) , j=0

where the terms ∆τ (σ(sj )) are as defined in Lemma 7.4. We wish to argue that, as m → ∞ and simultaneously τ → 0, expression (44) after the last equality sign converges to the right-hand side of (43). Note first that Lφ(σ) is determined by the restriction of σ to the set SK (σ) ∪ K. By Corollary 7.8 there exists a fixed compact set L such that SK (σ(s)) ∪ K ⊆ L for 0 ≤ s ≤ t with probability at least 1 − ε. By Corollary 7.6, the time evolution

T. Sepp¨ al¨ ainen

222

{σ(x, s) : x ∈ L, 0 ≤ s ≤ t} is determined by the finitely many Poisson points in the random compact rectangle [−M, M ]d × [0, t]. Consequently the process Lφ(σ(s)) is m−1 piecewise constant in time, and then the integrand j=0 1(sj ,sj+1 ] (s)Lφ(σ(sj+1 )) converges to Lφ(σ(s)) pointwise as m → ∞. This happens on an event with probability at least 1 − ε, hence almost surely after letting ε → 0. To extend the convergence to the expectation and to handle the error terms, we show that

2 (σ(s)) < ∞. (45) E σ sup ψK 0≤s≤t

Before proving (45), let us see why it is sufficient. Since |Lφ(σ)| ≤ 2φ∞ |SK (σ)|,

(46)

(35) and (45) imply that also the first expectation after the equality sign in (44) converges, by dominated convergence. The second and third terms of (44) vanish, through a combination of Lemma 7.4, (35), and (45). 2 (σ(s)) and by Lemma 7.7, it only remains By the bound in (40) for sup0≤s≤t ψK to show that ! 2 " < ∞. E σ J(K, σ(t)) − I(K, σ(0)) + 1

This follows from property (6) of σ and the bounds for H in Lemmas 7.1 and 7.5. We omit the proof since it is not different in spirit than the estimates we already developed. This completes the proof of Theorem 3.1. 8. Proof of the limit for the height function Introduce the scaling into the variational formula (7) and write it as (47)

σn (nx, nt) =

sup

{σn (ny, 0) + H((ny, 0), (nx, nt))}.

y∈Rd :y≤x

Lemma 8.1. Assume the processes σn satisfy (12) and (13). Fix a finite T > 0 and a point b ∈ Rd such that b > 0, and consider the bounded rectangle [−b, b] ⊆ Rd . Then with probability 1 there exist a random N < ∞ and a random point a ∈ Rd such that (48)

σn (nx, nt) = sup {σn (ny, 0) + H((ny, 0), (nx, nt))} y∈[a,x]

for x ∈ [−b, b], t ∈ (0, T ], n ≥ N . Proof. For β ≥ e2 T 1/(d+1) and b ∈ Rd fixed, one can deduce from Lemma 7.1 and Borel-Cantelli that, almost surely, for large enough n, H((ni, 0), (nb, nt)) ≤ βn|b − i|d/(d+1) ∞ for all i ∈ Zd such that i ≤ b and |i − b|∞ ≥ 1. If y ∈ Rd satisfies y ≤ b and |y − b|∞ ≥ 1, we can take i = [y] (coordinatewise integer parts of y) and see that (49) for all such y.

H((ny, 0), (nb, nt)) ≤ βn + βn|b − y|d/(d+1) ∞

Growth model

223 d/(d+1)

In assumption (13) choose C > β so that −C +(2+|b|∞ )β < u0 (−b)−1. Let N and M be as given by (13), but increase M further to guarantee M ≥ 1. Now take a ∈ Rd far enough below −b so that, if y ≤ b but y ≥ a fails, then |y|∞ ≥ M . [Since assumption (13) permits a random M > 0, here we may need to choose a random d/(d+1) a ∈ Rd .] Then by (13), if y ≤ b but y ≥ a fails, then σn (ny, 0) ≤ −Cn|y|∞ . Now suppose x ∈ [−b, b], y ≤ x, but y ≥ a fails. Then σn (ny, 0) + H((ny, 0), (nx, nt)) ≤ σn (ny, 0) + H((ny, 0), (nb, nt)) + βn + βn|b − y|d/(d+1) ≤ −Cn|y|d/(d+1) ∞ ∞ # $ d/(d+1) ≤ n (−C + β)|y|∞ + β + β|b|d/(d+1) ∞

≤ nu0 (−b) − n ≤ σn (−nb, 0) − n/2 [by assumption (12), for large enough n] ≤ σn (nx, 0) − n/2 [by monotonicity].

This shows that in the variational formula (47) the point y = x strictly dominates all y outside [a, x]. Starting with (48) the limit (15) is proved (i) by partitioning [a, x] into small rectangles, (ii) by using monotonicity of the random variables, and the monotonicity and continuity of the limit, and (iii) by appealing to the assumed initial limits (12) and to (50)

n−1 H((ny, 0), (nx, nt)) → cd+1 ((x − y)!t)1/(d+1) = tg((x − y)/t)

a.s.

To derive the limit in (50) from (3) one has to fill in a technical step because in (50) the lower left corner of the rectangle (ny, nx] × (0, nt] moves as n grows. One can argue around this complication in at least two different ways: (a) The KestenHammersley lemma [28], page 20, from subadditive theory gives a.s. convergence along a subsequence, and then one fills in to get the full sequence. This approach was used in [24]. (b) Alternatively, one can use Borel-Cantelli if summable deviation bounds are available. These can be obtained by combining Theorems 3 and 9 from Bollob´ as and Brightwell [6]. 9. Proof of the defect boundary limit In view of the variational equation (7), let us say σ(x, t) has a maximizer y if y ≤ x and σ(x, t) = σ(y, 0) + H((y, 0), (x, t)). Lemma 9.1. Suppose two processes σ and ζ are coupled through the space-time Poisson point process. (a) For a positive integer m, let Dm (t) = {x : ζ(x, t) ≥ σ(x, t) + m}. Then if x ∈ Dm (t), ζ(x, t) cannot have a maximizer y ∈ Dm (0)c . And if x ∈ Dm (t)c , σ(x, t) cannot have a maximizer y ∈ Dm (0). (b) In particular, suppose initially σ(y, 0) ≤ ζ(y, 0) ≤ σ(y, 0) + h for all y ∈ Rd , for a fixed positive integer h. Then this property is preserved for all time. If we write A(t) = {x : ζ(x, t) = σ(x, t) + h}, then (51)

A(t) = {x : σ(x, t) has a maximizer y ∈ A(0)}.

224

T. Sepp¨ al¨ ainen

(c) If h = 1 in part (b), we get additionally that (52)

A(t)c = {x : ζ(x, t) has a maximizer y ∈ A(0)c }.

Proof. (a) Suppose x ∈ Dm (t), y ∈ Dm (0)c , and ζ(x, t) = ζ(y, 0) + H((y, 0), (x, t)). Then by the definition of Dm (t), σ(x, t) ≤ ζ(x, t) − m = ζ(y, 0) − m + H((y, 0), (x, t)) ≤ σ(y, 0) + H((y, 0), (x, t)) − 1 which contradicts the variational equation (7). Thus ζ(x, t) cannot have a maximizer y ∈ Dm (0)c . The other part of (a) is proved similarly. (b) Monotonicity implies that σ(x, t) ≤ ζ(x, t) ≤ σ(x, t) + h for all (x, t), so A(t) = Dh (t). Suppose x ∈ A(t). By (a) ζ(x, t) cannot have a maximizer y ∈ A(0)c , and so ζ(x, t) has a maximizer y ∈ A(0). Consequently σ(x, t) = ζ(x, t) − h = ζ(y, 0) − h + H((y, 0), (x, t)) = σ(y, 0) + H((y, 0), (x, t)), which says that σ(x, t) has a maximizer y ∈ A(0). On the other hand, if σ(x, t) has a maximizer y ∈ A(0), then by (a) again x ∈ / A(t)c . This proves (51). c (c) Now A(t) = D1 (t) and A(t) = {x : σ(x, t) = ζ(x, t)}. If ζ(x, t) has a / A(t). While if x ∈ A(t)c , again by part maximizer y ∈ A(0)c , then by part (a) x ∈ (a) σ(x, t) must have a maximizer y ∈ A(0)c , which then also is a maximizer for ζ(x, t). This proves (52). Assume the sequence of processes σn (·) satisfies the hypotheses of the hydrodynamic limit Theorem 4.1 which we proved in Section 8. The defect set An (t) was defined through the (σn , ζn ) coupling by (27). By (51) above, we can equivalently define it by (53)

An (t) = {x : σn (x, t) has a maximizer y ∈ An (0) }.

In the next lemma we take the point of view that some sequence of sets that depend on ω has been defined by (53), and ignore the (σn , ζn ) coupling definition. Lemma 9.2. Let B ⊆ Rd be a closed set. Suppose that for almost every sample point ω in the underlying probability space, a sequence of sets An (0) = An (0; ω) is defined, and has this property: for every compact K ⊆ Rd and ε > 0,  −1 for all large enough n. (54) n An (0) ∩ K ⊆ B (ε) ∩ K

Suppose the sets An (t) satisfy (53) and fix t > 0. Then almost surely, for every compact K ⊆ Rd and ε > 0,  −1 for all large enough n. (55) n An (nt) ∩ K ⊆ W (B, t)(ε) ∩ K

In particular, if W (B, t) = ∅, then (55) implies that {n−1 An (nt)} ∩ K = ∅ for all large enough n.

Proof. Fix a sample point ω such that assumption (54) is valid, the conclusion of Lemma 8.1 is valid for all b ∈ Zd+ , and we have the limits (56)

n−1 σn (nx, nt) → u(x, t)

for all (x, t),

Growth model

225

and (57)

n−1 H((ny, 0), (nx, nt)) → tg((x − y)/t)

for all y, x, t.

Almost every ω satisfies these requirements, by the a.s. limits (50) and (15), by monotonicity, and by the continuity of the limiting functions. It suffices to prove (55) for this fixed ω. To contradict (55), suppose there is a subsequence nj and points xj ∈ K / W (B, t)(ε) . Note that this also contradicts such that nj xj ∈ Anj (nj t) but xj ∈ −1 {n An (nt)} ∩ K = ∅ in case W (B, t) = ∅, so the empty set case is also proved by the contradiction we derive. Let nj yj ∈ Anj (0) be a maximizer for σnj (nj xj , nj t). Since the xj ’s are bounded, so are the yj ’s by Lemma 8.1, and we can pass to a subsequence (again denoted by {j}) such that the limits xj → x and yj → y exist. By the assumptions on xj , x∈ / W (B, t). For any ε > 0, yj ∈ B (ε) for large enough j, so y ∈ B by the closedness of B. Fix points x < x and y  < y  so that x < x < x and y  < y < y  in the partial order of Rd . Then for large enough j, x < xj < x and y  < yj < y  . By the choice of yj , σnj (nj xj , nj t) = σnj (nj yj , 0) + H((nj yj , 0), (nj xj , nj t)) from which follows, by the monotonicity of the processes, −1  n−1 j σnj (nj x , nj t) ≤ nj σnj (nj xj , nj t) −1    ≤ n−1 j σnj (nj y , 0) + nj H((nj y , 0), (nj x , nj t)).

Now let nj → ∞ and use the limits (56) and (57) to obtain u(x , t) ≤ u0 (y  ) + tg((x − y  )/t). We may let x , x → x and y  , y  → y, and then by continuity u(x, t) ≤ u0 (y) + tg((x − y)/t). This is incompatible with having x ∈ / W (B, t) and y ∈ B. This contradiction shows that, for the fixed ω, (55) holds. We prove statement (29) of Theorem 5.1. The assumption is that  for all large enough n. (58) B (−ε) ∩ K ⊆ n−1 An (0) ∩ K ⊆ B (ε) ∩ K

We introduce an auxiliary process ξn (x, t). Initially set  σn (y, 0), y∈ / An (0) (59) ξn (y, 0) = σn (y, 0) + 1, y ∈ An (0).

ξn (y, 0) is a well-defined random element of the state space Σ because An (0) is defined (27) in terms of ζn (y, 0) which lies in Σ. Couple the process ξn with σn and ζn through the common space-time Poisson points. Then σn (x, t) ≤ ξn (x, t) ≤ σn (x, t) + 1. By part (b) of Lemma 9.1, An (t) that satisfies (53) also satisfies An (t) = {x : ξn (x, t) = σn (x, t) + 1}.

T. Sepp¨ al¨ ainen

226

Then by part (c) of Lemma 9.1, (60)

An (t)c = {x : ξn (x, t) has a maximizer y ∈ An (0)c }.

 (ε) ∩K The first inclusion of assumption (58) implies that n−1 An (0)c ∩ K ⊆ B c for large n. The processes ξn inherit all the hydrodynamic properties of the processes σn . Thus by (60) we may apply Lemma 9.2 to the sets An (nt)c and the processes ξn (nt) to get (61)

n−1 An (nt)c ∩ K ⊆ W ( B c , t)(δ) ∩ K

for large enough n. By (55) and (61), bd {n−1 An (nt)} ∩ K ⊆ W (B, t)(δ) ∩ W ( B c , t)(δ) ∩ K for large n. For small enough δ > 0, the set on the right is contained in ! "(ε) W (B, t) ∩ W ( B c , t) ∩ K = X(B, t)(ε) ∩ K. This proves (29). To complete the proof of Theorem 5.1, it remains to prove (62) W (B, t)(−ε) ∩ K ⊆ n−1 An (nt) ∩ K ⊆ W (B, t)(ε) ∩ K for all large enough n under the further assumption that no point of W ( B c , t) is an interior point of W (B, t). The second inclusion of (62) we already obtained in Lemma 9.2. (61) implies

c W ( B c , t)(δ) ∩ K ⊆ n−1 An (nt) ∩ K. It remains to check that, given ε > 0,

c W (B, t)(−ε) ∩ K ⊆ W ( B c , t)(δ) ∩ K

for sufficiently small δ > 0. Suppose not, so that for a sequence δj  0 there exist xj ∈ W (B, t)(−ε) ∩ W ( B c , t)(δj ) ∩ K. By Lemma 6.1 the set W ( B c , t) is closed. Hence passing to a convergent subsequence xj → x gives a point x ∈ W ( B c , t) which is an interior point of W (B, t), contrary to the hypothesis. 10. Technical appendix: the state space of the process We develop the state space in two steps: first describe the multidimensional Skorohod type metric we need, and then amend the metric to provide control over the left tail of the height function. This Skorohod type space has been used earlier (see [5] and their references). 10.1. A Skorohod type space in multiple dimensions Let (X, r) be a complete, separable metric space, with metric r(x, y) ≤ 1. Let D = D(Rd , X) denote the space of functions σ : Rd → X with this property: for every bounded rectangle [a, b) ⊆ Rd and ε > 0, there exist finite partitions i ai = s0i < s1i < · · · < sm = bi i

Growth model

227

of each coordinate axis (1 ≤ i ≤ d) such that the variation of σ in the partition %d rectangles is at most ε: for each k = (k1 , k2 , . . . , kd ) ∈ i=1 {0, 1, 2, . . . , mi − 1}, (63)

sup{r(σ(x), σ(y)) : ski i ≤ xi , yi < ski i +1 (1 ≤ i ≤ d)} ≤ ε.

Note that the partition rectangles are closed on the left. This implies that σ is continuous from above: σ(y) → σ(x) as y → x in Rd so that y ≥ x; and limits exist from strictly below: lim σ(y) exists as y → x in Rd so that y < x (strict inequality for each coordinate). We shall employ this notation for truncation in Rd : for real u > 0 and x = (x1 , . . . , xd ) ∈ Rd ,   [x]u = (x1 ∧ u) ∨ (−u), (x2 ∧ u) ∨ (−u), . . . , (xd ∧ u) ∨ (−u) .

Let Λ be the collection of bijective, strictly increasing Lipschitz functions λ : Rd → Rd that satisfy these requirements: λ is of the type λ(x1 , . . . , xd ) = (λ1 (x1 ), . . . , λd (xd )) where each λi : R → R is bijective, strictly increasing and Lipschitz; and γ(λ) = γ0 (λ) + γ1 (λ) < ∞ where the quantities γ0 (λ) and γ1 (λ) are defined by & & & λi (t) − λi (s) && & γ0 (λ) = sup &log & t−s i=1 s,t∈R d 

and

γ1 (λ) =



0

∞

# & & $ e−u 1 ∧ sup & [λ(x)]u − [x]u &∞ du. x∈Rd

For ρ, σ ∈ D, λ ∈ Λ and u > 0, define

  d(ρ, σ, λ, u) = sup r ρ([x]u ) , σ([λ(x)]u ) . x∈Rd

And then (64)

dS (ρ, σ) = inf

λ∈Λ

  γ(λ) ∨

∞

0

 e−u d(ρ, σ, λ, u) du .

The definition was arranged so that γ(λ−1 ) = γ(λ) and γ(λ ◦ µ) ≤ γ(λ) + γ(µ), so the proof in [11], Section 3.5, can be repeated to show that dS is a metric. It is clear that if a sequence of functions σn from D converges to an arbitrary function σ : Rd → X, and this convergence happens uniformly on compact subsets of Rd , then σ ∈ D. Furthermore, we also get convergence in the dS -metric, as the next lemma indicates. This lemma is needed in the proof that (D, dS ) is complete. Lemma 10.1. Suppose σn , σ ∈ D. Then dS (σn , σ) → 0 iff there exist λn ∈ Λ such that γ(λn ) → 0 and   r σn (x), σ(λn (x)) → 0 uniformly over x in compact subsets of Rd .

Proof. We prove dS (σn , σ) → 0 assuming the second condition, and leave the other direction to the reader. For each rectangle [−M 1, M 1), M = 1, 2, 3, . . . , and each ε = 1/K, K = 1, 2, 3, . . . , fix the partitions {ski } that appear in the definition (63)

T. Sepp¨ al¨ ainen

228

of σ ∈ D. Pick a real u > 0 so that neither u nor −u is among these countably many partition points.   d(σn , σ, λn , u) = sup r σn ([x]u ) , σ([λn (x)]u ) x∈Rd

    ≤ sup r σn ([x]u ) , σ(λn ([x]u )) + sup r σ(λn ([x]u )) , σ([λn (x)]u ) . x∈Rd

x∈Rd

The first term after the inequality vanishes as n → ∞, by assumption. Let ε = 1/K > 0, pick a large rectangle [−M 1, M 1) that contains [−u1, u1] well inside its interior, and for this rectangle and ε pick the finite partitions that satisfy (63) for σ, and do not contain ±u. Let δ > 0 be such that none of these finitely many partition points lie in (±u − δ, ±u + δ). If n is large enough, then supx∈[−M 1,M 1] |λn (x) − x| < δ, and one can check that λn ([x]u ) and [λn (x)]u lie in the same partition rectangle, for each x ∈ Rd . Thus sup r (σ(λn ([x]u )) , σ([λn (x)]u ) ) ≤ ε. x∈Rd

We have shown that d(σn , σ, λn , u) → 0 for a.e. u > 0. With this lemma, one can follow the proof in [11], page 121, to show that (D, dS ) is complete. Separability of (D, dS ) would also be easy to prove. Next, we take this Skorohod type space as starting point, and define the state space Σ for the height process. 10.2. The state space for the height process In the setting of the previous subsection, take S = Z∗ = Z∪{±∞} with the discrete metric r(x, y) = 1{x = y}. Let Σ be the space of functions σ ∈ D(Rd , Z∗ ) that are nondecreasing [σ(x) ≤ σ(y) if x ≤ y in Rd ] and decay to −∞ sufficiently fast at −∞, namely   (65) for every b ∈ Rd , lim sup |y|−d/(d+1) σ(y) : y ≤ b, |y|∞ ≥ M = −∞. ∞ M →∞

Condition (65) is not preserved by convergence in the dS metric, so we need to fix the metric. For σ ∈ Σ, h ∈ Z, and b ∈ Rd , let y b,h (σ) be the maximal y ≤ b in Rd such that the rectangle [y, b] contains the set {x ≤ b : σ(x) ≥ h}. Condition (65) guarantees that such a finite y b,h (σ) exists. In fact, (65) is equivalent to (66)

lim |h|−(d+1)/d |y b,h (σ)|∞ = 0.

for every b ∈ Rd ,

h→−∞

For ρ, σ ∈ Σ and b ∈ Rd , define θb (ρ, σ) = sup |h|−(d+1)/d · |y b,h (ρ) − y b,h (σ)|∞ h≤−1

and Θ(ρ, σ) =



Rd

  e−|b|∞ 1 ∧ θb (ρ, σ) db.

Θ(ρ, σ) satisfies the triangle inequality, is symmetric, and Θ(σ, σ) = 0, so we can define a metric on Σ by dΣ (ρ, σ) = Θ(ρ, σ) + dS (ρ, σ). The effect of the Θ(ρ, σ) term in the metric is the following.

Growth model

229

Lemma 10.2. Suppose dS (σn , σ) → 0. Then dΣ (σn , σ) → 0 iff for every b ∈ Rd , (67)

lim

h→−∞

sup |h|−(d+1)/d |y b,h (σn )|∞ = 0, n

or equivalently, for every b ∈ Rd (68)

lim sup

M →∞

n

sup y≤b |y|∞ ≥M

σn (y) d/(d+1)

|y|∞

= −∞.

We leave the proof of the above lemma to the reader. Lemmas 10.1 and 10.2 together give a natural characterization of convergence in Σ. Lemma 10.3. The Borel σ-field BΣ is the same as the σ-field F generated by the coordinate projections σ → σ(x). Proof. The sets {x : σ(x) ≥ h} are closed, so the functions σ → σ(x) are upper semicontinuous. This implies F ⊆ BΣ . For the other direction one shows that for a fixed ρ ∈ Σ, the function σ → dΣ (ρ, σ) is F-measurable. This implies that the balls {σ ∈ Σ : dΣ (ρ, σ) < r} are F-measurable. Once we argue below that Σ is separable, this suffices for BΣ ⊆ F. To show the F-measurability of σ → dS (ρ, σ) one can adapt the argument from page 128 of [11]. To show the F-measurability of σ → Θ(ρ, σ), one can start by arguing the joint BRd ⊗ F-measurability of the map (b, σ) → y b,h (σ) from Rd × Σ into Rd . We leave the details. The remaining work is to check that (Σ, dΣ ) is a complete separable metric space. Proposition 10.4. The space (Σ, dΣ ) is complete. We prove this proposition in several stages. Let {σn } be a Cauchy sequence in the dΣ metric. By the completeness of (D, dS ), we already know there exists a σ ∈ D(Rd , Z∗ ) such that dS (σn , σ) → 0. We need to show that (i) σ ∈ Σ and (ii) Θ(σn , σ) → 0. Following the completeness proof for Skorohod space in [11], page 121, we may extract a subsequence, denoted again by σn , together with a sequence of Lipschitz functions ψn ∈ Λ (actually labeled µ−1 n in [11]), such that (69)

γ(ψn ) < 21−n

and (70)

σn (ψn (x)) → σ(x) uniformly on compact sets.

Step 1. σ ∈ Σ. Fix b ∈ Rd , for which we shall show (66). It suffices to consider b > 0. Let bk = 2 b + k1. By passing to a further subsequence we may assume Θ(σn , σn+1 ) < e−n . Fix n0 so that   (71) exp |b2 |∞ + d(n + 1) − n2 < 2−n for all n ≥ n0 . Lemma 10.5. For n ≥ n0 there exist points β n in Rd such that b1 < β n+1 < β n < b2 , and θβ n (σn , σn+1 ) < 2−n .

T. Sepp¨ al¨ ainen

230

Proof. Let αn = b1 + e−n · 1 in Rd . e−n ≥ Θ(σn , σn+1 ) 2

≥ where

inf

x∈(αn+1 ,αn )

{1 ∧ θx (σn , σn+1 )} · e−|b

2

|∞

· Lebd {x : αn+1 < x < αn },

Lebd {x : αn+1 < x < αn } = (e−n − e−n−1 )d ≥ e−d(n+1)

is the d-dimensional Lebesgue measure of the open rectangle (αn+1 , αn ). This implies it is possible to choose a point β n ∈ (αn+1 , αn ) so that θβ n (σn , σn+1 ) < 2−n . n+1

n

β n+1 < β n implies y β ,h (σn+1 ) ≥ y β ,h (σn+1 ) − (β n − β n+1 ). For each fixed h ≤ −1, applying the above Lemma inductively gives for n ≥ n0 : yβ

n+1

,h

n

(σn+1 ) ≥ y β ,h (σn+1 ) − (β n − β n+1 ) n ≥ y β ,h (σn ) − |h|(d+1)/d 2−n · 1 − (β n − β n+1 ) n  n0 2−k · 1 − (β n0 − β n+1 ), ≥ · · · ≥ y β ,h (σn0 ) − |h|(d+1)/d k=n0

from which then (72)

inf y b

n≥n0

1

,h

(σn ) ≥ y b

2

,h

(σn0 ) − |h|(d+1)/d 21−n0 · 1 − (b2 − b1 ).

Now fix h ≤ −1 for the moment. By (72) we may fix a rectangle [y 1 , b1 ] that contains the sets {x ≤ b1 : σn (x) ≥ h} for all n ≥ n0 . Let Q = [y 1 − 1, b1 + 1] be a larger rectangle such that each point in [y 1 , b1 ] is at least distance 1 from Qc . By (69) and (70) we may pick n large enough so that |ψn (x) − x| < 1/4 and σn (ψn (x)) = σ(x) for x ∈ Q. [Equality because Z∗ has the discrete metric.] We can now argue that if x ≤ b and σ(x) ≥ h, then necessarily x ∈ Q, ψn (x) ≤ b1 and σn (ψn (x)) ≥ h, which implies by (72) that x ≥ ψn (x) − (1/4)1 ≥ y b

1

,h

(σn ) − (1/4)1 ≥ y b

2

,h

(σn0 ) − (5/4 + |h|(d+1)/d 21−n0 ) · 1.

This can be repeated for each h ≤ −1, with n0 fixed. Thus for all h ≤ −1, $ # 2 |y b,h (σ)| ≤ |b| ∨ |y b ,h (σn0 )| + 5/4 + |h|(d+1)/d 21−n0 ,

and then, since σn0 ∈ Σ,

lim |h|−(d+1)/d |y b,h (σ)| ≤ 21−n0 .

h→−∞

Since n0 can be taken arbitrarily large, (66) follows for σ, and thereby σ ∈ Σ. Step 2. Θ(σn , σ) → 0. As for Step 1, let us assume that we have picked a subsequence σn that satisfies 2 (69) and (70) and Θ(σn+1 , σn ) < e−n . Let φn = ψn−1 . If we prove Θ(σn , σ) → 0 along this subsequence, then the Cauchy assumption and triangle inequality give it for the full sequence.

Growth model

231

Fix an arbitrary index n1 and a small 0 < ε0 < 1. Fix also β ∈ Rd . For each h ≤ −1, fix a rectangle [y h , β] that contains the sets {x ≤ β : σn (x) ≥ h} for each σn for n ≥ n1 , and also for σ, which Step 1 just showed lies in Σ. This can be done for each fixed h because by (72) there exists n0 = n0 (β) defined by (71) so that the points y β,h (σn ) are bounded below for n ≥ n0 . Then if necessary decrease y h further so that y h ≤ y β,h (σ1 ) ∧ y β,h (σ2 ) ∧ · · · ∧ y β,h (σn0 −1 ). Let Qh,k = [y h − k1, β + k1] be larger rectangles. On the rectangles Qh,2 , h ≤ −1, construct the finite partition for σ which satisfies (63) for ε = 1/2, so that the discrete metric forces σ to be constant on the partition rectangles. Consider a point b = (b1 , b2 , . . . , bd ) < β with the property that no coordinate of b equals any one of the (countably many) partition points. This restriction excludes only a Lebesgue null set of points b. Find ε1 = ε1 (β, b, h) > 0 such that the intervals (bi − ε1 , bi + ε1 ) contain none of the finitely many partition points that pertain to the rectangle Qh,2 . Pick n = n(β, b, h) > n1 such that σn (ψn (x)) = σ(x) and |ψn (x) − x| < (ε0 ∧ ε1 )/4 for x ∈ Qh,2 . Since the maps ψ, φ do not carry any points of [y h , β] out of Qh,1 , y h,b (σn ) = y h,b (σ ◦ φn ). It follows that |y h,b (σ) − y h,b (σn )| = |y h,b (σ) − y h,b (σ ◦ φn )| < ε0 . The last inequality above is justified as follows: The only way it could fail is that σ (or σ ◦ φn ) has a point x ≤ b with height ≥ h, and σ ◦ φn (respectively, σ) does not. These cannot happen because the maps ψ, φ cannot carry a partition point from one side of bi to the other side, along any coordinate direction i. Now we have for a.e. b < β and each h ≤ −1, with n = n(β, b, h) > n1 : |h|−(d+1)/d |y h,b (σ) − y h,b (σn1 )| ≤ |h|−(d+1)/d |y h,b (σ) − y h,b (σn )| + |h|−(d+1)/d |y h,b (σn ) − y h,b (σn1 )| ≤ ε0 + θb (σn , σn1 ) ≤ ε0 + sup θb (σm , σn1 ). m:m>n1

The last line has no more dependence on β or h. Since β was arbitrary, this holds for a.e. b ∈ Rd . Take supremum over h ≤ −1 on the left, to get 1 ∧ θb (σ, σn1 ) ≤ ε0 + Integrate to get

sup {1 ∧ θb (σm , σn1 )}

for a.e. b.

m:m>n1



e−|b|∞ {1 ∧ θb (σ, σn1 )} db  e−|b|∞ sup {1 ∧ θb (σm , σn1 )} db ≤ Cε0 + d m:m>n1 R  m−1 '   −|b|∞ ≤ Cε0 + e sup 1 ∧ θb (σk+1 , σk ) db

Θ(σ, σn1 ) =

Rd

m:m>n1

Rd

= Cε0 + = Cε0 +

∞ 

k=n1 ∞  k=n1



k=n1

e−|b|∞ {1 ∧ θb (σk+1 , σk )} db Rd

Θ(σk+1 , σk ) ≤ Cε0 +

∞ 

k=n1

e−k , 2

T. Sepp¨ al¨ ainen

232

where C =

(

Rd

e−|b|∞ db. Since n1 was an arbitrary index, we have lim sup Θ(σn1 , σ) ≤ Cε0 . n1 →∞

Since ε0 was arbitrary, Step 2 is completed, and Proposition 10.4 thereby proved. We outline how to construct a countable dense set in (Σ, dΣ ). Fix a < b in Zd . In the rectangle [a, b] ⊆ Rd , consider the (countably many) finite rational partitions of each coordinate axis. For each such partition of [a, b] into rectangles, consider all the nondecreasing assignments of values from Z∗ to the rectangles. Extend the functions σ ) thus defined to all of Rd in some fashion, but so that they are nondecreasing and Z∗ -valued. Repeat this for all rectangles [a, b] with integer ) of elements of D(Rd , Z∗ ). Finally, each such corners. This gives a countable set D ) yields countably many elements σ σ )∈D

∈ Σ by setting  −∞, σ )(x) < h σ

(x) = σ )(x), σ )(x) ≥ h

⊆ Σ. for all h ∈ Z. All these σ

together form a countable set Σ

Now given an arbitrary σ ∈ Σ, it can be approximated by an element σ

∈ Σ arbitrarily closely (in the sense that σ = σ

◦ φ for a map φ ∈ Λ close to the identity) σ ) is close to y b,h (σ) for all on any given finite rectangle [−β, β], and so that y b,h ( b in this rectangle, for any given range h0 ≤ h ≤ −1. Since |h|−(d+1)/d |y β,h (σ)| < ε

, φ, u) for h ≤ h0 for an appropriately chosen h0 , this suffices to make both d(σ, σ

) small for a range of u > 0 and b ∈ Rd . To get close under the metric and θb (σ, σ dΣ it suffices to approximate in a bounded set of u’s and b’s, so it can be checked

is dense in Σ. that Σ

Acknowledgments. The author thanks Tom Kurtz for valuable suggestions and anonymous referees for careful readings of the manuscript. Hermann Rost has also studied the process described here but has not published his results. References [1] Aldous, D. and Diaconis, P. (1995). Hammersley’s interacting particle process and longest increasing subsequences. Probab. Theory Related Fields 103 199–213. [2] Aldous, D. and Diaconis, P. (1999). Longest increasing subsequences: from patience sorting to the Baik-Deift-Johansson theorem. Bull. Amer. Math. Soc. (N.S.) 36 413–432. [3] Baik, J., Deift, P. and Johansson, K. (1999). On the distribution of the length of the longest increasing subsequence of random permutations. J. Amer. Math. Soc. 12 1119–1178. ´zs, M., Cator, E. and Seppa ¨ la ¨inen, T. (2006). Cube root fluctua[4] Bala tions for the corner growth model associated to the exclusion process. Electron. J. Probab. 11 1094–1132. [5] Bickel, P. J. and Wichura, M. J. (1971). Convergence criteria for multiparameter stochastic processes and some applications. Ann. Math. Statist. 42 1656–1670. ´s, B. and Brightwell, G. (1992). The height of a random partial [6] Bolloba order: Concentration of measure. Ann. Appl. Probab. 2 1009–1018. ´s, B. and Winkler, P. (1988). The longest chain among random [7] Bolloba points in Euclidean space. Proc. Amer. Math. Soc. 103 347–353.

Growth model

233

[8] Cator, E. and Groeneboom, P. (2006). Second class particles and cube root asymptotics for Hammersley’s process. Ann. Probab. 34 1273–1295. [9] Dafermos, C. M. (1977). Generalized characteristics and the structure of solutions of hyperbolic conservation laws. Indiana Univ. Math. J. 26 1097– 1119. [10] Evans, L. C. (1998). Partial Differential Equations. Amer. Math. Soc., Providence, RI. [11] Ethier, S. N. and Kurtz, T. G. (1986). Markov Processes: Characterization and Convergence, Wiley, New York. [12] Ferrari, P. A. (1992). Shock fluctuations in asymmetric simple exclusion. Probab. Theory Related Fields 91 81–110. [13] Ferrari, P. A. and Kipnis, C. (1995). Second class particles in the rarefaction fan. Ann. Inst. H. Poincar´e Probab. Statist. 31 143–154. [14] Groeneboom, P. (2002). Hydrodynamical methods for analyzing longest increasing subsequences. J. Comput. Appl. Math. 142 83–105. Special issue on Probabilistic Methods in Combinatorics and Combinatorial Optimization. [15] Hammersley, J. M. (1972). A few seedlings of research. Proc. Sixth Berkeley Symp. Math. Stat. Probab. I 345–394. [16] Liggett, T. M. (1999). Stochastic Interacting Systems: Contact, Voter and Exclusion Processes. Springer, Berlin. [17] Logan, B. F. and Shepp, L. A. (1977). A variational problem for random Young tableaux. Adv. Math. 26 206–222. [18] Rezakhanlou, F. (1995). Microscopic structure of shocks in one conservation laws. Ann. Inst. H. Poincar´e Anal. Non Lin´eaire 12 119–153. [19] Rezakhanlou, F. (2002). Continuum limit for some growth models I. Stochastic Process. Appl. 101 1–41. [20] Rezakhanlou, F. (2001). Continuum limit for some growth models II. Ann. Probab. 29 1329–1372. ¨ la ¨inen, T. (1996). A microscopic model for the Burgers equation and [21] Seppa longest increasing subsequences. Electron. J. Probab. 1 1–51. ¨ la ¨inen, T. (1998). Large deviations for increasing sequences on the [22] Seppa plane. Probab. Theory Related Fields 112 221–244. ¨ la ¨inen, T. (1999). Existence of hydrodynamics for the totally asym[23] Seppa metric simple K-exclusion process. Ann. Probab. 27 361–415. ¨ la ¨inen, T. (2000). Strong law of large numbers for the interface in [24] Seppa ballistic deposition. Ann. Inst. H. Poincar´e Probab. Statist. 36 691–736. ¨ la ¨inen, T. (2001). Perturbation of the equilibrium for a totally asym[25] Seppa metric stick process in one dimension. Ann. Probab. 29 176–204. ¨ la ¨inen, T. (2001). Second-class particles as microscopic characteristics [26] Seppa in totally asymmetric nearest-neighbor K-exclusion processes. Trans. Amer. Math. Soc. 353 4801–4829. ¨ la ¨inen, T. (2002). Diffusive fluctuations for one-dimensional totally [27] Seppa asymmetric interacting random dynamics. Comm. Math. Phys. 229 141–182. [28] Smythe, R. T. and Wierman, J. C. (1978). First-Passage Percolation on the Square Lattice. Lecture Notes in Math. 671. Springer, Berlin. [29] Versik, A. M. and Kerov, S. V. (1977). Asymptotic behavior of the Plancherel measure of the symmetric group and the limit form of Young tableaux. (Russian) Dokl. Akad. Nauk SSSR 233 1024–1027. English translation: Soviet Math. Dokl. 18 527–531. [30] Winkler, P. (1985). Random orders. Order 1 317–331.

IMS Lecture Notes–Monograph Series Asymptotics: Particles, Processes and Inverse Problems Vol. 55 (2007) 234–252 c Institute of Mathematical Statistics, 2007
DOI: 10.1214/074921707000000382

Empirical processes indexed by estimated functions Aad W. van der Vaart1 and Jon A. Wellner2,∗ Vrije Universiteit Amsterdam and University of Washington Abstract: We consider the convergence of empirical processes indexed by functions that depend on an estimated parameter η and give several alternative conditions under which the “estimated parameter” ηn can be replaced by its natural limit η0 uniformly in some other indexing set Θ. In particular we reconsider some examples treated by Ghoudi and Remillard [Asymptotic Methods in Probability and Statistics (1998) 171–197, Fields Inst. Commun. 44 (2004) 381–406]. We recast their examples in terms of empirical process theory, and provide an alternative general view which should be of wide applicability.

1. Introduction Let X1 , . . . , Xn be i.i.d. random elements in a measurable space (X , A) with law P , and for a measurable function f : X → R let the expectation, empirical measure and empirical process at f be denoted by Pf =




n

f dP,

Pn f =

1 f (Xi ), n i=1

Gn f =

√ n(Pn − P )f.

Given a collection {fθ,η : θ ∈ Θ, η ∈ H} of measurable functions fθ,η : X → R indexed by sets Θ and H and “estimators” ηn , we wish to prove that, as n → ∞,     (1) sup Gn (fθ,ηn − fθ,η0 ) →p 0. θ∈Θ

Here an “estimator” ηn is a random element with values in H defined on the same probability space as X1 , . . . , Xn , and η0 ∈ H is a fixed element, which is typically a limit in probability of the sequence ηn . The result (1) is interesting for several applications. A direct application is to the estimation of the functional θ → P fθ,η . If the parameter η is unknown, we may replace it by an estimator ηn and use the empirical estimator Pn fθ,ηn . The result (1) helps to derive the limit behaviour of this estimator, as we can decompose √ √ n(Pn fθ,ηn − P fθ,η0 ) = Gn (fθ,ηn − fθ,η0 ) + Gn fθ,η0 + nP (fθ,ηn − fθ,η0 ). (2) If (1) holds, then the first term on the right converges to zero in probability. Under appropriate conditions on the functions fθ,η0 , the second term on the right ∗ Supported in part by NSF Grant DMS-05-03822, NI-AID Grant 2R01 AI291968-04, and by grant B62-596 of the Netherlands Organisation of Scientific Research NWO 1 Section Stochastics, Department of Mathematics, Faculty of Sciences, Vrije Universiteit, De Boelelaan 1081a, 1081 HV Amsterdam, e-mail: [email protected] 2 University of Washington, Department of Statistics, Box 354322, Seattle, Washington 981954322, USA, e-mail: [email protected] AMS 2000 subject classifications: 62G07, 62G08, 62G20, 62F05, 62F15. Keywords and phrases: delta-method, Donsker class, entropy integral, pseudo observation.

234

Random empirical processes

235

will converge to a Gaussian process by the (functional) central limit theorem. The behavior of the third term depends on the estimators ηn , and would typically follow from an application of the (functional) delta-method, applied to the map η → (P fθ,η : θ ∈ Θ). In an interesting particular case of this situation, the functions fθ,η take the form   fθ,η (x) = θ η(x) , for maps θ : Rd → R and each η ∈ H being a map η : X → Rd . The realizations of the estimators ηn are then functions x → ηn (x) = ηn (x; X1 , . . . , Xn ) on the sample space X and can be evaluated at the observations to obtain the random vectors ηn (X1 ), . . . , ηn (Xn ) in Rd . The process {Pn fθ,ηn : θ ∈ Θ} is the empirical measure of these vectors indexed by the functions θ. For instance, if Θ consists of the indicator functions 1(−∞,θ] for θ ∈ Rd , then this measure is the empirical distribution function n

θ → Pn fθ,ηn =

1 1{ηn (Xi ) ≤ θ} n i=1

of the random vectors ηn (X1 ), . . . , ηn (Xn ). The properties of such empirical processes were studied in some generality and for examples of particular interest in Ghoudi and Remillard [6, 7]. Ghoudi and Remillard [6] apparently coined the name “pseudoobservations” for the vectors ηn (X1 ), . . . , ηn (Xn ). The examples include, for instance, regression residuals, Kendall’s dependence process, and copula processes; see the end of Section 2 for explicit formulation of these three particular examples. One purpose of the present paper is to extend the results in these papers also to other index classes Θ besides the class of indicator functions. Another purpose is to recast their results in terms of empirical process theory, which leads to simplification and alternative conditions. A different, indirect application of (1) is to the derivation of the asymptotic distribution of Z-estimators. A Z-estimator for θ might be defined as the solution θˆn of the equation Pn fθ,ηn = 0, where again an unknown “nuisance” parameter η is replaced by an estimator ηn . In this case (1) shows that √ Pn fθˆn ,ηn − Pn fθˆn ,η0 = P (fθˆn ,ηn − fθˆn ,η0 ) + oP (1/ n), so that the limit behavior of θˆn can be derived by comparison with the estimating equation defined by Pn fθ,η0 (with η0 substituted for ηn ). The “drift” sequence P (fθˆn ,ηn − fθˆn ,η0 ), which will typically be equivalent to P (fθ0 ,ηn − fθ0 ,η0 ) up to √ order oP (1/ n), may give rise to an additional component in the limit distribution. The paper is organized as follows. In Section 2 we derive general conditions for the validity of (1) and formulate several particular examples to be considered in more detail in the sequel. In Section 3 we specialize the general results to composition maps. In Section 4 we combine these results with results on Hadamard differentiability to obtain the asymptotic distribution of empirical processes indexed by pseudo observations. Finally in Section 5 we formulate our results for several of the particular examples mentioned above and at the end of Section 2. 2. General result In many situations we wish to establish (1) without knowing much about the nature of the estimators ηn , beyond possibly that they are consistent for some value η0 .

236

A. W. van der Vaart and J. A. Wellner

For instance, this is true if (1) is used as a step in the derivation of M − or Z− estimators. (Cf. Van der Vaart and Wellner [12] and Van der Vaart [11].) Then an appropriate method of establishing (1) is through a Donsker or entropy condition, as in the following theorems. Proofs of the Theorems 2.1 and 2.2 can be found in the mentioned references. Both theorems assume that ηn is “consistent for η0 ” in the sense that (3)

sup P (fθ,ηn − fθ,η0 )2 →p 0. θ∈Θ

Theorem 2.1. Suppose that H0 is a fixed subset of H such that Pr(ηn ∈ H0 ) → 1 and suppose that the class of functions {fθ,η : θ ∈ Θ, η ∈ H0 } is P -Donsker. If (3) holds, then (1) is valid. For the second theorem, let N (, F, L2 (P )) and N[ ] (, F, L2 (P )) be the -covering and -bracketing numbers of a class F of measurable functions (cf. Pollard [8] and van der Vaart and Wellner [12]) and define entropy integrals by
δ  (4) sup log N (F Q,2 , F, L2 (Q)) d, J(δ, F, L2 ) = (5)

J[ ] (δ, F, L2 (P )) =




Q

0

δ

0

 log N[ ] (F P,2 , F, L2 (P )) d.

Here F is an arbitrary, measurable envelope function for the class F: a measurable function F : X → R such that |f (x)| ≤ F (x) for every f ∈ F and x ∈ X . We say that a sequence Fn of envelope functions satisfies the Lindeberg condition if P Fn2 = O(1) and P Fn2 1Fn ≥√n → 0 for every  > 0. Theorem 2.2. Suppose that Hn are subsets of H such that Pr(ηn ∈ Hn ) → 1 and such that the classes of functions Fn = {fθ,η : θ ∈ Θ, η ∈ Hn } satisfy either J[·] (δn , Fn , L2 (P )) → 0, or J(δn , Fn , L2 ) → 0 for every sequence δn → 0, relative to envelope functions that satisfy the Lindeberg condition. In the second case also assume that the classes Fn are suitably measurable (e.g. countable). If (3) holds, then (1) is valid. Because there are many techniques to verify that a given class of functions is Donsker, or to compute bounds on its entropy integrals, the preceding lemmas give quick results, if they apply. Furthermore, they appear to be close to best possible unless more information about the estimators ηn can be brought in, or explicit computations are possible for the functions fθ,η . In some applications the estimators ηn are known to converge at a certain rate and/or known to possess certain regularity properties (e.g. uniform bounded derivatives). Such knowledge cannot be exploited in Theorem 2.1, but could be used for the choice of the sets Hn in Theorem 2.2. We now discuss an alternative approach which can be used if the estimators ηn are also known to converge in distribution, if properly rescaled. √ Let H be a Banach space, and suppose that the sequence n(ηn − η0 ) converges in distribution to a tight, Borel-measurable random element in H. The “convergence in distribution” may be understood in the sense of Hoffmann-Jørgensen, so that ηn need not be Borel-measurable itself. √ The tight limit of the sequence n(ηn −η0 ) takes its values in a σ-compact subset H0 ⊂ H. For θ ∈ Θ, h0 ∈ H0 , and δ > 0 define a sequence of classes of functions by   (6) Fn (θ, h0 , δ) = fθ,η0 +n−1/2 h − fθ,η0 +n−1/2 h0 : h ∈ H, h − h0  < δ .

Random empirical processes

237

Let Fn (θ, h0 , δ) be arbitrary measurable envelope functions for these classes. √ Theorem 2.3. Suppose that the sequence n(ηn − η0 ) converges in distribution to a tight, random element with values in a given σ-compact subset H0 of H. Suppose that (i) supθ |Gn (fθ,η0 +n−1/2 h0 − fθ,η0 )| →p 0 for every h0 ∈ H0 . h0 , δ)| →p 0 for every δ > 0 and every h0 ∈ H0 ; (ii) supθ |Gn Fn (θ,√ (iii) supθ suph0 ∈K n P Fn (θ, h0 , δn ) → 0 for every δn → 0 and every compact K ⊂ H0 ; Then (1) is valid.

√ Proof. Suppose that n(ηn − η0 ) ⇒ Z and let  > 0 be fixed. There exists a compact set K ⊂ H0 with P (Z ∈ K) > 1 −  and hence for every δ > 0, with K δ the set of all points at distance less than δ to K,  √ lim inf Pr n(ηn − η0 ) ∈ K δ/2 > 1 − . n→∞

In view of the compactness of K there exist finitely many elements h1 , . . . , hp ∈ K ⊂ H0 (with p = p(δ) depending on δ) such that the balls of radius δ/2 around these points cover K. Then K δ/2 is contained in the union of the balls of radius δ, by the triangle inequality. Thus, with B(h, δ) denoting the ball of radius δ around h in the space H, p(δ)    √ ηn ∈ B(η0 + n−1/2 hi , δ) . n(ηn − η0 ) ∈ K δ/2 ⊂ i=1

It follows that with probability at least 1 − , as n → ∞, sup |Gn (fθ,ηn − fθ,η0 )| θ

≤ sup max i

θ

≤ sup max i

θ

sup h−hi <δ

sup h−hi <δ

|Gn (fθ,η0 +n−1/2 h − fθ,η0 )|

|Gn (fθ,η0 +n−1/2 h − fθ,η0 +n−1/2 hi )| + |Gn (fθ,η0 +n−1/2 hi − fθ,η0 )|

≤ sup max |Gn Fn (θ, hi , δ)| + 2 sup sup i

θ

θ

h0 ∈K



√ n P Fn (θ, h0 , δ)

+ sup max |Gn (fθ,η0 +n−1/2 hi − fθ,η0 )|, θ

i

√ where in the last step we use the inequality |Gn f | ≤ |Gn F | + 2 nP F , valid for any functions f and F with |f | ≤ F . The maxima in the display are over the finite set i = 1, . . . , p(δ), and the elements h1 , . . . , hp(δ) ∈ K depend on δ. By assumptions (i) and (ii) the first and third terms converge to zero as n → ∞, for every fixed δ. It follows that there exists δn ↓ 0 such that these terms with δn substituted for δ converge to 0. For this δn , all three terms converge to zero in probability as n → ∞. √ The rate of convergence n in the preceding theorem may be √ replaced by another rate, with appropriate changes in the conditions, but the rate n appears natural in the following context. For more general metrizable topological vector spaces there are similar, but less attractive, results possible.

A. W. van der Vaart and J. A. Wellner

238

The two conditions (i), (ii) of Theorem 2.3 concern the empirical process indexed by the classes of functions (7)

{fθ,η0 +n−1/2 h0 − fθ,η0 : θ ∈ Θ},

(8)

{Fn (θ, h0 , δ) : θ ∈ Θ}.

These classes are indexed by Θ only, and hence Theorem 2.3, if applicable, avoids conditions for (1) that involve measures of the complexity of the class {fθ,η : θ ∈ Θ, η ∈ H} due to the parameter η ∈ H. Condition (iii) of Theorem 2.3 involves the mean of the envelopes of the classes Fn (θ, h0 , δ). For the minimal envelopes this condition takes the form √ sup sup n P (9) sup |fθ,η0 +n−1/2 h − fθ,η0 +n−1/2 h0 | → 0 θ

h0 ∈K

h−h0 <δn

for all δn ↓ 0. This is an “integrated uniform local Lipschitz assumption” on the dependence η → fθ,η . In some applications it may be useful not to use the minimal envelope functions. The lemma is valid for any envelope functions, as long as the same envelopes are used in both (ii) and (iii). The set K in (iii) √ or (9) is a compact set in the support of the limit distribution of the sequence n(ηn − η0 ). In some cases condition (iii) may be valid for any compact K ⊂ H, whereas in other cases more precise information about the limit √ process must be exploited. For instance, if the sequence n(ηn − η0 ) converges in distribution to a tight zero-mean Gaussian process G in the space H = ∞ (T ) of bounded functions on some set T , then K may be taken to be a set of functions z : T → R that is uniformly bounded and uniformly equicontinuous relative to the semimetric with square d2 (s, t) = E(Gs − Gt )2 (and T will be totally bounded for d). Cf. e.g. van der Vaart and Wellner [12], page 39. Condition (iii) is an analytical condition, whereas conditions (i) and (ii) are empirical process conditions. In many cases the latter pair of conditions can be verified by standard empirical process type arguments. For reference we quote two lemmas that allow handling the empirical process indexed by a sequence of classes, as in (8) or (7). (For proofs see e.g. van der Vaart [10, 11].) Both lemmas apply to classes Fn of measurable functions f : X → R such that (10)

sup P f 2 → 0.

f ∈Fn

Lemma 2.1. Suppose that the class of functions holds, then supf ∈Fn |Gn (f )| →p 0.



n

Fn is P -Donsker. If (10)

Lemma 2.2. Suppose that either J[·] (δn , Fn , L2 (P )) → 0 or J(δn , Fn , L2 ) → 0 for all δn ↓ 0 relative to envelope functions Fn that satisfy the Lindeberg condition. In the second case also assume that each class Fn is suitably measurable. If (10) holds, then supf ∈Fn |Gn (f )| →p 0. Example 1 (Regression residual processes). Suppose that (X1 , Y1 ), . . . , (Xn , Yn ) are a random sample distributed according to the regression model Y = gη (X)+ e. For given estimators ηn we can form the residuals eˆi = Yi − gηn (Xi ) and may be . . , eˆn , i.e. for a collection interested in the empirical process corresponding to eˆ1 , .  n ei ) : θ ∈ Θ . This Θ of functions θ : R → R we consider the process n−1 i=1 θ(ˆ  fits the general set-up with the functions fθ,η defined as fθ,η (x, y) = θ y − gη (x) .

Random empirical processes

239

In many cases it will be possible to apply Theorem 2.1. For instance, if x ∈ Rd , gη is a polynomial in x, and Θ is the class of indicator functions 1(−∞,θ] for θ ∈ R, then the functions fθ,η are the indicator functions of the sets {(x, y) : y − gη (x) − θ ≤ 0}. Because the set of functions (x, y) → y−gη (x)−θ is contained in a finite-dimensional vector space, it is a VC-class, and hence so are their negativity sets (e.g. van der Vaart and Wellner [12], Lemma 2.6.18). Thus the class of functions fθ,η is Donsker, and Theorem 2.1 can be applied directly. Example 2 (Kendall’s process). Let ηn be the empirical distribution function of a random sample X1 , . . . , Xn from a distribution η0 on Rd . Barbe, Genest, Ghoudi and Remillard [2] and Ghoudi and Remillard [6] study the behavior of the empirical distribution function Kn of the pseudo-observations ηn (Xi ), n

Kn (θ) =

1 1{ηn (Xi ) ≤ θ}, n i=1

and the resulting Kendall’s process √ n(Kn (θ) − K(θ)), (11)

θ ∈ [0, 1],

θ ∈ [0, 1]

where K(θ) = P (η0 (X) ≤ θ). This fits the general set-up with fθ,η the composition function fθ,η = θ ◦ η, and θ the indicator function 1(−∞,θ] (where we abuse notation by using the symbol θ in two different ways). An attempt to apply Theorem 2.1 to this problem would lead to the consideration of the class of all indicator functions of sets of the form {x ∈ Rd : η(x) ≤ θ} for η ranging over the cumulative distribution functions on Rd and θ ∈ [0, 1]. This class is similar to the collection of all “lower layers” in Rd , and, unfortunately, fails to be Donsker for most distributions (cf. Dudley [3], page 264, 373 or Dudley [4]). In this √ case it appears to be necessary to exploit the limit behaviour of the sequence n(ηn − η0 ). Ghoudi and Remillard [6] have shown that (1) is valid in this case, under some strong smoothness assumptions on the underlying measure η0 . In Sections 4 and 5 we rederive some of their results by empirical process methods using Theorem 2.3. We also consider the empirical process of the variables ηn (X1 ), . . . , ηn (Xn ) indexed by classes of functions other than the indicators 1(−∞,θ] . If the indexing functions are smooth, then this empirical process will converge even without smoothness conditions on η0 . A proof can be based on Theorem 2.3. Example 3 (Copula processes). Suppose that X1 , . . . , Xn are a sample from a distribution η0 on Rd . Write Xi = (Xi,1 , . . . , Xi,d ) and let η0,1 , . . . , η0,d be the marginal distributions. Thecopula function C associated with η0 is the distribu −1 (u) = inf{x : tion function of the vector η0,1 (X1,1 ), . . . , η0,d (X1,d ) , i.e. with η0,j η0,j (x) ≥ u} for u ∈ [0, 1], −1 −1 (u1 ), . . . , η0,d (ud )) C(u1 , . . . , ud ) = η0 (η0,1

for (u1 , . . . , ud ) ∈ [0, 1]d . For j = 1, . . . , d let ηn,j be the empirical distribution function of X1,j , . . . , Xn,j (on R), and let ηn be the empirical distribution function of X1 , . . . , Xn (on Rd ). Then a natural estimator Cn of C is given by n

Cn (u) =

1 1{ˆ en,i ≤ u}, n i=1

u ∈ [0, 1]d ,

240

A. W. van der Vaart and J. A. Wellner

  for the “pseudo-observations” eˆn,i = ηn,1 (Xi,1 ), . . . , ηn,d (Xi,d ) . The resulting “copula processes”, √ n(Cn (u) − C(u)), u ∈ [0, 1]d , (12) have been considered by Stute [9], G¨ anssler and Stute [5], and Ghoudi and Remillard [7]. This example can be treated using Theorem 2.3, but also with the more straightforward Theorem 2.1, or even by employing the theory of Hadamard differentiability, as in Chapter 3.9 of Van der Vaart and Wellner [12]. 3. Composition In this section we consider the case where the functions fθ,η take the form (13)

fθ,η (x) = θ(η(x)),

for θ ranging over a class Θ of functions θ : Rd → R and η ranging over a class H of measurable functions η : X → Rd . We first give general conditions for the validity of condition (iii) of Theorem 2.3, and next consider also the conditions (i) and (ii) for the special cases of functions θ that are Lipschitz and √ monotone, respectively. We develop these results for the case that the sequence n(ηn − η0 ) converges in functions z : X → distribution in the space H = ∞ (X , Rd ) of uniformlybounded  Rd , equipped with the uniform norm z = supx∈X z(x). (Variations of these results are possible. For instance, Rd could be replaced by a more general Banach space, and H could be equipped with a weighted uniform norm.) 3.1. Condition (i) For fθ,η taking the form (13), condition (i) of Theorem 2.3 takes the form (14)

sup |Gn,Q f | →Q 0

f ∈Fn

for Q = P ◦ (η0 , h0 )−1 , Gn,Q the empirical process of a random sample from the measure Q, and Fn the class of functions   (15) Fn = (y, z) → θ(y + n−1/2 z) − θ(y) : θ ∈ Θ .

Condition (i) requires that (14) is valid for every fixed choice of h0 ∈ H0 , i.e. for every measure Q determined as the law of (η0 (X), h0 (X)) for some h0 ∈ H0 and X distributed according to P . This situation is of the form considered in Lemmas 2.1 and 2.2, and both lemmas may be applicable in a given setting. It is not especially helpful to restate these lemmas for the present special situation. Instead, we give one easy to check set of sufficient conditions. This covers VC-classes Θ, and much more. If Θenv : Rd → R is an envelope function for Θ, then

(16)

Fn (y, z) = Θenv (y + n−1/2 z) + Θenv (y)

is an envelope function for Fn . (A crude one, because we do not exploit that the functions in Fn are differences.)

Random empirical processes

241

Lemma 3.1. Suppose that J(1, Θ, L2 ) < ∞, that Θ is suitably measurable, that P (Θenv ◦η0 )2 < ∞, and that the functions Θenv ◦(η0 +n−1/2 h0 ) satisfy the Lindeberg condition in L2 (P ), for every h0 ∈ H0 . If 2  sup P θ ◦ (η0 + n−1/2 h0 ) − θ ◦ η0 → 0,

θ∈Θ

for every h0 ∈ H0 , then condition (i) of Theorem 2.3 is satisfied for the functions fθ,η given by (13). Proof. It suffices to prove (14) for the classes Fn given in (15). The class Fn is contained in the difference of the classes Fn = {(y, z) → θ(y + n−1/2 z) : θ ∈ Θ}, F  = {(y, z) → θ(y) : θ ∈ Θ}. These classes possess envelope functions Fn and F  defined by Fn (y, z) = Θenv (y + n−1/2 z), F  (y, z) = Θenv (y). The uniform entropy of F  relative to F  is finite by assumption. The uniform entropy of Fn relative to Fn is exactly the same, as the law of Y + n−1/2 Z runs through all possible laws on Rd if the law of (Y, Z) runs through all possible laws on Rd × Rd . The uniform entropy of Fn relative to Fn is bounded by the sum of the uniform entropies of Fn and F  . (Cf. e.g. Theorem 2.10.20 of van der Vaart and Wellner [12].) Now apply Lemma 2.2. 3.2. Lipschitz functions θ Assume that every function θ : Rd → R in the class Θ is uniformly Lipschitz in that |θ(r1 ) − θ(r2 )| ≤ r1 − r2 .

(17) Then, for every x ∈ X ,

     h(x) − h0 (x)  √ . θ η0 (x) + n−1/2 h(x) − θ η0 (x) + n−1/2 h0 (x)  ≤ n

The norm in the right side is bounded by the supremum norm h − h0  on ∞ (X , Rd ). It follows that the classes Fn (θ, h0 , δ) as in (6) possess envelope functions √ Fn (θ, h0 , δ) = δ/ n.

(18)

Theorem 3.1. If Θ is a suitably measurable collection of uniformly bounded, uniformly Lipschitz functions θ : Rd → R such that J(1, Θ, L2 ) <√∞ (relative to a constant envelope function), η0 ∈ ∞ (X , Rd ), and the sequence n(ηn − η0 ) converges weakly in ∞ (X , Rd ) to a tight random element, then    supGn θ(ηn ) − θ(η0 )  →p 0. θ∈Θ

242

A. W. van der Vaart and J. A. Wellner

Proof. With the envelope functions Fn (θ, h0 , δ) as defined in (18), condition (ii) of Theorem 2.3 is trivially satisfied because the envelopes are actually constants, and the validity of condition (iii) is immediate. By assumption we can choose the envelope function of Θ equal to a constant and J(1, Θ, L2 ) < ∞. This suffices for the verification of most of the conditions of Lemma 3.1. Finally, it suffices to note that 2  P θ ◦ (η0 + n−1/2 h0 ) − θ ◦ η0 ≤ h0 2 /n. By Lemma 3.1 we conclude that condition (i) of Theorem 2.3 is also satisfied, whence the theorem follows from Theorem 2.3.

For the verification of condition (i) of Theorem √ 2.3 it suffices to consider the functions θ on the range of the functions η√ 0 + h0 / n for a fixed h0 in the support of the limit distribution of the sequence n(ηn − η0 ). Thus we may restrict the functions θ to a subset of Rd that contains the ranges of these functions and interpret the condition J(1, Θ, L2 ) < ∞ in Lemma 3.1 accordingly. In particular, in Theorem 2.3 we may replace this condition by the condition that J(1, ΘK , L2 ) < ∞ for every norm-bounded subset K ⊂ Rd , where ΘK is the collection of restrictions θ : K → R of the functions θ ∈ Θ. Any collection of uniformly bounded, Lipschitz functions θ : K → R on a compact interval K satisfies J(1, Θ, L2 ) < ∞. (Cf. e.g. van der Vaart and Wellner [12], page 157.) Thus in the case that d = 1 the assertion of the preceding theorem is true for any collection of uniformly bounded Lipschitz functions. For d > 1 further restrictions on the class Θ may be necessary. For instance, any subset of the unit ball in the H¨ older space C α (K) for a compact interval K ⊂ Rd possesses a finite uniform entropy integral provided α > d/2. (Cf. e.g. van der Vaart and Wellner [12], page 157.) The assertion of the preceding theorem is also true for such a class. There are many other examples of classes of Lipschitz functions with finite uniform entropy integrals. For instance, VC-classes of Lipschitz functions. 3.3. Monotone functions θ Assume that every function θ : Rd → R in Θ is the survival function θ(x) =  1[x,∞) dθ of a subprobability measure on Rd . Then each θ is nonincreasing in each of its arguments. If H = ∞ (X , Rd ) equipped with the uniform norm relative to the max-norm on Rd , then     θ(η0 + n−1/2 h) − θ(η0 + n−1/2 h0 ) ≤ θ(η0 + n−1/2 h0 − n−1/2 h − h0 ) − θ(η0 + n−1/2 h0 + n−1/2 h − h0 ).

It follows that the classes Fn (θ, h0 , δ) possess envelope functions, with δ the vector (δ, . . . , δ), Fn (θ, h0 , δ) = θ(η0 + n−1/2 h0 − n−1/2 δ) − θ(η0 + n−1/2 h0 + n−1/2 δ). In order to verify condition (iii) of Theorem 2.3, we assume that for given (possibly infinite) a < b in Rd and every δn ↓ 0 and compact set K ⊂ H0 ∪ {0},   √ (19) sup sup nP 1η0 +n−1/2 h0 ≤t+n−1/2 δn − 1η0 +n−1/2 h0 ≤t → 0. t∈Rd ,a≤t≤b h0 ∈K

Random empirical processes

243

Theorem 3.2. Let Θ be a collection of survival functions θ : Rd → [0, 1]√of subprobability measures supported on an interval (a, b) ⊂ Rd . If the sequence n(ηn − η0 ) converges in distribution in ∞ (X , Rd ) to a tight Borel measure concentrating on the σ-compact set H0 , and (19) holds for every δn ↓ 0 and every compact K ⊂ H0 ∪{0}, then    supGn θ(ηn ) − θ(η0 )  →p 0. θ

Proof. The survival functions of subprobability measures are in the convex hull of the set of indicator functions 1[t,∞) , which is a VC-class. Therefore the entropy integral J(1, Θ, L2 ) of Θ relative to a constant envelope is finite. (Cf. e.g. van der Vaart and Wellner [12], page 145.) Defining Fn (θ, η0 , δ) as in the display preceding the theorem, we can write P Fn (θ, η0 , δ)
  = P 1(−∞,s] (η0 + n−1/2 h0 − n−1/2 δ) − 1(−∞,s] (η0 + n−1/2 h0 ) dθ(s)   ≤ θ sup P 1(−∞,s] (η0 + n−1/2 h0 − n−1/2 δ) − 1(−∞,s] (η0 + n−1/2 h0 ) . a≤s≤b

√ By assumption (19) the right side converges to zero faster than 1/ n, for every δ = δn ↓ 0, uniformly in h0 ∈ K, and uniformly in θ because the total variation norms θ are uniformly bounded. This verifies condition (iii) of Theorem 2.3. Because θ is monotone with range contained in [0, 1],  2   P θ(η0 + n−1/2 δ) − θ(η0 ) ≤ sup P 1(−∞,s] (η0 − n−1/2 δ) − 1(−∞,s] (η0 ) . a≤s≤b

√ By assumption (19) with h0 = 0 this converges to zero faster than 1/ n for every sequence δn ↓ 0. This can be √ seen to imply that the expression in the display (which n)√converges to does not have the leading  zero  also for√fixed δ. By monotonicity  of θ we can bound θ(η0 + h0 / n) − θ(η0 ) by θ(η0 − δ/ n) − θ(η0 ) for δ = h0 . By Lemma 3.1 we now conclude that condition (i) of Theorem 2.3 is satisfied. In the present case the envelope functions Fn (θ, h0 , δ) are equal to the functions fθ,η0 +n−1/2 h − fθ,η0 −n−1/2 h for h = h0 − δ. Therefore, the validity of condition (ii) of Theorem 2.3 follows by the same arguments as used for the validity of condition (i). Condition (19) is a uniform√Lipschitz condition on the distribution functions of the variables η0 (X) + h0 (X)/ n. If the distribution of η0 (X) is smooth, then we might expect that the distribution functions of the perturbed variables η0 (X) + √ h0 (X)/ n will be smooth as well. However, this appears not to be true in general, and it will usually be necessary to exploit some information about the functions h0 . (We √ need to consider functions in the support of the limit measure of the sequence n(ηn − η0 ).) In this respect the conditions of Theorem 3.2 for composition with monotone functions are much more stringent than the conditions of Theorem 3.1 for the composition with Lipschitz functions. The condition (19) is in terms of the indicator functions 1(−∞,s] , and would have exactly the same form if we considered only indicator functions θ = 1(−∞,θ] , rather than general monotone functions. Thus the restrictive condition is connected to studying the classical empirical process. The following lemma allows the verification of condition (19) in many cases. It will also be used in the next section to prove applicability of the delta-method. The lemma is similar to Lemma 5.1 of Ghoudi and R´emillard [6].

244

A. W. van der Vaart and J. A. Wellner

Lemma 3.2. Suppose that X, Y, Yt (with t > 0) are real-valued random variables on a common probability space such that (i) X possesses a Lebesgue density f that is continuous in a neighbourhood of x; (ii) Yt − Y ∞ → 0 and Y ∞ < ∞; (iii) the conditional distribution of Y given X = s can be represented by a Markov kernel K(s, ·) such that the map s → K(s, ·) is continuous at x for the weak topology. Then for every continuous function g : R → R and every converging sequences xt → x, at → a and 0 ≤ bt → b, as t → 0,
1 Eg(Yt )1xt <X+tat Yt ≤xt +tbt → b g(y) K(x, dy) f (x). t Proof. First consider the case that Yt = Y for every t. By the definitions of K and f , we can write 1 Eg(Y )1xt <X+tat Y <xt +tbt t
  1 E g(Y )1(xt −s)/t
 This reduces to b g(y) K(x, dy) f (x) by Fubini’s theorem. This concludes the proof of the lemma for the case that Yt = Y . Because g is continuous, Y possesses bounded range and Yt − Y ∞ → 0 we have that g(Yt ) − g(Y )∞ → 0. Therefore  1  E g(Yt ) − g(Y )1xt <X+tat Yt <xt +tbt t  1  ≤ o(1) Pr xt − tat Yt ∞ < X < xt + t + tat Yt ∞ . t

This converges to zero as X has a density that is bounded on bounded intervals. Finally the difference 1xt <X+tat Yt <xt +tbt − 1xt <X+tat Y <xt +tbt is nonzero only if X + tat Y is in the union of intervals of total length bounded by tat Yt − Y ∞ in a neighbourhood of x. By the lemma with Yt = Y and g = 1, which is already proved, t−1 Pr(xt < X + tat Y < xt + tct ) → cf (x) for any sequences xt → x and ct → c.

Random empirical processes

245

Hence this probability converges to zero for ct = tat Yt − Y ∞ , which satisfied ct → 0. We conclude from this that  1  Eg(Y )(1xt <X+tat Yt <xt +tbt − 1xt <X+tat Y <xt +tbt ) t 1 ≤ g(Y )∞ E|1xt <X+tat Yt <xt +tbt − 1xt <X+tat Y <xt +tbt | t converges to zero. The proof of the lemma is complete upon combining the preceding. In order to verify (19) with K = {h0 } a single function we can apply the preceding   (X), h (X) , b = δn , and t = lemma with (X, Y ) equal to the pair of variables η 0 0 t √ 1/ n. Then the conditions of the lemma require that the variable η0 (X) possesses a continuous density, and that the conditional distribution of h0 (X) given η0 (X) = z depends continuously on the value of z. The second condition is clearly unpleasant, but appears to be natural in the present situation. It will √ involve a closer analysis of the support of the limit distribution of the sequence n(ηn − η0 ). To verify (19) with a general compact set K ⊂ ∞ (X , R) we simply note that in view of the compactness it suffices to verify that for every sequence hn such that hn − h0 ∞ → 0 for some h0 ∈ H0 ,   √ n P 1η0 +n−1/2 hn ≤t+n−1/2 δn − 1η0 +n−1/2 hn ≤t → 0. sup t∈Rd ,a≤t≤b

 preceding√lemma with the variables (X, Y, Yt ) equal to Thus we can apply the η0 (X), h0 (X), hn (X) and t = 1/ n.

Example 2, continued. Suppose that ηn is the empirical distribution of a random sample from the cumulative distribution function η0 on Rd . Then the limit √ distribution of the sequence n(ηn − η0 ) is the d-dimensional η0 -Brownian sheet on Rd . If d = 1, then the Brownian sheet is a Brownian bridge and can be represented A typical function as B ◦ η0 for B a standard Brownian bridge on the unit interval. √ in the support of the limit distribution of the sequence n(ηn − η0 ) can be represented as h0 = h ◦ η0 for some function h : [0, 1] → R. The conditional law of the variable h0 (X) given η0 (X) = z is the Dirac measure at h(z). Because the standard Brownian bridge is continuous, the function h can be taken continuous and hence the corresponding Markov kernels K(z, ·) = δh(z) (·) are weakly continuous in z, as required by the preceding lemma. If d > 1, then we can, without loss of generality, suppose that η0 is a distribution function on [0, 1]d with uniform marginal distributions (i.e. a copula function). Then the conditioning event η0 (X) = z will typically restrict X to a one-dimensional curve in [0, 1]d . Under sufficient smoothness of η0 , this curve will vary continuously with z, and under smoothness conditions on the law of X, the conditional distribution of h0 (X) given η0 (X) = z for a continuous function h0 will vary continuously as well. Ghoudi and Remillard [6] give sufficient conditions for this continuity in a number of examples. The preceding lemma can be extended to the case of multidimensional variables. For simplicity we only consider the two-dimensional case. Lemma 3.3. Suppose that X, Y, Yt (with t > 0) are random variables in R2 defined on a common probability space such that

246

A. W. van der Vaart and J. A. Wellner

(i) X possesses a Lebesgue density f that has continuous conditional densities; (ii) Yt − Y ∞ → 0 and Y ∞ < ∞; (iii) the conditional distribution of Y given X = s can be represented by a Markov kernel K(s, ·) such that the map s → K(s, ·) is continuous at x for the weak topology. Then for every continuous function g : R → R and every converging sequences xt → x, at → a and bt → b > 0, as t → 0, 1 Eg(Yt )(1X+tat Yt ≤xt +tbt − 1X+tat Yt ≤xt ) t
x2
  → b1 g(y) K (x1 , s2 ), dy f (x1 , s2 ) ds2 −∞
x1
  + b2 g(y) K (s1 , x2 ), dy f (s1 , x2 ) ds1 . −∞

Proof. The event {X + tat Yt ≤ xt + tbt } ∩ {X + tat Yt ≤ xt }c can be decomposed in the three events I = {x1t < X1 + tat Y1t ≤ x1t + tb1t , X2 + tat Y2t ≤ x2t }, II = {x1t < X1 + tat Y1t ≤ x1t + tb1t , x2t < X2 + tat Y2t ≤ x2t + tb2t }, III = {X1 + tat Y1t ≤ x1t , x2t < X2 + tat Y2t ≤ x2t + tb2t }. In view of the boundedness of the Yt the event II is contained in an event of the form {X ∈ B1 } for Bt rectangles of area O(t2 ). Therefore, this event does not contribute to the limit. The contribution of the event I with Yt = Y can be written


  g(y)1u1
By arguments as given previously this can be shown to converge to


  g(y)1u1
The integral with respect to u1 can be computed explicitly and the expression reduces to the first term on the right of the lemma. The contribution of the event III gives the second term. We can replace Yt by Y by similar arguments as in the one-dimensional case. (In fact, bound x2t by ∞ and use exactly the same arguments.) 4. Pseudo observations √ In this section we consider the asymptotic behaviour of the process { n(Pn θ ◦ ηn − P θ ◦ η0 ) : θ ∈ Θ} for a given class Θ of functions θ : Rd → R. The set-up is the same as in Section 3. As explained in the introduction we can decompose this process as √ Gn (θ ◦ ηn − θ ◦ η0 ) + Gn θ ◦ η0 + nP (θ ◦ ηn − θ ◦ η0 ). Under the conditions of Theorem 3.1 or Theorem 3.2, Theorem 2.3, or their extensions, the first term will converge to zero in probability in ∞ (Θ). The second term

Random empirical processes

247

will converge in distribution to a Gaussian process in this space if and only if the class of functions Θ is Donsker for the law P ◦ η0−1 . If the third term also converges in distribution, then the sum of the three processes is asymptotically tight, and it will usually be straightforward to deduce its limit distribution from consideration of the marginal distributions. The behaviour √ of the third term will follow by the (functional) delta-method if the sequence n(ηn − η0 ) converges in distribution in the Banach space H and the ∞ map η → (P θ ◦ η : θ ∈ Θ) from √ H to  (Θ) is suitably differentiable. If the limit distribution of the sequence n(ηn − η0 ) concentrates on the space H0 ⊂ H, then it suffices that the map η → (P θ ◦η : θ ∈ Θ) be “Hadamard differentiable tangentially to H0 ”, i.e. for every converging sequence ht → h0 ∈ H0 ⊂ H  1

P (θ ◦ (η0 + tht ) − θ ◦ η0 ) → L(h0 )(θ), t uniformly in θ ∈ Θ, for a continuous linear map L : lin H0 → ∞ (Θ). Under the additional condition that L is defined on all H, this implies √ √ nP (θ ◦ ηn − θ ◦ η0 ) = L( n(ηn − η0 ))(θ) + oP (1).

Cf. van der Vaart and Wellner [12], page 374. As in the preceding section we consider the cases that the functions θ are smooth or of bounded variation separately. In the former case the differentiability is relative to a weak norm on H (and is easy to prove), but for discontinuous functions θ, such as the indicator functions 1(−∞,θ] , the differentiability requires a strong norm on H and some conditions on the underlying distribution. 4.1. Smooth functions θ If the functions θ are differentiable with bounded derivatives, then the Hadamard differentiability is true for H equipped with the L1 (P )-norm on H. Lemma 4.1. Let the functions θ : Rd → R in Θ be continuously differentiable with ˙ derivative θ˙ such that θ(x) ≤ 1 for every x ∈ Rd . Then the map η → (P θ ◦ η : ∞ θ ∈ Θ)  from L1 (X , A,  P ) to  (Θ) is Hadamard differentiable at η0 with derivative h → P θ˙ ◦ h : θ ∈ Θ . Proof. Given a sequence ht with P |ht − h0 | → 0 we can write, by Fubini’s theorem, 1      ˙  P θ(η + tht ) − θ(η) − P θ(η)h  t 
1     ˙ + stht )ht − θ(η)h ˙ = P θ(η ds 0
1   ˙ + stht ) − θ(η) ˙ ht  ds + P θ(η)h ˙ ≤ P θ(η t − h0 . 0

The second term on the right is bounded above by P |ht − h0 | and converges to zero by assumption. The first term on the right converges to zero by the dominated convergence theorem. 4.2. Functions θ of bounded variation In the second result we let Θ be a set of functions of bounded variation on a bounded interval in R, and consider the Hadamard differentiability of the map

248

A. W. van der Vaart and J. A. Wellner

η → (P θ ◦ η : θ ∈ Θ) as a map from ∞ (X ) to ∞ (Θ). For simplicity of notation, let X be a random variable with law P . Lemma 4.2. Let the functions θ ∈ Θ be distribution functions of subprobability measures supported on a compact interval I ⊂ R. Suppose that the variable η0 (X) possesses a Lebesgue density f that is continuous on a neighbourhood of I. Then the map η → (P θ ◦ η : θ ∈ Θ) from ∞ (X ) to ∞ (Θ) is Hadamard differentiable at η0 tangentially to the set of all h0 such that there exists a version of the conditional continuous in s ∈ I. The distribution of h0 (X) givenη0 (X)  = s that is weakly  E h0 (X)|η0 (X) = s f (s) dθ(s) : θ ∈ Θ . derivative is given by h0 → Proof. Let h0 be as given and suppose ht → h0 in ∞ (X ). For given s ∈ R and u > 0 let χs,u be the continuous function that takes the value 0 on (−∞, s − u], takes the value 1 on [s, ∞) and is linear on the interval [s − u, s]. Then 1[s,∞) ≤ χs,u , and hence     P (1s≤η0 +tht − 1s≤η0 ) ≤ P χs,u (η0 + tht ) − χs,u (η0 ) + P χs,u (η0 ) − 1[s,∞) (η0 ) .

Because η0 (X) possesses a Lebesgue density that is bounded on a neighbourhood of I and 1[s,∞) − χs,u vanishes off the set (s − u, s), the second term on the right is bounded in absolute value by a multiple of u, uniformly in s ranging through I, for small u. By choosing u = δt this term divided by t can be made arbitrarily small by choice of δ. Because χs,u is absolutely continuous with derivative 1/u on (s − u, s) and 0 elsewhere, the first term on the right divided by 1/t can be written in the form
1 1 P (ht 1s−u<η0 +vtht ≤s ) dv. u 0   For u = δt this converges to E h0 (X)|η0 (X) = s f (s), by Lemma 3.2, uniformly in s ranging over I. It follows that, uniformly in s ranging over I, 1    lim sup P (1s≤η0 +tht − 1s≤η0 ) − E h0 (X)|η0 (X) = s f (s) ≤ 0. t t↓0 A similar argument using the functions χs+u,u instead of χs,u gives a corresponding lower bound, whence the expression in brackets converges to zero, uniformly in s ranging through compacta. This concludes the proof of the lemma for Θ equal to the set of functions 1[s,∞) with s in a compact interval. For a general collection Θ of functions of bounded variation we can write
  P θ(η0 + tht ) − θ(η0 ) = P (1s≤η0 +tht − 1s≤η0 ) dθ(s). Next we use the assumption that the functions θ ∈ Θ are supported on the compact interval I with total variation bounded by 1.

The applicability of the second lemma depends on whether the set H0 of functions such that the conditional distribution of h0 (X) given η0 (X) = s is weakly continuous in s is large enough to support the limit distribution of the sequence √ n(ηn − η0 ). As noted in the preceding section, under some smoothness conditions on η0 and on the distribution of X, the set H0 typically contains all continuous functions. Then it suffices that the sequence possesses a continuous weak limiting process.

Random empirical processes

249

5. Examples: completion In this section we return to two of the three examples discussed at the end of Section 3, Example 2 and Example 3. We give the theorems (and proofs) resulting from our approach. The general theme here is that the traditional results given in Corollaries 5.1 and 5.3 for indicator functions involve non-trivial restrictions on the underlying distribution η0 of the data, while the results for indexing by Lipschitz functions given in Corollaries 5.2 and 5.4 involve almost no restrictions on η0 (but significantly smoother indexing functions θ). 5.1. Two corollaries for Kendall processes For the Kendall process, Example 2, it suffices to consider the case in which η0 is concentrated on [0, 1]d and has uniform marginal distributions (i.e. is a copula function), as noted by Ghoudi and Remillard [6]. We first give a corollary for indexing by indicator functions, and then a corollary for indexing by Lipschitz functions. Corollary 5.1. Suppose that for a given interval [a, b] ⊂ (0, 1): (i) The variable η0 (X) possesses a density k with respect to Lebesgue measure that is continuous on a neighbourhood of [a, b]. (ii) The conditional distribution of X given η0 (X) = s, has a regular version representable as a Markov kernel K(s, ·) such that s → K(s, ·) is continuous on [a, b] for the weak topology. √ Then the sequence of processes n(Kn − K) as in (11) tends in ∞ [a, b] in distribution to the process Gη0 fθ : θ ∈ [a, b] for Gη0 an η0 -Brownian bridge process and fθ : [0, 1]d → R defined as fθ (x) = 1η0 (x)≤θ − k(θ) E[1x≤X |η0 (X) = θ]. Corollary 5.2. (Kendall processes, Example 2, indexed by Lipschitz functions). Suppose that Θ is a suitably measurable collection of continuously differentiable ˙ functions θ : [0, 1] → [−1, 1] with derivatives θ˙ satisfying θ(x) ≤ 1 for every n  −1/2 θ(η (X )) − P θ(η0 ) tends x ∈ [0, 1]. Then the sequence of processes n n i i=1   in distribution in ∞ (Θ) to the process Gη0 fθ : θ ∈ Θ for Gη0 an η0 -Brownian bridge process in ∞ (Θ) and fθ : [0, 1]d → R defined as   ˙ x≤X ). fθ (x) = θ η0 (x) − P θ(1

Proof of Corollary 5.1. We apply the decomposition (2) with fθ,η (x) = 1{η(x) ≤ θ}, for distribution functions η on [0, 1]d , θ ∈ [0, 1] and x ∈ [0, 1]d . As discussed following the proof of Lemma 3.2, hypotheses (i) and (ii) imply that the condition (19) for Theorem 3.2 (with d = 1) holds by way of Lemma 3.2, and hence the first term on the right side of (2) converges in probability to 0 uniformly in θ ∈ [a, b]. The second term is simply the usual empirical process for the i.i.d. one-dimensional random variables η0 (X1 ), . . . , η0 (Xn ), and hence it converges weakly as claimed by standard theory. To handle the third term, note that (i) and (ii) imply that the hypotheses of Lemma 4.2 hold, and hence that the map η → {P fθ,η : θ ∈ Θ} from ∞ (X )

250

A. W. van der Vaart and J. A. Wellner

to ∞ ([a, b]) is Hadamard differentiable tangentially to C([0, 1]d ) with derivative L : C([0, 1]d ) → ∞ ([a, b]) given by   L(h0 )(θ) = −E h0 (X)||η0 (X) = θ k(θ).

Weak convergence of the third term then follows from van der Vaart and Wellner [12], Theorem 3.9.5, page 375. The joint limit law of the second and third term can be determined from the marginals, and the limit of the sum of the two terms can be represented in the form as given. An insightful way to derive this is from asymptotic linearity of the two terms as follows. The second term is already linear √ with influence  functions x → 1η0 (x)≤θ . The third term can be approximated by L(( n(ηn − η) , where ηn − 

n

n √ η = n−1 i=1 (1Xi ≤x − η(x)), so that L(( n(ηn − η) = n−1/2 i=1 L(1[Xi ,1] − η). The terms in the latter sum should be understood as L acting on the functions x → 1[Xi ,1] (x) − η(x) for fixed Xi . We thus obtain that n   √ √ L(( n(ηn − h) = n−1/2 L(1[Xi ,1] ) − nL(η) = Gη0 L(1[Xi ,1] ). i=1

The representation of the limit process as given in the corollary follows. For many distribution functions η0 the corresponding density k of K is unbounded at 0 and hence not continuous on [0, 1]. See Barbe, et al. [2], pages 202208, for a number of explicit examples. In particular this is true even when η0 is the uniform distribution on [0, 1]d . For such distributions the preceding corollary does not yield convergence of Kendall’s process in the space ∞ ([0, 1]). However, this convergence may be valid even when k is unbounded. Barbe et al. [2] show that under the growth condition k(t) = o(t−1/2 (log(1/t))−1/2− ),

t ↓ 0,

 > 0.

convergence in the full domain still holds. They √ achieve this using results of Alexander [1] to show that the empirical process n(ηn − η0 )1{η0 ≥ an } converges in the weighted metric  · /q(η0 )∞ if q(t) = t1/2 (log(1/t))p for some√1/2 < p < r/2 and an = n−1 (log n)r . This strengthening of the convergence of n(ηn − η0 ) then compensates for the growth of k at 0. Proof of Corollary 5.2. This follows by combining Theorems 3.1 and Lemma 4.1 with the fact that F = {θ ◦ η0 : θ ∈ Θ} is Donsker. 5.2. Two corollaries for copula processes For the copula processes (12) in Example 3 it again suffices to consider the case in which η0 = C, so that all all the marginal distributions η0,j , j = 1, . . . , d, are Uniform(0, 1). The first of the following two corollaries was obtained in Stute [9] and Ghoudi and Remillard [7]. Corollary 5.3. Suppose that: (i) η0 = C is continuous. (ii) The copula function η0 = C is continously differentiable on [0, 1]d with gradient ∇C(u).

Random empirical processes

251

√ Then the sequence of copula processes n(Cn − C) given in (12) converges in distribution in ∞ ([0, 1]d ) to the process (Gη0 fu : u ∈ [0, 1]d ) for Gη0 an η0 -Brownian bridge process, and fu : [0, 1]d → R defined as fu (x) = 1x≤u − ∇C(u) (1x1 ≤u1 , . . . , 1xd ≤ud ). Corollary 5.4. (Copula processes, Example 3, indexed by Lipschitz functions). Suppose that Θ is a suitably measurable collection of continuously differentiable d ˙ functions θ : [0, 1]d → R such that with derivative θ(x) ≤ 1 for every

n x∈ [0, 1] −1/2 and satisfying J(1, Θ, L2 ) < ∞. Then the sequence of processes n   i=1 θ(ηn (Xi ))− P θ(η0 ) tends in distribution in ∞ (Θ) to the process Gη0 fθ : θ ∈ Θ for Gη0 an η0 -Brownian bridge process in ∞ (Θ) and fθ : [0, 1]d → R defined as   fθ (x) = θ(x) − P θ˙ 1x1 ≤X1 , . . . , 1xd ≤Xd .

Proof of Corollary 5.3. We apply the decomposition (2) with

fθ,η (x) = 1{η1 (x1 ) ≤ θ1 , . . . , ηd (xd ) ≤ θd } for θ = (θ1 , . . . , θd ) ∈ [0, 1]d , x = (x1 , . . . , xd ) ∈ [0, 1]d , and ηj the jth onedimensional marginal distribution function on [0, 1] of the distribution function η (so ηj (uj ) = η(1, . . . , 1, uj , 1, . . . , 1)). To show that the first term in (2) converges to zero uniformly in θ ∈ [0, 1]d , we can apply Theorem 2.1. The class of functions fθ,η (x) = 1{x1 ≤ η1−1 (θ1 ), . . . , xd ≤ ηd−1 (θd )} is a class of indicators of a Vapnik-Chervonenkis-class of sets. Thus Theorem 2.1 applies if we show that (3) holds. But this is easily verified by the assumed continuity −1 for of η0 = C and the uniform consistency of the empirical quantile functions ηn,j j = 1, . . . , d. Thus (1) holds. The second term in (2) is simply the classical empirical process of the random vectors X1 , . . . , Xn in [0, 1]d , and converges weakly by classical theory. Finally, the third term in (2) converges weakly to ∇C(u) · Gη0 (v(X, u)), for v(x, u) = (1x1 ≤u1 , . . . , 1xd ≤ud ), by the delta-method for the map η → P fu,η . This map can be decomposed as      η → η1−1 (u1 ), . . . , ηd−1 (ud ) → C ◦ η1−1 (u1 ), . . . , ηd−1 (ud ) ,

u ∈ [0, 1]d

and can be shown to Hadamard-differentiable from the domain of distribution functions in ∞ (X ) = ∞ ([0, 1]d ) to ∞ (Θ) = ∞ ([0, 1]d ) by the chain rule, using the continuity of ∇C and the fact that the quantile transformation is Hadamard differentiable. It is possible to extend Corollary 5.3 to the case in which ∇C is continuous on (0, 1)d but satisfies certain growth restrictions at 0 and/or 1. Then weighted metrics are involved in the proof. Proof of Corollary 5.4. This follows by combining Theorems 3.1 and Lemma 4.1 with the fact that F = {θ : θ ∈ Θ} is Donsker and the delta-method, e.g. van der Vaart and Wellner [12], Theorem 3.9.5, page 375.

252

A. W. van der Vaart and J. A. Wellner

References [1] Alexander, K. (1987). The central limit theorem for weighted empirical processes indexed by sets. J. Mult. Anal. 22 313–339. [2] Barbe, P., Genest, C., Ghoudi, K. and Remillard, B. (1996). On Kendall’s process. J. Mult. Anal. 58 197–229. [3] Dudley, R. M. (1999). Uniform Central Limit Theorems. Cambridge Univ. Press, Cambridge. ´ ´ e de St. [4] Dudley, R. M. (1984). A Course on Empirical Processes. Ecole d’Et´ Flour 1982. Springer, New York. ¨nssler, P. and Stute, W. (1987). Seminar on Empirical Processes. DMV [5] Ga Seminar. Birkh¨ auser, Basel. [6] Ghoudi, K. and Remillard, B. (1998). Empirical processes based on pseudo-observations. In Asymptotic Methods in Probability and Statistics (Ottawa, ON, 1997) 171–197. North-Holland, Amsterdam. [7] Ghoudi, K. and Remillard, B. (2004). Empirical processes based on pseudo-observations. II. The multivariate case. In Asymptotic Methods in Stochastics 381–406. Fields Inst. Commun. 44. Amer. Math. Soc., Providence, RI. [8] Pollard, D. (1984). Convergence of Stochastic Processes. Springer, New York. [9] Stute, W. (1984). The oscillation behavior of empirical processes: The multivariate case. Ann. Statist. 12 361–379. [10] van der Vaart, A. W. (1998). Asymptotic Statistics. Cambridge Univ. Press, Cambridge. ´ ´ e de St. [11] van der Vaart, A. W. (2002). Semiparametric statistics. Ecole d’Et´ Flour 1999 331–457. Springer, New York. [12] van der Vaart, A. W. and Wellner, J. A. (1996). Empirical Processes and Weak Convergence. Springer, New York.