Selected Works of Murray Rosenblatt

Selected Works in Probability and Statistics For futher volumes: http://www.springer.com/series/8556 Richard A. Davi...

Author: Richard A. Davis | Keh-Shin Lii | Dimitris N. Politis

43 downloads 2164 Views 81MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

Selected Works in Probability and Statistics

For futher volumes: http://www.springer.com/series/8556

Richard A. Davis • Keh-Shin Lii • Dimitris N. Politis Editors

Selected Works of Murray Rosenblatt

Editors Richard A. Davis Department of Statistics Columbia University Amsterdam Ave. 1255 New York, NY 10027 USA [email protected]

Keh-Shin Lii Department of Statistics University of California Riverside, CA 92521, USA [email protected]

Dimitris N. Politis Deptartment of Mathematics University of California, San Diego La Jolla, CA 92093-0112 USA [email protected]

ISBN 978-1-4419-8338-1 e-ISBN 978-1-4419-8339-8 DOI 10.1007/978-1-4419-8339-8 Springer New York Dordrecht Heidelberg London Library of Congress Control Number: 2011925370 © Springer Science+Business Media, LLC 2011 All rights reserved. This work may not be translated or copied in whole or in part without the written permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)

Preface to the Series

Springer’s Selected Works in Probability and Statistics series offers scientists and scholars the opportunity of assembling and commenting upon major classical works in statistics, and honors the work of distinguished scholars in probability and statistics. Each volume contains the original papers, original commentary by experts on the subjects papers, and relevant biographies and bibliographies. Springer is committed to maintaining the volumes in the series with free access on SpringerLink, as well as to the distribution of print volumes. The full text of the volumes is available on SpringerLink with the exception of a small number of articles for which links to their original publisher is included instead. These publishers have graciously agreed to make the articles freely available on their websites. The goal is maximum dissemination of this material. The subjects of the volumes have been selected by an editorial board consisting of Anirban DasGupta, Peter Hall, Jim Pitman, Michael Sörensen, and Jon Wellner.

v

Murray Rosenblatt

Preface

Murray Rosenblatt was born in New York City on September 7, 1926. After completing an undergraduate degree in mathematics at the City College of New York, he entered graduate school in mathematics at Cornell in 1946. While at Cornell, he studied both physics and probability theory and benefited greatly from his contact and exposure to some of the great minds of the day including Feynman, Feller, and Kac. It was under the direction of Marc Kac that Murray wrote his PhD dissertation in 1949 on extensions of the famed Feynman-Kac integral. Murray met his wife Ady in the late 1940s; they were married in Ithaca in 1949, and subsequently had two children, Karin and Daniel. After receiving his PhD, Murray joined the “Committee on Statistics” at the University of Chicago. He held academic positions at the University of Indiana and Brown University before settling in the University of California at San Diego (UCSD) in 1964. He remains at UCSD to date— as Distinguished Professor Emeritus since 1994—having survived the loss of his wife Ady in 2009 after a courageous battle with cancer. The Statistical Science interview [1] provides further insights into Murray’s professional life and career; more details are also found in the short section on key events included in this volume. During the second half of the 20th century, Murray was one of the most celebrated and leading figures in probability and statistics with particular emphasis on time series, Markov processes, and nonparametric function estimation. In addition to being a fellow of IMS and AAAS, he was a Guggenheim fellow twice (1965–66, 1971–72), and was elected to the National Academy of Sciences in 1984. Among his many contributions, Murray conducted seminal work on density estimation, central limit theorems under strong mixing, spectral domain methods, long memory processes and Markov processes. During his long—and continuing—career, Murray has published over 130 papers, and 5 books. His earliest book, the time series monograph [2] written jointly with U. Grenander, was published in 1957 and remains a classic today. This book was the first to lay out a comprehensive and modern set of techniques for modeling time series backed up by rigorous arguments. It had wide appeal to researchers and practitioners alike. Many of the results in this book are as relevant today as when they first appeared more than 50 years ago. In tribute to Murray’s seminal contributions, the Rosenblatt name has been attached to two notable terms, the Rosenblatt Transformation (based on his 1952 paper [3]) and the Rosenblatt Process—a term coined by Murad Taqqu. At the close of 2009, these terms have an astonishing 506K and 1870K search listings, respectively, in Google.

ix

x

Preface

We are delighted that several leading experts agreed to provide commentary and reflections on various directions of Murray’s research portfolio. Rick Bradley gives a nice overview of some of Murray’s work related to the central limit theory under strong mixing conditions. Peter Bickel describes Murray’s insight into their papers on global measures of density estimates. David Brillinger provides a behind-the-scenes look at his papers with Murray on higher-order spectra. Murad Taqqu describes how Murray’s paper on independence and dependence [4] inspired him to look at a class of limit processes with long memory, the aforementioned Rosenblatt Processes. T.-C. Sun gives an interesting account of Murray’s work on Markov processes on semi-groups and the connections to results by Lévy, and Kawada and Itô. Finally, Keh-Shin Lii, a long time collaborator of Murray’s, describes some of their joint work and problems in deconvolution and non-Gaussian time series modeling. Murray was an accomplished advisor with over 20 PhD students to his credit. He had an interesting style of advising that may have reflected his experience with his own mentor Marc Kac. In his interview [1], Murray says Kac “helped occasionally with suggestions, but sort of left you alone without saying you’ve got to do this or that so forth and so on. He let you to go your own way.” Murray breaks down his own style of advising by “suggesting an area” and “if a student is bright enough to make his own way, why do you have to impose on him?” Of course, as two of the undersigned can attest, it was not always clear that we could make it on our own! In meetings during our graduate student days, Murray was fond of frequently uttering the “what if ” refrain. He was not only trying to probe our level of understanding, but trying to push us to think more broadly beyond the conventional boundaries. Murray has extraordinary insight which, unfortunately, is something that is not readily passed on to one’s students. It was often a revelation when after studying a topic, one would finally come to understand the insight Murray had expressed years earlier. This volume is a celebration of Murray Rosenblatt’s stellar research career that spans over six decades. While space limitations prevented us from publishing all of Murray’s numerous research papers, we have attempted to gather in this volume some of his most interesting and influential papers. The task was difficult—and in the end rather subjective—but we received help and guidance from our six expert discussants and from Murray himself. We would like to thank Anirban DasGupta, past editor of the IMS Collections and Lectures Notes—Monograph Series, who first proposed putting a volume together in honor of Murray. Even though IMS had to back out of its commitment to this series as a result of unanticipated financial concerns, Anirban, together with Jim Pitman, Peter Hall and Jon Wellner, have been stalwart supporters of these “Selected Works” projects. We are indebted to John Kimmel and Springer for stepping in and salvaging the “Selected Works Series” that honors some of the leading figures in our field. New York Riverside La Jolla

Richard A. Davis Keh-Shin Lii Dimitris N. Politis

Preface

Bibliography [1] D. R. Brillinger and R. A. Davis. A conversation with Murray Rosenblatt Statistical Science, 24:116–140. [2] U. Grenander and M. Rosenblatt. Statistical analysis of stationary time series. John Wiley & Sons, New York, 1957. [3] M. Rosenblatt. Remarks on a multivariate transformation. Ann. Math. Statistics, 23:470–472, 1952. [4] M. Rosenblatt. Independence and dependence. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. II, pages 431–443. Univ. California Press, Berkeley, Calif., 1961.

xi

Acknowledgements

This series of selected works is possible only because of the efforts and cooperation of many people, societies, and publishers. The series editors originated the series and directed its development. The volume editors spent a great deal of time compiling the previously published material and the contributors provided comments on the significance of the papers. The societies and publishers who own the copyright to the original material made the volumes possible and affordable by their generous cooperation: American Institute of Physics American Mathematical Society American Statistical Association Applied Probability Trust Bernoulli Society Cambridge University Press Canadian Mathematical Society Elsevier Foundation of the Scandinavian Journal of Statistics Indian Statistical Institute Institute of Mathematical Statistics International Chinese Statistical Association International Statistical Institute John Wiley and Sons l’Institut Fourier London Mathematical Society New Zealand Statistical Association Oxford University Press Polish Academy of Sciences Princeton University and the Institute for Advanced Studies Springer Statistical Society of Australia University of California Press University of Illinois, Department of Mathematics University of Michigan, Department of Mathematics University of North Carolina Press

xiii

Contents

1. Preface to the Series

v

2. Preface

ix

3. Acknowledgements

xiii

4. Table of Contents

xv

5. List of Contributors

xix

6. Professional Timeline

xxi

7. Complete Author Bibliography

xxiii

8. Commentary: Discussion of Rosenblatt’s work on Global Measures of Deviations for Density Estimates, by Peter Bickel

1

9. Commentary: Murray Rosenblatt’s contributions to strong mixing, by Richard C. Bradley

3

10. Commentary: Murray Rosenblatt and cumulant/higher-order/poly spectra, by David R. Brillinger

11

11. Commentary: Rosenblatt’s Contribution to Deconvolution, by Keh-Shin Lii

14

12. Commentary: Rosenblatt’s Contributions to Random Walks on Compact Semigroups, by T. C. Sun

23

13. Commentary: The Rosenblatt Process, by Murad S. Taqqu

29

14. U. Grenander and M. Rosenblatt. On spectral analysis of stationary time series. Proc. Nat. Acad. Sci. U. S. A., 38:519–521, 1952. Reprinted with permission of National Academy of Sciences.

46

15. M. Rosenblatt. Remarks on a multivariate transformation. Ann. Math. Statistics, 23:470–472, 1952. Reprinted with permission of Institute of Mathematical Statistics.

49

16. U. Grenander and M. Rosenblatt. Statistical spectral analysis of time series arising from stationary stochastic processes. Ann. Math. Statistics, 24:537–558, 1953. Reprinted with permission of Institute of Mathematical Statistics.

52

xv

xvi

Contents

17. J. L. Hodges, Jr. and M. Rosenblatt. Recurrence-time moments in random walks. Pacific J. Math., 3:127–136, 1953. Reprinted with permission of Mathematical Sciences Publishers. First printed in the Pac. J. of Math 3(1) 127-136 (1953).

74

18. J. R. Blum and M. Rosenblatt. A class of stationary processes and a central limit theorem. Proc. Nat. Acad. Sci. U.S.A., 42:412–413, 1956. Reprinted with permission of National Academy of Sciences.

84

19. M. Rosenblatt. A central limit theorem and a strong mixing condition. Proc. Nat. Acad. Sci. U. S. A., 42:43–47, 1956. Reprinted with permission of National Academy of Sciences.

90

20. M. Rosenblatt. Remarks on some nonparametric estimates of a density function. Ann. Math. Statist., 27:832–837, 1956. Reprinted with permission of Institute of Mathematical Statistics.

95

21. M. Rosenblatt. Some regression problems in time series analysis. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1954–1955, vol. I, pages 165–186, Berkeley and Los Angeles, 1956. University of California Press. Reprinted with permission of the University of California Press.

101

22. M. Rosenblatt. Some purely deterministic processes. J. Math. Mech., 6:801–810, 1957. Reprinted with permission of Indiana University Press.

124

23. M. Rosenblatt. Functions of a Markov process that are Markovian. J. Math. Mech., 8:585–596, 1959. Reprinted with permission of Indiana University Press.

134

24. M. Rosenblatt. Stationary processes as shifts of functions of independent random variables. J. Math. Mech., 8:665–681, 1959. Reprinted with permission of Indiana University Press.

146

25. M. Rosenblatt. Asymptotic distribution of eigenvalues of block Toeplitz matrices. Bull. Amer. Math. Soc., 66:320–321, 1960. Reproduced with permission of the American Mathematical Society.

163

26. M. Rosenblatt. Limits of convolution sequences of measures on a compact topological semigroup. J. Math. Mech., 9:293–305, 1960. Reprinted with permission of Indiana University Press.

165

27. M. Rosenblatt. Independence and dependence. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. II, pages 431–443. Univ. California Press, Berkeley, Calif., 1961. Reprinted with permission of the University of California Press.

179

28. M. Rosenblatt. Asymptotic behavior of eigenvalues for a class of integral equations with translation kernels. In Proc. Sympos. Time Series Analysis (Brown Univ., 1962), pages 316–326. Wiley, New York, 1963. Reprinted with permission of Wiley Press.

193

Contents

xvii

29. M. Rosenblatt. Asymptotic behavior of eigenvalues of Toeplitz forms. J. Math. Mech., 11:941–949, 1962. Reprinted with permission of Indiana University Press.

205

30. M. Rosenblatt and J. W. Van Ness. Estimation of the bispectrum. Ann. Math. Statist., 36:1120–1136, 1965. Reprinted with permission of Institute of Mathematical Statistics.

214

31. M. Rosenblatt. Remarks on the Burgers equation. J. Mathematical Phys., 9:1129–1136, 1968. Reprinted with permission of the American Institute of Physics.

231

32. M. Rosenblatt. Density estimates and Markov sequences. In Nonparametric Techniques in Statistical Inference (Proc. Sympos., Indiana Univ., Bloomington, Ind., 1969), pages 199–213. Cambridge Univ. Press, London, 1970. Reprinted with permission of Cambridge University Press.

239

33. M. Rosenblatt. Curve estimates. Ann. Math. Statist., 42:1815–1842, 1971. Reprinted with permission of Institute of Mathematical Statistics.

255

34. P. J. Bickel and M. Rosenblatt. On some global measures of the deviations of density function estimates. Ann. Statist., 1:1071–1095, 1973. Reprinted with permission of Institute of Mathematical Statistics.

283

35. K. S. Lii and M. Rosenblatt. Asymptotic behavior of a spline estimate of a density function. Comput. Math. Appl., 1(2):223–235, 1975. Reprinted with permission of Elsevier Inc.

308

36. M. Rosenblatt. Fractional integrals of stationary processes and the central limit theorem. J. Appl. Probability, 13(4):723–732, 1976. Reprinted with permission of the Applied Probability Trust.

321

37. M. Rosenblatt. Linear processes and bispectra. J. Appl. Probab., 17(1):265–270, 1980. Reprinted with permission of the Applied Probability Trust.

331

38. M. Rosenblatt. Limit theorems for Fourier transforms of functionals of Gaussian sequences. Z. Wahrsch. Verw. Gebiete, 55(2):123–132, 1981. Reprinted with permission of Springer Science+Business Media.

337

39. K. S. Lii and M. Rosenblatt. Deconvolution and estimation of transfer function phase and coefficients for non-Gaussian linear processes. Ann. Statist., 10(4):1195–1208, 1982. Reprinted with permission of Institute of Mathematical Statistics.

347

40. M. Rosenblatt. Asymptotic normality, strong mixing and spectral density estimates. Ann. Probab., 12(4):1167–1180, 1984. Reprinted with permission of Institute of Mathematical Statistics.

361

xviii

Contents

41. K.-S. Lii and M. Rosenblatt. Deconvolution of non-Gaussian linear processes with vanishing spectral values. Proc. Nat. Acad. Sci. U.S.A., 83(2):199–200, 1986. Reprinted with permission of National Academy of Sciences. 375 42. M. Rosenblatt. Scale renormalization and random solutions of the Burgers equation. J. Appl. Probab., 24(2):328–338, 1987. Reprinted with permission of the Applied Probability Trust.

377

43. J. A. Rice and M. Rosenblatt. On frequency estimation. Biometrika, 75(3):477–484, 1988. Reprinted with permission of Oxford University Press.

388

44. F. J. Breidt, R. A. Davis, K.-S. Lii, and M. Rosenblatt. Maximum likelihood estimation for noncausal autoregressive processes. J. Multivariate Anal., 36(2):175–198, 1991. Reprinted with permission of Elsevier Inc.

396

45. K.-S. Lii and M. Rosenblatt. Spectral analysis for harmonizable processes. Ann. Statist., 30(1):258–297, 2002. Reprinted with permission of Institute of Mathematical Statistics.

420

46. K.-S. Lii and M. Rosenblatt. Estimation for almost periodic processes. Ann. Statist., 34(3):1115–1139, 2006. Reprinted with permission of Institute of Mathematical Statistics.

461

47. K. S. Lii and M. Rosenblatt. Prolate spheroidal spectral estimates. Statist. Probab. Lett., 78(11):1339–1348, 2008. Reprinted with permission of Elsevier Inc.

487

Contributors

Peter Bickel Department of Statistics, University of California, Berkeley, California, 94720, E-mail: [email protected] Richard Bradley Department of Mathematics, Indiana University, Bloomington, Indiana, 47405, E-mail: [email protected] David R. Brillinger Department of Statistics, University of California, Berkeley, California, 94720, E-mail: [email protected] Keh-Shin Lii Department of Statistics, University of California, Riverside, California, 92521, E-mail: [email protected] T.-C. Sun Department of Mathematics, Wayne State University, Detroit, Michigan, 48202, E-mail: [email protected] Murad Taqqu Department of Mathematics, Boston University, Boston, Massachusetts, 02215, E-mail: [email protected]

xix

Murray Rosenblatt: Professional Timeline

1926 Born New York City, September 7, 1926 1946 B.S. City College of New York 1947 M.S. Cornell University 1949 PH.D. Cornell University thesis: “On distributions of certain Wiener functionals”. Supervisor: Mark Kac 1949–1950 ONR postdoctoral grant (at Cornell University) 1950–1951 Instructor, Committee on Statistics, University of Chicago 1951–1955 Assistant professor, Committee on Statistics, Chicago 1953 University of Stockholm 1955 Department of Statistics, Columbia 1956–1959 Associate professor, Indiana University 1959–1964 Professor of Probability and Statistics, Brown University 1959 Brookhaven Lab visit 1960 —Madhav Heble, Indiana University, dissertation: “Linear estimation of regression coefficients, orthogonal matrix polynomials and application to multidimensional weakly stationary processes; interpolation and regression” 1960 —Perry Scheinok, Indiana University, dissertation: “The error on using the asymptotic variance and bias of spectrograph estimates for finite observation time” 1961 —Jack Hachigan, Indiana University, dissertation: “Some further results on functions of Markov processes” 1962 Symposium on Time Series, Brown University 1963 —T.-C. Sun, Brown University, dissertation: “A central limit theorem for nonlinear functions of a normal stationary process” 1964 —Henry Krieger, Brown University, dissertation: “Toeplitz operators on locally compact abelian groups” 1964 —John van Ness, Brown University, dissertation: “Estimates of the bispectrum of stationary random processes” 1964–1994 Professor of Mathematics, University of California, San Diego 1965 —Mahendra Nadkarni, Brown University, dissertation: “Vector valued weakly stationary stochastic processes and factorization of matrix valued functions and strong mixing and uniformly ergodic gaussian processes” 1965 —Henry Richardson III, Brown University, dissertation: “Regression analysis when the least-squares estimate is not asymptotically efficient” 1965 Guggenheim fellowship, University College and Imperial College, London 1967–1968 Chair, Mathematics Department, UCSD 1970 —Seung Chay, USCD, dissertation: “On quasi-Markov random fields” 1970 Wald lecturer 1971 Guggenheim fellowship, University College and Imperial College, London xxi

xxii

Murray Rosenblatt: Professional Timeline

1972 —Michael Sturgeon, UCSD, dissertation: “The representation of invariant conservative Baire measures for certain Markov processes” 1975 —Keh-Shin Lii, UCSD, dissertation: “Density estimation and splines” 1978 —Richard Bradley, UCSD, dissertation: “Measures of dependence on stationary sequences of random variables” 1978 —Yue-Pok Mack, UCSD, dissertation: “k-nearest neighbor estimation” 1979 —Richard Davis, UCSD, dissertation: “Extremes of stationary processes” 1979 —Stuart Strait, UCSD, dissertation: “A quadratic measure of deviation of spectral estimates” 1979 Fellow at Churchill College, Cambridge 1984 —Larry Goldstein, UCSD, dissertation: “Extensions of stochastic approximation procedures” 1984 —Diane Marcus, UCSD, dissertation: “On the approximation of distributions of sums of independent summands by infinitely divisible distributions—n dimensional case” 1984 Member, National Academy of Sciences 1985 —Karen Messer, UCSD, dissertation: “Boundary effects of smoothing splines” 1991 —Bruce Wahlen, UCSD, dissertation: “A nonparametric measure of independence” 1993 —Michael Kramer, UCSD, dissertation: “The fluctuation of the gaussian likelihood for stationary random sequences” 1994 —Yukuang Chiu, UCSD, dissertation: “Topics on prediction and representation of stationary processes” 1998 —Anthony Gamst, UCSD, dissertation: “Stochastic Burgers flows” 1994–present Professor Emeritus, University of California—San Diego 1994–1996, Visiting Professor, Colorado State University (fall semesters) 2003 President, Emeriti Association, UCSD

Murray Rosenblatt: Bibliography

Books Authored . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Books Edited. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Mathematical Articles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interviews and Biographies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xxiii xxiv xxiv xxxv

Books Authored [B1] U. Grenander and M. Rosenblatt. Statistical analysis of stationary time series. John Wiley & Sons, New York, 1957. [B2] M. Rosenblatt. Random processes. University Texts in the Mathematical Sciences. Oxford University Press, New York, 1962. [B3] M. Rosenblatt and T. Czechowski. Procesy stochastyczne. Państwowe Wydaw. Naukowe, 1967. Polish translation of Random Processes. [B4] M. Rosenblatt. Markov Process: Structure and Asymptotic Behavior. Die Grundlehren Der Mathematischen Wissenschaften, Band 184. SpringerVerlag, 1971. [B5] M. Rosenblatt. Random processes. World Publishing Corporation, Beijing, 1974. Reprint of Springer-Verlag’s 1974 ed. of Random Processes. [B6] M. Rosenblatt. Random processes. Graduate Texts in Mathematics, No. 17. Springer-Verlag, New York, second edition, 1974. [B7] U. Grenander and M. Rosenblatt. Statistical analysis of stationary time series. Chelsea Publishing Co., New York, second edition, 1984. [B8] M. Rosenblatt. Stationary sequences and random fields. Birkhäuser Boston Inc., Boston, MA, 1985. [B9] M. Rosenblatt. Stochastic Curve Estimation, volume 3 of NSF-CBMS Regional Conference Series in Probability and Statistics. Institute of Mathematical Statistics, 1991. [B10] M. Rosenblatt. Gaussian and non-Gaussian linear time series and random fields. Springer Series in Statistics. Springer-Verlag, New York, 2000.

xxiii

xxiv

Murray Rosenblatt: Bibliography

Books Edited [E1] [E2] [E3]

[E4] [E5]

[E6]

[E7]

[E8]

M. Rosenblatt, editor. Proceedings of the Symposium on Time Series Analysis, New York, 1963. John Wiley and Sons. M. Rosenblatt and C. V. Atta, editors. Statistical models and turbulence. Lecture Notes in Physics, Vol. 12. Springer-Verlag, Berlin, 1972. M. Rosenblatt, editor. Studies in probability theory, volume 18 of MAA Studies in Mathematics. Mathematical Association of America, Washington, D.C., 1978. T. Gasser and M. Rosenblatt, editors. Smoothing techniques for curve estimation, volume 757 of Lecture Notes in Mathematics, Berlin, 1979. Springer. M. Rosenblatt, editor. Errett Bishop: reflections on him and his research, volume 39 of Contemporary Mathematics, Providence, RI, 1985. American Mathematical Society. D. Brillinger, P. Caines, J. Geweke, E. Parzen, M. Rosenblatt, and M. S. Taqqu, editors. New directions in time series analysis. Part I, volume 45 of The IMA Volumes in Mathematics and its Applications. Springer-Verlag, New York, 1992. D. Brillinger, P. Caines, J. Geweke, E. Parzen, M. Rosenblatt, and M. S. Taqqu, editors. New directions in time series analysis. Part II, volume 46 of The IMA Volumes in Mathematics and its Applications. Springer-Verlag, New York, 1993. P. M. Robinson and M. Rosenblatt, editors. Athens Conference on Applied Probability and Time Series Analysis. Vol. II, volume 115 of Lecture Notes in Statistics. Springer-Verlag, New York, 1996. Time series analysis, In memory of E. J. Hannan, Papers from the conference held in Athens, March 22–26, 1995.

Mathematical Articles [M1] M. Rosenblatt. On a class of Markov processes. Trans. Amer. Math. Soc., 71:120–135, 1951. [M2] U. Grenander and M. Rosenblatt. On spectral analysis of stationary time series. Proc. Nat. Acad. Sci. U. S. A., 38:519–521, 1952. [M3] M. Rosenblatt. The behavior at zero of the characteristic function of a random variable. Proc. Amer. Math. Soc., 3:498–504, 1952. [M4] M. Rosenblatt. Limit theorems associated with variants of the von Mises statistic. Ann. Math. Statistics, 23:617–623, 1952. [M5] M. Rosenblatt. On the oscillation of sums of random variables. Trans. Amer. Math. Soc., 72:165–178, 1952. [M6] M. Rosenblatt. Remarks on a multivariate transformation. Ann. Math. Statistics, 23:470–472, 1952.

Indicates article found in this volume.

Murray Rosenblatt: Bibliography

[M7]

[M8] [M9]

[M10] [M11]

[M12] [M13] [M14] [M15]

[M16] [M17] [M18] [M19]

[M20] [M21]

[M22] [M23]

K. A. Brownlee, J. L. Hodges, and M. Rosenblatt. The up-and-down method with small samples. Journal of the American Statistical Association, 48(262):262–277, 1953. U. Grenander and M. Rosenblatt. Comments on statistical spectral analysis. Skand. Aktuarietidskr., 36:182–202, 1953. U. Grenander and M. Rosenblatt. Statistical spectral analysis of time series arising from stationary stochastic processes. Ann. Math. Statistics, 24: 537–558, 1953. J. L. Hodges, Jr. and M. Rosenblatt. Recurrence-time moments in random walks. Pacific J. Math., 3:127–136, 1953. U. Grenander and M. Rosenblatt. An extension of a theorem of G. Szegö and its application to the study of stochastic processes. Trans. Amer. Math. Soc., 76:112–126, 1954. U. Grenander and M. Rosenblatt. Regression analysis of time series with stationary residuals. Proc. Nat. Acad. Sci. U. S. A., 40:812–816, 1954. M. Rosenblatt. An inventory problem. Econometrica, 22:244–247, 1954. J. R. Blum and M. Rosenblatt. A class of stationary processes and a central limit theorem. Proc. Nat. Acad. Sci. U.S.A., 42:412–413, 1956. U. Grenander and M. Rosenblatt. Some problems in estimating the spectrum of a time series. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1954–1955, vol. I, pages 77–93, Berkeley and Los Angeles, 1956. University of California Press. M. Rosenblatt. A central limit theorem and a strong mixing condition. Proc. Nat. Acad. Sci. U. S. A., 42:43–47, 1956. M. Rosenblatt. On the estimation of regression coefficients of a vector-valued time series with a stationary residual. Ann. Math. Statist., 27:99–121, 1956. M. Rosenblatt. Remarks on some nonparametric estimates of a density function. Ann. Math. Statist., 27:832–837, 1956. M. Rosenblatt. Some regression problems in time series analysis. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1954–1955, vol. I, pages 165–186, Berkeley and Los Angeles, 1956. University of California Press. J. R. Blum and M. Rosenblatt. A class of stationary processes and a central limit theorem. Duke Math. J., 24:73–78, 1957. M. Rosenblatt. The multidimensional prediction problem. In Proceedings of the National Academy of Sciences of the United States of America, volume 43, pages 989–992, November 15 1957. M. Rosenblatt. A random model of the sea surface generated by a hurricane. J. Math. Mech., 6:235–246, 1957. M. Rosenblatt. Some purely deterministic processes. J. Math. Mech., 6: 801–810, 1957.

Indicates article found in this volume.

xxv

xxvi

Murray Rosenblatt: Bibliography

[M24] J. R. Blum, H. Chernoff, M. Rosenblatt, and H. Teicher. Central limit theorems for interchangeable processes. Canad. J. Math., 10:222–229, 1958. [M25] C. Burke and M. Rosenblatt. Consolidation of probability matrices. Bull. Inst. Internat. Statist., 36(3):7–8, 1958. [M26] C. Burke and M. Rosenblatt. Consolidation of probability matrices. Bull. Inst. Internat. Statist., 36(3):7–8, 1958. [M27] C. J. Burke and M. Rosenblatt. A Markovian function of a Markov chain. Ann. Math. Statist., 29:1112–1122, 1958. [M28] M. Rosenblatt. A multi-dimensional prediction problem. Ark. Mat., 3: 407–424, 1958. [M29] J. R. Blum and M. Rosenblatt. On the structure of infinitely divisible distributions. Pacific J. Math., 9:1–7, 1959. [M30] M. Rosenblatt. Functions of a Markov process that are Markovian. J. Math. Mech., 8:585–596, 1959. [M31] M. Rosenblatt. Stationary processes as shifts of functions of independent random variables. J. Math. Mech., 8:665–681, 1959. [M32] M. Rosenblatt. Statistical analysis of stochastic processes with stationary residuals. In Probability and statistics: The Harald Cramér volume (edited by Ulf Grenander), pages 246–275. Almqvist & Wiksell, Stockholm, 1959. [M33] M. Rosenblatt. Asymptotic distribution of eigenvalues of block Toeplitz matrices. Bull. Amer. Math. Soc., 66:320–321, 1960. [M34] M. Rosenblatt. Limits of convolution sequences of measures on a compact topological semigroup. J. Math. Mech., 9:293–305, 1960. [M35] M. Rosenblatt. The multidimensional prediction problem. In Statistical methods in radio wave propagation : proceedings of a symposium held at the University of California, Los Angeles 1958. Symposium Division Pergamon Pr., 1960. [M36] M. Rosenblatt. Stationary Markov chains and independent random variables. J. Math. Mech., 9:945–949, 1960. [M37] M. Rosenblatt. Some comments on narrow band-pass filters. Quart. Appl. Math., 18:387–393, 1960/1961. [M38] J. R. Blum, J. Kiefer, and M. Rosenblatt. Distribution free tests of independence based on the sample distribution function. Ann. Math. Statist., 32: 485–498, 1961. [M39] M. Rosenblatt. Independence and dependence. In Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. II, pages 431–443. Univ. California Press, Berkeley, Calif., 1961. [M40] W. Freiberger, M. Rosenblatt, and J. W. Van Ness. Regression analysis of vector-valued random processes. J. Soc. Indust. Appl. Math., 10:89–102, 1962. [M41] J. Hachigian and M. Rosenblatt. Functions of reversible Markov processes that are Markovian. J. Math. Mech., 11:951–960, 1962.

Indicates article found in this volume.

Murray Rosenblatt: Bibliography

[M42] G. F. Newell and M. Rosenblatt. Zero crossing probabilities for Gaussian stationary processes. Ann. Math. Statist., 33:1306–1313, 1962. [M43] M. Rosenblatt. Asymptotic behavior of eigenvalues of Toeplitz forms. J. Math. Mech., 11:941–949, 1962. [M44] M. Rosenblatt and D. Slepian. nth order Markov chains with every N variables independent. J. Soc. Indust. Appl. Math., 10:537–549, 1962. [M45] M. Heble and M. Rosenblatt. Idempotent measures on a compact topological semigroup. Proc. Amer. Math. Soc., 14:177–184, 1963. [M46] M. Rosenblatt. Asymptotic behavior of eigenvalues for a class of integral equations with translation kernels. In Proc. Sympos. Time Series Analysis (Brown Univ., 1962), pages 316–326. Wiley, New York, 1963. [M47] M. Rosenblatt. The representation of a class of two state stationary processes in terms of independent random variables. J. Math. Mech., 12:721–730, 1963. [M48] M. Rosenblatt. Some results on the asymptotic behavior of eigenvalues for a class of integral equations with translation kernels. J. Math. Mech., 12:619– 628, 1963. [M49] M. Rosenblatt. Almost periodic transition operators acting on the continuous functions on a compact space. J. Math. Mech., 13:837–847, 1964. [M50] M. Rosenblatt. Equicontinuous Markov operators. Teor. Verojatnost. i Primenen., 9:205–222, 1964. [M51] M. Rosenblatt. Some nonlinear problems arising in the study of random processes. J. Res. Nat. Bur. Standards Sect. D, 68D:933–936, 1964. [M52] M. Rosenblatt. Products of independent identically distributed stochastic matrices. J. Math. Anal. Appl., 11:1–10, 1965. [M53] M. Rosenblatt and J. W. Van Ness. Estimation of the bispectrum. Ann. Math. Statist., 36:1120–1136, 1965. [M54] M. Rosenblatt. Remarks on ergodicity of stationary irreducible transient Markov chains. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 6: 293–301, 1966. [M55] M. Rosenblatt. Functions of Markov processes. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 5:232–243, 1966. [M56] M. Rosenblatt. Remarks on higher order spectra. In Multivariate Analysis (Proc. Internat. Sympos., Dayton, Ohio, 1965), pages 383–389. Academic Press, New York, 1966. [M57] D. R. Brillinger and M. Rosenblatt. Asymptotic theory of estimates of kthorder spectra. Proc. Nat. Acad. Sci. U.S.A., 57:206–210, 1967. [M58] D. R. Brillinger and M. Rosenblatt. Asymptotic theory of estimates of k-th order spectra. In B. Harris, editor, Spectral Analysis Time Series (Proc. Advanced Sem., Madison, Wis., 1966), pages 153–188. John Wiley, New York, 1967.

Indicates article found in this volume.

xxvii

xxviii

Murray Rosenblatt: Bibliography

[M59] D. R. Brillinger and M. Rosenblatt. Computation and interpretation of k-th order spectra. In B. Harris, editor, Spectral Analysis Time Series (Proc. Advanced Sem., Madison, Wis., 1966), pages 189–232. John Wiley, New York, 1967. [M60] M. Rosenblatt. A strong mixing condition and a central limit theorem on compact groups. J. Math. Mech., 17:189–198, 1967. [M61] M. Rosenblatt. Transition probability operators. In Proc. Fifth Berkeley Sympos. Math. Statist. and Probability (Berkeley, Calif., 1965/66), Vol. II: Contributions to Probability Theory, Part 2, pages 473–483. Univ. California Press, Berkeley, Calif., 1967. [M62] M. Rosenblatt. Remarks on the Burgers equation. J. Mathematical Phys., 9:1129–1136, 1968. [M63] M. Rosenblatt. Conditional probability density and regression estimators. In Multivariate Analysis, II (Proc. Second Internat. Sympos., Dayton, Ohio, 1968), pages 25–31. Academic Press, New York, 1969. [M64] M. Rosenblatt. Stationary measures for random walks on semigroups. In Semigroups (Proc. Sympos., Wayne State Univ., Detroit, Mich., 1968), pages 209–220. Academic Press, New York, 1969. [M65] M. Rosenblatt. Density estimates and Markov sequences. In Nonparametric Techniques in Statistical Inference (Proc. Sympos., Indiana Univ., Bloomington, Ind., 1969), pages 199–213. Cambridge Univ. Press, London, 1970. [M66] M. Rosenblatt. A prediction problem and central limit theorems for stationary Markov sequences. In Proc. Twelfth Biennial Sem. Canad. Math. Congr. on Time Series and Stochastic Processes; Convexity and Combinatorics (Vancouver, B.C., 1969), pages 99–114. Canad. Math. Congr., Montreal, Que., 1970. [M67] M. Rosenblatt. Curve estimates. Ann. Math. Statist., 42:1815–1842, 1971. [M68] M. Rosenblatt. Central limit theorem for stationary processes. In Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability (Univ. California, Berkeley, Calif., 1970/1971), Vol. II: Probability theory, pages 551–561, Berkeley, Calif., 1972. Univ. California Press. [M69] M. Rosenblatt. Probability limit theorems and some questions in fluid mechanics. In Statistical models and turbulence (Proc. Sympos., Univ. California, La Jolla, Calif., 1971), pages 27–40. Lecture Notes in Phys., Vol. 12. Springer, Berlin, 1972. [M70] M. Rosenblatt. Uniform ergodicity and strong mixing. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 24:79–84, 1972. [M71] M. Rosenblatt. Invariant and subinvariant measures of transition probability functions acting on continuous functions. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 25:209–221, 1972/73.

Indicates article found in this volume.

Murray Rosenblatt: Bibliography

[M72] P. Bickel and M. Rosenblatt. Two-dimensional random fields. In Multivariate analysis, III (Proc. Third Internat. Sympos., Wright State Univ., Dayton, Ohio, 1972), pages 3–15. Academic Press, New York, 1973. [M73] P. J. Bickel and M. Rosenblatt. On some global measures of the deviations of density function estimates. Ann. Statist., 1:1071–1095, 1973. [M74] M. Rosenblatt. Recurrent points and transition functions acting on continuous functions. Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 30:173–183, 1974. [M75] P. J. Bickel and M. Rosenblatt. Corrections to: “On some global measures of the deviations of density function estimates” (Ann. Statist. 1 (1973), 1071–1095). Ann. Statist., 3(6):1370, 1975. [M76] K. S. Lii and M. Rosenblatt. Asymptotic behavior of a spline estimate of a density function. Comput. Math. Appl., 1(2):223–235, 1975. [M77] K. S. Lii and M. Rosenblatt. Asymptotic results on a spline estimate of a probability density. In Statistical inference and related topics (Proc. Summer Res. Inst. Statist. Inference for Stochastic Processes, Indiana Univ., Bloomington, Ind., 1974, Vol 2; dedicated to Z. W. Birnbaum), pages 77–85. Academic Press, New York, 1975. [M78] M. Rosenblatt. The local behavior of the derivative of a cubic spline interpolator. J. Approximation Theory, 15(4):382–387, 1975. [M79] M. Rosenblatt. Multiply schemes and shuffling. Math. Comput., 29:929–934, 1975. [M80] M. Rosenblatt. A quadratic measure of deviation of two-dimensional density estimates and a test of independence. Ann. Statist., 3:1–14, 1975. [M81] M. Rosenblatt. Note correcting a remark in a paper of Karl Bosch (“Notwendige und hinreichende Bedingungen dafür, dass eine Funktion einer homogenen Markoffschen Kette Markoffsch ist”, (Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 31 (1974/75), 199–202)). Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 33(3):219, 1975/76. [M82] K. S. Lii, M. Rosenblatt, and C. V. Atta. Bispectral measurements in turbulence. J. Fluid Mech., 77, Pt. 1:45–62, 1976. [M83] J. Rice and M. Rosenblatt. Estimation of the log survivor function and hazard function. Sankhyá Ser. A, 38(1):60–78, 1976. [M84] M. Rosenblatt. Fractional integrals of stationary processes and the central limit theorem. J. Appl. Probability, 13(4):723–732, 1976. [M85] M. Rosenblatt. On the maximal deviation of k-dimensional density estimates. Ann. Probability, 4(6):1009–1015, 1976. [M86] M. Rosenblatt. Asymptotics and representation of cubic splines. J. Approximation Theory, 17(4):332–343, 1976. [M87] K. N. Helland, K. S. Lii, and M. Rosenblatt. Bispectra of atmospheric and wind tunnel turbulence. In P. R. Krishnaiah, editor, Applications of Statistics. North Holland Publishing Co., 1977.

Indicates article found in this volume.

xxix

xxx

Murray Rosenblatt: Bibliography

[M88]

P. A. W. Lewis, L. H. Liu, D. W. Robinson, and M. Rosenblatt. Empirical sampling study of a goodness of fit statistic for density function estimation. In Multivariate analysis—IV: proceedings of the fourth International Symposium on Multivariate Analysis, pages 139–174, 1977. [M89] M. Rosenblatt. Energy transfer for the Burgers’ equation. Phys. Fluids, 21(10):1694–1697, 1978. [M90] M. Rosenblatt. Dependence and asymptotic independence for random processes. In Studies in probability theory, volume 18 of MAA Stud. Math., pages 24–45. Math. Assoc. America, Washington, D.C., 1978. [M91] K. N. Helland, K. S. Lii, and M. Rosenblatt. Bispectra and energy transfer in grid-generated turbulence. In Developments in statistics, Vol. 2, pages 123–155. Academic Press, New York, 1979. [M92] K. N. Helland and M. Rosenblatt. Spectral variance estimation and the analysis of turbulence. Phys. Fluids, 22:819–823, 1979. [M93] Y. P. Mack and M. Rosenblatt. Multivariate k-nearest neighbor density estimates. J. Multivariate Anal., 9(1):1–15, 1979. [M94] M. Rosenblatt. Global measures of deviation for kernel and nearest neighbor density estimates. In Smoothing techniques for curve estimation (Proc. Workshop, Heidelberg, 1979), volume 757 of Lecture Notes in Math., pages 181–190. Springer, Berlin, 1979. [M95] M. Rosenblatt. Linearity and nonlinearity in time series: prediction. In Proceedings of the 42nd session of the International Statistical Institute, Vol. 1 (Manila, 1979), volume 48, pages 423–434, 1979. [M96] M. Rosenblatt. Some limit theorems for partial sums of quadratic forms in stationary Gaussian variables. Z. Wahrsch. Verw. Gebiete, 49(2):125–132, 1979. [M97] M. Rosenblatt. Some remarks on a mixing condition. Ann. Probab., 7(1):170–172, 1979. [M98] M. Rosenblatt. Correction to: “Some remarks on a mixing condition” [Ann. Probab. 7 (1979), no. 1, 170–172; MR 80k:60071a]. Ann. Probab., 7(6):1097, 1979. [M99] M. Rosenblatt. Linear processes and bispectra. J. Appl. Probab., 17(1): 265–270, 1980. [M100] M. Rosenblatt. Some limit theorems for partial sums of stationary sequences. In Multivariate analysis, V (Proc. Fifth Internat. Sympos., Univ. Pittsburgh, Pittsburgh, Pa., 1978), pages 239–248. North-Holland, Amsterdam, 1980. [M101] M. Rosenblatt. Erratum: “Asymptotics and representation of cubic splines” [J. Approx. Theory 17 (1976), no. 4, 332–343; MR 54 #5682]. J. Approx. Theory, 28(2):184, 1980. [M102] J. Rice and M. Rosenblatt. Integrated mean squared error of a smoothing spline. J. Approx. Theory, 33(4):353–369, 1981.

Indicates article found in this volume.

Murray Rosenblatt: Bibliography

[M103] M. Rosenblatt. Limit theorems for Fourier transforms of functionals of Gaussian sequences. Z. Wahrsch. Verw. Gebiete, 55(2):123–132, 1981. [M104] M. Rosenblatt. Polynomials in Gaussian variables and infinite divisibility? In Contributions to probability, pages 139–142. Academic Press, New York, 1981. [M105] K. S. Lii, K. N. Helland, and M. Rosenblatt. Estimating three-dimensional energy transfer in isotropic turbulence. J. Time Ser. Anal., 3(1):1–28, 1982. [M106] K. S. Lii and M. Rosenblatt. Deconvolution and estimation of transfer function phase and coefficients for non-Gaussian linear processes. Ann. Statist., 10(4):1195–1208, 1982. [M107] J. Rice and M. Rosenblatt. Boundary effects on the behavior of smoothing splines. In Statistics and probability: essays in honor of C. R. Rao, pages 635–643. North-Holland, Amsterdam, 1982. [M108] M. Rosenblatt. Corrections: “A quadratic measure of deviation of two-dimensional density estimates and a test of independence” [Ann Statist. 3 (1975), 1–14; MR 55 #1600]. Ann. Statist., 10(2):646, 1982. [M109] J. Rice and M. Rosenblatt. Smoothing splines: regression, derivatives and deconvolution. Ann. Statist., 11(1):141–156, 1983. [M110] M. Rosenblatt. Cumulants and cumulant spectra. In Time series in the frequency domain, volume 3 of Handbook of Statist., pages 369–382. NorthHolland, Amsterdam, 1983. [M111] M. Rosenblatt. Linear random fields. In Studies in econometrics, time series, and multivariate statistics, pages 299–309. Academic Press, New York, 1983. [M112] K. S. Lii and M. Rosenblatt. Remarks on non-Gaussian linear processes with additive Gaussian noise. In Robust and nonlinear time series analysis (Heidelberg, 1983), volume 26 of Lecture Notes in Statist., pages 185–197. Springer, New York, 1984. [M113] K. S. Lii and M. Rosenblatt. Non-Gaussian linear processes, phase and deconvolution. In Statistical signal processing (Annapolis, Md., 1982), volume 53 of Statist. Textbooks Monogr., pages 51–58. Dekker, New York, 1984. [M114] M. Rosenblatt. Asymptotic normality, strong mixing and spectral density estimates. Ann. Probab., 12(4):1167–1180, 1984. [M115] M. Rosenblatt. Short-range and long-range dependence. In H. A. David and H. T. David, editors, Statistics, an appraisal: proceedings of a conference marking the 50th anniversary of the Statistical Laboratory, Iowa State University, Ames, Iowa, June 13-15, 1983, pages 509–520. Iowa State University Press, 1984. [M116] M. Rosenblatt. A two-dimensional smoothing spline and a regression problem. In Limit theorems in probability and statistics, Vol. I, II (Veszprém, 1982), volume 36 of Colloq. Math. Soc. János Bolyai, pages 915–931. North-Holland, Amsterdam, 1984.

Indicates article found in this volume.

xxxi

xxxii

Murray Rosenblatt: Bibliography

[M117] K. S. Lii and M. Rosenblatt. A fourth-order deconvolution technique for non-Gaussian linear processes. In Multivariate analysis VI (Pittsburgh, Pa., 1983), pages 395–410. North-Holland, Amsterdam, 1985. [M118] M. Rosenblatt and F. J. Samaniego. Julius R. Blum, 1922–1982. Ann. Statist., 13(1):1–9, 1985. [M119] K. S. Lii and M. Rosenblatt. Estimation of a transfer function in a non- Gaussian context. In Function estimates (Arcata, Calif., 1985), volume 59 of Contemp. Math., pages 49–51. Amer. Math. Soc., Providence, RI, 1986. [M120] K.-S. Lii and M. Rosenblatt. Deconvolution of non-Gaussian linear processes with vanishing spectral values. Proc. Nat. Acad. Sci. U.S.A., 83(2):199–200, 1986. [M121] M. Rosenblatt. Convolution sequences of measures on the semigroup of stochastic matrices. In Random matrices and their applications (Brunswick, Maine, 1984), volume 50 of Contemp. Math., pages 215–220. Amer. Math. Soc., Providence, RI, 1986. [M122] M. Rosenblatt. Parameter estimation for finite-parameter stationary random fields. J. Appl. Probab., (Special Vol. 23A):311–318, 1986. Essays in time series and allied processes. [M123] M. Rosenblatt. Prediction for some non-Gaussian autoregressive schemes. Adv. in Appl. Math., 7(2):182–198, 1986. [M124] M. Rosenblatt. Non-Gaussian sequences and deconvolution. In Proceedings of the 1st World Congress of the Bernoulli Society, Vol. 2 (Tashkent, 1986), pages 349–353, Utrecht, 1987. VNU Sci. Press. [M125] M. Rosenblatt. Scale renormalization and random solutions of the Burgers equation. J. Appl. Probab., 24(2):328–338, 1987. [M126] M. Rosenblatt. Remarks on limit theorems for nonlinear functionals of Gaussian sequences. Probab. Theory Related Fields, 75(1):1–10, 1987. [M127] M. Rosenblatt. Some models exhibiting non-Gaussian intermittency. IEEE Trans. Inform. Theory, 33(2):258–262, 1987. [M128] K.-S. Lii and M. Rosenblatt. Estimation and deconvolution when the transfer function has zeros. J. Theoret. Probab., 1(1):93–113, 1988. [M129] K.-S. Lii and M. Rosenblatt. Nonminimum phase non-Gaussian decon volution. J. Multivariate Anal., 27(2):359–374, 1988. [M130] J. A. Rice and M. Rosenblatt. On frequency estimation. Biometrika, 75(3):477–484, 1988. [M131] M. Rosenblatt. Comments on structure and estimation for non-gaussian linear processes. In E. Weginan, S. Schwartz, and J. Thomas, editors, Topics in Non-Gaussian Signal Processing, pages 88–97. Springer-Verlag, 1989. [M132] M. Rosenblatt. Erratum: “Prediction for some non-Gaussian autoregressive schemes” [Adv. in Appl. Math. 7 (1986), no. 2, 182–198; MR0845375 (88a:62260)]. Adv. in Appl. Math., 10(1):130, 1989.

Indicates article found in this volume.

Murray Rosenblatt: Bibliography

[M133] M. Rosenblatt. Book Review: Correlation theory of stationary and random functions vol. I; Basic results, vol. II, Supplementary notes and references. Bull. Amer. Math. Soc. (N.S.), 20(2):207–211, 1989. [M134] M. Rosenblatt. A note on maximum entropy. In Probability, statistics, and mathematics, pages 255–260. Academic Press, Boston, MA, 1989. [M135] F. J. Breidt, R. A. Davis, K.-S. Lii, and M. Rosenblatt. Nonminimum phase non-Gaussian autoregressive processes. Proc. Nat. Acad. Sci. U.S.A., 87(1):179–181, 1990. [M136] K. S. Lii and M. Rosenblatt. Asymptotic normality of cumulant spectral estimates. J. Theoret. Probab., 3(2):367–385, 1990. [M137] K. S. Lii and M. Rosenblatt. Cumulant spectral estimates: bias and covariance. In Limit theorems in probability and statistics (Pécs, 1989), volume 57 of Colloq. Math. Soc. János Bolyai, pages 365–405. North-Holland, Amsterdam, 1990. [M138] M. Rosenblatt. Discussion of Bartlett’s paper “chance or chaos”. J. Roy. Stat. Soc. Ser. A., 153, 1990. [M139] F. J. Breidt, R. A. Davis, K.-S. Lii, and M. Rosenblatt. Maximum likelihood estimation for noncausal autoregressive processes. J. Multivariate Anal., 36(2):175–198, 1991. [M140] R. A. Davis and M. Rosenblatt. Parameter estimation for some time series models without contiguity. Statist. Probab. Lett., 11(6):515–521, 1991. [M141] K. Helland, K. S. Lii, and M. Rosenblatt. Monte Carlo and turbulence. In Nonparametric functional estimation and related topics (Spetses, 1990), volume 335 of NATO Adv. Sci. Inst. Ser. C Math. Phys. Sci., pages 405–418. Kluwer Acad. Publ., Dordrecht, 1991. [M142] M. Rosenblatt. Introductory remarks. In Spatial stochastic processes, volume 19 of Progr. Probab., pages viii–x. Birkhäuser Boston, Boston, MA, 1991. [M143] K.-S. Lii and M. Rosenblatt. An approximate maximum likelihood estimation for non-Gaussian non-minimum phase moving average processes. J. Multivariate Anal., 43(2):272–299, 1992. [M144] M. Rosenblatt. Gaussian and non-Gaussian linear sequences. In New directions in time series analysis, Part I, volume 45 of IMA Vol. Math. Appl., pages 327–333. Springer, New York, 1992. [M145] M. Rosenblatt and B. E. Wahlen. A nonparametric measure of independence under a hypothesis of independent components. Statist. Probab. Lett., 15(3):245–252, 1992. [M146] M. Kramer and M. Rosenblatt. The Gaussian log likelihood and stationary sequences. In Developments in time series analysis, pages 69–79. Chapman & Hall, London, 1993. [M147] K.-S. Lii and M. Rosenblatt. Bispectra and phase of non-Gaussian linear processes. J. Theoret. Probab., 6(3):579–593, 1993.

Indicates article found in this volume.

xxxiii

xxxiv

Murray Rosenblatt: Bibliography

[M148] K.-S. Lii and M. Rosenblatt. Non-Gaussian autoregressive moving average processes. Proc. Nat. Acad. Sci. U.S.A., 90(19):9168–9170, 1993. [M149] M. Rosenblatt. The central limit theorem and Markov sequences. In Doeblin and modern probability (Blaubeuren, 1991), volume 149 of Contemp. Math., pages 171–177. Amer. Math. Soc., Providence, RI, 1993. [M150] M. Rosenblatt. A note on prediction and an autoregressive sequence. In Stochastic processes, pages 291–295. Springer, New York, 1993. [M151] M. Rosenblatt. Parameter estimation for some finite parameter stationary sequences. In Probability, statistics and optimisation, Wiley Ser. Probab. Math. Statist. Probab. Math. Statist., pages 213–218. Wiley, Chichester, 1994. [M152] M. Rosenblatt. Prediction and non-Gaussian autoregressive stationary sequences. Ann. Appl. Probab., 5(1):239–247, 1995. [M153] K.-S. Lii and M. Rosenblatt. Maximum likelihood estimation for nonGaussian nonminimum phase ARMA sequences. Statist. Sinica, 6(1):1–22, 1996. [M154] K.-S. Lii and M. Rosenblatt. Non-Gaussian autoregressive sequences and random fields. In Stochastic modelling in physical oceanography, volume 39 of Progr. Probab., pages 295–309. Birkhäuser Boston, Boston, MA, 1996. [M155] M. Rosenblatt. The likelihood of an autoregressive scheme. In Athens Conference on Applied Probability and Time Series Analysis, Vol. II (1995), volume 115 of Lecture Notes in Statist., pages 352–362. Springer, New York, 1996. [M156] M. Rosenblatt. Comments on estimation and prediction for autoregressive and moving average non-Gaussian sequences. In Stochastic models in geosystems (Minneapolis, MN, 1994), volume 85 of IMA Vol. Math. Appl., pages 353–358. Springer, New York, 1997. [M157] M. Rosenblatt. Some simple remarks on an autoregressive scheme and an implied problem. J. Theoret. Probab., 10(2):295–305, 1997. Dedicated to Murray Rosenblatt. [M158] K.-S. Lii and M. Rosenblatt. Line spectral analysis for harmonizable processes. Proc. Natl. Acad. Sci. USA, 95(9):4800–4803 (electronic), 1998. [M159] M. Rosenblatt. Non-Gaussian autoregressive and moving average schemes. In Asymptotic methods in probability and statistics (Ottawa, ON, 1997), pages 731–737. North-Holland, Amsterdam, 1998. [M160] K.-S. Lii and M. Rosenblatt. Spectral analysis for harmonizable processes. Ann. Statist., 30(1):258–297, 2002. [M161] K.-S. Lii and M. Rosenblatt. Correction: “Spectral analysis for harmonizable processes” [Ann. Statist. 30 (2002), no. 1, 258–297; mr1892664]. Ann. Statist., 31(5):1693, 2003.

Indicates article found in this volume.

Murray Rosenblatt: Bibliography

[M162] M. Rosenblatt. Non-Gaussian time series models. In Time series analysis and applications to geophysical systems, volume 139 of IMA Vol. Math. Appl., pages 227–237. Springer, New York, 2004. [M163] F. J. Breidt, R. A. Davis, N.-J. Hsu, and M. Rosenblatt. Pile-up probabilities for the Laplace likelihood estimator of a non-invertible first order moving average. In Time series and related topics, volume 52 of IMS Lecture Notes Monogr. Ser., pages 1–19. Inst. Math. Statist., Beachwood, OH, 2006. [M164] K.-S. Lii and M. Rosenblatt. Estimation for almost periodic processes. Ann. Statist., 34(3):1115–1139, 2006. [M165] M. Rosenblatt. An example and transition function equicontinuity. Statist. Probab. Lett., 76(18):1961–1964, 2006. [M166] K. S. Lii and M. Rosenblatt. Prolate spheroidal spectral estimates. Statist. Probab. Lett., 78(11):1339–1348, 2008. [M167] K.-S. Lii and M. Rosenblatt. Correction: “Estimation for almost periodic processes” [Ann. Statist. 34 (2006), no. 3, 1115–1139; mr2278353]. Ann. Statist., 36(3):1508, 2008. [M168] M. Rosenblatt. A comment on a conjecture of N. Wiener. Statist. Probab. Lett., 79(3):347–348, 2009.

Interviews and Biographies

[I1] Publications of Murray Rosenblatt. J. Theoret. Probab., 10(2):287–293, 1997. [I2] T. C. Sun. Murray Rosenblatt: his contributions to probability and statistics. J. Theoret. Probab., 10(2):279–286, 1997. [I3] D. R. Brillinger and R. A. Davis. A conversation with Murray Rosenblatt. Statistical Science, 24(1):116–140, 2009.

Indicates article found in this volume.

xxxv

Discussion of Bickel and Rosenblatt's work on Global Measures of Deviations for Density Estimates By Peter Bickel The paper [1] of Murray Rosenblatt's, which I coauthored, seems to have had a reasonably long shelf life. I believe Murray enjoyed working on this paper—I certainly did! The question posed by Murray was that of obtaining asymptotic approximations to the distribution of the maximum deviation of a kernel density estimate fn of an unknown density / based on a sample X i , . . . , Xn. If a limit distribution existed for

o<x
/TTT

J

\l fn(x) where /i n , an > 0 are normalizing constants not depending on / , then it is clear how to (a) construct approximate uniform confidence bands for / of the form fn db cny

fn.

(b) construct goodness of fit tests of point hypotheses H : f = fo using

vMx) as a test statistic. The heuristic that Murray advanced was the following. If Un(x) denotes the stochastic process defined by

Un(x)

r-T-(fn(x)

-

f(x)

/(*)

then Un(xi),..., Un(xk) are asymptotically iid Gaussian and Un(x) behaves locally like a stationary Gaussian process. Hence one would expect that the statistic based on maxo< x
0

fo(x)

We showed that under strong conditions, this statistic had a limiting Gaussian distribution consistent with its x2 like nature. Following this up in [5], Rosenblatt showed that this result held under very weak conditions in view of the representation of the statistic as a U statistic with a kernel dependent on the sample size. This was indeed a nice and useful observation. Subsequently, we used our methods to consider the testing problem and examined the rate at which alternatives tend to the hypothesis with non-negligible power. Not surprisingly, the rate is less than n~z which is what one obtains for Kolmogorov-Smirnov and similar tests, although the quadratic deviation is consistently better. However, as results of Ingster and Suslina [4] show, minimax results for different smoothness classes of alternatives are obtained for tests which resemble quadratic density deviation measures. These results clearly depend on what class of alternatives is

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_l, © Springer Science+Business Media, LLC 2011

considered, {/ : ||F — Fo||oo > £} where F is the cdf and / is the density is suitable for KS tests while {/ : | / 0 ) - f(y)\ < M\x - y\a all x,y, \\f - f0\\2 > 5} is suitable for the test of [5]. Extensions of this work to the dependent and multivariate cases, and processes other than kernel density ones were developed quickly by Rosenblatt and others [6]. Murray and I only had one further paper in this area, [2], although we had certainly corresponded on [5] and others. The second paper was more probabilistic and derived results on the maxima of Gaussian random fields, which were later obtained more generally by Hogan and Siegmund [4] using a different method. We envisaged these results to be applied to the extension of the results of [1] to the multivariate case. It is clear from all of the follow-up work that Rosenblatt's initial question was important and his intuition perfect. References [1] Bickel, P.J. and Rosenblatt M. (1973). On some global measures of the deviations of density function estimates, Ann. Statist. 1, 1071-1095. [Correction: Ann. Statist. 3 (1975), no. 6, 1370]. [2] Bickel P.J. and Rosenblatt M. (1973). Two-dimensional random fields, Multivariate III, 3-15, Academic Press NY.

Analysis

[3] Hogan M.L. and Siegmund D. (1986). Large deviations for the maxima of some random fields, Advances in Applied Probability 7, 2-22. [4] Ingster Y. I. and Suslina, I. A. (2000). Minimax nonparametric hypothesis testing for ellipsoids and Besov bodies. ESAIM Probab. Statist. 4, 53 135. [5] Rosenblatt M. (1975). A quadratic measure of deviation of two-dimensional density estimates and a test of independence, Ann. Statist. 3, 1-14. [6] Rosenblatt, M. (1976). On the maximal deviation of A:-dimensional density estimates, Ann. Probab. I 1009-1015.

2

M U R R A Y ROSENBLATT'S C O N T R I B U T I O N S TO S T R O N G M I X I N G By Richard C. Bradley

1. Introduction. Murray Rosenblatt's research has contributed much to the field of "strong mixing conditions," (i) by providing many results of his own in that field, and (ii) by inspiring a vast amount of research in that field by other people. This note will give just a small snapshot of (i) and an even smaller snapshot of (ii). Within the broader field of probability theory, the field of "strong mixing conditions" seemed to "take off" with the publication in 1956 of the paper of Rosenblatt [17] titled "A central limit theorem and a strong mixing condition." Before that, weaker notions of "mixing" had been studied by Eberhard Hopf [9] and others in ergodic theory; and "strong" forms of "mixing" had been implicitly present in the work of Doeblin [4] involving what is now known as "Doeblin's condition" for Markov chains, and also in Doeblin's [5] work on continued fractions (see Iosifescu [13] for details on this latter work). But it was Rosenblatt's [17] paper — which introduced the "strong mixing" ("a-mixing") condition, showed how it "coordinated" well with a Bernstein-type blocking scheme and produced a central limit theorem under that condition — that really seemed to light a spark and lead to the development of "strong mixing conditions" as a field of its own. In particular, the 1956 paper inspired a considerable number of papers by other researchers in which variants of "strong mixing conditions" were proposed and used in various types of limit theorems. Those papers include, among many others, Ibragimov [10, 11], Kolmogorov and Rozanov [14], and Cogburn [2]. Subsequently, the field of "strong mixing conditions" exploded into an active and vibrant area of research. Strong mixing conditions and the associated central limit theorems have enjoyed broad appeal beyond probability and into the mainstream statistics community. Applications of strong mixing include applications as diverse as block or stationary bootstrapping of time series (see e.g., Politis and Romano [16]); inference for linear time series; and parametric and nonparametric inference for nonlinear time series (see e.g., Fan and Yao [7]). Strong mixing is often the key tool for establishing asymptotic normality of various estimators of complicated time series models. In particular, inference for popular financial time series models such as GARCH and stochastic volatility models, often rely on the process being strongly mixing. A short note such as this on Rosenblatt's work on strong mixing conditions cannot do justice to all of his own contributions to this field, and it cannot even begin to touch on the many key advances in this field that have been made by others. For details on the

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_2, © Springer Science+Business Media, LLC 2011

3

history of t h e development of t h e field, see e.g. [6] or t h e series [1]. This article will provide a sampling of just a few of t h e m a n y contributions t h a t Rosenblatt has m a d e to this field, with a couple of examples of their direct inspiration of subsequent research by others. To set t h e table for t h e discussion to follow, t h e rest of Section 1 here will be devoted to t h e definition of strong mixing and two other mixing conditions. Suppose (fi, T, P) is a probability space. For any two cr-fields A and B C J-, define t h e following three measures of dependence: a(A,B)

:=

sup \P(A n B) - P(A)P(B)\, AeA,BeB {A,B)\= sup \P(B\A) - P(B)\, AeA,BeB,P(A)>0 p(A,B):=sup\Corr(fl9)\

(1.1) and

(1.2) (1.3)

where this last s u p r e m u m is taken over all pairs of square-integrable r a n d o m variables / and g such t h a t / is A-measurable and g is S-measurable. Now suppose X : = (Xkl k G Z) is a strictly stationary sequence of r a n d o m variables on our given probability space ( 0 , J F , P). For each positive integer n, define t h e following three dependence coefficients: a{n) := a(a(Xk,

k < 0),a(Xk,

<j>(ri) : = (j){a{Xk, k < 0), a(Xk, p{n) := p{a(Xk,

k < 0), a{Xk,

k > n)), k > n)), k > n)).

(1.4) and

(1.5) (1.6)

Here and below, t h e notation a(...) means t h e cr-field of events generated by ( . . . ) . T h e (strictly stationary) sequence X is said to be "strongly mixing" (or "a-mixing") if a(n) -^ 0 as n -^ oo, "0-mixing" if <j)(n) -^ 0 as n -^ oo, and "p-mixing" if p(n) -^ 0 as n —> oo. T h e "strong mixing" ("a-mixing") condition is t h e one t h a t was introduced and used in a central limit theorem in t h e 1956 paper of Rosenblatt [17]. T h e "0-mixing" condition was introduced by Ibragimov [10] and was studied by b o t h Cogburn [2] and Ibragimov [11]. T h e "p-mixing" condition was introduced by Kolmogorov and Rozanov [14]. (The "maximal correlation coefficient" defined in (1.3) had been studied earlier in statistical contexts t h a t did not involve "random sequences.") It is well known t h a t 0-mixing implies p-mixing, t h a t p-mixing implies strong mixing (a-mixing), and t h a t t h e converses of b o t h of those s t a t e m e n t s are false. For a formulation of other classic strong mixing conditions and their connections with t h e ones here, see e.g. [6] or [1, V I , C h a p t e r s 3 and 5].

2. M a r k o v c h a i n s . An ongoing t h e m e of research by m a n y people is t h e question of what strong mixing properties are satisfied by r a n d o m sequences with various other specific types of dependence structure. T h e books by Rosenblatt [20, 24, 26] gave (along with much other material) a t r e a t m e n t of different facets of this general question. In particular,

4

strong mixing conditions for strictly stationary Markov chains (with, say, state space R) has been a topic of much research by Rosenblatt. A small selection of specific contributions of his on this topic will be briefly reviewed here. Strong mixing conditions for Markov chains are treated extensively in Chapter 7 of Rosenblatt's 1971 book [20] on Markov processes. Along with much other information, that chapter gave what were, to the best of my knowledge, (1) the first known examples of strictly stationary Markov chains that are ^-mixing in the "usual" direction of time but fail to be 0-mixing in the "reversed" direction of time, and (2) the first known examples of strictly stationary p-mixing Markov chains such that for every positive integer n, the n-step (conditional) transition probability distributions are almost surely singular with respect to the marginal distribution. Central limit theory under the p-mixing condition apparently started with central limit theorems of Rosenblatt [19] [20, Chapter 7] for square-integrable "instantaneous" functions of strictly stationary p-mixing Markov chains, and also with his paper [18] giving central limit theorems for some kernel-type estimators of marginal probability density for p-mixing Markov chains with an absolutely continuous marginal distribution. Shortly thereafter, the development of central limit theory for general (not necessarily Markovian) strictly stationary p-mixing sequences was promoted by Ibragimov [12]. (For more on that theory, see e.g. [1, VI, Chapter 11].) In [18] and [20, Chapter 7], Rosenblatt used the Riesz Convexity (Interpolation) Theorem to prove the equivalence of a class of mixing conditions (including p-mixing) for Markov chains. That was apparently the first use of "interpolation theorems" of that type for comparison of different mixing conditions. This work subsequently inspired a more extensive use (not restricted to Markov chains) of various interpolation theorems to establish "covariance" and "cumulant" inequalities under various strong mixing conditions, and to compare different measures of dependence on which strong mixing conditions are based. For some of the main results and history of that later development, see [1, VI, Chapter 4], including the notes at the end of that chapter. In [23], Rosenblatt showed that (in essence) an ultra-strong version of a "Markov regularity condition" that had been used in various papers, could not be satisfied by strictly stationary Markov chains other than the i. i. d. sequences. Around the same time, Rosenblatt conjectured to me that this condition could not be satisfied by any strictly stationary sequences, Markovian or not, except the i. i. d. sequences. Shortly thereafter, I was able to confirm the conjecture. For a detailed exposition, see [1, V2, Theorem 24.2]. 3. A little known contribution. A remarkable paper of Rosenblatt [21] contains some ingenious fundamental insights in connection with the strong mixing condition for Markov chains. Unfortunately, this paper seems to have been practically unknown for at least thirty years after its publication. In this section, the content and the connections of this paper with the work of other researchers will be reviewed in some detail.

5

It will be useful to first formulate a classic result of Doeblin [4] (just the "aperiodic" case) involving what is now known as "Doeblin's condition" for Markov chains. It is well known that Doeblin's result can be reformulated in a different but equivalent way in terms of the 0-mixing condition (which was defined a couple of decades later), as follows: Theorem 3.1 (Doeblin (reformulated)). Suppose X := (Xj^k G Z) is a strictly stationary Markov chain which is ergodic, aperiodic, and satisfies (j)(n) < 1 for some n > 1. Then 4>(n) —» 0 (at least exponentially fast) as n —» oo. We shall return to this theorem below, after picking up some other threads. For strictly stationary random sequences, Cogburn [2] introduced, under the name "uniform ergodicity," a "Cesaro" variant of the strong mixing condition, and (along with other results) proved a central limit theorem involving this condition. The uniform ergodicity condition need not be formally defined here. Now (for strictly stationary sequences) strong mixing implies uniform ergodicity, and uniform ergodicity implies ergodicity. A natural question, pertinent to the development of limit theory for strictly stationary sequences, is how much weaker uniform ergodicity is than strong mixing. Rosenblatt [21] examined this question and proved the following result involving Markov chains: Theorem 3.2 (Rosenblatt). Suppose X := {X^^k G Z) is a strictly stationary Markov chain which is "mixing" (in the ergodic-theoretic sense) and satisfies uniform ergodicity; then a(n) —> 0 as n —> oo (strong mixing). In the same paper, Rosenblatt [21] also suggested that this result might still hold for general strictly stationary sequences (i.e., without the Markov assumption). That now seems to be an open question, with (so far) no decisive evidence either way. Formally, Theorem 3.2 was the "goal" of Rosenblatt's [21] paper. However, the various arguments in this paper contained further peculiar and ingenious insights that have remained little known. In particular, Rosenblatt showed that if X := (Xf^k G Z) is a strictly stationary Markov chain that fails to satisfy strong mixing, then there exists a number q G (0,1) such that the following condition holds: For every positive integer N and every 8 > 0, there exist events G G cr(X^, k < 0) and H G cr(X^, k > N) such that q — 0 < P(H) < q + 6 and P(GAH) < 6. (Here A denotes symmetric difference. Obviously, adjusting the parameters a little, one can have q — 0 < P(G) < q + 0 as well.) Rosenblatt also showed that this condition holds for every q G (0,1) under the additional assumption of a trivial (past or) future "tail cr-fleld." Combining this fact with q = 1/2 and other arguments in Rosenblatt's [21] paper, implicitly yields the following theorem: Theorem 3.3 (Rosenblatt). Suppose X := (Xk,k G Z) is a strictly stationary Markov chain which is ergodic, aperiodic, and satisfies a(n) < 1/4 for some n > 1. Then a(n) —> 0 as n —> oo.

6

Now for any two cr-fields A and 23, one has by trivial arguments that a(A,B) < 1/4 and c/)(A,B) < 1. Hence for a given strictly stationary sequence (Markovian or not), one has that a(n) < 1/4 and (f>(n) < 1 for every positive integer n. Thus Theorem 3.3 is a nearly exact analog, for strong mixing, of Doeblin's result (transcribed to "0-mixing") in Theorem 3.1. The only significant difference is that in the context of Theorem 3.1, the mixing rate has to be (at least as fast as) exponential, whereas in Theorem 3.3, the mixing rate need not be exponential (as was shown, for example, for the Markov chains studied by Davydov [3]). Yet while Theorem 3.1 receives frequent attention in the literature, Theorem 3.3 was apparently "absent from the radar screen" for three decades after the publication of [21]. For a detailed exposition of the material in Rosenblatt's [21] paper (including Theorems 3.2 and 3.3 and the other insights from that paper described here), see [1, V2, Sections 21.25-21.28 and 24.17-24.22]. 4. Strong mixing conditions and estimation problems for random fields. At this point, we depart from Markov chains and turn to some contributions of Rosenblatt involving more general random sequences as well as random fields. As Richard Olshen explained to me about 30 years ago, a major motivation behind Rosenblatt's [17] paper and the subsequent development of the field of strong mixing conditions, was to provide a way of proving limit theorems for use in statistical inference for data from time series that seemed to be "weakly dependent" but did not necessarily seem to "fit" more specific models of dependence such as Markov chains, martingales, Gaussian sequences, or ARMA models. Since then, in that spirit, a vast amount of research has been done by many researchers on statistical estimation of various kinds under strong mixing conditions. The 1985 book of Rosenblatt [24] was a prominent treatise on this topic. Further, much of that book gave a treatment of random fields (not just random sequences) satisfying strong mixing conditions, an ongoing topic of research of Rosenblatt and many others. We shall return to that book after providing some background information. Suppose d is a positive integer, and X := (Xkj k G Z d ) is a strictly stationary random field. For a given positive integer n, define a*(n) := sup a(a(Xk, k G A),a(Xk,k

G £)),

(4.1) d

where the supremum is taken over all pairs of nonempty, disjoint sets A, B C Z such that distM, B) :=

min

\\a - b\\ > n.

(4.2)

aeA,beB

Here, for a given k G Zd1 \\k\\ denotes the Euclidean norm of k. The (strictly stationary) random field X will be said to be "a*-mixing" if a*(n) —> 0 as n —> oo. In the case d — 1 (the classic case of strictly stationary random sequences), a*-mixing is a stronger condition than the strong mixing (a-mixing) condition (involving (1.4)), because in (4.1) the two index sets A and B are not restricted to "past" and "future" (as

7

in (1.4)); they can instead be "interlaced," with each set having elements between ones in the other set. Extending well known arguments and results of Kolmogorov and Rozanov [14] involving the usual strong mixing and p-mixing conditions in the case of stationary Gaussian sequences, Rosenblatt [22] showed that for a given stationary Gaussian random field X := (Xk^k G Z d ), (i) the a*-mixing condition is equivalent to the corresponding "/?*mixing" condition (defined using in the right hand side of (4.1) the maximal correlation coefficient from (1.3)), and (ii) those equivalent conditions are satisfied if X has a continuous positive spectral density function. (An exposition of those results was also given in [24, pp. 73-77].) Later, it became known that observation (i) holds for general (not necessarily Gaussian) strictly stationary random fields X := (Xk, k G Zd). (See [1, V3, Theorem 29.12(II)(A)] for the result, its proof, and its history.) We shall continue to use the term "a*-mixing" (instead of "p*-mixing") in the discussion below. Of course in contrast, for random sequences (the case d = 1), the strong mixing (a-mixing) condition (based on (1.4), with the two index sets being just "past" and "future") is strictly weaker than the p-mixing condition (based similarly on (1.6)). (In a slightly different vein, the paper of Rosenblatt [22] also gave a central limit theorem for random fields under "martingale-like" assumptions similar to those used by Gordin [8] in a classic central limit theorem for random sequences.) Now let us return to the 1985 book of Rosenblatt [24]. This book gives an extensive treatment of various estimation problems for strictly stationary a-mixing random sequences X := (Xfc, k G Z) and strictly stationary a*-mixing random fields X := (X&, k G 7id). The problems considered there include the estimation of (under appropriate assumptions) autocovariances, spectral densities, higher-order spectral densities, marginal probability densities, and regression functions. The book contains a wealth of results and opened up a broad spectrum of new research problems. In the formulation of most of the theorems in the 1985 book involving a-mixing or a*-mixing, there is no assumption on the rate of convergence of a(ri) or a*(n) to 0; instead the "mixing rate" is allowed to be arbitrarily slow. To compensate, there typically are assumptions on higher-order moments and/or higher-order cumulants. For example, in a central limit theorem for some estimators of spectral density for strictly stationary random fields, Rosenblatt [24, p. 157, Theorem 7] assumes (in addition to a*-mixing) the summability of cumulant functions of the X^'s up to order 8. Now the usual estimators of spectral density (such as ones based on periodograms) involve quadratic forms of the Xfc's, and hence the natural basic moment assumption for a central limit theorem for those estimators is EX$ < oo, in order for the quadratic forms to have finite second moments. A couple of years after the publication of his book, Rosenblatt [25] posed the problem of whether in theorems such as the one alluded to here, one could get by with

8

|

just the summability of cumulants of order 4 (instead of order 8), or even with just the assumption of finite fourth moments (of course in addition to the assumption of ce*-mixing). In his 1994 Ph.D. thesis at Indiana University (see also [15, Theorem 2]), Curtis Miller gave an affirmative answer to that question for a class of estimators that involve "blocks" of the Xfc's. Miller's result only assumes EXQ < oo; no assumption of moments of higher than fourth order and no further assumption on covariances or cumulants are required. (Miller formally used the "p*-mixing" condition, but under strict stationarity that condition is equivalent to a*-mixing as noted above.) The question is apparently still open for other estimators of spectral density, such as ones that involve averaging the periodograms over neighboring frequencies. Anyhow, this is just another particular example of the many ways in which the work of Rosenblatt on strong mixing conditions has directly inspired the research in the field by others, including young researchers. Starting with his 1956 paper [17], Murray Rosenblatt has contributed a great deal to the field of strong mixing conditions, and has helped establish it as an active field of research. His work has directly inspired many contributions in that field by other researchers. Strong mixing conditions continue to be an active field of research that has important modeling and inference ramifications in a variety of applications. Acknowledgement: My thanks go to Richard Davis, who made many helpful comments and suggestions and provided the third paragraph of this article. References [1] R.C. Bradley. Introduction to Strong Mixing Conditions, Volumes 1,2, and 3. Kendrick Press, Heber City (Utah), 2007. [2] R. Cogburn. Asymptotic properties of stationary sequences. Univ. Calif. Publ. Statist. 3 (1960) 99-146. [3] Yu.A. Davydov. Mixing conditions for Markov chains. Theor. Probab. Appl. 18 (1973) 312-328. [4] W. Doeblin. Sur les proprietes asymptotiques de mouvement regis par certains types de chaines simples. Bull. Math. Soc. Roum. Sci. 39 (1937), no. 1, 57-115, no. 2, 3-61. [5] W. Doeblin. Remarques sur las theorie metriques des fractions continues. Compositio Math. 7 (1940) 353-371. [6] P. Doukhan. Mixing: Properties and Examples. Springer, New York, 1995. [7] J. Fan and Q. Yao. Nonlinear Time Series: Nonparametric and Parametric Springer, New York, 2003.

Methods.

[8] M.I. Gordin. The central limit theorem for stationary processes. Soviet Math. Doklady 10 (1969) 1174-1176.

9

E. Hopf. Ergodentheorie. Springer-Verlag, Berlin, 1937. LA. Ibragimov. Some limit theorems for stochastic processes stationary in the strict sense. Dokl. Akad. Nauk. SSSR 125 (1959) 711-714. LA. Ibragimov. Some limit theorems for stationary processes. Theor. Probab. Appl. 7 (1962) 349-382. LA. Ibragimov. A note on the central limit theorem for dependent random variables. Theor. Probab. AppL 20 (1975) 135-141. M. Iosifescu. Doeblin and the metric theory of continued fractions: A functional theoretic solution to Gauss' 1812 problem. In: Doeblin and Modern Probability, (H. Cohn, ed.), pp. 97-110. Contemporary Mathematics 149. American Mathematical Society, Providence (Rhode Island), 1993. A.N. Kolmogorov and Yu. A. Rozanov. On strong mixing conditions for stationary Gaussian processes. Theor. Probab. Appl. 5 (1960) 204-208. C. Miller. A CLT for the periodograms of a p*-mixing random field. Process. Appl. 60 (1995) 313-330.

Stochastic

D.N. Politis and J.P. Romano. The stationary bootstrap. J. Amer. Statist. Assoc. 89 (1994) 1303-1313. M. Rosenblatt. A central limit theorem and a strong mixing condition. Proc. Natl. Acad. Set. USA 42 (1956) 43-47. M. Rosenblatt. Density estimates and Markov sequences. In: Nonparametric Techniques in Statistical Inference, (M.L. Puri, ed.), pp. 199-210. Cambridge University Press, London, 1970. M. Rosenblatt. A prediction problem and central limit theorems for stationary Markov sequences. In: Proceedings of the Twelfth Biennial Seminar of the Canadian Mathematical Congress on Time Series and Stochastic Processes; Convexity and Combinatorics (Vancouver, B.C., 1969), (R. Pyke, ed.), pp. 99-114. Canadian Mathematical Congress, Montreal, 1970. M. Rosenblatt. Markov Processes. Structure and Asymptotic Verlag, Berlin, 1971.

Behavior.

Springer-

M. Rosenblatt. Uniform ergodicity and strong mixing. Z. Wahrsch. verw. Gebiete 24 (1972) 79-84. M. Rosenblatt. Central limit theorems for stationary processes. Proceedings of the Sixth Berkeley Symposium on Probability and Statistics, Volume 2, pp. 551-561. University of California Press, Los Angeles, 1972. M. Rosenblatt. Some remarks on a mixing condition. Ann. Probab. 7 (1979) 170-172. M. Rosenblatt. Stationary Sequences and Random Fields. Birkhauser, Boston, 1985. M. Rosenblatt. Private communication, 1987. M. Rosenblatt. Gaussian and Non-Gaussian Linear Time Series and Random Fields. Springer, New York, 2000.

10

Murray Rosenblatt and cumulant/higher-order/poly spectra By David R. Brillinger The bispectrum, or third order spectrum, of a stationary process has been around at least since the early 50s, for example it appeared in the paper Tukey (1953). It was studied in some detail in John Van Ness's thesis, "Estimates of the Bispectrum of Stationary Random Processes", supervised by Murray Rosenblatt. A 1965 Annals of Mathematical Statistics paper, "Estimation of the bispectra", Rosenblatt and Van Ness (1965) followed. There is a Soviet history extending the concept. In particular Shiryaev (1989) writes: "In the late 1950s and early 1960s Kolmogorov suggested to his pupils V. P. Leonov and A. N. Shiryaev a series of problems related to the issue of nonlinear analysis of random processes (in particular, in radio technology) which brought about the techniques of calculating cumulants under nonlinear transformations, and the development of the theory of spectral analysis of high-order moments of stationary random processes." I first met Murray in New Jersey in 1963. He was then consulting at Bell Telephone Laboratories Murray Hill under a Brown University program. In particular he was developing properties of a cepstrum estimate, the cepstrum being the Fourier transform of the log of a second-order spectrum. It had been developed as a tool for estimating how deep explosions, as opposed to earthquakes, were sited. The study in the polyspectra paper, Brillinger (1965), started out as a young academic having fun. Independently of the Russian work I had found that cumulants were a pertinent tool. They were a multilinear operator with the property that if for a vectorvalued variate some subset of its entries was independent of the complementary subset, then the joint cumulant of all was zero. This property takes one directly to a definition of mixing for general stationary processes. Perhaps the Russians already knew that type of result. At one point Tukey had mentioned the word polyspectra, and a connection was made. The 1965 paper surely led to my getting to collaborate with Murray later when we were both in London. Ady and Murray had come to London on a Guggenheim Fellowship in the Fall of 1965. On meeting Murray again I was surely a bit disrespectful going right up to Murray and saying something like, "How about we do some work and write a paper together?" Murray replied with a remark like "Sure.", and the papers Brillinger and Rosenblatt (1967a, b, c) resulted. The first, Brillinger and Rosenblatt (1967a,) summarizes the work to be presented in (1967b) and (1967c). It is noteworthy for also seeking to develop a tensor-like notation for joint cumulant spectra after encouragement from John Tukey.

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_3, © Springer Science+Business Media, LLC 2011

11

Brilllinger and Rosenblatt (1967b) sets down some asymptotic theory of estimates of a cumulant spectrum developed under an asymptotic independence or mixing assumption. The estimate is based on an empirical Fourier transform of a long stretch of data. A higher order periodogram is defined and smoothed to form the estimate. An important insight in the work is to base the estimates on the Fourier transforms evaluated at the Fourier frequencies. Large sample means and variances of the estimate were developed, with uniform error bounds. In developing the results substantial use was made of the theorem in Leonov and Shiryaev (1959). The mixing condition involved the absolute summability of the time domain joint cumulant functions. Asymptotic normality was shown by proving that the standardized cumulants of order greater that 2 tended to those of the normal, which is determined by its moments. Brillinger and Rosenblatt (1967c) concerns properties and interpretations of cumulant spectra as well as details of computation and a scientific example. The thorny problem of aliasing is addressed with fundamental domains for the second, third and fourth order cases presented. Estimates of the second, third and fourth order spectra are computed for a long series of monthly relative sunspot numbers. A fast Fourier transform is employed in the computations. Murray was an academic role model for me as my career progressed. He was someone to ask to write recommendation letters and to provide advice concerning the professional sides of academic life. For example, I remember being quite concerned when I found an error in a paper that I had published. Murray's reaction was, "Is that the first one?" There have been others since.

References. D. R. Brillinger, "An introduction to polyspectra", Ann. Math. Statist. 36 (1965) 13511374. D. R. Brillinger and M. Rosenblatt: "Asymptotic theory of estimates of kth order spectra," Proc. Nat. Acad. Sci., 57 (1967a), 206-210. D. R. Brillinger and M. Rosenblatt: "Asymptotic theory of estimates of kth order spectra," Spectral Analysis of Time Series, John Wiley (1967b), 153-188. D. R. Brillinger and M. Rosenblatt. "Computation and interpretation of kth spectra," in Spectral Analysis of Time Series, John Wiley (1967c), 189-232.

order

M. Rosenblatt: "Remarks on higher order spectra," in Multivariate Analysis, Academic Press (1966), 383-389 M. Rosenblatt: "Cumulants and cumulant spectra," in Handbook of Statistics, vol. 3 (eds. D. Brillinger and P. Krishnaiah), (1983), 369-382.

12

M. Rosenblatt and J. W. Van Ness: "Estimation of the bispectra," Ann. Math. Stat., 36 (1965), 1120-1136. A. N. Shiryaev, "Kolmogorov: life and creative activities." Ann Probab. 17 (1989) 866944. J. W. Tukey: "The spectral representation and transformation properties of the higher moments of stationary time series." Reprinted in The Collected Works of John W. Tukey, Vol. 1 (Edited by D. R. Brillinger), 165-184. Wadsworth, Belmont. J. Van Ness "Estimates of the Bispectrum of Stationary Random Processes" 1964 Ph.D. Dissertation, Brown University.

13

Rosenblatt's Contributions to Deconvolution By Keh-Shin Lii

A simple model of deconvolution can be described as observing {x(t)} which is a convolution of a signal {s(t)} with a filter {f(j)},x

= s*f.

More specifically, we have

oo

x(t)= £

j=-oo

fUHt-j).

The problem of deconvolution is to recover {s(t)} based on the output process {x(t)}. the filter {f(j)}

If

is known then the problem is fairly straightforward. The blind deconvo-

lution, in signal processing terminology, is to recover {s(t)} based solely on {x(t)} without knowing {/(j)}.

Statisticians may be more interested in the estimation of {f(j)}

under

certain conditions on {s(t)} and {/(j)}- This problem and its many variations have very broad applications in signal processing, image restoration, geo-exploration, seismology, radio astronomy among others [11, 33, 38, 39]. Assume that the signal random variables {s(t)} are independent and identically distributed with mean 0 and variance 1. Let the filter {f(j)}

be a sequence of real constants

such that oo

£ f(j) < °°

j=-oo

and f(z) = J2j f(j)zj

be the z-transform corresponding to the process {x(t)}. f(e~iX) = £ / 0 > " y A =

Then

\f(e-*x)\exp{th(X)}

j

is the frequency response function or the transfer function where h(X) is the phase function of the transfer function. If we know f(e~tX) for all A G [0,27r], then we can obtain

{f(j)}

for all j . The modulus / ( j ) , |/(j)|, of the frequency response function can be obtained from

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_4, © Springer Science+Business Media, LLC 2011

14

the spectral density of {x(t)} which is 5(A) = ^ | / ( e - * A ) | 2 .

It is clear that if the {x(t)} process or the random signal process {s(t)} is Gaussian then the full probability structure of {x(t)} is determined by g(X) or equivalently by |/(e _ z A )| which is determined by the second order covariance property of the process. The phase information h(X) of f(e~lX) is not identifiable in the Gaussian case. Any hope of getting information on the phase function will require the process to be nonGaussian. Murray Rosenblatt's interest and insight in this problem may have stemmed from his interest in the higher order spectra, especially higher-order cumulant spectra which is fundamental in dealing with nonGaussian processes [34, 4, 5, 19]. The kth-order cumulant spectral density of {x(t)} is given by 6fc(Ai, A2) • • -, Afc_0 = j^-J{e-lM)f{e-^)

• • • /(e-^-i)/(e<(^+"+**-i>),

where 7^ is the kth order cumulant of {s(t)}. Therefore the phase of the kth order spectrum &fc(Ai, A2, • • •, Afc-i) is related to the phase h(X) of the the transfer function

f{elX).

In the paper [35], Murray laid out the basic idea on how the phase function h(X) can be identified up to a linear shift of cX using the phase of the bispectra of the process. It was noted that the same result holds by using any kth-order spectra with k > 2. A similar but different idea of using bispectra to obtain phase information of the transfer function is outlined in Brillinger [6]. Of course for different c's we will get different transfer functions and the deconvolution can not be realized. The problem of indeterminancy of the linear shift of the phase was solved in [20], where it was shown that the phase function can be identified up to an integer shift rX with r an integer. The basic idea is that the transfer function f(e~tX) has to be real when A = TT and

15

hence its phase at TT has to be TTT with r an integer. Since this integer phase shift corresponds to reindexing the noise sequence {s(t)}, the deconvolution problem is essentially solved in this case. If the linear process {x(t)} is a finite parameter ARMA(p,q) process then there are other methods available to identify the phase function. In [20] a few other ideas were laid out to identify the correct ARMA model in terms of the location of the roots of the characteristic functions of the ARMA model. Again, the basic idea here is to match various higher order moments or cumulants of the process with the correct coeffients or roots. The paper [20] seemed to have generated quite a bit interest in the signal proceesing community. The importance of the phase in signals has been noted in signal and image processing and geoexploration problems [31, 33, 38, 39]. Many papers were published using higher-order statistics and higher-order spectral analysis to analyze signals, images and systems. These activities resulted in the first 'Workshop on Higher-Order Spectral Analysis' held at Vail, Colorado in 1989 with a Proceedings of the Workshop and a 'Special Section on Higher Order Spectral' that appeared in the July 1990 issue of the IEEE Transactions on Acoustics, Speech and Signal Processing. Following the first workshop, four more 'workshops on higher-order statistics' were held in odd years with the location alternating betwwen Europe and North America until the last one in 1997. The last two Proceedings published in the 1995 and 1997 Workshops have more than 450 pages of articles each. The 1995 workshop resulted in IEEE Signal Processing September 1996 issue being the 'special issue on higher-order statistics'. A book [32] was published in 1993 on these topics.

If the output process {x(t)} is observed with an independent Gaussian noise rj(t) it is shown in [21] that we can still estimate the filter {f(j)}

up to an unknown scale factor c. If

the additive Gaussian noise rj(t) is white then we can estimate the filter consistently up to a time shift. A detailed discussion on the use of fourth-order cumulants to estimate the filter function for deconvolution is given in [22], where a discussion on the sample size relative to various orders of cumulants of {s(t)} is given. In [23] it is demonstrated with a well-log

16

data and water-gun signature that the deconvolution can be more effective using the Fourier transform of the tapered sample higher-order cumulants to obtain higher-order spectrum instead of using the smoothed higher-order periodogram. For this reason, theoretical properties of such higher-order spectral estimates are given in [25, 26]. The particular formula using the phase of a higher-order spectrum to obtain the phase of the transfer function in [20] utilized only a subset of the full higher-order spectral phase function. Another method which utilized the whole effective higher-order spectral phase function is given in [28] with better asymptotic convergence properties as a function of sample size. Generlization of the previous results to the deconvolution problem in autoregressive random fields is given in [30,37]. In the deconvolution problem described above it is generally assumed that the spectral density of the process is strictly positive. In [24] our attention turned to the blind deconvolution problem when the transfer function has zeros, which do occur in geophysical investigations [31]. In [24] it is shown that if the zeros are finitely many and are of finite order then the transfer function can still be consistently estimated without the minimum phase assumption when the process is nonGaussian. A procedure is given so that the deconvolution can be effectively carried out. If the transfer function is zero in an interval for A G (a, b) with 0 < a < b < ir, then the smallest mean square error in deconvolution that can be achieved is (6 — a)/2n even if we assume that the phase can be estimated. It was also shown that the phase is not identifiable if the transfer function is zero in an interval around TV with positive length i.e., the process is band-limited. However if all higher order cumulants of {x(t)} are available then the phase is identifiable up to a linear phase shit SX with 5 a real number.

In the previous discussion the process is a general linear process and the approach to the identification of the phase information is the use of higher-order cumulant spectra which in general, is monparametric' in the sense that the probability distribution of the signal process

17

{s(t)} is not used explicitly other than that it is nonGaussian and that a certain kth order cumulant exists and is non-zero. Now if the process {x(t)} can be represented by a finite parameter ARMA(p,q) process, then as noted before certain methods based on moments can be used to identify the phase information and in this case it is equivalent to finding the locations of the roots of the charateristic functions associated with the AR and MA parts of the process or eqivalently the coefficients of the ARMA model without the usual causal or invertible conditions under the Gaussian assumption. Maximum likelihood estimation of the parameters of an ARMA model under the Gaussian assumption has been discussed widely in the literature [9, 36]. Of course in this case the phase information is not available and the ARMA model is assumed to be causal and invertible. In [7] it is shown that given a nonGaussian probability density for the independent and identically distributed process {s(£)}, the stationary AR(p) process x(t) = (j>ix(t — 1) + . . . + (f)px(t — p) + s(t) is identifiable through a maximum likelihood procedure whether the process is causal or not. The idea is to reparametrize the model by decomposing the autoregressive polynomial into its causal and purely non-causal components and then analyzing the corresponding AR processes that resulted from this decomposition. The likelihood function can be approximated using these processes and the estimates of the parameters of the possibly non-causal AR process are the solutions to the likelihood equations. Similar results for possibly non-invertible MA(q) process are given in [27] and for general ARMA process in [29]. These results give possibly 'efficient' methods for blind deconvolution. If the nonGaussian probability distribution of the {s(t)} process is unknown then it is demonstrated in the previous papers that a quasilikelihood method can be used to estimate the parameters of the ARMA process by assuming that the unknown probability density is Laplace (two-sided exponentail) which leads to a least absolute deviation criterion. A modified least absolute deviation method is shown to be consistent when the input process in the AR(p) model has a stable law distribution with index a G (1,2) [13].

18

Without the causal and invertible conditions for nonGaussian ARMA processes one is naturally led to an interesting subclass of processes called all-pass ARMA models. An ARMA model is all-pass if every root of the AR polynomial (f)(z) is a reciprocal of a root of the MA polynomial 9(z) matched with multiplicity and vice versa. In such a case the corresponding filter is f(e~lX)

= (f)(e~tX)/6(e~tX) and \f(e~tX)\

is a constant. This means that whatever

the spectum of the input process {s(t)}, it will pass the ARMA filter unchanged except for a multiplicative constant, hence the name all-pass. So the spectral density of the process

is a constant. This means that the process {x(t)} is uncorrelated or white but not independent if the input independent process is nonGaussian. Processes which are second order uncorrelated or white but with higher order dependence occur often in financial data. The estimation of parameters of such all-pass ARMA models using least absolute deviations is given in [8], using maximum likelihood in [1] and using rank based procedures in [2]. The rank based method can have the same asymtotic efficiency as maximum likelihood estimators and are robust to some distributional assumptions. There are deconvolution problems in signal processsing when the probability distribution of the signal process {s(t)} is discrete with finitely many points of support such as in the finite alphabet transmission. These deconvolution problems were treated in [17, 12]. The finite tone image deconvolution or debluriing problems were treated in [18] where the distribution of the pixels is two-tone (or finite-tone) without the stationarity assumption. Methods using the maximization of the standardized cumulant of the deconvolved process to estimate the filter are given in [10].

Murray's interest in higher-order spectra began more than four decades ago [34], which ultimately led to the deconvolution problem [35]. Murray's fundamental contributions in both areas have had a long lasting impact on many aspects of statistical probems and applications as described in this brief article. His influence in the area of blind or noncausal

19

deconvolution is still ongoing [3], and has expanded to many related problems in economics [15], in medicine [14], and in signal processing [16,40], among others.

References [1] Andrews, B., Davis, R.A., and Breidt, F. J. (2006). "Maximum Likelihood Estimation for All-Pass Time Series Models". J. Multivariate Analysis. 97, 1638-1659 [2] Andrews, B., Davis, R.A., and Breidt, F. J. (2007). "Rank Estimation for All-Pass Time Series Models". Annals of Statistics, 35, 844-869. [3] Andrews, B., Calder, M. and Davis, R.A. (2009). "Maximum likelihood estimation for a-stable autoregressive processes". Annals of Statistics, 37, 1946-1982. [4] Brillinger, D. R., Rosenblatt, M. (1967a) "Asymptotic theory of k-th order spectra." In Spectral Analysis of Time Series, Ed. B. Harris, 153-188. New York, Wiley. [5] Brillinger, D. R., Rosenblatt, M. (1967b) "Computation and interpretation of k-th order spectra." In Spectral Analysis of Time Series, Ed. B. Harris, 189-232. New York, Wiley. [6] Brillinger, D. (1977). " The Identification of a Particular Nonlinear Time Series System," Biometrika, 64, 509-515. [7] Breidt, F. J., Davis, R. A., Lii, K. S., and Rosenblatt, M. (1991). "Maximum Likelihood Estimation for Noncausal Autoregressive Processes", J. Multivariate Analysis, 36, 175-198. [8] Breidt, F.J., Davis, R.A., and Trindade, A. (2001). "Least Absolute Deviation Estimation for All-Pass Time Series Models". Annals of Statistics 29, 919-946 [9] Brockwell. P. J., and Davis, R. A. (1987). Time Series: Theory and Methods. SpringerVerlag, New York. [10] Cheng, Q. (1990). "Maximum standardized cumulant deconvolution of non-Gaussian linear processes", Annals of Statistics, 18, 1774-1783. [11] Donoho, D. (1981). "On minimum entropy deconvolution", In Applied Time Series Analysis, II. Ed. by Findly, D. F. 565-608. Academic Press New York. [12] Gamboa,F., and Gassiat, E. (1996). "Blind deconvolution of discrete linear systems", Annals of Statistics, 24, 1964-1981. [13] Jian, H., and Pawitan, Y. (1999). "Consistent estimation for non-Gaussian non-causal autoregressive processes", J. of Time Series Analysis, 20, 417-423.

20

[14] Kannnan, G,, Milani, A. A., Panahi, I., and Briggs, R. (2008). "Equalizing secondary path effects using the periodicity of fMRI accoustic noise", 30th Annual Intrernational IEEE EMBS Conference, Cancouver, British Columbia, Canada, August 20-24, 2008, 25-28. [15] Lanne, M., and Saikkonen, P. (2009). "Noncausal vector autoregression", Bank of Finland Research, Discussion papapers 18, 2009. [16] Larue, A., Mars, J. I., and Jutten, C. (2006). "Frequency-domain blind deconvolution based on mutual information rate", IEEE Transactions on Signal Processing, 54, 1771-1781. [17] Li, T. H. (1995). "Blind deconvolution of linear systems with multilevel stationary inputs", Annals of Statistics, 23 690-704. [18] Li, T. H., and Lii, K. S. (2002). "A joint estimation approach for tw-tone image deblurring by blind deconvolution", IEEE Tansactions on Image Processing, 11, 847-858. [19] Lii, K. S., Rosenblatt, M., and Van Atta, C. (1976). "Bispectra measurements in turbulance", Part 1, J. Fluid Mechanics, 77, 45-62. [20] Lii, K. S., and Rosenblatt, M. (1982)."Deconvolution and estimation of the transfer function phase and coefficients for non-Gaussian linear processes," Ann. Statist., 10, 11951208. [21] Lii, K. S., and Rosenblatt, M. (1984). "Remarks on nonGaussian linear processes with additive Gaussian noise". Lecture Notes in Statistics, V. 26, 185-197. In "Robust and Nonlinear Time Series Analysis" Ed. by Franke, J., Hardle, W., and Martin, D. [22] Lii, K. S., and Rosenblatt, M. (1985). "A fourth order deconvolution technique for nonGaussian linear processes". In Krishnaiah, P. R. (eds), Multivariate Analysis VI, 395410. [23] Lii, K. S., and Rosenblatt, M. (1988). "Nonminimum phase non-Gaussian deconvolution", J. Multivariate Analysis, 27, 359-374. [24] Lii, K. S., and Rosenblatt, M. (1988)."Estimation and Deconvolution When the Transfer Function Has Zeros," J. Theoretical Probability, 93-113. [25] Lii, K. S., and Rosenblatt, M. (1990)."Cumulant spectral estimates: Bias and covariance," Proc. Third Hungarian Colloquium on Limit Theorems in Probability and Statistics (held at PECS), 365-405. [26] Lii, K. S., and Rosenblatt, M. (1990)."Asymptotic normality of cumulant spectral estimates," J. Theoretical Probability, 3, 367-385. [27] Lii, K. S., and Rosenblatt, M. (1992). "An Approximate Maximum Likelihood Estimation for Non-Gaussian Non-Minimum Phase Moving Average processes", J. Multivariate Analysis, 43, 272-299.

21

[28] Lii, K. S., and Rosenblatt, M. (1993). "Bispectra and phase of nonGaussian linear processes", J. Theoretcal Probability, 6, 579-593. [29] Lii, K. S., and Rosenblatt, M. (1996). "Maximum Likelihood Estimation for NoGaussian NonMinimum Phase ARMA Sequences", Statistica Sinica, 6, 1-22. [30] Lii, K. S., and Rosenblatt, M. (1996)."NonGaussian autoregressive sequences and random fields". In Adler, R. J., Muller, P., and Rozovskii, B. (Eds). Stochastic Modeling in Physical Oceanography, 295-309. [31] Matsuoka, T., and Ulrych, T. J. (1984). "Phase Estimation Using Bispectrum", Proc. IEEE, 72, 1403-1411. [32] Nikias, C. L., and Petropulu, A. P. (1993) Higher-order Spectra Analysis, a Nonlinear Signal Process Framework, Prentice Hall, Englewood Cliffs, NJ. [33] Oppenheim, A. V., and Lim, J. S. (1981). " The importance of phase in Signals," Proc. IEEE, 69, 529-541. [34] Rosenblatt, M., and Van Ness, J. W. (1965). "Estimation of bispectra", Annals of Mathematical Statistics, 36, 1120-1136. [35] Rosenblatt, M. (1980). "Linear processes and Bispectra", J. Appl. Probab., 17, 265-270. [36] Rosenblatt, M. (1985). Stationary Sequences and Random Fields. Birkhauser, Boston. [37] Rosenblatt, M. (2000). Gaussian and Non-Gausian Linear Time Series and Random Fields. Springer, New York. [38] Scargle, J. D. (1981). "Phase-sensitive deconvolution to model random processes with special reference to astronomical data". In Applied Time Series Analysis, II. Ed. by Findly, D. F. 549-564. Academic Press New York. [39] Wiggins, R. A. (1978). "Minimum entropy deconvolution", Geoexploration 16, 21-35. [40] Xia, B., and Zhang, L. (2007). " Blind deconvolution in nonminimum phase systems using cascade structure", EURASIP J. on Advances in Signal Processing, V. 2007, Article ID 48432, 10 pages.

22

Rosenblatt's Contributions to Random Walks on Compact Semigroups By T. C. Sun

Murray Rosenblatt's interest in random walks on compact semigroups probably came from his work on representations of stationary processes as shifts of functions of independent random variables described in [11], where products of matrices were studied. His papers [12],[5],[13],[14] generalized the work of Levy [2] on random walks on the circle and the work of Kawada and ltd [4] on random walks on compact groups. In [14], he also completely characterized the structure of the limit measures for the special case of compact semigrougs of n x n stochastic matrices. To describe Rosenblatt's work in this area in more detail, we first introduce some definitions and basic facts about compact topological semigroups (see [15] and [3]). 1. Every compact topological semigroup S has a minimal two sided ideal K which is called the kernel of S. 2. The kernel if is a compact completely simple semigroup which has a Rees structure 1 x 6 x 7 , where (a) X is a compact left-zero semigroup, Y is a compact right-zero semigroup and G is a compact group. (b) The topology on X x G x Y is the product topology. (c) The binary operation on X x G x Y is given by (x,g,y)(x',g',yf)

=

(x1g<j)(x,1y)g,1y,)1

where c/)(x'', y) : X x Y —» G is a continuous function which, in our case, can be chosen to be (j)(xf,y) = yx'. The identity in each group {(x,g^y\g G G} is (x, (yz) - 1 ,y). 3. The convolution of two regular probability measures v and /i on S is defined by z/*/i(A)= / ii(s~lA)v(ds)=

\

u{s~lA)ii{ds).

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_5, © Springer Science+Business Media, LLC 2011

23

4. Let v be a regular probability measure on S and denote its n with itself by z/ n *\ i.e., l/2*) = ! / * ! / ,

-fold convolution

Z > * } = l / ^ - 1 ) * ) * !/.

5. Suppose Xi1X2l'-' ,Xn are independent and identically distributed random variables with values in S and common distribution v on S. Then the distribution of Wn = Xi + X 2 + • • • + X n is z/ n *), which has support contained in 5 and {W n , n = 1, 2 , . . . } is called a random walk on 5 . In the following we shall always assume that v is a regular probability measure on S and U£° =1 supp(z>*)) = S or S is the compact semigroup generated by the support of z/, because the random walk can never exit the semigroup which is generated by the support of v. In [12], Rosenblatt first showed that the sequence (1/n) Y^=i to a measure fi on S satisfying,

u

^ converges weakly

and where the support of /i is the kernel K of S. So, unlike the case of compact groups where the limit measure is the Haar measure on all of S, here in the case of compact semigroups the probability measure is absorbed into the kernel K of S in the limiting process. In the next step, Heble and Rosenblatt in [5] determined the structure of this limit measure JJLOI±K = XXGXY. They showed that if a regular idempotent probability measure \i on S has a completely simple subsemigruoup K = X x G xY as its support, then // is a product measure and

where x is ^ n e Haar measure of the group G and a, j3 are regular probability measures o n l , 7 respectively. So the components a and j3 of the product measure are the extra elements in the semigroup case. J. S. Pym also obtained this results in [10]. Later in [13], Rosenblatt found a necessary and sufficient condition for the sequence {z>*)} to converge weakly. He showed that if v is a regular probability measure on S whose support generates 5 , then the sequence {z/ n *)} will not converge as n —> oc if and only if there is a proper closed subgroup Q (proper inclusion) of G such that YX C G' and the support of v is contained in

(X x G' xY)~l(X

24

xgG' x F ) ,

where g 0 G' and further, U<j^1g^Gf = G. In the compact group case, Kawada and Ito's result says the sequence z/(n*) will not converge if and only if there is a proper closed normal subgroup G of G such that for some element g ^ G the support of v is contained in the coset gG'. This is because the support of v ^ jumps from one coset of G to another coset of G as n changes. However the sequence (1/n) XT=i u ^ always converges. In a more concrete setting, when the semigroup S is the set of all m x m stochastic matrices, one wonders what a completely simple kernel semigroup K would look like? In [14], Rosenblatt gave an answer about the structure of such a K as the following. 1. There is a partition of integers {1,2, . . . , m } into disjoint classes of integers T, Ci, C2, • •., Cr of numbers n 0 , n i , n 2 , . . . , nr respectively with YJi=o ni = rn a n d T = {1,2, . . . , n 0 } , Cx = {n 0 + 1 , n 0 + 2 , . . . , n 0 + n i } , . . . , Cr = {n 0 + ni + hnr_i + 1,... ,ra}. 2. There are column vectors

a=l

^ = (uf;jGCa),

«f>>0, ^uf

= l,

a=l

l& =

(l;jeCQ

3. H is the permutation group on { 1 , 2 , . . . , r } . Then a completely simple kernel semigroup K in S contains all matrices of the form (for all heH), o(M 7/ M' f 0 pMuW , \ 0 S^lWuW ... <Ji-*rl<W

where 1. the vectors {u^\a

= 1, 2, • • • , r } corresponds to F ,

2. the vectors { p ^ ; a = 1, 2, • • • , r } corresponds to X , 3. i J corresponds to the group G?, 4. the map 0(x, y) maps everything into the identity in G.

25

One may also wonder what the limit measures a and /3 would look like. In [15], in the Notes of Chapter 5, Rosenblatt gave a simple example of a measure v on the semigroup S of 2 x 2 stochastic matrices whose support contains only two points. Each 2 x 2 stochastic matrix fa 1— a \

\b

i-b)

can be identified with a point (a, b). In this setting both the X and G components of the kernel K = X xG xY of S degenerate into one point and Y = {(a, a); a G [0,1]}. So K = Y and \i = /?. With proper choice of the two points in the support of z/, the limit measure / 3 o n 7 can be discrete, singularly continuous or absolutely continuous. In [17], T. Sun generalized this result and, in particular, showed geometrically how this sequence z/n*) of measures is attracted to the kernel K = Y. More examples in this direction can be found in [18] and [3]. To summarize, Rosenblatt's work completely generalizes Kawada and Ito's result from compact groups to compact semigroups. Of course the result on compact semigroups is much more complicated because of the appearance of the two components X and Y in addition to the group component G. However his work on the structure of the kernel K for the semigroup of m x m stochastic matrices at least reveals some mysteries about the 2 components X and Y. Rosenblatt's seminal research in this area inspired a number of related papers. Here we mention just a few. 1. Mukherjea and Tserpes showed in [8] that any idempotent probability measure on a locally compact semigroup must have support which is a completely simple subsemigroup and the measure is a product measure. They generalized Pym, Heble and Rosenblatt's result from compact semigroups to locally compact semigroups. 2. After Rosenblatt's results, it was natural to study recurrent random walks on compact and locally compact semigroups [19], [9]. It is not surprising that any recurrent state must belong to some subgroup and, in fact, in a compact semigroup the set of recurrent states is exactly the kernel of the semigroup. Since a completely simple semigroup is really a union of isomorphic groups, should there be any type of harmonic analysis on a completely simple semigroup? 3. A study of products of random matrices on the semigroup of all m x m matrices with positive entries [6] and on the semigroup of all m x m matrices with real entries is contained in [1], [7]. In [16] Rosenblatt posed the question of determining the structure of a completely simple semigroup in these types of matrix semigroups. It seems that this question remains unanswered. For more recent results on the random walks on semigroups we refer to the excellent reference book [3] by Hognas and Mukherjea.

26

References [1] Bougerol, P., Tightness of products of random matrices and stability of linear stochastic systems, Ann. Prob. 15, 40-74 (1987). [2] Levy, P., L'addition des variables aleatoires definis sur une circonference, Bull. Soc. Math. France, 67, 1-41 (1939). [3] Hognas, G. and Mukherjea, A., Probability Measures on Semigroups, Press (1995).

Plenum

[4] Kawada, Y. and Ito, K., On the probability distribution on a compact group, Proc. PhysrMath. Soc. of Japan, 22 977-998 (1940). [5] Heble, M. and Rosenblatt, M., Idempotent measures on a topological semigroup, Proc. Amer. Math. Soc. 14, 177-184 (1963). [6] Kesten, H. and Spitzer, F., Convergence in distribution of products of random matrices, Z. Wahr. verw. Gebiete 67, 363-383 (1984). [7] Mukherjea, A., Tightness of products of i.i.d. random matrices II, Ann. 22, 2223-2233 (1994).

Prob.

[8] Mukherjea, A. and Tserpes, N., Idempotent measures on Locally compact semigroups, Proc. Am. Math. Soc. 29, 143-150 (1971). [9] Mukherjea, A., Sun, T. and Tserpes, N., Random walks on compact semigroups, Proc. Am. Math. Soc. 39, 599-605 (1973). [10] Pym, J., Idempotent measures on semigroups, Pacific J. Math. 12, 685-698 (1962). [11] Rosenblatt, M., Stationary processes as shifts of functions of independent random variables, Jour, of Math, and Mech, 8, 665-682 (1959). [12] Rosenblatt, M., Limits of convolution sequences of measures on a compact topological group, Jour, of Math, and Mech. 9, 293-395 (1960). [13] Rosenblatt, M., Equicontinuous Markov operators, Theory of Prob. and 9 205-222 (1964).

Appl,

[14] Rosenblatt, M., Products of independent identically distributed stochastic matrices, Jour. Math. Anal. Appl. 1 1 , 1-10 (1965). [15] Rosenblatt, M., Markov Processes. Structure and Asymptotic Verlag (1971).

Behavior, Springer-

[16] Rosenblatt, M., Convolution of sequence of measures on the semigroup of stochastic matrices, 215-220, Contemporary Math. vol. 50, Am. Math. Soc. (1986).

27

[17] Sun, T., Limits of convolutions of probability measures on the set of 2 x 2 stochastic matrices. Bull. Inst. of Math. Acad. Sinica, 3, 235-248 (1975). [18] Sun, T., Random walks on semigroups, Contemporary Soc. (1986).

Math. vol. 50, Am.

Math.

[19] Sun, T., Mukherjea, A. and Tserpes, N., On recurrent random walks on semigroups, Trans. Am. Math. Soc. 185, 213-227 (1973).

28

The Rosenblatt Process* By Murad S. Taqqu

Abstract This is a brief history of the Rosenblatt process, how it came about, the role it played, its properties and a detailed description of its various representations.

1

Introduction

I shall describe here the Rosenblatt process and the history behind it. The Rosenblatt process is important because it is one of the so-called "Hermite processes", which are limits of normalized sums of long-range dependent random variables. The simplest Hermite process is the Gaussian fractional Brownian motion. The Rosenblatt process is the simplest non-Gaussian Hermite process. It has continuous non-differentiable paths and is self-similar with stationary increments. It is Murray Rosenblatt who first conceived of it, and this is why I coined it the Rosenblatt process in [32]. I will start with Rosenblatt's strong mixing condition, quote the corresponding central limit theorem and describe Rosenblatt's two counterexamples. The second counterexample is of interest. It involves a limiting non-Gaussian distribution which is now called the Rosenblatt distribution. This marginal distribution has been expanded in [32] to the finite-dimensional distributions of a stochastic process called the Rosenblatt process. The distributions are not known in closed form and the corresponding characteristic functions are expressed in terms an expansion involving cumulants. There are, however, a number of integral represen tations which clarify the nature of the Rosenblatt process and which I will describe in the sequel. These include the time, spectral and finite time interval representations. There is also a wavelet representation due to Vladas Pipiras which extends to the Rosenblatt process the wavelet representation of fractional Brownian motion in [20]. I shall not describe this representation here but will direct the reader to [26], and in par ticular, to the first two sections which explain heuristically the ideas behind the expansion. This expansion is implemented in [1]. I will also go over some properties of the Rosenblatt process. The paper of Ciprian Tudor [35] provides additional information, in particular, about the stochastic calculus associated with the Rosenblatt process. The paper is organized as follows. The strong mixing condition and Rosenblatt's corresponding central limit theorem are in introduced in Section 2. The second counterexample, mentioned above, is described in Section 3. The joint characteristic function of the Rosenblatt process and the associated cumulants are given in Section 4. The process is represented through Wiener-Ito stochastic integrals and these are introduced in Section 5. The corresponding stochastic integral representations for fractional Brownian motion and for the

*This work was partially supported by the NSF grant DMS-0706786 at Boston University.

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_6, © Springer Science+Business Media, LLC 2011

29

Rosenblatt process are discussed in Section 6 and Section 7 respectively. Section 8 describes some properties of the Rosenblatt processes and an Appendix provides details about the derivation of the characteristic function.

2

Strong mixing

The classical central limit theorem (CLT) states that the distribution of suitably normalized sums of in dependent and identically distributed random variables converges to the normal distribution. This result continues to hold if there is some dependence, for example, if the random sequence is m-dependent, that is, if random variables are independent when separated by m time lags. What if we increase the range of the dependence? More generally, will there be a natural boundary, that is, a dependence structure beyond which the CLT will not hold. This is unlikely to be the case as the notion of dependence is not linearly ordered. Nevertheless, Murray Rosenblatt wanted to pursue A. Markov's idea "that one expects a central limit theorem to hold for X\, X2,... if the random variables behave more like independent random variables the further they are separated (assuming appropriate moments exist)". Rosenblatt focused on the following condition, known as the strong mixing condition, which he expresses in the following way in [29]. Definition 2.1 (Strong m i x i n g condition) Consider a strictly stationary sequence {X^}. Let A and B be any events determined by conditions on the random variables Xk, k < 0 and Xk, k > n, respectively with n > 0. The process {Xk} is said to satisfy the strong mixing condition if sup \P(A f\B)-

P(A)P(B)\

A,B

< d(n),

where the sup is over all such events A and B, and where d is a function decreases to zero as n —> 00.

(1) of the positive integers n that

Since that time, research on mixing conditions has been extensive. See for example the surveys of Magda Peligrad [25] and Richard Bradley [7], [8], and the three volumes Bradley [9, 10, 11]. For the CLT to hold, often mixing conditions need to be accompanied by moments conditions. Typically, the slower d(n) —► 0 as n —> 00, the more moments should exist. Rosenblatt in [30] states the following central limit theorem for strictly stationary random variables. It is proved in [29] without the assumption of stationarity x . T h e o r e m 2.2 (CLT for strong mixing sequences) Suppose {Xj} is a strictly stationary sequence with mean zero satisfying the strong mixing condition and let Sn = Y^j=\ Xj. Suppose also that the following two moment conditions: E 5 ^ = crj; —>• 00

ES*=0(a*)

as

as

n —>> 00,

n -► 00

(2)

(3)

hold. Then as n —> 00 ; ^

i

iV(0,1)

(4)

In this theorem, there are no assumptions on the speed at which d(n), defined in (1), tends to zero, but it is required that all moments up to order 4 exist. The following stronger result sheds light on Theorem 2.2 (see Denker [13], Theorem 3). 1 In [29], the relation corresponding to (3) below is stated as o(afi) but the proof holds with the weaker condition 0(afi) stated in [6].

30

as

T h e o r e m 2.3 Suppose {Xj} is a strictly stationary sequence with mean zero satisfying the strong mixing condition and condition (2). Then the CLT (4) holds if and only if S2 —^

<

is uniformly integrable.

(5)

Thus o~~xSn tends to a N(0,1) distribution if, for example, E | £ n | 2 + 5 = 0{a2^5) for some 5 > 0. Rosenblatt's Theorem 2.2 is stated with 8 = 2. For the history behind Theorem 2.3, see [9]. To get an idea of the limitations of Theorem 2.2, Rosenblatt provides two counterexamples in [30]. The first is Xk = €k — €fc_i, where the ek are i.i.d., non-Gaussian random variables, with zero mean and finite fourth order moment. Then, a^ = E| Y17=i ^i\2 = E|e n — eo| 2 = 2ECQ does not tend to infinity with n. While the strong mixing condition and condition (3) are satisfied, condition (2) fails, and in fact, the conclusion (4) fails as well.

3

Rosenblatt's counterexample

The second counterexample is more intriguing and is the basis of later developments which gave rise to the Rosenblatt process. Let {Yk} be a Gaussian sequence with mean zero, unit variance and covariance rk =EY0Yk

= (l + k2)~D/2

~k~D

as

fc^oo

with 0 < D < 1/2. Consider Xk = Y2 - 1. and choose a > 0. Then, n Z

« = ^DY.X*

(6)

fc = l

tends to a non-Gaussian distribution as n —> oo. Since the Xk are dependent chi-squared random variables, the characteristic function of Zn can be readily written and one can then let n tend to infinity. The detailed computation is instructive and can be found in the Appendix. Rosenblatt shows in [30] that the sequence Zn converges in distribution to a random variable Z(l) whose characteristic function is

V^)=exp(if>^fV J 2^ v k

(7)

k=2

where ck=

dx!... Jo

Jo

dxk\x1-x2\

D

\x2-x3\

D

. . . | x f c _ i -xk\

D

\xk-x1\

D

.

In this example, both conditions (2) and (3) are satisfied. Since the conclusion of the theorem is not the sequence {Xk} cannot be strong mixing. Since Xk = Y2 — 1, the cr-fields generated by {Xkl and {Xk, k > n} are respectively smaller than the cr-fields generated by {Yk, k < m} and {Yk, Therefore {Xk} not strong mixing implies that {Yk} is not strong mixing either. Rosenblatt, in was able to characterize a non-trivial, non-strongly mixing Gaussian sequence. The parameter a is arbitrary. As we will see below, if one sets

(8)

satisfied, k < m} k > n). this way,

1/2

\{l-2D)(l-D)

31

(9)

the limit of Zn has unit variance. In fact, (7) is an expansion in cumulants. Indeed, if we set K

lnp(0) = £ ( « \k

k

k=2

k=2 we can identify the cumulants as

2k-1(k-l)\<7kck.

Kk =

With a as in (9), the variance, which equals the second cumulant, becomes ft2=2c2cr

ri ri 2 = 2a* I I \xi - x2\ -2D dx\dx2 = 2a1 (1-2D)(2-2D) Jo Jo

= 1,

so that the distribution is standardized. I have been intrigued by this counterexample. What is so special about the Xk = Yk2 — 1? Why is the usual central limit theorem failing? Why was D taken to about weak convergence to a stochastic process? The answers turn out to be very interesting. The transformation Xk = Yk2 — 1 it involves in fact the Hermite polynomial of order 2, #2(2/) — U2 ~ 1- The Hermite H0(v) = l,

H1(y) = y,

H2(y)=y2-1,

H3(y) = y3 -

2

square transformation satisfy D < 1/2? What is not an arbitrary one: polynomials

3y,...

span the L space generated by the Gaussian sequence. This space contains any non-linear function where Y - N(0,1) with EG2(Y)

G(Y),

e—2~ G2(y)^=dy<(K.

:

2TT

One is then led to consider not only Xk = Y2 — 1 = H2(Yk), but also Xk = Hm(Yk) for m > 2, and more generally, Xk = G(Yk) with EG2(Yk) < 00. This is done in Taqqu, [32], [33], Dobrushin and Major [14], Major [16]. What about the condition on Dl Observe that

E^ = E(! +

fe2 -D/2

1 1-D

The condition Y^kLi Tk = °° ^s n o w known as "long-range dependence", "strong dependence" or "long memory". Since 0 < D < 1/2, the {Yk} above are strongly dependent ( t h e y would be so even if 0 < D < 1). What about the {X&}? One nice property of the Hermite polynomials is their orthogonality. In fact, EHmi(Y0)Hm2(Yk)

E(Y0Yk)mi6mi,n

= rrnl

In particular, lXQXk

= 2{EY0Yk)2

= 2r2.

Since 0 < D < 1/2,

J>XoX f c = 2 ^ ( l + /c2

1-2D

and

EK>

n-1 j \

((l-2D)(l-D))

i=i fc=i

32

(10)

which explains the normalization factor in (6) and the value of a in (9). Observe that {Xk} has a weaker dependence than {Yfc} since r\ tends faster to 0 than r& as k —> oo. Nevertheless, the condition 0 < D < 1/2 ensures that {Xk} remains long-range dependent, that is, one has not only YlT=i ^^o^fc = ° ° ^ u t a ^ s o ^2T=i ^XQX^ = oo. This would not be the case anymore if D were t o be greater than 1/2.

4

The joint characteristic function of the Rosenblatt process

What if one considered not random variables Zn as defined in (6), but a stochastic process \nt\

where |_-J denotes the integer part. Observe that our previous Zn is Zn(l). We shall then choose a as in (9) so that lim^^oo KZn(l)2 = 1. The limiting characteristic function of the vector (Zn(ti),..., Zn(tp)) is given in Taqqu [32]. Denoting it as the characteristic function of the limiting vector ( Z ( t i ) , . . . , Z(tn))1 it is 2 ip(Ou . . . , dp) =

^eWiZ^)+-+0pZW)

= e x

P U E ^ V

E

k

=2

esl...eSksD{tslJ...1tSk)Y

si,..., Sfe e{i,...,p}

(12) /

where rtSl

SD(tSl,...,t8k)=

/ Jo

rtS2

dxi I Jo

rtSk

dx2...

Jo

dxk\xi-x2\~D\x2-x3\~D

...\xk-i-Xk\~D\xk-x1\~D. (13)

As noted in [32], since we have by the Cauchy-Schwarz inequality, (

l

SD
1 1/2 f

i

i

"1

\j^dy1->.j^dyk-1\y1\-2D---\yk-1\-2D\

1/2

f

=\ [

i

1

k/2

\y\-2Ddy\

then for \6i\ < e, i = 1, • • • ,p, k

(2aCpe) \^,...,ep)\<e,P(lj:^L).

(14)

Thus the series in (12) converges for small values of e, and this is enough to characterize the distribution (see [5], Theorem 30.1). It is also convenient to consider the cumulants Kk of the random variable "52? 0iZ(ti). These are K,I = 0 and nk = 2k-\k-l)\ak Yl 6Sl...6SkSD(tSl,...,tSk), k>2, (15) si,...,sfcG{l,...,p}

and they characterize the finite-dimensional distribution of the vector (Z(ti),..., Z(tp)) uniquely. In fact, the sequence Zn(t) converges t o Z(t), not only in terms of the finite-dimensional distributions, but also as weak convergence of the probability measures in the space D[0,1] endowed with the Skorohod topology (see [33]). In [32], the limiting process Z(t) was called the Rosenblatt process and this name has been used since then. 2

Replace the expression in [32] by the one given here.

33

5

Wiener-Ito stochastic integrals

The Rosenblatt process can be represented in terms of Wiener-Ito stochastic integrals. We shall now introduce them. Let {B(t), t e M.} be a standard Brownian motion with B(0) = 0. A Wiener-Ito stochastic integral of order m is an integral of the form ,(/)= /

(16)

f(x1,...,xrn)dB(x1)...dB(xm),

where / is a non-random kernel and dB(x) can be interpreted as Gaussian noise with mean 0 and variance dx with EdB(xi)dB(x2) = 0 if x\ ^ x^ and E(dB(x))2 = dx. The prime indicates that one does not integrate over the diagonals, that is, one always supposes x\ ^ X2 ^ • • • ^ xm. This ensures that EdB(xi)... dB(xm) = 0 and thus that Im(f) has mean 0. Moreover, Im{f)=Im{fsym}

(17)

where fsyrn denotes the symmetrized version of / and where ' = ' means equality in distribution (either of the marginal or joint distributions, depending on the context). The integral (16) is defined for all / satisfying \f(x1,...,xm)\2dx1...dxm

II/IIL2(R-) = /

that is, for all / G L 2 ( R m ) . To define the integral precisely, one first defines it for simple functions. Then one takes a limit in L 2 ( R m ) , while ensuring that the isometry E(/m(/))2=m!||/|||2(Rro) holds. In the case m = 2, the distribution of 12(f) is determined by its cumulants. If / is symmetric, these are Kk(f)

= 2k

1

(k-l)\

/

f(xllX2)f(x2,x3)...f(xk-llxk)f(xk,x1)dx1...dxk

JRk

(18)

(see [15]). Circular integrals of this type appear in (8) and (13). See [24] for more information about multiple integrals including formulas for moments and cumulants. It is also possible to view the integral (16) from the perspective of Parseval's formula, and write Im(f)=

f

/(Ai,...,Am)d5(Ai)...dB(Am),

(19)

where /(Ai,...,Am) =

/

eiT
,xm)dx1...

dxm

is the Fourier transform of / in L 2 ( R m ) and dB(X) is viewed as the complex-valued Fourier transform of dB(x). Since dB(x) is real, dB(X) will satisfy dB(\) = dB(—X) and thus, in the integral (19), the double prime on fRm means that one excludes integration of Ai = ±A2 = • • • = ± A m . This is made precise in [33] where the equality in distribution between (16) and (19) is established as well as the relation mm{f)2=m\

[

|/~(A1,...,Am)|2dA1...rfAm
If one interprets the x's in (16) as "time" and the A's in (19) as "frequency", then one has two representations for J m ( J ) , one in time, and the other in frequency.

34

6

Stochastic representations of fractional Brownian motion

To better understand the representations of the Rosenblatt process which involve double integrals (ra = 2) in (16) and (19), it is best to describe first the representation of fractional Brownian motion, since it involves only a single integral (ra = 1). Fractional Brownian motion {£#(£), t > 0} is characterized by the following properties: • It is Gaussian with mean zero. • It has stationary increments with BH(0) = 0. • It is self-similar with index H G (0,1), that is for any a > 0, {5jf(ot), t > 0} and {aHBn(t), have the same finite-dimensional distributions.

t > 0}

For more details, see for example [34]. These three characterizing conditions are very handy because in order to check that a given single Wiener-Ito stochastic integral (ra = 1) represents fractional Brownian motion, it is sufficient to verify that the three conditions mentioned above are satisfied. Self-similarity and stationary increments imply moreover that EB%(t) = a2t2H

(20) 2

EBnMBHfo)

= RH(hM) = y {if + if - \tx - t2\2H} .

This process is "standardized" if a2 = 1. If one sets Yk = BH{k + l)-BH{k),

(21)

/c>0,

then {Y&} is Gaussian, stationary and has long-range dependence if 1/2 < H < 1 because rk = EY0Yk - a2H{2H

l)k2H~2,

-

and hence oo

y^rk = oo. fc=i

Observe that D = 2 — 2H £ (0,1). The increments {Yk} are called fractional

[34]).

6.1

Gaussian noise (see [31] and

T i m e r e p r e s e n t a t i o n of fractional Brownian motion

The following time representation of fractional Brownian motion is the most familiar one:

BH(t) = C\(H) f ((t - x)l~112 - (-x)^-1/2)dB(x), V J JR

t > 0,

(22)

where 3

C (H) V ^^^ + ^ S l n ( ^ ) ) l / 2 U

3

j

'

T(H+l/2)

J

Sln

(^ g )

\/3{H + 1/2, H +1/2)

1 V2

f

j

\ (H - l/2)j3(H

The constant C\(H) here is the inverse of the constant C\(H) in [31] and [34].

35

2

JL

"l/2

- i / ^ , ^ - z,n} j

(23)

ensures that EBJj(t) = t2H, that is, a2 = 1. For the value of the constant Ci(H), see relation (9.3) in [34], Appendix A in [21] and also (25) below. Here [3(a b) = /

Jo

xa-1(l-x)b-1dx=

/

Jo

x a - 1 ( l + ^ ) - a - 6 d x = r ( a ) r ( 6 ) / r ( a + 6), a , 6 > 0 ,

denotes the Beta function. Since there is a single integral in (22), the process BH(t) is Gaussian. Moreover, the form of the integrand is (22) also ensures that the process has stationary increments. The self-similarity follows from the fact that the integrand scales and so does dB{x) because dB(ax) = a}'2dB(x). Thus the representation (22) is that of fractional Brownian motion, but it is far from unique and different time representations are possible. If 1/2 < H < 1, one can express the kernel in (22) as an integral and obtain BH{t) = d ( t f )(ff - 1/2) / (j\s

- x)^-3/2ds\

dB(x).

(24)

This is a useful expression. Firstly, with (41) below, it can be used to verify that EBH(l)2

= d(H)2(H =

- 1/2) 2 /

/ (si - x) + " 3 / 2 ( 5 2 -

Jo Jo

d(H)2(H

- 1/2)2H~1(2H

x)*-3/2dSlds2dx

- l ) - 1 / 3 ( # - 1/2,2 - 2H) = 1,

(25)

that is, the constant C\(H) ensures standardization. Secondly (24) suggests a heuristic way of representing fractional Brownian motion. Although it is strictly speaking incorrect, it can be made precise by using "generalized functions". The idea is as follows. Since Bn(t) has stationary increments, let us represent it as the integral of its "derivative". To do this, suppose 1/2 < H < 1 and interchange the integral signs in (24), writing BH(t)u

= "Ci(tf)(tf - 1/2) I ( f(s- x)H-V2dB(x))

ds

= d ( H ) ( # - l / 2 ) / B'H{s)ds.

(26) (27)

Jo

The change of order of integration in (26) is not justified and neither is the derivative B'H(t) well-defined because the integrand (s — x) + ~ ' in (26) is not square integrable. Nevertheless (26) and (27) provide a physical interpretation of the time representation (22) and a better understanding of the corresponding representation for the Rosenblatt process.

6.2

Spectral representation of fractional Brownian motion

The spectral representation 4 for Bn(t) is BH{t)±C2(H)l^wL^dB(X)

where 5 21 4 5

_ f

>'

H(2H - 1)

\2r(2-2H)sm[(H-l/2)ir]

1 1/2 _

(tfr(2tf)sintf^ 1 / 2

In [31], the spectral representation is called the harmonizable representation. The constant CQ,(H) here is the inverse of the constant C2(-£/") in [31] and [34].

36

(28)

(see [33], Theorem 6.3 and [34], relation (9.8)). The term (etXt — l)/(zA) is the Fourier transform of l[o,t]? yielding stationary increments and the term c(A)|A| _ ^- ff_1 / 2 )|, with |c(A)| = 1, can be viewed heuristically and up to a multiplicative constant as the Fourier transform of the kernel s+ in (26) (see the proof of Lemma 6.2 in [33]). The equivalence between the time representation (22) and the spectral representation (28) follows from the discussion in Section 5.

6.3

Finite time interval representation of fractional Brownian motion

There is a third representation for fractional Brownian motion which involves only an integral from 0 to t. It is related to the so-called Molchan martingale and can be found in [23] and [27] where it is used for prediction. See Molchan [22] for a brief history and a discussion of the prediction problem. See also Mishura [21] who treats the case 0 < H < 1/2 as well. We suppose here 1/2 < H < 1. The representation is:

L / KH (t,x)dB(x), Jo

BH(t)

(29)

where H

(t, x) = C3(H)x^2-H = C3(H)x^2-H

f\u

- x)H-^2uH-^2du,

0 < x < t,

f\u

- x)l-Z/2uH-^2du,

0 < x < t,

Jo

and H(2H - 1) /3{2-2H,H - 1/2)

C3(H) :=

(30)

1/2

(31)

(see [35]). Since KH(t,x) we can express (29) as

=KH(t1x)

-KH{x,x)

= / ——(u,x)du, Jx

B it)l

»l

rt

r

rt

if.

0<x
^^

dKL

(32) (u,x)du dB(x), Jo Ux 9u a form which will be handy when we will consider the corresponding extension to the Rosenblatt process. Observe that we are proceeding as in (24) by involving the derivative of the original kernel. Here dKH

/x\1/2~H

^-(u,x)

M 1/9

(u-x)f-3/2,

= C3(H){^)

so that one has

BH(t)±C3(H)

f\f'[-)■ JO Ux

1/2-H Ku/

(33)

0<x
(u-x)%H-3/2

(34)

du dB(x).

The self-similarity follows from KH(at,

ax) = aH~1/2KH(t,

x) for any a > 0.

(35)

The stationarity of the increments is not obvious, and since the process is Gaussian, it is best to compute the covariance directly. The relation (uv)1*-1/2 0{2-2H,H-l/2)Jo

(u — x)+

37

(v — x)+

/

x1

2H

dx

(36)

for u,v > 0 is very handy (see (3.4) in [27]), because it shows that EBH^BH^)

=

/•tlAt 2 / KH(tux)KH(t2,x)dx

Jo

given in (21). Hence, the choice of C^,{H) does ensure that E i ^ ( l ) = 1.

7

Stochastic representations of the Rosenblatt Process

The Rosenblatt process {Z(t), t > 0} is if-self-similar with 1/2 < H < 1, has zero mean and has stationary increments. It has therefore the same second order properties as fractional Brownian motion, namely, EZ2(t)

= a2t2H,

¥.Z(h)Z(t2)

= RH(h,t2)

= ^ {t\H + t2H - \h - t2\2H) ■

It is, however, non-Gaussian and is expressed as a Wiener-Ito stochastic integral of order m = 2. R e m a r k . It is important to distinguish between the self-similarity index of fractional Brownian motion which we now call HQ (instead of H), and the self-similarity index of the Rosenblatt process which we will denote by H.

7.1

Time representation of the Rosenblatt process

The time representation of the Rosenblatt process is Z(t)

=

ai{HQ)

f ( f ( pf

=

A^H)

/

I

s

ft

-

^ i ) + ° - 3 / 2 ( S - x2)f°-3/2ds) 2—H

2—H

dB{x1)dB{x2)

(37)

\

(s-x1)~~(s-X2)+~ds)dB(x1)dB(x2),

(38)

where 01 (

^o) '

V(ffo-l/2)(4g0-3) P(H0-1/2,2-2H0)

y/(ff/2)(2g-l) P(H/2,1-H)

. ' M{H)

(39)

ensures that KZ2(1) = 1 (see relation (1.6) together with (6.1) in [33]6 ). I have expressed the time representation in two ways: one, involving iio to indicate the analogy with the corresponding representation of fractional Brownian motion in (24), the other involving ii", the self-similarity index of the Rosenblatt process. The relation between HQ and H is Ho = ^

(40)

obtained by comparing the exponent iio — 3/2 in (24) with the exponent —(2 — H)/2 in (38). Observe also that 1/2 < H < 1 ^ ^ 3/4 < i i 0 < 1. 6 One sometimes replaces the integral J R 2 by the integral J \ normalization constants.

38

-, which is smaller by a factor of 2!. This affects the

Moreover, the relation between the H in (38) and the D in (6) and (11) is H =

l-D.

The self-similarity with index H and the stationarity of the increments can be readily verified. To check that the process in (38) represents the Rosenblatt process, it is sufficient to show that the cumulants of Y^QiZ^i) are given by (15). This is easily done using (18) and the relation

/

(u - x)l~\v

- xY^dx

= (3(a, 1 - 2a)\u - v ^ '

1

(41)

Jm

valid for 0 < a < 1/2. Interchanging the integrals in (37), we could write heuristically, Z(t) = ai{H0)J

(l'2(s-x1)f'-3/2(s-x2)f'-3/2dB(x1)dB(x2))ds.

(42)

This is not well-defined because the function fs{x\,X2) = (s ~ xi)+ 2 (s — x^) ~ is not in L 2 (R 2 ). If we ignore this fact and also suppress the prime in JR2 which would have ensured mean 0 (if the process were well-defined), then we could view the Rosenblatt process as Z(t)=ai(H0)

f {B'Ho(s)fds Jo

(43)

where B'H denotes the "derivative" of fractional Brownian motion with self-similarity index given by (40).

7.2

Spectral representation of the Rosenblatt process

The corresponding spectral representation of the Rosenblatt process is

= MH) L tr+l") ' | A ^ | A 2 ; g / 2 ^^(A 2 ),

(45)

where ,H a2{Ho

>

,

I -

(2ffo-l)(4ff0-3)

/

H(2H-i)

" V 2[2r(l - f f ) s m ( ^ / 2 ) ] 2 " '

V 2[2T(2-2H0)sm(MH0-l/2W

M

'

(

6)

with (40) ensures that EZ 2 (1) = 1 (see [33], Theorem 6.3 and Section 5). Compare this spectral represen tation with the one of fractional Brownian motion in (28) and note that in view of (40), the powers of Aj, i = l , 2 are H/2 = H0 — 1/2. Thus one has in the spectral domain, a structure consistent with the heuristic interpretation (43).

7.3

Finite time interval representation of the Rosenblatt process

Let us now turn to the third representation involving integration over the finite range [0, t] only. We have seen that fractional Brownian motion has the representation (32). The corresponding representation of the

39

Rosenblatt process is Z(t)

J[0,t}

dKH° dKH° -(u,xi)—-—{u,X2)du du \JXl\ , du

2

f (f

A3(H)

oK 2 du

(lt,Xi)

I

(47)

dB{xi)dB(x2)

oK 2 (u,X2)du ] du

dB(xi)dB{x2),

(48)

J[o,t}2 \JXlv where KH° is defined in (30), H0 = (H + l ) / 2 as in (40) and where 1 2H0

a3(H0) :=

/ 2 ( 4 ^ 0 - 3) 2H0-1

1 H + l

l2(2H-l) --■• A3(H) H

(49)

with (40) ensures that KZ2(1) = 1 (see [35]). The self-similarity follows from (35) which yields

<9w

-(au,ax)

= a °~

7

Q,

K^),

<9w

and hence one gets 2(if 0 — 3/2) + 1 + 1/2 + 1/2 = 2H0 — 1 = H by applying (40). Again, the stationarity of the increments is not obvious. But it will follow by showing that Z(t), as defined in (47), has the same distribution as the Rosenblatt process defined through the time representation (22). We will show in fact that they share the same cumulants. But let us compute first the covariance of Z(t). Observe that dKH° ' dKH° (u,x) (v,x)dx du dv

UAV

Q

/a^l/2-Ho

«>■!"(:

C3(H0)2(3(2

/xxl/2-Ho

H0{2H0-l)\u-v\

-

xfl2~Hodx

v\2H°~2

- 2H0l H0 - l/2)\u 2H

(u - x)3_/2-Ho(v

°-21

(50)

by (31) and (36). Hence the covariance equals EZ(ti)Z(* 2 ) :

2(a3(H0))2

/

Jo pti

pt2

dx! /

JO r^2

= 2{H0(2H0 - l)]2(a3(H0))2 = H(2H -1) =

f1 Jo

du f2 Jo

dKH0

r pt

dx2\ r

du

Ux1 uAv ru

/ Jo dv\u -

dKH°

du / Jo

dKH0

(u,xi)

dv\u -

du

dKH0

t2

(u,X2)du

/

dKH° . . dKHo . dv (v,£i) — dv— ( v 1 X 2 ) d u

(v,x)dx

v\H°~4

v\2H~2

Rlf(ti,t2),

by (47), (50) and (49). Thus, Z(t), as defined in (47), has the same covariance as standard fractional Brownian motion, and in particular, we have EZ2(t) = 1. Moreover, the cumulants of X]f=i 0iZ(ti) are the same as the cumulants of the corresponding time representation (22), namely they are given by (15). To see that this is the case, apply again (18) and the relation (50), using computations similar to those performed in the computation of the covariance.

40

The Rosenblatt process is not different iable, but as for fractional Brownian motion, one could view it heuristically as the integral of its derivative. One merely has to interchange the integrals in (47) and write Z(t)±a3(H0)

8

-^(u,x1)^^(u,x2)dB(x1)dB(x2) du. du

\ JO

\J[0,u]2

°U

(51)

Some properties of the Rosenblatt process

The equalities in the previous sections hold in the sense of finite-dimensional distribution. Rosenblatt process {Z(t), t > 0} has an almost sure continuous path version since

nz(h)-z{t2)\2

In fact, the

= \h-t2\2H

with 2H > 1 and thus Kolmogorov's criterion holds (see [4], Theorem 12.4). But the Rosenblatt process is not differentiable in mean square since h m E f ^ V t->o \ t J

= nin^t^o

2

= oc.

In fact, it is almost surely non differentiable. The argument is based on self-similarity and is identical to the one for fractional Brownian motion (see [19] and Proposition 1.7.1 in [3]). We have seen that the Rosenblatt process is self-similar with index 1/2 < H < 1 and has stationary increments. Therefore its increments AZ(k)

= Z(k + 1) - Z(k),

k > 0

have long-range dependence, that is, ^E[(AZ(0))AZ(fc)] =oo. k=0

In fact, they have the exact same covariances as the increments of fractional Brownian motion, namely 2

E[(AZ(0))AZ(fc)] = °-\\k

+ l| 2 - 2\k\2H + |ife - l | 2 } - a2H(2H

-

l)k2H~2,

as k —> oo, and where 1/2 < H < 1. In Section 7, we presented several representations of the Rosenblatt process. While representations are very physical, they do not provide immediate information about the distributions. However, since the Rosenblatt process is represented as a double Wiener-Ito stochastic integral (ra = 2), one can use a general result due to Major [17] on probability bounds for double integrals. Namely, there is a universal constant C > 0 such that P[Z(l)>w]
u>0,

(52)

where a2 = EZ(1) 2 . The result is proved in [18]. Observe, for example, that if Y is N(0,1), then P[Y2 > u] = P[\Y\ > uxl2\ which is consistent with (52). Albin [2] focuses on extreme values. One has clearly P[sup t G r 0 1 i Z(t) > u] > P[Z(1) > u], but Albin also shows that p su [ P t e [ o , i ] ^ W >u] r h m s u p —7-—„. /„*.—— < oo. u^oo u^-H)/HV[Z{l) >u]

41

Observe that the factor u^~H^H tends to infinity with u. Observe also that the exponent (1 — H)/H decreases from 1 to 0 as H increases from 1/2 to 1. Hence the greater H, the greater the similarity between P[sup £G r 01 i Z(t) > u] and P[Z(1) > u] for large values of u. The marginal distribution of Z(t) has a probability density function. This follows from a general result on multiple integrals (see for example [12]). We also remarked in Section 7, that the Rosenblatt process can be viewed heuristically as the integral of its derivative. However, as noted in Tudor [35], one can use this point of view to show that the Rosenblatt process can be approximated by a sequence of semi-martingales. Indeed, define Z^ (t) by replacing each KH°(u,x) in (51) by KH°(u + e,x) where e > 0, that is, C1 \ C' 8KH° / / ——( u 2 JO \J\0,u} °U

A zU(t)±a3(H0)

r)KH° + e,x1)-^—(uU °

e,X2)dB(x1)dB(x2) du.

+

by using (17). We then observe that Z^e\t)

We can express the integral L, ,2 as 2 Jx

(53)

is the integral

e

over [0,£] of an adapted process and hence Z^ \t) is a semi-martingale. Since K\Z^(t) — Z(t)\2 —>■ 0 as e —► 0, one obtains that the Rosenblatt process can be approximated by a sequence of semi-martingales. Because Z^ (t) is well-defined as the integral of its derivative, the semi-martingales are in fact processes with bounded variation.

9

Appendix

We provide here a proof of Rosenblatt's result, that the random sequence Z n , n > 1 in (6) converges in distribution to a random variable Z(l) with characteristic function given by (7) and (8). Let dn = n 1 _ D and let Rn denote the covariance matrix of the Gaussian vector ( Y i , . . . ,Yn). Each component has mean 0 and unit variance. Let y' = ( ? / i , . . . , yn) denote the row vector and | • | a determinant. Then the characteristic function of Zn is ®ei9Zn

n

r

= /

n

i

exp {iOad-1 $ > ?

l)}-===e-^'R^y^y V^\Rn\

~

3= 1

expi-iOad-'n}

J—^ ^2n\Rn\ 1

JRn 2

exp {-iOcrd^n}

li^l" / ^"

exp {-idad^n}

\I -

■ exp^jy-^ad3=1 1 ■ exp

1

exp f -^[R'1

f 1

I

- 2i6(rd-1I\y}

z

dny J

1

1 2

- 2z0crd" J I " /

2i6ad-1Rn\-1/2

- ln(l - 2i0ad-1Xjin)]} n

i=l

where the Aj>n, j = 1 , . . . , n denote the eigenvalues of Rn. Let Tr denote the trace. Expanding the logarithm, we get

Hi-™od-%,n) =

42

-±{™°dlX^\

and hence n

- J2 M l - 2i0<7d"1 A;,n) = ^ d "

n

1

n oo M . / ,

^ Ai>n + J2 E

j= l

j=l A

Since i? n has 1 in its diagonals, Y^j=\

J > = n,

an

, - n \fc

fe

j = l fc=2

d thus

■lit'-

Ee^=eW{^^dnkf^,n\, (

(54)

k=2

where n

d-kJ2Xln=d-kTr(Rkn) n

= dnk

r

E

d ^ ~i<2^

r

(\i<2 -h\)---r{\ik-i

-ik\)

r{\ik - h\) -► ck

«l,«2,---j*fc =

as n —► oo. The expansion (54) converges absolutely for |0| < e, e small enough, as in (14). This proves (7) and (8). □ Observe that the k — 1 term, d" 1 J^?=i ^j> = d~lrTi{Rn) — nD~ln — nD —> oo, but this term has been canceled and creates no trouble. The k = 2 term provides the variance and hence the normalization dn. Observe also that the expression (7) for the characteristic function is not valid for all values of 6 G R, but only for small values of 9. This, however, is enough to characterize the distribution. This also suggests the following representation for the limiting random variable Z ( l ) (see [28]). Let Cj, 3' > 1 be i.i.d. N(0,1) and let Aj, j > 1 be such that

E A * = c*>

(55)

for k > 2, where c& is given by (8). Then one has oo

Z O L ^ ^ A ^ - l ) , i=i

(56)

because they both have the same characteristic function (7). Moreover,

Var a

oo

A

2

oo

{ E i ( s - ! ) } = J2X2J 2(J2

= 2(j2c2

< °°'

and therefore the sum in (56) converges almost surely. A c k n o w l e d g m e n t s . I would like to thank Magda Peligrad for comments on strong mixing and also Vladas Pipiras and Mark Veillette for interesting discussions.

43

References P. Abry and V. Pipiras. Wavelet-based synthesis of the Rosenblatt process. Signal processing, 86:23262339, 2006. J. M. P. Albin. On extremal theory for self-similar processes. The Annals of Probability, 26(2):743-793, 1998. F. Biagini, Y. Hu, B. 0ksendal, and T. Zhang. Stochastic calculus for fractional Brownian motion and applications. Probability and its Applications. Springer-Verlag, London, 2008. P. Billingsley. Convergence of Probability Measures. Wiley, New York, 1968. P. Billingsley. Probability and Measure. Wiley, New York, 3rd edition, 1995. J.R. Blum and M. Rosenblatt. A class of stationary processes and a central limit theorem. Proceedings of the National Academy of Sciences USA, 42:412-413, 1956. R. C. Bradley. Basic properties of strong mixing conditions. In E. Eberlein and M. S. Taqqu, edi tors, Dependence in Probability and Statistics, Progress in Probability and Statistics, pages 165-192. Birkhauser, 1986. R.C. Bradley. Basic properties of strong mixing conditions. A survey and some open questions. Proba bility Surveys, 2:107-144 (electronic), 2005. Update of, and a supplement to, the 1986 original. R.C. Bradley. Introduction

to strong mixing conditions.

Vol. 1. Kendrick Press, Heber City, UT, 2007.

R.C. Bradley. Introduction

to strong mixing conditions.

Vol. 2. Kendrick Press, Heber City, UT, 2007.

R.C. Bradley. Introduction

to strong mixing conditions.

Vol. 3. Kendrick Press, Heber City, UT, 2007.

Yu. A. Davydov. Distributions of multiple Wiener-Ito integrals. Theory of Probability and its Applica tions, 35(l):51-62, 1990. M. Denker. Uniform integrability and the central limit theorem for strongly mixing processes. In Ernst Eberleein and Murad S. Taqqu, editors, Dependence in probability and statistics, volume 11 of Progress in Probability and Statistics, pages 269-289. Birkhauser, Boston, MA, 1986. R. L. Dobrushin and P. Major. Non-central limit theorems for non-linear functions of Gaussian fields. Zeitschrift fur Wahrscheinlichkeitstheorie und verwandte Gebiete, 50:27-52, 1979. R. Fox and M. S. Taqqu. Central limit theorems for quadratic forms in random variables having longrange dependence. Probability Theory and Related Fields, 74:213-240, 1987. P. Major. Multiple Wiener-Ito Integrals, volume 849 of Springer Lecture Notes in Mathematics. Verlag, New York, 1981.

Springer-

P Major. Tail behaviour of multiple random integrals and [/"-statistics. (electronic), 2005.

2:448-505

Probab. Surv.,

P. Major. On a multivariate version of Bernstein's inequality. Electron. J. Probab., 12, 2007. B. B. Mandelbrot and J. W. Van Ness. Fractional Brownian motions, fractional noises and applications. SIAM Review, 10:422-437, 1968.

44

Y. Meyer, F. Sellan, and M. S. Taqqu. Wavelets, generalized white noise and fractional integration: the synthesis of fractional Brownian motion. The Journal of Fourier Analysis and Applications, 5(5):465494, 1999. Y.S. Mishura. Stochastic calculus for fractional Brownian motion and related processes, volume 1929 of Lecture Notes in Mathematics. Springer-Verlag, Berlin, 2008. G. M. Molchan. Historical comments related to fractional Brownian motion. In P. Doukhan, G. Oppenheim, and M. S. Taqqu, editors, Theory and Applications of Long-range Dependence, pages 39-42. Birkhauser, 2003. I. Norros, E. Valkeila, and J. Virtamo. An elementary approach to a Girsanov formula and other analytical results on fractional Brownian motions. Bernoulli, 15:571-587, 1999. G. Peccati and M. S. Taqqu. Moments, cumulants and diagram formulae for non-linear functionals of random measures. Preprint: arXiv:0811.1726vl [math.PR], 2009. M. Peligrad. Recent advances in the Central Limit Theorem and its weak invariance principle for mixing sequences of random variables. In Ernst Eberlein and Murad S. Taqqu, editors, Dependence in Probability and Statistics, Progress in Probability and Statistics, pages 193-224. Birkhauser, 1986. V. Pipiras. Wavelet-type expansion of the Rosenblatt process. The Journal of Fourier Analysis Applications, 10(6):599-634, 2004.

and

V. Pipiras and M. S. Taqqu. Are classes of deterministic integrands for fractional Brownian motion on an interval complete? Bernoulli, 7(6):873-897, 2001. A. Plikusas. Some properties of the multiple It6 integral. Lithuanian 191, 1981.

Mathematical

Journal, 21 (2): 184-

M. Rosenblatt. A central limit theorem and a strong mixing condition. Proceedings of the Academy of Sciences, 42:43-47, 1956.

National

M. Rosenblatt. Independence and dependence. In Proceedings of the 4-th Berkeley Symposium matical Statistics and Probability, pages 431-443. University of California Press, 1961.

Mathe

G. Samorodnitsky and M. S. Taqqu. Stable Non-Gaussian Variance. Chapman and Hall, New York, London, 1994.

Infinite

Processes: Stochastic Models with

M. S. Taqqu. Weak convergence to fractional Brownian motion and to the Rosenblatt process. far Wahrscheinlichkeitstheorie und verwandte Gebiete, 31:287-302, 1975. M. S. Taqqu. Convergence of integrated processes of arbitrary Hermite rank. Zeitschrift fur lichkeitstheorie und verwandte Gebiete, 50:53-83, 1979.

Zeitschrift Wahrschein

M. S. Taqqu. Fractional Brownian motion and long-range dependence. In P. Doukhan, G. Oppenheim, and M. S. Taqqu, editors, Theory and Applications of Long-range Dependence, pages 5-38. Birkhauser, 2003. C.A. Tudor. Analysis of the Rosenblatt process. ESAIM Probability and Statistics,

45

12:230-257, 2008.

ON SPECTRAL

ANALYSIS

OF STATIONARY

TIME

SERIES*

B Y U L F GRENANDER AND MURRAY ROSENBLATT D E P A R T M E N T OF MATHEMATICAL STATISTICS, UNIVERSITY OF STOCKHOLM, AND COM MITTEE ON STATISTICS, UNIVERSITY OF CHICAGO

Communicated by Saunders MacLane, April 30, 1952

1. The present statistical theory of analysis of stationary time series (e.g., extrapolation) has assumed complete knowledge of the covariance sequence or equivalently of the spectrum of the process. I t is, therefore, of great importance to be able to estimate one of these. However, knowl edge of the spectrum seems to yield greater immediate insight into the structure of the process. This seems to have first been noted in a funda mental paper by Bartlett. 1 An unpublished paper by Tukey 2 deals with some aspects of the problem of estimating the spectrum. 2. Consider a discrete parameter stochastic process xt (t = . . ., — 1, 0, 1, . . .) of the following form: CO

v

=

— co

where {an} is a real sequence and CO

IX 2 < ». The £„'s are assumed to be independent and identically distributed realvalued random variables with EL = 0, EL2 = <J\ EL4 = M4. The conditions cited above imply that the stochastic process x^ is strictly

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_7, © Springer Science+Business Media, LLC 2011

46

520

MATHEMATICS:

GRENANDER

AND ROSENBLATT

PROC, N. A, S.

stationary. Such a process is known to have an absolutely continuous spectral distribution function F(\) with a density given by iv\\ F'O) =/(X) = ^ £ a*6

Proofs of the results cited below will require /(X) to satisfy a smoothness condition and the £/s to have moments up to the eighth order. As the process is real-valued, the spectral density is symmetric about zero and hence one need only consider a spectral distribution function F(\) as the integral of the spectral density from zero to X, 0 < X < 7r. A reasonable estimate of the spectral distribution function F(\) is

^v*(X) =

S^INWI

where N

IN(J)

2TTN

I

is the so-called periodogram. Professor Doob suggested that we investi gate the problem of giving confidence bands for the estimation of F(k) in terms of FN*(\). In the case of a normally distributed process, Theorem 3 yields a solution which is non-parametric with respect to the spectrum. M4

In the general case, Theorem 1 and an appropriate estimate of e

- 3

can be used to obtain conservative bands non-parametric with respect to the spectrum. Theorem 4 can be used to test whether two independently drawn time series come from processes having the same spectrum. 3. The theorems are as follows: THEOREM

1.

lim P) N-*°°

max. )0<\<7T

N[FN*(\)

- F(k)]\ < a\ = P \ max. ZQi)

<

where Z(X) is a normal process whose distribution is completely determined by the mean value EZ(X) = 0 and the covariance EZ(\)Z(ji)

= eF(\)F(ji)

+

2TTG

(min. (X, /*)),

where

G(X) = COROLLARY.

If e =

f0xf(l)dL

0

lim P\ max. \VN[FN*(\)

- F(\)l < al = P\

where W(\) is the Wiener process, i.e., 47

max.

\W{\)

<

VOL. 38, 1952

MATHEMATICS:

EW(\)

= 0

GRENANDER

AND ROSENBLATT

EW(\)W(fi)

521

= min. (A, M)

and W(X) is normally distributed. Note t h a t e = 0 when the £„ are normally distributed. T H E O R E M 2. r&e statistic 1

&v*

2

z „ = __£#<* iV2

converges to 2TrG(ir) in probability as N —► oo if k > 0,0 < a < 1. cpys are the product sums

The

N ~ |H

1

T H E O R E M 3.

Ife

= 0

Km P ( max. VN[FN*(\) N^m

)0<X<7T

-\

~ F(\)]

—-— P */2 w

= P tt ax

L o3< x <- i W ( X ) | <

aL

I t is well known t h a t the probability on the right can be explicitly evaluated in a convenient form. T H E O R E M 4* If e = 0 and if FN1*(\), Fm* (A), i?ivi, i?iV2 are the statistics defined above corresponding to two independently drawn samples from the same stochastic process, then lim P ( max. VN[FN1*(\) N-

- /W(x)]

)0<X<7T

(Rm + J?j\r2) A

<

I

=P\m*x.\W(\)\
4. Although the plausibility of the basic Theorem 1 is apparent by heuristic arguments, the proof requires a long and formidable sequence of lemmas. The practical value of the theory cited above can only be judged after having completed an extensive computational program in volving analyses of artificially generated time series. Some preliminary analyses have already been made and tables of the related distributions are being compiled. Considerable investigation must be made of the practical application of the theory. The proofs of the above theorems and considerable elaboration of the theory (e.g., solution of the corresponding problem for the spectral density) as well as aspects of the practical applica tion of the theory will be contained in a forthcoming publication. * This work was supported by a grant from the Office of Naval Research. Bartlett, M. S., "Periodogram Analysis and Continuous Spectra," Biornetrika, 37, (1950). 2 Tukey, J. W., "Measuring Noise Color," unpublished manuscript. 1

48

470

MURRAY ROSENBLATT

REMARKS ON A MULTIVARIATE TRANSFORMATION1 B Y MURRAY ROSENBLATT

University of Chicago The object of this note is to point out and discuss a simple transformation 2 of an absolutely continuous fc-variate distribution F(xx, • * • , Xk) into the uniform distribution on the ^-dimensional hypercube. A discussion of related transformations has been given by P. L6vy [1]. Let X = (Xx, • • • , Xk) be a random vector with distribution function F(xt, • • • , xk). Let z = (zx, • • • , zk) = Tx = T(x\, • • • , xk), where T is the transformation considered. Then T is given by zx = P{Xi ^ xx} = Fx(xx), z2 = P{X2 S x21 Xx = xx\ = F2(x21 xx), zk = P{ Xk^Xk | xk-x = %k-x, • • • , Xi=xt

} =

Fk(xk | Xk-i, • • • , #1).

One can readily show that the random vector Z = TX is uniformly distributed on the ^-dimensional hypercube, for P{Zi ^ Zi\i = ! , • • - , & } = /

• • • / dxkFk(xk | afc_i, • • • , xi) • • • 4 i ^ i ( # i )

J

{Z\Zi^i)

= /

J

••• /

JO

(fel • •• tfeft = I I ^M

JO

i=l

when 0 ^ £»• ^ 1, t = 1, • • • , fc. Hence Z i , • • • , Z* are uniformly and independently distributed on [0, 1]. Let Xa) = (Xi(0 , • • • , -Xfc(o), f = 1, • • • , w, be a random sample of n vectors from a population with distribution function F(xx , • • • , # * ) and let (7(^i, • • •, Xk) be the corresponding sample distribution function. I t has been found that the probability distribution of the Kolmogorov-Smirnov statistic, max

| F(xu

• • •, xk) - G(xif ••-,#*) |,

is not the same for all continuous F when k > 1 [2]. The same can be said for the multidimensional von Mises statistic / J—oo

••• / v—oo

(F(xi, • • •, xk) - G(xx, • • •, #*))2 ^ . . . ^ F f o , • • •, #*).

However, it would still be of interest to study the Kolmogorov-Smirnov and von Mises statistics for the case of sampling from a population uniformly distributed on the ^-dimensional hypercube. One could test whether the X& , i = 1, • • • , n, are a sample from a population with distribution function F(xx, * • • , xk) by 1

Work done under ONR contract. 1 have recently learned that J. H. Curtiss and I. R. Savage have also considered this transformation. 2

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_8, © Springer Science+Business Media, LLC 2011

49

471

MULTIVARIATE TRANSFORMATION

using either the Kolmogorov-Smirnov or von Mises statistic to test whether the Z(o = TXa) are a sample from a population uniformly distributed on the kdimensional hypercube. Both statistics are invariant under permutation of the X(,->, i = 1, • • • , n. The transformation is of some interest in setting up regions of equal probability mass in a ehi-square test of goodness of fit. Assume again that one wishes to test whether the X (1 >, i = 1, * • • , n, come from a population with absolutely continuous distribution function F(x\, • • • , #*). This, as remarked before, is equivalent to testing whether the 2^> = TX^ , i = 1, # • * , n, come from a population with distribution function z\ * • • Zk , where 0 ^ z3- ^ 1, j = 1, • • • , k. Divide the ^-dimensional hypercube into the Nk cells ^h—h

~~ *z ~

< Zi < ~jT~

, % = % 1, ' « • , fc> ,

where j i , * • • , jk = 0, 1, • • • , N — 1. Each of these cells has probability mass l/Nk under the null hypothesis. Let v3l...jk be the number of transformed observations Z(i) = TXa) in Cjx...jk. The chi-square statistic k

2

2LY

can be used to test whether the sample X<;> comes from a population with distribution function F(x\, * * * , Xk). A test for goodness of fit based on the chi-square statistic above is consistent as n, N —> <*>. There are M transformations T of the type described above corresponding to the M ways in which one can number the coordinates x\, • * • , a:*. One might think this unsatisfactory because of a suspicion that the experimenter might carry out one of the procedures described above with each of the k ! transformations and then choose that particular transformation which yields the result he wishes to obtain. But this situation can arise in any case where there is a multitude of tests in the same context. The transformation T can be written down explicitly in several cases. We shall write down the transformation when F(xt, • • • , Xk) is a normal distribution with mean M = (mi, • • • , mk) and covariance matrix A = { X*-,- } , i,j = 1, * • * , k. Let A(r) = { \ij } , iy j = 1, * • * , r ^ k, and A|j ) be the cofactor of { X# } in A(r). The transformation T is then given by

1

V

VA<»/A$|>

r

/

a;ft — wit + 23 (Akj/Akk)(xj - vy)

Fk(xk

J=1

| % _ i , • • • , Xi) = $ '

VA/A **• 50

472

ABSTRACTS

Let F(xi, x%) be a normal distribution with means mi, m 2 , variances
x% — m% + — (xx — mi) F%(x*1 Xi) « * cr 2 Vl - P2 REFERENCES [1] P. U V Y , TMorie de VAddition des Variables AUatoires, Gauthier-Villars, Paris, 1937, pp. 71-73, 121-123. [2] PAUL B. SIMPSON, "Note on the estimation of a bivariate distribution function," Annals of Math. Stat., Vol. 22 (1951), pp. 476-478.

ABSTRACTS OF PAPERS (Abstracts of papers presented at the Eugene meeting of the Institute, June 19-21, 1952)

1. The Auditory Cortex—A Probability Model. ARCHIE R. TUNTURI, University of Oregon Medical School. The role played by the brain in communication is well known, but in what manner the brain handles information is not understood. Some progress has been made in this direction by studying the anatomy and physiology of the auditory cortex in the anesthetized dog with controlled acoustic signals. Communication may be thought of as making a representation in a space of a representation in another space. In three of the four auditory areas (on one side of the brain), the entire frequency spectrum from 100 to 12800 cps is represented literally spacewise by groups of cells that respond only to a narrow range of frequencies. A special method increases the signal to noise ratio, by augmenting the electrical response of the cells, thereby permitting exact measurements of the characteristic frequency and intensity for each group of cells. This is similar to a narrow band filter, and does not reveal the effect of other frequencies on the information. The information capacity of the system can be inferred if it can be assumed that occurrence of the augmented response for the group of cells follows some probability function. These probabilities for all groups of cells can be assembled into a model representing the behavior of the system as a communication device. If there are 70 groups of cells between 100 and 12800 cps, the probability of any particular combination would be 1/270, if the selections were equally probable, The effect of noise on this system will be considered. (Research sponsored in part by the Office of Naval Research.)

2. Testing Message Diffusion: The Utilization of Mathematical Models. STUART C. DODD, RICHARD J. HILL, AND SUSAN HUFFAKER, University of Washington. In connection with the study of interpersonal verbal communication, the Washington Public Opinion Laboratory designed an experimental procedure which yielded data on the temporal diffusion of thirty-three different messages in a population of 184 individuals. Data (including the recipient of each message, the initiator of communication, and the time of communication) were obtained on 5,522 separate instances of communication. The 51

STATISTICAL SPECTRAL ANALYSIS OF TIME SERIES ARISING FROM STATIONARY STOCHASTIC PROCESSES1 BY ULF GRENANDER AND MURRAY ROSENBLATT

University of Chicago Summary. We consider time series which are realizations of a stochastic process. From the time series we construct various estimates of the spectral distribution function of the process (Section 3) and we study the sampling dis tributions of some functionals of these estimates (Sections 4-7). We then obtain confidence regions for the spectral distribution function and various tests of hypotheses in the normal case (Sections 8-10). 1. Introduction. Let us consider a real discrete parameter stochastic process' that is, a sequence of stochastic variables xt, t = • • • , — 1, 0, 1, • • • .2 We introduce the quantities Ext = mt, the mean value sequence, and the covariances p8tt = E(x8 — m8)(xt — mt). The process yt = x% — mt is said to be sta tionary in the wide sense if p8tt = ps-t • Then it follows from a theorem of Herglotz [10] that

Pt

=

/

eia dF(\) where F(X) is a bounded and nonde-

creasing function in (—TT, W). F(X) is called the spectral distribution function of the process as it can be said to describe the distribution of the spectral energy of the process (see Wold [22], p. 16). As xt is real, pt = p~* and the distribution of the spectral energy is symmetric about zero. Since knowledge of the distribution of the spectral energy is equivalent to knowledge of the covariance sequence, it is a matter of convenience which to choose in analyzing the process. The reduced process y% can be written as

(1.1)

yt =

fe i t x dZ(\) J—T

[3] where .Z(X) is an orthogonal process such that EZ(\) - 0 (L2)

E | ZOO - Zip) | 2 = | F{\) - F(M) I. (For a discussion of stochastic integrals used in this paper see J. L. Doob, Stochastic Processes^ John Wiley and Sons, 1953.) Hence dF(X) can be inter preted as the variance of the stochastic amplitude dZ(X) corresponding to the harmonic euX in the Fourier expansion of yt. F(\) seems to be more directly related to the structure of the process than the covariance sequence. Later on Received 10/22/52. This work has been carried out with support of the Office of Naval Research. 2 Any notation introduced in the paper will be adhered to in the remainder of the paper without any further explanation. 1

537 R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_9, © Springer Science+Business Media, LLC 2011

52

538

U L F GRENANDER AND MURRAY

ROSENBLATT

we shall find other reasons for preferring to deal with the spectrum instead of the covariances. We can decompose F(X) into three parts, F(X) = Fa(X) + Fd(K) + FS(X), where Fa(X) = / /(/) dl, Fd(X) = X ^ x AF(X„), and FS(X) is constant except on a set of Lebesgue measure zero. Here/(X) = F'(X), the spectral density, is a nonnegative integrable function and the X„'s are the points of discontinuity of F(X) with associated jumps AF(X„) > 0. We are going to deal with the absolutely continuous case, but will make a few remarks later on concerning possible dis continuities of the spectral distribution function. A model of some practical interest is the following one. A linear filter $ can be characterized by its output at at time t generated by a unit input at time 0. It is not necessary for our purpose to demand that at = 0 for t < 0. Suppose we feed independent random impulses & , identically distributed with mean 0 and standard deviation 1, into the filter. Following Tukey we shall call a process of the type £* "pure white noise" as contrasted with a more general process t]t with Erjt = 0, E7jsrjt = 5st (called "white noise"), but where the rj's are not nec essarily independent. Note that in both cases one has a uniform spectrum, F'(X) = (27r)_1. The resulting output at time t is then 00

(1.3)

yt = X) a>n£t-n. n=—oo

a

If XX> * < °°> this sum converges with probability 1 (see Levy [15J, p. 139™ 142). If the input had been t)t we would only be able to state convergence in the mean square. In both cases the spectrum is absolutely continuous with spectral density (see Karhunen [12], p. 71)

w

' -h

ane

—in\

More restrictive conditions will be imposed on the process later. A process of the type (1.3) is called a linear process. On the other hand, if a real stationary process yt has an absolutely continuous spectral distribution function F(X) = / f(l) dl, Doob (see Proceedings of the Berkeley Symposium on Mathematical Statistics and Probability, University of California Press, 1949, p. 327) has shown that

(1.4) where we can choose an =

yt= / J—r

E

an 7]t-n »

etn)iy/f(X) d\ and Etjt = 0, E^srjt = 8st. The

?j's are elements of the real Hilbert space spanned by the y's and an appropri ately chosen set of real random variables (see [12], p. 42). They are then real and so are the a's. In the present paper we shall deal only with the linear proc53

SPECTRAL ANALYSIS I N TIME

SERIES

539

esses. It would be interesting to extend the domain in which our results are valid to a more general class of processes. A normally distributed process with an absolutely continuous spectrum is clearly linear as the rfs appearing in (1.4) are then automatically independent and identically distributed. 2. Some methods that have been proposed for time series analysis. An im portant group of statistical problems arise in the following wray. We observe a sample Xi, x2, • • • , xN (a time series) from the stochastic process xt and want to make inferential statements concerning the covariances, or equivalently the spectrum of xt. There has been a long history of work aimed at answering such problems. A considerable part of the literature has been devoted to testing the independence of the observations, usually in the case of normal time series (for a list of references see [14]). Other statisticians have studied what one could call finite parameter schemes. Two important schemes of this type are the autoregressive scheme (2.1) and the moving average (2.2) (2.1)

£

bny*-n = .{,

rc=0

(2.2) iSnfc-n Vt = £ where the Vs and the /3's are constants. Specifying a priori an order p for the process, they have given estimates for the coefficients and have devised tests for various hypotheses (for a list of references see [14]). Whittle [21] has gone further and has given tests for discriminating between finite parameter schemes of different orders without specifying the coefficients a priori. Any linear process can be approximated as closely as is desired by a finite parameter scheme of sufficiently high order. In most practical situations, how ever, it does not seem realistic to take the order of the scheme as some number p, usually small, given a priori. A great deal of the older literature has been devoted to the so-called periodogram analysis (see Kendall [13] for references). This was originally devised to deal with processes of the type xt = X) (A* s i n M + Bv cos \yt) + £« where m, Av, Bv, and A„ are unknown. Clearly xt has a spectrum with a dis crete component. To estimate the frequencies \v the following statistic called the periodogram INO)

=

2

2wN

E Xp e—iv\ J>=I

N 1l [Y V /N V C X S n 2irN -~Tr\ ( 23 %" °S ^ ) + ( ]C v i ^ )

54

540

ULF GRENANDER AND MURRAY ROSENBLATT

has been used. A rationale for this method is that IK(X) can be shown to diverge to infinity as N —> °o if X coincides with some frequency X„. Hence if I#(k) is large we suspect that X is one of the frequencies X„ of the scheme. For a corre sponding test of significance see Fisher [6]. Periodogram analysis is not immediately applicable to the case of an ab solutely continuous spectrum. However, some work of the last few years has indicated that when it is properly modified, it is the most powerful method that has been found to work without very special assumptions concerning the covariance structure of the process. We would like to mention especially Bartlett [1], [2], and Tukey [20]. A brief discussion of some of the results of this paper can be found in [9]. 3. Some preliminary considerations. Consider a process yt

= f e~itx dZ{\) = E Zve~itx + y\

where y\ is the component of the process with a continuous spectrum and the other term is the one with a discrete spectrum. If we observe a complete realiza tion yt, — oo < t < oo, we can specify the sample value of any Zv with proba bility one. However, we cannot estimate E\ZV\2 = AF(X„) consistently unless \ZV\ is constant with probability one. We note in passing that the model of random phases in which Zv = Aye%(p% where the
Note that e is the fourth cumulant of the random variables £. The periodogram ZAT(X) of the process yt then is N,

IM = 2TNl

12

Vte JV-l

2irN where Cv = ^IZl (3.1)

TTN £a

ytyt+v ■ We are interested in statistics of the form $*

=

f

INUMD

55

dl

541

SPECTRAL ANALYSIS I N TIME S E R I E S

where (l) is bounded, symmetric about zero, and has at most a finite number of discontinuities. Let (3.2)

$

= r jaw) di. J—ir

1. Let $ # , 3> be defined as in (3.1), (3.2) and let SEv , & be defined analogously with weight function $. If the spectral density f(l) is continuous THEOREM

lim E$N = ,

lim E9N = #

and (3.3)

lim N cov ($x, *N) = e$* + 4a- [ fUMDtU)

PROOF.

dl.

By definition of $#

(3.4)

E$N = f^ElAUd)

dl

But using (1.1)

IAD =

1 2TN

/»T

1

—ivl

1

iNQi—l) iiv*.A— t ;

i

and hence from (1.2) iN(K-l)

-—r

dE | Z(X)

s i n — (X — Z)

sin

:X

f(\)dh

- I

which tends to/(Z) as 2V —»• °o. But the integrand in (3.4) is bounded by !
covC^^.y^^^'--^5'.

£

But ^2/a 2//s y7 2/5 =

X)

= (/i4 — 3 ) 2L^ o>a—vO'fir-vO>y—va,t—v + oo

+

2J

JL*

(x a _^a / 3-_ p a ? _j li a5_ /i

QO

aa-pa^ay-pas-y + 56

2-, aa-va>fi-ixa>y-va>h-n

542

U L F GRENANDER AND MURRAY

ROSENBLATT

so that cov

(Vcxl/P,

VyVd)

=

(M4

— 3 )

2 2

da-pCt^-vdy-vClb-v

+

Pa-8

P&-y

+

Pa-y

P&-6

•

The fourth moments of Ya can be shown to exist and the operations that have been carried out can be justified by a repeated application of Schwarz' inequality and Fatou's lemma. Hence N cov ($N, AT

tyN)

1

=

4TT2AT

r

oo

2 2 (M4 — a,P, 7 , 5 = 1 |_

3)

2 2 0>a-vQ>$-va>y-vO>b-V v=—oo

+

Pa-6p0-y

+

Pa-y

P0-<

= $1 + $2 +

where v =

$3

e l v V(^) d^ a n d ^„ is defined analogously.

/

We first deal with S i . Note that /»7T

mlT

47T^iV

00

N

J - T T J-TTv=-oo a,0=1

7,5=1

= ^TTAT (' f £ I E °— ei<"X f I E a*-^*'™ f *(X)*G«) <*X d„. 47rZiV

J—7i J - T T v = - o o I a = l

I

I 7=1

I

First we consider the special case The function z 3 akdk-ne1'

Cn,m(X) =

z2

i—«

aia>i-me u\

is continuous. We have 7T~ JlW

l

£n,m\Xj6

==

^A

-Z-/

J—TT

$a—v<*a—n—i> dy—v

&y— m—v

J » = — 00

so that 2

S,(«-V') = ,-4v (2*0

m i n (iV.iST+m)

minCJV^V+n)

But then 7r 7T /

SIO

)

c„.m(x)
— t(a—7)X

J-7T

dx.

dX

c«,m(X)

/

sin 7T

=—f sin —-—- X 2 .

^-

7 = m a X ( l , l + n ) ^?T

. N —m ^ . N —n^ sin — X sin — X

t(n—m)X

— mX — *m/x\

e

E

E

a=max(l,l+m)

2X

i(n—m)X

- sm {N — n)\ sm — - — X n —m A , 1 2 dX. cos — - — X + sin 2

sm -

57

543

SPECTRAL ANALYSIS IN TIME S E R I E S

Using [5] we see that the second term is dominated by constant (log N)/(N) —> 0 as N —> oc. The first term tends to ecn,m(0) = epnpm . Now let us consider a finite trigonometric polynomial &(X,M)

=

hntme~inXe~imfi

£

in place of <£(X)^(X) in Si. Then Hm

& ( / f c ( \ , M)) =

7-2

2

hn,mPnpm

= £ /

/

&(X, M ) / ( X ) / ( / X ) ^

(fc.

Now given any <£(X) and #(/*), we can choose two finite trigonometric poly nomials hi(\, fi), &2(X, p) such that /* If

h(\

M) ^ 0 ( A ¥ W ^ fo(A, M),

pit

/

/

MX, n) - h(X,

M)]

dX dy. < e.

Then Si(
N—>oo

, lim Si(<j>(\Mn)) iV—►oo

lies between 6 / / /ii(X, M)/(X)/(M) dX <2M and e / / A»(X, . o Jo Jo *o On letting e —> 0 we see that lim &(0(X)*GO) = 6 f *(X)/(X) dX f

*GI)/(M)

M)/(X)/(AO

dX d/i.

d/i.

To evaluate the remainder £ 2 + £ 3 of cov ($JV , *#) we note that it would be the actual covariance if the process were normal. But in this case the limiting covariance has been evaluated in [7] when and $ are even, although under the restriction that the spectral density has a continuous derivative. However, an argument similar to that just carried out above indicates that the result in [7] is valid for continuous/(X). This proves (3.3). One should note that as the periodogram is an even function, it is sufficient to consider estimates of the form $# =

/

IJV(X)$(X)

Jo

d\ corresponding to the

weight function §[<£(X) + (—X)]. Expression (3.3) then becomes lim AT cov ($. v , *„) = e$# + 2x / f {1)^(1)^(1) dl. N-t-oo

•0

The following theorem gives an estimate of the speed with which E$N con verges to $>. THEOREM

2. Suppose thatf(X) has a bounded derivative. Then BbK

= $ + O(log N/N).

By the proof of Theorem 1 r 1 r sm — (X - I) E<$>N - * = -i- / ?— \f(l) - /(X)]*(X) dX dl. J-T dlTi\ J-r . 2X — I sin 2

PROOF.

58

544

ULF GRENANDER A N D MURRAY ROSENBLATT

But we have uniformly f(l) — / ( \ ) = 0(1 — A) so that . >K JSft*

~*- A

2N

f" S^J2~# °d * l>V2(2r - I * I) dx ^

* ^

sin-

according to [5]. We remark that if the function /(X) defined on the real line with period 2ir has a bounded second derivative everywhere we get instead E$x — $ = 0(1/N). We now prove three elementary lemmas that are required in the develop ment of the main results of the paper. These results have been proved under conditions that are probably far from necessary. But we believe these condi tions give an idea of the practical domain of applicability of the results. In the remainder of the paper we shall assume that E [ £„ |8 < oo. vl LEMMA 1. Consider the covariances cv = X)nli £n£n+i»j . Then if *>, ju ^ 0 (3.5)

(Ai N | Ecc+jCpCu+j | ^ < {A2N2

if

\i ■£ v

if

fjL « v.

The reader should notice that Cv refers to the y-process and cv to the ^-process. PROOF. I t is clear that (3.5) is made up of terms of the type J ? I l L i ?», • If one of the indices n* is different from the rest this term vanishes. As each of the terms is bounded by E j £„|8, it is sufficient for our purpose to enumerate the nonvanishing terms. But we have restraints, say n% = ni + a,

n* = nz + b,

fte

= ^s + c,

n& = n7 + dy

where we do not yet specify the integers a, &, c, d except that they should all be different from zero. We can then treat the eight variables in a completely symmetric way. Let us fix n\. As n2 -^ n\, n 2 has to be equal to some other n t -, say n 3 . Now we separate two cases. As n& T± nt, it has to be equal to one of n\ , n2, n 3 , ni or to one of n7, n 8 . Let us consider the first alternative. Then n7 has to be equal to one of n\, n 2 , nz, n±, n s , n6 and whichever we choose, we have (3.6)

ni = nx + a{,

i = 2, 3, • • • , 8.

As 1 ^ ni ^ N this gives us at most N possibilities. In the second alternative let us take n$ — n7. If any of n±, n 2 , nz, n4 is equal to any of n& ,n* ,n7, n 8 we have again a system of restrictions of type (3.6) and hence again at most N possibilities so we can exclude this case. But n2 = n% ^ n\ and n6 7* n7 = n$ so that the only other way of getting a nonvanishing term is n2 = n 4 , n6 = fts, which requires that a = b, e = d. If that is the case we clearly get at most N2 possibilities. The result now follows easily. __ LEMMA 2. 7%e distribution of the variables (c0 — N)/y/N9ci/y/N, • • • , ck/\^N tends to the distribution of k + 1 normal and independent variables with mean zero and variances e + 2, 1, 1, • • • , 1. 59

SPECTRAL ANALYSIS IN TIME SERIES PROOF.

545

Introduce Zj = fo(& — 1) +
so that UN

1 v 1 = — ^ Z) 2; = 7 7 = [to(co - N) + ix c[ + • • • + fe cj[]

where N

Cv = Cv +

Yl

&&+*,

V > 0.

It is evident that (c, — cv)/{-\/N) —» 0 in probability as N —» oo. But zy is a stationary (fc + l)-dependent sequence of random variables. Applying a theorem of Hoeffding and Robbins [11] we see that Ujr is asymptotically normal with mean zero and variance tl(e + 2) + J^i £„. Hence

Be*™ = 1? exp { * ^

+ i±

U ^ i

->

eW™i*

wrhich proves the lemma. LEMMA 3. If yt is a normal process with a positive spectral density having an integrable second derivative, then av can be chosen so that av = 0(l/v ). PROOF.

Choose av so that av =

e%vX\/f(X) d\ and integrate by parts

/ J—TT

twice. The result follows immediately. 4. Treatment of pure white noise. THEOREM 3. Consider the empirical spectral distribution function .x F$,iQ0 = IN.&) dl where

M

'-^ - mt^~ Then the limiting probability distribution of max

VN\F%,^(\)

- —

as N —> <*> is the same as that of max | f (A) where f (X) is a normally distributed process with mean zero and covariance

£f(X)f(M) = g^ + ^min(A, M ). 60

546

U L F GRENANDER AND MURRAY ROSENBLATT

PROOF.

VN

We have

^* / x

^1

X / Co

sin v^ 1 c„ si

/-—\

J/X

7TJ>

= sN,k(\)

+

rN,k§i)

where sN>k(X) consists of the first term and the k — 1 first summands of the sum. We shall show that with probability close to one, for sufficiently large k, | rjv,&(X) | is small uniformly in N7 X. Let us consider J^

cv

sin v\

cv eivX

X

v

VN where 0 < m < I < N. But A

cv eivX

i-j

^2Z

£i» VN v

Cy Cy-\-j

To get a bound for this sum we consider i-i

E

(4.1)

Cv Cv-{.j

2

l-j

Nv(v + j)

j^

m

Ecv

Cy-{.j Cy.

Cff\-j

Z

N V(V + j)fl(jl +

j)

We know by Lemma 1 that (A2N2 JbCvCv+j Cy Cp+j j ^

if

v = n

if

^ / i .

i

[AxN

Thus the expression in (4.1) is bounded (use the Schwarz inequality) by ^4.2X1 2

'^ + £2

1 ^ A*' + j) 2 ^ Ar ,,^n y(i/ + j)fx(fi + j)

=

„«=7n v\v ^3 + y)2 *

Now we choose m = 2 P and £ = 2 p + \ Then again using the Schwarz inequality we get 2

E max

V^

cv sin ^X j

A

S^

2P+1-

1

1/2

V-2P V*(v + j T ' J

0 < X <7T

2P

1

=5 ^.5 2-f ^3^/2 = A s / 2

Let k = 2 n . Then with probability greater than 1 — ^ 5 / 2 p / 4 op+l

Tp =

max

0 <;X<7r

v^p

VN

v

<2W4.

Consequently [log.V]+l

61

J

SPECTRAL ANALYSIS IN TIME SERIES

547

with probability greater than 1 — ^ 6 / 2 n / 4 . Choosing n sufficiently large j rNtk(\) | is uniformly small. We want to find the limiting value of

p(max

-A.

VNIFZM

But from what we have just shown, we see that for any e, 5 > 0 P ( max | sNfk (X) | < a — e) — 5 ^ P ( max

S/N

KM

- 2x

^ a)

^ P{ max | s*,fc (X) | < a + e) + 5 for ft sufficiently large, uniformly in N > 2V(«, 5). But A

\

SNAX)

Co

=

~~ N 27rVF

X +

If c sin v\ 7rVViV *

and we have shown in Lemma 2 that the joint distribution of (co — N)/\/N, CI/\/N, • • • , Ck/^N (k fixed) converges to the distribution of & independent normal variables with common mean zero and variances e + 2, 1, 1 , - - - , 1 . Consider the related process

Sk(X)

= f. V7T2 x V t y-

^

where the 7*s are 2V(0, 1) and independent. It is easily seen that lim P ( max | sN,k (X) | ^ a) = P ( max | sk (X) | ^ a) iV—>00

O^X^TT

O^X^T

as the relevant point set in (k + l)-space is closed [18]. But on letting k tend to infinity s*(X) converges uniformly to

(see Paley-Wiener [17], p. 148-151). But then P{ max | f (X) | < a - e)

5 0. Since the distribution function of m a x ^ x ^ , | f (X) | is continuous (see Section 7), this completes the proof. x 5. Reduction to pure white noise* Let G(X) = j p{l) dl. Jo THEOREM

4. Let /(X) be a nonnegative absolutely continuous function.

lim P < max N-*QO

lo<X<x

VN

/ /(Z){ 2 * 1 ^ ( 0 -

Jo

1} dl

^ a> = P{ max | »j(X) j < «} 0<X
62

Then

548

ULF GRENANDER AND MURRAY ROSENBLATT

where r?(X) is the normal process with, Eri(\) = 0, Er)(\)i)(/j.) = eF(X)F(n) + 2irG (min (X, /*)). PROOF. Integrating by parts we have (

J(1){^INAI)

~l}dl

=

2TT/(X)

fe!?(X) -

A]

Given e > 0, by Theorem 3 we know that max

VN ( i ^ ( X ) - £-) - **.* (X)

<

€

with probability 1 — e uniformly in N > N(e) for k sufficiently large. But then max /(x) V t f ( * l * ( X ) - £ ) -

8JM (X)

O^X^TT

fV(0

VNIFIAD

dl\ < Bit

SNAD

2TT

with probability 1 — e. However, P < max U<X^7T

/(x)«*.*(x) - f f'(i)sNAD di £ a P< max 10 < X < ir

/(x)«*(x) - f / m o ) (2Z *>0

<

a

as iV -^ oo as the relevant set is again closed in (k + l)-space. We know that for any e > 0 max | sk(X) - f(X) | < e with probability 1 — e for sufficiently large k. Let i?(X) - /(X)f (X) -

f /(Of (0 #.

In summary P < max | T?(X) | < 10 <X<7T

-.V

8 < Pi max [osX
VN j

ffl{2KlN,iW-Udl\£a

< P< max j rj(X) | < lo<x<x

a

+ e> + 5

and on letting N —» oo so that e, 5 —> 0 we obtain the desired result. Note that the event max0^x^7r | i?(X)| ^ a has a well defined probability since T?(X) is a process with continuous sample functions with probability one. 63

SPECTRAL ANALYSIS IN TIME SERIES THEOREM

549

5. Let av = 0(v8) where (3 < — %. Then given any e > 0

VN max o <x<

[IAD - 2wf(i)iN,i(i)} di

<

€

T

with probability 1 — e for Ar > Are P R O O F . W e have (5.1)

2irVN

I [IN(l) - 2wf(l)IN,M

dl = ^ 7 - Z

arasdrs

where t(n—m)X ^r,s

^ ^ £n—r fra—s ""77 n,w=l t(n

-4

— m)

i(n—ra)X

\~"

^L-/ ? » - r ?m-« n=l+r,iV-fr m=l-j-s,2V-i-s

+

%\Th — TYl)

The coefficients of the terms when n = ra should be interpreted as X. W e note t h a t there m a y be a lattice rectangle R^J of points (ft, m) common t o both sums. Let us call the complement of R^J with respect to the set consisting of all the lattice points in both summations CrV . Then we have E max I dr,s | ^ 0^X<|7r

m

f ^ w g(n - m)

where "2

flf(s) = 4 I x I

if x 9* 0

7T if X = 0 .

I t is easily verified that y,

[2N\ogNii\r\>

n mEC

'

r,s

N or\s\>

N

U ( s + r) log i\T otherwise.

The expression in (5.1) is then bounded b y O

oo

—7= Z ViV

O

Z

oo

I a, a. | 2Ar log N + 77== Z

a=—oc |r|>A'

ViV

Z

«=-«, |r|gAT

I ara. | r log iV

= g ( Z I a. |V4 ViV log iV Z K I + ^ # \s=-oc

[r|>AT

/ \

ViV

Z r|a,|Y

jr[^Ar

/

Under the assumption made above, this expression tends t o zero as N —> <*> which proves t h e theorem. 6. Treatment of the general case. Let F*(X) - / 7*(Z) efl. •'0

T H E O K E M 6. Let

1. /(X) 6e absolutely continuous 2. a, = 0 ( / ) , 0 < -V2. 64

550

ULF GHENANDER AND MURRAY ROSENBLATT

Then lim P{ max y/N

\ F*(\) - F(\) | £ a] = P{ max | v(\) | ^ a}

w&ere r}(\) is the process defined in Section 5. The proof follows immediately from Theorems 3, 4 and 5. An important special case is contained in the following corollary. COROLLARY 1. When yt is a normal process with a positive spectral density with an integrable second derivative, Theorem 6 reduces to the following statement: lim P\ max VN JV-KJO

| F*(X) - F(\) \ S a}

O^X^TT

(6.1) =

E

(-l)*[*((2fc + \)a/x) - *((2fc - l)a/x)]

=

A(a/x)

t^ftere *(w) is #ie normal distribution function and x = S/2TG(T). PROOF. Lemma 3 implies that the assumptions of Theorem 6 are satisfied. As e = 0 the process 17 (X) reduces to the Wiener process with the following changed scale of time: t = 2wG(k), 0 ^ X ^ 7r. The reader may note that (6.1) is the probability that a particle in Brownian motion on the line is not absorbed by the barriers a, —a in the time interval 0, 2x6? (71-), given that the particle starts from 0 at time 0 (see Sommerfeld [19], pp. 74-79). 7. The limiting distribution in the case of pure white noise. When the process yt is pure white noise (so that/(X) = constant, say 1) the limiting probability we are interested in is given by (7.1)

P{ max | f(X) | ^ a} o < ; x ^ -a-

where f (X) is the normal process with # f (X) = 0 # f (X)f (M) = e\n + 2w min (X, M ). Now (7.1) has been explicitly evaluated when 6 = 0 (see 6.1). However, the limiting distribution can still be evaluated when e ^ 0 . THEOREM 7.

(_i)y-*'« > /-'m-
P{ max | f(X) | ^ a\ = t O^X^TT

k=—

00

ft^KM^H))]

where y = e + 2. PROOF. Let r = A/TT. Let X be a normal random variable with mean zero and variance one. Let X{T) be a normal process with EX(r)

= 0

£ X ( T ) X ( T ' ) = min

65

(T, T') -

TT',

SPECTRAL ANALYSIS IN TIME SERIES

551

that is, X(T) is the Wiener process conditioned so that X ( l ) = 0. X and X(r) are independent. Consider the process U(T) = s/ywTX + W\/2X(T). The proc ess U{T) has the same probability distribution as f (X) so that P{ max | f(X) | ^ a] = P{ max | u(r) j ^ a } . But P{ max | U(T) I ^ a} (7 2) =

— r J-a-VyTrrx ^ •*-« I 7rV2

^ a-vW XV2

x (

0

^

T

g A J

(aj) d x

where ^(#) is the normal frequency function. Let Y(t) = (t + 1)X ^ - i ^ ) ,

0 ^ < < oo.

Then F ( 0 is the Wiener process (see [4]), 0 ^ t < °o. We have then to consider (7.3) P{ - a - (a + &)* ^ y(t) S a + (a - &)*; 0 ^ < < °o } where a = a/ir\/2, b = y/yx/y/2. The integrand in (7.2) is clearly zero unless | V S ^ I ^ a/V. But supposing this inequality to hold, Doob [4] has evaluated (7.3) as

E

{

-2[(2m-l)2a2_( 2m -l)6a]

{6

.

-2[(2m-l)2 0 2+(2m-l)6a]

- 2 [(2m) 2a2-2m&a]

+e

—e

-2[ (2m) 2a2+2mba] >

—e

= 2£

)

( - D ^ + V 2 " 2 " 2 cos/* 2»&a.

Thus (7.1) is equal to \7rV7/

\irVT/

»=i

\v\/yj

/* ( ^ 5 ? L \

7T _

+

«*-<*/*V"Y

WT/

«=I

« \ _ ^ ( n ^ _ « \ + $ / zWJ? +

7TV7/

/ —

\

n

x

\

V Ta

^

__

a

TT-y/y/

\ \

7TV7/J

=

^

7r

\

/

*

a\ WVy/

-j\n -(n*aS/ir»)(l-(7/2))

n^-oo

. [ # ( ^ ( . + i ) ) _ . ^ ( . i))]. This expression parallels (6.1) and will be used in a later publication in which we will study some applications of the theory. 8. Statistical applications. The corollary of Theorem 6 lends itself to impor tant statistical applications. First, however, we will have to estimate G(ir) which 66

552

ULF GRENANDER AND MURRAY ROSENBLATT

is in general unknown. We note that G(ir) = (47r)~123!!00 pi. Of course the Cv/N are consistent estimates of pv. It is natural to consider an estimate of the form (8.1). One reason for this particular choice is that in practice one will try to avoid the cumbersome calculation of all the lagged product sums Cv. LEMMA 4. If av = 0 ( / ) , j8 < — 1, the statistic 1

(8.1)

[KN<*]

G*M = ^ 5

is a consistent estimate of

[Co2 + 2 E

Cl],

0 < a < 1,

K>0,

G(TT).

PROOF.

(8.2)

(?*« = - L j C„2 + 2 E Cl\ + 5^= E Cl

But we see using (3.5) that the last sum is less than or equal to [KNa] N oo Z-/ 2s 2s { | fl/-0i ai+v-$i j>=m-fl j,k=X fiifiz——oo

dk-02 G>k+p-p2 I +

I aJ~01 aJ+>-&2 ak-Pi ak+y-02 |

+ | aj~p1 aj+v-02 Q>k-$2 0'k+v-p11} (4 + fn) = QCi + Z-/2 + ZJ3) (4 + /**) * Let the rv be understood to be the covariances of the process ]TX-*> I an-v | £*. The spectral density (27r)_1 | 2 I a^ I e%vX ? °f ^ s Process is continuous and hence quadratically integrable, so that ]>2-«> rl < *>. But then

| Z i | - Z Nyl^N2

E rl

so that | ]T)i |/Ar2 < € for m sufficiently large uniformly in iV\ Also we get iVw

iV Z y=m+l i,fc=l

iV

m+1 n=-co

iV

-oo

as iV —> 00. The third sum is handled in almost the same way but using Schwarz' inequality. We now choose m so large that the last sum in (8.2) is less than 5 with probability larger than 1 — 5. The term in brackets in (8.2) consists of a fixed number of terms each of which converges in probability to the correspond ing term in the expression for G(TT). This proves the lemma. The corollary of Theorem 6 shows that in the normal domain the asymptotic distribution of max x s/N \ F*(X) — F(X) \ depends only on one parameter G(TT). This together with Lemma 4 enables us to construct confidence bands for F(\). THEOREM 8. Suppose that yt is a normal process and that its spectral density is positive and has an integrable second derivative. Then (8.3)

#00 - a y ^ | ©

^ F(X) ^ Ft(X) + «

| / ^ ^

holds with a probability tending to A (a) as N tends to infinity. The proof follows immediately from Lemma 4 and Corollary 1 as A (a) is continuous. Clearly Theorem 8 also gives a test of significance for the simple hypothesis 67

SPECTRAL ANALYSIS I N TIME SERIES

553

of a completely specified spectrum. A more interesting case, however, is the following. THEOREM 9. From a stochastic process satisfying Theorem 8 we observe two independent time series of length Ni and N2. Compute the corresponding estimates F%x(\), F*2(X), Gi (7r), Gt(IT). Then, assuming that N1/N2 —> c > 0 and putting N = 2N1N2/(N1 + N2),

max VN\FU\) x

(8.4)

°* **

,

-F£ 2 (X)|

— —

±

> a,

holds with a probability tending to 1 — A (a) as N tends to infinity under the hypothesis FX(X) = F2(X). Theorems 8 and 9 are somewhat restricted in their applicability as they stand, since in many contexts mt = Ext is not identically zero. We shall consider the case mt = dip^ + d2%
(8.5)

R = lim A-i A'—oo — *

Tf-L(l)* V *

Z^
(2)2

< 1.

Zs
We will use the least square estimates df , dt of d\, di as they have been shown to be asymptotically efficient [8]. THEOREM 10. Under the conditions of Theorems 8 and 9 and (8.5), formulas (8.3) and (8.4) remain valid if F*(X) is computed using xt — dt
max

0 < X ^ IT

s/N

IT A,

Jn

A'

2

eft

„„ 1

■J

/.X I *V

2

'n 27TiV Jo I 11

tends to zero in probability as N —> cc. The expression inside the absolute value sign is 2ir2V

V,M=I

+ (d* — d^^pv
554

U L F G R E N A N D E R AND MURRAY ROSENBLATT

where the same convention as before is used in interpreting the terms with v = fx. The least square estimates d* , dt are unbiased linear estimates with coefficients depending only upon the
(8,,

var g

S

*

„ «

JM

(2)2

s , , . . . ^ . ' : ^ , , , ^ , max /(A)

2TT

1 -

R

,

£ *SU )

J

Now TTVN max

| S i | ^ 77~ S 1 <£ - di | £ z2

£

^>+;

12

N

+ 7r | d* — di

-

1

VVVP

1

\ +

\ d*

— di \

-j

zl

i— p=l—p

But we have 'N—p

(8.7)

E

Z

2

(i)

AT

^

2TT

max /(A) £

^1)2-

We know from (8.6) that with as large a probability as desired \d* -~ di\ \s less than V ^ X ^ i ^ * 2 * * - Then it follows from Schwarz' inequality and (8.7) that with large probability WN

max | E i I < K' log N/VN

- » 0.

2^2 can be handled in the same way. Now 2WN

max | Z a | = @-=M

f \ ± ,< V " \ dl S

(dt

~ ^

T

t

v?

)*

and the expectation of this tends to zero as N~~\ ]T^ a n d ^ s can be treated in a similar manner which proves the theorem. An important special case is
n.„(x) - ^ + i i a ^ 4 ff 7,(0sin sin(h (I+ - §)(£fi/2)— n)dl dfi N

Z7TiV

7TiV v=l

V

ZlT JQ J - T T

69

SPECTRAL ANALYSIS IN TIME SERIES

555

where hM —> <*> as N —> «>, as they can be proved in exactly the same manner since the weighting factors are 1 from v = 0 up to v = hN and 0 from that point on. Note that the estimate F*,*r(A), in general, is not nondecreasing with prob ability 1 which may at times be an unpleasant feature. We can choose hN = [kNa] as in Lemma 4. This reduces the computational work considerably as one then only needs to compute the CVs for v S [kNa]. We shall now consider a class of estimates of the spectral distribution func tion with nondecreasing weight functions. Let W#(\) = I

Wx(l) dl where

J—TT

Wx(l) ^ 0 in (—7r, 7r), /

WKQ)

dl = 1 and

J—r

fo if x < o lim W„(\) = j *-«>

Let

[1 if X > 0.

K<\, WN) = f I„(l)W„(\ - I) dl Jo

F(\, WN) = ( f(l)W„(\ - I) dl «'0

We prove the following theorem. THEOREM 11. Under the conditions of Theorem 6 lim P{ max VN \ K(\, PROOF.

WN) - F{\, WN) | ^ a} = P{ max |

On integrating by parts we have

VN[F%(\ WN) - F{\, Ws)] = \/N

f [IAD - f(l)\ dlWN(X - TT)

+ [ VN f [IAn) - /GO) d»w„(\ - I) dl JQ

JQ

By Theorem 5 we can replace (9.1) by VN (9-2)

ff{l)[2irI N S)

Jo

- 1] dlWN{\ - T) „ _ « + [ VN f [2T/ J ,. { 0*) Jo

Jo

1]/(M)

dnwN(\ - I) dl

committing an error of at most e > 0 uniformly in X with probability 1 — c. However, on integrating by parts twice we obtain

VNMK.M

- i]fMWN(\ - x) - f

VN2T \F%,&)

- i-1

(9.3) • | (f(DWN(X - /)) dl. 70

556

TJLF GRENANDER AND MURRAY ROSENBLATT

But y/N max F%,t(\) - ^

~7^ sN,m(\);

< €

with probability 1 — e where m is a large but fixed number. We can then replace (9.3) by j[ fflW„(\ - l)s'„,m(Q dl = ^y=

[ /(Z)TfJV(X - I)dl 1 A W

1

/" _£j_ JQ

cos vlj(l)WN(\ - Z) dZ

with an error of at most e uniformly in X. But then reasoning as in Theorem 4 we get lim P< max

f(l)W„(\ - lK,Jl)

dl

>( P< max

a

j f(l)sUDdl

Jn

< a

making use of the fact that max

/ cos vlf(l)WN(\ — I) dl — / cos vlf{l) dl\ ^ max /(X) i [ WN{1) dl + [' \ WN(l) - 1 | dl\ -» 0

asiV

But max 0<X< r

h

f(l)sUD dl - *(X)

<

€

with large probability if m is sufficiently large and the theorem follows imme diately. We are usually more interested in estimating F(X) than F(X, WN). The follow ing corollary enables us to do this. COROLLARY 2. Theorems 6, 8-10 remain valid when F%(\) is replaced by F*(X, WN) if (9.4)

[ WN(X) d\ + l * [1 - WN(\)] dX = o(N-*). J—TT

PROOF.

JO

The proof follows immediately as

max VN I P(X) - P(X, WN) j ^ VN

max /(X) ( f WN(\) d\ + f [1 - Wjtf] 71

d\[ = o(l)

SPECTRAL ANALYSIS IN TIME SERIES

557

It is clear from the comment above that the time series analyst has a large class of estimates of the spectral distribution function at his disposal. At present we are not able to claim that certain estimates are preferable to others. We hope to investigate such problems in a later paper. It is worthwhile noting that the Fej£r kernel 1 sm Y = wN(\) 2whN —— sin2 2il satisfies (9.4) if log hx/hN = o(N~~*)9 so that we can choose the truncation point hx as hx = [kN% § < a < 1. Note that the estimate corresponding to this kernel is

This estimate of the spectral distribution function is closely related to an esti mate of the spectral density given by Bartlett [1]. It is nondecreasing and does not require the computation of all the CVs. 10. Alternative statistics and the corresponding limit theorems. In some in stances one might prefer to consider statistics other than max^x^* | F%(X) — F(X) | in analyzing time series. We shall therefore consider some alternative statistics in this section. Consider the linear space consisting of continuous functions c = c(X) on 0 ££ X ^ 7r with the norm \\c\\ = sup0^x^x | c(X) |. Consider a functional
P{
PROOF.

- F(X)]) ^ «,0 ^ X S

TT}

= PMtiQO) ^ a,0 ^ X £ ir}.

Writing

VN [KOO - F(\)] = s*,*(X) + r*,*(X) as before we note that || r^^CX) || < e if k is chosen sufficiently large. Hence we commit only a small error by considering instead the probability of the event { oo. But we can choose k so large that || s&(X) — i?(X) || < 5 with prob ability 1 — 5. This together with the uniform continuity of
^^/jjcc^r^cx)^ 72

p^ i,

558

ULF

GRENANDER AND

MURRAY ROSENBLATT

where n(\) is bounded and nondecreasing. This will give a statistic of the von Mises type. 11. Acknowledgement. We are indebted to J. L. Doob who suggested the problem that led to the investigation resulting in this paper. REFERENCES [1] M . S. BARTLETT, "Smoothing periodograms from time series with continuous s p e c t r a , " Nature, Vol. 161 (1948), p p . 686-687. [2] M. S. BARTLETT, "Periodogram analysis and continuous s p e c t r a , ' ' Biometrika, Vol. 37 (1950), p p . 1-16. [3] H. CRAMER, "On harmonic analysis in certain functional s p a c e s , " Ark, Mat. Astr. Fys., Vol. 28B (1942), p . 12. [4] J. L. D O O B , " H e u r i s t i c approach to the Kolmogorov-Smirnov theorems," Ann. Math. Stat., Vol. 20 (1949), p p . 393-403. [5] L. F E J E R , "Lebesguesche k o n s t a n t e n und divergente fourier-reihen," / . Reine Angew. Math., Vol. 138 (1910), p p . 22-53. [6] R. A. F I S H E R , " T e s t s of significance in harmonic analysis," Proc. Roy. Soc. London, Ser. A, Vol. 125 (1929), p p . 54-59. [7] U. GRENANDER, " O n empirical spectral analysis of stochastic processes," Ark. Mat., Vol. 1 (1951), p p . 503-531. [8] U. GRENANDER, "On the estimation of regression coefficients in the case of an a u t o correlated d i s t u r b a n c e , " unpublished manuscript. [9] U. GRENANDER AND M . ROSENBLATT, "On spectral analysis of s t a t i o n a r y time series," Proc. Nat. Acad. Set., U. S. A., Vol. 38 (1952), p p . 519-521. [10] G. HERGLOTZ, "tTber Potenzreihen mit positivem reelem Teil im E i n h e i t s k r e i s , " Sitzgsber. Sachs. Akad. Wiss., Vol. 63 (1911), p . 501. [11] W. H O E F F D I N G AND H . R O B B I N S , " T h e central limit theorem for dependent random v a r i a b l e s , " Duke Math. J., Vol. 15 (1948), p p . 773-780. [12] K. K A R H U N E N , " U b e r lineare Methoden in der Wahrscheinlichkeitsrechnung," Ann, Acad. Sc. Fenn. Ser. A I, M a t h . - P h y s . N o . 37, 79 p p . 1947. [13] M. G. K E N D A L L , The Advanced Theory of Statistics, Vol. 2, C. Griffith and Co., London, 1946. [14] T . KOOPMANS, Statistical Inference in Dynamic Economic Models, John Wiley and Sons, 1950. [15] P . LEVY, Thiorie de VAddition des Variables Aleatoires, Gauthier-Villars, Paris, 1937. [16] P . LEVY, Les Processus Stochastiques et le Mouvement Brownien, Gauthier-Villars, Paris, 1948. [17] R. E . A. C. PALEY AND N . W I E N E R , Fourier Transforms in the Complex Domain, Amer. M a t h . S o c , New York, 1934. [18] H . SCHEFFE, " A useful convergence theorem for probability d i s t r i b u t i o n s , " Ann. Math. Stat., Vol. 18 (1947), p p . 434-438. [19] A. SOMMERFELD, Partial Differential Equations in Physics, Academic Press, New York, 1949. [20] J. T U K E Y , "Measuring noise color," unpublished manuscript. [21] P . W H I T T L E , Hypothesis Testing in Time Series Analysis, Uppsala, 1951. [22] H . W O L D , A Study in the Analysis of Stationary Time Series, Almqvist and Wiksells boktr., Uppsala, 1938.

73

RECURRENCE-TIME MOMENTS IN RANDOM WALKS J. L. HODGES, J R . AND M . R O S E N B L A T T

1. Introduction. Consider

an irreducible

time-homogeneous

Markov

chain

with discrete time. The recurrence-time moments of the s t a t e s of such stochastic processes are studied. We point out that if the recurrence time of one state has its first k moments finite, then the recurrence times of all the other s t a t e s have their first h moments finite. We then specialize and investigate the recurrencetime moments of random walks. The main result of the paper consists of exhibiting random walks whose first k - 1 recurrence-time moments exist and whose higher moments are infinite, for k - 1, 2, • • • . A comparison theorem is derived that permits the moment properties of recurrence times of a large c l a s s of random walks to be determined. 2. Preliminary considerations. We begin with the following: DEFINITION. By the index of the random variable X, denoted by I(X), k

shall mean the largest integer k such that E(X ) finite, we write I(X)

= oo. Clearly I(X)

we

< oo. If all moments of X are

>_ 0. We shall consider only nonnegative

random variables. LEMMA

I.1 (a) I(Xi

/ / Xl9 •••

9

Xn

are independent

+ • •• + Xn ) = min \ 1{XX\

( b ) If X has the geometric

• •• ,

random variables, I(Xn)\r

distribution

P(X = n) = p(l-p)n-1 then I{X)

(p>0),

= oo . ( c ) / / N, Xx, X2, • • • are independent

positive

then

integer valued, Xx> X29 • • • are identically

random variables,

distributed,

N is

and then

The proof of these simple facts is omitted. Received April 28, 1952. This work has been carried out under ONR contract. Pacific J. Math. 3 (1953), 127- 136

127

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_10, © Springer Science+Business Media, LLC 2011

74

J. L. HODGES, JR. AND M. ROSENBLATT

128

l(Xl

+...+XN)

= min|/(/V),/(^l)}.

We adopt the terminology and notation for Markov c h a i n s u s e d by F e l l e r [ 1, C h a p . 1 5 ] • L e t El9

E2,

• • • be a denumerable s e t of s t a t e s . We a s s u m e the

chain i s i r r e d u c i b l e ; i . e . , for every pair £ ; , Ej there e x i s t s n(i9 t r a n s i t i o n from £ / to Ej in n{i9

j) s u c h t h a t

j) s t e p s h a s p o s i t i v e p r o b a b i l i t y . L e t /?,• be the

random number of s t e p s for first return to £ ; , and l e t /?jy be the random number of s t e p s for first p a s s a g e from £ ; to Ej, s t a r t i n g at £ j . It i s well known [ l ] that if, for any / , P(Rj

< o o ) = 1, then t h i s h o l d s true for a l l / , and we s h a l l

a s s u m e t h i s to be the c a s e ; i . e . , our p r o c e s s i s a s s u m e d to be r e c u r r e n t . LEMMA 2 . 2 ( a )

/ ( / ? ; ) has a constant

(b)

value,

say / ( C ) , for all i= 1, 2 , • • • .

For every pair i I(Ri)

= mini/(/?,-/),

HRji)\.

T h i s lemma i s r e l a t e d to the well-known r e s u l t t h a t in a recurrent i r r e d u c i b l e time-homogeneous

Markov c h a i n , if the e x p e c t e d time for first return to any

s t a t e i s finite, then the same h o l d s for all s t a t e s . In the l a n g u a g e of [ 1 , C h a p . 1 5 ] , if a n y s t a t e i s null, then every s t a t e i s n u l l . T h e lemma e x t e n d s t h i s r e s u l t from first moment to arbitrary m o m e n t s . E x a m p l e s of Markov c h a i n s of index k are to be found in a c l a s s of Markov c h a i n s c o n s i d e r e d by F e l l e r [ 1 ] . L e t X{n) p . = P{X(n)

= Ej

q. = 1 - p .

+1

| X(n-l)

= P\X{n)

= El

be the Markov c h a i n , and l e t = £. !,

| X(n-l)

= E}\,

;=l,2t... .

Then

t=l

We o b v i o u s l y obtain a Markov chain with E(R^)

i=l

t=l

< oo, £ ( / ? ^ + i ) = oc, where

i _> 1, if the p. are such t h a t n-1

n

f

El Pi-UPi = 1

1

n

(0< e < D .

k + l + e

2

After obtaining this result we learned that it had been obtained earlier by K. L. Chung and by R. N. Snow, in a more general form (unpublished). We omit the proof. 75

129

RECURRENCE-TIME MOMENTS IN RANDOM WALKS

where

= l.

CT nk+1+€

i

3. Comparison theorem for random walks. We Markov chains to random walks. There is no loss states of the walk to be consecutive integers, and probability of transition from i to i + 1. To ensure

shall now specialize from of generality in taking the we shall let p. denote the irreducibility, we shall re-

quire 0 < p. < 1 for all interior s t a t e s . To apply Lemma 2, we walk recurrent, so we shall assume that the boundaries, if any, and that (if the walk is unbounded in one or both directions) of escape to infinity is 0. Denote the walk by W and its index by THEOREM 1. ( a ) / (/?;.) has a constant

must have the are reflecting, the probability I(W).

value, say 1 , for all i > j , and

a constant value, say I , for all i < j . ( b ) Further, I(W) = min {IL, IR \ . Proof. We shall first show that /(/fj +1, i) = I(Ri, j-i )• Consider a walk starting at i and indefinitely prolonged. It is certain eventually to reach i — 1; let M denote the number of times the walk is at i, including the start, before reaching i - 1; M has the distribution P(« = m)-(l-p|.)p™-1; and given M = m, the walk consists of m — 1 steps from i to t + 1, m - 1 first passages from i+ 1 to i (denote these by R\\

• > • • • » / ? . ? , . ), and a terminal

step from i to i — 1. Thus R. .

= M + R(\\

I, l - i

.+ - . . + R^

I +1, I

..

1+1,1

Apply Lemma 1 ( b ) , ( c ) , ( a ) . To complete the proof of (a ), represent a walk from i to /, i > /, as the sum R

i,j

= Ri,i-i

+ #*-i,i-2> + ••• + #/ + i,/>

and apply Lemma 1 ( a ) . As for ( b ) , this follows from Lemma 2 ( b ) . It is clear that in a walk over finitely many s t a t e s , the index is oo. (See [ l , problem 8, p . 345].) Similarly, passages away from a reflecting boundary 76

130

J. L. HODGES, JR. AND M. ROSENBLATT

have moments of all orders. The interest attaches to passages away from an unbounded side, and by virtue of Theorem 1 ( b ) we may consider the two sides separately. We may therefore, without loss of generality, consider only passages to the left in semi-infinite walks unbounded on the right, and we may take the states to be nonnegative integers. We now approach the following general question: given the sequence of transition probabilities p , p , • • • o f such a walk, to determine the index of first passages from right to left. This question is somewhat analogous to the question of determining the minimum integer k such that the series 2-,X. is convergent. As in the convergence of series, all that matters is the performance of \p\

in the tail —any

finite number of terms may be changed or suppressed without altering the result. We also have a "comparison" theorem. THEOREM 2. Consider two random walks, W and V, with transition probabilities { p. } and { q- |, respectively. If p. < q- for every i = 1, 2, • • • , then I(W)>I(V). Proof. We may assume that the walks begin at the same state. Refer both walks to the same infinite process Xi9 X2, ••• * where Xi9 X2, ••• are mutually independent random variables, each uniformly distributed over ( 0 , 1 ) . If the walk W is at state / for trial j , it proceeds to the right if and only if X. < p., and similarly for walk V, which proceeds to the right if and only if X. < q.. Consider Xl9 X2, ••• fixed at observed values xl9 x2, ••• . Observe that $ and V always differ by an even number and that this difference changes by either 0 or 2 on each step. We shall show that walk W can never be to the right walk V* If it were, there would have to be a first time on which this occurred, and on the preceding time, say trial iy the walks would coincide, say at state /. But p. < q., so that x. < p. and x- > q- are contradictory. As an application, we remark: if lim sup p. < 1/2, then the index is oc; and if lim inf p. > 1/2, when the walk is nonrecurrent. This fact follows from comparison with the classical walk with constant transition probability p. The fact just cited indicates that the interest will lie in those sequences I p.} for which lim sup p. _> 1/2 and lim inf p. <^ 1/2. We shall in the next section investigate a class of walks in which lim p^ = 1/2, and these will serve, together with the comparison theorem, to handle a large class of problems. 4. A class of random walks. We now consider a class of random walks which are related to the ordinary unbiased coin tossing. The random walk has as its possible states the integers r, r + 1 , r + 2 , • • • • Let 77

RECURRENCE-TIME MOMENTS IN RANDOM WALKS

pk = P\X(n) l-pk

= k + l } X(n-l)

= qk = P\X(n)

^

= k-l

=

131

k\,

j A(/i-l) = *},

= P{ going from A: to r for the first time in n steps !

(k > r).

Then ?M +i

=

Pfc?* + l.n + ?* 1k-i,n

(A > r ) .

The state r will be an absorbing barrier. This implies that 9r,0 = !» We shall set qr

?r,n

=

°

if

*

>

°•

= 0 for all n as a convenient convention. Note that while

?r + i.i - ?r + i»

?*,o-°

while

if

k>r

?i(1 = 0 i f

* > r + l ;

-

Define

Gfc(s)- E

%,nsn

(& = r - l , r, r + 1 , . . . ) ,

and observe that G r . l ( s ) = 0,

Gr(s)=l.

Let

P

^\{1~T\

9

*-i( 1 + i)

( A; = r, r + 1, • • • ) <

Note that r = 0 gives us the ordinary case of unbiased coin tossing. The generating functions satisfy the following equations: Gk(s)

= pksGk

+ i{s)

+ qksGkml(s)

Gr(s)

= P r « G r + l ( s ) + qrsGrml(s)

Let 78

(k > r ) , + 1 =qrsGr^(s)

+ 1.

132

J. L. HODGES, JR. AND M. ROSENBLATT

G(t,s)

s

= ]TGk(s)tk

\G(t>S)-tr

=

r

l

+ tG (t, s) + * + —\\ G{t9

s)dt-

f

t G(t,

s)-tr

We now differentiate with respect to £ (0 < £ < 1 ) , use [ l - ( s / 2 ) Q A +

dt

t)Y*1

as an integrating factor, and integrate the differential equation between the limits t and a = [1 - (1 - s 2 ) l / 2 ] / s , where 0 < t < o. We then obtain

(i)

- G (t, s )

-;(Hr ■ / *

s t r-2

+ rT r-i

-i(Hr

rfT.

Let C/Xi denote the cth factorial moment of the first passage time from k to r. It is easily seen that the c\i^ (c fixed), if they are finite, cannot increase with k more than exponentially. Therefore, if c\i, < oo for some positive c and some k > r, then k-c k=r is convergent for sufficiently small U Therefore

Hm

dc

G{t, s) = £

cH **"°

exists for sufficiently small t if and only if c/x, < oo for some k > r ( and hence for all k > r ) . It is now easily shown that the first r derivatives of the integral on the right of ( 1 ) with respect to 5 are bounded in the closed interval (ty 1 ) , and hence have finite limits as s —> 1~. The ( r + l ) s t derivative, however, contains a term with the factor do /ds, which diverges as 5 —» 1 — . The same techniques can be used to prove the following result: 79

RECURRENCE-TIME MOMENTS IN RANDOM WALKS THEOREM 3 . If one has a onesided

a>

0 ( A; = CC, a + 1, • • • ) then

while

f\Lk

random walk

133

with

< oo if r - 1/2 < a < r + 1/2 and k >

a,

cfi, = oo for c > r and k > CC •

We s h a l l now c o n s i d e r w a l k s with pn > 1/2. It i s c l e a r from Theorem 2 t h a t for s u c h w a l k s the index i s 0. T h e i n t e r e s t now a t t a c h e s to the q u e s t i o n of whether first p a s s a g e s are c e r t a i n to o c c u r . LEMMA 3 . 3 Consider p, _> 6 for some

a random

walk

e > 0. The walk is recurrent

* = l PXP2 is

over

the states

0, 1, 2, • • • •

if and only if the

Assume

series

•••?*

divergent. Proof.

C o n s i d e r first the finite walk over 0, 1, 2, • • • , n, and l e t Pj^'

de-

note the p r o b a b i l i t y t h a t the w a l k , s t a r t i n g a t k, r e a c h e s n before r e a c h i n g 0. Clearly

and

p n)

= pAnA +
l

We may s o l v e for P^n\

^(^

( n )

P$n\

(ft-i.2,....»-i).

••• , P ^

" * & > ) " Pk {P&\

in terms of P[n\

~ t ' )

getting

U - 1. 2, . . . , » -

1),

and h e n c e

P i » > - ( l +fc1+ 62 + ... where bs = (q q ••• q-)/(p

+

i 5fc . 1 )P 1 <»>.

p • • • p . ) . Now s u p p o s e 2Z^L fe. = oo. Since

3

We were informed by the referee that this lemma and the following theorem have also been obtained by T. E. Harris. 80

134

J. L. HODGES, JR. AND M. ROSENBLATT

PM-PM/

l

+

"£ bk < l / l / £ k=l

we must have P^n*—»0

b\t

k=l

as n—> co* ^ e conclude that the walk is recurrent.

Conversely, suppose 2Lk = i b, < oo. Then £ & = ! b, = b , where 6ft is some k ~ o' positive number. Using the fact that P<»> a t rc-2 P{nl n-\ = rpn-if + ^rc-l we have n-2 * =1

/

\

/c=l

and therefore pM

_ i

l

n-2

+ Z h + K-JPn-l k=l

Thus lim

P™ = 1/b

> 0,

whence the walk is nonrecurrent. As an illustration, consider a random walk over the nonnegative integers with reflecting barrier 0 and p*

2\

U = l , 2, . . . ; £ > 1 ) .

Bk/9

THEOREM 4. The walk is recurrent if and only if ft >_ 2. Proof. Let P^ = P \ walk never reaches 0, given that it starts at k \ ( k = 1, 2, • • • ) . We have 81

135

RECURRENCE-TIME MOMENTS IN RANDOM WALKS

p

' - ? K h l-3(l-jfirK.+5(I + £ K '

<*-*».->•

Rewriting, we have /3-1 „ L. P

P

„ pk-1 _ P = 11

Thus f / 3 - 1 j 8 - l 2)3-1 /iS-1 (ft-l)0-l\1 r, = r . | l + + • +•••+[ ••• L * M j S + 1 0 + 1 2)3+1 \ / 8 + l (A-l)jB + l/J Recalling the preceding lemma, we see that the recurrence of the walk is equivalent to the divergence of

1 +

0-1 j8+l

+

j B - 1 2 ) 3 - 1 / 8 - 1 2 j S - l 3y8^1 . + • • + ••• = / (p). j8+l 2)3+1 j8+l 2/3+1 3)3+1

We easily see that 1 1 / ( 2 ) = 1 + - + - + . . . =oo, 3 5 so the walk i s recurrent when j3 = 2. That it i s recurrent for /3 > 2 then follows from the lemma. The series / ( ) 3 ) ( 1 < /3 < 2 ) , may be shown to be convergent as follows. For 1 < )3 < 2,

r

/ 3 + 1 2/3+1 \

<1

+

/ 3 + 1 / 3/3+1 \

j8+l/\

2/3+1/

I+I(i-^Ul(i-^(i-^) + .... 2

3\

3 / 4 V

3 / \

5 /

But ( l _ ^ \ ( l _ ^ j . . . ( l _ i l ) <

c-(2-)8)(i/3

< e[-(2-/8)/4]log (2A+I) = (2A; + 1 ) " ( 2 " ' 8 ) / 4 . Thus 82

+ i/5 + . . . + i/aA+0

136

J. L. HODGES, JR. AND M. ROSENBLATT

1 1 + - + 2 3

f(B)
1 3(2-/3)/4

+

1 4

1

5 (2-/3)/4

+ •••
The preceding class of examples of random walks together with the comparison theorem (Theorem 2 ) now permit us to determine the moment properties of a large class of random walks. For example, the one-sided walks with p

n

= _ (l + 2

\

) n/3

(n - 1, 2, . . . )

I

have the same moment index as the walk with pn = 1/2 if (X > 0, /3 > 1, while if )8 < 1,

a < 0 the index i s infinity (all moments e x i s t ) . For a > 0, the walk

will be certain to return if j8 < 1, but not if /3 > 1.

REFERENCE

1. W. Feller, New York, 1950.

An introduction

to probability

U N I V E R S I T Y OF CHICAGO

83

theory

and its

applications,

Wiley,

A CLASS OF STATIONARY PROCESSES AND A CENTRAL LIMIT THEOREM BY J. R. BLUM AND MURRAY ROSENBLATT

1. Summary. A certain class of stationary processes is discussed. It is shown that each process in the class has an absolutely continuous spectrum. Under some moment conditions, it is shown that such processes satisfy the central limit theorem. 2. Introduction. Let rj = (• • • , r].x , rj0 , rjx , • • •) be a doubly infinite sequence of independent identically distributed random variables. Let T be the translation operator defined on rj by (2.1)

Trj = n' = ( • • • ,770 , i?i , ife , • • • ) .

Let B be the Borel field generated by v\ and let g = #(??) be a function defined on JS such that i?{ | gr j2} < °°. Given any such function gr, we define a stationary process {Xn} where (2.2)

Xn = g(rii),

n = 0,

± 1 , •••.

We assume without loss of generality that E{Xn} = 0. Let rs = i£{X n X n+s }. Then r8 = / I T eisXdF(h), where F(X) is the spectral distribution function of the process. In §3 the spectral distribution function of any process of the form (2.2) is shown to be absolutely continuous. Finally it is shown in §4 that under some additional assumptions on the moment structure of the process the central limit theorem is applicable. 3. Absolute continuity of the spectral distribution function. we prove

In this section

I. / / F(X) is the spectral distribution function of a process of the form (2.2), F(\) is absolutely continuous. THEOREM

Proof. In proving the theorem we assume that each % is uniformly dis tributed on the unit interval. It is easily seen that this may be done without loss of generality. For if £ is uniformly distributed on the unit interval, we can always construct a monotone function
84

74

J . R. BLUM AND MURRAY ROSENBLATT

For each integer n define a process {Xn,w}, m = 1, 2, • • • , by (3.1)

Xn>m

=

E{X

j rjn-m+l

n

j ' *'

j ??n + m - l } »

It is easily verified that for nfixed,the process {Xn, m} is a martingale. Further more # { | Xn,m |2} < E{\ Xn |2} = # { | g(rj)\2} < co. I t then follows from well-known theorems on martingales that Xnttn converges in mean square and with probability one as m approaches infinity. To see that lim^oo Xn>m = Xn , note that Xn,w is the projection of X n on the space generated by all measurable functions of rjn„m+1 , • • • , rjn+m-i . Since Xn lies in the space generated by the entire t\ sequence, it follows that lim^o, E{\ Xn — Xn>m \2} = 0. (See e.g. Dpob [1; 331, Theorem 4.3]. Let $i(u); j = 0, 1, • • • be an orthonormal sequence of polynomials where Oi{u) is of degree j on [0, 1] i.e., jl O^u) dk(u)du = Sjk where 8ik is the Kronecker delta. The functions {0,-(w)} are complete in L2[0, 1]. Let (ux , • • • , uk) be a point in the /c-dimensional unit cube Uk . The sequence of products {0jt (u^ • • • Bjk{uk)) with j1 , • • • , j f c nonnegative integers is a complete orthonormal system for L2[Uk]. To get an orthonormal system for the infinite-dimensional unit cube U we take the function which is identically one and the system of products {8u(urm) • • • 0jk(umk)} where mx < m2 < • • • < mk and k = 1, 2, • • • . This system is clearly orthonormal. Now let k be an arbitrary positive integer. From the discussion above it follows that we can expand X0,k in the form 2k — 1

(3.2)

X0.k = E ?=1

oo

E — k + l<mi<

E

Cfa , m,)^,.(^,)

• • •
where wy = (mi , • • • , m,), % = ( % , • • • , n ? ), and *s,(w,) = ft.,00 ••• 9ni(r,mi). Since E{\X0tk

|2} < £ { | X 0 |2} < -

it follows that

Z

E c% , m,)2 < «.

T,

3 = 1 m i < m 2 < * • *<mj

» i , • • • ,ttf=l

Now X 0 = lim^co X0,& with probability one so that 00

(3.3)

00

X0 = E

E

1=1 m i < " , < m j

E

C(&*

»ffidvssivsi)

wi,*»*,nj=l

with probability one Similarly we have 00

(3.4)

X„ = E

CO

E

E

j»»l m i < • • • < * » j n\ , • • • ,nj = 1

where m, — n = (m-^ — n, • • • , m, — n). Now letfcbe a positive integer, n = {nx , ♦ • - , % ) be a fc-tuplet of positive integers, and m = (mx , • • • , mk) be a fc-tuplet of integers with mx < • • • < mk ♦ 85

75

A CLASS OF STATIONARY PROCESSES

Given any integer /, we define Xt (m, n) by CO

(3.5)

Xt(m,n)

= £

C(fi> ™ + J ~ 0^»0&w)

J3-00

where m + I = (ml + ly • • • , mfc + I) for any integer Z. Xt(m, ft) is a moving average, and such processes are known to have an absolutely continuous spectrum [1; 499]. Now let (m', nf) be different from (m, n) in the sense that ft ^ n' or that m — m! is not a vector with equal coordinates. Then it is easily seen that the processes Xt(m, n) and Xt(in\ nf) are mutually orthogonal. There is a denumerable collection of such distinct processes and we enumerate them. Let {X\{)} be the i-tib. process and let / ( °(X) be its spectral density. Consider the process ]C;=i X*U) and let /„(X) be its spectral density. From the mutual orthogonality of the processes {X,(°} it follows that /n(X) = X X i / c o (X). Since the fio(\) are nonnegative, /n(X) is non-decreasing in n. Let /(X) = limw_*„ /n(X). Then we have f (3.6)

/(X) d\ = f

J —w

v — -jr

lim /m(X) dX < lim f' fn(X) dX n-»«>

n—><» «/ — TT

= lim#{| E * . ( ° \2} = E{\g\2} n~>oo

(J

»»l

< oo.

[ )

Hence /(X) is almost everywhere finite and integrable. An easy computation shows that /(X) is the spectral density of the process {Xt} and the theorem is proved. It follows from the theorem just proved that if h is any square integrable function defined on the probability space generated by X = ( • • • , X_x , X 0 , Zi , • • •) and if {Un} is the process defined by Un = h(TnX) then the process {Un} also has an absolutely continuous spectrum. It would be of some interest to determine if the converse of this proposition is also true. That is, if {Xn} is strictly stationary and if {X„} and any process {Un\ obtained from {Xw} in the manner indicated above has an absolutely continuous spectrum, does it follow that Xn is of the form g{Tny\) where g is square integrable and q = (• • • , 17-1 , rjo , Vi } •' 0 is a doubly infinite sequence of independent identically dis tributed random variables? 4. The central limit theorem. In this section it is shown that under some additional moment conditions, the processes we have discussed satisfy the central limit theorem. As before let Xn>k = E{Xn \ yn-k+1 , • • • , yn+k-i} when k = 1, 2, • • • , and let Xn>Q = 0. Furthermore, let Vn,k = X»th+1 - Xn>k, and for any integer s let a9thtk> = E{Vn,kVn+8tk>\. Then we have THEOREM 2.

(4.1)

//

E{\ Xn \a} < oo for some a > 2, 86

76

J. R. BLUM AND MURRAY ROSENBLATT

(4.2)

Z

E

!«..».*' I < « ,

s » —co &,&' ==1

and

(4-3)

E

S

a..».». * 0

*«= — co A ^ & ' ^ l

then N~* 23*-1 X* is asymptotically normal with mean zero and variance 2vf(0), where f(\) is the spectral density of the process {Xn}. Proof. For fixed k it can easily be verified that the process {Xn,k} is a stationary (2k — Independent process. Marsaglia [2] has given conditions under which such a process satisfies the central limit theorem. Let s%tk = E{\ /Lrt-i-Xi,* H- Then we can state Marsaglia's conditions in the way we shall use them: (4.4)

E{\Xn,k |2}
(4.5)

#-<£,

k = 1,2,

iV=l,2, .--,

* iV.fc

(4.6)

limy

v

^(|3r..>n 1 / i, 0 >

2V->oo

forsome

$tftk

We first show that these conditions are satisfied for k sufficiently large. Con dition (4.4) follows immediately from the fact that E{\Xnfk\2} < E{\g\2} < co. To prove (4.5) we write .2

M s §*■<)'

N ~ N

1

k-1

N

fc-1

iV-l

_

-I

= AT 23

Z

a*,i,j>(N - \U I),

where u = i' — L Hence 2

lim-|~ =

k-1

oo

2w 22 auti,i' •

Given hypothesis (4.3) we can clearly choose fc so large that (4.5) is satisfied. Jensen's inequality is used to obtain E{\ Xnk \a] < E{\ Xn \a} for all k% Con dition (4.6) follows immediately. Now let r{ = E{XnXn+i}, ri%k = E{XnfkXn+jtk], and let fk(\) be the spectral density of the process {XUtk}. Then

87

77

A CLASS OF STATIONARY PROCESSES

and

/*(X) = h E *v V x ^7T

4*.-oo

Hence IA(X)-/(X) I < | l E

k y . » - r , |.

Z7T / - - c o

For each,? we have lim^oo ritk = rf . Furthermore it is easily established that

E \r< | < E E I <*/.<.<' J>3o

3>/o

*,t'«0

and E

I ^.* I < E

j> jo

E

j>)'o

i , t '

a

I «/,<,<' I uniformly in fc. 0

Applying hypothesis (4.2) we see that 1 °° lim«I E |*V.* - r, | = 0. Thus it is clearly seen that lim*-,* fk(\) = /(X) uniformly in X. Since

/(o) = ^ E ^ 4 f

E «*.».»' > o.

we may choose k so large that /*(0) > 0 and so that Marsaglia's theorem applies. Then limPJ(2ri\r/ t (0))-* E X , - , ,
JV->co

V.

t' = l

/

/2

efot.

J—co

Now £•

isr

(tf/»(o))-* E (x, - x,.,,)

iV/ft(0) <

Njk{0)

a.

E E K„-

#

♦-1

j-&+i

JV

oo

2ij

2-/

i,*'-l j.j'-ft +l

I

a

i'-*,/,}'

where u = if — i. On letting iV approach infinity in the last sum we obtain -f

-f /A\

J k\yj

CO

^^

j,j'=k

CO

.Z^

+ l w=-oo

I ^u,3,j'

|»

Clearly we may make this sum arbitrarily small by choosing k large enough.

78

J . R. BLUM AND MURRAY ROSENBLATT

Since lim^*, fk(0) = /(0) we obtain

which is the desired result* REFERENCES

1. J. L. DOOB, Stochastic Processes, New York, 1953. 2. G. MARSAGLIA, Iterated limits and the central limit theorem for dependent variables, Proceed ings of the American Mathematical Society, vol. 5(1954), pp. 987-991. INDIANA UNIVERSITY

89

VOL.

MATHEMATICS:

42, 1956

M.

ROSENBLATT

43

3~ p , either F(nh) = 0 or $n(nh) = 0. Thus the sum on the right-hand side of equation (3) is identically zero. Combining equation (3) with equation (2), we have the desired result. * The research for this paper was supported by the United States Air Force under Contract No. AF18(600-685) monitored by the Office of Scientific Research. 1 A discussion of the problem along with the necessary references will be found in Harry Pollard, "The Harmonic Analysis of Bounded Functions," Duke Math. / . , 20, 499-512, 1953. 2 See ibid. 3 See S. Bochner, Fouriersche Integrate (Leipzig, 1932), p. 33. 4 The Levitan polynomial. See N.I. Achieser, Approximationstheorie (Berlin, 1953), p. 146.

A CENTRAL

LIMIT

THEOREM AND A STRONG MIXING

CONDITION

BY M. ROSENBLATT COLUMBIA UNIVERSITY

Communicated by P. A. Smith, November 10, 1955

Introduction.—This paper presents a central limit theorem for a sequence of dependent random variables Xi, A 2, . . .

The assumptions required are the usual assumptions on second and 2 + 6 order moments and a strong mixing condition. The theorem is of interest for two reasons. All general central limit theorems for dependent random variables formalize in some sense a heuristic notion of A. Markoff to the effect that one expects a central limit theorem to hold for Xh X2, . . . , if the random variables behave more like independent random variables the farther they are separated (assuming that ap propriate moments exist). An interesting discussion of this intuitive notion is given in S. Bernstein's paper on the central limit theorem. 1 The strong mixing condition used in this paper seems to he a more intuitively appealing formalization of this notion than most others. The condition is also of interest because it is a strong version of the mixing condition encountered in ergodic theory (see Hopf,2 p. 35). The Mixing Condition.—The mixing condition is assumed to hold for sets of the form akr < XkT < bkr,

r = 1, . . . , $,

(1)

where ki < . . . < ks. Given any two such sets A and B, introduce the following notion of distance d(A, B) between them. Consider the smallest interval contain ing the indices of the random variables in terms of which A is defined. Call this interval 1(A). In the case of set (1), this interval is [kh k8]. The distance d(A, B) is set equal to the distance between the intervals 1(A) and 1(B). Note that d(A, B) does not satisfy the usual properties of a distance; for example, one may have d(A, B) > d(A, C) + d(B, C) for some set C. The strong mixing condition is satisfied by the sequence of random variables Xh X2,..., if R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_12, © Springer Science+Business Media, LLC 2011

90

MATHEMATICS:

44

M.

ROSENBLATT

\P(A n B) - P(A) P(B)\
PROC. N. A. S.

B))

(2)

for all sets A, B of form (1), where f(n) is a function defined for n = 0, 1, 2, . . . , and decreasing to zero as n -► o°. 77ie Central Limit Theorem,—Assume that the mean values EX% = 0, as one could in any case subtract the mean values EXk from the random variables Xk, respectively. The assumptions are 1.

E

E j = a

Xtf^hib-a) I

as b — a —*■ co, where h(m) | E

E xj

2+5

«> as m - * » ;

= o(h(b - a) 1+ */ 2

as fe — a -> oo for some 8 > 0. 2. The strong mixing condition (2) is satisfied by the sequence of random variables Xh X 2 , . . . Introduce the following notation: j = 1

Ur=

rpn + (r -

£

l)qn

j = (r - D(Pn + qn) + 1 r(pn + fln)

Vr =

E

Xj,

** r - 1, . .

• , &,

J = r^n + (r — l)$n + 1

where fc(pn + n -^0asn-^ co. We expect to show that

Vkh(Pn) is asymptotically normally distributed. E

l '/2

k

Fr

E Vkh(Pn)

Now <

'i

(kh(qn) ~ A(p„))'A" Choose the sequences qn, p„, k such that kh(qn) h(Pn)

— 0.

Then k

E

* #1/!|Fr|2

VM( P B ) 91

Vkh(Pn)

VOL.

MATHEMATICS:

42, 1956

M.

ROSENBLATT

45

in probability as n -* °°. It can therefore be neglected in further computations. We shall later see that this can be done. Set Or. n(x) = P

Wkh{

= < 4, Pn)

'

and let the set Ur Vkh(Pn)

<mr8 < where mT is an integer. E (nit +...+

< (mr + 1)5> = A(r, n, mr, 5),

Note that

P(

fl A ( r , n , m , 5 ) ) ««) < e

Wkh{Pn)

(k/e)*. E r =

\,...,k

k

2

Ur

max

< E

Vkh(pn)\

E r = 1

t/r

VM(P„)

< h.

The result is obtained by an application of the Tchbycheff inequality. LEMMA 2

E

(m + . . . + mk)8 < x

P ( {) A(r, n, to,, &)) \r

= 1

/

- ( ! ' ) '

E

n

( m i + . . . + wu-)« < x r = 1

P(A(r,n,mr,6))\ |

/(?.) + 2e.

The probability contributed by all the sets f| A{r, n, mT) 8) for which max r = 1

I

/

1

*

| UT/vkh(pn)\

> tk is at most e. Consider the sets f) A(r, ny mr, 8) for which r = I

max | Ur/y/kh{pn) \ < tk. By repeated application of condition (2), one can see that ( fl Mr, n, mry 8)) - n P(A(r, n, mr, 8)) < kf(qu). \r

= 1

/

r = 1

k

Since there are (2tk/8) sets of this form, we obtain the desired inequality. Let Gi. n * • • • * Gk, n (X) 92

MATHEMATICS: M. ROSENBLATT

46

be the convolution of G\, n(x), . . . , Gk, n(x). Gi. n * . . . * Gkt n(x) <

PROC. N. A. S.

Note that

E

n

P(A(r, n, mr, 6))

(mi + • • • + mk)S <> x r = 1

/j\

+ kg)

and ( ? , , „ * . . . * < ? * . „ (x -

fc5)<

£

n

P(A(r, n, «,, a))

(mi + . . . + m* + *)« ^ * r = 1

/g\

Now (?i, n * . . . * Gk> n{x) tends to the normal distribution as n -* <» if fc —► oo. We want to letfc,p n , gn -► » f kpn ~ n, and kd -► 0 in such a way that

fc(f*)*/(g»)-o

(6) (7)

h(Pn) Set 5 = 1/fc2. Then A; 5 ->- 0. Since k

-

(;)"'

it follows that

j b ^ y -jb«/» + i ^<*«c*, where (7 = 2/61/l. The function / can always be chosen in such a way that /(n) > 1/n for all n. If k is chosen so that * < [-log/(?„)]*A,

(8)

clearly condition (6) will be satisfied. Now h(n) ~ E\ by the Minkowski inequality.

*»©

i = i

But then k2 h{n)

1 h(pn)'

Thus kh(qn) < k%{qn) KVn) ~ h(n) Condition (4) will then be satisfied if

h(qn) =

° (^r) 93

(9)

VOL.

42, 1956

MATHEMATICS:

M.

ROSENBLATT

47

Sequences k, pnj g„, -► °°, kpn ~ n can be chosen so that conditions (8) and (9) are satisfied and hence conditions (6) and (7) are satisfied. On using inequalities (3), (4), and (5) and Lemma 2, we see that *

U T rE = 1 Vkh(pn) is asymptotically normally distributed.

£ r

=i

Since

Vr Vkh(pn)

in probability as n —► °° y it follows that y/kh(pn) is asymptotically normally distributed. Remarks.—The result obtained includes the result of Hoffding and Robbins. 3 It would be of very great interest to see how much stronger the notion of a strong mixing condition is than that of an ordinary mixing condition in the case of a strictly stationary process. 1

S. Bernstein, "Sur l'extension du theoreme limite du calcul des probability aux sommes do quantity dependents," Math. Ann., 97, 1-59, 1927. 2 E. Hopf, Ergodentheorie ("Ergebnisse" series). J. Springer, Berlin, 1937. 3 W. Hoffding and H. Robbins, "The Central Limit Theorem for Dependent Random Vari ables," Duke Math. / . , 15, 773-780, 1948.

94

832

MURRAY ROSENBLATT

REMARKS ON SOME NONPARAMETRIC ESTIMATES OF A DENSITY FUNCTION1 By MURRAY ROSENBLATT2

University of Chicago 1. Summary. This note discusses some aspects of the estimation of the density function of a univariate probability distribution. All estimates of the density function satisfying relatively mild conditions are shown to be biased. The asymp totic mean square error of a particular class of estimates is evaluated. 2. Estimates of the density function. Let Xi, • • • , Xn be independent and identically distributed random variables with continuous density function f(y). Let S(y; Xi, • • • , Xn) be an estimate oif(y). The function S(y; xi, • • • , xn) is assumed to be jointly Borel measurable in (y, xi, • • • , xn). It is also assumed that S(y; xi, • • • , xn) ^ 0, since f(y) ^ 0. It can easily be shown that S(y; Xi, • • • , X n ) is not an unbiased estimate of/(y). Suppose to the contrary that (1)

ES(y,X1,*--,Xn)^f(y)

for all continuous / and all y. Condition (1) implies that for each y9 ES(y;Xl9 ••• ,X») < «>. Assume that S(y; Xi, • • • , xn) is a symmetric function of x±, • • • 9xn, since the symmetrized n-tuple (Xi, • • • , Xn) is a sufficient statistic for the problem. But then (2)

fsiviXw-tXJdy

is a symmetric estimate of F(b) - F(a) - [ f(y) dy. Moreover, (2) is an unbiased estimate of F(b) — F(a), since E f SiviXw-tXJdy**

f ES(y;X1} • • -7Xn) dy

= f f(y) dy = F(b) - F(a) Received April 27, 1955. 1 Research carried out at the Statistical Research Center, University of Chicago, under the sponsorship of the Statistics Branch, Office of Naval Research. The comments of R. R. Bahadur have been very helpful. 2 Now at Indiana University. R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_13, © Springer Science+Business Media, LLC 2011

95

833

NONPAKAMETKIC ESTIMATES

by Fubini's theorem. However, the only unbiased estimate of F(b) — F(a) sym metric in the observations Xi, • • • , Xn is Fn(b) — Fn(a)> where Fn(y) is the sample distribution function. This follows immediately from the fact that the symmetrized n-tuple (Xi, • • • , Xn) is a complete statistic [2]. Thus, Fn(b) - Fn(a) = [ S(y;Xl9 - - ^ X J dy for all a and b and almost all Xi, • • • , X n . But then Fn(y) is absolutely con tinuous in y for almost all X x , • • • , Xn , which is impossible. One need not require S(y;xi, • • • , xn) to be nonnegative. An assumption like ( E\S(y;Xl9---,Xj\dy<

oo

for some two values, a, 6, with a < 6, would lead to the same conclusion, that is, that there are no unbiased estimates S(y; Xi, • • • , Xn) of f(y) satisfying this condition. 3. The difference quotient of the sample distribution function. An obvious estimate of f(y) is the difference quotient S(y;Xl9~;Xl

= /»(») =

Fn(y + fc) - Fn(y ~ h) 2h

of the sample distribution function Fn(y), where h = hn is a function of the sample size n and approaches zero as n —> °o. The asymptotic behavior of this estimate as n —> <*> is examined in terms of its mean square error. Fix and Hodges have used an estimate of this form in their discussion of a nonparametric dis crimination problem [1]. Now, EFn(y) = F(y)9 E[Fn(y)Fn(yO) = \ F(min (
F(y)F(y')>

so that cov (Fn(y),Fn(y'))

= £ [F(min C?/, 2/0) -

F(y)F(y%

But then, cov (fn(y),fn(y'))

= - ^ [F(min (y + h,y' + ft)) - F(y + A ) W + h)

- F(min (y + h, y' -h)) + F(y + h)F(y' - h) - F(min (y - ft, y' + h)) + F(y - ft)F(y' + h) + Fdnin (y - ft, y' - ft)) - F(y - h)F(y' - h)}. On setting y — y', Afniy))

= | ^ [F(y + h) - F(y -h) 96

+ (F(y + h) - F(y - h))%

834

MURRAY ROSENBLATT

Now consider the behavior of fn{y) where y is fixed as n —> <*> and h —* 0. The mean square error E\Uv) - f(y)\2 = *2(f«(y)) + (MfM - f(v))2 1

4h?n

[F(y + h) - F(y - h) + (F(y + h) - F(y - h)f] 12

+ [ ^ (F(y + h) - F(y - A)) - f(y)~J is a reasonable measure of how good an estimate fn(y) is of f(y) locally at y. The density function / is assumed to be sufficiently regular for the following evaluation of the mean square error to be carried through. It will be enough to assume that the first three derivatives of / exist at y. Then 1

Jy-h v—h ry+h

= /

f(u) du \f{y) + (u - y)f'(y) +

§(u-y)2f"(y)+0\u-yf}du

= 2hf(y) + \f"{y)ti + 0(\h\l). Assume that/"(y) 7* 0. Then

(Efn(y) - f{y)f ~ ( | f"(y)J = |

\f"(y)f

as h —» 0. The variance of the estimate

as h —> 0. The asymptotic mean square error

(3)

B |/. W - fy)f ~ f + | | / - W p + » (I + »•)

as & / —> 0 and ft —> <*>. The question of an optimal choice of h = hn as a function of ft now arises. If & / is set equal to krTa, a > 0, it is easily seen from (3) that the optimal choice of a is a = -§-. The optimal value of k is then the one mini mizing

This value of k is n/5

- ["£ _J»LT L2|/"(t/)|2J 97

NONPARAMETRIC ESTIMATES

835

With this choice of k and a, we find that

E\UV) - f(y)\2 ~

Wu&2-i,5f(yfY(v)rnil6.

The choice of k would be based on guesses as to the magnitude of f(y)} f'(y). One is led to a choice of h as a function of n that is independent of y by considering a global measure of how good /„ is as an estimate of /. The inte grated mean square error

C

E\fM-f(y)?dy

J— oo

is a simple measure of this type. Let f(y), f'(y) be bounded continuous functions that are square integrable. One is then led to the following asymptotic expression

£ »i/.w - m\< * ~ -L + 1 £ \rm> * + . ( £ + »«) as h —*■ 0 and n —> «>. The optimal choice of h as a function of n is h = hrTm, where k is no v 9

& =

1 1/5

2 f\rw*v J—QQ

and

IJlUiy)

- M\* dy ~ 2- 4/5 9- 1/5 |[£jf (y)|2 dyj n"4'5

a s n - ^ oo. 4. A class of estimates of the density function. The discussion of the previous section suggests that the following class of estimates will be of interest. Let wn(u) be a nonnegative function such that /

Wn(u)

du

The sequence of functions {wn(u)} is chosen so that the total mass concentrates in the neighborhood of zero as n —-> oo ; that is, given any e > 0, /

J

\U\<€

wn(u) du —» 1

as n —> oo. Corresponding to each sequence of weight functions {wn(u)) of this type, there is an estimate oo

/

-j oo

wn(y - u) dFn(u) = - Z) wn(y - X,-). 98

836

MUEEAY EOSENBLATT

Now /»00

Efniy) = /

/»0O

J—00

wn(y - x) dF(x) = / w„(y - x)f(x) dx, v—00

and cov (fn(y),fn(yf)) = -

/ w»(y - aOwn(y' — #)/(#) cfo

71 L J-oo

— (J

^n(2/ ~ X)f(x) dX

On setting y = ?/', we have
£ wl(y — aO/G&) dx —ij

wn(y — 3)/G*0
.

Note that all estimates of this form are themselves density functions; that is'

Uy) ^ 0, and / fn(y) dy = 1. J—00

An estimate fn(y) with any desired regularity properties can be obtained by choosing a weight function wn(u) with these same regularity properties. Thus, fn(y) will be analytic if wn(u) is. As an example, consider

wn(u) w

=i (i)>

where h = hn—>0asn—><*>, and / w(t&) rfu = 1. J—00

The estimate discussed in the previous section is obtained on setting w(u)

= /* \0

when |ti| < 1, otherwise.

The function w(u) is assumed to be square integrable. Then
99

f(y).

,

837

NONPARAMETRIC ESTIMATES

It is clear that J w(u)f(y + hu)

du-*f(y),

and that / w2(u)f(y + hu) du -» f(y) I w2(u) du as h —> 0 for every continuous density function /. Hence,
J w^u)

du

-

The integral / w(u) \u\z du is assumed to be finite, and f(y) is assumed to have continuous derivatives of the first three orders in the following computation of the bias. The bias of the estimate is then Efniy) - f(y) = / w(u)[f(y + hu) - f(y)] du = hf(y) J w(u)u du + %h2f"(y) f w(u)u2 du + 0(\h\z). It is now clear that it would be advantageous to have / w(u)u du = 0. This condition will be satisfied if w(u) is symmetric about zero. Using the same sort of argument as was used in the last section, it is easily seen that the mean square error of these estimates can be made no smaller than 0(n~4/5) for all ad missible /. It would be very interesting to find out whether there are other esti mates fn(y) with an asymptotic behavior of the order 1/n as n —» op. REFERENCES [1] E. Fix

AND J. L. HODGES, J R . , Discriminatory Analysis, Nonparametric Discrimination: Consistency Properties, USAF School of Aviation Medicine, Project No. 21-49004, Report No. 4. [2] E. LEHMANN, "Notes on the theory of estimation,'' University of California, Berke ley (1950).

100

U.S. copyright law (title 17 of U.S. code) governs the reproduction and redistribution of copyrighted material.

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_14, © Springer Science+Business Media, LLC 2011

101

SOME REGRESSION PROBLEMS IN TIME SERIES ANALYSIS MURRAY ROSENBLATT1 COLUMBIA UNIVERSITY AND NEW YORK UNIVERSITY

1, Introduction Estimates of the regression coefficients which are unbiased and linear in the observa tions are discussed in this paper. The residual is assumed to be a stationary process* Two specific estimates are discussed, the least-squares estimate and the Markov estimate. I call the estimate which is computed under the assumption that the residual is an orthogo nal process the least-squares estimate. The Markov estimate is the linear unbiased esti mate with minimal covariance matrix. The basic assumptions made in the paper are dis cussed in section 2 and are held to throughout the paper. In section 3 some remarks about the approximation of a continuous positive definite matrix-valued function by finite trigonometric forms are made. These remarks are used in section 4 to obtain the main results about the asymptotic behavior of the covariance matrices of the leastsquares and Markov estimates. The next section discusses the many interesting cases in which the least-squares estimate is asymptotically as good as the Markov estimate. The first really systematic discussion of some of these problems was given by IL Grenander [1]. Further work was carried out by U, Grenander and M. Rosenblatt in [2], [3], and [4]. The author considers some of these problems in the case of a vector-valued time series in [5]. Some of the results of this paper are a generalization of some of those ob tained in [5]. A few cases in which the least-squares estimate is not asymptotically efficient in the class of linear unbiased estimates are discussed in sections 5 and 7. Some small sample computations for a linear regression with a residual which is a first order autoregressive scheme are carried out in section 6 to test the asymptotic theory. 2. Assumptions and notation I assume that the observed process yt is a vector-valued process (a ^-vector) (2.1)

yt^xt+mt,

J-

•-•, — 1, 0, 1,«",

where ntt = Eyt is the mean value sequence and xh Ext ss 0, is the sequence of residuals. The residual xt is assumed to be weakly stationary, that is, the covariances (2.2)

rt T = rt-T=E%tx!r=E(yt-

md(yr-

m7)!

2

depend only on the difference / — r. For mathematical convenience, in sections 3 and 4, I assume that the components of the vector observations are complex valued. The realBased in part on research supported by the Office of Naval Research at the Statistical Research Cen ter, University of Chicago. 1 Now at Indiana University. 2 Xt is column vector. Given a matrix At Af denotes the conjugated transpose of A. 165 102

166

THIRD BERKELEY SYMPOSIUM: ROSENBLATT

valued case is the one of statistical interest and will be examined in detail later on. The mean value mt is assumed to be of a regression form (2.3)

WjBSs/ 5 i ^i) +

/ 5 2 ^) +

... +

/5i^),

where the regression vector sequences (2.4)

^

l

^

>

' " , ^

>

i = l,-",* ,

are assumed known and the regression coefficients ft., - • •, & are unknown. I shall discuss the problem of estimating the regression coefficients by unbiased estimates linear in the observations (2.5)

yu'-^yn.

Two specific linear unbiased estimates will be discussed in some detail, the "leastsquares" estimate and the Markov estimate. The least-squares estimate is obtained by treating the residuals as if they were orthogonal, that is, (2.6)

£#i#T=

St.rl

where Sit r is the Kronecker delta and I is the identity matrix, The Markov estimate is the optimal linear unbiased estimate in the sense of minimal covariance matrix of the estimate. The covariance sequence rt of a weakly stationary process has the representation

(2.7)

rt** Heil*dF{\)

where F(X) is a nondecreasing matrix-valued (* X *) function, that is, AF(X) g> (X* I shall assume that F(\) is absolutely continuous, that is,

(2.8) so that (2.9)

FM = Pfitidfi rt=

/*V*/(X)dX.

The function F(X) is called the spectral distribution function of the process while /(X) is called the spectral density of the process. The spectral density is a nonnegative function of X since ( 2 ,o,

,c»-^*o.

I shall assume that /(X) is a continuous function of X [each element of /(X) is a con tinuous function of X] and that/(X) is nonsingular for all X. For convenience I introduce the following notation. Let

yJ » Given a square matrix A, A 2:0 means that the corresponding quadratic form is positive semidefinitfi. A >Q means that the corresponding quadratic form is positive definite. 103

TIME SERIES ANALYSIS

167

Let (2.12)

)

and (2.13)

p: \
Equation (2.1) can then be rewritten in the form (2.14)

y~
The matrix R is the covariance matrix of y (or %). Here

r

(2.15)

0=1

The matrix R is nonsingular since /(X) is continuous and nonsingular for all X* The ma trix
(y-vfi'(y-vfi)

and is given by (2.17)

pL=

foVl-Vy.

It is clearly an unbiased estimate and has covariance matrix (2.18)

EtfL-p)(0L-p)'-

[
The linear unbiased estimate with minimal covariance matrix or Markov estimate is given by (2.19)

fo,=

[f/JK-irf -V'JK^y •

Its covariance matrix is given by (2.20)

£(ftif-ft < & £ - « ' = [ ^ - V l - 1 -

These remarks on the least-squares and Markov estimates are well known. The techniques used in the paper can be considered a sort of generalized harmonic analysis. In order to carry out the analysis, various assumptions on the asymptotic be havior of the regression vectors are introduced. These assumptions are broad enough to include most of the usual types of regression, such as polynomial and trigonometric re gression. They do not include the case of exponential regression, Let (2.21)

*» = j j ^ p V p , 104

i - i , — ,*.

l68

THIRD BERKELEY SYMPOSIUM: ROSENBLATT

It is assumed that (2.22)

*«->»

as n —> °°, i = 1, * • *, s. It is also assumed that ,(i)

Hmfe-1 $(0

(2,23)

for every fixed A. Consider the ^-vectors

(2.24)

! > • ' ■ > *

^0 =

*-l.

■>

» .

^ CO

Let the limits 0)

(2.25)

exist,y, 2 = 1,-- • •, 5 and p, q = 1, ■ • •, k [if t < 0 set gp4(/) = 0]. Set (2.26)

iM*~

[nn&'*

\

*, « = i , - " , M

and (2.27)

Af*-{„Ar»;

i , / = l,•■-,*}.

The matrices Mh, h = • ■ •, —1,0, 1/ •■ form a positive definite sequence, that is, given any finite collection of ks-vectors {Op]

(2.28)

^

a M

'" >~» ° * ^ ° •

It then follows that the matrices MA have the representation (2.29)

14=

fe^dMW

where ifcf (X) is a nondecreasing matrix-valued (ks X fo) function of X, In accordance with the notation introduced in (2.26) and (2.27) I write (2.30)

M(X) = 1/iATCX);

j , Z — 1,---,J}

and (2.31) Note that (2.32)

yjHf (X) - {^AfM(X);

# , ? = !,•••,*}>

■Miiff(X) = {M^ 5 (X);

;, J =!,•••, * } ,

J£ 0 « f*dM{\)

=MW

-M(-v).

It will be convenient to introduce some additional notation. Given a k X k matrix/ and a form M = iififhvi hj — V • •, J ; «, v «= V • -? &}, let (2,33)

( / -M) = | 2 ) fuwinii; a, j

105

«, i> - ! , • • • , J £ J

TIME SERIES ANALYSIS

169

By the integral (2.34)

/ " [ / ( X ) -dJf(X)]

I shall mean (2.35)

j r V fu (X) ^Jlfiy (X);

#, q - 1, • - • ,s\ .

The matrix (2.36)

T = (/-Mo)

is assumed to be nonsingular, This means that the vector sequences (2.37)

v®>

i=-l,'--,jf

are asymptotically linearly independent in a sense which is relevant in this context. Con ditions (2.22) and (2.23) are introduced to ensure that the estimates of the regression coefficients converge to the true regression coefficients in the mean square. It will also be convenient to introduce the matrix 11 ■SCi) ' 0 1■ ft

(2.38)

Dn =

0

'sw 1 "

3. Remarks on approximation The arguments used to obtain results on the asymptotic behavior of the covariance matrices of the least-squares and Markov estimates are approximation arguments. They make use of iinite trigonometric polynomials that uniformly approximate the matrixvalued spectral density /(X). LEMMA 1. Letf(\) be a continuous positive definite matrix-valued (k X k) function of X. Given any e > 0, there is a positive definite matrix-valued trigonometric polynomial

(3.1)

g (X) = J £ g.«** U——p

with coefficients gu Hermitian k X k matrices such that (3.2)

*z'z> s'[/(X) - g ( X ) ] %> - e * ' *

for every k-vecior z. There are finite trigonometric polynomials gu(X) such that (3.3)

ga(\)

= g;*(X)

and (3.4)

\fijW

-gijOO

\<8

>

S<0 ,

where ij = V • •, h But then on setting g(X) « {gy(X)j i,j = V ' •, &}, (3.5)

H'[/(X)

-g(X)] z^ ]£§h<|

|2/l

If 5 > 0 is chosen sufficiently small (3.6) *'[/(X) - g ( X ) ] a<e«/* . 106

^kSz'*-

170

THIRD BERKELEY SYMPOSIUM: ROSENBLATT

One can show similarly that (3.7)

- « « < * ' [ / ( X ) - g ( X ) ] z.

Note that p is the maximal order of the polynomials pij(X). LEMMA 2. Lelf(k) be a continuous positive definite matrix-valued (k X k) function of X. Given any e > 0 sufficiently small, there are positive definite matrix-valued trigonometric polynomials p •u=—p

(3.8) A(X) = £ > «
0 < / ( X ) - e I < g ( X ) < / ( X )
where I is the identity matrix (k X k). Since /(X) is a positive continuous function of X, for all sufficiently small e > 0 (3.10)

0
and clearly (3.11)

/(X) - « 7 < / ( X ) .

By lemma 1, there is a trigonometric polynomial g(X) such that (3.12)

-ieZ
But then (3.13)

0
One can similarly show that there is a trigonometric polynomial A(X) such that (3.14)

/(X) < * ( X ) < / ( X ) + € 7 .

LEMMA 3. Let

(3.15) /(X) - ^ 0 + ^ cos XH

h«p cos £X + b\ sin X + - - * + £ P sin p\

be a positive definite matrix-valued (k X k) function of X with the coefficients av, b» (k X k) matrices, Thenf(\) can be written in the form

(3.16)

/(X) = ^ ( S ^ ^ m ) ( £ ^ ^ A )

where cQ is nonsingular and

(3.17)

f V " x ( ^ cie-iA''ld\ = 9

where I > 0. Since/(X) is a positive definite matrix-valued (& X *) function of X, there is a A-di107

T I M E S E R I E S ANALYSIS

1H1

mensional weakly stationary process xh Ext s 0, having /(X) as its spectral density. The covariance matrices (3.18)

rt*aaJBxt+TiXr'=

/ " V V ( X ) d\

are the null matrix if |*| > p. L e t Pxt be the projection of xt on the closed linear mani fold spanned by %t~i} « w , " ** Consider nt = *h - P#*. The ^ are orthogonal to each other, that is, Er\t^rT = 0 when t ^ T\ Since &* and 77* are stationary I can write (3.19)

Xt=

re**dZ*W,

*,= r^MZ,(X)

where 2X(X), 2„(X) are processes with orthogonal increments EdZx (X) dZx (ix) ' - «„„/ (X) dX j EdZv ( X) iZ„ (M) ' = __ S^Nd-k.

(3.20)

Here iV = J S ^ / . Because rt = 0 for t < ~p it is clear that (3.21) %t = fy*HZA\) ~£hsVi-j = fe**(£hje-**)dZ,(\), and &0 = 7, On approximating the characteristic function of the set [— TT, X] in the mean square by linear combinations of the exponentials exp ii\ equation (3.22)

f*dZmW

«

r^he^HZ^X)

is obtained. On taking the covariance matrix of both sides of equation (3.22)

(3.23)

F.(X) =—£ (J2 hjer**yN(£h,r<*yd\

or

(3.24)

/(X )

= * ( £ A,*-"*WjSft,,-'*)'

is obtained. Since/(X) is nonsingular, the matrix N must be nonsingular. Norm the yfe so as to get (3.25)

^-jy-va^.

The |e's are an orthonormal process, that is, I can now write

<3'27>

^=± C y^-y 3-0

where c0 = #V*. By using the argument that led to equation (3.22) one can see that

108

172

THIRD BERKELEY SYMPOSIUM: ROSENBLATT

Since £< is weakly stationary (3.29) (3.30)

£t = f_y*dZ(

(X) =£/* ( J £ Ci*-**yldZu

(X)

£dZ«(X) <*£«(/*)'-8*.— IrfX.

By our construction & is in the linear manifold spanned by xh #e-i>* •' so that r^m(Y\

(3.31)

cje-*A~ld\ = 0

when I > 0, 4. The asymptotic covariance matrices In obtaining the asymptotic form of the covariance matrices of the least-squares and Markov estimates, it will be convenient to deal with (4.1) DnE(^L-p)^L-^)fDn^

DnW^^DnD^RipD^Dnl^^^Dn

and (4.2)

DnE(pM-f3)

(fa- /?) ' D n - Dn toft-Vl - i A .

THEOREM: 1. Under the conditions on the spectrum of the process %i given in section 2 and the conditions on the regression vectors specified there

(4.3)

timDnEtfx,-p)(pL~P)'Dn=2TT-i

/ " [ / ( - X ) -dJIf (X) ] r ^ 1 .

The conditions on the regression vectors specified in section 2 imply that (4.4)

D^WtiD^T

as »—* oo and the limiting s X s matrix T is nonsingular, I therefore need only consider the asymptotic behavior of (4.5)

Dn*
Given any sufficiently small e > 0, by lemma 2 there are finite matrix-valued (k X k) trigonometric polynomials

(4.6) MX)

=~y>„e^

such that (4.7)

0 < / ( X ) - « 7 < g ( X ) < / ( X )
Let

TIME SERIES ANALYSIS

173

I shall obtain the limit of (4,5) as n —> <» when the covariance matrix of % is G. The matrix n

(4.9)

v'tii-rVr.

t, r - 1

A typical element of (4.9) is therefore of the form

(4.10)

!LZ±

.

[$G>)$(8)] 1/2

Since there are only a finite number of nonzero gu's, the limit of expression (4.10) is (4.11) 2a-V rgiii-Xid^vM. Thus l i m ^ V , G v ^ 1 = 2 7 r / " [ g ( - X ) -dM (X) ] .

(4.12) Similarly

lim Dn1?'E
(4.13)

-dM(\)].

On letting e —> 0, it is clear that lim Dn^RcpDZ1 ~ 2r r[f(-\)

(4.14)

-dM(\)].

Note that the expression

jC[/(~X) -^(X)]

(4.15)

is nonsingular for/(X) > el if € > 0 is sufficiently small But then fV[f(~\)

(4.16)

-dlf(\)]

>ef*[I
=e(/-M0)

which is nonsingular. THEOREM 2. Under the conditions on the spectrum of %t and the regression vectors as sumed in section 2 (4.17)

lira DnE(pM-l3)(!3M-l3)'Dn=2Tr(

rif~H-\)

-dM(.\)])~\

By lemmas 2 and 3, for every sufficiently small e > 0 there are finite trigonometric polynomials p

g (x) = u=0 2 *•*-*"'

(4.18)

h (A) = ]£} ^ e_iuX M= 0

with go, /zo nonsingular and satisfying relation (3.17) and such that 0
-eK7i-g-1(X)g^(X)/
IT

110

174

THIRD BERKELEY SYMPOSIUM: ROSENBLATT

Let G, H be the covariance matrices of the vector oc if the process xt has the spectral densities WWrWrW > (l/27r)^-1(X)/ri(X)/, respectively. In obtaining our result it will be sufficient to consider the asymptotic behavior of D~i
0 < 'G-V .

Assume for the moment that #< has the spectral density {l/2ir)g-l0C)fl{\Y and con sider the asymptotic behavior of D i r V G ^ A T 1 as »-+ °o4 Now carry out a GrammSchmidt orthogonalization procedure with respect to xh • ■ •, #„ starting with #i, The first p equations are ^ n # i = £i (4.20)

^ 2 i % + ^22^2 = ?a

The residual £'s are orthonormal. At the next step I have (4.21)

&*i + • • • + &>tfp+i = £*+i *= ifo+i

since #* satisfies the difference equation (4.22)

#*-

*?*

with the rjtS orthonormal. From then on we have the equation (4.23)

gP%k-\

Let the matrix

h go *«?+!»= h+p— Vk+p J

«

!

,

■

w-

du. 0

diid^i

(4.24)

*

dpidP2

A =

' ' dpp

ip & ~ i * - gi go

&

'

' g2 g l gQ

Then (4.25)

A*=£ =

where £ is a vector of orthonormal random vectors. A is a nonsingular transformation taking a v •, #n into fi,* * •, £w. On taking the covariance matrix of both sides of equa tion (4.25) I have (4.26)

AGA' = / 111

TIME SERIES ANALYSIS

175

so t h a t

14.2 7)

G~i = A'A.

Let {g„y, u,v=

1,- ■ •, »} = G-\ Then oo

1

(4.2 8)

^M =2^+^"+« « = — CD

if p or ji is greater than £. Here gv is understood to be zero if v < 0 or v > p. But CO

(4.29)

2

^

«;+•&.+««^r^-^(X)'jf(X)rfX.

Now the (£, $)th element of D^vG^ipD^1

is

Here ym - ( 1 / 2 T ) / V**(X)'*(X)I*X. Now 8n is the sum of at most ip2 terms of the form

Since every element of gz}* is smaller in absolute value than Sr1 for some small but fixed 5 > 0, the elements of (*)'

-i

(a>

converge to zero as ^—> oo, But then

(4 33)

'

Jg»'' »<»»») V«

=

2 ^ £ S U(-X)-g(-X) l^M^CX).

I have now shown that (4.34)

H m D - V G - V ^ 1 ^ / " U ( ~ ?0'g ( ~ X) .
In the same way one can show that (4.35)

l i m Z ) - y # - V Z t I a s s /"* [A ( - X ) ' A ( - X ) - d t f ( X ) ] .

On letting e | 0 the desired result (4.36)

lim^ViJ-V^-T^-rt/^C-X)

-rfJf(X)]

is obtained. The matrix (4.36) can be seen to be nonsingular by using the argument used at the end of the proof of theorem 1. 112

176

THIRD BERKELEY SYMPOSIUM: ROSENBLATT

5, Asymptotic efficiency It is interesting to investigate those types of regression for which the least-squares estimate is asymptotically efficient in the class of linear unbiased estimates for all ad missible spectra/(X). In most cases the covariance matrix R is unknown and a reasonably large sample size is required to get adequate precision in estimating it. For this reason it would be convenient if one could use the least-squares estimate instead of the Markov estimate since the least-squares estimate does not require knowledge of R. Even if R is known, it may be difficult to compute R"1 which is required for the Markov estimate. In view of the results already obtained, the least-squares estimate will be asymptotically efficient if (5.1)

r - i f [ / ( - X ) -dM{\)]T-ir[f-n-W

-
for all admissible /(A). The case of interest is that in which the process %% and the regres sion vectors have real components. Because of this/(X) and M(X) must satisfy the addi tional restraints (5.2)

/CXW(-X)

When k = 1, asymptotic efficiency of the least-squares estimate has been discussed in U. Grenander [1] and U. Grenander and M. Rosenblatt [2], [3]. In the one-dimensional case it is convenient to set (5.3)

r(X) - M ( X + ) - A f ( - A - )

and rewrite (5.1) in the form (5.4)

T"1 f*f

(\) dTMT-i

Hf-HD

dT{\)

=1.

Equation (5.4) is satisfied for all positive continuous /(X) if and only if T(X) increases only at a finite number of points 0 g Xi < * ■ • < Xg g w, q < s, and the jumps (5.5)

2\«AT(X<)

satisfy the relations TiT-lTj~8ijTi.

(5.6)

These conditions are satisfied if one has a polynomial regression (5.7)

w i - M — ' + A-i*1-1,

a trigonometric regression (5.8) mt « ft cos JXH

V &h cos t\8i + p,1+1 sin 2Xi+ • • - + ft^+a, sin t\$t

(with the points X* distinct), or more generally a mixed polynomial and trigonometric regression q

(5.9)

»« = £

Uz^i

a 1 U

S

v*=i

A..*""1 c o s ^ + £

q

u^i

„* 2 li

S i f c , , * ' - 1 sin«X„ v=*i

(with points X* distinct) [3]. Obviously the sine terms in this last regression form dis appear if \i = 0. Notice that these regression sequences include most of those used in 113

TIME SERIES ANALYSIS

177

standard statistical work. It is easy to construct a regression sequence where the leastsquares estimate is not asymptotically efficient. Consider (5.10)

w< = /?(a 0 + #icos JXH

hflfcCos*Xft),

0 < X i < - - - < \* > where the constants ai are known. Such a regression has a form similar to the pulse trains encountered in communication theory. In the case of such a regression T(\) in creases only at the points 0, Xi, • ■ •, X*. The jump of T(\) at 0 is aQ

(5.11)

z

1=1

and the jump at X^ j = 1, • • •, k, is i^2

(5.12)

1

h

* ?'=i

The asymptotic efficiency of the least-squares estimate #L in the class of linear unbiased estimates is

(5,13)

g1 .

In section 7 I shall discuss the question of how much additional information about the spectrum/(X) is required to construct an estimate with the same asymptotic mean square error as the Markov estimate. In the case of multidimensional time series, new phenomena arise. Considerfirstthe case of a polynomial regression. If each component of the time series has a polynomial regression, that is, (5.14)

imt^y^ifaP-1,

i=l,<••,£,

the least-squares estimate #& is still asymptotically efficient. However, if the different components have polynomial regressions of different orders, the least-squares estimate is no longer asymptotically efficient [5]. A simple example is that in which the mean value of the first coordinate of a two-dimensional time series is unknown while the mean value of the second coordinate is known to be zero, Then (5.15)

mir=z^(pt)

^ = (0/'

The function M(X) increases only at zero and the jump at zero

114

1

7§

THIRD BERKELEY SYMPOSIUM: ROSENBLATT

Thus

fUi-V, (5.17)

-dM(\)}

=/u(0)

•;-* / * [/- 1 ( - X W M (X) ] = -

fn(0)

/n (°) /» <°) ~ /?« CO)

The asymptotic efficiency of the least-squares estimate is (5.18)

1

/l2 (0)

/u(0)/«(0) '

If there is a mixed polynomial and trigonometric regression (5.9) not only must the same regression form occur in each component, but one must also have xsu = 2su if X„ ■* 0. Thus, the least-squares estimate will be asymptotically efficient in the case of a regression (5.19)

W i - f t ^ + ft^' + ft^ +

ft^

but not in the case of a regression

(5.20)

mi

= ft p? + ft v? = ft (COS0'X) + ft ( c o s ° J ,

X* 0.

It is worthwhile examining this last regression in a little more detail. The function M(X) increases at two points, X and — X. The jumps AAfii(X) = A M U ( - X ) = ( * jj) (S.21)

A4f»(X) = AM12(-X) =(2 *) AM21(X) = A M 2 i ( - X ) = ( ^ Q ) (5.22)

AM22(X) = A M 2 2 ( - X ) = ( Q ° ) . Thus

(5. (5. 23,

£i/(-x).« ( x„=( Re ; ;: ;»

£/«<»)

and md (5.24) / l l r ^ - X ) -cZM(X)] 1 / / » (X) / u (X) / « ( X ) - | / H ( X ) |» V - Re / „ (X)

- Re / u (X) \ /1X (X) >*'

It is worthwhile noting that one does have asymptotic efficiency of the least-squares esti mate if Im /w(X) = 0, In the multidimensional case k <£ 2 set (5.25)

2V(X) = { r - V 2 / , M ( X ) T - ^ ; 115

j , 1= 1, • . . , * } .

TIME SERIES ANALYSIS

179

Then equation (5.1) can be rewritten (5.26)

T [ / ( - X ) -dNW)

/ " [ / - K - X ) -dN(\)}

« J.

I have not been able to get simple necessary and sufficient conditions on N(\) for equa tion (5,26) to be satisfied for all admissible/(X). However, one can get simple conditions of this type when N(\) is known to increase only at zero. This corresponds to the inter esting case of a polynomial regression. Equation (5.26) can then be written as U*N)(f~l
(5.27)

«J,

where f = /(0) >0

(5.28) and

N=AN^0.

(S.29) Set (5.30)

Ntt-

{ptNa;

,s]

P,q=l,

and (5.31)

19* J .

WU-NU+NH,

Since/ = /(0) and N = AiV(O), the elements of / and N are all real THEOREM 3. Letf and N be positive definite and positive semidefiniie symmetric matrices, respectively. The equation (/•iV)(/- 1 -iV r ) « f

(5,32)

is valid for all positive definite symmetric f if and only if h j =
NuNti = SUNH ;

k ,

2 Nu = I (5.33)

WuWik = Nu Wjk = Wih

j ^ k ,

Nu, i^k,

Wu Wki = 0 , WijNM=NkicWi}-

= 0,

I;

j?*k,l; i,

J9*k.

Consider first the case in which/ is a diagonal matrix

(5.34) Equation (5.32) then becomes (5.3 5)

/=

\ °

\ J

,

^liNu^tfNij-I. 116

X,- > 0 .

186

THIRD BERKELEY SYMPOSIUM: ROSENBLATT

This equation is valid for all positive X* if and only if (5.36)

NuNjj=

StjNu

a n d ^ i V i i = L Since N is positive semidefinite and NUNH — SijNa, it follows that (5.3 7)

WijNkk^NkhWu-O

if i,j T* k and (5.38)

WijWki^O

if i 7* k, I and j ^ k, L Now consider the case in which

(5.39)

/ - ( / a fz J

and is positive definite, On differentiating equation (5.32) with respect to/a twice, equa tion (5.40)

WU^Nu+Nn

is obtained. On differentiating equation (5.32) first with respect to ft and then with respect to/2, equation (5.41)

NiiWw=WiiNn

is obtained. Now let (5.42)

/ l /2 fz J h /4 /B 0 / fz /B /« 0 i>

and be positive definite. Differentiate equation (5.32) with respect to ft, f2, and/g. I now get (5.43)

MuTii**TiJTn.

All the other equations (5.33) are obtained by taking some subscripts other than 1, 2, 3 or interchanging subscripts. By using the conditions (5.33), one can readily verify that equation (5.32) is satisfied with any positive definite symmetric/ 6. Some computations It is worthwhile looking at a process with a stationary residual of a special and simple form to see how good the asymptotic theory considered is for finite samples. Consider a process (6.1)

y« = *< + jSi + ft*

where the residual xt is a first order autoregressive scheme with covariances (6.2)

'.~T?r-3' 111

- K p < l .

181

TIME SERIES ANALYSIS

Here p is the correlation coefficient of the scheme. The regression coefficients ft, £2 of the scheme are unknown and to be estimated. The covariance matrices of the leastsquares estimate f}L and the Markov estimate i3M are considered. I have already noted that (6.3)

E (pL - p) (0* - jS) ' - (*V) ~VR
~l

and (6.4)

( & * - « ' « (^i?~V)~ : <

E{fa-P)

Here when the sample size is n (6.5) so that (6.6)

(*V> - 1 -

/2(2»+i) ' n(n — 1) 1

- 6

i

« ( » — 1) 12 «(» 2 — 1)

- 6

\»(M — 1) Straightforward but tedious manipulations lead to

(n— l)n

(«-2)(l-p)H-2(l-p) (6.7) <prR-lv = (re—l)w. -(1-p)2

(1-P)»+(»+P)(1-P)

(»-l)»(2n-l)

d-p)!

+«2(l-p)-p2+»pJ

+ <»+p)(l-p) One can similarly show that n (l-p)» (6.8)/Ity-

»(»+!) 2(l-p)«

2p(l-pB) (l-p«)(l-p)»

w(»+l) 2(l-p)»

(«+l)p(l-pw) (l-p2)(l-p)2

(»+l)p(l-p») (l-p»)(l-p)»

*»(»+!) ( 2 » + l ) 6(l-p)«

'*

»(»+!) P

(1 + P ) ( 1 - P ) 3

, 2p2(l-pn+1) s (I+P)(I-P)* ' (i+P)(i-p) ; 2/*»+«(»+1)

The covariance matrix as given by the asymptotic theory is

(6.9)

I (l-p)M

1n -6 n2

«2 12 nz

to the first order. The (i}j)th elements of the covariance matrices of both the least-squares and Markov estimates, % j = 1,2, are given in Table I for the sample sizes n = 10,15, 20, SO and cor relation coefficients p = — .8, — .6,- • •, .8. The approximation suggested by asymptotic theory is also given. 118

TABLE I COVARIANCE MATRICES OF THE LEAST-SQUARES AND MARKOV ESTI MATES (AND AN ASYMPTOTIC APPROXIMATION OF THE COVARIANCE MATRICES) OF A LINEAR REGRESSION, RESIDUAL FIRST-ORDER AUTOREGRESSIVE MATRIX ELEMENTS p

(1, 2)-(2, 1)

(i, i)

(2, 2)

»=10

+ .2

(a) (b) (c)

0.65196 0.64406 0.62500

-0.091312 -0.090046 -0.093750

.016602 .016372 .018750

-.2

(a) (b) (c)

0.35898 0.34109 0.27778

-0.052116 -0.049425 -0.041666

.0094753 .0089863 .0083333

+ .4

(a) (b) (c)

0.99189 0.94826 1.11111

-0.13464 -0.12785 -0.16667

.024880 .023245 .033333

-.4

(a) (b) (c)

0.29685 0.27749 0.20408

-0,043812 -0,040614 -0.030612

.0079656 .0073844 .0061224

+ .6

(a) (b) (c)

1.69108 1.54892 2.50000

-0.21501 -0.19421 -0.37500

.039090 ,035311 .075000

-.6

(a) (b) (c)

0.27142 0.22345 0.15625

-0.040923 -0.032950 -0.023438

.0074402 .0059909 .0046876

+ .8

(a) (b) (c)

3.48150 3.17040 10.00000

-0.35878 -0.32391 -1.50000

.065229 .058893 ,30000

(a)

0.32541 0.18379 0.12346

-0.051326 -0.027257 -0.018518

.0093318 .0049559 .0037037

-.8

(h)

(c)

« = 15

+ .2

(a) (b) (c)

0.42883 0.42456 0.41667

-

.040942 .040469 .041667

.0051178 .0050587 .0055556

-.2

(a) (b)

0.21958 0.21716 0.18519

-

.021498 .021227 .018519

.0026873 .0026534 .0024692

(a)

0.68983 0.66268 0.74075

-

.064546 .061576 .074075

.0080684 .0076970 .0098767

(a) (c)

0.17508 0.16642 0.13606

-

.017363 .016383 .013606

.0021704 .0020478 .0018141

(a) (b) (c)

1.29297 1.18142 1.66669

-

.11604 .10428 ,16669

.014505 .013034 .022223

+ .4 -.4 + .6

(c)

(b) (c)

(a) Least-squares, (b) Markov, and (c) Asymptotic.

182

to o

oo

CT*P

crP

Q*P

CNCocn s e n rf*.

CN cncn

O.CNJ--.

O O O

ONCnC

ON ON ON ON 0 0 0 0

0 0 0

§ 0O 0O

O00h»

ON C O t ^ ONCnCo ^rtoco

-<*ON\0

OOO

Cn J-*. Co O >£* »-*. O O O

ggg

_ 5 0 ON COrfx CN ON >£*

OOO OOO

>o ggg

tototo cn en o s O C N H* OCnlo

O O O

o

M

CAVOQN

Cnoooo Cntocn

CnCnCn

OOO 0 0 0

o

+

Mi

tofcoiss tO 1— *->■ tO tf^-00 tOrf^O

poo

o

+

O O O O O O 0 0 0 if^Oi-sr ON c o s tO t O S \OOVO ONONrf^

O N * - * Co

t o OO O N

O O O 4^ Cnoo

OOO

I l l

tOJ^OO OOO

cn i-»- vo O 0 0 Co

*
O O O Co *-»• >-»■

CnCO -
]

O O O O

O O O O O O O O O CnONS OOCnCn CnQOCO VOOOO CnOOCo

Cn»—O NO 0 0 Cn CnCoOO

OOVONO

O O O O O O cnON-
Ml

tOCOOO ON>f>.

O b ►—• ^'"OO oototf^

O O O

ON

O O O O O O NO ON ON Co t o 0 0 >JOCn CnON^ OONCn

ovoo

* > 4 M NO cnONCN

OOO

M

O H ^ O

t o NO O CnCnCo OONOO

♦-»• O >-*■

ON

+

S^S? t^pw 0000

Q ***. 0 0 S f ° ^ ^ ^ S

^P?&

CnbOtO

O O O O O M ^ ^ ^ ^tfe^

bo

+

O O O OO o s

eC n s s o O O Otos

O O O O O O SCO ON C

soo
O O O O O O ON s t o CnOO O Co O O S O*>*00

III

rfi.rfi.H*

ocn£»

I-* t-L K * O *-*• t o toooco

O O O

rf^ Co co >-»• if* cn ON4*. 00 ONJ^S SCn(-».

OOO 0 0 0

O O O rf^CO CO •—■ ON s ON J-^CN CN ON Cn S S N ©

I I I

C N C N ^

cn\o-
O O O CnCnCn Cn o t o Cnvo«
H M K O O »-^ rf^vo 0 H* 00 «o SCo s

O O O 0 0 0

*-»• C o c n s j N J O

s—* >-* J—«l O M l-A ^ Cn C N

O O O

I I I

VOO«
oorfs.^

O O O H-> H-> K->. Oo Cn cn ooc\~o;

rf*00O Co t o co O O H ' CO

888

Co »~* Co 00 to 0

O O O to to to CO tO CO

I I I

Co Co Co t-^ t-^ )-* tOONVO cn-vjtfx O O O

OOO

+

to

u

a

1 00

+

Cn c o Co

ON S H - ^ ON CN ON ON-O;-O; S O N ON

C \ t o co

I

1—*.

Ococn lpM-*tO *-^Cnto S N O I-*

1-1»—».

0 0 0

^s&

ON

ocoo

rf^-oco

NO t o Cn s t f s ^

O O O O O O t-L ►— tO

O O O O *-^ H* 0O O C N tO Co rfs. CO NO O 0 0 0 VO Cn

o^#-

00 t o 00 VOCWO

O O O 00 t o co O O C N O

CNts>NJ CNKt^ 0 > 0 < t S O >—■ CnOOCn

{—»• J - i 1—k.

oNOOOrfi* osro

0 0 t o bo

0 * 0t—»•0 1—». H Co ON VO

O O O

rf* O C o V-* t O ON S C O Co

O O O

1 1 1 ! I I 1 i1

cn

0 J—»• *-* OOOcn t o ON 0 0 COCN W* O C n s

O O O

£*E& &&£,

C>/-v/^

00

8.

to

1

^ 1 ^ u g ) i? ^* to J^ « $* V a to 1-3

3

^ »H*

"O

5

184

THIED BERKELEY SYMPOSIUM: ROSENBLATT TABLE I—Continued MATRIX ELEMENTS

(l, i)

p

(i, 2 ) » ( 2 , 1)

(2,2)

n«50-—Continued

8t

0,044081 0.043302 0,040816

— ,0013209 .0012933 .0012245

(c)

0,46697 0.44575 0.50000

-

.013595 .012856 .015000

,00053312 .00050415 .00060000

(a) (b)

0.035245 0.033457 0.031250

— .0010643 .0010010 ,00093751

.000041737 .000039256 .000037500

+ .8

(a) (b) (c)

1.61368 1.43485 2,00000

— .045419 .039365 .060000

.0017810 .0015437 .0024000

-.8

(a) (b) (c)

0.031137 0.026625 0.024691

-.4

+ .6 -.6

(a)

j (c)

-

,00095752 .00079766 .00074074

.000051799 ,000050719 .000048979

,000037549 .000031281 .000029629

7. Some special examples In section 5 a few special but interesting types of regression sequences were considered where the least-squares estimate of the regression coefficient was not asymptotically efficient in the class of linear unbiased estimates. I now consider two of these regression sequences to find out what information about the spectrum /(X) is required to construct an estimate of the regression coefficient with the same asymptotic mean square error as the Markov estimate. The first example is that of a one-dimensional process (7.1)

yt=*t+P
where xt is stationary and (7.2)
V&k cos t\k ■>

0 < Xi< ••• <X fc .

Note that

(7.3)

®n = 2 V*~n C^ + ^ S aH '

An estimate of 0 which has the same asymptotic mean square error as the Markov esti mate is

for (7,5)

Ep*~(3 + 111

0(~)

TIME SERIES ANALYSIS

185

and

(7.6)

^(iB»)~lf-4^ + l V

ai.2

\

\ -l i

Notice that the only information about the spectrum/(X) required for the construction of jS* is knowledge of the ratios

The second example is a two-dimensional process (7.8)

Vt=**i + P
with #* stationary and

(7.9)

„-(J).

An estimate of /3 which has the same asymptotic mean square error as the Markov esti mate is

where iy<, 2;y* are the components of yt. Note that (7.11)

Jg|3*«p

and

„I(^)~i[/„(0)-|HW].

8. Final remarks There are many interesting open problems. It is clear that one ought to be able to obtain analogues of the results obtained thus far in the case of a continuous time parame ter. It is likely that such a program would require heavier tools. The results obtained thus far have an immediate implication for various types of nonstationary processes, specifically processes which are integrals or sums of stationary proc esses. Consider as an example

(8,1)

Zi«2>

where %t is a stationary process. Results on estimation of regression coefficients with Zt as a residual can be obtained from corresponding results with %i as a residual. A much more detailed investigation of specific types of regression sequences would be worthwhile pushing through. It is worthwhile noting that all the main results obtained can be derived for processes with a vector time parameter in the same way, I wish to thank M. de Groot and the computing staff of the Statistical Research Cen ter at the University of Chicago for carrying out the computations in section 6, 122

186

THIRD BERKELEY SYMPOSIUM: ROSENBLATT

REFERENCES fl] U. GRENANDER, "On the estimation of regression coefficients in the case of an autocorrelated disturb ance," Annals of Math. StaL, Vol. 25 (1954), pp. 252-272. [2] U. GRENANDER and M. ROSENBLATT, u An extension of a theorem of G. Szego and its application to the study of stochastic processes," Trans. Amer. Math. Soc.t Vol. 16 (1954), pp. 112-126. [3] , "Regression analysis of time series with stationary residuals," Proc. Nat. Acad. Sci.} Vol. 40 (1954), pp. 812-816. [4] , A monograph on time series analysis, to be published. [5] M. ROSENBLATT, "On the estimation of regression coefficients of a vector-valued time series with a stationary residual," to be published in Annals of Math, Stat.

123

Some Purely Deterministic Processes MURRAY

ROSENBLATT

1. Introduction. The linear prediction problem as it arises in the case of stationary processes has attracted much attention. The problem has been studied most intensively when the prediction error is known to be positive. I t is of some interest to consider a few simple situations in which the prediction error is zero, especially as a situation of this sort arises in NEUMANN'S theoretical model of storm-generated ocean waves [3]. I shall, unfortunately, not be able to discuss the problem that arises in the context of NEUMANN'S model in any detail. Let xt , Ext = 0, be a weakly stationary process, that is, Ttfr

:=:

xLXtXT

==

Tt—T

depends only on the time difference t — r. Our process is assumed to be realvalued. The time parameter t may either range over the real numbers or it may range over the integers. The first case is the continuous parameter case and the second is that of a discrete parameter. Examples of both continuous parameter and discrete parameter processes will be discussed. The discrete parameter processes discussed will be of much greater interest than the continuous parameter processes. 2. Preliminary Remarks. If xt, Ext = 0, is a weakly stationary process, it has a random Fourier representation of the form (1)

xt = [

eitXdZ{\),

EdZ(\)dZ(fij

J — 00

=

8x,dF(X)

in the continuous parameter case — °° < t < °°, and a random Fourier representation (2)

xt = /

eia dZ(\),

E dZ(\) dZ(fi) = 5XM dF(X)

in the discrete parameter case t = • • • , — 1, 0, 1, • • • . Here 8Xfi is the Kronecker delta (l

if

[0

if 801

K =\

X = ix X 4= A*-

Journal of Mathematics and Mechanics, Vol. 6, No. 6 (1957).

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_15, © Springer Science+Business Media, LLC 2011

124

802

M. ROSENBLATT

Note that dZ(\) is the random amplitude of the harmonic e,x. The covariances

(3)

rt = [ eiadF(X),

- « < t < „>,

J — oo

in the continuous parameter case. In the discrete parameter case (4)

rt = f\iadF(\),

t=

•■• , - 1 , 0 , 1 , ••• .

F(\) is a bounded nondecreasing function and is called the spectral distribution function of the process x, . Note that dF(\) is the variance of the random amplitude dZ(\), dF(\) = E \dZ(X)\2. The case of greatest interest is that in which F(X) is absolutely continuous, that is,

TO) = f /GO dn J •— 00

in the continuous parameter case, and

TO) = fX /GO <*M J —T

in the discrete parameter case. The function /(X) ^ 0 is called the spectral density of the process xt . In the linear prediction problem, the parameter t is thought of as time. The process xt has been observed over the time t S r and one wishes to predict s time units into the future (t = r is thought of as the present) by a predictor that is linear in the observations. Thus, one wishes to predict xT+a, s > 0 by a predictor pxT+a which is linear in the observations xt , t ^ r. The desired predictor is the one that minimizes the mean square error of prediction E \xT+8 — px T + S l

2 •

Note that this prediction problem is somewhat unrealistic as it assumes knowledge of the entire past of the process. Nonetheless, in certain situations the solution to this problem will lead to reasonable procedures in the more realistic problem in which only a finite amount of the past of the process is known. A process xt is called "purely deterministic'' if the mean square error of prediction is zero in this idealized problem, that is, if one can predict perfectly by linear techniques when the entire past of the process is known. Some specific discrete parameter purely deterministic processes will be discussed. A finite amount of the past of the process xt, r — m S t S r, is assumed known. The prediction error in these cases is positive since only a finite amount of the past is known. The rate at which the prediction error decreases to zero as m —» oo is obtained. The discussion of these discrete parameter processes is based on the helpful comments of G. SZEGO and some computations he made. 125

PURELY DETERMINISTIC PROCESSES

803

3, Continuous Parameter Processes. Let xt, — oo < t < <», be a continuous parameter process. Assume that its spectral distribution function F(X) satisfies the condition

f

(5)

et]h] dF(\) < oo

for some t > 0. It is then clear that the process xt is purely deterministic (see J. L. DOOB [1] p. 584). Let the fcth derivative of xt , if it exists, be denoted by x\k\ The derivative x\l) is the limit in the mean square of (xt+h — xt)/h as h —> 0, that is, the random variable x such that Xt+h

E

0

as h —> 0. Condition (5) implies that J1)

x 2) exists. The derivative x\ is defined similarly in terms of x™. By proceeding recursively in this manner one can define x(k) in terms of x\k~1). Condition (5) implies that all the derivatives xik) exist and are given by Ak)

f

(tX)V' k dZ(\).

Let rm

(6)

dF(X) < oo

T > 0.

This implies that x?\t -

E

T)'

fc!

if | * - T | < 42", or that (7)

a;.

z

^r

(< — r )

fc!

if I * — -r I < f I7. In particular, if x(Qk\ k = 0, 1, 2, • • • , is known, _

^X(0k)tk fc = 0

7

A'i

if | f | < I? . Given that we now know xt , | t \ < fl 7 , by applying (7) to £ values in the range \t\ < f T we see that x« is determined in the range \ t\ < T. By applying formula (7) repeatedly xt is determined for all t Thus, knowledge of Xok), k = 0,1, 2, • • • , w/ien condition (5) £s satisfied, determines the past and future of xt completely. This means that knowledge of the process xt over any small interval \t\ < €, e > 0, determines the full history of the process. This is unreal126

804

M. ROSENBLATT

istic as in practice there is always some noise perturbing the process of interest so that condition (5) is not satisfied. Nonetheless, the study of this idealized situation implies that in practical work one might be able to use a truncated power series expansion with approximations of the derivatives inserted in the expansion. The point at which the power series is truncated and the type of approximation of the derivatives to be used (one might use difference quotients) would be dictated by some a priori knowledge available about the form of the spectrum of the process and the perturbing noise. 4. Prediction Error for Discrete Parameter Processes when the Spectral Density is Zero on an Interval. Let xt , t = • • • , — 1, 0, 1, • • - , be a discrete parameter weakly stationary process with mean value Ext s= 0. The process is assumed to have an absolutely continuous spectral distribution function with spectral density /(X). I t will be convenient for us to let X vary from zero to 2w instead of •— w to x. Further let /(X) be continuous and positive on the closed interval [Jx — a, §?r + a] and zero elsewhere. Thus /(X) is zero on a set of length 2w — 2a. This may disturb the reader a little as such a spectral density corresponds in general to a complex-valued process since it is not symmetric about T, and real-valued processes are of interest. However, this can easily be remedied by adding \-K to X. This convention is held to because of the convenience in the later discussion. The prediction error for such a spectral density is the same as in the corresponding problem with X shifted by \>K units. We would like to consider /(X) as a function on the unit circle | z | = 1 in the complex plane. This can be accomplished by setting /(A) = / ( ! log (eiX)) = W(eiX). W(ea) is now a positive continuous function on the arc z = e*x, ^w — a ^ X ^ f x + a, of the unit circle. The error in predicting x0, given that X—\ , X-—2 y * * * f X-n are known, is o-» =

min

E \xo — X) cix~i j - i

min f \einX - i > / ( " - , ) X /(X) T+ a /»fh 77r-r<

min 5 l » ' ' * ,Cn

I " §

7r

~

>*wX -

a

2>ie'(n~'

- j ) X

d\

Tf(eiX) dX.

2-1

This means that the prediction error a\ is equal to min f ^

| r f ' x ) | 2 W{eiX)

where pn{z) is a polynomial of degree n in z with the coefficient of zn one. In our problem the process xt is purely deterministic since 127

PURELY DETERMINISTIC PROCESSES

805

f log f(\)d\ But we want to study the rate at which the prediction error a2n approaches zero as n —> oo. In evaluating the rate at which <J\ —* 0 as n —» oo we will lean heavily on some results of G. SZEGO that are given in Chapter 16 of his monograph Orthogonal Polynomials. The crucial results will be given in our discussion. SZEGO considers the following problem. W(z) is a positive continuous function defined on a Jordan curve C in the complex plane. C is the boundary of a simply connected region T in the complex z-plane containing z = °o as an interior point. Let L be the length of the curve C. Now let pn(z) be a polynomial of degree n in z with the coefficient of zn one. One wishes to obtain ~fc\pn(z)\2W(z)\dz\

min

over the class of such polynomials. In our problem the curve C is the arc of the unit circle e*x, JTT - a ^ X ^ \ir + a, and PF(e a ) = /((I/O log (etX)) on this curve. T consists of all the points of the plane off the arc. Define the inner product of two functions h(z), g{z) where z is on C by

(Kg) =±foh(z)j®W®\(k\. If the system 1, z} z2} • • • is orthogonalized by the Gramm-Schmidt orthogonalization procedure, one obtains an orthonormal set of polynomials 0»(s), jr J n(z)(j>m(z)W(z) \
n, m = 0 , 1 ? 2 , • • • ,

where <j>n(z) is a polynomial of degree n with the coefficient of zn real and positive. Let kn be the coefficient of zn in 0»(s). Then 1 _ L __ 2a 2 <Jn

-

j 2 /V n

7,2 /€n

in our case since L = 2a. Let 2 = 17G-O = aix + a0 + ai/T 1 + a2/x""2 + • • • be the analytic function, regular and one-valued for | /x | > 1, which maps | ju | > 1 conf ormally onto the region T, preserving the point at infinity and the direction there. The function r] (M) is uniquely determined and a is called the capacity of the curve C. Let JJL — N(z) be the inverse function of z = rj(jx). Now let W(z) be the positive continuous function on C. Then W{r)(e~%e)} is positive and continuous on the unit circle ju = e%\ 0 ^ 6 ^ 2 T . Let D(TFn; M) - exp { ^ £ '

log [ r T { i > ( O H } * % - « * } • 128

806

M. ROSENBLATT

On substituting into the analytic function D(Wrj; ju) the function fx = {N(z) } ~ l we obtain the analytic function A(z). A(z) is regular in T including z = <». Moreover A(z) 4= 0, A( » ) is real and positive, and lim \A(z)\2 -

W(z0)

where z0 is a point on C and 2! approaches £0 from within the region T. SZEGO reduces the minimum problem on the curve C to the problem on the full unit circle by the mapping given above (see pages 355-356, 270 of SZEGO [4]). We now cite two interesting results of SZEGO (see pages 365-367 [4]). If A(z) is regular in the closed exterior of C (the closure of T) (8) when z is in the closure of T. Moreover

Now let us see what we can derive from these general results in our problem. In our problem *?(M)

2 a . cos r sin . a, . 2 2« . /A sm - + i cos - ~1 2 2 . fi + .i .sin

a s: 2 a-

maps the region | ju | > 1 onto the complex 2-plane cut by the circular arc z = e ,x , §7r — a ^ X ^ ^TT + a, 0 < a < IT. The function -(£ — z) — */(£ — z)2 + 4d sin2 ~

/i = TO = 2sm Here A(oo) = D(TT^; {iV(co)}"1) = D ( T ^ ; 0)

= exp{^J o " log [FM
exp \ - |

r 27r

2it

xlogTf

sm ~ + le I
where

W X ) = /(X),

<* ^= X ^

129

§7T + GJ.

te

PURELY DETERMINISTIC PROCESSES

807

Note that the capacity of the arc in our problem is a = sin fa. Thus

(

\2n +l

sinf)

2

.

By using (8) we can also obtain information about the orthogonal polynomials n(z) as n —> oo (or equivalently the predictors). However, we shall not pursue this topic any further. We have already evaluated the asymptotic behavior of the prediction error and we see that it approaches zero as fast as the (2n + l ) t h power of a positive number less than one. Note that the result obtained can be used to obtain upper and lower bounds for the rate at which the prediction error approaches zero when the spectral density is a continuous function that is zero on a set of positive measure and positive on a set of positive measure. It is clear that in such a case the prediction error approaches zero as fast as the (2n + l) t h power of a positive number less than one. 5. Prediction Error for Some Discrete Parameter Processes Having Spectra with a High Order Contact with Zero. The spectral density that NEUMANN (3) suggests as the theoretical spectrum of a storm-generated ocean surface is positive except for one point. However, the order of contact of the spectral density with zero at this one point is so high that the corresponding stationary process is purely deterministic. Thus far, no one has been able to discuss the prediction problem with such a spectral density in any detail. G. SZEGO has suggested looking at a discrete parameter weakly stationary process xt, Ex% ss 0, t = • • • , — 1, 0,1, • • • , that has a spectral density positive everywhere except for one point. At this one point it has a very high contact with zero that is not exactly of the same character as that shown by NEUMANN'S spectral density. Nonetheless, it is close enough to give some small insight into the asymptotic behavior of the prediction error. The computations of this section were carried out by G. SZEGO. Throughout this discussion we shall refer to SZEGO'S book Orthogonal Polynomials and a paper of his on POLLACZEK'S polynomials Let the spectral density of the process xt be (2X~ir)
/(X; a) = W (cos X; a) |sin Xj = jf(-X;a) = /(X;a),

cos X(7np(X))

0 £ X S r,

where /A N
actn X —r—

Here a is a fixed parameter. Note that 7T

/(X; a) ~ 2 exp <j - -.--rf |sin X 130

808

M. ROSENBLATT

as X —> 0 so that /(X) has a very high order contact with zero at X = 0. Let n(z) — k„zn -f- • • • + ln be the orthogonal polynomial of degree n with respect to /(X), z = e*'\ that is J —r

We know that the prediction error one step ahead, given observations at n successive time points, is 2_

i w

The evaluation of the asymptotic behavior of k~x as n —> °o is required. The asymptotic behavior of &n is evaluated by studying a closely related family of orthogonal polynomials on the interval ( — 1 , 1). Let g(x, z) = #(cos X, z) = X) ^»<>; a> n

(9)

- (1 - se f y* + < * ( X ) (l - aT**)-*-"™,

x - cos X,

be the generating function of the polynomials Pn{x) of degree n. Let Pn(x) = pnXU +

Pn(x).

n +

The polynomials pn(#) are orthonormal with respect to the weight function (2X-r)^(X)

W(x; a) = IPX cos X; a)

cosh (TT
on the interval — 1 S % ^ 1. Replace z by Z/E in formula (9) and let x •—> «> (see [5] p. 731-732). One then obtains

E

^p .

-rzn = (1 - 22)-* exp ^a X) ~ 2 m ~7 = (1

2s) - / 2

But then

(•

+

P» ^ — - > = ( - 1)a -f- 1 i

a + 1 2 2n =

rin + a

+ A

a+ 1

T(n + 1) 2n (a-l>/2 » a + 1

so that Vn

2"

la

l a+ 1 n .

131

PURELY DETERMINISTIC PROCESSES

809

Also p„(l) ~ nl exp {2 Van] (see [5] p. 733). Now let g» = qX + • • • be the orthonormal family of polynomials with weight function (1 — x*)W(x; a) on the interval — 1 2* x ^ 1. Then it is known that q„(x) = cQn(x) where Qn(x) =

Pn(x) pa+3(x) 1 -x-

= p„+2pn(l)x" + ••• = ^ f p „ ( x ) + •••

pJX) P.+a(l)

(see [4] pp. 28-30). Now J 1 (1 - x%)Wix){Qn{x)f dx = f = ^

Pn(x)

Pn+2(X)

Pw(l)

P»+a(l)

Tr(x)^pw(l)pw(a!)dx

p»(l)p-«(l)

so that C= J ^

bn(l)Pn+2 (l)]^

and

The orthonormal polynomials pn(x), qn(x) can be expressed in terms of the orthonormal polynomials n(z), x = \{z + z"1), z - e'\ Pn(x)

= (2*r*{l + ^ } " l { 2 " 0 a . ( « ) + ^ n ^ " 1 ) }

(see [4] p. 287). Thus pn = (2ir)-*(l + ~2^) V(fc2n + l2n) = (2T)-i2"/bL(&2„ - W 1 and

=ss

\

I

"

«^2n+2V^2n+2

132

*2n+2/ •

810

M. ROSENBLATT

By (11.3.6) on p. 283 [4] we know that \ln\ < kn . But i

n2

=
Thusfc2n~ Y». Now &2n~i ~ T» • Therefore h

^

—

r

n*«

and al ~ n 6. Conclusion. It is worthwhile contrasting the asymptotic behavior of the prediction error al for the processes dealt with in sections 4 and 5. In section 4 the spectral density is positive and continuous except for an interval of length 2w — 2a3 T > a > 0, where the spectral density is zero. The prediction error al approaches zero geometrically

(sin|J

\2n+l

.

In section 5 the spectral density /(A; a) is positive away from X = 0 and has a very high order of contact with zero at X = 0 /(X; a) ~ 2 exp | - j | r j |sin X|. The prediction error al approaches zero as n —> <» but at a slower rate than (10) 2

„-

an ~n

This paper was prepared for the Office of Naval Research under contract Nonr 285(17) at New York University, It may be reproduced in whole or in part for any purpose of the United States Government. REFERENCES [1] J. L. DOOB, Stochastic Processes, John Wiley, New York, 1953. [2] U. GRENAKDER, On Toeplitz forms and stationary processes, Arkiv for Matematik 1 (1951). [3] W. J. PIEESON, JR., Wind Generated Gravity Waves, Advances in Geophysics, Vol. 2, Academic Press, New York, 1955. [4] G. SZEGO, Orthogonal Polynomials, Amer. Math. Soc. Colloq. Pub. Vol. 23, 1939. [5] G. SZEG5, On certain special sets of orthogonal polynomials, Proc. Amer* Math. Soc* 1, pp. 731-737 (1950). 133

Functions of a Markov Process that are Markovian M. ROSENBLATT*

Summary. In this paper we are primarily concerned with discrete time parameter Markov processes {X(n)}, n = 0, 1, 2, • • • , with stationary transition mechanism. The processes {Y(n)} = {f(X(n))} generated by a given manyone function / and the processes {X(n)} with a fixed stationary transition mechanism are constructed. The processes {Y(n)} are in a one-to-one correspondence with the possible initial distributions of {X(n)\. The object of the paper is to determine conditions under which {Y(n)} is Markovian, whatever the initial distribution of {X(n)}. Necessary and sufficient conditions for the new processes {Y(n)\ to be Markovian are obtained under the assumption that the family of measures corresponding to the fixed transition mechanism (of {X(n)}) is dominated [4]. The conditions are expressed, of course, in terms of the function /(•) and the transition mechanism. Generally the processes {Y(n)} do not have stationary transition mechanism. The conditions simplify in the case of a continuous time parameter Markov chain. Some of the discussions may at times have the flavor of those used in considering the concept of sufficiency [4]. Some aspects of the problem discussed in the paper are touched on in [2], Basic Concepts. Let Abe a space of points x and (B a Borel field of A sets. The sets consisting of single points {#}, x e A, are assumed to be in(B. A function P(-) = P0(') of B z(R and a family of functions P»(«, •) (n = 0, 1, • • •) of x t A, B E(B, satisfying the following recursive conditions are assumed given: (i) P{B) = PQ(B) is a probability measure m(B. (ii) P0(x, B) for almost any fixed x (P0 measure) is a probability measure in (B and for every B (c (B) is a measurable function with respect to (B in x. (iii) Let P»(-)> P«('> ') be given where Pn(B) is probability measure in (B and where Pn{x, B) is a probability measure in (B for almost every fixed x (P n measure) and for every B e (B is a measurable function with respect to (B in x. Then (1)

Pn+i(B) = f

Pn(dx)Pn(x,B);

*This research was supported by the Office of Naval Research under Contract Onr 908 (10). Reproduction in whole or in part is permitted for any purpose of the United States Government. 585 Journal of Mathematics and Mechanics, Vol. 8, No. 4 (1959). R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_16, © Springer Science+Business Media, LLC 2011

134

586

M. ROSENBLATT

and Pn+1(x} B) is a probability measure in (B for almost every x (P„+i measure) and for every B e (B is a measurable function with respect to (B in x. Define a probability measure in the space Q of sequences w = (x0 , xx , • • -), Xi t A, as follows. Let Xn(w), n = 0, 1, 2, • • • , be the nth coordinate function, t h a t is, Xn(w) = xn if w is the point (x0, xx , • • •)• Set (2)

P(X0(w) t B0) = P0(Bo),

B01
and, more generally,

P(X0(w)zBo, ( 3 )

=

f

••• , X » e P w ) Po(^o)

f

P o ( * 0 ,
• • •

f

P^CXn-x

,«„),

P

7

t(B.

The sequence of functions {Xn(w)\ is called the Markov process with initial distribution Po(-) cmd stochastic transition functions (or, more loosely, transition probabilities) P w (-, •)> n = 0, 1, • • • . Notice that a Markov process, as introduced above, may have a non-stationary stochastic transition function (not independent of the time n) and our interest is in functions Y(n) = f(X(n)) of Markov processes X(n) with stationary transition functions. Since we may be led to processes with non-stationary transition mechanism, the more general definition above has been given. As already remarked, the Markov process is said to have a stationary transition function if P«(#, B) = P(x, B) (independent of n). Consider the family of Markov processes with stationary transition function generated by the fixed transition function P(x, B) and any initial distribution P ( P ) . Since we speak about any initial distribution P ( •), a more stringent condition has to be imposed on P(x, P ) . P(Xj B) must be assumed to be a probability measure in (B for every fixed x e A. The family of Markov processes introduced will be referred to as the family of Markov processes generated by P(x, B). Let A' be a space of points y with a Borel field(B' of A' sets. Let /(•) be a manyone function from A onto A' measurable (with respect to (B). The field A' is also assumed to contain the sets consisting of single points {y}, y t A', Given a function /(•) and a stationary transition function P(x, B)7 we are interested in finding conditions under which all the processes Y(n) = f(X(n)), derived from /(•) and the family of processes generated by P(x, P ) , are Markovian. Notice that the sub-Borel field of (B induced by /, f J /(B = {f'fB

| P e ( B } - f x (B',

is of importance, The results may be phrased in terms of f 1 / ® or (B;, depending on which is more convenient. We shall want to obtain stochastic transition functions (in the sense given above) for the derived processes Y(n). This will mean that the conditional probabilities computed have to be consistent in such a way that conditional probability measures (the stochastic transition functions) can be manufactured 135

FUNCTIONS OF A MARKOV PROCESS

587

from them. This is possible if there is a one-to-one mapping of the range space A'(of /) into the real line taking the sets®' into Borel sets (of the real line) with the inverse images of Borel sets of the family (B' (see [1, 3, 5]). This assumption is made throughout this paper. This includes the case where A' is a Borel set of countably dimensional Euclidean space with (B' the Borel field of Borel subsets of A'. A Dominated Stochastic Transition Function. Consider the family of Markov processes generated by the stochastic transition function P(x, B), x t A, B e(B. The transition function is a family of probability measures indexed by the points x t A. The family of measures P(x, f^B'), x z A, Bf cffi', on f1®' = f~lf(& is induced by the transition function and the function /. Necessary and sufficient conditions for the processes Y(n) = f(X(n)), derived from /(•) and the Markov processes X(n) generated by P(x, JB), to be Markovian will be obtained under the assumption that the measures P(x, j~1Bt) are dominated by a sigma-finite measure (see [4] for a discussion of this terminology). If the family of measures P(x, f~lBf) is dominated, it is well known that one can find an equivalent countable subset of measures P(x{ , /~\B'), #» e A , i = 1, 2, • • • [4]. The dominating (in fact, equivalent) measure can be taken, and, shall be taken, as the probability measure Hij-'B') -

I ) aJP(Xi , rlB'),

a, > 0,

i = 1, 2, • • • .

Call a set S t j~x($> a single entry set if n(S) > 0 and there is a point y z Kf such that P(x, S) = 0 if x JL f1 {y}. Note that since JJL(S) > 0 Qi equivalent to the family of measures P(x, f~xBf)) there are points x e / _ 1 {y} such that P(x, S) > 0. $ is a maximal single entry set if it is a single entry set and is not contained in a single entry set of larger ju measure. We shall identify a single entry set with all sets differing from it by at most a set of /x measure zero. The point | / E ( E ; from which one can enter a single entry set in one step with positive probability is called a single entry point Lemma !• If the measures P(x, f~1B,)) x t A, Bf e(B', are dominated, there are at most a countable number of disjoint maximal single entry sets and corresponding single entry points. This follows immediately from the fact that JJL is a finite measure. Lemma 1 ensures the existence of a unique (up to sets of ju measure zero) countable maximal collection of maximal single entry sets. Let the single entry sets of this maximal collection be Si , S2 * • • • . Let M = A - U S , - . Lemma 2. Under the hypothesis of Lemma 1, if fx(M) = 0 then the processes Y(n) = f(X(n)) are Markov processes. The space A; is then the countable set of single entry points. Let y,- be the single entry point leading into the maximal single entry set 136

588

M. ROSENBLATT

Si . Since ii{M) = 0 any point x z A must fall in some set f~l {VJ}, j ~ 1,2, • • • . The space A' is therefore the countable set of points y,- . Every set f 1 {y,} for which nif1 {#,-}) > 0 is a single entry set. In verifying that the processes Y(n) are Markovian it is enough to look at the probabilities P(Y(j)

= ymU) ; j = 0, • • • , n) -

•/

f

P(dx0)

p(xQ,
P(S»-I

, rMy«(n)})

where P ( - ) is the initial probability distribution. Let Pw(x,B) <5)

=

p ( w + 1 ) ( x , B) -

P(x,B), f P ( n ) (s, dx')P{x', B),

n = 1, 2, • • • .

I t is apparent that under the above assumptions P(Y(j)

- y m(/) ; j = 0, • • • , w) = P ( l » =

(6)

=

ymM) f p W P

w

( x o , n u , | )

if (4) is positive. Notice that we need only consider points y,- for which P(fl {y,-}) > 0. Let Q(-) be the initial distribution (on(B') and Qn(- ,•) the stochastic transition functions (of y z A' and J3' e ©0 of Y(n) that are to be constructed. Clearly Q({y?}) = P ( / _ 1 fa}), li Pn(fl fa}) > 0 let

Q«iVi , \Vk}) - P(Y(n + 1) = yh \ Y(n) = (7)

Vi)

\pn+i(r{yk})/pn(ri{yi}) if y, is the single entry point for the set of f1 0

{yk}

otherwise.

I t is then clear that Y(n) is a Markov process with initial distribution Q(-) and transition functions Qn( •, •) since (3) is satisfied with Q( •), Q n (•, •), A', (B' in place of P(-)> ^n(-, •)> A , ®- Since this is true for any initial distribution P ( - ) the lemma has been proved. The Markovian property of Y(n) was trivially verified when fx(M) = 0. The interesting case is clearly that in which p(M) > 0. There are then single entry sets if and only if p(M) < 1 (assuming M to be a probability measure). If S is a single entry set, we may be able to enter S from a point xzM outside the single entry sets in a finite number of steps with positive probability. If this is possible, there is an integer n and there are points ym{1) , • • • , y w(n) effi' with the following properties. The sets/" 1 {ymU))j j ~ 2, • • • ,n, are single entry sets while/" 1 {yma)} is not. Further, the point ymU) is the single entry point for f1 {ymU+1)}, j = 1} • • • , n — 1, and the point ymin) is the single entry point for S. Given any single entry set S into which entry is possible from points xzM with positive probability 137

589

FUNCTIONS OF A MARKOV PROCESS

(that is, there is an integer nf ^ 1 such that Pinn (x, S) for some x z M), the integer n and the points ymU) as specified above are uniquely determined. We shall then say that a thread of length n consisting of the points ymil) , • • • , ym(n) eCB' enters S. Theorem 1. Let the measures P(x, f 1 !?'), x z A, B' t(&'9 be dominated. A necessary and sufficient condition for the processes Y(n) = f(X(n), derived from /(•) and the family of Markov processes X(n) generated by P(x, B), to be Markovian is given as follows: (a) There is a function R(x, B) of x z M, B z f"1®' that is measurable in x with respect to f'1®' for each B z f"1®' and satisfies (8)

f P(x0 , dx1)P(x1 , B) = [ P(x0 , dx1)R(x1 , B)

JA

JA

1

for every xQ z A and A, B z jT ®' where A (Z M. (b) / / S is a maximal single entry set with a thread of length n entering it, there is a function nR(%, B) of x z S, B z / -1 (B' that is measurable in x with respect to /~\B' for each B z f 1 ® ' and satisfies f P ( n + 1 ) (* 0 , dxn+1)P(xn+1 , B) = f P{n+l)(x0

(9)

JA

for allxoz

JA

, dxn+1) nR(xn+i , B)

A, A, B z / " V , A C S.

First consider the necessity of the conditions. If p(M) = 0 the conditions are vacuously satisfied. Assume therefore that p(M) > 0 and let the necessity of condition (a) be our concern. Take any two points x, xr z A and the corresponding measures P(x7 B), P(x', B) on the sets B z f"1®', B QM. Let M(x9 xf) C M be the set on which these two measures are mutually absolutely continuous in M . Set (10)

P(x, A,B)

= f P(x, dx')P(x', B),

A, B z f V ,

A

CM,

and let P(x, df^y, B)/P(x, df~~1y) denote the Radon-Nikodym derivative of (10) with respect to P(x, A) on the Borel field f'1®' restricted to subsets of M, We want to show that K

}

P(x9df-1y9B) P(x, df^)

Pix'tdf^B) P{xf9 df'y)

almost everywhere with respect to the dominating fi measure on M (x, xf) for each B z f~l(&\ Let us first consider points x} x' z A with xf t f1 f{x}. Let P ( - ) be the initial distribution with mass a9 0 < a < 1, at x and mass j8 = 1 — a at xf. Then

(12)

p(v(9\ * m I vm - ^ "

aP x

( > df~ly>^ + $p(x'> ^ " l y ? aP(x >^id + # * < * ' ' ^ _ag(x,rly,B) agixj-'y) 138

g)

l + fig(xf 9f y9B) + (3g(x,f1y)

590

M. ROSENBLATT

where g(x, j~xy, B), g(xf, j~xy) are the Randon-Nikodym derivatives of P(x, A, P), P(x, A) respectively with respect to p(A), A, B t f 1 ®, A Q M. Since the Y(n) process is Markovian, whatever the initial distribution of X(n), we must, in particular, have P(F(2) e fB, 7(1) e fA | 7(0) = fx) = f P{x, dx')P(x', B) (13)

JA

= f P(s, dr1y)P{Y{2) £ /P | 7(1) = y) for A, 5 £ f"1®', A C Af (#, a?'). But this cannot happen for every a unless (11) holds almost everywhere with respect to p measure on M(x, xf). Thus (11) is valid for pairs of points x, xf z A with x' % j ~ l f{x}. Now consider a pair x, xf z A with x' £ fx f{x} and the set M(x, xf) as defined above. Assume that fi(M(x, xf)) > 0 since otherwise (11) is obviously valid. Consider all points xn £ A, xn ? f'1 j{x) and the corresponding sets M(xn\ x). There is an integer n > 0 such that for some such x", ix{M{xff, x)M(x'} x)) > (|) n . Choose the smallest such integer n and a corresponding point as" % / _1 f{x} and call them n1 , xx . Assume that one has points x{ % f1 /{#} and corresponding integers nt- , i = 1, • • • , m, rii ^ n2 ^ • • • ^ nm , with (14)

//M(S,

s')ilf(s, x,) - 0 ^ ( s , *,)) > ^ r ,

i = 1, • • • , m.

If (15)

M ( ^ ( S , a?0 -

0

M(s, »,-))

> 0

there is an integer n ^ nm and a corresponding point #" £ f1 f{x} with (16)

M(M(X, X')M(X,

X")

- \jM(xy

Xi)J

> |r

Choose the smallest such n and a corresponding point xn and call them nm+1 , xm+i . If this construction stops in a finite number of steps, (17)

v\M(x, x') - (jM(x,

xf)) = 0

where m is the number of steps. Otherwise we have an infinite sequence of x/s with corresponding ni —> oo as j —» <». Moreover, (18)

/*(jf(x, S') - (jMfa

, a)) = 0

for otherwise we would have a contradiction. Now P(x, d/_1t/, B)/P(x, rf/_1y), P(x', d f xy, B)/P(x', df'y) are both equal to P(x, , df'y, B)/P{xi , d f V), P £ /^(B', on Af (x, X')M(XJ , x) and hence equal to each other there with respect 139

FUNCTIONS OF A MARKOV PROCESS

591

to ix measure. Because of (17) or (18) they are therefore equal almost everywhere on M(x, x') with respect to ju measure. Assume that we have taken the equivalent measure CO

V(P) = Z CljPix', , B),

CLj > 0,

£ CLi = 1,

B£ fV,

where {P(x', JB) } is an equivalent countable subset of the collection of measures {P(x} B)y x t A}, B e f 1(B'. Now consider the Radon-Nikodym derivative of f J2 <*iP(x'i , dx)P{x, B)

with respect to p(A)} A, B t j~l(8>, and call it R(x, B). Now it is clear that R(x, B) is measurable in x with respect to /~l(B' for each B t f"1®'. Further,

almost everywhere with respect to P(x, A) measure, A t f~~l(&', A C M. But this is condition (a). An argument paralleling the one given above for the necessity of condition (a) suffices for the necessity of condition (b) if £ is a maximal single entry set with a thread of length n entering it. The function nR(x, B) can be taken as the Radon-Nikodym derivative of [ E a/P (n+1) fo' , dx)P(x, B)

(20)

JA

i

( +1)

with respect to £ f a,P * (x$ , A), A, B t f 1 ®', ACS. We now consider the sufficiency of conditions (a), (b). If n(M) = 0 conditions (a), (b) are vacuously satisfied and Lemma 2 tells us that the new process Y(n) is Markovian. Let us therefore consider n(M) > 0. Let P(-) be the initial distribution of X(n). The initial distribution of Y{n) is then Q{B') P(flB')7 Bf t <£'. Now consider how to construct the stochastic transition functions &(•, •) of the Y(n) process. liytfM let Qn(y, Bf) = R(x, flB'), n £ 1, where l f x t f~ {y} and B is any set in (B\ Set Q0(y, Bf) equal to the Radon-Nikodym derivative of P(F(0) £ A', 7(1) e B') with respect to P(F(0) e A'), Af, Bf e ®', y z Af. If y $ fM, f1 \y) is contained in some single entry set S. If there is no thread of length m < n entering S let (21)

QJy, BO = [

Pn(dx)P(x, r'B'WPniT1 M),

& e «',

if P n (/ ml {$/}) > 0. If there is a thread of length m < n entering S, let (22)

Qn(y, BO = mR(x, f ' P ' ) , 1

* ' e ©',

where x zf {y}. The Qw(•, • )'s have been set equal to conditional probabilities. Since we have assumed that there is a one-to-one mapping of the range space 140

592

M. ROSENBLATT

A' into the real line taking the sets of (B' into Borel sets of the real line and conversely, the Qn(-, •) can be chosen so that they are conditional probability measures. The process Y(n) must be shown to be Markovian with the specification of the stochastic transition functions Qn(*, •) &s given above, namely P(F(0) e BS , • • • , Y(n) z BO = [ (23)

JB

• f

Qo(l/o , dyX)

Q(dy0)

*'

••• f

Qn-liVn-l

, Bn),

BJ * ffi'.

I t is enough to show that relation (23) holds for sets of the form (24)

( r G ) e / B , J i = 0, ••• , n ) ,

B,e(E,

where for each j separately either B, C M or else JB, is a single entry set, since every set (Y(j) t fB,- ; j = 0, • • • , n), Bi c(B, is the sum of a countable number of sets of the form (24). However, it is apparent from the construction of the stochastic transition functions Q„(-, •) that (23) holds for sets of the form (24). The proof of the sufficiency is complete. Probably the most interesting context covered in Theorem 1 is that in which there are no single entry sets. Now the original process X(n) has stationary transition mechanism. Suppose we insist that the new process Y(n) (assuming it to be Markovian) also have stationary transition mechanism (which might conceivably depend on the initial distribution P ( - ) of the X(n) process). Condition (8) simplifies since stationarity of the transition mechanism implies that the Radon-Nikodym derivative of (25)

f

Jf~1Af

P(dx)P(x,

f'B')

with respect to PQ^A'), A', Br e (B', whatever the initial distribution P(-), must be equal to Q(y, Bf) almost everywhere with respect to the jit measure. Corollary 1 follows. Corollary 1. Assume that the hypotheses of Theorem 1 are satisfied and that there are no single entry sets. A necessary and sufficient condition for the processes Y(n), derived from /(•) and the processes X(n), to be Markovian and have stationary transition mechanism (Q„(*, •) = Q(-, •) ) i$ given by (26)

P(x, f~xBf)

measurable in x

with respect to (B; for every B1 e(B'. However, condition (26) of Corollary 1 is not generally valid if one allows single entry sets. The following example illustrates this. Consider a Markov chain X(n) (A a countable set which is, for convenience, labeled by the integers) with transition probability matrix P = (p<,-). The function / collapses the states (elements of A) into three distinct points y1 , y2 , y3 (A; = {2/1,2/2,2/3}) 141

593

FUNCTIONS OF A MARKOV PROCESS

in such a way that at least two X(n) states are mapped into each y point. Let (27)

p

=

fc 8S«u

o

if *>t, ij ce ff 1Ms/il, ^},

if

if itrivi},

< 1, 00 <
iefMy.}

while (28)

P(i, fl{y,})

=

E

P„ = Qfo« , {%})

for i e f 1 {y a }, a, /? = 2, 3. The remaining pu (i z f1 arbitrary aside from the non-negativity condition and (29)

P(i, A - r'iVi})

= 1 - c,

{yx}, j % f1

\yi}) are

tef1^}.

The processes F(n) = f(X(n)) are Markovian whatever the initial distribution P({i}) = Pi of X(n). Further, the transition probability matrix of Y(n) is stationary, whatever the initial distribution pi of X(n). Of course, it does depend on the initial distribution of X{n) since (30)

P(X(n + 1) e rMife} I X(n) t rx{!/i})

=

E j^P&rMv.})/ E P«.

If we insist that the transition mechanism of Y(n) not only be stationary, but also independent of the initial distribution P(-) of X(ri), we are again led to (26) as a necessary and sufficient condition. Condition (26) arises in some of the results of [2, 6, 7, 8, 9]. Notice that condition (8) implies that the transition mechanism of Y(n) is almost stationary. It is, in fact, then stationary for n ^ 1. There are simple conditions that imply (8), for example, condition (26). The assumption that the measures P(x, B) are of the form (31)

P(x9B) -

f f(x,z)v(dz),

where v is a probability measure on(B and f(x, z) is measurable in (x, z) on the product space A X A with respect to the Borel field (B X f'1®', is enough to ensure (8). It is reasonable to ask whether (26) or (31) or trivial variants of them are the only conditions that imply (8). A simple discussion indicates that this is not the case generally. Suppose that the function P(x, B) is given as a function in x z A and B z /~1(B'. It is measurable for each fixed B z f^ffi' in x with respect to(B and a probability measure for each fixed x e A on f^ffi'. We wish to find the family of stochastic transition functions P( •, •) of x z A and Be(E which are consistent with P(x> B)} B z Z"1®', that is, they agree with the latter on the restricted field Z"1®' and, further, condition (8) is satisfied. Assume that the given family of measures P(x, B), x z A, B z Z"1®', is equivalent to the probability measure p on /-1(B'. Call n a refinement of ju if it is a probability measure on (B and agrees with JL 142

594

M. ROSENBLATT

on /~1(B/. The following corollary characterizes the family of stochastic transition functions with corresponding sets of measures dominated by a probability measure M and consistent with P(x, B), B z f 1 ® ' . Corollary 2. Let P(x, B), x z A, B z f'1®', be given and be dominated by the probability measure p on f 1(B'. Then P ( - , •) is a dominated (by a probability measure) stochastic transition function consistent (so that (8) is satisfied) with P(x} B), B z f'1®*', if and only if it is a transition function that can be expressed in the form (32)

P(x, B) = [ (1 + a(x, z))g(x, z)n(dz),

xtA,

B z (B,

where ii is a refinement of p, where g(x, z) ^ 0 is a function of A X A measurable with respect to (B X f"1^' and a(x, z) is a function on A X A measurable with respect to (B X (B and satisfying (33)

a(x,z) ^ — 1 ,

(34)

/

(35)

f

#, Z E A ,

a(x}z)g(xyz)n(dz)

= 0,

a(s, s)flrGc, z)ix{dz)P{z, f'B')

= 0

/or all x t A and A'} Bf effi'. Suppose P ( - , •) is a dominated stochastic transition function consistent with P(x, B), B z f 1 ® ' . Let / i b e a probability measure on(B equivalent to the family of measures generated by P(«, •). Since P ( - , •) agrees with P(x, B), B z f"1®', on P £ f"1®', ju can be taken to be a refinement of p. Let #(#, 0) be the RadonNikodym derivative of P(x, B)} B z f"1®', with respect to /L The function g(x, z) is defined on A X A and measurable with respect to (B X / -1 (B'. Let A (a, 2) be the Radon-Nikodym derivative of P(x, B), B c(B, with respect to (36)

f g(x,z)ii(dz),

BE(B.

Set a(#, 2;) = /&(#, 2) — 1. Then P ( - , •) has the representation (32) and gr(#, 2) *z 0. We only have to verify conditions (33-35). But h(x, z) = 1 + a(x, z) i> 0. Condition (34) follows from (37)

/ h(x,z)g(x,z)ti(dz)

= [ g(x,z)p(
BtfV,

and (35) follows from (8). On the other hand, if P ( - , •) has a representation of the form (32) it is clearly a stochastic transition function consistent with P(x,B)f x z A, B z / -1 (B'. Actually (34) is just the special case of (35) obtained by taking Bf = A'. I t was written out because we made use of it to get (35), Corollary 2 provides us with a method of constructing transition functions 143

595

FUNCTIONS OF A MARKOV PROCESS

consistent with a specification of P(x, B), x e A, B e f1®'. The specific transition function P(-, •) obtained will depend on the choice of the refinement fx of p, and the function a(x, z). There is always one possible choice of a(x, z)} namely a(x, z) s 0. Generally, there will be many other nontrivial choices of a(x, z) but this will, of course, depend on how strong the restriction (35) is (see the example in section 3 of [2]). The condition a(x, z) = 0 amounts to the representation (31). The statement of Theorem 1 in the special case of a countable state space is given in [10]. It is clear that the notion of a maximal single entry set does not make sense if the condition that the measures P(x, f^B'), x e A, Bf c (B;, be dominated is relaxed. The question of determining when a function Y(n) = j(X(n)) of a stationary Markov process X(n) (not any initial distribution) is Markovian was brought up in [2]. It is clear that (8) provides a sufficient condition broader than (26). Continuous Parameter Markov Chains. Now consider what happens in the case of a continuous parameter Markov chain X(t) with stationary transition mechanism, 0 ^ t < oo. Let the state space A be labeled by the positive integers. The chain is assumed to have a decent transition probability function P{t) = (Pii(t);i,j

= 1,2, .-•) = (P(X(t + T) = j | X(r) = i»,

0 £ *,

r < co,

continuous in t, 0 ^ t < «>, with (38)

lim P(t) = I.

Let Sa = Z"1 {y*}, <* = 1? 2, • • • , be the disjoint sets of states generated by the function /. Necessary and sufficient conditions for Y(t) = f(X(t)) to be Markovian, whatever the initial distribution p{ = P(X(0) = i), i = 1, 2, 3, • • • , of X(t), are obtained. We shall say that Sa is (is not) a single entry set at time t > 0 according as to whether it is (is not) a single entry set with respect to the matrix P(t) and its powers (in the sense of the previous section). If Sa is not a single entry set at time t > 0, it is not a single entry set for all r ^ I Suppose Sa is a single entry set at time t > 0. Then Sa can only be entered from itself in time r ^ t because of (38). It must then be a single entry set for all r ^ t For if it were not, there would be a maximal T < oo such that for t < T Sa is a single entry set and for t > T Sa is not a single entry set. Then for every € > 0 there is an i % Sa such that (39)

Vi98m(T

+ e) = J2 Vu(T + €) > 0.

But (40)

Vi,Sa,{T

+ «) = T,Pii(T

itS

a

- e)p,.s„(2€) = £ Vu{T - e)Pi,Sa(2e) = 0

if € < |JT, a contradiction. Thus there are only two possibilities. Either Sa is a single entry set for all t or else it is not a single entry set for alU, a = 1,2, • - • . 144

596

M. KOSENBLATT

If Sa is a single entry set for all t (41)

Pi.s.W

= 0

for all i % Sa . Suppose Sa is not a single entry set for all t. Then (42)

D p
= Qs..s,(«, T),

£, T > 0, for all i c $ a and all j8. As t [ 0 the left side of (42) approaches p{ , ^ ( r ) . B u t the right side of (42) depends on i only through Sa . Thus (43)

pitB,(r)

= Qa„ f 5,(r)

for all i £ >Sa and we have the following result. Theorem 2. Let X(t), 0 ^ f < » , be a continuous parameter Markov chain with stationary transition probability function P(t) = (pa(t); i, j = 1, 2, • • •)• Lei P ( 0 be continuous in t, 0 ^ t < co ? with (44)

lim P(«) = / .

T&e process Y(t) = f(X(Jt)) will be Markovian, whatever the initial distribution of X(t), if and only if each set Sa of states generated by f separately satisfies one or the other of the following two conditions for all t: (45)

(i)

(46)

(ii)

pitsM

= 0

for all

Pi,sfi(t) = Qs„,sfi(t)

i?Sa for all

, izSa

and all

]8.

It would be interesting to find out under what conditions the results of Theorem 2 hold in the case of a more general state space for the processes X(t). REFERENCES [1] R. R. BAHADUR, Sufficiency and statistical decision functions, Ann. Math. StaL, 25 (1954), pp. 423-462. [2] C. J. B U R K E & M. ROSENBLATT, A Markovian function of a Markov chain, Ann. Math. Stat., 29 (1958), pp. 1112-1122. [3] J. L. DOOB, Stochastic Processes, New York (1952). [4] P. R. HALMOS & L. J. SAVAGE, Application of the Radon-Nikodym theorem to the theory of sufficient statistics, Ann. Math. Stat, 20 (1949), pp. 225-241. 15] M. LOEVE, Probability Theory, New York (1955). [6] B. R A N K I N , The concept of enchainment—a relation between stochastic processes, unpublished (1955). [7] D . ROSENBLATT, On aggregation and consolidation in linear systems, to be published in the Naval Research Logistics Quarterly. [8] D. ROSENBLATT, On aggregation and consolidation in linear systems, Technical Report C, prepared under Contract Nonr-1180 (00) (NR-047-012) 1956. [9] D . ROSENBLATT, Abstracts of "On aggregation and consolidation in finite substochastic systems I,' I I , I I I , I V , " Ann. Math. Stat., 28 (1957), pp. 1060-1061. [10] M. ROSENBLATT, Abstract of "Functions of Markov Chains," Ann. Math. StaL, 29 (1958), p. 1291. Indiana University Bloomington, Indiana 145

Stationary Processes as Shifts of Functions of Independent Random Variables M. ROSENBLATT* 1, Summary. Let xn , n = 0, ± 1 , ± 2 , • • • , be a strictly stationary process. Two closely related problems are posed with respect to the structure of strictly stationary processes. In the first problem we ask whether one can construct a random variable £„ = g(xn , xn~t , • • •)> & function of xn , xn„x, • • • , that is inde pendent of the past, that is, independent of xn-x , £w_2 , • • • . Such a sequence of random variables {£n} is a sequence of independent and identically distributed random variables. Further, given such a construction, is xn a function of ?» , £n_i , • • • . Necessary and sufficient conditions for such a representation are ob tained in the case where xn is a finite state Markov chain with the positive transition probabilities in any row of the transition probability matrix P = (p tJ ) of xn distinct (Section 3). Such a representation is comparatively rare for a finite state Markov chain. In the second problem, the assumption that the independent and identically distributed £n's be functions of xn , xn-x , • • • is removed. We ask whether for some such family {£n} there is a process {yn}, yn = gr(£n f g n - 1 f . . . ) , with the same probability structure as {xn}. This is shown to be the case for every ergodic finite state Markov chain with nonperiodic states (Section 4). Sufficient conditions for such representations in the case of a general strictly stationary process are obtained in Section 5. 2. Introduction. There are problems on the representation of strictly stationary processes that are motivated by certain results in the linear prediction problem and remarks of P. LEVY. In the linear prediction problem one considers a weakly stationary process Xn 9 n = 0, ± 1 , • • • , that is, a process xn with (1)

Ea£ < » ,

Exnxm = rn_m ,

Ezn s 0.

Under certain conditions on the spectrum of the process (the spectral distribution function F(K) absolutely continuous with log F'(\) integrable), the process xn can be represented as a one-sided weighted moving average of orthonormal random variables £n defined on the present and past of the process. More explicitly, *This research was supported by the Office of Naval Research. Reproduction in whole or in part is permitted for any purpose of the United States Government. 665 Journal of Mathematics and Mechanics, Vol. 8, No. 5 (1959). R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_17, © Springer Science+Business Media, LLC 2011

146

666

M. ROSENBLATT

£n is an element of the closed linear manifold (closure in mean square) generated by xm , m S n, with (2)

E& s 0,

E ^ w = 5n,m

and xn has a representation CO

xn = X ) a*£n-* >

(3)

X ) a* < °°

(see [1]). This suggests a corresponding representation problem with orthogonality replaced by independence and linear relationship by functional dependence. Before going on to present the problem, it should be noted that any state ment made in the paper is to be understood modulo sets of measure zero, where the measure is the probability measure implicit in the problem at hand. Thus if £(•) is said to be measurable with respect to a Borel field of sets (B we under stand that it is equal almost everywhere to a function strictly measurable with respect to (B. Further, two Borel fields (B, (B' will be said to be the same if given any set B e (B there is a set Bf t (B' with the measure of the symmetric difference of B and B' zero, and if (B' has the same property relative to (B. Clearly the two Borelfieldsare not the same strictly, but they are the same modulo sets of measure zero and that is of interest to us. Let xn , n = 0, ± 1 , • • • , be a strictly stationary process with T the corresponding one-step shift operator. (Bn is the Borel field of sets generated by Problem 1. (a) When can one find a random variable £0 measurable with respect to (B0 , independent of x-x , x~2 , • • • , and such that (B0 = (B_i X d0 (product field) where ®0 is the Borel field generated by £0 ? (b) If (a) is satisfied, the random variables £rt = Tn%0 , n = 0, ± 1 , • • • , are a sequence of independent, identically distributed random variables. Let an be the Borelfieldgenerated by £n . Is xn measurable with respect to • • • X a»-i X A second problem is suggested by some remarks by P. LEVY [5]. Let xx,x2, be a sequence of random variables. Under rather broad conditions P. LEVY has shown that one can find a sequence of independent, uniformly distributed random variables & , £ a , • • • and Borel functions yx = 0i(&), y2 = Q*(& > &), • • • of £i , £2 , • • • such that yt , y2 , • • • have the same probability structure as X\ , X2 , *

•

Problem 2. Let zn , n = 0, dbl, • • • be a strictly stationary process. Can one find a sequence of independent, uniformly distributed (on [0, 1]) random variables £n , n = 0, ± 1 , • • • , and a Borel function g(£) = g(%0 , £-i , * • 0, £ = (••• , £-i , & , & , * - 0 , such that (4) y n = fl
STATIONARY PROCESSES

667

Notice that g is a one-sided function of the sequence of random variables £n . If problem 1 is soluble, problem 2 is soluble. The converse is, however, not true. Notice that the solutions to these problems have nothing to do with the existence or nonexistence of moments of the process considered. They are prob lems about the Borel fields (Bn generated by the process and the probability measure of the process on these fields. Let H (Bn be the Borel field of sets common to all the Borel fields (B„ , n = 0, ± 1 , • • • . Lemma 1. If problem 2 is soluble, the only functions measurable with respect to r\ (Rn are the constant functions. Consider any random variable z, E \z\ < <», measurable with respect to the Borel field generated by a stationary process {yn} for which problem 2 is soluble. We can assume that the random variables yn are bounded. Just consider any monotone one-one function h(-) mapping (— <», <») into (0, 1). The new process lVn}, yn = h(yn)9 is bounded and has the same Borel fields Owe can find a polynomial p(yn , • • • , yn-k) in an appropriately large but finite number of random variables yn , • • • , yn-.k such that (6)

E |s - p| < €.

Since the yn's are one-sided functions of the £„'s (independent and uniformly distributed [0, 1]), there is a Borel function g(£n , • • • , (■„_,) in £n , • • • , £„.., for j sufficiently large such that (7)

E \q - p\ < 6.

But then (8)

E \z - q\ < 2e.

Now (9)

E(g|H(Bw) = ECfflfflJ - E(g|k , Zm-X ,-•■) = Eg

for m < n — j . But E(s | (Bm) —> E(s | C\ (B„) as ra —> — <» in absolute mean by a martingale convergence theorem [1], [6]. Further, (10)

E |E(*|(Bm) - E(«|(Bm)| £ E \z - g| < 2e.

This implies that (11)

E(E(z(n$8n) - E ( z ) | < 4 e .

Since this is true for every e > 0, E(z j H (Bn) = E(s) with probability one. Now consider any function w measurable with respect to C\ , since otherwise one could find a one-one monotone function &(♦) such that z = h(w) has finite first moment. If z is a constant 148

668

M. ROSENBLATT

almost everywhere so is w. But clearly z is a constant almost everywhere since E 0 | C\ (Rn) = Es. The proof is complete. It is an open question as to whether the converse of Lemma 1 is true. We shall look at this question in a few special but interesting cases. The necessary condition for the representation of problem 2 obtained in Lemma 1 is, of course, a necessary condition for the representation of problem 1. Let {xn\ be a strictly stationary process with the only functions measurable with respect to f~\ (Bn the constant functions. It is natural to call such processes purely nondeterministic processes. From this point on assume that the conditional probability distribution function of the process {xn}, (12)

F(an\an-1 , aw_2 , • • • ) = Pfr» ^ a»K-i = aw_2 , %n-2 = an_2 , • • •],

is a Borel measurable function of the real variables a» , an_i , * * * - Given this weak assumption, we shall look for conditions on the conditional probability distribution that imply that part (a) of problem 1 is satisfied. The following convenient notion of equivalence of two distribution functions F, G is introduced. F and G are equivalent if there is a one-one function mapping the jumps of F onto the jumps of G and preserving the size of the jumps. Notice that any two continuous distribution functions F, G are equivalent according to this definition. Lemma 2. There is a random variable Jn satisfying part (a) of problem 1 if and only if the distribution functions F(an | an_i , an_2 , • • •) (as functions of an) are equivalent for almost all an^ , an„2, • • • (with respect to the probability measure of the process {xn}). Since £n is independent of xn~x , xn~2 , • • • the conditional probability distribu tion of £„ given #»_i , xn-2 , • • • is independent of a;n_i , &„_» , ♦ • • . Moreover, since (Bn = (Bn_2 X GLn the conditional probability distribution of %n given #w-i , xn-2 > - — must with probability one be equivalent to the conditional probability distribution of xn given # n _i, xn„2 9 ' • • • The probability distribution functions F(an | an~x , • • •) must therefore be equivalent for almost all an-x , an-2 } - - • • Conversely, assuming the equivalence of the distribution functions F(an | On.! , an_2, • • •) for almost all a»_i , an_2 , • • • we must show how to con struct a random variable £n with the desired properties. If the distribution functions F(an | an_i , • • •) are continuous for almost all an_i , a«_2 , • • • there is no difficulty. Simply set (IS)

| w = r (xn\Xn-i

, Xn-.2 ? * * *)?

and an argument of P. LEVY [5] indicates that £n is uniformly distributed [0, 1] and independent of #w_i , xn-2 , • • • . Further, (14)

xn = F'\^n\xn^

, xn-2 , • • •)

with probability one. Let us therefore consider what happens if the distribution 149

669

STATIONARY PROCESSES

functions F(an | o».i , an_2 , • • •) have with probability one a discrete part. Let the jumps in the discrete part, ordered in terms of magnitude, be pt ^ p 2 ^ • • • . For convenience assume there is strict inequality among the p4 . An obvious elaboration of the argument given in the case of strict inequality holds in the more general context with equality allowed. The magnitudes of the jumps Vi i V2 j •# • with probability one do not depend on an_i , an_2 , • • • because of the equivalence of the conditional probability distributions. Let A,- be the set of points (15)

Af = \{an , an_x , • • ')\LF{an\an.x , • • • ) = vA

where AF denotes the magnitude of the jump (if there is any) of the conditional distribution function at the argument indicated. We associate a set of integers S((an , an^ , • • •)) with every point (an , a*_i , • • •) % KJ Ai . The integer j e S((an , an_i , • • •)) if there is a set Ai and a point (a'n , an_x , an_2 , • • •) e Af with afn < an . Of course, the set S((an , att_i , • • •)) m a Y be the empty set. Set [j n

""

if

jFfel^n-1 , • • • ) —

J2

Vi

if

(Xn , Xn-i , • • •) £ Ai fan

, ^n-1 , • • ') t U i , -

.

Then fn is independent of a?n-i , xn_2 , • • • . The random variable £n is uniformly distributed on (0, 1 — X) Vi) with density function one and assumes the integral value j with probability p, ♦ Further, the construction indicates that xn is given with probability one by the knowledge of £n , #w_i , x„^2 , • • • • There is a case in which part (a) of problem 1 is satisfied trivially and yet there is no hope of having part (b) satisfied. Suppose F(an \ aw_i , • • •) is for almost all an_i , an_2 , • • • a pure jump function with a jump of size one. Then xn is determined with probability one given xn-x , xn„2 , • • • so that (Bn = (Bn.i = • • • for all ft. It is natural to call such a process a purely deterministic process. Such a process will satisfy the necessary condition of Lemma 1 only if the random variables xn are a constant c with probability one. Since we are interested in problems 1 and 2, the purely deterministic processes are to be avoided. 3. Markov Chains and Problem 1. We shall not be able to obtain very broad results in the general context spoken of in Section 2. The discussion in the general context of strictly stationary processes is relegated to a later section (Section 5) because more detailed and enlightening results can be obtained for strictly stationary Markov chains with a finite number of states. Let {xn} be a strictly stationary Markov chain with transition probability matrix (17) P = (p„), pa = P[xn+l = j\xn = i], and stationary vector of probabilities (18)

Pi - V[xn = *] > 0. 150

i, j = 1, 2, • • • ,

670

M. ROSENBLATT

It is already clear from Lemma 2 that we shall not be able to construct a random variable £n measurable with respect to (Bn satisfying the conditions of part (a) of problem 1 unless the probability distributions {pi}- ; j = 1, 2, • • •} indexed by i are equivalent. An alternative way of stating this is as follows. Given any i, ip there is a permutation M (i, if) of the integers j such that (19)

Pa =

Pi>,M(i,i>)i

for all j . Such Markov chains are admittedly rather special. Nonetheless they are interesting and give some insight into problem 1. We shall call such Markov chains uniform chains because the row probability distributions are the same except for permutation. Let the positive probability masses common to the distributions {pti , j = 1, 2, • * •} be qx ^ q2 ^ • • • > 0 in order of magnitude, 22 Qi = 1- When the q^s are all distinct one can characterize the random variable £w simply. Lemma 3. If xn is a stationary uniform Markov chain with g/s distinct, £n is uniquely determined (up to a one-one transformation modulo sets of measure zero) and is of the form (20)

£n = |(a;n , xn^) = k for (xn , xn^)

such that pXn_t , Zn = qk .

Without too much abuse of notation we can write £« = %(xn , xn^ , • • •). It is already clear that £ must take m distinct values (m ^ » the number of distinct q's) with probability one, and we label the values 1, 2, • • • for con venience. Let the inverse function of £, as a function of xn for fixed xn-i, £n_2, • > • , be 7}(%y xn-i , xn^2 , • • •)• Since £„ is independent of z»_i , zn_2 , • • •

= P[£« = i] = p.x ,vu,i1«..

•••)

> o.

But the g/s are distinct, and therefore for fixed ix , j the function rj(j, ix ,i2, • • •) must be independent of i2 , s3 , • • • . Thus ri(j, it ,i2 , • • •) = r?0*, ^) and hence %(xn , ajn_! , . • • ) = = £(#n , rcn-i). It is clear that £ must be constant on the set (22)

{V,i)\Pn = «*},

and for convenience we set £ = fe on this set. Any other £ satisfying the con ditions of part (a) of problem 1 must be a one-one function of this £. The random variable £n was seen to be essentially uniquely determined when the q^s are distinct. It is easy to see that it is not uniquely determined if the q/s are not distinct. For example, if the transition probability matrix is \a a

(23)

b

a b a\ , P = \a

J)

a

a, b > 0,

a\

we can take 151

a 4= b,

STATIONARY PROCESSES

0 if (xn,xn-0 (24)

%(Xn , %n-V

— )

1 if

671

= (0,0), (0,1), (1,2)

(x. ,x._0 = (1,0), (2,1), (2, 2)

2 otherwise or, alternatively, [0 if (25)

£(#n j %n~l) — ]

( ^ , ^ . 0 = (0,0), (2,1), (2, 2)

1 if (xn,xn^)

= (1,0), (0,1), (1,2)

2 otherwise. In fact, £ need no longer be a function of only xn , xn-i , for one can take [0 if (xn , #„_! , xn-2) = (0, 0, 0), (1,0, 1), (0,0, 2), (0,1,0), (2, 1,1), ( 0 , 1 , 2), (1,2,0), (2, 2,1), (1,2, 2) (2u)

%\Xn , Xn-i

, Xn-2J —

1

if (XH , Xn-! , ZM_2) = (1, 0, 0), (0, 0, 1), (1,0, 2), (2, 1,0), (0, 1,1), (2, 1,2), (2, 2,0), (1,2,1), (2, 2, 2)

,2 otherwise. Let us restrict ourselves to uniform Markov chains with distinct #/s and try to find out under what circumstances the answer to problem 1 is in the affirma tive. We have already seen that a positive answer to problem 2, and hence to problem 1 (making use of Lemma 1), can be given only if all the functions measurable with respect to C\
Mk - {6„(fc)|

where (28)

ea(k) = II if Vu = & [0 otherwise.

Given the Markov chain \xn} there are m corresponding matrices Mk . Each matrix Mk has precisely one element equal to one in each row and all other elements zero. No two Mks have a one in the same entry. Every product of a finite number of the Jlf^s also has these properties. The family of matrices {Mk} corresponds to a family of mappings of the set of states of the process into the same set of states induced by the process {xn}. Consider the semigroup 152

672

M. ROSENBLATT

(under multiplication) generated by the matrices Mk . This semigroup may or may not contain a matrix with a column of ones. If it does, we call the semigroup point collapsing. Theorem 1. Let {xn} be a finite state stationary uniform Markov chain with q{'s distinct. Let {Mk} he the family of mapping matrices generated by {xn}. The answer to problem 1 is positive if and only if the semigroup generated by {Mk} is point collapsing. First assume the semigroup generated by the family of matrices {Mk} is point collapsing. There is then a finite product Mkl • • • Mka of matrices of this family (with possible duplication) which has all the elements in the ith column one. Now the random variable %n satisfying the conditions of part (a) of problem 1 is uniquely determined by Lemma 3 and is given by (29)

£n = %{xn , xn-i) = k if eXn_ltXJk) = 1.

Consider an event Ai of the form (30)

Aj

=

{£,■

==

fci , • • • , qj+a — ks \.

If Ai occurs, it is clear that xi+8 ~ i with probability one. The events An„s , An_2. , • • • are independent, and hence by the lemma of Borel-Cantelli at least one of the events will occur with probability one. Thus at least one of the events Ai , j ^ n — s + 1, must occur with probability one. Let Th be the mapping induced by Mk , that is, Tki = j where j is the unique j such that e{i(k) = 1. We now construct a function g(£n , £n_! , • • •) such that xn = g(%n , £n_i , • • •) with probability one. Consider the sequences (• • • , £n_x , £n). Let S,- be the set of sequences (• • • , £n_i , £n) for which some A{ , i ^ n — s + 1, occurs and Ai is the A{ with largest index, i gg n — s + 1, that occurs. The sets Si , j ^ n — s + 1, are disjoint and P(VJ$3) = 0 (S denotes the complement of S) as already remarked above. Let (31)

g& , ^

, • • • ) = \Ti" *'' r*'+,+1* [0 otherwise.

lf

(

' *' ' L~' ' ^ '

Si

It is then clear that xn = g(%n , ^n_i , •••) with probability one since (• • • , £»-i , £n) * &i ior some j i£ n — s + 1 with probability one. Thus the answer to problem 1 is positive. Now, on the contrary, assume that the semigroup generated by the family of matrices {Mk} is not point collapsing. Every matrix of the semigroup has a certain number of distinct columns containing a one as a column entry. The minimal number of distinct columns among all matrices of the semigroup is certainly greater than one (let us say r > 1) since the semigroup is not point collapsing. But this means that knowledge of all the i-Zs, i ^ n, can tell no more than that xn takes one of r distinct values. The answer to problem 1 must therefore be negative. 153

673

STATIONARY PROCESSES

One might hope for a positive answer to problem 1 if there is a random variable £n satisfying part (a) of the problem and the necessary condition of Lemma 1 is satisfied. However, by making use of Theorem 1 a large number of processes satisfying these conditions for which the answer to problem 1 is negative can be constructed. We give the simplest example exhibiting this interesting sort of pathology. Let {xn} be the strictly stationary Markov chain with the two states 1, 2, the stationary instantaneous distribution P[x„ = 1] = T[xn = 2] — I

(32)

~~

2

and transition probability matrix 0 < p , g = 1 — p < 1,

P =

p * g.

.2 V) Now Mx =

(33)

1 0

M,

0 1

0 1 1 0

so that Sn

SV^» ? %n—l)

fl

if

(aB_1 ,xn) = (1,1), (2, 2)

12 otherwise

is measurable with respect to (B„ and yet independent of X —i , X ~2 j * * * • By Lemma 3 it is essentially the only random variable with these properties. Further, {xn} is purely nondeterministic. Nonetheless xn is not a function of £» , f»-i , since E(xn | £n , £n_! , - . - ) = Exn . One would like to give a neat characterization of the point collapsing semi groups generated by the matrices {Mk) induced by a uniform Markov chain. This does not appear to be possible. However, one can easily sketch the details of a computational procedure that would determine whether or not such a semigroup is point collapsing. Let N < <*> be the number of states of the finite state uniform Markov chain {xn}. Let {Tk} be the family of mappings of the integers 1, • • • , N into the integers 1, • • • 9N determined by the matrices {iff*}. If the semigroup is point collapsing some Tk must map the N states onto iVi < N states. Given Nx states, consider the ensemble of all distinct iVVtuples (including the original one) that can be obtained from this iVi-tuple by applying a finite number of the Tk mappings. Some one of these iVVtuples must be collapsed further onto 2V2 < N% states by some Tk mapping if the semigroup is point collapsing. Continue in this manner until the original set of states is collapsed onto one state. If, at some stage in the construction, all the distinct iVVtuples (Ni > 1) obtained cannot be contracted, the semigroup is not point collapsing. If N is not small, this is admittedly a very tedious procedure. n

154

n

674

M. KOSENBLATT

4. Markov Chains and Problem 2. The answer to problem 1 is generally negative, even in the case of a finite state purely nondeterministic Markov chain. However, the answer to problem 2 for such Markov chains will be shown to be positive. The following theorem will be proved. Theorem 2. Let {xn} be a stationary purely nondeterministic Markov chain. Let there be an e > 0 and a finite number of states M such that (34)

iLVii

= Pi.M > €

jzM

for all L A representation of the type mentioned in problem 2 holds for such a process. Notice that this result holds for all ergodic finite state stationary Markov chains with nonperiodic states. Actually Theorem 2 is proved by enriching the probability space of the {xn) process in an appropriate way. Introduce the Markov process yn , 1 < yn < °°, with stationary probability density (35)

p(y\y') = pa

if yr t (i, i + 1],

yt (j, j + 1],

i, j = 1, 2, ••• ,

and stationary instantaneous probability distribution (36)

P[yn t A] = £ pMAr\(i,

i + 1]),

where p{ = P[xn = %[. We shall construct a process on the probability space of the yn process with the same probability structure as that of the xn process. In spite of a possible abuse of notation we shall call this new process the xn process. The process xn can be taken as xn = g(yn) where g(y) = jifyt (j, j + 1] (see [7]). The probability space of the yn process is rich enough so that one can construct random variables an = G(yn | yn„0 independent of yn-i , yn-2 , • • • with an uniformly distributed on (0, 1). G(y \ y') is a Borel function of y, yf and is so constructed that, for fixed yn-x , G(yn | yn^) is with probability one a one-one function of yn . We can therefore write yn = G~1(an | yn-i). It will be convenient to introduce the associated family of transformations Ta , 0 < a < 1, where (37)

Tay =

G-\a\y).

Notice that it is equivalent to give either G(y | yf) or the corresponding family of transformations {Ta} due to the essentially one-one character of G. Such a family of random variables an can be introduced, for example, by taking G{y \ yf) = F(y I V*) where F(y | yf) is the conditional distribution function of yn given yn„x . Note that in this case Ta for each a, 0 < a < 1, maps each interval (j, j + 1] into a single point. There are other families of transformations Ta , associated with random variables an of the above form, that have this property. We shall construct a convenient family Ta for our purposes in the proof of Theorem 2. By (34) for some fixed e > 0 there is a finite set of states which we can take as 1, • • • , M (by relabeling the states if necessary) such that J^jli Pa > e 155

STATIONARY PROCESSES

675

for all i. Since the Markov chain \xn) is irreducible and the states are nonperiodic, there must be two paths from state one to state one of positive probability with lengths that are relatively prime. Add all the states passed through in these paths to the states 1, • • • , M. Further, add all the other states passed through in a path of minimal length and positive probability from state i to state j , i, j = 1, • • • , M. Call the collection of states included thus far S. We can then connect any two states i, j z S by a path of positive probability con taining only states of S. If the states of S are N(S) in number it may be con venient to relabel the states of the process so that they appear as the states 1, 2, • • • , N = N(S). Notice that the matrix of transition probabilities of the process {xn}, as restricted to the states of S, is irreducible and aperiodic, that is, some sufficiently high power of the restricted matrix has all its elements positive. Let S' be any subset of S and N(S') the number of states in S'. Call Sf a contracting set if there is a set of states S" C S with N(S") < N(Sf) such that for each i z S; there is a j z S" with pu > 0. The set S" C S can be reached from S' C S if for each i z S' there is a j z S" with p{i > 0. S" is accessible from S' if for some k there is a sequence of subsets (of S) Si = S', S2 , • • • , # * = S" such that Sf can be reached from $?-_i , j = 2, • • • , k. If N(S;) > 1, S' C S, there is a set S" C S with N(S") < N(S') that is accessible from S'. For suppose there were not. Every m-tuple of states of S is accessible from every other m-tuple of states of S because of the irreducible and nonperiodic character of the transition matrix when restricted to the states of S. If there were no set of states S" with the desired property no m-tuple with m = N(S') > 1 could be contracted. Take any two distinct states i, j z S. Since they are in some m-tuple and no m-tuple is contracting, there is no state k t S such that p%k j Pik > 0. This means that for any i t S there is one and only one j = j(i) such that Pit > 0. But this implies that the matrix of transition probabilities, as restricted to the states of S, is periodic, a contradiction. Start with the whole set of states $ 0 = $ = {1, 2, • • • , N}. This set is con tracting. Let Sx C S be a set of states with N(Si) < N(S) that is accessible from S0 in a minimal number of steps. Let S2 C S be a set with N(S2) < N(SX) that is accessible from Si in a minimal number of steps. Continue the construction until a set, say Sk , with one element is obtained. Let Mi , • • • , Ma be sets of states with Mx = £ 0 , M2 = Si , • • • , M2+rt = S2 , • • • , M2+rt+ra = Sz , • • • , Ma = Sk where a = 2 + rx + • • • + rk and ifcff+1 can be reached from M,- . In other words, the M/s are the sets of states passed through in going from S0 to Sk. The number of steps in going from £,• to £,-+1 can be at most N2 = N(S)2 in number (see [8]). Therefore the total number of steps from Mi to Ma can be at most Nz. Let the minimal positive transition probability pa , i, j z S, be 5 ( > 0). Given any state i z M,- , let I = fj(i) be the state of lowest index in Mi+X for which pu > 0. Further, given any state i % S. there is a state I = f(i) z S such that pit > e/N. Let 6' = min (e/N, 5). Now consider the Markov process {yn}, constructed at the beginning of our proof, and the Markov chain xn == g(yn). We now set up random variables 156

676

M. ROSENBLATT

an = G(yn | yn-i)7 uniformly distributed and independent of yn-x , yn-2 , • • • with the associated family of one-one transformations Ta satisfying conditions now to be specified. Consider disjoint subintervals (7, , 5<), i = 0, 1, • • • , a, of (0, 1) with 0 < di - 7* < 8'/(N* + 1). The transformations Ta are to be set up (and can be set up) so that they satisfy the following conditions. Let (38)

r . ( M + i ] C ( / ( t ) , / ( H i ] if

i*s,

Ta(i,i+

izS

1] C(1,N

+ 1]

if

when 70 < a < 80 . Thus if 70 < an < 50 we know that 1 < yn ^ iV + 1 (or equivalently avc S). Further, let (39)

Ta(j,i+l]C(Jt(i),m

+ l]

for all z e M, if y,- < a < 5,- . Moreover, let the family of transformations Ta , 0 < a < 1, map each interval (i, i + 1], t = 1, 2, • • • , into a single point. The point will generally depend on a. Now if the event A} occurs, (40)

Aj = {70 < oti < S0 , • • • , 7a < «,-+« < Sa , 70 < ai+a+1 < S0},

we know with probability one the value of y3+a+i . For the first a + 1 conditions imply that i < yi+a S i + 1 where i is the single state of Ma . Thus #,-+«+1 = Taj+a+xih i + !]• The random variables {<*,«} are independent uniformly distri buted random variables. Just as in the proof of Theorem 1, the Borel-Cantelli lemma implies that at least one of the events Ai , j ^ n — a — 1, occurs with probability one. Let 22, be the set of sequences (• • • , an-t , an) for which some Ai, i ^ n — a — 1, occurs and A, is the ^4t- with largest index, i ^ n —• a — 1, that occurs. The events Ej are disjoint, and with probability one every sequence (• • • , an-i , an) lies in some Ef , j £ n — a — 1. Let /^X (,41)

X/ Ai^a„ , an-t

x _ \Tan , • • • ; — -€

' • " TaHa+1(i,

l+l]

if

(• • • , « n - l , <*«) « # /

[0 otherwise. Then yn ~ h(an

y

an^x , • • •) with probability one and hence #M = g(h(an, a n -i , • • •))

with probability one. The proof of Theorem 2 is complete. 5. Stochastic Processes. We now consider some conditions sufficient (but not necessary) for a positive answer to problem 1, and hence problem 2 in the case of a general strictly stationary stochastic process {xn}. There are several conditions that will be introduced to avoid unduly tedious measure theoretic complications. We have already seen that there are difficulties in obtaining random variables £n satisfying the conditions of part (a) of problem 1 if the conditional distribution function F(an | aw_x , •••) has a discrete component with positive probability. To avoid this assume that F(an | a n _i, • • •) is continuous in an with probability one. Further, let the random variables xn of the process be 157

STATIONARY PROCESSES

677

hounded between zero and one. This last assumption is on restriction since it can always be brought about by considering a process zn = h(xn) where h is a one-one monotone function mapping (— <», oo) into (0, 1). Further, letF(an j an-x, • • •) be a strictly increasing junction of an , 0 < an < 1, with "probability one. By Lemma 2 there is a random variable £n satisfying part (a) of problem 1. Such a random variable is, in fact, given by (42)

£n = FiXnlXn-t , Z n - 2 , ' ' 0-

This is not the only possible construction of such a random variable. The con ditions to be specified will be enough to ensure that the process xn is of the form

(43)

xn = gfa ,£ n _! , • • • ) ;

in other words problem 1 has a positive answer with this particular construction (see formula (42)) of such a random variable. In a report KALLIANPUR & W I E N E R [3] discuss some work in this direction. The conjectured result is that a repre sentation of the form (43) holds under the above conditions and under the assumption that {xn\ is purely nondeterministic. The argument given implies that the conjectured representation can be established in terms of the specific random variables (42). We shall later give an example for which a representation of the form (43) is valid, but not in terms of the random variables (42). This indicates that generally one has to consider constructing random variables %n satisfying part (a) of problem 1 which are not of the form (42). Let fi be the measure on half infinite sequences (a„ , an-x , • • •), 0 < a{ < 1, induced by the probability measure of the process {xn}. Let Ta , 0 < a < 1, be the family of transformations of the set of sequences (an , an_! , • • •) into the real numbers an+1, 0 < an+1 < 1, induced by the conditional distribution function, that is, (44)

Ta(an , an-t , • • • ) = F~\a\an , an_x , • • •).

Notice that (45)

Xn = 1 £n\Xn-i , Xn-2 } * ' ')

with probability one where £n is the uniformly distributed random variable (46)

£, = F(xn\xn-X , • - •)

independent of £n_i , xn„2 , • • • . If £„ = g(£n , £n-i , • • •) then clearly xn = E(x n | in , £n_t , • • •) and hence (47)

xn - E(xn\£n , £ , - ! , - • • , &-*) -> 0

in probability as k —> oo or equivalently (48)

E t a - E ( x . | f c , . . . ,$n-,)|~>0

because of the boundedness of the random variables xn . Because of the stationarity of the {xn} process, condition (48) could equally well have been written 158

678

M. ROSENBLATT

(49)

E | I

S

- E W | „ , • • • , €0! —> 0

a s w - 4 c». To avoid cumbersome formulas we introduce the following notation. Let Xm

(50)

\Xm

j Xm—1 ?

r u 2 » . i = Tu(xm^

* )j

, xw_2 , • • •),

Theorem 3. The condition (51)

E \xn - E(#j£n , • • • , $01 -* 0 as n -^ oo

is satisfied if and only if there is a set S of y0 points of jx measure one such that (52)

Ttn{T(,_t{-..{TM---}\

-Ttn{Tt._l{-~{TM--'}}-*0

in £x , • • • , £n , • • • measure for all y0 , % e S as n —» oo. Suppose there is a set # of /x measure one satisfying condition (52). Let z01 S. Then for all x0 t S the measure as n —> oo for any fixed e > 0. But then

r{\xn-T(A--{Ttjk}'--}\

>€} = /m{(fe ,••• ,M|r t .{...{r 4 t s„}...}

as n —» oo and hence in probability a s n - ^ oo. On the other hand,

- E | E ( r { i l { . . . { r ^ } . . - } -r,.{---{r l A}---}l«., ••• ,&)l gE(E(|r € .{...{r^o}---} - ^ . { • • • { r « A } . . . } | | & , . . . ,&» = Ek-ru{-.-{r€l2o}---}|-*o. But then xn — E(#n |£n , • • • , £i) —» 0 as n —» oo in probability. Now suppose that xn — E(xn | £n , • • • , £t) —» 0 in probability as n —» oo and condition (52) is not satisfied. Then given a sufficiently small e > 0 there are sets Sx , S2 with M(#I), M ( ^ ) > 5(e) > 0 such that for x0 z St , xf0 z S2 m{(&, '•• , f O | | ^ . { - - - { ^ A } - - - } - r e . { - . - { r ^ } . . . } | >*)} > 5(e) for an infinite number of values of n. However, 159

679

STATIONARY PROCESSES

xn - Efolfc, , • • • , &) -> 0 implies that for almost all x0 (n measure) Tin{->>{TM--}

-E{Tin{~-{TM---\\Sn,

••• , * i ) - 0

in £1 , • • • , £n , • • • measure, a contradiction. Corollary 1. Let the family of transformations Ta , 0 < a < 1, be such that, given any two x0 , x'0 with 0 < #_, , xLi < 1, 00

(53)

E \T^x0 - Tux'*\ S Z J8/ b - / - x'-f\

/or some sequence of positive constants &• with ]T) &• < 1. 77&en {#w} Aas a representation of form (43). TTie expectation in formula (53) is understood to be an average with respect to the uniformly distributed random variable £. It is clear that

E|r € .{...{r l l « 0 }---} - r ^ - . . { z v « } - - - } | £ ( 2 > , r so that

^{•••{r £ l x 0 }---}

-T(n\---{T(x}---}-+o

as n —> oo in probability for all xQ , x£ and hence the hypothesis of Theorem 3 is satisfied. Corollary 2. Suppose there is a unique ergodic distribution corresponding to the transition probability measure generated by the conditional probability distribution F(an | an^x , •••) and that the Doeblin condition (D) ([1], [6]) is satisfied by this transition probability measure. Further, given any two points x0 , XQ t S (see Theorem 3) with 0 < # _ 3 < xJ{ < 1 for all j , let (54)

Tax0 < Tax£

for all a, 0 < a < 1. Then {xn} has a representation of the form (43). Notice that if x0 < xo , that is, if x-f < x-\ for all j ( ^ 0). However, the hypothesis of the corollary implies that xn , xfn have a common limiting distribution, say F(a). But V[x'n > a, xn ^ a} + ¥{x'n S a] = Y{xfn £ a] so that P{#n > a, xn j£ a} —> 0 at every point a (F(a) is continuous since F(an | an-x , • • •) is continuous in a n ). 160

680

M. ROSENBLATT

Given any € > 0 we can find k(e) points 61 < • • • < bk such that k

U \Xn ^ h , Xn > &/} D {X'n — Xn > e} . 3 - 1

But then ¥{x'n - xn > e} - > 0 for every € > 0 as n —> 00} and hence xrn — #n —> 0 in probability as n —> 00. This argument has been carried out for any two points xQ , x'0 with x0 < x'0 . However, given any two points x0 , x'0 (without such an order relation) we can find a third point x'0f such that xfQr < x0 , x'0 and hence x'n — xn —> 0 for ££ , a:n derived from any two points £0 > #o . Now simply apply Theorem 3. We now construct the simple example mentioned at the beginning of this section to show that even though a strictly stationary process {xn} may not be a one-sided function xn = g(%n , £w_i , • • •) of the random variables (42), it may have such a representation in terms of random variables (£ satisfying part (a) of problem 1 and defined in a different and more suitable manner. Set up a sequence of points 6t , i = 0, =fcl, • • • , with 0 < • • • < &»• < bi+l < .. • < 1. Let S C (0, 1) be (55)

S = 0 %=—00

{y\hi < V ^ b2i+1}

and S = (0, 1) — S. Let \xn) be the strictly stationary Markov process with transition probability density

(56)

p(y\y') - l p ' / ( 6 ' + 1 ~

5i)

iPi/ih - 6,-0

if fei < y

if

-

hi+1

h-x
'

v eS

'

r

'

y e8 ,

where the p t 's are distinct positive numbers with 23 P* = 1- Consider the family of transformations Ta generated by the conditional distribution function of the process {xj. If X)*--« P, < « ^ £ j — » P, , then bt < Tax ^ bi+l when a: e 8 and b ^ < T a # g 6,- when a; £ S. Thus if we start out with two points x} xf with x z S, x' z 8 their transforms Tu • • • T^o:, T^ • • • T^s' will never be both in 8 or both in 8. Moreover, at any stage n we can separate the transforms by a quantity bounded below by a positive number by specifying an appropriate condition of the form / ? < ? „ < 0'. Thus the condition of Theorem 3 is not satisfied and hence \xn) does not have a representation of form (43) with the £n's given by (42). Nonetheless the process \xn] is of the same type as the process {yn} constructed in the proof of Theorem 2. By using the construction given in Theorem 2 we can generate random variables & satisfying the conditions of part (a) of problem 1 and such that xn = g(& , ££-1 > * • •)• In conclusion, it is worthwhile noting that problem 1 can be rephrased in the terminology of information theory [4]. The original process is passed through a deterministic channel that manufactures independent random variables £n 161

STATIONARY PROCESSES

681

independent of the past, that is, independent of xn„x , xn„2 , • • • . We ask under what circumstances the original process {xn} can be expressed in terms of the output of the channel {£„} with the time sense preserved, that is, when xn = g(Zn , £«-i , • • •) for some Borel function g. REFERENCES [1] J. L. DOOB, Stochastic Processes, New York (1953). [2] W. FELLER, An Introduction to Probability Theory and its Applications, New York (1950). [3] G. KALLIANPUR & N . W E I N E R , Non-linear prediction. Technical Report No. 1 (1956), Office of Naval Research, Cu-2-56-Nonr-266, (39)-CIRMIP, Project NR-047-015. [4] A. I. KHINCHIN, Mathematical Foundations of Information Theory, New York (1957). [5] P . LEVY, Theorie de l'addition des variables aleatoires, Paris (1937). [6] M. LOEVE, Probability Theory, New York (1955). [7] M. ROSENBLATT, Functions of a Markov process t h a t are Markovian, Journal of Mathematics and Mechanics, 8 (1959), pp. 585-596. [8] H. WIELANDT, Unzerlegbare, nicht negative Matrizen, Math. ZeiL, 52 (1950), pp. 642-648. Indiana University* Bloomington, Indiana T h e author is now located in the Division of Applied Mathematics, Brown University.

162

ASYMPTOTIC DISTRIBUTION OF EIGENVALUES OF BLOCK TOEPLITZ MATRICES M. ROSENBLATT 1

Communicated by Edwin Hewitt, June 1, 1960

Let g(X), —T^\^TJ be a pXp (£ = 1, 2, • • • ) matrix-valued Hermitian function. Further g(X) is bounded on [■— TT, TT], that is, its elements are bounded on [ — TT, 7r]. The Fourier coefficients (1)

e*x*(X)<*X,

*» = — f

* = 0, ± 1, • • • ,

2ir J -*

are then bounded in k. We call the npXnp

matrix

An = (aj-*;j,k = 1, • • • , » ) (an n X ^ matrix of the £ X £ blocks a,-*) the nth section block Toeplitz matrix generated by g(X). Notice that the block Toeplitz matrix An is generally not Toeplitz. Our interest is in obtaining the asymptotic distribution of eigenvalues of An as n—^vo. The proof is suggested by an argument given in the one-dimensional case (p—1) (see [3]) and is based on results in the multidimensional prediction problem [5]. If the real number a is sufficiently small in absolute value /(X) = [lP+ocg(K)] is positive definite for all X and bounded (Ip is the identity matrix of order p). Let Rn = Inp-i-aAn be the nth section block Toeplitz matrix generated by/(X). Further denote the (i, j)th block element (pXp matrix), if j = l, • • • , n, of the inverse R„x of Rn by nTfj^. The basic result on the determinant of the prediction error covariance matrix in the multidimensional prediction problem [5] tells us t h a t limdet(nfn

)

= (2?r)*exp <— I

W-+00

log det (

\2wJ-r

\2w

)dX> /

)

since / ( X ) / 2 T can be regarded as the spectral density function of a ^-vector weakly stationary stochastic process. However, det ( n rn )

= det (jR»)/det

(see [l, p. 21]). Let X„,w, v=l, 1

(JR»-I)

=
• • • , np9 be the eigenvalues of An.

This research was supported in part by the U. S. Army Signal Corps.

320 R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_18, © Springer Science+Business Media, LLC 2011

163

ASYMPTOTIC DISTRIBUTION OF TOEPLITZ MATRICES

321

Now np

det (Rn) = I I (1 + ak,tn) and 2

1 A

2

1 ££,

lim log
n

v=i

Thus lim — £ log (1 + a\v,n) = — I log det (/, + «g(X))dX. On taking $»,* = X^=i X*,», it is readily seen that

for | a | sufficiently small (tr(A) denotes the trace of -4). This cannot hold unless (2)

lim ^

= ~

f *tr(g*(X))<*\.

L ^ jui(X), • • • , /Xp(X) &e £/ze eigenvalues of g at X, let us say for con venience in order of magnitude. Then relation (2) implies that the asymptotic distribution of eigenvalues X,,» = 1, • • • , pn, is given by (3)

lim

number of eigenvalues < a 2 n_

n-+co

pn

=

1 ? _ _ _ £ Pfo(*) ^ o] p

y=i

where X is a random variable uniformly distributed on [ — x, TT]. Aside from the interest in this result for its own sake, it is clear that it sug gests a number of good approximations for the joint distribution of spectral estimates in the case of multidimensional normal stationary processes [2; 4], REFERENCES 1. F. R. Gantmacher, The theory of matrices, vol. 1, New York, Chelsea Publishing Co., 1959. 2. N. R. Goodman, On the joint estimation of the spectra, cospectrum and quadrature spectrum of a two-dimensional Gaussian process, Scientific Paper No. 10, Engineering Statistics Laboratory, New York University, 1957, p. 168. 3. U. Grenander and G. Szego, Toeplitz forms and their applications•, University of California Press, 1958. 4. M. Rosenblatt, Statistical analysis of stochastic processes with stationary residuals, The H. Cramer Volume, Stockholm, Almquist and Wiksell, 1959. 5. N. Wiener and P. Masani, The prediction theory of multivariate stochastic processes, Acta Math. vol. 98 (1957) pp. 111-150. BROWN UNIVERSITY

164

Limits of Convolution Sequences of Measures on a Compact Topological Semigroup M. ROSENBLATT Introduction. Limit properties of the convolution sequence of a regular measure on a compact topological semigroup are examined in this paper. Similar questions, as they arise in the case of a compact group, were examined byKAWADA & ITO [4], Recently BELLMAN [1] and GRENANDER [3] considered special limit theorems for products of independent identically distributed random operators. Such problems are closely related to those in this paper. It should be noted that similar questions arise when considering the structure of stationary stochastic processes [7]. Various results on compact semigroups are used in characterizing the class of limit measures [5], [6], [8]. Basic Concepts. Let S be a compact Hausdorff space. Take (B as the Borel field generated by the open sets of the topology. Further, let S be a topological semigroup. This means that there is a product for pairs of elements of S and that the product operation is continuous with respect to the given topology (see [5], [6]). Let (B X (B be the Borel field in the product space S X S generated by the Borel field (B in the component spaces. The set (1)

AB = {(x, y)

\xyzB)z($>X($>

if B 8 (B since it is true for open sets B. We can define the product measure v X M on (B X (B in the usual manner [2] and the convolution v * \x of two measures as (2)

v * ix(B) - f cB{xy) d(v X **)((*, 2/))

for B z (B, where cB is the characteristic function of B. In Lemmas 1 and 2 some simple properties of a measure on a topological space are obtained. No use is made of the semigroup property. Lemma 1. Let v be a measure on a Borel field 9 of a topological space. Let $ be the collection of sets E z (B such that for any e > 0 there are open and closed sets This research was supported by the Office of Naval Research under contract Nonr 908(10). Reproduction in whole or in part is permitted for any purpose of the United States Government. 293 Journal of Mathematics and Mechanics, Vol. 9, No. 2 (1960). R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_19, © Springer Science+Business Media, LLC 2011

165

294

M. ROSENBLATT

0, C such that (3)

CQECO

and (4)

v(0 -E)

<e,

v(E - C) < e.

Then ff is a monotone field. Consider any two sets Ex , E2 t $. Given any e > 0 there are open sets 0< and closed sets C< , (5)

CtCEtCOt,

* = 1,2,

with (6)

APt - Et) < h,

v{E, - C<) < i«.

But (7)

v{01 U02-E1U

E2) ^ K(Oi - #i) U (02 - E2)) < e,

v(E1 VJ E2 - Cx W C2) ^ viiE, - C,) VJ (# 2 - C2)) < e. Thus ExKJ E2z JF. We now show that Ex — E2t $?. There is an open set Oi,2 and a closed set C2 such that (8) with (9)

El\JE2C01,2, KOi.a -ErU

C2CE2,

E2) < h,

v(E2 - C2) < *e.

Now Oi,2 — C2 is open and Ex — E2 C 0 l i 2 — C2 . Further, (10)

0 1)2 - C2 - ( ^ - # 2 ) = (01>2 - # t W E2) \J (E2 - C2)

and hence (11)

v(0 li2 - C2 - (Ex - E2)) < e.

Note that the empty set 0 and the whole space B obviously belong to JF. One can reduce the problem of rinding an approximating closed set within Et — E2 to the argument given above by complementation. Thus Ex — E2 t 3\ Now consider a monotone sequence of sets En c JF, En \ E. We shall show that E t 5. Let n be so large that v(E — En) < §e. There is then a closed set C C E» with v(En - C) < h- But then v(E - C) < e. Given each En there is an open set On , En C On , with v(On - En) < e/2B+1. Now 0 = \J On is open, E C VJ On , and (12)

KO - £) £ *(U (0. - 1Q) < €.

Thus 5 is a monotone field. 166

CONVOLUTION SEQUENCES OF MEASURES

295

// v is a measure on the Borel field g and the monotonefield$ of Lemma 1 is identical with S, we say that v is regular on g. Lemma 2. If v, ix are regular measures on (B, the product measure v X fx is regular on (B X (B. If the product measure v X \x is regular, the factor measures v, fx are regular. Clearly the product sets with projections on the factor spaces elements of (B have the property of Lemma 1. Therefore the cr-field (B X (B generated by these sets has this property. Conversely let v X M be regular and B e (B. Then there is a closed set C e
v{B - C") = y X /*(£ X 0 - CO ^ ? X fx(B X fi - C).

Lemma 3. If v, \x are regular measures on (B, the convolution v * \x is regular {on (B). Let B e (B. Set (14)

A* = {(«,K) laveJ?}.

Now AB e (B X (B, and hence by the regularity of P X M there is a closed set C C AB with * X ix{AB - CO < €. Let (15)

C = {s | 2 = xy for some pair (x, y) e C'}.

It is clear that C C Ac C ^ B . Thus (16)

v * M(B - C) = » X M(AB - Ac) < 6.

Further, C is a closed set since it is the image of a continuous function on the closed set C . We can find an open set containing B with the desired properties by complementation. Let us now consider the family of regular probability measures 911 on the compact topological semigroup S. We shall say that the sequence of regular probability measures txn on S converges to a regular probability measure fx, /xn —»ju, if

(17)

/ fd^-iffdv

for every continuous function f on S. Since the continuous functions on a compact space are bounded, it is clear that it is enough to have convergence for continuous functions /, 0 ^ / ^ 1. 167

296

M. ROSENBLATT

Let / be the closed unit interval [0, 1]. Consider the product space H / If where the coordinate spaces If are indexed by continuous functions /, 0 ^ / < 1, with the usual Tychonov topology [2]. H 7/ is a compact space since If is compact. Each measure M E 3TC can be considered as a point in J J If with component in the fh coordinate space / / = = / / dp. Let us consider a limit point c c H 7/ of the family of measures 9TL Clearly cf 2£ 0 for all / and c% = 1 (1 is the function / « 1 ) . Further, given any two functions fx , /2 , 0 ^ jx , f2 , /i + U = 1? a n d any e > 0 there is a measure fi t M in the neighborhood {C' | K - CA| < €, K ^ C/t| < 6, K

(18)

+ /2

- C / l + /2 | < €}.

But this implies that cfl+fa = cfl + cf% . Let cf = £(/). L(/) is a functional on the continuous functions /, 0 ;£ / ££ 1. We can similarly show that c a / = acf for a ^ 0 if 0 ^ a/, / ^ 1. Further, |c,| ^ ||/|| if 0 ^ / ^ 1 (absolute supremum norm). We now extend L to all continuous /. First suppose / ^ 0. Then there is a positive constant a such that 0 ^ af ^ 1. Let £(/) = L(af)/a. Thus L(/) is defined on all continuous nonnegative /. If / is not nonnegative, introduce / + = max (0, /) and f = / — / + . Then / + , —f are nonnegative continuous functions. Let L(f) = L(f+) — 7,(—/"). L is linear on the continuous functions since L(af) = aL(f), L(fx + /2) = L(/,) + L(/ 2 ). L is positive, that is, L(/) ^ 0 if / ^ 0. Further, L(j) S \\f\\ for / ^ 0. But then ~f)) ^max(||ni,H-rll) = for all continuous /, and thus ||L|| ^ 1. By the Riesz representation theorem [2] there is a regular probability measure M on S (since L(l) == 1) such that L(f) = / / dp. Therefore c £ 9TC and 9fK, as a point set in J J 1/ , is closed. Thus 311 is compact. (19)

|L(/)I =

|L(/+)

"

L(

~r)l ^

m a x (L(/+}

>

L(

Lemma 4. The set of regular probability measures 3TTT on S is a compact set

in II If • Now let us investigate the continuity of the convolution operation on the set of regular probability measures. First consider a continuous function f(x, y) on S X S. It is easy to see that the set of finite sums of products of continuous functions of the form

(20)

£

g^Hy)

with g4(x), h{(y) continuous on S is an algebra of functions that separate points. For if (x, y) 4= (xf, yf) either x 4= xr or y #= y'. Suppose x =f= xf. Take a continuous function g(x) that separates x, xf and let h(y) s= 1. Then g(x)h(y) = g(x) separates (x, y)} (a/, yr). Since S X S is compact, by the Stone-Weierstrass theorem any continuous function f(x, y) on S X S can be uniformly approximated by a function of the form (20) with g{ , h{ continuous on S. We shall now prove Lemma 5. 168

297

CONVOLUTION SEQUENCES OF MEASURES

Lemma 5. The convolution operation is continuous on the probability measures arc. It will be enough to show that the inverse image of a set in the subbasis for regular probability measures is open. Let us consider two probability measures v, ix t 9TL A subbasis neighborhood of v * fx is of the form lc | J / d v * ft - C < e>

(21)

where / is a continuous function, 0 ^ / :£= 1. Now (22)

f i(x) d v * /*(») = f f(xy) d v X /*((*, v))>

But / is a continuous function of (x, y), and hence there is an approximating function if

(23)

u(x, y) = £ gi(x)ht(y)

with hi , gt continuous on S such that (24)

\u(x, y) - f(x, y)\ < |e.

Then (25)

\J M%> V) - K%y)] dvX

ti((x, y))< ze.

Let M < oo be such that [g^, \h{\ < M for all i. Consider the following neighborhoods of v, n respectively: N(v) =rn\\j

9
,NJ,

N(fi) = }v \ \

hi{x) di)(x) — I h{(x) dy,(x) < 8;i = 1, •••

,N\-

(26)

If / e N(y), ix' e N(jx) then (27)

|J

/ u(x, y) dvX N

^ E

n&x, y)) -

J

/ u(x, y) dv' X n'((x, y))

I p

/ 9i(x)ht(y)[d v X /*((«, V)) - dv' X ix'{(x, 2/))]

and thus if 5 < e/4NM, (28) f fdv*

n - f fdv' * ix'

<

^ 2N &M,

€.

The following theorem is an immediate consequence of the results obtained thus far. 169

298

M. ROSENBLATT

Theorem I. The family of regular probability measures 9TI on S with the convolution operation as a multiplicative operation is a compact Hausdorff topological semigroup. Limit Properties of Convolution Sequences. Let v be a regular probability measure on S. We shall now consider limit properties of the sequence of measures va) = v, J>(2) = v * *>, • • • , v{n+l) = vin) * v} • • • generated from v by convolution. A classical probability problem of this sort arises in considering the limit properties of sums of independent, identically distributed random variables. The values taken on by the random variables (since they are numerical-valued) commute with each other. In our case, we are interested in limit properties of products of independent, identically distributed elements of a semigroup (possibly a semigroup of operators) and a non-abelian semigroup is of considerable interest to us. The notion of the spectrum of a regular measure v on S is of particular interest. The spectrum X(v) of a measure v e 91Z consists of the points s e S such that any open set 0 containing s has positive v measure. The spectrum is clearly a closed set. Given any two subsets A, B of the semigroup S let AB = {ss' \ s t A, s' t B}. Now the closure of the set of elements \Jn (2(*0)n is a closed semigroup and hence is compact. This is the relevant semigroup insofar as the sequence of convolutions vin) is concerned. Let us call this subsemigroup the semigroup generated by the spectrum of v. Since this semigroup is of importance in considering the limit properties of the sequence v(n), we shall take S = \<Jn (X(v))n (A is the closure of A). An element e of S is called an idempotent if e2 = e. Every compact Hausdorff semigroup has at least one idempotent [5], By a left (right) ideal of S we mean a nonvacuous subset L (R) of S such that SL C L (RS C. R).If M is both a left and right ideal of S, M is called a (two-sided) ideal of S. The minimal twosided ideal K of S exists and is given by K = f}e SeS [5]. The minimal ideal K is called the kernel of the semigroup S. K is closed and hence compact [5]. We shall say that M is a limit measure of the sequence v(n\ n = 1, 2, • • • , if there is a subsequence nk such that n = lim/ w *\ Theorem 2. Let v be a regular probability measure on the compact Hausdorff semigroup S. Every limit measure /x of the sequence v(n) (v regular) has all its mass concentrated on the kernel of the semigroup generated by the spectrum of v. It is clear that we can take S = \J{*Z{v))n- Let k be an element of the kernel K of the semigroup generated by the spectrum of v. Let N(k) be any neighborhood of k. Then for m sufficiently large there are elements sx , • • • , sm t 2(*>) such that XIr=i S* £ N(k). By the continuity of the multiplicative operation there are neighborhoods N(s{) of s4 , i = 1, • • • , m, such that I l f - i #($<) C N(k). Let 0 be any open set containing K. Then there must be a neighborhood of &, N(k), 170

CONVOLUTION SEQUENCES OF MEASURES

299

such that SN(k), N(k)S C 0. For given any point s e S there are neighborhoods Nu(k)9 N28(k) of k and a neighborhood N(s) of s such that Nls(k)N(s), N(s)N2a(k) C 0 since SK - KS = X. Let #.(fc) - #lt(fc) H #2.(fc). Then N(s)N8(k), N,(k)N(s) C 0. The open sets N(s) cover $, and hence there are a finite number of &)% i = 1, • • • , n, such that the N{s)) cover $ (because of the compactness of S). N(k) = f \ Nail(k) is a neighborhood of k such that SN(k), N(k)S C 0. Given this neighborhood N(k) of & choose a finite number of elements $i , • • • , sw with corresponding neighborhoods iV(s*)> * = 1, • • • , m, such that HiT-i ^( s *) C N(k). Now consider the infinite product space of sequences (tx , 12 , • • •) of elements t{ of £ with product measure generated by v. Now v(N{Si)) > 0, i == 1, • • • , m. By the lemma of Borel-Cantelli there is at least one block of m successive elements from N(st), • • • , 2V(«m) respectively in any such sequence with measure one. Take n sufficiently large so that the measure of the set with a block of m such successive elements in truncated sequences of length n is greater than 1 — e. It then follows that v(n+1) (0) > 1 — e. But this implies that for any limit measure /* of the sequence v(n), M(0) = 1. However, this is true for any open set 0 containing K. Therefore p(K) = 1. Lemma 6. Let /((#, y)) be a continuous junction on S X S where S is a compact Hausdorff space. Then the family of functions fy(x) = f((x, y)) defined on S and indexed by y t S is eguicontinuous for all y t S. To say that the family {fy(x)} is equicontinuous is equivalent to stating that for any given e > 0 and any x e S there is a neighborhood N(x) of x such that for all x' z N(x) (29) |/,(*0 ~ /,(*)| < e independently of y. Suppose the family is not equicontinuous. Then for some e > 0 and some x t S there is no neighborhood N(x) of x such that \fy(x') — fy{x)\ < € for all x' z N(x) and all y. Index the neighborhoods Na of x by a. Given any neighborhood Na of x there is a point (xa , ya) z S X Sf xa z Na 9 such that \fya(xa) — fya(x)\ ^ e. The indices a are partially ordered if we set a ^ af when N«* QNa. Further, the indices are a directed set since any finite subset of them has an upper bound. Thus {xa} and {(xa , ya)) are generalized sequences of elements in S and S X S respectively [2]. Further, lim„ The generalized sequence (xa , ya) has a cluster (limit) point (x, y). But then \fv(x) — fy(x)\ = |0| = 0 ^ e > 0, a contradiction. Theorem 3. Let v be a regular probability measure on the compact Hausdorff semigroup S. There is then a unique limit measure of the averaged convolution sequence

(30)

M=

n

Further,

(31)

limply".

M

n p[

* „ = „ * M = M(2) = M# 171

300

M. ROSENBLATT

Consider the regular probability measure v as an operator T on the family of continuous functions / on S} that is, (Tf)(x) =

fj{xy)dv{y).

The family of functions fy(x) = f(xy) is an equicontinuous family of functions by Lemma 6, and hence T maps the continuous functions into continuous functions. Consider iterates of the operator T, Tn, acting on /. Then (32)

(Tnf)(x) = f (T^fiixy)

dv{y) = f f(xy) dv{n\y).

Let

(33)

r = ~ £ r.

The sequence of functions (Tnf)(x) is an equicontinuous family of functions since fy(x) = f(xy) is equicontinuous. This implies that (Tnf)(x) is equicontinuous and hence {Tnj} is strongly compact (sequentially). Further, ||T|| = 1 and Tnf/n ~-» 0. Therefore Tn ~» T strongly where T is linear, (34)

||f|| = 1,

TT = TT - T2 = f,

and !f is positivity preserving. Hence there is a regular probability measure y such that (35)

f Tf(x) dv{x) = f f(x) duix).

But then (36)

\ f fa) d( E ^"(s)) - / /(or) dM(x)

for all continuous /, and hence (l/ri) 23?-1 ? (,) "~* J*. Limit Measures on the Kernel of the Semigroup. Consider limit measures of the kind discussed in Theorem 3. These measures are the idempotents in the semigroup of probability measures on S. For simplicity, we restrict ourselves to those measures v in this semigroup of measures whose spectrum generates S. By Theorem 2 the limit measure ju of (1/n) 23i ? (,) is concentrated on the kernel K of S. In Theorem 4 we shall show that the spectrum of a limit measure /x is K. The detailed structure of such measures will be examined in this section. However, we will require certain basic results on the structure of the kernel if of a compact Hausdorff semigroup. When studying K> it is natural to look at it with the relative topology, if is a completely simple semigroup, that is, KkK = K for all k z K and K has a minimal left and a minimal right ideal [8]. Every compact completely simple semigroup can be represented as the product 172

CONVOLUTION SEQUENCES OF MEASURES

301

space T X X X Y of a compact topological group T and compact Hausdorff spaces X and Y where the multiplication is given by (t, x} y)(t', z', yf) = (t y y)

(37)

with
0. Any point k t K has the form k = (t, x, y). Let x = x(k), y — y(Jc) be the x, y coordinates of k. Further, let NXtV(k) be the collection of all points k (in K, of course) with x, y coordinates the same as those of points in N(k) r\ K. Take k' to be an element in the spectrum of fi. There is then a neighborhood Nx(k) such that N1(k)kfN1(k)

(38)

CNx,y(k).

This follows from the continuity of the multiplicative operation. But then n(NXtV(k)) > 0 for any neighborhood N(k) of k. For (39)

/*(#..,(*)) « / <*..,<»>(W3) dv^is,) d»(s2) dvin)(sz)

and for some n (40)

*(n)(A^(&)) > 0.

Thus there is a point k" e NXfV(k) in the M spectrum. Given the neighborhood N(k) there is a point &' e i\T(fc) with a corresponding point fc" t Nx,y(k), x(k") = #(&'), y(k") = 2/(&')> that is, in the /x spectrum. There are then points k{', k'2f such that k['knkf2> = kf. This implies that there are neighborhoods N(k['), N(k"), NQc'2') of k[f , &", k'2f such that (41)

N{k[')N{k")N(V2') C W ) C N(k)

by the continuity of the multiplicative operation and the structure of K, Then all of K is the spectrum of ju since (42)

v™ * M * »(m) = /i.

The proof is complete. Let us now consider a product measure p on K of the form (43)

!z =

xXaX(3

where x is the normed Haar measure (x(T) == 1) of the compact topological group T and a and # are regular probability measures on X, Y respectively. 173

302

M. ROSENBLATT

We shall show that such a measure is an idempotent on K. First consider a set G t (B which is a product set, that is, G = A X B X C where A, B, C are sets of the Borel fields on T, X, Y respectively. Let $~XG = {$' | ss' t G}. But (44)

p(2\G) « f p(s^G)p (ds) » [

pis'^Pids).

Now (45)

Afr-1©) - £ £ x(Mx(s), y")TlA)

da(x') dfi{y") -

x(A)a(B)

by the invariance of Haar measure. Thus (46)

p(2\G) -

f

x(A)«(B)/i (da) = x(A)a(B)p{0 = /z(G).

Since Pm(G) = /z((?) for all product sets, the relation holds for all G in the field of finite sums of product sets and hence for all G in the Borel field (B generated by these sets. Lemma 7. Let K be a completely simple compact semigroup. Any measure p of the form (43) is an idempotent measure on K. We shall now show that for a fairly wide class of completely simple compact semigroups, the p measures are the only idempotent measures. Theorem 5. Let K be a completely simple compact semigroup with the function
f(k) - ftCidtoOc).

Now \f/(k) is a continuous function of h since

v{A) = M(*(fc) e A)

on the compact group T. Here A is an element of the Borel field on T. Since M is an idempotent measure on 2£, it follows that v is an idempotent measure on T [4]. The spectrum of v is the whole group T since the spectrum of M is the whole semigroup K. However, this implies that v is the normed Haar measure of T. Let /49x

MA, C) = fi(k | Upx(x) zA,yt

C)}

v2(A, B) = fx(k |
CONVOLUTION SEQUENCES OF MEASURES

303

Now if ju is an idempotent measure M(3> = i"> and hence (50)

- f v2 (dt, B)v{rxAt)Vl

v(AXBXC)

(df, C) - *>2(-, £ K ( - , C>(ii)

since *> is the Haar measure of Z\ Thus ju is a p measure. Corollary. If S is a compact Abelian Hausdorff semigroup, the kernel of S is an Abelian group. Any idempotent measure is concentrated on the kernel. If the spectrum of the measure is the kernel, it is the normed Haar measure of the kernel. This is almost an immediate consequence of Theorem 4. We shall now show that the result of Theorem 5 is true for a general finite completely simple semigroup. This makes one suspect that Theorem 5 very likely holds without the restriction on
(51)

M(*0

= E

Kk'k-'Mk)

k

where k'k'1 = {a | ak = k'\. Note that k'k'1 is the null set except when x(k) = x(k'). Thus /iCfc"*;-1) =

(52)

k'

Yl^kf'k~1k'-1)^k').

But V'k'1 = {a | ak - k"}, k'^-'k''1 = {b \ bk'k = k"} = k'T"'1 where k'" = k'k, so that Wk-1)

(53)

Y,n(k"k'~%(k'k-1).

= k'

We can then regard ^{kk'"1) = p(k \ k') as the conditional probability of going from kf to k for a Markov chain with state space the states k. The measure fx assigns positive measure to each point k. Notice that there is an irreducible class of states for the Markov chain with transition probabilities ix{kk'~l) corresponding to each distinct x value. By the classical results on Markov chains {k) V{k\k')=\P

(54) This tells us that

,/ / 1 M(fc * " )

(55)

if

*M-*®> { 0 otherwise.

= M( U ( r r V f e yT1, x, y")) X

is independent of t', y' if x(k") — x(k'). But expression (55) is then independent of t", f, y'. Then (56)

„(&') =

Z

nik'k-'Mk)

=

x(.k')~x(.k)

£ x{k)~x'

175

a(x,2/')M(s) =

Kx',y').

304

M. ROSENBLATT

Thus

(57)

/*((•, x', y')) = E *»((•, *", vOMit, *', V)) = *V)W) x*'

,tty

and

(58)

»((t, x , y)) = I a(x)b(y) 71/

where n is the number of elements in the finite group T. Notice that if K contains exactly one minimal left (right) ideal, the idempotent measure /* is an r* (Z*) invariant measure in the sense of ROSEN [6]. A Special Class of Semigroups* In certain contexts it is of especial interest to determine whether the semigroup S is such that all elements of the kernel K are left (right) annihilators, that is, kS = k (Sk = k) for every k t K. This is of particular interest in certain structure problems for stationary processes (see [7]). We shall show that in this case not only do the averaged convolutions converge, but also the convolution sequence itself. Theorem 7. Let S be a compact Hausdorff semigroup with kS = k (Sk = k) for every k t K, the kernel of S. If the semigroup generated by the spectrum of the regular measure v on S is S, lim vin) =

(59)

M

n—*co

exists with M having K as its spectrum. Suppose kS = k for all k t K. Let

(60)

M

= limiiyn.

We shall show that n = lim„ v{n)'. Let N(k) be a neighborhood of k c K. For sufficiently large n vM(N(k)) > 0. Let KN = {k' e K \ ¥ e N(k)}. Take G C KH a closed set of points such that n(C) > n(KN) — e, e > 0. Given a point k' t C, there is a neighborhood N(k') of k' such that N(k')S C N(k). For, given each point s 2 S, there are neighborhoods N(s) of s and iV.(fc') of k' such that N,(k') N(s) C iV(fc). Since S is compact there are a finite number of points s< such that iV(s,-) cover S. Let #(&') = O #.,(&')• It is then clear that N(k')S C #(&)• Now \Jk,tC N(k')S C A^(/c). Let N = U * £ c #(&')• Then for some sufficiently large n, vM(N) > n(K„) - e. But then for all m > n (61)

v'm\N{k)) = f vM(N(k)s-iym-n)

(ds) ^ pM(N) > n(KN) - e.

Thus (62)

lim v(m\N(k)) ^ ix(N(k))* m—>co

176

305

CONVOLUTION SEQUENCES OF MEASURES

This is not possible unless (63)

limvim\N(k))

= n(N(k)).

m-+a>

Thus lim v = /z. The proof is essentially the same when Sk = k for every hzK. A very simple but interesting application of Theorem 7 can be given. Let v be a regular measure on the n X n probability matrices, that is, matrices with nonnegative elements and row sums one. Suppose the spectrum of v contains a matrix with identical rows. The assumption of Theorem 6 is then satisfied by the semigroup of matrices S generated by the spectrum of v. Thus lim vim) exists and is concentrated on the kernel K of S which consists of matrices with identical rows. Let ax, • • • , an, ]T) a{ = 1, be the random elements of the common row for the limit measure concentrated on the matrices of K. Then the characteristic function
n-l

z^? r

I n-\

= E exp Y X) s»( Z) akVu + (64)

- Ej^E^exp U £

Zipni>

n-1

"j

N^

1 " X) ak \PniJf

exp U £ z1 £ (pki - pnj) \ pkjjjj

BIBLIOGRAPHY [1] R. BELLMAN, Limit theorems for noncommutative operations I, Duke Math, Journ. 21, (1954). [2] N. DUNFORD & J. T. SCHWARTZ, Linear Operators, Part I, 1958 (New York). [3] U. GRENANDER, Some nonlinear problems in probability theory, H. Cramer Volume (1959). [4] Y. KAWADA & K. ITO, On the probability distribution on a compact group I, Proc. Phys.Math, Soc. of Japan, 22 (1940), pp. 977-998. [5] K. NUMAKURA, On bicompact semigroups, Math. Journ. of Okayama Univ. (1952), pp. 99-108. [6] W. G. ROSEN, On invariant means over compact semigroups, Proc. Amer. Math. Soc, 7 (1956), pp. 1076-1082. [7] M. ROSENBLATT, Stationary processes as translates of functions of independent random variables, Journ. Math, and Mech., 8 (1959), pp. 665-682. [8] A. D. WALLACE, The Rees-Suschkewitsch structure theorem for compact simple semigroups, Proc. Nat. Acad. Set. U.S.A., 42 (1956), pp. 430-432. Indiana University Bloomington, Indiana

177

Addendum to "Limits of Convolution Sequences of Measures on a Compact Topological Semigroup"* M. ROSENBLATT

It should be noted that in Theorem 5 the functions
681

178

U.S. copyright law (title 17 of U.S. code) governs the reproduction and redistribution of copyrighted material.

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_20, © Springer Science+Business Media, LLC 2011

179

INDEPENDENCE AND DEPENDENCE M. ROSENBLATT BROWN

UNIVERSITY

1. Introduction A stochastic process is commonly used as a model in studying t h e behavior of a random system through time. It will be convenient for us to t a k e the sto chastic process {xt} as discrete in time I = • • •, —1, 0, 1, • • ♦. Processes of inde pendent random variables are the simplest and most completely understood. I t is, however, clear that these are extremely limited in scope as models and one must have recourse to dependent processes (the random variables xt not independent) in order to have any power in description. For simplicity, lei us further restrict ourselves to processes t h a t are stable through time, stationary processes. For such processes the probabilities of events shifted through time remain the same, that is, the probability (1)

P{xti+h ^ ai, • - , zin+h S an]

is independent of h. Such models occur fairly often in the physical sciences. If mean properties of the process are to be capable of being estimated reasonably well from part of a realization of the process, some form of asymptotic inde pendence for blocks of random variables of the process that are widely separated must be satisfied. This is, in effect, the gist of many of the results in ergodic theory. Two types of interesting problems are posed. The first of these is con cerned with reasonable notions of asymptotic independence and what types of proc esses satisfy them. The second is that of characterizing those processes {xt} thai can be constructed, out of independent processes by a function and its shifts, that is, (2)

^ = / ( " ^ ~ i , ^ V i , •••)

where {&} is a process of independent random variables. Neither of these ques tions have elicited satisfactory answers. However, there are some small results t h a t do give insights into the problems. The object of this paper is a presentation and discussion of a few of these limited results. 2 . Mixing Ergodicity itself might be thought of as a form of asymptotic independence. However, the most obvious formulation of asymptotic independence is the This research was supported by the Office of Naval Research. 431

180

432

FOURTH BERKELEY SYMPOSIUM: ROSENBLATT

stronger mixing condition. Suppose T is the shift (in time) operator associated with the stationary process. The process is said to be mixing if (3)

P{A 0 T»B) -» P{A}P{B}

as n —► co for any two events A, B. The normal stationary processes are the simplest stationary processes and Maruyama [6] has obtained a simple and intuitive necessary and sufficient condition for such a process to be mixing. To verify the mixing property, it is enough to verify property (3) for any two events A, B determined by conditions on a finite number of the random variables Xi* MAKTJYAMA'S THEOREM. A necessary and sufficient condition for a normal sta tionary process to be mixing is thai the covariance rt = Cov (#r+*, xT) tend to zero as jif| —^ co. It is clear that no generality is lost by assuming that the mean of the process is zero. The necessity of the condition that rt —* 0 as \t\ —> <*> is then almost com pletely apparent. We approximate the random variables xT+t} xT by step func tions in the following way. Let (4)

A(kt e) = {o>|/ce ^ ;eT(a>) < (fc + 1)€}

and J l if co G A(k, e)
(5) Set (6)

4Ar» =

K N

f fcCA(U<»)-

k= — N

Then for sufficiently large N Elx^i*) - zr(o>)\ = EWiT'a)

- xl+r(o>)\ < 8,

\n - ExyiTtyx^Ml ^ Elxt+r^Xrifi)) ~ x^^(Tl)\ Since the mean of the process is zero, for N sufficiently large (8) \Ex™A(co) | = \Ex^(T'«) | < 5. However by the mixing condition (9)

Exi™(T)x^\o>) =

£

kejeEcA^AT'^CAuM

h J * — iV

-

£

kjW[AU,e)]P[A(k,e)]

ktj « — N

But this limit relation coupled with the inequalities (7) indicate that (10)

fim

\rt\ < 35

and since this is true for any 5, we have lim^-** rt = 0. 181

< 8.

INDEPENDENCE AND DEPENDENCE

433

The sufficiency of this condition follows just as easily. Let A, B be determined by conditions on blocks of k successive random variables (these could in fact be taken to be the same blocks if k is sufficiently large). Then (11)

P{A O T*B) = {2*)-h\M\-w

f A

f

exv(-^(ii,v)M-i(uivy\dudv)

B

where uf v are two row /c-vectors and (u9 v) the 2&-vector formed by joining them. The matrix M is the covariance matrix of two blocks of 7e successive random variables of the process separated by n — k random variables. Write M in the form - (Rl

R

^

where Ri is the k X h covariance matrix of a block of k successive random vari ables and R2 the k X k matrix of covariances of the one block of k with the other block removed from it by n — k random variables {R% is the transpose of R2). The fact that rt -* 0 as \t\ —>co implies that M~l exists. It is in fact given by (13)

M-

(A B\ \B' C/

with A = {Rx- RzRr^z)-1, C - (Bi - BiBf 1 ^)- 1 , B = -R^RJJ 1 ~ARiRr . The matrix Rx is fixed since k is fixed. But i?2 is a 7c X 7c matrix whose entries are the covariances rt with \t\ > n — 7c. They approach zero uni formly as n -> 00 so that E 2 approaches the null matrix. Thus (14) (15)

M- 1

'Bf 1 .

0

P{ii n T*B} -> (27r)-fc|i21]"1 f f exp j - | [uB x -V + t>22r V] j
- P{4}F{J3}. Stronger mixing conditions are sometimes invoked in deriving results like central limit theorems for dependent processes. The following condition is an example of such a strong mixing condition: Let A} B be any events determined by conditions on the random variables Xh9 k ^ my and Xk, k ^ n, respectively with n > m. The process {xk} is said to satisfy the strong mixing condition M if (IB) \P{A n B} - P{A}P{B}\ S d(n - m) for all such events A} B and some function d where d is a function on the positive integers n that decreases to zero as n —» <*>. Notice that such a mixing condition is a uniform mixing condition in what we might call the distance n — m between the events A} B, Let {%ic} be a stationary process with mean zero} Exk ^ 0, thai satisfies the strong mixing condition M, Further lei the process satisfy the two moment conditions

182

434

(17) (18)

FOURTH BERKELEY SYMPOSIUM: ROSENBLATT

#|jba:y|»= £ | £ *;|4 = £

E

(n - \k\)n = h(n) ~+&

as

w-*«>,

Ezj&sw*

71

W(ai, a2, a3)

= 0[h(n)f. The sums E"=i^V can then be shown to be asymptotically and nontrivially normally distributed when suitably normed as n —* co. Of course, the statement of nontrivialness means that the limiting normal distribution does not have variance zero, The proof of this limit theorem can be found in [7]. The follow ing two examples in part at least indicate the limits of the central limit theorem cited above. In the first example the process {#*} is taken to be (19)

Xk =

CA

— €*-i

where the €& are independent, identically distributed nonnormal random vari ables with finite 4th moment and vanishing 1st moment. Since E5Li&& = en — to, it is readily seen that h(n) does not diverge as n —► <*>. Thus, we get a nonnormal limiting distribution, namely that of €i — e0, without norming the sum at all. The process certainly satisfies the strong mixing condition and moment condi tion (18) but not moment condition (17). The difficulty arises because the process {xh} has too little spectral mass in the vicinity of A = 0. Its spectral density is /(A) = (1/2TT)!1 - exp (-tX)l*. The second example has greater interest. Consider a normal process {yu} with mean zero and covariance sequence Vk = (1 + 7c2)~r with 7 > 0, The spectral density of the process {y$ is (20)

a(X) =

£

&(X + 2/CTT)

where

(21)

MX) - b(|x|) = {r(7)}-2e-w fj M^(M + |xl)^- 2 ^ d/i.

The spectral density a (A) is continuous and bounded away from zero if |X| > e > 0. If 7 < 1/2, it has a singularity of the form |A|2r_:L in the vicinity of X = 0. However, it should be noted that the process is in any case, purely nondeterministic and therefore mixing in the ordinary sense. Let the process {xk} be of the form xu = yl — 1. The covariance sequence of the process {xk} is Tu = 2(1 + ¥)-%i. We shall be interested in processes for which 7 < 1/4. The spectral density /(A) of {xu} has a singularity of the form \\\^~l in the neighbor hood of A = 0 as can be seen from (21). We shall see that n~lJrly ££=1 xu has a 183

435

INDEPENDENCE AND DEPENDENCE

limiting nonnormal distribution as n —>» For the characteristic function of (22)

\I - 2iln~l-^R\-li* exp {-in**lr0}

exp«{ E CZiin-^)h

sp (Rh)/h

and (23)

{n-^yf

sp (Efc)

-> J • • • J |OJI - x2\~2y\x2 - x*\-*r • • * |&* - ail""87 dxi • ■ ■ Ac* o 7c

- 0, as n—>oo (here sp (22) denotes the trace of the matrix jR). The characteristic function of the limiting distribution is

(24)

«P{£ <W *}

the characteristic function of a nonnormal distribution. The curious aspect of this example is that moment conditions 1 and 2 are satisfied. We must therefore conclude that the strong mixing condition is not satisfied by {xh}, But this means that the normal process {*/&} does not satisfy a strong mixing condition, since {xk} is derived from it by an instantaneous nonlinear filter, even though {yk} is purely nondeterministic and hence mixing. This is rather surprising. It certainly indicates that there would be a great deal of interest in a result that would character ize the Gaussian stationary processes that satisfy the strong mixing condition dis cussed in this section, 3. Stationary processes generated by independent random variables Most of the stationary processes that arise in practice are generated from independent identically distribution random variables by a linear or nonlinear operation and its shifts. Let rj = (• •«, q_i, ??o, ??i, • ■ •) be a doubly infinite se quence of independent random variables with a common distribution function. Take T as the shift operator defined by (25)

Tt) = v = (• • •> m m, ^ - • •)>

that is; the nth component of i/,(2V)n. = T?H-I- T i i e Processes we are thinking of are generated in the following way. Let g be a Borel function of an infinite number of variables such that the random variable g(rj) has finite second moment -Z%0?)l2 < °°. A stationary process {xn} is generated by taking (26)

s*« g(T*ifi,

184

n = 0j=fcl

436

FOURTH BERKELEY SYMPOSIUM; ROSENBLATT

We can take E{xn} = 0 without any loss of generality. In this section we shall prove a result of Doob and Leibler [3] indicating that the spectral distribution function of any process of this type is absolutely continuous. The form of the proof given is that in [2]. In proving this result it is enough to assume that the ^ are uniformly distrib uted on the unit interval. For if y is not uniformly distributed, one can find a monotone function

Xntm — E\Xn\Vn~7n^l)

' * *, tyn+w-l/ •

For fixed n, {xn,m} is a martingale. It follows from well-known theorems on martingales that .r„|TO converges to xn in mean square and with probability one as m —> oo, Let 6j(uj)} j = 0, 1, • • •, be a sequence of orthonormal polynomials on the unit interval [0, 1] with 83(u) of degree j precisely. It is well-known that the functions {63{u)} are complete in L2[0, 1]. The family of products {fyx(wi) • • • Ojk{uk)} is a complete orthonormal system for L2[Uk] where Ujc is the /c-dimensional unit cube. To get a complete orthonormal system for the infinite-dimensional cube U we take the function identically one and the system of products {fljifami) • • *flfcfym*)}where m* < m2 < • • • < m*, ji, • • •, jk ^ 1 and h ~ 1, 2, • • •. Let k be an arbitrary positive integer. From the discussion given above it follows that jco.A-caii be expanded in the form 2fc-l

(28)

«

a*,* = Z

E

£

C(Wy, Wh)
where ra,- = (mi, • • •, mi), %j = (n1; ■ ■ •, %) and
E

E

E

Cto,?B,)*
Since .TQ = lim*-* #0l/« with probability one it follows that 00

(30)

CO

.To = £

£

E

i * 1 ?/it < • * • <mj

MI,

C(W/, Wly)<SH,(W

• ■ ♦ ,«/ = 1

with probability one. The identity 00

(31)

Xn = E

60

E

E

Cfay,ffiy- n)*>#(lW

/— 1 mi < • •' < m / m, • • ' t ?t/= 1

with Mj — n = (mi — w, * • •, rrij — n) follows in a similar manner. Let /c be a positive integer, n — (m, • • -,ft&)a /c-tuplet of positive integers and m = (mi, • * •, m&) a /c-tuplet of integers with mx < • • * < m&. For any inte ger I let $<(ffJ, 75) be given by (32)

xt(m, n) = £

Cfa m+j- t)
INDEPENDENCE AND DEPENDENCE

437

where m + I = (mi + h • * *, wto + 0- { ^ ( ^ ^)} is a moving average and such processes have an absolutely continuous spectrum, We shall distinguish (M1, W) from (M} fl) )£% 5* W orm — W is not a vector with equal coordinates. Then the processes xt(m} n) and Xt(W, ft') are orthogonal if (m, %) and (ra'3 nf) are distinct in the sense indicated above. There is an enumerable collection of such processes. Let {xP} be the ith process w i t h / w ( X ) its spectral density. Because of the orthogonality of the processes {xl**} the spectral density/ n (X) of £ ? - i xf is given by 2D?=i/Ci)(X). The sequence/ n (X) is nondecreasing in n since the/ (i> (X) are nonnegative. Let/(X) = lim^^/nCX). Then (33)

fx /(X) d\ -

f*

lim/„(X) dX S lim f* /»(X) dX

« lim i
!

2

of } = #{M } <«>.

The function /(X) is finite almost everywhere and integrable. An easy computa tion indicates that /(X) is the spectral density of the process {xt} and the desired result is established. Analogous results for continuous pai*ameter processes have been considered by Anzai [1], Kakutani [5] and It6 [4]. The importance of such processes has been stressed in the work of Wiener [10]. I t follows from the result obtained above that if h is any square integrable function defined on the probability space of x = (* • -, x-ls xo} xh • • *) and if {Vn} is the process given by Vn = h(Tnx)} then {Vn} has an absolutely con tinuous spectrum. One would like to know whether one can characterize such processes in terms of independent random variables with a representation like that of (26). At present this question is unresolved. In fact, not much work has been done on the question of determining those processes that can be given a representation of the form (26). Some insight into this question is given by the discussion in the remainder of this paper. 4. Structural questions for stationary processes We now pose two interesting structural questions for stationary processes. Let {xt}, with i — 0, ± 1 , • ••, be a stationary process and T the corresponding one-step shift operator. Further, let (B» be the Borel field generated by xn) Xn-i, • • •. The first problem can be conveniently split into two parts. PROBLEM 1. (a) When can one find a random variable f0 measurable with re spect to (Bo, independent of x^i, x-2, • * * and such that (Bo =
186

438

FOURTH BERKELEY SYMPOSIUM:

ROSENBLATT

This problem can be easily.rephrased in the language of information theory. The random variable .(•* can be-regarded as the output of a noiseless channel £ that {xn} is sent through, that is, %n =* %(xn)xn-h * • •)• ^ n P a r ^ ( a ) w e a s ^ ^ a t the output at time n, £n, be independent of the past of the input (£n~i, xn-2, • • •) and yet that £n coupled with knowledge of the past of the input be enough to de termine the input xn at present. Thus we can write xn == /(£n* Xn-i3 £n-2, • • 0 and by iteration xn as a function of &, fw_i, * * ■, (•«-*, #»-*-i, £«_ft-2, •• • for any finite k. In part (b) one asks whether one can go to the limit so that xn can be considered a function of (•», £n_ij • • • alone. Essentially the second question can be regarded as a reformulation of the first question where the condition that £ be a noiseless channel is relaxed. PEOBLBM 2. Can one find a sequence of independent uniformly distributed (on [0, 1]) random variables £«, with n = 0, ± 1 , • • ■ and a Borel function g(0 = fftto, £-i> • • •), £ = (• • •, l-i, £o, fc, • • •) such that (34) y. - g{T^) n = 0, ± 1 , ±2, • • • has the same probability structure as #n, with n = 0, ± 1 , • * •? Here T is the shift operator on £• Of course, a positive answer to problem 1 implies that prob lem 2 can be answered in the affirmative. Let D(S>n be the Borel field of sets common to all the Borel fields (Bn, with n — 0, ± 1 , • • •. Call a stationary process {xt} purely nondeterministic if the only functions measurable with respect to C\(Rn are the constant functions. This means that there is no information in the infinite past of the process. A little reflection indicates that if problem 1 or problem 2 are soluble for {xn}, it must be a purely nondeterministic process (see [8]). As remarked before, the answer to part (a) of problem 1 is not difficult. Let (35) F(an\an-i} a«-2, * • •) — P{xn ^ an\xn-i = a„_i, £*-2 = a«-2, • • •} be the conditional probability distribution function of {xn}. Assume that (35) is a Borel measure function of the real variables anj a«_i, • • •. Given two distri bution functions F and G, we shall say they are equivalent if there is a one-one function mapping the jumps of F and G onto each other and preserving the size of the jumps. In [8] it is shown that there is a random variable £n satisfying part (a) of problem 1 if and only if the distribution functions F(an\an-i} • - •) (as functions of a«) are equivalent for almost all a n -ij a«-2, * * * (with respect to the probability measure of the process {arn}). Let us assume that part (a) of problem 1 is satisfied and inquire under what conditions part (b) is fulfilled. The remarks of section 3 indicate that there is no loss in generality in assuming the processes {xn} and {£„} bounded. Naturally take £(#„, xn-i} • • •) as a Borel measurable function of its arguments. Since (Bo =
INDEPENDENCE AND DEPENDENCE

439

by £» = £(fftt, #n-i, • * •)• To say that xn is measurable with respect to • • • X Ctn-i X <%n is equivalent to stating that xn = gigm £n-i, • • 0 where ^ is a Borel measurable function. But this will hold if and only if (37)

Xn ~ E(xn\£n,

| n - l , • • ', fn-fe) - > 0

in probability asfc—»<*> making use of the boundedness of the {xn} process. Let ix be the measure on half infinite sequences induced by the probability measure of the process {xn}> For convenience we introduce the following notation. Let Xffi

(38)

=::

{Xtfij Xm~-ij

* * *j

S$$m-l = SsJfZm-i, #w-2, *' ' ' )

We now give an argument proving the following theorem. THEOREM. The condition that (39) Zn~E(xn\Sn, " * , £l) ~> 0 in probability as n —> <*> is satisfied if and only if there is a set of points M of y$ points of ji measure one such that (40) St.{S^{- • • {Stfo} • • • } } - S,„{S^{- ■ ■ {StfQ • ■ ■}} - 0 OT £1, • • •, £„, • • • measure for all y0> j}'0 in M as n—> «>. Assume there is a set itf of /* measure one satisfying (40). Take zQ G M. For all So G .M" and any fixed e > 0 the measure (41)

m{(fc, ■ • -, $n)| |^„{- • • {£*&,} • • •} - £«,{■ • • {S6z0} •■•>! > «} - ^ 0

as n —> co. This implies that (42)

P{\xn-8t.{-'-

{S&o} ••■}!>€}

= [m{fa,

•■■,Zn)\\Sh{'-'

as n —> oo so that

{StZo} ■■•}

- ^ „ { . - . {S&}

•••}! >e}^(so)-*0

(43) xn- Sin{--- {S^o} • • • } - > 0 in probability as n —> oo. On the other hand (44)

40(s B |*„, • ■ • , & ) - #*„{■ • • {S#o} ■ • -}| = ^(5€.{. • • { «

• • •} - ^ { . • • {Site} ■ ■ -}|*», • • -, €i)|

£ £[#(|^„{- • • {&&} • • •} ~ S?n{- • • {£#«} • • -}| |€n, • • •, $1)] = E\xn - Sin{'• ■ { - W ■••}|-»0. Therefore #« — i?(a;B|f„, • ■ •, £0 —» 0 as 71 —» « in probability. Now assume that xn — E(xn\l-n, ■ • •, £1) —»0 in probability as n -> co but that (40) is not satisfied. Given a sufficiently small e > 0 there are then sets ilf1, ilf2 with p(Mi), n(Mt) > fi(«) > 0 such that for So G JMi, aio G M2 188

440

FOURTH BERKELEY SYMPOSIUM: ROSENBLATT

(45) m{fa, ■ . ., Cn)| \SU' • • {S&o} ■ • •} - Sfc{. • • { ^ 0 } • • -}| > e} > 8(e) for an infinite number of values of n* However (46) Xn-E(Xn\in, •••,€!)—>0 implies that for almost all 5D0 (M measure)

(47)

Sh{- ■ • { W • • •} - ^OVr • • {S&ft • • -}|fc, •••,&)-><)

1

i* ftj • ••> €n measure, a contradiction. What results exist on these two problems are rather scattered. The most coherent set of results are those on Markov chains. Let us therefore consider problem 1 for Markov chains. Let the transition probability matrix of the chain be P = (p#). It has already been remarked that part (a) of problem 1 has an affirmative answer if and only if the conditional distribution functions are equiv alent for almost every past. This means that the row probability distributions in the transition matrix must be the same except for permutation. Let the posi tive probability masses common to the distributions {p^j = 1,2, ■ • •} be qx ^> £2 ^ • • ■ > 0 in order of magnitude, £ qi — 1. We call such Markov chains uniform chains. When the g* are all distinct one can characterize the random variable (•„ simply. If {xn} is a stationary uniform Markov chain with the qi distinct^ £tt is uniquely determined (up to a one-one transformation modulo sets of measure zero) and is of the form (48) £* = $(xn) E«-I) = k for (xni s n -i) such that p * , . ^ ^ &. This is implied by the following argument. Write £n = %(xn) rrn_i, • • •). Now £ must take m distinct values (where m g <*> is the number of distinct q) with probability one. Label the values 1, 2, • • • for convenience* Let the inverse function of £ as a function of xn for fixed Since £n is independent of Xn—1, 3-n—2, (49)

P{£„ ~ i|^_i = *l, £n~2 = 4, • * '} = Pfei = j) = Pu,nOfii.».--0-

Since the ffn-i). The function £ must be constant on the set (50) {&i)|p.v = <7*} and for convenience we set £ = k on this set. The remarks just made indicate that the answer to part (a) of problem 1 is rarely in the affirmative in the case of Markov chains even though the chain is purely nondeterministic- Nonetheless it is of some interest to find out when part (b) of problem 1 is satisfied for stationary uniform Markov chains with the qi distinct. We have already noted that processes for which problem 1 (or 2) has a positive answer must be purely nondeterministic. In the case of Markov chains this means thai the set of all states must be an irreducible closed set of persistent states. An interesting necessary and sufficient condition for a positive answer to problem 1 can be given in algebraic terms. Let Mu be the matrix 189

INDEPENDENCE AND-DEPENDENCE

(61) where

441

Mk = {*„(*)}

,-,-(/=) - j 1 if " " = "" [0 otherwise. There are m such distinct matrices M^ Each such matrix Mk has precisely one element equal to one in each row and all other elements zero. Notice that every product of a finite number of the Mk also has these properties. No two distinct Mk have a one in the same location. The matrices {Mk} correspond to a family of mappings of the set of states of the process into itself induced by the transition probability matrix P, First consider the semigroup generated (by finite opera tions) from the {Mk}. Notice that the matrices of this semigroup are of the same type, that is, have precisely one element equal to one in each row and all the rest zero. Complete this semigroup by adding to it all limit matrices. Call this semigroup the completed semigroup generated by {Mi). We shall say that this semigroup is point collapsing if it has as an element a matrix with all the elements in some column one. A simple application of the theorem of this sec tion indicates that a purely nondeterministic stationary uniform Markov chain with distinct g* has a representation of the type indicated in problem 1 if and only if the completed semigroup generated by the {Mi} is point collapsing. Since the proof is almost immediate we do not go through the details. It is easy to give examples of purely nondeterministic Markov chains for which 1(a) is satisfied and 1(b) is not. The simplest such example is the Markov chain with the two states 1, 2 the stationary instantaneous distribution (52)

(53)

P{xn - 1} = P{xn = 2} = |

and the transition probability matrix (54)

P =

(g

p}

0
=

l-p
Here

<»>

so that (56)

* - (I !> * - (! S> $ft = %(xn, Xn-l)

=

1

if

(sn-i, x») - (1, 1), (2, 2)

2

otherwise.

Notice that xn is not a function of £n, £n-i, * • • since E(xn\£n) £n-i, • * •) = JEx^ It is generally a tedious computation even in the case of a finite state uniform Markov chain to find out whether the completed semigroup generated by the {Mk} is point collapsing. One might hope for a simple criterion that would provide either a necessary or a sufficient condition for a positive answer to problem 1 in terms perhaps of the invariant vector (stationary instantaneous distribution) of the Markov chain, In the case of the simple example given 190

442

FOURTH BERKELEY SYMPOSIUM: ROSENBLATT

above, we note that the stationary distribution is uniform. Perhaps this would provide such a criterion. We give two simple examples to show that a satisfac tory criterion must be somewhat more elaborate than a simple statement about uniformity of the invariant distribution. Consider a Markov chain with transi tion probability matrix ?1

(57)

P =

2a ?3 ff*

?2 <& ffl & 2i Qi

ff4 ?3 Q* ff*

ff* g3

with the (ji positive and distinct and qi + g2 — Qz +£4' = 1/2. The invariant distribution is clearly uniform. Nonetheless, a simple computation indicates that the semigroup generated by the {Mjc} is point collapsing so that the answer to problem 1 is positive. A second example is provided by the Markov chain with transition matrix A?i

(58)

?3 q*

P = {* * Q2 I ?2 «* g3

with the qi positive and distinct. If the g< are appropriately chosen, the in variant distribution will have distinct elements. Nonetheless the answer to problem 1 is negative since the semigroup generated by the {Mk} is not point collapsing. For reasons of clarity it is worthwhile seeing what one can say about the problem in terms of the concepts associated with the coding problem of infor mation theory, even though it is trivial. Let us consider the capacity of the channel £ = £(a?n, ffn-i) assuming that we are dealing with finite state processes (let us say m states). Since the channel is a noiseless channel with finite memory and essentially one-one given knowledge of rcn-i, it follows that the average entropy of an input, the input and output jointly, and the output alone are all the same. But this implies that the channel capacity is the maximum of average entropies for all possible inputs. Thus, the channel capacity is m log m. None theless, as we have noted there are inputs {xn} with average entropy smaller than m log m that cannot be reconstructed from the output. On reflection, the reason is clear. Here we do not allow coding. The channel £ is determined so as to generate independent random variables from the input where the input is to be directly transmitted without coding. Given £», • • •, %n-k the knowledge lack ing to reproduce xn is that of xn-k-i< For some processes we recover this infor mation as k —► co and for others we don't. Problem 1 is that of determining the processes for which this information is recoverable. The second problem has a more natural answer in the case of Markov chains. In fact, in [9] it is shown that the purely nondeterministic stationary Markov chains are precisely the Markov chains for which a representation of the form (19) is possible. In the case of Markov chains this is equivalent to a positive 191

INDEPENDENCE AND DEPENDENCE

443

answer to problem 2. Clearly this is the best one can hope for. It, is an interesting and open question as to whether the same is true of all purely nondeterministic stationary processes with respect to problem 2. Essentially the proof proceeds by first passing the Markov chain {xn} through a channel with noise and with out memory so as to enlarge the probability space. The output of this channel is a Markov process {yn} from which one can immediately reproduce {#»}. A deterministic channel £ is then determined in terms of {yn} ■ This channel manu factures independent random variables and it is in terms of this output that {%n} is constructed. In obtaining the results cited above, the simplicity of the state space of the process (the fact that it is countable) plays an important role. In the case of a Markov process with a conditional probability distribution F(an\an~.i) continu ous in an for almost every past, the state space is continuous and one can im mediately construct a noiseless channel satisfying part (a) of problem 1 by setting (59)

& - £(**, Xn-l) -

F(xn\xn^).

Here £„ is independent of the past Xn~u ffn-2, - * • and yet £n and xn-i determine xn. Nonetheless, even if {xn} is purely nondeterministic, it is generally not true that problem 1 has an affirmative answer for this channel. In fact there may be some other channel i satisfying 1(a) for which problem 1 has an affirmative answer even though the answer is negative for | (see the example on page 680 of [8]). Therefore, one must generally take care in choosing the proper £. A discussion of some scattered results in the case of continuous state space station ary processes can be found in [8]. I would like to thank D. Slepian and S. P. Lloyd for their helpful comments. REFERENCES [1] H. ANZAI, "A remark on S]:>ectral measures of the flow of Brownian motion," Osaka Math. /., Vol. 1 (1949), pp. 95-97. [2] J. R. BLUM and M. ROSENBLATT, "A class of stationary processes and a central limit theorem," Duke Math. J., Vol. 24 (1957), pp. 73-78. [3] J. L. DOOB and R. A. LEIBLEE, "On the spectral analysis of a certain transformation/' Amer. J. Math., Vol. 65 (1943), pp. 263-272. [4] K ITO, "Spectral type of the shift transformation of differential processes with stationary increments," Trans, Amer. Math. Soc.} Vol. 81 (1956), pp. 253-263. [5] S. KAKTJTANI, "Determination of the spectrum of the flow of Brownian motion," Proc. Nat. Acad. Sci. U.S.A., Vol. 36 (1950), pp. 319-323. [6] G. MARTJYAMA, "The harmonic analysis of stationary stochastic processes," Mem. Fac. Sci. Kyushu Univ., Ser. A, Vol. 4 (1949), pp. 45-106. [7] M. ROSENBLATT, "A central limit theorem and a mixing condition," Proc. Nat. Acad. Sci. U.S.A., Vol. 42 (1956), pp. 43-47. [8] , "Stationary processes as shifts of functions of independent random variables," / . Math. Meek., Vol. 8 (1959), pp. 665-682. [9] , "Stationary Markov chains and independent random variables," / . Math. Mech., Vol. 9 (1960), pp. 945-950. [10] N. WIENER, "The homogeneous chaos," Amer. J. Math., Vol. 60 (1939), pp. 897-936.

192

U.S. copyright law (title 17 of U.S. code) governs the reproduction and redistribution of copyrighted material.

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_21, © Springer Science+Business Media, LLC 2011

193

CHAPTER 21

Asymptotic Behavior of Eigenvalues for a Class of Integral Equations witk Translation Kernels* M . Rosenblatt, Division of Applid Mathematics Brown University

1. INTRODUCTION

The results obtained in this chapter may be of some interest from the point of view of analysis. However, they have an immediate interpretation in terms of certain representation theorems for stationary random processes on a finite time interval, and this provided part of the motivation for the investiga tion. Our interest is in finite interval translation kernel integral equation eigenvalue problems, that is, in the integral equation / J r K * - r ) * ( r ) d r = X4(0.

(1)

Here (t) is an eigenfunction and X the associated eigenvalue. The principal result of the investigation runs as follows. Let r(t) be a positive definite even function on [-22 1 , 2T] of the form r(i) = (-l)kc\i\2k^

+ b(t)9

c>0,

(2)

with b(l) a function 21c — 1 times continuously differentiable whose (2fc — l ) s t derivative is absolutely continuous (ft = 1, 2, • * •)• We can then show t h a t the jih eigenvalue \j of the integral equation (1) is asymptotically the same as 2(2fc- l ) ' c ( ^ J

(3)

as j —> w, The interest from the point of view of random processes is cany to explain. The positive definite function r(t) can be considered as the covarianeo function * This research was supported by the Office of Naval Research. Reproduction in whole or in part is permitted for any purpose of the United States Government. 310

194

EIGENVALUES OF TRANSLATION KERNEL INTEGRAL EQUATIONS

317

of a real-valued random process X(t) weakly stationary on [—7, T]; that is, EX{t) as 0 (4) EX(t)X(r) « r(< - r), |i|, |r| ^ T. The Karhunen-Lo&ve theorem [7] then tells us that X(t) has the following representation in mean square: X(l) « %\j4>i(l)Xj,

(5)

y

where the j(t)}& are the eigenfunctions of the integral equation (1), the X/s are the corresponding eigenvalues, and the X / s are orfchonormal random varia bles given by Xj « ^ f X(r) Ur) dr h'J~T EXjXh = 5^jfc.

(G)

The eigenfunctions 0,-(£) are, of course, assumed to bo normalised over the interval [— T} T], Estimates on the rate of decay of the eigenvalues are help ful in obtaining information on the speed of convergence in the representation (5). It is also useful in determining the sample function regularity of the process. The result obtained is reminiscent also of the relationship between the tail behavior of a distribution function and the local behavior at zero of the cor responding characteristic function (see [3]). Assume that r(l) is the restriction to [ — 2T} 2T] of a positive definite symmetric function on (--co, «>). We then know that r(t) has the Fourier-Stieltjes representation KO = /_"„ eitx dF(\),

(7)

where F(\) is a nondescreasing function of bounded variation on (— <», <*>). Of course, dF(k) = dF(-X). Then 1 - r(l)££c\t\" at zero if and only if 1 - F(X) ^ c/X1+* as X -* oo. Note that the asymptotic behavior of the eigenvalues \d [see (3)] when r(i) satisfies (2) (here a = 2k - 1) is essentially the same as the tail behavior of F as X —► oo. Our assumptions are such that the main irregularity of r(t) is at t - 0 and r(t) is by comparison smooth away from zero. Recently related results have been obtained for the special case in which F is absolutely continuous with /(X) = F'(\), a rational function of X[l]. A few remarks made later on in the chapter indicate that we can no longer expect the correspondence between asymptotic behavior of Xy and tail behavior of F if r(t) has irregularities as bad as that at t = 0 away from zero. Our results are in the case a = 2ft - 1 with h a positive integer, Correspond ing results can be obtained for a nonintegral by means of more detailed estima tion procedures and they have recently been written up by the author [8]. The derivations use the variational characterization of eigenvalues and a number of simple comparison lemmas for eigenvalues. 195

318

STRUCTURAL PROBLEMS

2, COMPARISON LEMMAS FOR EIGENVALUES The estimates we obtain for the eigenvalues of the integral operators we deal with depend on a few well-known results that follow almost immediately from the classic variational definition of the eigenvalues. These results are stated and discussed for the sake of completeness. Consider a completely continuous symmetric operator A on a Hilbert space II mapping II into itself. Let /4", /*£■, ft ~ 1, 2, •. • • , be the nonnegative and nonpositive eigenvalues of A on II in nonincreasing and nondecreasing order of magnitude, respectively. We shall understand by 4 4=

inf

sup IM-i.

(A}4>)

(8)

where (•, •) and ||-j| are the inner product and norm of an element in H. Here $ _L tf means (<£, <£') = 0. Further, if there is no element 4> _L <£i, * ' * , 07C_i with || 0|| = 1, the supremum in formula (8) should be identified with zero. This is not quite the usual definition of eigenvalue. It always leads formally to an infinite number of eignevalues, even in the case of a finite dimensional Hilbert space. However, if H is n-dimensional (n < co), then /4~ = 0 for all k > n. The eigenvalues ^ are given by a formula analogous to (8), namely /ifc" =

sup

inf

(A(j>} )

(9)

Note that M£, Mfc\ ' * " > r u n s "through all the eigenvalues as defined in the con ventional way [5] and that all the remaining eigenvalues as defined through (8) and (9) are zero. Lemma 1. Let Al9 A2 be two symmetric completely continuous transformations and set A = A\ + A^ Denote the nth nonnegative eigenvalue of Ah A 2, and A [as given by (8)] by /z£n, jx£tn, cind i4;> respectively, and the nth nonpositive eigenvalue [as given by (9)] by /*£», ^%n} and M~, respectively. Then

and

(10>

4 H , _ I < fit* + f4.q l4 £ vttP+q-i + /*i;«.

(H)

The proof of this simple but extremely useful lemma can be found in [5], It enables us to get upper and lower bounds for /*+. Corresponding upper and lower bounds for ju~ can be written down. We are also interested in a result that will let us compare corresponding eigenvalues for the two integral equations

flr/& *>*to dr = **(*>>

* = ^ 2'

(12)

where r(i} r) is Hermitian and square integrable in t} r with 0 < 1\ < T% A 196

EIGENVALUES OF TRANSLATION KERNEL INTEGRAL EQUATIONS

319

result of this type is stated in a more general context that may be of some inde pendent interest. Lemma 2. Let Abe a symmetric completely continuous operator on a Hilbert space II mapping II into itself, hot II1 he a sulhllilhert space of II and P the projection operator onto II\. p,f, M F ( ^ ^F)> ^ s 1, 2, • ' ' arelhenonnegaiive and nonposiiive eigenvalues of A (PA = PAP) on II (IIi) in nonincreasing and nondecreasing order of magnilzide} respectively, Then """ * *

(, 3 )

k = 1, 2, • - . A discussion of the application of this lemma can bo found in [(!], Its proof is so simple that we give it here. I t is enough to show that fxf ^ Xj|\ Given any e > 0, there are #i, • • • , 4>k-~i G H such that sup

(A0, 0) < 4+

e

(14)

0 X 0 1 , ♦• ' >0fc~l

where the supremum is understood to be zero if there is no <j>} \\\\ = 1, orthogo nal to 0i, • • • , <j>ic~i. If there is a ^ G II\, ||$|| = 1, orthogonal to Pi, • • • , P^/c-i, it is certainly orthogonal to #i, • • • , 4>k~-i. Hence sup

(PA, ) ^

sup

(Act>} 0).

(15)

This implies that \£ ^ jxf and the proof is complete. In the integral equations (12) H is the Hilbert space of square integrable functions 4> on [— T%y JT2], HI is the subspace of such square integrable functions cj> equal to zero when T\ < \t\ < T2, and (A)(t) « j * * T t r(«, r)*(r) dr.

(16)

3. SOME BOUNDS ON THE ASYMPTOTIC BEHAVIOR OF EIGENVALUES Crude bounds on the asymptotic behavior of eigenvalues of finite interval translation kernel integral equations are obtained in this section. The basic idea is that of extending the kernel so that it is a section of a sufficiently smooth periodic function and then using one of the comparison lemmas of the last section. The following elementary lemma is derived first: Lemma 3. Let r(i) be a function of period 2T symmetric about zero that is h times continuously differentiable (fc — 0, 1, 2, - « •) and whose Hh derivative rih)(l) is absolutely continuous. The eigenvalues Xj (in order of decreasing abso lute magnitude) of the integral equation

/ _ V # - r)(r) dr - X*(0 are then o(j~~Ic~l) as j —> oo. 197

(16')

320

STRUCTURAL PROBLEMS

The eigenfunctions of the integral equation (16') are exp(^f)

i = 0, ± 1 , ; • • ;

since r(t) is periodic with period 2T. Fourier coefficients

(17)

The eigenvalues are simply given by the

e, = f^r(t)

exp (*f^j

dl.

(18)

Clearly all the derivatives of r(t) that exist have period 2T, The odd and even derivatives of r(t) are odd and even functions, respectively, about zero. Hence On integrating by parts, we find that r ^ ) ( f ) = r o»)(—y) for 0 ^ m ^ k.

( i)i+

+,,

<>Kir - i> Mf)* - otr*-1).

(is)

The condition of periodicity is discarded in the following lemma: L e m m a 4. Let r(t) be an even function h times continuously differeniiable (7c = 0, 1, 2, • • • .) whose kth derivative is absolutely continuous. The eigenvalues \j of the integral equation (16) are o(j~k~l) as j —» oo, If r(t) is infinitely differentiable^ the eigenvalues \j are o(j~m) as j —> <*> for every m > 0. Consider the function r(J) on [ —2!T, 2T7]. We can find an even function r*(£) 7c times continuously differentiable with 7cth derivative absolutely con tinuous that is periodic with period 4( J7 + e) for some e > 0 such that r*(0 = r(0 on |i| ^ 2T. The eigenvalues My of the integral equation

JZTI)

r (t r)

* -

*(T) dr = M®

(20)

are o(f~h~~l) by Lemma 3 as j—► » . However, by comparison Lemma 2 it then follows that the eigenvalues X^ of the original integral equation (16) are o(j~~h~l) as j —> to. The statement in Lemma 4 about infinitely differentiable r(t) follows immediately. 4. BERNOULLI POLYNOMIALS AND PERIODIC FUNCTIONS Properties of the Bernoulli polynomials and periodic functions are useful in obtaining detailed estimates on the asymptotic behavior of eigenvalues of a certain class of finite translation kernel integral equations. The Bernoulli polynomials on the interval [0; 1] (see [2]) are recursively generated by setting Bo(s) s 1 and SJH-iCc) = where

Bh(x)

/ o a Bh(x) dx - 0,

fc 198

= 1, 2, ■ • • .

EIGENVALUES OF TRANSLATION KERNEL INTEGRAL EQUATIONS

321

The first few polynomials can be easily written down as follows B0{x) - 1, X6

B1{x) = x - \, X*

B2(x) = j

- | +

^

X

* » < * > - 3 I - 7 + 12 £4(2) = 77 - T — - + — 4! 2 -3! 24 (24-30)

We are interested in the Bernoulli polynomials of even order. If the Fourier series of period unity of the polynomial B^x) is written out, we obtain the following expansion: 2mrx P2^) = (-i)fc-i^2L-1r::2fcn-1

w

<2i>

Note that except for the factor ( —l)*"""1 we have a positive definite function. Consider

= 2T y

c o s

(mrt/T)

T(2mr)

i < 2T

(22)

as the kernel of the integral equation \1T n(t - T) 4>{r) dr = \
|i| < T.

As we have already remarked, the kernel rje(t) is positive definite. functions of the integral equation are

The eigen-

(23) * „ ( « ) - 2 - * sin * £ T j = 1, 2, • • • , with corresponding eigenvalues 2T X 8 ^! - X2i = ^ a » -

(24)

Since the only odd power of the Bernoulli polynomials J52fc(^) is xu~~x, the 199

322

STRUCTURAL PROBLEMS

kernel f/c(i); except for the term 2- (2/c - 1)!(2T)2/C-1? is a polynomial of degree 2ft in t in |g| < 2T with only the even powers of i appearing, Theorem 1. Let p(t) be a positive definite even function on [—2Tf 2T] of the form (25) p(t) = (_ X )* c | t |»-i + 6(i ) mWi c > 0, &($) a function 2k ~ 1 foraes continuously differentiate whose (2/c — l)st derivative is absolutely continuous (7c = 1 ; 2, • ■ ■)• ^/ien £A<3 jiA eigenvalue \j of the integral equation

f^p(t-r)Hr)dr

= M(i)

(26)

is asymptotically the same as

2(2fc- l)fcf=r;J

( 27 )

Now p(f) = 2e(2/c - l)!(2T)a*-Vft(i) + d(«),

(28)

where r&(J) is given by (22) and d(t) is an even function 2/c — 1 times continu ously differentiable whose (2/c — l)st derivative is absolutely continuous. By Lemma 4 the translation integral equation of the form (26) with d(t — r) as kernel has eigenvalues JULJ = o(j~2k) as j -* «>. Further, the eigenvalues t?/ of the translation integral equation with kernel 2e(2/c - l)!(2r)2fc"~"1r/c(i - r) are given precisely by 2c(2/c - l)!(2T) 2/c_1 times (24) in view of our earlier discussion*. Comparison Lemma 1 tells us that Xp+a-i ^ Vp + MQ

and

Xp ^ ^p+g-l + M7-

Since My - o(j~u) as j —> oo, we can find a positive integer-valued function q = ff(p) such that g(p) = o(p) and /xfl(p) = o(p^2&) as p - * «>. It then immediately follows that the eigenvalues \3- are asymptotically the same as (27) as j —»- co, With this theorem, we can easily obtain the asymptotic behavior of eigen values for the integral equation (26) when p(t) is of the form p (i)

J

f

e<*(l + X2*)-1 dX. 200

(29)

EIGENVALUES OF TRANSLATION KERNEL INTEGRAL EQUATIONS

323

The integral (29) can be readily evaluated by a contour integration and is given by h-l

PW - -1*2, z^tzi)

i>

j~0

°>

with p(t) = p( — t), where zj = exp {[HU + *)]/&},

j « 0, 1, • • • , * - 1.

Now

"--il^l*

n+1

where jfc-i 2

/ 0 if n + 1 is even and not a multiple of 27c zp-1 = I - i [ s i n ^ ^ r

1

if n + 1 is odd

i==0

( (-l) a & if n+ 1 - 2s7c. Thus p(f), as given by (29), satisfies the assumptions of the theorem with c = [(2k — 1)!] _1 ; hence the eigenvalues X^ are asymptotically

as j —► oo. This suggests the following result for positive definite functions p(t) of the form Pit) = / _ ! *"7G*) Ac

(30)

with /(a?) ^ 0. Theorem 2. Consider the integral equation (26) w#A p(i) of the form (30) wftere f(x) is even with f(x)c*~>

c>0,

(31)

as a; —» oo. T&e eigenvalues \j of the integral equation (26) are asymptotically of the form

XyS&rc^J •

(32)

Given any e > 0 there is a K(e) > 0 large enough so that for \x\ > K(e) 1

+

a.2* < ^ /J(w* ) <- j

+

xn

Let

/*M H f V ' * > ^ '

w

'0

otherwise 201

(33) v

'

324

BTRUOTUEAL PROBLEMS

and

x/(*)- { f T ^ * M > * « 0

(34)

otherwise C — 6

a/(«) =

if |B| > i?(e) 1 + x2k " m ' " w 0 otherwise.

(35)

Call the Fourier transforms of fjc, if, 2f derived by formula (30) ph(t), ip(t)f 2p(t), respectively. Since /& is bandlimited, pk(i) is analytic in f, and the eigenvalues Xy of the integral equation (26) with py, as kernel approach zero faster than any inverse power of j as j —► » by Lemma 4. Lemma 1 implies that the eigenvalues {Kj and 2X,- of integral equation (26) with kernels ip + pk, 2P + Phi respectively, behave like /2T\2k iXy S 2TT(C + e) f - ^ J

^ 2 X ^ 2 ^ ( 0 - e){— J

(36)

as j —■* co. But the eigenvalues X / of the integral equation (26) with kernel p satisfy the inequality (37) 2Xy < X,- ^ iX,. The inequality (37) with the asymptotic estimates (36) immediately imply the theorem. It is curious that the kernel ( - l ) ^ " 1 Pihip) used to prove Theorem 1 is at the limit for which one can expect a result like that cited in the theorem to hold. If we consider this kernel over the interval \x\ ^ 2 instead of \x\ ^ 1 [it is positive definite on the infinite line (—<*>, «>)L we find that the eigen values of the corresponding translation kernel integral equation are X 2 i_l = X2j' =

22ft-l/.

N2*'

^38)

whereas Theorem 1 would lead us to expect that

v

'-(l)

2k

(39)

as j —> oo . The actual eigenvalues are easily read off, since essentially they are the Fourier coefficients of the function. Here the asymptotic behavior of the eigenvalues is the same type as that in Theorem 1 but the coefficient is not that given in the theorem. The reason why the result of Theorem 1 does not hold is given in the introduction. The function r(t) of Theorem 1 has a jump in its (2k — l)st derivative at t = 0 but nowhere else in [ —2T, 2T]. How ever (—1)*~1P2*(0 not only has a jump at t = 0 but also at t = ± 1 in 202

EIGENVALUES OF TRANSLATION KERNEL INTEGRAL EQUATIONS 325 \t\ ^ 2 . In this example we have seen that the asymptotic behavior is the same type as that given in Theorem 1 but the coefficient is off, However, even more erratic examples indicating the limits of Theorem 1 can be con structed by considering positive definite functions r(t) with singularities of the same type as that at the origin distributed more widely in the interval [ - 2 7 , 2 7 ] . Let r(t) = ^

c ^

1

j

with

{

7jq-§ if j = [n*\ for some integer n > 0, a > I, 3 0 otherwise,

where j ^ 0. Here [%] is the largest integer less than or equal to x. The nonzero eigenvalues are essentially the Fourier coefficients if T = |-, hence

If F(x) = V cs then 1 - F(x) ~ l/x^,

hence

r(t)~c\t\Mtt

+ b(t)$

where b(t) is smoother than \t\fifa at I = 0. If we set f}/a « 2k - 1, we shall see that the eigenvalues \j decrease at a rate faster than that suggested by Theorem 1. 5. CRUDE UPPER BOUNDS FOR EIGENVALUES OF ANALYTIC KERNELS Let r(i) be a positive definite function on (—<*>, <*>) that is the restriction to t h e real axis of a function analytic in the entire complex J-plane. Such a function, of course, has a power series representation 00

Kt) = 2 c^h

(4o>

valid for all L We shall obtain crude upper bounds for the eigenvalues of (1) for an analytic kernel in terms of this power-series representation. Suppose we wish to obtain an upper bound for A% , Let

ri(t) - 2

c

r*(t) - r(t) -

^ rx(t).

Call the integral operator generated by r{t) as a translation kernel on [ - T} T] A a n d the corresponding integral operators generated by ri(t), r 2 (J), the oper203

326

STRUCTURAL PROBLEMS

ators A h A2) respectively, The eigenvalues of A in o r d e r of magnitude are Xj. The nonnegative eigenvalues of Ah A 2 are \f%5 a n d X^~. in order of magnitude. Of course A = Ai + A 2 . By Lemma 1

However, A\ has at mostj — 1 nonzero eigenvalues s o t h a t X^- = 0.

Further

$ max [T T \ck\ \t - T\h dr. (41) T fe = y — i l \i\
»*z

H '

'

rm 2A;-1-1

m

/c!(2/c +

1)

The kernel r(t) = sin i/i has been of especial i n t e r e s t [4].

I n this case

00

H- <

I

(2T)2k+1 (2k + l)!(2fc +

1)

In both cases it should be noted that the e i g e n v a l u e s go to zero faster than geometrically, that is, given any c with 0 < c < 1 , for sufficiently large k: \h

< cK

ACICNOWLEDGMENT Theorem 2 of this paper can be regarded a s e x t e n s i o n of an unpublished result of A. Beurling (see the footnote in [9]). I s h o u l d like to thank H, 0. Pollak and D. Slepian for many helpful discussions. REFERENCES 1. Capon, J. Asymptotic eigenfunctions and eigenvalues of a homogeneous integral equation. IRE Trans. Inform. Theory, 8, 2-4 (1962). 2. Franklin, F. A Treatise on Advanced Calculus. N e w Y o r k : Wiley, 1940. 3. Pitman, E. J. G. Some theorems on characteristic functions of probability distributions. Proc. Fourth Berkeley Symp. Math. Statist. Prob., 2, 393-402 (1961). 4. Pollak, H. O., and Slepian, D. Prolate spheroidal wave functions, Fourier analysis and uncertainty I. Bell System Tech. J., 40, 43-64 (1961). 5. Riesz, F,, and Sz.-Nagy, B. Lecons d'analyse fimctionelle. Budapest, 1952. 6. Rosenbloom, P. Linear Partial Differential Equations, Surveys in Applied Mathematics V. New York: Wiley, 1958. 7. Rosenblatt, M. Random Processes. New York: Oxford University Press, 1962. 8. Rosenblatt, M. Some results on the asymptotic b e h a v i o r of eigenvalues for a class of integral equations with translation kernels. To be published in the J\ of Math, and Mcch. (1963). 9. Slepian, D. Fluctuations of random noise power. Bell System Tech. J., 37, 163-184 (1957). 204

Asymptotic Behavior of Eigenvalues of Toeplitz Forms" M. ROSENBLATT 1. Introduction. Let F(X) = {Fik(\); j , k = 1, • • • , s} be an s X s matrixvalued function of bounded variation on [•— w, ir]. By this we mean that every complex-valued element Fik(\) is of bounded variation. Further, let every difference F(\t) — F(k2) be Hermitian. It will be convenient and in no way less general to take F(-~ir) = 0, the null matrix. Let (1)

rk(dF) - ~ f

eikX dF{\)}

k = 0, ± 1, • • ■ ,

be the sequence of Fourier-Stieltjes coefficients of F. The sequence rk(dF) is a sequence of s X s matrices. Let the Hermitian matrix (2)

Rn(dF) = {r^k(dF);j} k = 1, - < • , n)

be the ns X ns block Toeplitz matrix generated by dF. Since F is of bounded variation, its derivative (3)

/(A) =

^

exists almost everywhere and is Hermitian. Let /^(A) ^ ^2(X) ;S • • • S M«00 be the eigenvalues of /(A) in order of magnitude. The functions M* (X), j = 1, • • • , s, are measurable (see GYIRES [3]). Let ju*,n , j = 1, • • • , ns, be the eigenvalues of Rn{dF) and JV»(aO the number of eigenvalues of Rn(dF) less than or equal to x. We show that

(4)

lim ^ ^ = ~ ?IS

1 : m{\ | M,(X) ^ *}

Z7TS ,-„!

at every point of continuity of the right hand side, where m is Lebesgue measure on [■—7r, 7r]. The case of Toeplitz matrices arises when s = 1 and was treated by SZEGO and GRENANDER & SZEGO (see [2]). The result obtained in this paper is a generalization of that given by GYIRES [3] and ROSENBLATT [6] for block Toeplitz matrices. * The results presented in this paper were obtained in the course of research carried out under grant NSF-G19046 of the National Science Foundation. 941 Journal of Mathematics and Mechanics, Vol. 11, No. 6 (1962). R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_22, © Springer Science+Business Media, LLC 2011

205

942

M. KOSENBLATT

A crude upper bound is obtained for the maximal eigenvalue of Toeplitz matrices generated by an absolutely continuous F. This upper bound is used to obtain a result on the asymptotic distribution of covariance estimates of normal stationary processes. 2. Nondecreasing F(\). We first prove the result on the asymptotic distribution of eigenvalues of block Toeplitz forms in the case of nondecreasing F(k). The function F(\) is said to be nondecreasing if F(\i) — F(k2) is a positive definite matrix whenever Xi ^ X2 . Lemma 1. Let F(X) be an s X s nondecreasing matrix-valued function of bounded variation on [—-ir, x]. The limiting distribution of the eigenvalues of the block Toeplitz matrix Rn(dF) as n —» o° is given by (5)

^

E m{\ | Ml(X)

^x}.

Let Gn(x) be the distribution function of the eigenvalues of Rn(dF), that is,

(6)

We need only consider the distribution functions Gn(x) on [0, <»). Now (7)

f x dGn(x) = ± £ /i,-,n = ^• sp (Rn(dF)) = ^ sp (r0(dF))

where sp (M) denotes the trace of the matrix M. Notice that (8)

y[l - Gn(y)] £ f * dGn(x) ^ ~sp (r0) = c,

so that (9)

1 - GM S fy

Consider any weak limit G(y) of a subsequence of the sequence Gn(y). Clearly any such limit must also satisfy (10)

1 - G(y) ^ *

and hence is a distribution function. Let z be a nonnegative number. The basic result on the determinant of the prediction error covariance matrix in the multidimensional prediction problem tells us that -j

ns

/»oo

— Y, log (1 + zni>n) = /

(ii)

ns

3== i

Jo

log (1 + zx) dGn{x)

~* i /.',logdet {I+2/(x)) dx = i S /_'r log (1+^(x)) dx 206

943

TOEPLITZ FORMS

as n —•> a> (see [5] and [6]). On integrating by parts we see that (12)

[ log (1 + zx) dGn(x) = log (1 + zx)(Gn(x) - 1) |"

Jo

+ /„" I T S <x " GM) dx = /." IT^(1 " G -«"» *• Relations (11) and (12) imply that for any weak limit G(y) of a subsequence of Gn(y)

( 13 )

[ Y+Vx (1 " G{x)) dx^^sti f\ log (1 + *M/(X)) dX-

The distribution function (5) obviously satisfies (13). To complete the proof for nondecreasing F we only have to show that 0 is uniquely determined by the transform (13). Consider the uniqueness problem for the transform (14)

f~j^H(z)dx,

z^O,

where H is of bounded variation and satisfies \H(x)\ S k/(l + x) for some constant k. This is equivalent to the uniqueness problem for the Stieltjes transform (15)

f^^xH{x)dx}

z>0.

A proof of uniqueness is easily given. Suppose f~7^H(>x)dxEs0

(16)

for all z > 0. Equation (16) can be rewritten as (17)

[ e~zu [ e~xuH(x) dx du = 0,

But this implies that (18)

f e~xuH(x) dx S3 0. Jo

It follows that H{x) = 0 almost everywhere. The proof of Lemma 1 is complete. It is readily seen that the conclusion of Lemma 1 holds for every F of bounded variation such that F(\) + a(X + TT)I is nondecreasing for some real value a. It similarly holds for every F of bounded variation such that F(k) + a(X + ir)I is nonincreasing for some real value a. 3. General F. In this section it is shown that the conclusion of Lemma 1 holds for general Hermitian F of bounded variation. The extension will depend on the following remarks. Given two n X n Hermitian matrices A, B with 207

944

M. ROSENBLATT

A — B ^ 0 (difference positive definite), the eigenvalues A, and fif of A and £ respectively (assume the eigenvalues are indexed in order of magnitude) satisfy the corresponding inequalities, that is, (19)

A, ^ & ,

j = 1, ••• ,n.

This follows readily from the minimax property of eigenvalues (see HILBERT & COURANT [1]).

Theorem 1. Let F(k) (F( —7r) = 0) be an SXS Hermitian matrix-valued junction of bounded variation on [—7r, IT}. The limiting distribution of the eigenvalues of the block Toeplitz matrix Rn(dF) as n —> » is jrwen fey 2^Em{AU(A)

^*1-

We first prove the desired result for such a function F whose singular part is nondecreasing. Let /(A) = dF(k)/dk. The singular part of F is given by F(\) - f_T /(M) cfo. Let (20)

£«== {A|/(A) + a / ^ 0 } ,

and Sa be the complement of Sa . The sets Sa are nondecreasing as a —» » and such that the Lebesgue measure m(Sa) —> 27r as a —» » . This follows from the fact that /(A) is integrable, that is, every element of / is integrable. Given any value A, let (21)

f(X)= Z

s|/u.„(X)|-7.

Note that (22)

/(X) ^ f(X)

(.A i^ J3 means that A — JS ^ 0). Inequality (22) is to be understood as holding almost everywhere where the elements of the matrices are finite and hence well defined. Let (23)

F\X) = F(K) -

[X m djx + t

f GO
where (24)

/(X)

°n

Sa

lf(A)

on

&.

f (A) = i

Because of (22) it follows that (25) Rn(dF) ^ Rn(dFa) a for all a. Further, Rn(dF ) is nonincreasing as a —> » . Let G(w)(a;), G(an)(x) be the distribution functions of the eigenvalues of Rn(dF) and Rn(dFa) respectively. 208

945

TOEPLITZ FORMS

It then follows that G(an)(x) £ G(n\x)

(26)

for all n, a, and x. The integrability of /(A), f+(X), inequality (25), and the fact that m(Sa) —> 0 as a —> <» imply that for every e > 0 there is an a(e) > 0 such that for a > a(e) 0 ^ -sp(Rn(dFa) (27)

m

-Rn(dF))

f x d[Gln\x) - GM(x)] = [ J— CO

[GM(x) - G?\x)] dx

< 6

J — CO

independently of n. By the remarks at the end of section 2, it follows that G(an)(x) has the limiting distribution (28)

Ga(x) = ~

Z m{\ | „;(X) ^ *}

as n —> oo where the /x"(X) are the eigenvalues of / a (X). Let H(x) be the limit of any subsequence of the functions GM(x). It follows that (29)

Ga(x) ^ H(x)

for all x and a. Inequality (29) implies that (30)

0 ^ [

[H(x) - Ga(x)} dx S e

«/—00

for all a > a(e). The family of functions Ga(x) is nondecreasing at each x as a —»• <» and has as limit as a —» °° (31)

G(x) = ^

g m{\ | M,(X) ^ x } .

Thus (32)

#(z) ^ (?(*)

and (33)

0g f

[#(*) - (?(*)] cte ^ <=

«/—oo

for every €. But this implies that H(x) = G(x). Since this is true for the limit H(x) of any subsequence of G(n)(x), there is a limiting distribution which is given by G(x). The proof is complete for F with a nondecreasing singular part. We now complete the proof of Theorem 1 by showing that (5) is the asymptotic distribution of eigenvalues of Rn(dF) for a general Hermitian F of bounded variation. The singular part of F is (34)

H{\) = F(X) 209

f

f(fx) d»

946

M. KOSENBLATT

where /(X) = dF(\)/dh. Let (35)

H\\)

= £

s T

|dff„.,G0|.J.

J —T

U,V-1

Set (36)

/(M) dy. + J/+(X),

FX(X) = f J

F2(X) = t

/(M) dM ~ #+(A)-

*J —TV

—IT

Fx has a nondecreasing singular part H* and F2 a nonincreasing singular part -H+. Further, (37)

RJflFO ^ Rn{dF) ^ Z 4 W

for all n. Since RnidFx), Rn{dF2) have the same asymptotic eigenvalue distribu tion (5) by the result on the asymptotic distribution of eigenvalues for a function with a nondecreasing singular part given in the last paragraph, it follows from (37) that Rn(dF) has the asymptotic eigenvalue distribution (5). 4. Asymptotic distribution of covariance estimates. It will be useful to have an upper bound on the maximal eigenvalue of the ordinary Toeplitz matrix Rnif) generated by a nonnegative function / £ L2 in our derivation of the asymp totic distribution of the covariance estimates of a normal stationary process. There are refined estimates of Xn,n when / is a bounded function satisfying certain regularity conditions (see [2]). However, we wish to allow / to be unbounded. Let G(x) = ^ m { A | / ( A )

Sx}.

Lemma 2. The maximal eigenvalue XWf„ of the Toeplitz matrix Rn(f) generated by f zL is bounded above by (38)

n [

xdG(x)

Jb(n)

where b(n) is the supremum of the values a such that 1 — G(a) > 1/n. If f t L2, this upper bound is of smaller order than n* as n —> <». The eigenvalue Xn,n is the maximal value attained by the quadratic form cRncf with c an n-vector subject to the restraint ccf = 1. But (39)

cRnc' = ^ / '

|cn(ea)|2 /(A) dk,

cn(ea) = ] £

c/a

and the absolute maximum of \cn(ea)\2 subject to the restraint ccf = 1 is n. Let (40)

& = {A|/(X)^z}.

We certainly overestimate XWfW by replacing |cn(e*x)|2 by a function equal to n on the set $6(w) . Now 210

TOEPLITZ FORMS

ir™ \

ffr)d\ = n [

947

xdGix).

If / e L2 then J x2 dG(x) < °° and by the Sehwarz inequality n

xdG(x) ^n\

x2 dG(x)

dG(x)

= o(n1/2).

An upper bound for the maximal eigenvalue of a block Toeplitz matrix can be obtained by similar techniques. We now consider the asymptotic distribution of the estimates 1 N-\k\

(41)

rt = ^ iV

J2 XiXj+k ,

k = 0, 1, • • • , s,

j=i

of the covariances rk , k = 0, 1, • • • , s, of a normal stationary process {X*}. Theorem 2. Let {Xk ; & = 0, ± 1 , • • •} be a normal stationary process with mean E I j = 0 and covariances (42)

rk = cov (X, , Xy+/C) = ~ f

eik*f(\) dX fc = 0, ± 1, • • • ,

M ! / e L2. Then (43)

ViV (rt - E r | ) ,

Jfc = 0, 1, • • • , «,

are asymptotically jointly normally distributed with mean zero and covariances (44)

c,f

k

1 fT = - / cos jX cos &X /2(X) dX;

j ,fc= 0, 1, • • • , s.

The joint characteristic function of the random variables (43) is E (exp | E ih{r% - E r%) VNj) = exp { - ± ihVNE l

(45) •/

k=0

r*]^1'2" }

••• / exp {ixTNx'/V~N

\RN\~1/2 - fraR^V} (fo

= |J - 2iR*2V V t f |~1/2 exp { - E i4 V iV E r*J where a; is a row iV-vector, 22^ = RN(1), and 2V = {tith ; ;, fc = 1, • • • , iV} is the Toeplitz matrix with £it< == f0 9 tifi+h = | 2 m if 0 < k ^ s, and all other elements zero. Now expression (45) can be written as

exp | - i £ [log (1 - 2ipi%N/VN) + 2 *7W V F j j where the JK,-,*- are the eigenvalues of R^TNR1^2. 211

Here i ? | is the positive definite

948

M. ROSENBLATT

square root of RN . However, liUN = OCmaxo^ E I*/1); i

i

where the aJtN are the eigenvalues of R^. By Lemma 2 the eigenvalues of RN are o{Nh) uniformly in j so that /*/,#/ ViV = o(l) uniformly in j . Therefore (45) can be given as (46)

exp { - sp {RNTNf/N

+ o( E

|M,^|V^3/2)}.

But (47)

E k , * | 3 ^ ( E M2,* E M/^)1/2 = fe

{RNTN)2

sp (I^7V) 4 ] 1/2 .

Furthermore (48)

0 S sp ( l ^ ) 2 * ^ sp (M„)2*, fc = 1, 2, • • • ,

where MN is the N X N Toeplitz matrix with elements m

i-i = E

|*W-„|-K|.

Notice that MN is generated by a real-valued function g t L2 since m* = ra__* and E k*l* < °°. The fact that E ™>l < °° implies that sp (M^) = 0(iV). Since g e L2, Lemma 2 indicates that the eigenvalues iqJtN of MN are all uniformly o(Nh). But then (49)

sp (MNy = E

HJ.IT

= o(N) sp (ilf„)2 - o(2V2).

This implies that

(50)

t

k.*|3/tfs/2 = o(l)

j-1

as N —> oo. A simple computation indicates that (51)

1 lim-^sp (RNTN)2 =

5

1 ^Lr 1(1 + */,o)(l + **.o)*m*i*i 77~

where c,,& is given by (44). The proof of asymptotic normality is complete by the correspondence between characteristic functions and distribution functions, for the limit of (45) as N —> °° exps — | E c,-,ktjtk\ is the characteristic function of s + 1 jointly normal random variables with mean zero and covariances citk . Some results related to Theorem 2 are discussed in [4]. Relatively simple 212

TOEPLITZ FORMS

949

examples can be constructed to show that one need no longer have asymptotic normality if / 4 L2 (see [7]). In fact, given any p with 1 ^ p < 2 one can construct an / c U but not in L2 such that one does not have asymptotic normality of the covariance estimates. REFERENCES [1] R. COURANT & D . HILBERT, Methods of Mathematical Physics, Vol. 1, New York, 1953. [2] U. GRENANDER & G. SZEGO, Toeplitz Forms and Their Applications, Berkeley, 1958. [3] B. GYIRES, Uber die Spuren der verallgemeinten Toeplitzschen Matrizes, PubL Math.Debrecen, 8 (1961), pp. 93-116. [4] E . J. H A N NAN, Time Series Analysis, New York, 1960. [5] P . MASANI & N . W I E N E R , The prediction theory of multivariate stochastic processes, Ada. Math., 98 (1957), pp. 111-150. [6] M . ROSENBLATT, Asymptotic distribution of eigenvalues of block Toeplitz matrices, Bull Amer. Math. Soc, 66 (1960), pp. 320-321. [7] M . ROSENBLATT, Independence and dependence, Proceedings of the 4th Berkeley Symposium on Probability and Statistics, 2 (1961), pp. 431-443. Brown University Providence, R. I .

213

ESTIMATION OF THE BISPECTRUM1 BY M. ROSENBLATT AND J. W. VAN NESS

University of California, San Diego, and Stanford University 1. Summary. Recently interest has arisen in statistical applications of the bispectrum of stationary random processes. (The bispectrum can be thought of as the Fourier transform of the third-order moment function of the process.) The principal area of statistical harmonic analysis to receive attention previous to this time has been second-order (i.e. spectral) theory on which there is a vast literature. However, the spectrum is most useful in problems of a "linear nature" (see discussion beginning on p. viii of Blackman and Tukey [2]) and provides insufficient information in nonlinear problems. A desire to study phenomena of a nonlinear character has attracted attention to the higher order theory. Such was the case, for example, in a recent study by Hasselmann, Munk and MacDonald [8] where the bispectrum is used in connec tion with oceanographic problems, among which, as the authors state, a number of interesting phenomena such as surf beats, wave breaking, and the energy transfer between wave components can be explained only by the nonlinearity of the wave motion. The bispectrum therefore provides a first glimpse at the nonlinear effects. It is the purpose of the present paper to discuss estimating the weighted and un weighted bispectral density given a set of observations of the process. The rele vant properties (consistency and asymptotic unbiasedness) of the estimates are derived for certain general classes of processes. 2. Introduction. The indexed set of random variables, {Xt}, is called a con tinuous parameter random process if the index set, T, is the real line and it is called discrete parameter if T is the positive and negative integers. For our purposes we take Xt to have mean zero and finite sixth-order moments and to be real-valued sixth-order weakly stationary all of which implies that for all t, Ext = 0, EXtXt+v = m2(t, t + v) = r(v), (2.1)

EXtXt+VlXt+V2 = mz(t, t + vi, t + v2) = rz(vi, *>2), EXtXf+n

••• Xt+V6 = me(t, t + vi, ••• , t + vh) = n(vi,

••• , vb).

Consider r(v) and rz{vi, *>2) to be in Lx (or k) and in the continuous parameter case to be continuous. It is well known from second-order harmonic theory that Received 24 June 1964; revised 17 February 1965. This research was supported in part by the Office of Naval Research under Contract Nonr 562(29) at Brown University. 1120 1

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_23, © Springer Science+Business Media, LLC 2011

214

ESTIMATION OF THE BISPECTRUM

1121

r{v) has a Fourier-Stieltjes representation in terms of the spectral distribution function, F{\). Take F{\) to be absolutely continuous with a continuous density, /(A); then this representation becomes (2.2)

j-»eivXf(\)d\.

r(v) =

(Throughout this paper, all formulae shall be written for the continuous param eter case; the discrete case can be obtained by easy modifications.) Further (2.3)

(2ir)-l$!„e-iv\tv)dv.

/(A) =

Analogously, in third-order harmonic theory we can define the bispectral density function, gr(Ai, A2), as (2.4)

gr(Ai, A2) =

(2TT)~2J!!0O

J~oo exp ( — iv{kx — ^2A2)r3(^i, v2) &vxdv2

and, assuming g(\i, A2) e Za(# 2 ), (2.5) rz(vi, v2) = J-oo J-oo exp (tViAi + zV2A2)gf(Ai, A2) dAi dA2. (For a discussion of the existence of the higher order spectral representations under some conditions, see Blanc-Lapierre and Fortet [3] and Sinai [20].) This then is the harmonic representation of the third-order moment function under the above assumptions. Since the process is real, the following symmetries occur in the third-order functions (2.6) (2.7)

^3(^1, n) = ^3(^2, vi) = rz( — vi, v2 — vi), ?(A!, A2) = (?(A2, A0 = g(\t, -A x - A2) = ^ - A x , - A 2 ) .

The symmetries (2.6) imply that rz('vi, v2) is completely specified over the en tire plane by its values in any one of the six sectors, (a) through (f), shown in Fig. 1. These sectors include their boundaries so that, for example, sector (a) is (2.8)

0 ^ vi < 00,

FIG.

0 ^ p2 ^

1

vi.

FIG.

215

2

1122

M. ROSENBLATT AND J. W. VAN NESS

Similarly gr(Xi, X2) is, by (2.7), completely specified by its values in any one of the twelve sectors (including boundaries) shown in Fig. 2. A later paper will discuss the slightly more complicated situation which arises in the discrete param eter case. Recall from second-order theory that the process permits the representation (2.9)

Xt=

J^e^dZCX),

where Z{\) is a random process of orthogonal increments with (in the case of a real process) EdZ(\) (2.10)

EdZfa)

dZ(\2)

= 0, = 8(Xi + X2) dF(\i),

d(x) = 1

if x = 0

= 0

i f ^ O .

Equation (2.10) yields an interpretation of the spectrum. When there exists a Fourier-Stieltjes representation of the third-order moment function in terms of a complex-valued bispectral distribution function of bounded variation, a similar result holds: (2.11)

EdZ(\)

dZ(\2) dZ(\z)

= 8(Xi + X2 + X3) dG(\x, X2).

More particularly a real process (implies dZ{\) sentation (2.12)

= dZ(—X)) has the real repre

Xt = jo cos tk (iZi(X) + J"? s i n

tx

dZt(\)

where dZi(X) = 2 Re dZ(X) and dZ2(X) = —2 Im dZ(X). I t is interesting that these functions allow us to obtain the following insight into the role of the real and imaginary parts of the bispectrum. In these relations note that when E dZ/^Xi) dZh(\2) dZjz(\z), j% = 1, 2, is nonzero, it is equal to some multiple of the real part of dG if an odd number of the jVs equal 1 and equal to some mul tiple of4the imaginary part of dG if an even number of t h e j / s equal 1. EdZ 1 (Xi)dZi(X 2 )dZ i (X 3 ) = 2 R e [
+ dG(\2, X,)8(-Xi + X2 + Xs)J E dZ^Xi) dZi(\2)

dZ2(\z)

= 2 Im [dG(\u X2)5(Xi + X2 - X3) - dG(\i, 216

Xi)«(Xi - X2 + X3)

ESTIMATION OF THE BISPECTRUM

1123

- d(?(X 2 ,X 3 )5(-X 1 + X2 + X3)] ^dZi(Xi) dZ2(X2) dZ2{\z) = 2 Re [c«?(Xi, X2)8(Xi + X2 -

X3) +
X2 + X3)

- dG(X2, X 3 )5(-Xx + X2 + X3) - d(?(0, 0)5(X1)S(X2)3(X3)]. I t is also worth noting that the symmetries indicate that the integrals in Equations (2.4) and (2.5) can be taken over one (or more) of the sectors in stead of over the entire plane. 3. Estimation. Given observations, xt, for 0 ^ t ^ N, natural estimates for the bispectral density, g(\i, X2), are those based on a function, #* (Xi, X2), which is analogous to the periodogram of second-order theory (see, for example, Rosenblatt [17] for a discussion of periodogram theory). The usual estimate for Tz{vi, v2) is PN{V\ , V2> = ^l}DN{vltv2)

(3.1)

XtXt^y1Xt+y2

dt

where the interval DN restricts xt, xt+vi, and xt+V2 to the domain in which they are*defined (The set $ is the empty set.): DN{vi, v2) = *

if \vi\ or \v2\ or \vx — v2\ > N

= [—min[0, vi,v2], iST—max[0, vly v2]]

otherwise.

I t is intuitively plausible that if rz(vi, v2) is replaced by pN(i>i, v2) in (2.4), the resulting function is an estimate of g(\i, X2), thus (3.2)

gN(\i,

X2) = (2T)~~2J-N

f-N exp ( -tViXi - iv2\2)pN{vi,

v2) dvx dv2.

There are two customary requirements made of such estimates: (i) the estimate should be asymptotically unbiased, (ii) the variance of the estimate should go to zero as N —► oo. Theorem 1 states that under certain conditions on the process, {Xt}, PN(V\ , ^2) has both properties. However, ^ ( X i , X2), like the periodogram, has only Property (i) and not Property (ii). Prompted by the results of second-order theory, one corrects this difficulty by considering weighted estimates of the form (3.3)

9N*(W)

= j-00 J-oo W(m , ^)gN{m

, M2) dm dm

where W(m , ii2) is a "bispectral averaging function" to be defined later. By Theorems 2 and 3 below, it is seen that under certain conditions on W and the process, gN*{W) is an asymptotically unbiased estimate of (3.4)

g(W)

= J-00 J-00 W(m,

m)g(m > M2) dm dm

and that cr2(gN*(W)) —► 0 as N —► <». Therefore gN*(W) suffices as an estimate of (3.4) but it obviously is not an asymptotically unbiased estimate of g(\i, X2). To get an estimate of the bispectral density itself with both Properties (i) and 217

1124

M. ROSENBLATT AND J. W . VAN NESS

(ii), a sequence of weight functions, {WN(m , /x2)], can be used so that (3.5) gN (Xi, X2) = J ~oo J -oo WN(fxi — Xi, /X2 — X 2 )P^(MI , M2) d/xi d/x2. Appropriate choice of the weight functions will give 0**(Xi, X2) Property (ii). As N increases, the weight functions should narrow the region averaged over (that is tend to behave more and more like a two-variate 8-function with the "spike" at the origin); this tends to give the estimate Property (i). A proper rate of concentration by the weight functions must be chosen so that (under certain conditions on the process and on {WN}) both properties can be obtained simultaneously (see Theorems 4 and 5). 4. Hypotheses characteristic of this paper. Fourth and sixth-order moments arise during the investigation of the covariance properties of the estimates. It is necessary to make some kind of assumption on these moments, A very advan tageous assumption involves the use of cumulant (semi-invariant) functions (see Stratonovich ([21], Chapter 1) and Leonov and Shiryaev [11] for a discus sion of cumulant functions). These functions are defined in a manner completely analogous to the way ordinary cumulants are defined for a single random variable. Let a random vector 77 = (771, • > • , rjk) be given. Its characteristic function is (4.1)

Assume E\rij\n < <*> ; then the mixed moments (4.2)

m(n,...,n>

Erjn

=

. . . ^n

exist for all v\, • • • , vk such that VJ ^ 0 and v\ + • • • + vk ^ n. Consequently
ft(ttl,

• • • , ak) = £ , l + . . < - w (iVl+-"+nM

I . . . 1* O m ^ ' - ' V

1

• • • akn

+ o((\a1\ +... + w n

where the sum is over all non-negative v\, • • • , vk whose sum does not exceed n. Furthermore, log (pn has a Taylor expansion exactly as in (4.3) except with min"",Vk) replaced by the coefficient sin,""n\ The quantities sin'""Vk) are called (n n) the cumulants of the vector 77. Also m can be expressed as a polynomial iyi yk in the s \ 0 ^ yj ^ VJ , j = 1, • • • , k (and vice versa) (see Leonov and Shiryaev ([11], p. 320)). For a process with zero mean, the two expressions needed in the following are (4.4)

m 4 (^i, • • • , VA) = $4(^1, • • • , VA) + {8*(vi, ^2)52(^3, vi))z

and (4.5)

mt(v\,

• • • , vt) = s6(vi ,•••,**) +

where (for 77 = (XVl,

+ {^3(^1, v^, vz)sz(v* , H , Pe)}io

{§2(^1, ^ 2 ) ^ ( ^ 3 , • • • , ^ ) } i 5 +

• • • , X,t))

for example 218

{^2(^1, ^2)52(^3, ^4)^2(^5,

ve)}n

ESTIMATION OF THE BISPECTRUM

1125

„ / \ (1,1,0,0,0,0) M ^ l , V%) = S , / x (1,0,1,0,0,0.)

M ^ i , vz) = s , \ (1,1,0,0,1,1) .

/

SA{PI , P2, n , Pe) = s

,

etc.,

and where the notation {• }j denotes the sum of all j different terms obtained by interchanging the arguments of the terms in brackets (the order of the arguments of the sj being immaterial) (see Tables I, I I and I I I below.) Thus (4.4) is m^Pi,

• • • , PA)

=

SA(VI

,

• • • , j>4)

+

s2(vi,

+

P2)S2(PZ,

PA)

$ 2 ( ^ 1 , ^3)^2(^2, ^4) +

S2(PI,

^4)^2(^2, ^ 3 ) .

Note that in the case of zero mean, si(p) (4.6)

=

0,

S2(PI , v%) = m2(pi,

P2),

Sz(vi9 p2, PZ) = rnz{pi, p2, v*). Due to stationarity we write s2(t, t + v) = fe(i'), (4.7)

: Se(t9 t +

Pi,

•••

,t +

vi)

=

&(PI , • • • , J*)-

Then the basic assumption on the process used in the following is that &(PI , P2, ^3) £ Li(Rz) and &(vi, • • • , P*>) £ LiC-Bs). This is a large class of processes which includes, for example, normal processes (trivially) and linear processes. More significantly, it includes all fc-step dependent processes (see Hoeffding and^Robbins [9]) (linear or nonlinear) which have sixth-order moments. The as sumption that £4 £ Li(Rz) was used by Magness [12] and Parzen [13] in works on spectral theory. 5. Third-order moment estimation. It was stated in Section 3 that, under certain restrictions, PN(PI , P2) is an asymptotically unbiased estimate of VZ(PI , P2) and that a2(pN(pi, P2)) —> 0 as N —» 00. The specific result is THEOREM 1. If (a) {Xt}, EXt = 0, is a real, 6th-order weakly stationary process, (b) r{v), rz(pi, P2), £4(^1, P2, PZ), &(vi, • • • , PS), flf(Xi, X2) £ Lx, and (c) y(N) (5.1)

(1) (2)

(5.2)

is a real function that is o(N), then l i m ^ o o Y W I ^ P ^ i , vt) - rz(pi, P2)\ = 0, lim^oo N cov

[PN(PI

, ^2),

PN(PZ , PA)]

= j-00dy[n(vi,P2,y,y

+ pz,y + Pi) - n(vi,

= J-00 dyMpi ,P2,y,y

+ pz,y + p*)

+ {md(0}

PI,

P2)md(y, y + pz,y + 219

PA)\IQ~I

p2)rz(pz, PA)]

1126

M. ROSENBLATT A N D J. W. VAN NESS

+ {m2(0, i>i)sA(v2, y, y + v%, y + y4)}16 + {m2(0, vi)m2(v2, y)m2(y + vz, y + */4)}15] {w3(0, n , v2)mz(y, y + v%,y + ^)}io-i = {m3(0, *>i, v2)m3(y, y + i>z,y + *>4)ho - r3(>i, %)r8(i^, VA)> (3) Equation (5.2) canfeewritten in the "frequency domain" provided £4 and £6 have Fourier transforms which are suitably regular (see [18]). PROOF. Assertion (1) is immediate and (2) is almost as immediate but will be proved in order to introduce some notation. NE[pN(vi, (5.3)

=N

v2) — EpN(vi,

jDN(vltv2)

dt JDN(VZ}PA)

v2))(pN(vz dr[mt(t,

, vA) — EpN(vz t +vi,

t +

, vA))}

v2 , r, r + ~

J>3 , r + P 4 )

^ 3 ( ^ 1 , ^2)^3(^3, ^ 4 ) ] .

Let y = r — £, < = t to get (5.4)

jlN %[Cisr(^i, ^ , vi, *>4, y)/i\T][r6 0>i, ^ , y, y + vz, y + y4) — ^ 3 ( ^ 1 , ^2)^3(^3, *>*)]

where CN is defined as follows (see Fig. 3): 1. Construct the set DN(vi, v2) X D ^ s , v*) = ^ , 2. take the intersection of this set with the line r — t = y and call that seg ment CN , 3. then CN(vi ,v2,vz,Vi,y) is equal to the length of the projection of CN onto either axis.

ESTIMATION OF THE BISPECTRUM

1127

Note that CN can be written out analytically in terms of its arguments but the expression is cumbersome and not as enlightening as Fig. 3. Also, 0 ^ CN/N g 1 and CN/N —» 1 as N —> <*> uniformly on any fixed finite set of ( V s arguments. Using (4.5), (5.4) becomes J - * dy(CN/N)Mvi

,vr,y

+ vz,V + VA)

(5.5)

+ {m3(0, vi, v2)m9(y, y + vd, y + *>4)}io-i + {m2(0, vi)sA(v2, y, y + vZjy

+ VA)}U

+ {m2(0, Vi)ni2{v2y y)m2(y + v*, y + v*)}*]. Tables I, II, and I I I display the individual terms (later called minor terms) which make up (5.5). Looking at each of these 40 terms separately, it is obvious TABLE I The ten terms which sum to give the expression {s3(0, v\ , v2)sz{yi y + vz , y + ^)}io . minor term number 1 2 3 4 5 6 7 8 9 10

«3 ( •,

7*3 (^3 , Vi)

r*(vi , v%) n(vi, y)

n(y + vz — v2, y + v* — v2) rs(y + vz — vi , y + V4 — vi) rz(y — v2 ,y + vi — v2) rz(y — V2 ,y + vz — v2) rz(y - vi, y + vi — vx) rz(y — vi, y + vz — vi) rs(v2 — vi , y + vi — vi) rz(v2 — vi , y + vz — vi) rz(v2 — vi , y — vi)

r*(v2, y) nin }y + vz) n(vi

y

y +

•, •)*«(•, •, 0

vi)

nfa , y + vz) n(v2, y + vi) r*(y, y + vz) n(y, y + vi)

n(y + vz, y + v*)

TABLE II The fifteen terms which sum to give the expression {s2(0, vi)si(v2 , y, y + vz , y + minor term number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15,

™>2(', *)S4(', ', r(vx) r(v2) r(vz) r(vi) r(y) r(v2 — r(vi — r(y r(y —

0

£4(^2 ~ y> VZ , Vi) h(vX — y, VZ , Vi)

fcfri , v2 , vi + £i(vi , V2 , vz +

y) y)

&O2 — vi , y + vz — vi , y + vi — vi) fc(—y, ^3 , ^4) &0i ,v2,y)

vi) vz) vi) v%)

ii{v2 , y + vz , y + vi) b(vi , y + vz , y + vi) £i(vi — y,v2 — y, vi) U(vi — y} v2 — y, vz)

r{y + vz) r(y +

J^JIS

vi)

r(y + vz —Vl) r(y + vz — V2) r(y + vi — r(y + vi —

&(v2 ,y,y

+ v^

& 0 i , y} y +

vi)

&(v2 , y, y + vz) &0i , y, y + v3)

*i)

V2)

221

1128

M. ROSENBLATT AND J. W. VAN NESS

TABLE I I I The fifteen terms which sum to give the expression {s2(0, VT)B%{V2 , y)s2(y + v% , y + VA)}U . minor term number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

S2O, O^G, O^C*, •) r(n) r(v2) r(pi) r(v2) r(vi) r{v2) r(y)

r(v2 r(vi r(y r(y r(y r(y r(v2 r(y r(y r(v2 r(p2 r(y r(y r(y r(y

Hy)

r(y) r(y r(y r(y r(y r(y r(y

+ vz) + Vi) + vz) + vz) + Vi) + Vi)

r(vz — Vi) r(j>3 — vi) r(v*)

— y) — y) + v3 - V2) + vz — vi) + V4 - V2) + VA — vi) — vi) + vz — Pi) + vz — V2) — vi) — vi) - vi) — v2) — vx) — v2)

T(VA)

r(vs) r(vz) r(vi — vs) r(y + VA — v2) r(y + vi — vi) r(vA) r(vz) r(y + V* — v2) r(y + VA — vi) r(y + vz — v2) r(y + v3 — ?i)

that the expression in square brackets is absolutely integrable with respect to y on (— 00, oc ). Thus writing j - N dy as /_«> dyxxiy) where XN is the characteristic function of the interval [—N, N], (5.5) can be looked upon as the infinite integral of a sequence of functions, XN(U)(CN/N)[• •], which (by the properties of CN) tend to zero pointwise and are dominated by the fixed Li( — «>, °°) function given by the absolute value of the expression in square brackets. Consequently, Lebesgue's dominated convergence theorem gives the result. Q.E.D. 6. Estimation of the weighted bispectral density. Next consider estimates of the form (3.3), (3.3)

g**(W)

= J-00 J-00 W(m,

M2)0JT(MI , M2) dm

dfx2.

The following conditions will be placed on W. DEFINITION 1. A bispectral averaging function, W{in, M2), of order a > 0 is (i) real-valued, (ii) eLi fl L 2 , and (iii) its Fourier transform (which is bounded and L 2 ), (6.1)

w(vi, v2) =

(2T)~2

J~OO

J-00 exp ( — ivifxi — iv2ix2)W(ii\, n2) din d[i2

is such that \w(v\, v%)\ = 0(|^i| + M ) " " Condition (iii) could be weakened. Note that (iii) implies that there are constants K\ and 6 > 0 such that for all |*>i| and/or \v2\ ^ b,

(6.2) THEOREM

\w(viyv2)\ ^ Xi(W + M)" a . 2. / / (a) {Xt} is Srd-order weakly stationary, real, 222

ESTIMATION OP THE BISPECTRUM

1129

(b) r3(vi, vt), g(\i, X2) £ I/i(JS2), and (c) W(ni, in) is a bispectral averaging function of order s + e, e > 0 , 0 < s < 1, then\\mN^N'\EgN*{W) - g(W)\ = 0. PROOF. By (3.2), (3.3), and (6.1) N' \Eg„*{W) -

(6.3)

g(W)\

= N' \E J ^ Jf, w(n , *j)[p*(?i, v2)

"" ^3(^1, ^2)] dv\ dv2

— J I\VX\OT\V2\>N w{v\, v2)rz(vi, v2) dvi dv2\. The second integral on the left hand side of (6.3) tends to zero in absolute value by the boundedness of w(v\ , v2) and the fact that n(vi, v2) e LL. The first integral is (6.4)

£N9 \j^N j ^ N w(v!, v2)[l -

[cN{Vl, i*)/tf] -

l]r3(*>i, v%) dvx dv2\

where (6.5)

cN(vi, v2) = min [cN(vi, *>2), JV] Cj\r(^i, ^2) = max [|*>i|, \v2\, \vi — *>2|].

Since c* ^ |*>i| + \v2\ and by (6.2), the integrand of (6.4), [cN(vi, v^/N^^wivi, ^2)^3(^1, n)j is bounded in absolute value on (— «>, °°) by K 2 | ^ ( ^ i , v2)\ for some K2. This integrand converges pointwise to zero so that again by Lebesgue's dominated convergence theorem the result is proved. Q.E.D. Property (i) of Section 3, the asymptotic unbiasedness, is therefore shown. Now turn to Property (ii) of Section 3. THEOREM 3. If (a) {Xt} is Qth-order weakly stationary, real, (b) r(v), nfa , v2), £4(^1> "2, vz), &(vi j "• , ^s), g(vi, v2) eLx, and (c) Wj(ii\, M2), j = 1, 2, are bispectral averaging functions of order 1 + ey, ej > 0, then (1)

lim^oo^covt^TFO,^^)] = jlw

• • • J-n J^oo dj/i • • • dv± dywi(vk, ^2)^2(^3, PA)

•Ke(^i, ^2, y, y + ^3, y + vA) + {m8(0, ?i, v2)mz(y, y + vZyy + j>4)ho-i

(6.6)

+ {m2(0, ^)S4(^2 , 2/, 2/ + *>3 , y + *>4)}l5 + {m2(0, vi)m2{v2, y)m2(y + vz, y + PA)}V>],

(2) Equation (6.6) is written in the "time domain"; the result can be written in the ufrequency domain" {see [18]). PROOF. Recalling the proof of Theorem 1, NE[{gN\Wi) =

- EgN\W1)){gN\W2)

J-iV • • • j-N

-

EgN\W*))]

J-N Wi(vi , ^2)^2(^3 , V^[CN{V\ , • * ' , ^4 , * / ) / # ] 223

1130 (6.7)

M. ROSENBLATT AND J. W. VAN NESS

•[&(?!, v2, y, y + vz, y + vA) + {m3(0, n , v2)mz{y, y + ?3, y + ^)}io-i + {m2(0,

J>I)S4(>2

, y, y + vz, y + *>4)}i5

+ {m2(0, vi)m2(v2, y)m 2 (y + *>3, y +

VA)]IB]

dvx • • • d*>4 dy.

It remains to be shown that the integrand with the (CN/N) term replaced by 1 is absolutely integrable on (— 00, 00 ). To do this, consider the right hand side of (6.7) as made up of four "major terms" or 40 "minor terms", where the first major term and the first minor term are identical and the remaining minor terms are enumerated in accorance with Tables I, II, and I I I . Each minor term is treated separately. Minor term 1 is obviously summable since Wi and w2 are bounded and ^eLi(Rh). The remaining minor terms are grouped according to the major terms within which they are contained. LEMMA 1. The hypotheses of Theorem 3 imply that (1) (6.8)

J-00 • • • J-co J-00 |{m3(0, V!, v2)mz{y, y + vz, y +^)}io-i Wi(vi, v2)w2{vz, vi)\ dv\ - - - dvt dy < 00,

(2) (6.9)

J-00 • • • J-00 J-00 |{m2(0,

J>I)SA(P2

, y, y + vz, y + ? 4 )U Wi^i, v2)w2(vz, VA)\ dvi • • • dv4 dy < 00,

(3) (6.10)

J-00 • • • J-00 j-00 \{m2(0, vi)m2(v2, y)m2{y + vz, y + *>4)}i6 ^ 1 ( ^ 1 , ^ ) ^ 2 ( ^ 3 , *>4)| di'i • • • dv4 dy

<

co.

PROOF. The proof follows using the integrability properties of r 2 , rz and £4 and (6.1) and (6.2) (see [18] and [19]). Q.E.D. The previous statements and Lemma 1 show that there is an Lx upper bound for all N on the integrand. Therefore the properties of {CN/N) and Lebesgue's dominated convergence theorem give (6.6), Q.E.D.

7. Estimation of the bispectral density. The most interesting properties are exhibited by the third form of estimate introduced in Section 3, (3.5)

gN (Xi, X2) = J!!« J-00 WN(fii — Xi, ix2 — \2)gN{ix\, ix2) dm dfi2.

Properties (i) and (ii) of Section 3 will be discussed not for the general form (3.5) but for a subclass of estimates described by first defining a bispectral esti mating kernel. DEFINITION 2. A symmetric bispectral estimating kernel, w(vi, *>2), of order a > 0 is (i) real-valued, € L2, (ii) w(vi, v%) ^ Mi < oo, 224

ESTIMATION OF THE BISPECTRUM

1131

(iii) w(vi, v*) = w(v2, vi) = w( — v\, v2 — vi) (same symmetries as those of (iv) for every e > 0 there exists an M2 such that uniformly in v2, (v) and w(a) = sup P lim, l l F l ^o[|l - win,

v%)\/{\vi\ + \v%\)%

where P = all possible paths, is well-defined and finite. Letting {BN} be a sequence of positive constants tending to zero as N —* 00, the estimates then discussed are of the form (7.1)

g*r*(\i, X2) = (2TT)~2 /-a- j - N exp (—zX^i — zX2*>2) -w(BNvi,

BNV2)PN{V\

, ^2) c^i d*>2.

By Condition (i) above, w ^ i , *>2) is a Fourier transform in the L2 sense, (7.2)

W(-PI , V2) = J-00 J-00 exp (— iv{ki — ^2X2)TT(Xi, X2) d\i

d\2,

Using this, (7.1) can be rewritten as (7.3) 0j\r*(Xi, X2) = J-00 f-JBN~~2W[(m

— \i)/BN),(fjL2 — X 2 ) / J B ^ ] ^ ( M I , M2) dm dm

which is of the form (3.5). Parzen [13] uses an analogous estimate for the spectral density. The rate at which BN —> 0 governs the rate of concentration of the weight functions. Condition (iii) of Definition 2 implies that gN*0
g(q)(m^2,h)

= (2x)~ 2 J-00 J-00 exp ( — imn — W 2 X H + \v2\)qn(vi, V2)h{vx, v2) dv\ dv2. Then the bias of (7.3) satisfies the following. THEOREM 4. If (a) {Xt\ is drd-order weakly stationary, real, (b) rz(v!, V*), jf(Xi, X2) e L x (# 2 ), (c) J-co j-00 (\vi\ + \v2\)qrz(vi, V2) dvi dv2 < «>, q > 0, (d) w{v\, V2) is a bispectral estimating kernel of order a > q, and (e) BN chosen such that as N —> 00 (i) BN - > 0,

(ii) BNN - ^ 0 0 BNqN -+00

if if

q ^ l q> 1 225

1132

M. ROSENBLATT A N D J. W. VAN NESS

then (7.5)

lim^oo BN~q \EgN*(ni, /*2) — g(m , M2>| = 0 )

)

= |^ (Mi,M2,^ )|

if

a > q

if

a = q

where in the case a = q} w{vi , v2) is assumed to have the property that for almost all (vi, v2), (7.6)

lim^oo (1 - w(BNVl,

PROOF.

BNv2))/(\BNVl\

+ \BNv2\)q = w{q\vx,

v2).

Break up the bias expression into three terms,

BN~9(EgN*(ni

> M2) —ff(Ml, M2))

= [B^"V(27r) 2 ][J^ J-3T exp ( - W i - ifx2v2)(w(BNply (7.7)

BNv2) — l ) r 3

• (?i, v2) dv\ dv2 — jH.N J-Nexp • w(BNv\,

(—iiHV! -

BNv2)rz(v\,

W2){min [max (|*>ij, \v2\, \vi -

v2\),

N]/N}

v2) dv\ dv2

~~ J Un\or\v2\>Nexp (—ifiivi — iii2v2)n{vx, v2) dv\dv2]. The third term is ^ (NBN)~q

j Jin|or|,2|>isr ( M + h | ) 5 |r 3 (^i, v2)\ dvx dv2 - ^ 0

by (c) and (e). The second term of (7.7) is S {M,/BNqN)

(7.8)

/ * * SZN ( H + H ) \ u i v t , *>2)| dn dv2.

First suppose g ^ 1; then (7.8) is ^[M./iNB^y]

jiN J!LN K\Vi\ + n r Y ^ K h i

+ HY

M*

, , 2 )i ^

<&* -> o

by (c) and (e). Next suppose q > 1; then (7.8) is ^ (M1/NBNq)

J?LN j»N (\Vi\ + W ) r , ( n , v2) din dv2 -> 0

again by (c) and (e). Therefore lim^oo BN~q \EgN*(ixi, M2) — g(m , M2>| (7.9)

= (27r)" 2 lim^ 0 0 |J_^ J - ^ e x p ( ~ W i - W * ) ( N + M ) S ( * i , "2) •[(w(B^i, B^2) - 1 ) / ( | B ^ | + | B ^ 2 | ) a ] ( ^ i I +

|5*H)a~*^i
Since W(J>I, ^2) is a bispectral estimating kernel of order a, there exists a constant 2£3 such that the integrand of (7.9) is bounded by JRL8(|I'I| + \v2\)q \n(vi, v2)\ which is integrable. Q.E.D. Secondly, the asymptotic behavior of the variance can be described under the assumptions listed in the next theorem. In the proof of this theorem the following lemmas are needed. Lemma 2 is motivated by the Riemann-Lebesgue lemma. The proofs are simple and are omitted (see [18]). 226

1133

ESTIMATION OF THE BISPECTRUM

2. The hypotheses of Theorem 5 imply that (1) J-M J-jf dv\ dv^ exp [—ivifii — iv2{ix2/BN)]w{BNvl, v2)r{v^) —> 0 as iV —> 00 unless \i% = 0, (2) J 5 M • • • J ^ J 5 M dz/x • • • dvt dy exp {— ivi(in/BN) — ^ 2 + i*>3M3 + in *(JH/BN)}W(VI, BNv2 + v4)w(BNvz + vu v4)r(y)r(y + vz)r(y — v2) -> 0 unless LEMMA

Mi = M4 = 0. LEMMA 3. / /

(a) {y(M, N)} is a sequence of constants, (b) y(M) = lim y(M, N) exists far all M, (c) 7 = KWM-OO T(-M") exists, and (d) /or any c > 0 £Aere eziste an Mo(e) such that for all N > M > MQ, \y(M,N) -y(N,N)\ < e, ifeen lim^-oo 7(0 as N-+ 00 (ii) BN2N —> <*> as JV —> °o, (e) W(J>I, i/2) is continuous a.e. and for a < <x>, w(BNa, v2) —> w(0, v2) for almost all v2, and (f) /or brevity in writing results, (MI, M2) and (M3, M4) are taken in their first sections of definition as shown in Fig. 2, i.e. 0 ^ MI , M3 < °°, 0 ^ M2 ^ Mi > 0 ^ M4 ^ /x3, then lim*-* NBN (7.10)

cov \gN (MI , M2), gn (M3 , M4)]

= (2TT)- 1 [/(MI)/(M2)/(MI + M2)/(MS)/(M4)/(M3 + M4)]1/2 •{^I5(M 2 )5(M4)[1

+ 25(MI)][1 + 28(/*s)]

+ W2S(MI - M3)5(M2 - M4)[l + S(MI - Mi) + 45(Mi)8(M2)]}, where wx = [/-« w(0, v) dvf, w2 = /-<» /-<» w2(v\, v2) dv\ dv2. PROOF. The proof will be given in a series of lemmas and individual discussions of major and minor terms. Recalling Theorems 1 and 3, write NBN2

COV [fltor* (MI , M2), flhr* (MS , Au)]

= (BN2/(2w)A)J-N •w(BNvi,

• • • J-JV dvi • • • aV4 exp (— ^ w i — ^2^2 + ^3^3 + tyui'O

BNv2)w(BNvz,

NNvA)j-N

dy(CN(v!,

• • • , *>4, # ) / #

• [&(^i, ^2, y, y + vz, y + 1/4) + {m3(0, i/t, v2)mz{y, y+v$,y + {m2(0, ^i)s4(^2 ,y,y

+ vz,y

+ ^)}io-i

+ i/4)}.is

+ {m2(0, vL)m2(v2, y)m2(y + vz, y + v4)}u]. The boundedness of 11? and (CN/N)

and the Li(i2 5 ) property of ^ imply the 227

1134

M. ROSENBLATT AND J. W. VAN NESS

first major term is 0(BN2). Further, by also using the integrability properties of 7*2, rz, £4 and w, the second and third major terms are seen to be 0(BN), By these statements lim^oo NBN

cov [gN (jui, /X2), QN (M3 , m)]

= lim^oo ( 5 / / ( 2 T T ) 4 ) J1K

• • • fl„ j ^ N dv1 • • • d,4
•exp (—ifiivi — i(x2v2 + ipzvz + imt>i)w(BNvi, •w(BNvz, Bm)(CN(vi,

5^2)

• • • , «4, y)/N){m2(0y vi)m2(v2,2/)m2(y + ^ , y + ^Mis.

Look at the 15 minor terms, listed by Table III, separately. The first term yields lim^oo (BMW)!-* (7.12)

• • • / - * J - * <**••• * 4 %

»exp ( — ifxxvi — W 2 + W 3 + ifXAV4)w(BNpiy BNv2)w(BNvz, •(CN(vly

••• , ? 4 , y)/N)r(vi)r(v%

-

BNv4)

y)r(^ 3 - *>4).

Let y = y — ^2, ^4 = *>4 — v*, h = BNv2, £3 = S ^ 3 ; then (7.2) becomes l i m ^ ( 2 T T ) - 4 J ^ dv^N-BAN dP2jlBAN dh / f ^ V / X ) &*{-*%*&)) (7.13)

«exp {—MMI — ih(v*/BN) -w(-h,BNP*)[CN(v1,(P2/BIf),

+ iP*[(jit + in)/BN]

rfy

+ iPun}w(BNvi, P2)

(P*/BN),P*+ (Pz/BN),y

+

(P2/BN))/N]

•r(vi)r($)r(Pi). An inspection of Fig. I l l reveals that (see [18] or [19]) (7.14)

[C*(vi, (v2/BN),

(vz/BN), v, + (vz/BN), y + (v2/BN))/N]

- * 1 as N -> 00.

Denote by [JM , the ./-dimensional hyper cube centered at the origin with sides of length 2M parallel to the j axes. The dimension, j , will be obvious from the context. Also let \Z\M' denote the complement of D M in Rj . We have the following easy lemma (see [18] or [19]). LEMMA 4. The hypotheses of Theorem 5 imply that for any e > 0, there is an M2(e) independent of N such that for all M > M2 and for all Ny JfjjjuM'

\w(BNvly

v2)w( — vz, B^4)r(^)r(j/)r(y 4 )|(iKi ••• dvAdy < e.

Let yi(M,N)

= (27r)" 4 J7/J7 dvi ••• dvAdy exp { — ivnn — iv2(fx2/BN) + iv*[(fiz + IH)/BN]

+ iv4m}w(BNvi, v2)w{ — vz,

BNvi)r(vi)r(y)r(vt),

7i(M) = l i m ^ o o T i W ^ ) . Lemma 4 and (7.14) indicate that the limit (7.13) is the same as lim^eo yi(N,N). Lemma 2 says that yi(M) = 0 unless /i2 = M3 + M4 = 0. Using the boundedness and continuity properties of w, Lebesgue's convergence theorem, and Lemma 3; 228

ESTIMATION OF THE BISPECTRUM

limN^7l(N,N)

=

1135

(2TT)-W(0)/(MI)/(M3)5(M2)5(M3

+ M4).

Minor terms 2 through 7 and 10 and 11 follow by similar arguments. Minor term 8 offers a slight variation. P u t h = v$ — v\, P2 = v2 — VA , Pi = BNvi, Pi = BNVA to get (omitting the "hats") {4W)

limjv-*ooJ -NBN J -NBN UVl CIVA) -N-(vjBN)

« ^ J -N-(vxlBN)

•exp {ivi[(fJLi — ixz)/BN] — iv2fi2 + ivm + (7.15)

•w{v1, 5 ^ 2 +

VA)W(BNVZ

-j-vdylCvavt/Bv) •r(y)r(y

, ,2 +

+ vz)r(y -

+ i/i,

^ 4 [(M4

^3

— in)/BN]}

VA)

(VA/BN)

, ,3 + ( * / £ * ) ,

(VA/BN),

y)/N]

v2).

Again it is seen that (CN/N) —> 1 as 2V —* <*>. Corresponding to Lemma 4 is the following: LEMMA 5. T%e hypotheses of Theorem 5 impZy that for any c > 0, £&ere is an Mz(e) independent of N such that for all M > M*(e) and for all N > M, nninM

dvi -"

dvAdy\w(vi, BNp2 +

VA)W(BNVZ

+ v1, vA) •r(y)r(y

+ vz)r(y - v2)\ < e-

Proof: The lemma follows using Schwarz's inequality (see [18] or [19]). Q.E.D. Define y8(M, N) =

• • • J-M

(2T)-*J?M

J-M

+ ivziiz + mlifiA — •r(y)r(y 7s(M) =

dv, • • • dvA dy exp {&i[(Mi - m)/J5ir] - tiwt

II
, BNV2 +

VA)W(BNVZ

+ *>i, y4)

+ vz)r(y - ?2),

limN^y8(M,N).

Lemma 2, Lebesgue's theorem, and Lemma 3 gives minor term 8 as limj^oo TsC-W, N)

= (2w)~1w2f(fJi2)f(fJ>z)f(iJ>2 + /xs)8(/ii —

MS)5(M2

—

AM).

Certain generalizations of Theorem 5 could readily be obtained. For example, two weight functions, w^ and W2, could be used; different (but interrelated) sequences, {BNa)} and {BNi2)}, could also be used; and as pointed out earlier certain restrictions on w and the process could be relaxed slightly. 8. Further results. A later paper (part of the results to appear in this paper are contained in Rosenblatt and Van Ness [18].) will discuss the estimates in the discrete parameter case, in particular their asymptotic distribution which (for certain processes) turns out to be complex normal with independent real and imaginary parts. In the second and third-order results the assumptions that the moment functions r(v) and rz(vi, *>2) are in Li were crucial to the estimation of their transforms. When we go to higher order theory the assumption that rn(v\, • • • , J>»_I) e Li, n ^ 4 is too restrictive. This difficulty can be overcome 229

1136

M. ROSENBLATT AND J. W. VAN NESS

by estimating the Fourier transforms of the cumulant functions, £n(*>i, • • • , ^n-i), instead of the transforms of the moment functions. (This is actually what we've done in the second and third-order cases since r(v) = £(*>) and n(vi, v2) = ^3(^1, ^2).) As stated earlier, the assumption %n(vi, • • • , vn-i) e Li(Rn-i) is not so restrictive. REFERENCES [1] [2]

BARTLETT, M. BLACKMAN, R.

S. (1955). An Introduction to Stochastic Processes. Cambridge Press. B. and TUKEY, J. W. (1959). The Measurement of Power Spectra from the Point of View of Communications Engineering. Dover, New York. [3] BLANC-LAPIERRE, A. and FORTET, R. (1953). ThSorie des Fonctions AUatoires. Masson et Cie, Paris. [4] DOOB, J. L. (1953). Stochastic Processes. Wiley, New York. [5] GNEDENKO, B. V. and KOLMOGOROV, A. D. (1954). Limit Distributions for Sums of Independent Random Variables. Addison-Wesley, Cambridge. [6] GRENANDER, U. and ROSENBLATT, M. (1957). Statistical Analysis of Stationary Time Series. Wiley, New York. [7] HANNAN, E. J. (1960). Time Series Analysis. Methuen, London. [8] HASSELMAN, K., MUNK, W. and MACDONALD, G. (1963). Bispectrum of ocean waves. Proc. Symp. Time Series Analysis (M. Rosenblatt, ed.). 125-139. Wiley, New York. [9] HOEFFDING, W. and ROBBINS, H. (1948). The central limit theorem for dependent ran dom variables. Duke Math. J. 15 773-780. [10] JENKINS, G. M. (1961). General considerations in estimation of spectra. Technometrics 3 319-329. [11] LEONOV, V. P. and SHIRYAEV, A. N. (1959). On a method of calculation of semi-in variants. (English translation). Theor. Prob. Appl. 4 319-329. [12] MAGNESS, T. A. (1954). Spectral response of a quadratic device to non-Gaussian noise. / . Appl. Phys. 25 1357-1365. [13] PARZEN, E. (1957). On consistent estimates of the spectrum of a stationary time series. Ann. Math. Statist. 28 329-348. [14] PARZEN, E. (1961). Mathematical considerations in estimation of spectra. Technometrics 3 167-190. [15] PARZEN, E. (1961). An approach to time series analysis. Ann. Math. Statist. 32 951-989. [16] ROSENBLATT, M. (1959). Statistical analysis of stochastic processes with stationary residuals. Probability and Statistics—The Harold Crame'r Volume (Ulf Grenander, ed.). 125-139. Wiley, New York. [17] ROSENBLATT, M. (1962). Random Processes. Oxford, New York. [18] ROSENBLATT, M. and VAN N E S S , J. W. (1964). Estimates of the bispectrum of stationary random processes. Report Nonr 562(29) 11, Division of Applied Mathematics, Brown Univ. [19] ROSENBLATT, M. and VAN N E S S , J. W. (1964). Estimation of the bispectrum. Report Nonr 562(29)13, Division of Applied Mathematics, Brown Univ. [20] SINAI, Y A . G. (1963). On the properties of spectra of ergodie dynamical systems. Dokl. Akad. Nauk} CCCP 150 1235-1237. [21] STRATONOVICH, R. L. (1963). Topics in the Theory of Random Noise, 1 (English transla tion). Gordon and Breach, New York. [22] TUKEY, J. W. (1959). An introduction to the measurement of spectra. Probability and Statistics—The Harold Cramtr Volume (Ulf Grenander, ed.). 300-330. Wiley, New York. [23] TUKEY, J. W. (1959). The estimation of power spectra and related quantities. On Numerical Approximation (R. E. Langer, ed.). 389-411. Univ. of Wisconsin Press. 230

JOURNAL

OF

MATHEMATICAL

PHYSICS

VOLUME

9,

NUMBER

7

JULY

1968

Remarks on the Burgers Equation M.

ROSENBLATT

University of California, Sun Diego, California

(Received 21 November 1967) Periodic and aperiodic solutions of the Burgers equation ttt + uut — ^ , p > 0. are studied in this paper. A harmonic analysis or the solutions is carried out and the form of the spectrum is estimated for large time. Corresponding estimates of energy decay are also made. In Burgers1 work on this equation, the case in which /< { 0 with ifixed,and one then lets t -*■ oo, is studied. In our investigation, a fixed value of /t > Ois taken and then one lets' -t- cc, A similar analysis is also carried out for an irrotational solution of a similar 3-dimensional system of equations. For large time and moderate wavenumbers there is, to the first order, a drift of spectra] mass from low wavenumbers to higher wavenumbers. Comments arc also made on the asymptotic distribution of a class of random solutions. INTRODUCTION Periodic and aperiodic solutions of the Burgers equation are studied in this paper. A harmonic analysis of the solutions is carried out and the form of the spectrum is estimated asymptotically for large time. Corresponding estimates of energy decay are also made. A similar analysis is also carried out for an irrotational solution of a similar 3-dimensional system of equations. For large time and moderate wavenumbers there is, to the first order, a drift of spectral mass from low wavenumbers to higher wavenumbers. Comments are also made on the asymptotic distri bution of a class of random solutions of the equations. 1. THE BURGERS EQUATION Burgers introduced the equation ut + u«, = fiu^,

jt* > 0,

X — u(x, 1) = ■

t

exp

0(x')

0

[-5*'-* ]"

—y)i 2r

+ (*«M Jo

(4^r*£%*0) exp [- (~^j dy,

(1.4) it is clear that
u(x, t) = -2ft — log
(1.6)

(1.7)

It is sometimes convenient to consider solutions of Burgers equation on the circle (or equivalently periodic solutions of the equation). One then has a correspond ing result on existence and uniqueness of the solution without condition (1.2). A standard argument indi cates that the solutions of (1.1), of period 2?r with continuous initial data u0(x) and

j>

(1.3)

x) dx = 0,

(1-8)

are given in terms of solutions tp(x, t) of the heat equation with period 2?r, fix, t) •= J y, exp (-ffit)

exp (ijx),

(1.9)

with initial condition

with iX

9(x> l)

lim u(f, y) = u„(x).

(1.2)

£>[-£«***]<*

FT*. F, 1) =

(y) = =*p - — J » * M * ! .

(1.1)

as a model equation for t > 0, —oo < x < 03, in order to get some insight into turbulence.1 Hopf2 and Cole3 showed that the study of this equation eould be reduced to that of the heat equation. Specifically, if M*) = "(Ofx) is continuous and '„(£) dx = J> as \x\ -*■ co, then

is the unique solution of the initial-value problem. Letting

tp(x, 0) = tp^x) = 2 y, exp (ijx)

dy,

1 G. K- Batchdor. The Theory of Homogeneous Turbulence (Cam bridge University Press, 1953). * E, Hopf, Commun. Pure Appl. Math, 3, 201 (1950). ' J. D. Cole, Quart. Appl. Math. 9, 12o" (1911).

(1.10)

given by (1.4) in terms of u„(x). Again, tp is positive so t h a t / , — y_s and y„ > 0. Then u(x, t) = -2/t — log >(X, ox

I).

1129

Downloaded 18 May 2009 to 1J2.2J9.146.117. Redistribution subject to AlP license or copyright: see http://jmp.aip.orgOmpfcopyrighl.jsp

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_24, © Springer Science+Business Media, LLC 2011

231

M. ROSENBLATT

1130

It will sometimes be convenient to look at random solutions of the Burgers equation that are stationary in jr. We look in particular at some stationary solu tions obtained from solutions of the heat equation that are stationary in x. If y is 8 solution of the heat equation on the circle [0, 2TT) (or, equivalents, a periodic solution with period 2ir) it takes the form (1.9). Stationarity up to second-order moments implies that the coefficients ys in representation (1.10) are orthogonal

large t. We shall obtain estimates o(Zt(t) with error term valid for large r and moderate k from the representation (2,2). These estimates will be used to show that, in the range of large / and moderate k, there is a unidirectional transfer of spectral mass from wavenumbers 1,2, • • ■, k to k + 1 to the first order. This is consistent with some of the heuristic arguments employed in the discussion of nonlinear phenomena like turbulence (see Refs. 1-3). Make the strong assumption

£ysy* = &s-t?i ^ 0, 2 > , < co.

£W<*.

Stationarity up to second-order moments for solutions of the heat equation on the full real line implies that one can write '}< dYik),

E dim

It ftp) = >*-, dF(X),

or the stronger assumption (for a later computation)

1. PERIODIC SOLUTIONS Consider a periodic solution of the Burgers equation with period 2IT and spatial mean zero; that is, con dition (1.8) is satisfied. For continuous initial data, it has already been noted that the solution can be given in terms of a positive solution (,>0. The continuity of initial data implies that

2WS<°°.

so that the expansion

(2.1)

yo — v

(2.5) (2.6)

Even if (2.4) or (2.4') are not satisfied, they will be satisfied at some later time t by y, exp (—/ V ) so that the qualitative features of our conclusions are valid for any periodic solution satisfying (1.8). Now

zt(0 = 2 M S — -

I

>v-*,tl

x exp [-(jj + ■ • • + sJ+JpfK'J (2.7) and

*-«0 + l) *

I

J V ■" >".t+1C-Ji

x exp [-f>* + - ■ ■ + O ^ ' K * - 1 (2.2)

If we formally took at the Burgers equation in the Fourier domain,

- -M%(0

'

s'+i)

'1+ " ■ +•(+!-*

I^exp(-./V) «?((/*)

4(0 + IZWVm t

•*

P tyx)

" 2 Z » ( 0 « P («*}■

'-°

is valid, where

In view of (1.6), the solution u(x, t) of the Burgers equation can be written, 2 yiW) exp ( - J V ) uf>,.) = -2/*-i-

1*0

(1.12)

with F a bounded nondecreasing function. The sequence {aj and function F(X) correspond to the second-order spectral decomposition of the process in the periodic and general case.

ex

(2.4')

(1.U)

where Y(l) is a process with orthogonal increments

(2.4)

Our aim is now to give the principal terms for Z t (i), Z'k{t) together with bounds for the error terms. First consider Zt(t) with k ^ 1 [Z_Jf) — Zt(i) since the solutions are real]. The principal term for large (

V W

(2.3)

is obtained for the Fourier coefficients Zt(t) of u(x, r). This equation is certainly satisfied for sufficiently

(2.8)

ex

P C—*£*)

(2.9)

corresponding to the terra in the inner summation of (2.7) withy + 1 = k and J , = J , = ■ ■ • = st = I. All the other terms of the inner summation corresponding

Downloaded 18 May 2009 to 132,239,145,117, Redistribution subject to AIP license or copyright: see http:Wjmp.aip.org(jmpfcopyrigrit,J5p

232

REMARKS ON THE BURGERS EQUATION toy + 1 = it are in total bounded by

1131

1 <,}
W \ I \yM ™v C-sVoT'1

2Iz»-J(')z,(0y, as

ven i n a

(2- W

num ,er

x r T It /y 1 exp(—s*u')l- (2-10) S" ' °^ heuristic discussions of |_i#o.i J J turbulence. This is enough to counterbalance the The total of the terms of the inner summation corre- dissipative term -pk'Z^i) to such an extent that spending to fixed; + 1 with / + 1 >ftis bounded by z*(<) i s o f o r d e r o f magnitude exp {-kpt) [see (2.14]) ratherthanexp(—ftVO.asit would beiftheconvolu, ,rTl , , , j ^lf+1 , T , , \ tion transfer term did not enter. Notice that all three W\£\yJy.\«P(-»rt\ (2.11) t e r m s o f ^ ( 2 3 ) a r e o f M d e r M p {_kfi!) ,, . _ _ ... , , ... ... Cole* has recently obtained estimates related to Under assumption (2,4), the total of the terms of the n ... inner summation corresponding to fixed j + 1 with * , ENERGY ESTIMATES j + 1
-VftChMfr*'

(2.15)

and that |Z;(() + 2F!.-ft{yJp0)e-*"| < ft'fcC exp [-(ft + l)pt], (2.16) with C an absolute constant. Let us now look at Eq. (2.3) with the estimates given by (2.14) and (2.16). Notice that 2 Zk_l,t)Z,(t)ij <. C f t W " + 1 1 ' " , (2.17) ,r*i>e .., f, ■ __ „,„_, , . T, ,. .. . where C is an absolute constant. This implies that 7'fi-i J - W w +

(,\v fiV J . t*7 < \

^**-^ r ^Ayy + *« ^*w

= 0 ( ( t V " ) e x p [—(ft + 1)^1]}, (2.18)

fJf

~/M " " I I ^ "" ., = /«»"J-w ~ ^{"Ux.

(3-2)

The relation \ d f f - — " l dx = - / J U\ dx (3.3) is obtained in the limit astf—<x> and describes the rate of decay of energy. A similar argument can be used to give a corre.. , °. , , b sponding result in the case of a random solution of the Bur crs S equation stationary in x. It will be more t h a n e n o u g h t 0 assume

rha! mmat3

0/ u and te

partfaJ derivatives ut, «,., and H „ ori» up /o fourth order. Multiply the Burgers equation by u and take as ( -► cc where the constant, corresponding to the O expectations to obtain order of magnitude estimate, is independent offt.At E the very least, ifft3exp {—/it) = o(l) as t — txi, the ^t + ^ " ^ = ^ ™ J (3.4) error estimate on the right of Eq. (2.18) is negligible T h ( , s j m . ,itjes compared to the individual terms on the left of the equation since each of those terms is precisely of order Eujf = - — Eul m 0 of magnitude exp (—kfif). It follows that we have a '"* drift of spectral mass from lower wavenumbers ' j . D. Cole {unPubushe4).

Downloaded 18 May 2009 to 132.239.145.117. Redistribution subject to AIP license or copyright: see http:Wjmp.aip.org(jmpteopyright.jsp

233

M. ROSENBLATT

1132

as t —► oo, then

and

\\
— Euux = Eu\ + Euuxx — 0 dx imply that \_d_Eu2= 2dt

(3.5)

-pEu\.

as r -^ oo with jC a constant. If h(X) ^ A:Aa, a > 0, and k 5*£ 0, as A J, 0, we shall have the second term on the right of relation (3.9) of smaller order of magnitude than the first as t -> oo, so that

We shall get estimates of the energy and the rate of decay of energy when t is large for a class of solutions of the Burgers equation in both the deterministic solution of the Burgers equation of the form u — —2^(px{c +
c + cp

(3. 10)

as t -> oo. The asymptotic rate of decay of energy will then be the same as for the solution 2juc~1(px of the heat equation. Notice that our discussion of total energy and its rate of decay for large t depends on the (3.6) low-frequency range of X in the neighborhood of zero. Similar conclusions can be obtained for a random solution u = —2(jb(px(c + fp)~Xt where

X

f|u(x,0f
j\ux\*dx^4^c~*{\cpxx\*dx

dx

and, since \ 0 as t —► oo,

(3.11)

with (p(x, 0) a bounded process stationary in x, of (3.7) absolute value less than c and mean zero. Here Z(-) is the random spectral function corresponding to as t -* oo. The energy content of the solution u of the 9?(x, 0). We shall assume that cp(x, 0) has second-order Burgers equation given above is the same as that of the spectral density f(X) and fourth-order cumulant solution ific1^ of the heat equation as t -»• oo. spectral density g(A , A , A ). These will be well x 2 3 The rate of decay of total energy is governed by J ]uj2 defined and continuous if the second-order moments dxmd and fourth-order cumulants are integrable as functions of the one and three linearly independent differences of , - r f ] ( * + ?)-*. (3.8) spatial arguments respectively (see Ref. 5 for a dis Thus cussion of higher-order spectra). As before, \ oo, so that f M x , OP dx ^ V c - 4 f J(c + ? ) < ^ -
as f -> oo. Since

(3.12) as t —► oo. The rate of dissipation of energy is given by (3.5), with ux given by (3.8). Then

as r -^ oo and

with

E \(c +
\(c + 0 ^ ^ , ^ ; ^ = \(f(pxx(pl dx = - - [q?x dx,

Now, £>* = cA(q>x) + 3o4(9?a.), where c4 is the fourth cumulant of
it follows that Uux\% dx ^ V r as f —► oo. Since

2

f | ^ | 2 ^x + V ^

£ u ^ V r 4 £ | ( c +
E
f W 4
4- / x / 2 ^(—-Aj —A2 —A3) x exp {-MA 2 + X\ + A2 + (Ax + A2 + A3)2]} x g(A1,A2,A3)flfA1c/A2^A3. (3.13)

sup \ fexp ( - ^ A 2 ) \X\ \h(X)\ dX 1

Ya. G. Sinai, Theory Probability Appl. (USSR) 8, 429 (1963).

Downloaded 18 May 2009 to 132.239.145.117. Redistribution subject to AlP license or copyright; see http://jmp.aip.org/jmp/copyright.jsp

234

REMARKS ON THE BURGERS EQUATION Assuming that/and g are bounded and t h a t / i s not zero in the neighborhood of zero,

1133

for integral (4.3) to the first order and therefore 2^iexp(-jUtA,/*)[j'ir,*(^)]V'W"r,M'/,*-!

E
(4.S)

while

Eft = Ofct)-*}, as (-* co. In that case the principal contribution to the decay of energy would be expected from the term Etplx when (is large.

as an estimate oi/Q., ()■ For A . (— (fit)~'> it is clear that only the term corresponding to k = 1 is of interest as t -» co. Consider a fixed X > 0 and assume that h(X) = ha + ikt + o(X), X > 0,

(4.6)

in the neighborhood of X = 0. The following approxi mation is obtained by taking into account those terms In this section we shall again look at a solution of in the summation (4.5) for values of k in the neighbor the Burgers equation of the form u = IfWJj: — Y)-> hood of fcmll,, which maximizes the typical term with max | T | < c and V given by (3.6). The function i h is assumed to be bounded, real, and continuous / a , J ) ~ 2 , U e x P ( - A j , t [ j l o g ( ^ ) ] ) with A(0) = c > 0, Our object is to get an estimate of the Fourier transform f(i, 1) (with respect to x) of 0[log(^)A. (4.7) u(x, t) at a fixed A ^ 0 as / -* co. This is a limited parallel to the computation carried out in Sec. 2 for a 5, AN OVERSIMPLIFIED VERSION OF THE periodic solution. Notice that as / gets large this will NAVIER-STOKES EQUATIONS fall outside of both the energy-containing and energyA three-dimensional example like the Burgers dissipative range for which X ~ (itt)~^ as t -HHJC by arguments like those given in the section on energy equation can be obtained by looking at the NavierStokes system of equations without the continuity estimates. equation and the pressure terms, and by looking for Now solutions that are gradients. We wish to consider Cauchy's problem for the equations dx (4.1) ^ + ( T .V)y = ^Av, fi>0, (5.1) i-a (k + 1) 4. A SOLUTION ON THE REAL LINE

Consider the Fourier transform o f T ' (k > 1)

where v = (u : , vt,«,) is a 3-vector depending on 3 spatial variables and time, and the initial vector v0 = v(x, 0) satisfies curl v„ = 0. This system is briefly mentioned at the end of Cole's paper3 but is studied at some length in the paper of Kuznetsov and Rozhdestvenskii,6 The fact that curl vq = 0 implies that a solution with this initial specification is irrotational for all time. By introducing a scalar function ': satisfying v = -IftV log ip, (5.2)

C2»r^" f«p \-M* -pfy +|M]) Set ^ = 7 +

it

(2H()-^.

Integral (4,2) can then be rewritten as

the initial-value problem for Eq. (5.1)is reduced to that of

(2*)-»-» exp ( - p i J l ' / i ) ^ / ) - * - 1 1 "

(5.3) x i l » ( | + ( W ^ ) & , ■ ■ • dt^.

If we set q>(x, t) = ya — T , with max \
This suggests the estimate

1-0

(5.4) exp(-(ia*jk)lhL)\

(4«/ifl-'*-»"jfc-i

(4.4)

■ N. N. Kuznetsc-v and B- L. Rozhdcslvemkij, 2. Vycisl. Mat- Fit. 1,217(19*1),

Downloaded 18 May 2009 to 132.239.145.117. Redistribution subject to AIP license or copyright: see http:Wjmp.aip.org(jmpfcopyright,J5ji

235

M. R O S E N B L A T T

1134

An argument like that given in See. 4 now suggests approximating the Fourier transform of Y* [k ^ 1) by

expC-^tXWJhQjV^t)-*"-1^-3

(5.5)

ifT is given by T(x, () = fexp (J'X ■ Jv - .uft'WX) dX

( - 1 , 0 , 0 ) . Let us set ■JWS**- (5.12) Then, for fixed k.

z„(t) = 2^ik^(k) exp [ - 2 W /«][1 + <><■''*)]

(5.6)

where \ and A are 3-vectors, and h is bounded, real, and continuous. One can then proceed further and obtain estimates somewhat like (4,7) under further assumptions on h analogous to those made in Sec. 4. Instead of proceeding in this direction, we shall consider periodic solutions of system (5.1) in the manner of Sec. 2. Let v be an irrotational solution of Eq. (5.1) with period 2TT in each spatial variable and

(5.13)

and K(t) = - 2 ^ i M ( k ) ( | 2 l k ( A x exp l-2Ve,\t*t)ll

+ o^~"%

(5-M)

as / -+ oo. Notice that the analog of the system (2.3) is ■HO + IW kJ

• W ) 1 M 0 = - M W'Mfc

(5-15)

This is certainly satisfied by (5.11), for t sufficiently large. We shall just briefly remark on the analog of = r ! V a , ( x , ( ) ^ , = 0. (5.7) Eq. (2.18) for wavevectors k all of whose components are nonnegalive. Givers any two wavevectors k and Then the solution v is given by (5.2) in terms of a solu k', k ^ k' means that k — k' is a vector all of whose components arc nonnegativc. The estimates (5.14) tion and (5.15) with error terms imply that ¥> = y«-V = J , g - 2 > ' k « P ( - , ' I / J 0 e ! ' P ( ' t ' - t ) r V " ( x , () dx, = f V"(x, I) dxt

(5.8) of the heat equation. Notice that k is a wavenumber 3-vector of integers and 0 is the zero 3-vector. Assume, as in Sec. 2, that ZW\k* < Vo, (5-9) v We can then get estimates for the vectors z k (0 and zi('L for large i, where the z t (r) are the Fourier coefficients in the harmonic analysis of v: v(x, r) = £ z t ( r y

Then z„(() = 2/*ik2

§t*4

(5.10)

zi(0+ 2 *'•%*(<)&*<*) + j»|kr**M = ojexp f- (\ + hkAptll

(5.16)

if k > 0, k ^ 0 with k fixed as / -*■ oo. The error estimate is small compared to each of the terms on the left of (5.16) which are of order of magnitude exp {—2?-i l^il !"'}■ O n e ^ a s a drift °? Spectral mass from wavevectors k', 0 <, k' ^ k to k through the truncated convolution term as before. If k <, 0, the convolution term would be summed over k', k <, k' <; 0, with a similar modification for k belonging to some other octant.

1

6. ASYMPTOTIC DISTRIBUTION OF A CLASS OF RANDOM SOLUTIONS.

(5.11)

For large f, it is clear that the first-order contribution to z^C) in (5,11) will come from vectors s ^ 0 with the following properties. The component $t of s will be zero if fcs = 0, while J, = 0,1 (s, = 0, - 1 } if kt > O (kt < 0). Call this set of vectors f(k). Thus, if all the components of k are positive, f(k) consists of the vectors (1,0,0), (0, 1,0), (0,0, I), (1, 1,0), (1,0, I), <0, 1, 1), and (t, !, 1). If k has its first component negative and the others zero, U(k) contains only

Let us now consider random solutions of the Burgers equation that are stationary in x. Our object is to get some information on the asymptotic distri bution of the solution as r - >■ co. The case of periodic solutions satisfying condition (1,8) is rather un interesting, since it is clear from (2.2) that

u(x,t)s*-2rfyjyjfe-

(6.1)

as t ~* aa. The asymptotic distribution of u(x, /) (assuming that yt > 5 > 0 with probability one) is determined by the joint distribution of yt and yt.

Downloaded 18 May 2009 to 1J2.239.146.117. Redistribution subject to AlP license or copyright: see http:Wjmp.aip.orgOmpfcopyrighl.jsp

236

REMARKS ON THE BURGERS EQUATION Consider now a random solution of the Burgers equation given by with c a positive constant and f a strictly stationary solution of the heat equation (1.5} In x with max |Y„(x)| < c, £T 0 (x) = 0, ■

(6.3)

where *F0(x) = ^{x, 0). Notice that (6.3) implies that max [Y(x, t)\ < c for all t > 0. The stationarity of T 0 in x implies that ua(x) = u(x, 0), u(x, (). and ¥ ( x , () are strictly stationary. Let the second-order spectral distribution function of "F, be F(X),

1B5

is strongly mixing). Under appropriate conditions on second and fourth order moments (see Ref. 8), one would expect Yz to be asymptotically normally distri buted (when properly normalized) as f -» oo. The assumption that T„ is strongly mixing implies that F is absolutely continuous with spectral density f(X). But then T ^ has spectral density AtfiA.). One of the conditions required for asymptotic normality of Vm is that the variance of (Airp,t}Wx diverge as t-* oo. But this variance o»t(4*pf)rVj = 4"f*t fAV ! *"'/(;i) dX = O(l). (6,9)

We do not have divergence of (6.9) as I-+ co and so u(x, t) as given by (6.7), is typically not asymptotically normal. The following objection might be brought Since «„(*) - 2 ^ ^ — t^r1, it is clear that at the against our argument. Even though u as given by (1.6) very least we ought to have is stationary in x, the function tp might not be and it is this class of random solutions one ought to look h*dF{k}< oo, (6.5) at. Nonetheless, we shall see that random solutions of (5,!) stationary in x that are obtained from corre in order that u„(x) be well defined at least in mean sponding stationary solutions of the heat equation square. Let us assume that T 0 is strongly (or uni (53) by (5,2)are typically asymptotically normal under formly) mixing in the following sense.' 8 Let A, B be analogous conditions. This could be thought of as a two events with A determined by conditions on the difference between the one- and three-dimensional random variables Y^x) with x <,y and B determined equations. by the random variablesY„(jr) with x ;> y + d, d > 0. Let us consider a random solution of Eq. (5.1) The process T 0 is said to be strongly mixing if given by \P{AB)-P{A)P(E)\x ■ X) dF(K). (6.12) mixing. Since One would have to require that i EV& + xW&)

^je'^dFW.

(6.4)

vi*. o =£W)- «P (- —f-^y) 'J.

it is clear that max !¥(*, t)| ^ 0 as t -*. co and that u(x,t)^2tic-lV„ (6.7) as r-j- oo. Now:

VJx, I) = -JWflr* «p [ -
jWdtf(A)
(6.13)

if vis to be well defined in mean square. The following definition is a possible generalization of the concept of strong mixing for a process Y 0 (x) with parameter x a point in 3-space. Consider any two events A, B determined by the random variables {Y„(x), x e / } and {T 0 (x'), x' S /"'}, respectively. Let C t , C„ be the convex hulls of the set of points { X , X E / ) and {x', x' e / ' } , respectively. Set d equal to the distance between C,, Ct. We shall say that x'„(x) is strongly mixing if condition (6.6) is satisfied for all events A, B, where a(d) is a function of d that tends to zero as d-+ oo. IfY„ is strongly mixing, then F is absolutely

Downloaded 18 May 2009 to 132,239,145,117, Redistribution subject to AIP license or copyright: see ht1p://jmp.aip.org(jmpfcopyright.jsp

237

M. ROSENBLATT

1136

continuous with spectral density/(X). As (-> oo one can show that ffcOtttycOT (6.14) and QbrptyVYlx,!)

--£H

s I* - yl-JW,<J> ), rfy.

4^tt

The spectral density of (4ir/t/)tT I| (x, () is

(6.15)

as t~+ co if f(X) is continuous and bounded away from zero atX — 0, Actually, less stringent conditions on/(X) at zero would suffice for (6.16). Again, it is clear from (6.15) that we are narrow band pass filtering about X = 0. But now because of (6.16), broad conditions on fourth-order moments assure asymptotic normality of (6.10). ACKNOWLEDGMENTS

**K4»¥rt)*IVx, ()] = (4i7^()31"*! exp (-2pJ |X|!)/(X) dX ■* so, (6.16)

I should like to lhank Professor Burgers, Professor Saffman, and Professor Whitham for their helpful comments.

Downloaded 18 May 2009 to 1J2.239.146.117. Redistribution subject to AlP license or copyright: see http:Wjmp.aip.org(jmpfcopyright,jsp

238

U.S. copyright law (title 17 of U.S. code) governs the reproduction and redistribution of copyrighted material.

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_25, © Springer Science+Business Media, LLC 2011

239

DENSITY ESTIMATES AND MARKOV SEQUENCESf M. R O S E N B L A T T 1

INTRODUCTION

Estimates of the density function of a population based on a sample of independent observations have been considered in a num ber of papers [1, 6-7]. Questions of bias, variance and asymptotic distribution of the estimates have been dealt with at greatest length. Our object is to look at such estimates of the density function when the observations are dependent. The results will not be dealt with in the most general context or under very general conditions. To obtain results in a simple and readily understandable form, the observations are assumed to be sampled from a stationary Markov sequence with a fairly strong condition on the Markov transition operator. However, the extent to which some of the conditions can be obviously relaxed will be indicated. I t should be noted that the asymptotic results we obtain in the case of dependent observations are essentially the same as those in the case of independent observations. This initially is surprising because it is certainly not true when estimating the distribution function by means of the sample distribution function. The more complicated nature of asymptotic results for this problem in the case of dependence can be seen in Billingsley [2]. However, the happy fact that the results we obtain in estimating the density have the same character as in the case of independence is due to the local character of the estimates. 2

R E M A R K S O N T H E CASE OF I N D E P E N D E N T SAMPLING

We just briefly make a few remarks on estimation of the density function of a population with a sample of independent observations. More detailed and precise results can be found [1, 6-7]. Consider a population with continuous positive density function f(x). Let the sample of independent observations be Xl9X2i ...9Xn. Let the estimate off(x) be n

fn(x) = w-i&fa)"1 S Hb{n)-Hx-Xt))> 3= 1

f This research was supported by the Office of Kaval Research. [199] 240

(!)

200

M.ROSENBLATT

r e h(x) is a given bounded continuous density function and b(n) fro-8, a,3:icl"WridLl3h parameter such, that b{n) -> 0 and nb(n) -> co as n-+ oo. T h e b i a s of t h e estimate is &»0») = Efn{x)-f[x)

= jh(a,){f(x-b(n)<x)-f(x)}da.

(2)

i t J i s c o n t i n u o u s l y differentiable up to second order and the moments i == / « * A ( a ) da, > = 1,2, exist then

m

bn(x) ^f'(x)mlb(n)+f"(x)mib(n)ij2 T h u s 6^(ic)

+ o(b{n)2).

(3)

2

o(6(») ) if % = 0. The variance

=

= n~1b{n)-'i-f{x)\hz{a)da

^(LW)

as w-^oo.

(4)

I t is c l e a r t h a t the estimates are asymptotically normally distributed. S p e c i f i c a l l y +J{nb(n))[fn(x)-Efn(x)] is asymptotically normally dis t r i b u t e d , -with mean zero and variance f(x) j h2(a) da. I t is also interesting t o l o o l c a t t h e covariance of estimates at different points x, y. T h e covariance G

°v(?n{x)Jn(y))

=

n-ib(n)-*cov{li{b{n)^{x-X)),h{b{n)-i{y-X))

= n-^b^-n -[hibin)-1

\h{b{n-x{x-oi))'h{b{n)-1{y-<x))f{a)doc (x - a)) f(a) da [h{b{n)-1 (y-a))f(a)

= n^bin)-1!

da)

!h{j3)h{b{n)-1(y-x)+/3)f(x-/3b(n))dj3

-Hn)JMfi)f(x-b(n)fi)dfijh^)f(y-b{n)jS)d/^. (5)

T h i s c e r t a i n l y implies that for fixed x, y with ^J(nb(n))[fn(x)-Efn(x)]

and

x^y

J (nb(n))[fn(y) - Efn(y)]

a r e a s y m p t o t i c a l l y independent. 3

T H E

MARKOV

ASSUMPTION

L e t *us n o w assume that Xv X2,..., Xn is a sequence of observations on a s t a t i o n a r y Markov process with stationary continuous density function f(x) > O a n d continuous transition probability density function f{y\x). L e t f?c(2/\%) &ndfk(y,x) denote the conditional probability density of 241

DENSITY ESTIMATES AND MARKOV SEQUENCES

201

■X/s+i given Xx and the joint probability density of Xh+1 and Xt respec tively, k = 1,2,..,. Clearly

ttn

d

fh{y>x)=fk(y\x)f{v)

(6)

A+i(y|«)=J/*(y|«)/(«l«)^ (*«i,2,...).

(?)

Wo UBO the Bamo sort of estimate fn(x) of the density function f(x) as in the ease of independent observations, The bias of the estimate in the case of a Markov sequence is the same as in the case of independent ob servations. Lot us now look at the covariance of the estimate when one ban dependence. Clearly

xfo-X,))]. (8) In §5 we will determine the asymptotic behavior of the covariance and of the distribution of the estimates fn{x) themselves, given certain asymptotic eonditions on the character of the dependence of the Markov sequence. Some of these conditions will be introduced and discussed in §4, 4

REMARKS ON THE TRANSITION OPERATOR

PROBABILITY

In order to determine the asymptotic behaviour of covariance of density estimates at different points as well as the asymptotic distribution of estimates we shall have to impose certain conditions on the transition probability function of the Markov sequence. The conditions discussed here are undoubtedly too strong. Certainly weaker conditions that ensure the desired results can be determined. However, these conditions have a certain interest even outside the problem of density estimation and the discussion will be given in a broader context. Let T be the transition probability operator of a stationary Markov sequence. In our problem (Th)te)=jh(x)f{x\y)dx9

(9)

where h can be thought of as a bounded function. The condition to be imposed on T corresponds roughly to what has been called 'geometric ergodieity' (see Kendall [5]) in some of the probability literature. Let ||74„ 1 < # < oo, be the D} norm of the function h with respect to 242

202

M. ROSENBLATT

the invariant measure fi (d[i — f(x) dx in our case of density estimation) so that -,yp x /r» \h(x)\p/i(dx)\ if 1

W

H

,lni

-f

(10)

essstip \h(x)\ it p = co, where ess sup is understood as referring to the measure JLC. The Lv norm of the operator T77*, n= 1, 2, ,.,,

i** - E H C -J

(11)

for 1 ^ p ^ oo since 5PH = 1. However, since we are interested in the rate at which Tnh -> Eh — jh(x)fo{dx) as n -> co, the modified norm

is introduced where by hx 1 one means t h a t J h{x)fi{dx) = 0. The transition probability operator T is said to satisfy the condition Gp (1 < p ^ oo) if there is some positive integer n such that \T»\P < a < 1.

(13)

The reason for 'geometric ergodicity ? is apparent since if (13) holds, it followsthat < L H m \Tmyamln (14) Lemma, If T is a transition probability operator of a stationary Markov sequence satisfying condition Gp for some p, 1 < p < oo, then T satisfies the condition Oqfor allq,l < q < GO. Let U be such t h a t Uh = Th-Eh. Notice t h a t \\h^Eh\\q^\\hl+\Eh\^2\\hl and that

U»h = U*-\TK - Eh) = T«^{TK - Eh).

This implies that However

\Un\Q ^ %\Tn\q ^ 2,

1 ^ £ ^ oo.

\Tn\q ^ \\U"\\Q.

(15) (16) (17) (18)

Since ||f/ n ||i, l&^IU < 2 the Riesz convexity theorem [4] can be applied to this situation. Suppose t h a t for some p, 1 < p < oo (for example p = 2), there is a positive integer n such that 1 2 ^ = a < 1. 243

(19)

DENSITY ESTIMATES AND MARKOV SEQUENCES

203

I t then follows that [I7*1*^ ^ ak. Consider any other q with 1 < q < oo. For q > p we have ljq = (pjq) (1/p) + (1 —pjq), while if g p

l o g | | ^ ^ l o g M ^ (l-IJlogl^lU

(20)

and if q < p

Since \\Unk\\p ^ 2afc and |I?7m||w, (1*7% < 2 it is clear that for large enough k

\\U^%
(22)

and therefore

\Tnk\Q < / ? < ! .

(23)

It is of some interest to consider the well known Doeblin condition and see how it relates to the conditions Gp. Let P(x> A) be the transition probability function of the stationary Markov sequence defined for each real x and each Borel set A, Then (Th)(x)=jP(x,ay)h(y)

(24)

for bounded h and the n step transition probability function Pn(x,A) is defined recursively via Pn+1(x,A)=jp(xydy)Pn(y,A)

(»=i,2,...).

(25)

The Doeblin condition D is said to be satisfied if there is a finite measure on the Borel sets with positive mass, an integer n > 1, and e > 0 such that

Pn(z,A)
(26)

for all x if {A) ^ e. Doob (see [3]) considers the more stringent condition D 0 where one assumes that D is satisfied, there is only one ergodic set, and there are no cyclically moving sets. Under condition D0, in Lemma 7.2 on p, 224 of Doob's book on stochastic processes, it is shown that there are constants y and p9 with 0 < p < 1, such that if h is bounded (|A| ^ M < oo) then

\(TVi)(x)-Eh\<2ypK

(27)

However, it is easily seen that (27) is just the condition G^. By the Riesz convexity theorem, it follows that if G* is satisfied then 0p holds for 1 < p < oo. Similarly, if Ox is satisfied then Gp holds for 1 ^ p < oo. 244

204

M.ROSENBLATT

Notice t h a t the condition JD0 is satisfied if we have an ergodic aperiodic stationary Markov sequence with continuous stationary probability density/(a) and transition probability density p(y\ x) satisfying p{y\x)*kK{l+y*Y*

(28)

for some constant K and a > 1/2. However, it is easy to give a simple example which doesn't satisfy D or (?co but for which Gpy 1 < p < oo, holds. This example is provided by the aperiodic Gaussian stationary Markov sequences. For convenience, assume t h a t the random variables of the sequence have mean zero and variance one, The invariant probability density is then

and the transition probability density is

^-^£=?)M-w^

(29)

with p a constant such t h a t \p\ < 1. Mehler's formula indicates that (29) can be written _y*l% m P(y\x) =^fK(y)hix)^y (30) where the hu are the Hermite polynomials orthonormal with respect to t h e standard Gaussian density f{x). By using (3) one can see that \T% = \p\^

(31)

so that condition G% is certainly satisfied. By the Lemma of this section it follows that Gp is satisfied for 1 < p < oo. 5

ASYMPTOTIC BEHAVIOR OF THE COVARIANCE AND D I S T R I B U T I O N OF ESTIMATES

The asymptotic behavior of the covariance of a density estimate at two points x, y will first be determined before the asymptotic distribution is considered. Assume that h is a continuous density function with h(u) = od^) - 1 )

as

\u\ -> oo.

The d e n s i t y / i s taken to be bounded and continuous. Consider the term corresponding to j — 0 in the sum (8). 245

DENSITY ESTIMATES AND MARKOV SEQUENCES

Then

205

cov {h{b{n)^ {x - X))9 h^n)"1 {y - X))} = b[n) h(u)h(b{n)"1(y-x) +

u)f(x-b(n)u)du

— b{n)% h(u)f(x — b(n) u) clu h(u)f(y — b(n) u) du „Mn)f(x)\h*{u)du [0{b(nf)

if x = y\ if x =h*/l

wlienf(x) > 0 as n -> oo. The terms with j 4= 0 have the following asymp totic behavior oov{A(6(w)-M«-i0)).M6(w)-My-^))} = fc(^2) h(u) h{v)fj(x — b(n) usy — b(n) v) dudv — b(n)2 h(u)f{x~b(n)u)du

h(u)f(y — b(n)u)du

s6(n)»{/,(a;,y)-/(»)/(y)} (33) as % -> oo if the joint density functions /,,(•, •) are bounded continuous functions. We shall now get a bound on (33) under the assumption that the Markov sequence satisfies condition 02. Let gx(a) = Win)-1 (»-«)) - jh(b(n)^(x-a))f(a)da. Then

(34)

|cov{A(&(%)-^-^)),7*(&(w)-%-X 0 ))}| =

\E(gy(X)(T%)(X))\

^{E\gv(X)\*E\(Tigx)(X)\*}i *Mpfi{E\gv(X)\*E\gx{X)\y ^ b(n)M'pH h*(u)duj(f(x)f(y)) (35) J for sufficiently large n with M, Mf and p constants, 0 < p < 1, i£f(x) and /(y) are positive. The estimates (32), (33) and (35) indicate that limnb{n)coY[fn(x)Jn(y)]

_ j/wjh (u)du 2

if x = y\ if x 4= y\

under the conditions assumed. 246

.

206

M. ROSENBLATT

We now consider the asymptotic distribution of

V(^(^))ll?n(^)-^(^)] U « !,...>«), for any finite ra-tuple of points xj9 These random variables are asymptoti cally jointly normal and independent with means zero and variances given by (36) as n -> oo under the assumptions made. So as to limit the. notation the argument on asymptotic normality will be given for a single point, that is, m = 1. However, exactly the same argument and estimates can be used to obtain the multivariate result by applying them to any given linear combination

V(«6(n))S«^CfJ^)-%(^)) of the random variables. This is the standard reduction of a multivariate limit theorem to a univariate limit theorem. Consider 4(nh(n))[fjx)-Efn{x)l The argument for asymptotic normality will proceed by the classic device of proving such a limit theorem for dependent random variables, that is, by writing the sum as asum of big blocks separated by small blocks. The contribution due to the small blocks is shown to be negligible and the big blocks approximately independent. A standard central limit theorem for independent random variables (the Liapounov theorem) is then used to get asymptotic normality. Now, <j(nb(n)U?n(x)-Efn(x)]

=

£ 7 P R ) [h{b{n)~1 {x ~Xj))~ Eh{h{n)~x {x ~ z>-))]

= S ( 4 + *,) + 2=1

with

ff

lm+(.l-l)r

4,=

and

(37)

Um+r)

£

, 5 , =

H=

S

(38)

£

(39)

j—lc{m+r)+l

Here m = m(n) and r = r(n) are the summation ranges of the big and small blocks Al and Bx respectively. They will be taken so that m(n)>r{n) —> co

as

n -> oo but

m(n) — o(n)

and

r(n) = o(m(n)).

Further k = Jc(n) = [nj(m + r)] is the greatest integer less than or equal to n\{m + r). The term II accounts for the additional few terms at the 247

DENSITY ESTIMATES A N D M A R K O V

SEQUENCES

207

end not included in the big blocks or small blocks. Notice that 7c(n) -> co as ?^->oo since m,r = o(n). Estimates like those used in (32), (33) and (35) imply that E Y>BX + H 2 ^2(kj:E\Bl\^ U=i

+ E\Bk + H\^j

(Twin) m\ \ n nf

<0

with G a constant,

/1X

(*0)

k

Thus 2 Bi + H can be neglected. Consider the characteristic function *«1

of 2 J.j. We use the 6r2 property of the Markov sequence to compare this i=i

characteristic function with the product of the characteristic functions of the individual AXB and find that E Jexp ( « S A \ } - fi ^{exp(ftA,)}

j=Z

^

-Sfexp ft* 21 4,)} - ${exp ( ^ ) } # fexp u / s ^z)l {k~l)Mpr^\

(41)

where i f is a constant and 0 ),r(w) will be chosen so that k(n)pr^ -> 0. However this means that we have asympft

totic normality if we can show that 2 Ax is asymptotically normal when the A/s are treated as independent random variables with the same marginal distributions. This will be shown to be true by using the Liapounov theorem. For this we require an estimate of the fourth moment of the A{&> Given that the joint density functions up t o fourth order are continuous and bounded, estimates like those obtained in (32) and (33) show that h E\At\^ %mmb(n))i, (42) j=l

where the M/s are constants. But then JcE\A\'

(^|2A|2)2

£^+lf

no{n)

a

^

+

n

Jf

8d

—B(n)+J^—&(n)« n n

(43)

If rib(n) ->oo, b(n) -> 0, b u t m(n) = o(n$) the expression (43) tends to zero as n -> co and the Liapounov theorem is applicable. We can still choose r(n) = o{m(n)) so t h a t (41) tends to zero as n -^ co. 248

208

M. ROSENBLATT

Theorem 1, Let Xnin — ..., — 1,0,1,..., be a stationary Markov sequence with bounded continuous density function f(x) and bounded transition probability density p(y\x) continuous in the variables {y>x). Consider the estimate fn{x) of the density f(x) given by fax) = *-i&(»)-i S A(6(n)-i ( a - * , ) ) , where h is a given uniformly continuous density function. If f is continuously differentiable up to second order and the moments mi = J a%(a) da, i =3 1, 2, o/ /& exist then the bias Efn{x)-f{x)

- m x f (a:)&(^) + m 2 r(a;)6(^) 2 + o(&(n))23

w&ere £7&e bandwidth b(n) ~> 0 as n-> GO. 1/ ifte sequence {Xn} satisfies condition G2 and nb(n) -^coasn^oo (b(n) -> 0) am/ m-tuple {m finite) of the normalized deviations

(ity distinct) are asymptotically normal and independent with mean zero and variances * ffa) \W(a)da {i =* l , . . . , m ) . O n e could equally well consider estimation of the joint density func tion of a fixed number of random variables. Under conditions correspond ing t o those assumed in Theorem 1, one again finds t h a t the asymptotic results are the same as those in the case of independent observations. Actually we shall explicitly note one such result under somewhat broader conditions of dependence. Now Xn, n = ... 3 — 1, 0,1, ♦.., is assumed to be a stationary sequence of random variables. Assume that all joint distributions of a finite number of distinct X/s are absolutely continuous with uniformly bounded continuous density functions, Let f{x,xf) be t h e joint density function of Xp X,-+1. A natural estimate of f(x9 xf) is given by fjx, x') = n~lb{n)-* 2 MHn)-1 (x-Xt),

bin)-1 (x' -Xj+1)),

(44)

w h e r e hfaaf) is a bounded continuous density function. Let 3Bn be the Borel field generated by Xk, h^n, and J*"n be the Borel generated by Xjc, h > n. We shall say that {Xm} satisfies the condition S if for every random variable g with Eg2 < oo, Eg = 0, that is S^n+k measurable it follows that

E\E(g\mn)\*
(45)

DENSITY ESTIMATES AND MARKOV SEQUENCES

209

where a(k) = 0(7c~4""e) for some e > 0 as /c-^co. By making estimates quite similar to those in the proof of the Theorem one can prove the following result. Theorem 2. Let {Xm,m = ..., - 1 , 0 , 1 , . . . } be stationary with uniformly bounded continuous joint density functions. Consider the estimate fn(x,x') of the joint density f(x,x') of Xp Xj+1 given by (44). Iff is continuously differentiable up to second order and the moments, m^

= L^h(a, fi) da dfi

(i,j = 0,1,2),

when i + j ^ 2 exist then the bias Efn{x,x')-f{x,x')

= K 0 / fls (a;,a:')+w 0tl / a ,(aj,» , )]6(n)

+!Ko/^

+ 2%,i/^'+ m o,W6W a +o(6W 2 )

(46)

as b(n) -»- 0. If {ZOT} satisfies the condition 8 and rib(n)2 -> oo as n-^-oo

{b(n) -» 0)

then any finite m-tuple of the normalized deviations

VM")') &(*»x') ~ Eh& x'^

<47)

corresponding to distinct points (x, x') are independent with mean zero and variances *

/(s,»')Ua(a,/?)&*dA

(^8)

Regression estimates have heen considered in [8] and [10]. Without going into details, one can show that under conditions similar to those specified in [8], the asymptotic behavior of these in the case of depen dent observations is the same as the asymptotio behavior in the case of independent observations. In [9] estimation of the transition probability density for Markov sequences is discussed under condition DQ. 6

ACKNOWLEDGEMENT

I should like to thank D. Brillinger and M. Woodroofe for their helpful comments on this paper. A. Garsia suggested that the Riesz convexity theorem might be helpful in comparing the norms considered in §4.

250

210

M. R O S E N B L A T T

REFERENCES [1] Bartlett, M. S. (1963). Statistical estimation of density functions. Sank7iya, Ser. A 25, 245-54. [2] Billingsley, P. (1968). Convergence of probability measures. New York: John Wiley. [3] Doob, J. L. (1953). Stochastic processes. New York: John Wiley. [4] Dunford, N. and Schwartz, J. L. (1958). Linear Operators, Part I. Interscienee. [5] Kendall, D. G. (1959). Unitary dilations of Markov transition operators, and corresponding integral representations for transition probability matrices. In Probability and statistics (ed. U. Grenander). New York: John Wiley. [6] Parzen, E. (1962). On estimation of a probability density and mode, Ann. Math. Statist 33, 1065-76. [7] Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a density function, Ann. Math. Statist. 27, 832-5. [8] Rosenblatt, M. (1969). Conditional probability density and regression estimators. In Multimriaie Analysis, vol. n. Academic Press. [9] Roussas, G. (1967). Nonparametric estimation in Markov processes, Technical Report No. 110, Department of Statistics, University of Wisconsin, Madison, Wisconsin. [10] Watson, G. S. (1964). Smooth regression analysis, Sankhyd, Ser. A 26, 359-72.

251

DISCUSSION ON ROSENBLATT'S PAPER MICHAEL W O 0 D R 0 0 F E I believe that Professor Rosenblatt's paper constitutes a significant contribution to the nonparametric literature and therefore feel quite honored to be a discussant of it. The paper n o t only contributes a major result by giving sufficient conditions for the asymptotic normality of density estimates from stationary Markov processes, but should also stimulate more research on the problem of density estimation from such processes. Indeed, it is to be hoped t h a t Professor Rosenblatt's present paper on density estimation from dependent observations will stimulate as much research as did his first paper on density estimation from in dependent observations [4]. Among other things, I will indicate some directions in which I feel this research might profitably proceed. In considering the asymptotic normality of the estimates, Professor Rosenblatt has solved a rather difficult problem and has left some less difficult results unstated. Since I feel these results to be important, I will take the liberty of stating some of them together with an indication of their proofs. Let Xn be a stationary process satisfying condition S with a(h) summable, (This includes the case of a stationary Markov Process which satisfies oo, provided t h a t / is uniformly continuous, *][nb(n)] -* co, and the Fourier Transform of h is integrable. Thereafter, it follows, again as in [3], that the sample mode is a consistent estimate of the true mode, provided that the latter is unique. I t would also be of interest to know the rate of convergence t o zero o£Mn and such related random variables as were studied in [6] a n d to determine conditions for and the exact rate of convergence to zero of t h e integrated mean square error. _ J —00

To the best of my knowledge, the only other work which seriously considers density estimation from Markov processes is the unpublished report [5], in which consistency and asymptotic normality of kerneltype density estimates are obtained under the condition DQ. I t is there fore of interest to consider Professor Rosenblatt's condition # 2 with some [211] 252

212

MICHAEL W O O D R O O F E

care. I t is quite intuitively appealing in t h a t it requires the dependence between segments of the process to decrease as the distance between them increases, I was therefore intrigued by the G% condition and hoped that it might be satisfied by a large enough class of stationary Markov pro cesses to provide a reasonable setting for asymptotic inference from such processes. I still have this hope, but in the course of attempting to verify Cr2 for linear processes of the form X*«2jflTw

(£ = 0,1,2,...),

(2)

where the Yi are independent and identically distributed and |/?| < 1, I discovered an example which has shaken t h a t hope slightly. If the Yi have the distribution function Ftte) = l-Qogy)-*:x>e,

(3)

then (2) does not satisfy (?2. Indeed, by considering LaPlace transforms and applying the Tauberian theorems of [1], one may show that F, the stationary distribution function, has a slowly varying tail. Letting gn(x) = (l-F(n))-i

for

x >n

and

gn{x) = 0 for

x ^ n,

--0(1) -» 1 as

n

it is then easily verified that \T*gn\l=\\TKgn\\l-o(l)>

l-F{nfi-*) I-Fin)

!

■GO,

for each fixed k. Since \gn\% ^ |flrj|| = 1 for all n9 G2 cannot be satisfied. The distribution function in (3) is, of course, sufficiently pathological to be of little practical interest but I still find this example disturbing. Since Xk = fikX0 + Z, where Z is independent of XQs the process defined by (2) has the property that the dependence of Xk on XQ decreases geo metrically as k -*> oo, which property led me to believe that any process of the form (2) should be geometrically ergodic, t h a t it should satisfy G2> I believe the generality of the Gz condition to be a question on which research could profitably be done. If it should turn out to be reasonably general, it might well simplify subsequent work on asymptotic inference from Markov processes. Another area in which I feel research could profitably be done is that of density estimation from general linear processes of the form Xk=^ojYk^

(ft = 0,1,2,...), 253

DISCUSSION ON ROSENBLATT'S PAPER

213

where the Yi are as in (2). What conditions on the coefficients cj and/or the distribution of Yi are sufficient to insure the consistency and asymp totic normality of density estimates from such processes ? These questions could be attacked either by verifying an appropriate version of condition S or from scratch, perhaps by using the techniques of [2], Part II. The former approach would be more desirable since conditions similar to S appear throughout the probability literature (e.g. in [2]), but the latter approach would probably be the easier.

REFERENCES [1] Feller, William (1966). An Introduction to probability theory and its applications, 2. New York: John Wiley. [2] Ibragimov, I. A. (1962). Some limit theorems for stationary processes, Theory Probab. Applic. 7, 349-82. [3] Parzen, E. (1962). On estimation of a probability density and mode, Ann. Math. Statist. 33, 1065-76. [4] Rosenblatt, M. (1956). Remarks on some nonparametric estimates of a density, Ann. Math Statist. 27, 832-5. [5] Roussas, G. (1967). Non-parametric estimation in Markov Processes, Technical Report No. 110, Department of Statistics, University of Wisconsin, Madison, Wisconsin. [6] Woodroofe, M. (1967). On the maximum deviation of the sample density, Ann. Math. Statist. 38, 475-81.

254

The Annals of Mathematical Statistics 1971, Vol. 42, No. 6, 1815-1842

CURVE ESTIMATES1 BY MURRAY ROSENBLATT

University of California, San Diego 1. Introduction. There is a large class of problems in which the estimation of curves arises naturally (see [15], [34]). It is curious that one of the earliest extensive investigations of this type involves the estimation of the spectral density function when sampling from a stationary sequence ([1], [17], [27], [33]). Even though the simple histogram has been used for years, it was only later that the simpler question of estimating a probability density function was dealt with at some length ([26], [25], [9]). Because the final character of the usual results obtained in both problem areas is quite similar, and the arguments are much more transparent in the case of the probability density function, we shall develop the results for the probability density function first. Later some corresponding results for spectra will be given. The similarities and differences in the two areas will be noted. Since the literature is rather extensive by now, any presentation of theory as given can only be a selec tion of topics and cannot claim to be exhaustive or perhaps even representative. There are a number of attractive open problems that one can suggest solutions to on heuristic grounds. A few of these problems will be examined. In most cases it is clear that one will not use the techniques to be proposed in estimating a density function unless there is a good deal of data (many observations), little a priori information about the density function available, but a great need to get additional information about the density function, even if it is fairly crude. 2. Estimating the probability density function by independent observations. Con sider a population with absolutely continuous distribution function F(x) and pro bability density function/ (x) = F'{x). A simple sequence of estimates is determined by the choice of an integrable bounded weight function w(u) with (1)

jw(u)du = l

and a sequence of bandwidths b(n) I 0 as n -> oo. Notice that this implies that j w2(u) du < oo. An estimate fn(x) off(x)

(2)

f x)

1

»

fx-X\

* =^)&\'m)

is determined by a sample Xu • • •, X„ of independent observations from the popula tion. If w is chosen to be nonnegative, the estimate fn(x) itself will be a probability Received February 11,1971. Invited Wald Lectures given at the annual meeting of the Institute of Mathematical Statistics held at Laramie, Wyoming, August 25-27, 1970; research supported by Office of Naval Research Contract N00014-69-A-0200-6016, Task NR-042-217. Reproduction in whole or in part is permitted for any purpose of the United States Government. 1

1815 R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_26, © Springer Science+Business Media, LLC 2011

255

1816

MURRAY ROSENBLATT

density function. Usually the weight function w will satisfy some additional regu larity conditions. First consider the mean and variance of the estimate/^*). The mean 1 f (x — u Ef„(x)=^jw^)f(u)du

(3)

— J w(v)f(x ~ b(n)v) dv while the variance
(4)

J w2(v)f(x - b(n)v) dvl~\ - (J w(v)f(x - b(n)v) dv)2

^

The mean square error can then be simply written as (5)

E\f„(x)-f(x)\2

=c2lfn(x)-]

+

\Ef„(x)-f(x)\2

1 — J w2(v)f(x - b(n)v) dv - (J w(v)f(x - b(n)v) dv)2 \$w(v){f(x-b(n)v)-f(x)}dv\2.

+

A simple bound shows that ifnb(n) -> oo as n -> oo and f is a bounded function that is continuous at x, then there is consistency in mean square at xasn -> oo. The bound is now given. Let swpxf(x) ^ M. Then (6)

v2Un{x)\

S M\nb(n)Y'

J w2(v) dv.

For each s > 0 let (5(g) = s u p ^ j ^ \f(x + y)—f(x)\. (7)

| £ / n ( x ) - / ( x ) | ^ 2M^b(n)[vl^\w(v)\

Then dv + 3(e).

The result follows by first letting n -> oo and then g -> 0. R. H. Farrel [14] showed that one cannot get a uniformly consistent sequence of estimators of the density function at a point (say x = 0) for a plausible class of density functions. He specifically showed that if CM is the class of density functions / o n (—oo, oo) with (a) / (b)

continuously differentiate everywhere supXERf(x)^M,

then even with a sequential estimator SN of/(0), the supremum of the mean square error supfECME\8N-f(0)\2^^ if M ^ 3 and sup f e C M EfN < oo. However, if one requires a Holder condition for /itself in the neighborhood of zero, a uniformly consistent sequence of nonsequen tial estimators of f(0) is easy to obtain. Let CMe be the class of continuously 256

CURVE ESTIMATES

1817

differentiable density functions satisfying conditions (a) and (b) and the additional requirement (c)

\f(y)-f(0)\£M\y\'

for

\y\£l

with M, s > 0 independent off. The types of bounds obtained in (6) and (7) indicate that the sequencefn(0) is uniformly consistent in mean square ifn->co and nb(n)->co, forfeCMf£. A number of people have suggested using the integrated mean square error £ j \fn(x)-f(x)\2dx

(8)

\E\fn{x)-f(x)\2dx

=

as a global measure of how good the estimator f„(x) is over R. Notice that 1

o\fn{y)-\dy^

(9)

w (v)f(x — b(n)v) dv dx

nb(n) 1 nb(n)

w2(v)dv.

Further ^00(Efn(x)-f(x))2dx=^a>{S™„W(u)[f(x-b(n)u)-f(x)-]du}2dx

(10)

^ Aft,, K«)| J-»{/(x-&(*)«)-/(*)}2 dxdu

where (11)

A = J [W(M)|(/M

i f / e L 2 . Let (12)

e~lxX(l>(X)dL

/ ( * ) = 2n

Then (13)

|^{/(x-K^-/(*)}^

Since \exp[ — b(n)uX\ — 11 5j 6(w)|iiil| it follows that (10) is bounded above by Ab(n)2p„

\^u)\u2du^

\4>(l)\2X2dL

We make use of (9) and (10) and note that iffn(x) is a sequence of estimators derived from a weight function w with (14)

off(x)

J |w(tt)|u 2 dtt < oo,

then the integrated mean square error of fn tends to zero uniformly as n, nb(n) -> oo for all f with j \f(x)\2dx, J \f'(x)\2dx ^ k < oo. The condition J \f(x)\2dx ^ k < oo actually implies that j |/(x)| 2 c/x ^ k+1. For convenience assume that f and its first two derivatives are continuous and bounded. Then simple approximations can be given locally for the variance and 257

1818

MURRAY ROSENBLATT

bias offn(x) as n -> oo, nb(n) -» oo, if j* | w(u) \u2du < oo. It is then immediately clear from (3) and (4) that a2

(15)

U^

=

^))

w 2

^

d v

if f(x) > 0 and (assuming j w(u)udu = 0) that (16)

Ef„(x)-f(x)

= \b(n)2f"(x)\w{u)u2

du+o(b(n))2.

Notice that if/(x) = 0 with/"(x) > 0, then

b

-^-f\x)lw2{v)dv

as n -> oo, b(n) - > 0 . With additional conditions o n / ( i n terms of smoothness) and on w one can improve the rate at which the bias tends to zero. Such a discussion has been given in Bartlett [2] and we shall briefly indicate how this might be done. However, it will be clear that in order to bring this about weight functions which assume negative values must be used. In many situations such weight functions would not be plausible. Suppose that f and its first four derivatives are continuous and bounded. Further, let w be such that (17)

\w(u)udu

=\w{u)u2du

=$w(u)u3du

= 0,

$\w(u)\u*du < oo.

Then (18)

Ef„(x) -f(x)

= i b{n)Y*Kx) 1 w(ti)u4 du + o(b(n))\

A weight function satisfying the conditions (17) is given, for example, by 3/ M2X w(<«) = 7 I " ? K 1 " 2 There have been discussions in the past about the "optimal" shape of a window or weight function in spectral analysis (see [24]). The arguments usually given have been of an asymptotic character and it is a mistake to take them too literally from a finite sample point of view. But even asymptotic arguments if used and inter preted with care can yield meaningful ideas. Let us briefly consider a simple dis cussion due to Epanechnikov [13]. Under the assumptions leading to (17) and (18), the local estimate of mean square error obtained is (19) E\Ux)-Kx)Y

=^)Jw2(1.)rfi.+i[(.(-.)]4[/"W]2(i"'(»)«2<'«)2

258

1819

CURVE ESTIMATES

The rate at which the mean square error tends to zero as n -> oo is maximized (if j w{u)u2du 7^ 0) if we set b(n) = Kn-x/5

(20) with K the constant

_ f 4f(x)\w2(v)dv (21)

X =

[_(f"(x)$w(v)v2dv)2\

11/5 •

The mean square error then becomes (22) E\fn(x)~f(x)\2 s 2 3 / 5 [ / ( x ) J w 2 ( i ; ) d t ^ and this suggests that the integrated mean square error jE\fn(x)— f(x)\2dx be no smaller than

will

2 3/5 J/(x) 4/5 | A x ) | 2 / 5 dx [J w2(*;) ^] 4 / 5 (J w(v)v2 dv)2l5n'A'5 to the first order as n -> oo. However, this would require locally scaling b(n) with i£ depending on x when estimating /(x). If the scaling were to be global with K independent of x, we would expect \E\fn(x)-f{x)\2dx

^ j ^ M ' O ' f o + W ' O j - o o |/"(x)| 2 ^(Jw(u)« 2 J«) 2

This suggests taking b(n) = A>J

1/5

\nb(n)

'

with

2 2/5 [Jw 2 (i;)^] 1/5 K = (j(f"(x))2 dx)l'5{jw(u)u2 du}2/5 leading to (23) $E\fn(x)-f(x)\2dx

=2 3 / 5 [Jw 2 (t;)^] 4 / 5 a \f\x)\2

dx(\w{v)v2

d v ^ u ^ + o(n" 4/5 ).

The estimate (23) can be obtained rigorously under the assumptions that (i) / is bounded, twice continuously differentiable with f9 f" e L2 (ii) w is bounded nonnegative and symmetric with J w(u)u2du < oo. For then we can make use of the simple bound J [ J w(v)f(x - Kn)v) dvf dx = J w(v)w(v'){ J/(x - b(n)v)f(x - b(n)v') dx] dv dv( ^[$w(v)dvf$f(x)2dx and the estimate f(x - b(n)u) -f(x) = ■~b(n)uf,(x) + ^b(n)2u2f,(x-eb(n)u). The 2 2 2 estimate implies that {j w(u)[f(x-b(n)u)-f(x)]du} = {j w(u)jrb(n) u f"(x-0b(n)u) du}2. The expression for K and (23) can then be obtained with a small additional argument by using (4) and (10). 259

1820

MURRAY ROSENBLATT

The type of discussion has been carried out several places some time ago. Epanechnikov's simple observation is that one simply ought to look for the weight function w minimizing (24)

\w\v)dv

subject to the restraints (i) \w(v)dv = \ (25)

(ii) w(v) = w(-v) (lii) §v2w(v)dv = 1.

If 8w represents a small deviation for an extremum subject to the restraints (i)-(iii), the variation of $ w^dv

+ X^tf w(v)dv-i} + X2{$ w(v)v2 dv-±}

should be zero (Xt and X2 are multipliers) and one is therefore led to 2 $ w(v) 5w(v) dv + Xi $ dw(v) dv + X2 $ 5w(v)v2 dv = 0. Thus 2w(v) + XX + X2v2 = 0,

w(v) = ( - Xx - X2v2)/2.

The function w(v) is zero at

v=±(-X1/X2)*. To have a function that is integrable and symmetric one ought to set w(v) = (-^-X2v2)l2

if \v\ ^ (-A,M 2 )*

=0

otherwise.

The constants Xu X2 are determined by the conditions (i) and (iii). Then Xx = — £5""*, A2 = J* ~* so that the conjectured function is (26)

w(v) = f5"*(1 -v2/5) =0

if \v\ ^ 5* otherwise.

This is a nonnegative weight function. Consider a variation 5w(v) about (26) which is such that J 8w(v)dv = 0, $ v28w(v)dv = 0 and 8w( — v) = Sw(v) with dw(v) ^ 0 for \v\ ^ 5*. Then $(w(v) + 8w(v))2dv = $(w(v) + 8w(v))2dv-%5-±{$(w(v) + 5w(v))dv-l} (27)

+i5-*{$(w(v) + Sw(v))v2dv-l} = $w(v)2dv + ${2w(v)-%5-±-%5-*v2}dw(v)dv + $(5w(v))2dv ^$w2(v)dv + $(dw(v))2dv 260

1821

CURVE ESTIMATES

because the integrand in the second integral of line (27) is zero for \v\ ^ 5* and nonnegative for \v\ ^ 5*. Thus the function w(v) given by (26) gives an absolute minimum for (24) under the restraints (i) to (iii) in the class of nonnegative weight functions. Exactly this variational problem was solved by Lehmann and Hodges in a nonparametric investigation [21]. Condition (ii) could be replaced by (ii)' J" uw(u)du = 0. However, one should note that if weight functions w with negative values are allowed, expression (24) can be made as small as is desired even though the re straints (i)-(iii) are satisfied. We reproduce a small part of a table of Epanechnikov (this is the point of greatest interest in his paper) to show how insensitive (24) is to the shape of w when dealing with nonnegative weight functions satisfying (i)-(iii). The minimizing weight function (26) is referred to as w0. TABLE 1 L = $w2(u) du

3.5"*

Wo

1/6*-1^|/6 0

if

W^6

if

\y\>6

(2w)-* e'y2/2 i3~* 0

r — $w2(u) dul$w02(u) du

if

\y\ £ 3*

1

6*/9

1.015

2-1JI-*

1.051

i3~*

1.077

if \y\ > 3*

From this table it is clear that one does almost as well with the Gaussian or even the rectangular weight function as one does with the optimal nonnegative weight function w0. In computation, there would be clear advantages in dealing with a bandlimited weight function like w0. When nb(n) -> oo as n -> oo (b(n) -+ 0) the central limit theorem can be used to get a normal approximation to the probability distribution offn(x). Some idea as to the error in the approximation is given by the Berry-Esseen theorem. Specifi cally, if the weight function w is bounded and nonnegative with / continuous, bounded and positive then (28)

\Pio(fn{x)Yl{fn{x)^Efn{xj]

£ x ] - 0 ( x ) | ^ Kl(nb(n)f

where Kfor sufficiently large n is bounded above by (29)

9\w{uydu{lw(u)2du}-%f{x)Y±. 261

1822

MURRAY ROSENBLATT

Iff is twice continuously differentiate with f(x) = 0 and f"(x) > 0, one would require nb(ri)3 -> oo and J w2(v)v2dv < oo for asymptotic normality of the estimate fn(x). It is of some interest to see whether one can get better and more detailed approximations for the distribution of such estimates when using a simple bandlimited weight function w(u) (say triangular or even uniform). Steepest descent methods (see Daniels [12]) may be useful. An alternative procedure would be to condition the number of summands in (2) that are nonzero, if the weight function is bandlimited. Approximations suggested by a paper of Kolmogorov [23] may then be helpful. It is clear that corresponding results can be obtained for estimates of multi dimensional probability density functions. Just a few remarks will be made in the case of two dimensional estimates. Let w(u, v) be a bounded integrable weight function with (30)

J w(w, v) du dv = 1.

Again let b(n) be a linear bandwidth tending to zero as n -> oo. The population is assumed to have an absolutely continuous two dimensional distribution function F(x, y) with continuous probability density

The estimate fn(x, y) off(x, y) 1

(3D

»

fx-Xj

y-Yj\

^y^^z^y-KnT'W))

is computed from a sample (Xu Yx), • • •, (Xn9 Yn) of independent observations from the population. Assume t h a t / a n d its partial derivatives up to second order are bounded and continuous. Further let (32)

J \w(u,v)\u2dudv,

J \w(u,v)\v2dudv

< oo

with (33)

J w(u, v)u dudv = J w(«, v)v du dv = 0.

Then the variance and mean offn(x9 y) are given asymptotically by (34)

o2Un(*,y)~] =

n-1b(ny2f(x,y)$w2(u,v)dudv

and (35)

Efn(x,y) =f(x,y)+$b(n)2[fxx(x,y)m2y0

+ 2fxy(x,y)rnul

+fyy(x,y)m0t2] + o(b(n)2)

as n -* oo, b(n) -> 0. Here the ma> fi are the moments (36)

J uavpw(u, v) du dv 262

1823

CURVE ESTIMATES

of the weight function w. The estimate is asymptotically normal if nb(n)2 -> oo as n -> oo. A more detailed discussion of multidimensional density estimates can be found in [7] and [13]. 3. The process [fn(x)-Efn(x)]. In this section the object is to study the process [fn(x)-~Efn(x)]> o v e r a n interval (say x e [0, 1]), suitably normalized so as to obtain a nontrivial result as n -> oo. The simple result obtained will be derived under unpleasant and completely impractical conditions. However, heuristically the result ought to hold under reasonable conditions. As we shall see, the type of result obtained suggests that limit theorems for sequences of random processes over finite intervals whose length is asymptotically unbounded may be of considerable interest. The asymptotic behavior of the process [fn(x)-Efn(x)] at a special set of points xj e [0, 1], j = 1, •••, n, has been considered by Woodroofe [38]. To make it easier to follow the argument, the exposition is broken down into a number of stages. At the end the assumptions made in these stages are put together to get a coherent result. The argument is due to joint work with Peter Bickel. (i) Consider the estimate fn(x) off(x) given by (2). Let us look at the covariance of Mx),fH(y) (37) Cov(/n(x),/B(y)) = j ^ 3

w

x—u\

(y—u ]f(u)du

\mr\m

i

w(z)f(x-b(n)z)dz

!■ w(z)f(y-b(n)z)dz

From remarks made in Section 2, it is clear that to get a nontrivial limiting distribu tion at a fixed point x as n -> oo, one ought to look at (38)

(nb(n)fUn{x)-Efn(x)l

We shall be interested in (38) as a process with parameter x varying over a finite range, say 0 ^ x ^ 1, for convenience. To get a reasonable limiting covariance function, the scale in x has to be renormalized so that we look at (39)

(nb{n)f\_fMn)x)

- Efn{b(n)x)~]

withO ^ x ^ bin)-1. Thus (40) nb(n) Cov [fn(x)Jn(x + aZ>(n))] = J w(z)w(z + a)/(x - b(n)z) dz - bin) J w(z)f(x - b(n)z) dz J w(z)f(x + (a - z)b(n)) dz and (40) tends to (41)

f(x)$w(z)w(z

+ a)dz

as n -> oo and b(ri) -> 0 iff is bounded and continuous at x while w is integrable and bounded. For asymptotic normality of (39) as n -> oo, we require that 263

1824

MURRAY ROSENBLATT

nb(ri) -^ co. Let us now assume that/is continuous and bounded away from zero on [0, 1]. Then we could look at (42)

(nb(n)f{f{x)-] " *[/,(*) - £/„(*)]

so as to get the limiting covariance function (43)

r(a) = J w(z)w(z + a) dz,

with the proper change of scale. (ii) We now compare the process (42) with the process obtained by looking at (42) at the points Xj = jc(n) where j = 1, 2, •••, [b(n)~lc{nyl] ([y] denotes the greatest integer less than or equal to y here) and interpolating linearly over the range 0 ^ x ^ bin)'1. To estimate the difference between these two processes it will be enough to look at (44)

supo^^^-^nK^I/nC^W-^/nCK^) -fn(b{n)lxlc^

The expression (44) can be conveniently rewritten as (45) s\\p0^b{n)-i(nb{n)f\

J {exp [-itb(n)x~]-exp [-»&(n)[x/c(n)]c(n)]} -h(b(n)t){(pn(t)-(p(t)}dt\

where (46)

h(t)=$eituw(u)du

and

(47)

%(0 = » ' 1 I i - i ^ ,
if h e L. Notice that the second moment (48)

E(sup0^Hnr.(nb(n))*\>.-\)2 ^n&(n)£{sup0^5(B)-iJ|l^ ^\h(b(n)t)\\cpn(t)-cp(t)\2dt}

and that the right-hand side of inequality (48) is bounded by

(49)

nfe(n)M«)2J^|^(K«)0|^^J|^(K^)0|£|^(0-^(0|2^^

However, (50)

E\(pn(t)-(p(t)\2^n-K

It follows from (49) and (50) that (44) tends to zero in probability as n -> oo if (51)

c(n)2lb(n) -+ 0 264

1825

CURVE ESTIMATES

(b(n) -> 0) and h,t2he L. Van Ryzin's paper [36] should be referred to for a related set of estimates. (iii) Let £i9 E^t = 0, / = 1, •••, n be independent identically distributed random k vectors. Set (52) i = l,-,k Pi=E\t;uYlEH\i, where £u is the /th component of the random variable £ t . Fn(x) is to denote the distribution function of n~*Yj=i^t a n d G(x) the /c-variate normal distribution function with means zero and the same covariance structure as Fn(x). Sazanov's Theorem 3 in [31] then states that (53)

sup x e R k \Fn(x)-G(x)\

^ C"(fc)Ei=i

PIFT*]*

where C\k) = Ck2 with C an absolute constant. In our context k = [c{ri)b(ri)Yl (the number of points xi = jc(n) in 0 ^ x ^ £(«) _1 ) and p f ^ C'b{n)'~* for all / with C" an absolute constant. The estimate of the error on the right side of (53) becomes C"c(ny1/3b(n)-5/2n-116 with C" an absolute constant and if we take c(ri), b(n) n -> oo, this estimate of the error will tend to zero, (iv) The covariance function of the process (42) is (54)

{f{x)f{y)}-*b{nTl

x—u

w b(n)

-b(n)2

0 sufficiently slowly as

y—u

w \b{n) f(u)du

w(z)f(x — b(n)z) dz w(z)f(y — b(n)z)dz

Let 0Yn(x), 0 ^ x ^ 1, be the Gaussian process with mean zero and covariance function (54). By using the Sazanov theorem, one can approximate the process (42) at the points jc(n)b(ri),j = 1, ♦•♦, [c(ri)b(ri)]~"*, in distribution by the Gaussian pro cess 0 Yn(x) at those same points. The type of argument used in (ii) shows that the supremum of the absolute difference between 0 Yn(x) and the process obtained from x 0Yn by linearly interpolating between the points jc(n)b(ri), j = 1, ••♦, [c(ri)b(ri)]~ 2 tends to zero in probability as n -> oo if c(ri) /b(ri) -> 0 and b(ri) -> 0. Let (55)

i Y„(x) = 0 Yn(x) + [(&(«)//(*))* J Hz)f(x

- b(n)z) d£] U

where U is a normal random variable with mean zero and variance one independent of the process 0 Yn. The process t Yn(x) can be written as 1Yn(x)=(b(n)Y

w

Kn) \f(x)

' dB(u)

where B{u) is a Brownian motion process with — co < u < oo. Here B(u) is a Gaussian process with increments over non-overlapping intervals independent, B(0) = 0, and E\B(u)-B(v)\2 = \u-v\,EB(u) = 0, 265

1826

MURRAY ROSENBLATT

for — oo < u, v < oo. Further, the process x Yn(x) can be approximated in the sup norm over 0 ^ x ^ 1 by

Jn(x)=(b(n))-^w(^£jdB(u). This can be seen by making use of an idea from a paper of Garsia, Rodemich, and Rumsey [16]. Let (56)

V(x) =

tY„(x)-2Yn(x).

We are interested in estimating sup o g ; t g 1 |t/(x)|. U(x) is a Gaussian process with mean zero and covariance function (57)

R(x,y)

1 bin),

w

x — u\

1 1 du.

iJ-Kl) -

ly—u

'W)Hwy

Let

AR(x, y) = R(x, x) + R(y9 y) - 2R(x, y). The following Lemma is an immediate corollary of Theorem 2.2 of [16]. LEMMA. Let U(x), 0 ^ x ^ 1, be a Gaussian process with mean zero and covariance function R(x, y). Consider p(u) a nonnegative even function that is continuous and nonnegative for u ^ 0 and such that

rdjiu) Jo u •< If then

11

C'ARjx,

oJoP2(x-

oo.

dx dy < oo

s u p o ^ s l | l / ( x ) | ^ | l / ( 0 ) | + B±

ri

dp(u)

o w

where B is a nonnegative random variable with EB
o

p2(*-y)

dxdy

and c an absolute constant. We shall now apply this Lemma to estimate E f s u p o ^ ^ |t/(x)|] 2 where U(x) is the Gaussian process (56) with covariance (57). In this case AK(x,)0 = with

1 bin),

a{x, u) = w

[tf(x, u) — a{y, w)]2 du x—u

W) 266

/(*)

■1

1827

CURVE ESTIMATES

We clearly have |A/?(x, y)\ ^ - ^

{a(x, u)2 + a{y, u)2} du.

L e t / b e continuously differentiable w i t h / , / 7 bounded a n d / b o u n d e d away from zero on the interval [0, 1]. It then follows that ^a(x,u)2du

Kl^(z)2[f(x-b(n)z)-f(x)fdz

^

^ K2b(n)2\w(z)2z2dz

K3b(n)2

^

if j w(z)2z2dz < oo with the K{ constants. The uniform bound |A/?(x, y)| = 4K3b(n)2 for AR(x, y) is obtained. However, another bound will be required for \AR(x, y) | when x is close to y. Let y — x = ab(n) with |a | ^ M < oo where M is a constant. Then 2r

y—x w(z) — w | 7777^ + z b(n)

\&R(x,y)\^2

+

W

y-x b(n)

+ z)

f(x-b(n)z)\±

12

f(x-b(n)z){f(x)-±-f(yy±}2

dz

dz

^ X 4 j w / ( z ) 2 d z a 2 + X5a2 if w7 is continuous and bounded and in L 2 . Thus \AR(x, y | ^ # 6 a 2 if j> —x = a6(«) with |a| ^ M < oo. Let us set p'(u) = w _ i , 0 < w, in applying the lemma since jl dp(u)ju = jl u'^du is finite. Also ''ARQcjQ

dxdy^K

b(n)2 ]b(n)2^\x\<2

X

dx + K'

dx

^K"b(n)2\logb(n)\. Notice that the Gaussian process Y(x) = 2 l^C^CO) has the desired covariance function r(a) = §w(z)w(z + (x)dz. We should now like to apply the result on the asymptotic distribution of the maxi mum of a Gaussian process as given, for example, on pages 271-272 of Cramer and Leadbetter [11] to the process Y(x) on the range 0 ^ x ^ Kn)'1. Assume that the covariance function r(a) of the process Y(x) as given by (43) is an even function that is continuously differentiate up to fourth order and such that (58)

r(a) = 0(|a|" £ )

as |a| -> oo for some e > 0. Let (59)

^o = KO) X2 = r"(0). 267

1828

MURRAY ROSENBLATT

Then the result cited in [11] indicates that (60)

max0^rAo*Y(x)^(21ogT)±+

A+z (21ogT)^

as T -> oo where (61)

A=log{(-2A 2 M 0 )*/2*}.

The estimates made in (ii), (iii) and (iv) (or obvious modifications of them) indicate that under appropriate conditions o n / a n d w the distribution of the process (42) can be sufficiently well approximated by that of Y(x) so as to obtain the following theorem. Let f be continuously differentiable and bounded away from zero on [0, 1]. Assume that the bounded, integrable weight function w used in the estimate fn (see (2)) is twice continuously differentiable with (1 + I>4)|H>"(I;)| bounded and has a Fourier transform h(t) with (1 +t2)h(t) e L. Then if n~1/24 = 0(b(n)) as n-> oo, it follows that THEOREM.

(62)

P max 0 ^x<\

nb(n)

{f„(x)-Efn(x)}^{2\ogb(n)

>H A+z

+(21og&(n)_1)* as n -> oo, where a = Jw2(u)(iw

(63) and (64)

A =log

El

2rc'

2 d \ B = --^- 2 (Jw(z)w(z + 0 d z ) | ^ o .

The interest in the theorem given above is not in the detailed conditions. The result, in particular, must hold under much weaker requirements on b(n). It is rather in its asymptotic nonparametric character and the ideas behind the proof. Notice that in the proof one approximates the process (42) in distribution with a corresponding Gaussian process. The heuristics that suggest a result like that in the above Theorem also imply an asymptotic nonparametric character (under appropriate conditions) for 1

(65)

[/»(*)-/(*)]: dx fix)

when suitably normalized. There are the corresponding open questions for twodimensional (and higher dimensional) density functions. For, example, let/Xx, y) 268

1829

CURVE ESTIMATES

be the estimate given in (31) of the density function f(x,y). estimates are (66)

The marginal density

gn(x)=ifa(x,y)dy K(y)=ifn(x,y)dx.

A natural statistic to test independence over [0, 1] x [0, 1] is given by 11

(67)

i

[l\fJLx9y)-gn(x)hn(y)\2 r<

\

dxd

y

and its distribution should be investigated. 4. Probability density estimates for dependent sequences. It is also curious and pleasantly surprising that the asymptotic results on the behavior of probability density estimates obtained in Section 2 still hold even when sampling from a stationary process, if appropriate and rather reasonable conditions are satisfied. This is not true at all if the distribution function is estimated (see Billingsley [3, page 195] for a discussion) and the fact that it holds when estimating the probability density is due to the local character of the estimate. Let {Xj9j = •••, - 1 , 0, 1, •••} be a strictly stationary process. Assume that the instantaneous distribution function F(x) = P[Xj £ x] is absolutely continuous with continuous spectral density f(x) = F'(x). We wish to estimate/(x) by/„(x) as given in (2). The assumptions on the weight function w, the density function/, and the bandwidth b(n) are the same as those in Section 2. The results on the bias of the estimate as given in (16) hold under the same condi tions as in Section 2 since dependence has no effect on the bias. Let us now consider the covariance of the estimate. At this point the character of the dependence must be examined. Let the two dimensional distribution functions (68)

F/x, y) = P[X0 g x, Xj £ y],

j * 0,

be absolutely continuous with continuous density functions (69)

fJ(x,y)=fxFyFj(x,y).

The covariance offn(x) and f„(y) is (70) Cov[fn(x),fn(y)-]

=n-2b(n)-2YZ=-„(n-\j\)CovWb(nri(.x-X0), wWnTHy-Xj))-]

with (71) Cov [w(b(nyl(x - X0)), w(b(nr' (y - X,))] = b(nf j J w(u)w(v){fj(x - b(n)u, y - b(n)v) -f(x - b(n)u)f(y - b(n)v)} du dv 269

1830

MURRAY ROSENBLATT

if j # 0 and (72)

Cov [w(b(nyl (x - X 0 )), w(b(n)-l (y - X 0 ))] = b(n) j w(u)w ( u + ^r— Jf(x - b(n)u) du — b(n)2 J w(u)f(x — b(n)u) du j w(v)f(x — b(n)v) dv.

Thus, if (73)

!j*o\fj(x,y)-f(x)f(y)\

^ M < oo

for all x and y, we have 74) nb(n)[Cov(fn(b(n)x)Jn(b(n)y)y]

= $ w(u)w(u + y - x)f(x - b(n)u) du + 0(b(n))

for all x and y as « -> oo. The first term on the right of (74) is fix) J w(u)w(u + y-x)du

+ 0(b(n))

i f / h a s a bounded continuous derivative. Additional conditions on the type of dependence are required to get a result on the asymptotic normality of the estimate fn(x). A standard result making use of the conditions will be described. The details of the proof can be found in [29]. Let £%n and !Fm be the backward and forward Borel fields generated by the random variables {XyJ S n} and {Xj\j ^ m) re spectively. The stationary process {Xk} satisfies the condition S if for every «^"w+fc measurable random variable Y(k > 0) with EY2 < oo, EY = 0, (75)

E\E(Y\@n)\2

^a(k)EY2

where (76)

0(k-4-£)

a(k) =

for some s > 0 as k ~> oo. If {Xk} satisfies condition S and the other requirements already specified, then { J w2(v) dv} - Hnb{n)f{fn{x)

- £/„(*)]

is asymptotically normal with mean zero and variance one as n -> oo, nb(n) -> oo, b(n) -> 0. The paper of Roussas [30] discusses related questions. The papers of Van Atta and Chen [34] and Frenkiel and Klebanoff [15] look at estimates of univariate and multivariate probability density functions without mentioning the estimation problem explicitly. They are interested in the probability density of the distribution of a component of the velocity in grid turbulence as well as the bivariate distribution of velocity components at the same point in space but with a time difference. They find that the univariate distribution appears to be normal but not the bivariate distribution. A recent refinement in the "theory" of turbulence suggested by Kolmogorov is discussed in a paper of Yaglom [39]. The refinement suggests that a local spatial average of the energy dissipation might have a lognormal distribution. Currently, 270

1831

CURVE ESTIMATES

experimental results are being analyzed to estimate the probability density of such a local average of the energy dissipation to see whether it is reasonably approxi mated by a lognormal density. 5. Expansions in orthogonal functions. There are many ways of estimating a prob ability density function from a sample of independent observations on a popula tion. Thus far in this paper, only estimates of the form (2) have been considered. Still another interesting class of estimates has been suggested by Van Ryzin [36]. An obvious class of estimates is also suggested by the notion of an orthogonal expansion. Just a few remarks will be made on this class but they will be enough to show that qualitatively results similar to those already obtained would be expected. However, it should be noted that there are still many open detailed questions. Let {
7LJCJ
with (78)

Cj = $f(x)(pj(x)w(x) dx

will converge t o / i n L2(w). Let Fn(x) be the sample distribution based on a sample of n independent observations Xu X29 •••, Xn from the population with absolutely continuous distribution function F(x) and continuous probability density f(x) = F'(x). It is plausible to estimate Cj by (79)

c*{n) = S^Jxjw(x)dFn(x)

=

fl-'EUi^W**)-

Then (80)

Ec/

= cj

and (81)

Cov(c/,c k *) = n~' =

n

Cow(cpj(X)w(XXcpk(X)w(X))

" * {J ^/*)%(*)w 2 (x)/(x) dx - Cjck}.

In all the cases we shall think of, the functions cpj will be continuous in the domain in which they are defined. Consider a sequence of weights {ctj(ri)} that are zero except for a finite number of values of y. Let the estimate of f(x) be (82)

fn(x)=YjCj*(n)aj(n)cpJ(x).

The bias (83)

Efn(x)-f(x)

= Ic»)-l)«p/x) 271

1832

MURRAY ROSENBLATT

and the covariance (84)

Co\ (f„(x),f„(y)) = n

1

Yj,k(Pj(x)(pk(y)aJ(n)ak(n) ■ { J
If the Fourier expansion of/converges to/absolutely, one would obviously require that (85)

a » -+1

as n -> oo for each fixed j . The integrated weighted mean square error has mean value (86)

EJ\Mx)-f(x)\2w(x)dx=lc2[Mx)Mx)dx

j\Efn(x)-f(x)\2dx

+

= n~l £ K-(n)| 2 U 2 2

\aj(u)\2W2(u)f(u)du-\cj\2}

+EN k(«)-i| .

It is sometimes convenient to rewrite fn(x) in the form (87)

fn(x) = n" > £," = 1 w(Xy)fcB(x, Xj)

where (88)

fc„(x,

u) = £,. ccj(n)(pj(x)(pj(u)

is a "generalized" kernel function (see G. S. Watson [37] for a related discussion). The variance of fn(x) can then also be written as (89)

n-Hfw2(u)|fcw(x,iO|^

The asymptotic behavior of the kernel function kn(x, u) is obviously of importance in determining the behavior of the variance (89) as n -> oo as well as the asymptotic distribution of/„(x). We consider the case of the trigonometric functions (90)

eijx q)j(x) = - ^ ,

- n ^ x ^ n, w(x) = 1,

because it is simple and is amusingly like the case of spectral estimation. Here (91)

kH(x9u) = K(x-u)

=

Yj*Ju)etJ<*-u\

For economy in computation one would require a/w) = 0 for |y| ^ n. In fact, the usual situation will be that in which Gtj(n) = 0 for \j\ ^ m(n) with m(n) = o(n). Assume that (92)

\kn{x — u)du = a 0 (n) = 1.

For convenience, assume that kn(u) is a nonnegative weight function that behaves asymptotically like a 5 function, that is, given any e > 0

(93)

JW*A(iO0 272

1833

CURVE ESTIMATES

as n -> oo. Asymptotic unbiasedness of the estimate fn(x) follows since (94)

Efn(x) = J kn(x, u)f(u) du - / ( x )

as n -> oo if f is bounded and continuous. The variance (T2[fn(x)] is given asymptoti cally by ~ , - . . / ( * ) k (u)2 du. cr22r[/,(x)]^ n n

(95)

A similar argument shows that (96)

E\kn(x,X)-Ekn(x9X)\*

^f(x)lkn{ufdu.

A sufficient condition for (87) to be asymptotically normal with mean Efn(x) and variance (95) as n -> oo is that v

f kju)4' du n{)kn{u)2du)z

J

by the Liapounov form of the central limit theorem. Suppose we look at the case of simple truncation. (98)

« » = 1

if

\j\£m(n)

=0

if

\j\ > m(n)

or the Fejer weights (99)

aj(n)

= l-\j\/m

if

\j\£m(n)

=0

if

\j\>m(n).

Then (100)

J/c„(w)2dw ^ cxm(ri) \kn{u)A'du ^ c2m(n)3

with cl9 c2 > 0 so that (97) is satisfied if m(ri) = o(n) as n -> oo. Similar computations will now also be carried out for orthonormal polynomials with respect to a weight function. The conditions assumed will be somewhat re strictive but the results obtained are of interest because they suggest what ought to hold under much weaker conditions. Let w(x) be a weight function on — 1 S x ^ 1 mthpj(x)9j = 0, 1, •••, the orthonormal polynomials with respect to this weight function. We wish to consider estimating a continuous density function f(x) on - 1 S x ^ 1 by (87) with (101)

kn(x, u) = ^jaj(n)pj(x)pj(u).

Let x = cos 0, z = el° and (102)

/i(0) = w(cos0)|sin0|. 273

1834

MURRAY ROSENBLATT

Assume that w is such that h(9) is a positive continuous function on the unit circle z = eie, — 9<9^n(9 = — n is identified with 6 = n), and h satisfies a uniform Lipschitz condition. Let „(z), n = 0, 1, 2, •••, with z = el6, — n < 0 ^ n, be the orthonormal trigonometric polynomials with weight function h(9) generated from 1, z, z2, •••. The coefficient of z" in 4>n{z) is Kn. Then by [32, page 292]

(103)

pn(x) = (2*)-*|l+^'J {z^^K^C*" 1 )}

for n ^ 1. Also, if (104)

ie

ie

g(h;re )=g(re )

I (*" e'^ + re1'9 = exp< — -X—^log/j(A)rfA

then by [18, page 51] (105)

lim^ic^lXo)]-1 lim»-oo0»(o)=O.

Further, by [18, page 53] one knows that (106)

logn znlg(h;z)+—0(l)

4>n(z) =

with the 0(1) uniform in n and z. Assume that |a/n)| ^ 1 with <x/ji) = 0 for |y'| ^ m(«) w/?ere /w(w) = 0(n) and lim„^00a» = l /or eachfixedj . Then, if the expansion of/ in terms of the orthonormal polynomials pn is absolutely convergent, the estimate fn(x) will be asymptotically unbiased. To avoid difficulties arising from possible singularities at x = + 1 , we shall assume that (107)

f(x)(l-x2yl

is bounded on — 1 ^ x ^ 1. The object is to again determine the asymptotic behavior of the variance offn(x) and sufficient conditions for asymptotic normality of/„(*). From (103) and (106) it follows that (108) k„(x, y) = £ ccj(n)Pj(x)pj(y)

=Y 7 C I«>y.+ ^ - J

^lg(h;z) + z^lg(h,z-1)

+ 1 ^O(l)J|f'^(/i; Z 0 + z'"^(A;z'- 1 )+^yO(l) where x = cos 9, z = eie, y = cos 9\ z' = ei0'. Let (109)

cn(0)=£}=1a»^9 274

1835

CURVE ESTIMATES

and assume that (110)

\c„(0)\ ^ M(e) < oo

for \0\ ^ e.

Then (111)

a 2 [/„(x)] =^w2(u)\kn(x,u)\2f(u)du

^ i

( T

^

i

J

\cn(e)\2d6

as n -> oo, if |x± 11 ^ 8 > 0 and (112)

(logm(n))2=o(J|cn(0)|W.

Further, (113)

J W \u)\k n (x,u)\ 4 f(u)du

^ A^n

|c8(0)|4rf0

with ^4 a constant, as w -> oo, if |x± 11 ^ 8 > 0 and (114)

(logm(«)) 2 =o(J|c„(0)| 4 d)i.

Thus, a sufficient condition forfn(x) to be asymptotically normally distributed with mean Efn(x) and variance (T2[fn(x)] as n -> oo, given the requirements already specified, is that

(U5)

«(JM9|W*°

as A -> oo. The condition required on the weight function w(u) is appropriate for the Chebyshev polynomials of the first kind but not for the other classical ortho gonal polynomials. For just the Chebyshev polynomials of the first kind one would go through a direct and simple computation to get the results (111) and (115). The interest in deriving (111) and (115) under the conditions we have specified is that it suggests what ought to hold more generally for density estimates based on expan sions in orthonormal polynomials on a finite interval. The case of classical orthonormal polynomials other than the Chebyshev polynomials of the first kind should be considered. Notice that asymptotic results like those obtained for expansions in the trigonometric functions can be derived for expansions in terms of the eigenfunctions of certain types of Sturm-Liouville problems by using the remarks made on page 117 of [18]. 6. Spectra and stationary processes. There are many extended developments of the theory of stationary processes and discussions of spectral estimation [17], [33]. We shall briefly remark on some of the results that relate or are comparable to those already mentioned on estimation of the probability density. Let X(t)9 t = •••, - 1 , 0 , 1, •••, be a discrete time parameter r-vector (column vector) valued stationary process with real-valued components. If second order moments exist, then (116)

X(t)=^neitXdZ(l) 275

1836

MURRAY ROSENBLATT

where Z is an r-vector valued process (with complex-valued components) of orthog onal increments such that (117)

EdZ{X)dZ(n)'

= 5(A + ii)dF(X)9

-TI< ^

< TT.

In formula (117), 8 is the Kronecker 5 symbol 8(1) = I

if 1 = 0

= 0

otherwise,

F i s an rxr matrix-valued Hermitian non-decreasing function (F(X) — F(fi) is posi tive semidefinite if X ^ //), and A' denotes the transpose of A. Since the components of X{t) are real-valued (118)

dZ(-X)=dZ(X)

and dF(X) = dF(—X)'. In the case of a Gaussian process, the full probability struc ture is determined by the first and second order moments. It is convenient to assume that EX{t) = 0. The covariance matrices (119)

K 0 = £ X ( T ) X O + T)'

are related to the second order spectral distribution function Fby r(t) = \n_neitXdF(X).

(120)

If the existence of all moments is assumed, we have (121)

m ^ . . . , ^ , . . ^ ^ * ^

where Xa{t) is the ath component of X{t). Assume that the moments have Fourier representations. (122)

maum..tak(tl9..;tk)

= E(Y\UZaj(wj)) with the functions G of bounded variation. The stationarity of the process X(t) implies that (123)

dG = 0 unless

Xiwy

= 0

modulo 2n.

The assumed representation (122) can also be written as (124)

cau...,ak{tu-,tk) = jln---$lnexp(iY*=ltJWj)c{dZaj(Wj);j = c{dZa.(wj);j = l,---,k} 276

=

\,---,k}dFau...,Jwl,■■-,*,)

1837

CURVE ESTIMATES

where c is the corresponding cumulant function. It is convenient to assume that the cumulants ca • , f l f c V M > tk) are in Lx as functions of their k — 1 (or k) t argu ments (see Condition I in (137)). This can be regarded, if assumed up to order k, as a curious mixing condition of kth order. This then implies that

c.

(125) dFait...}ak(wu-'-,wk)ri(YJltWj)

=/«,,...,«>>,, - , w^QTi wj)dw1 - dw*

(^(w) = 0 if w 7^ 0 modulo 27i and is equal to one otherwise) where the cumulant spectral density/ is continuous and a function of only k—l variables w since Y*L Wj = 0 modulo 2n. Let us first look at a real-valued stationary process X(t) (r = 1) and consider estimating its spectral density of second order. Such an estimate is given by (126)

1

fN\X)=T

2n

with

£ *„<">/?,<"V"

wv{N)=$!KeivXWN(A)dA

(127) and (128)

^

t,T=

1 , t~

T=V

Formula (126) could also be written as (129)

f
with (130)

7(/V)(a) =

1 2niV

I *(0<

-if A

the periodogram, and it is convenient to assume that WN(u)=ANBN-iW(BN-1u)

(131)

if | « | ^ 7 T

with BN | 0 as N -> oo and v4w a normalizing factor so that (132)

$!KWN(u)du = l.

Assume WN is periodically extended with period 2n. Then if W has a finite second moment, is nonnegative and symmetric about zero and NBN -> oo, we can show that (133)

BN-2lEfN\X)-f(X)-]

u2W(u)dJ~+0(BN-3)

and (134)

Cov{/(N>(A),/(W)0i)} = InN-1^

WN(X-a)WN(ii + a)f2(a)da

+ jlKWN(A-<x)WN(n-a)f2{a)dz 277

+ 0(N-1)}.

1838

MURRAY ROSENBLATT

By (131) and (134), it follows that l i m ^ BNN Cov (135)

{fiNWN\ii)}

=0

if

X^ii

= 2nf\X)\n_nW2{u)du

if

= Anf\X)\n.nW2{u)du

if X = ix = 0,7i

^=/x#0,7c

when 0 ^ A, ju ^ 7c. If we look at the asymptotic mean square error of estimates, the analysis of Epanechnikov given in Section 2 still holds and one is led to the weight function W(u) given in (26). The weight function W{u) (26) is bandlimited but it leads to a sequence wv{N) that is not bandlimited. But this need not be too frustrating since direct Fourier analysis via the fast Fourier transform (see [10]) may be preferable. One would directly compute the periodogram and then smooth it. The asymptotic results obtained in spectral estimation differ a bit from those obtained in estimating the probability density function. The basic range over which (135) holds is 0 < X < n with a slightly different result at X = 0, n. On the other hand, the results obtained in (15) for probability density estimation hold over the entire range of the independent variable x. Also, in the asymptotic formulas for the variance of the estimates, the constant/(x) in (15) is replaced by 2nf2(X) in (135). Notice that if we rescale locally in the covariance result (134) for spectral estimation, let X = n + BNa (A, \i # 0, n) and then normalize appropriately, as N ~> oo one would expect a Gaussian process with covariance function (136)

$W(b + a)W(b)db.

This is again very much like what was obtained in Section 3 when estimating a probability density function. It would be interesting to look at the functional of a global character such as (65) mentioned in Section 3, when estimating the spectral density. The applications of second order spectral techniques are legion and the development of theory is due to many people—Bartlett [1], Tukey [33], Grenander and Rosenblatt [17], Parzen [24] and others. The success of these second order techniques suggested that it might be worth while developing ways of estimating higher order spectra for situations in which there are nonlinear or nonGaussian effects. Tukey [33], Van Ness [34], Brillinger and Rosenblatt [4] developed a certain theoretical background. There have been a limited number of applications of which a few are mentioned here—the work of Hasselman, Munk and MacDonald [19] on bispectra of ocean waves, Haubrich [20] on earth noise, Brillinger and Rosenblatt [5] on sunspots, D. Cartwright [8] with a 4th order analysis of tides and surges, and Huber, Kleiner, Gasser and Dumermuth [22] on electroencephalographs. We shall just briefly mention some asymptotic results due to Brillinger and Rosenblatt [4], Let c'au ...tak(v l9 •••,i>fc_1) and f'au ...,ak(^i> ♦•♦,/lfc-!) be the cumulantfunctioncfll,...,aieand cumulant spectral density fau ...tQk looked at as functions of A:— 1 variables. First assume

(137)

I

£»

....^^I^ 278

>„-,Vi)|<»

1839

CURVE ESTIMATES

fory = 1, • • •, k- 1 and any /c-tuple al9-~9ak with k = 2, 3, • • •. Condition I implies that/fl/])...)flk(/lj?--s/l/c_1)has a bounded uniformly continuous gradient. Let Wbe a weight function with (138)

W(-uu—,

-uk) =

W(u1,u2,---,uk)

and set ^ ( M l , " - , U t ) = B JV - t+1 H/(5 N - 1 Ml ,.--, J B JV - 1 Mt ).

(139) If (140)

d a (W) W=L W ="o 1 ^(0exp(-iA0,

then (141) with ]T* Ay = 0 modulo 27r, is a &th order analogue of the periodogram. Consider the estimate (142)

f^...,ak(^,-,)-k) k-1 {Inf-'N

i

AT-k+1 5l = — 0 0

'2TTSI

'

, 5 / c = — 00

2TTSA

J"V ' ' " '

2ns±

2ns^

i V

\

/2JMI

i V

2TT^

A/ / "1- -•"k I jy ' ' " ' jy JV

of/ ai afc(A!, •••, Ak) where it is understood that YA^J = ® modulo 2n. In (142) («!, •••, uk) = 1 if £* uk = 0 modulo 2^ but £ J e J wy # 0 on any nonvacuous proper subset / of {1, •••, k}. Make the additional assumption (143) II

/

fc-1

>

Wiui,■•■,ttk_1>-

X «/

3

/

*-i

£7; W("i> •••."*-!»- I 3«

M

a-i

i \i\-(*+«-n

<^(1+(E«/ for / = 1, •••, k — 1 with A, e > 0. Under assumptions I and II Covl/W...,^,-,^),/^.,^^,-,^)} (144)

=27riV-1Xp^co-J^ai-«1,-,4-a*)

•nf4,;«,(«i)^. • • • ^ ^ 1 -j^+o(BJV-'!+2iv-1) if B^'^-N -> oo as i?N -> 0 and JV -> oo. The summation in (144) is over all permu tations P on the integers 1, ••♦,/: and it is understood that Zi h

=

Zi My = ° 279

m

odulo 27T.

1840

MURRAY ROSENBLATT

Let X(t) be a stationary process satisfying assumption I. Assume that jf{N\ j = 1, •••, b9 are spectral estimates of order kx ^ ••• ^ kb whose weight functions satisfy assumption II. Let the bandwidths BNU) be such that BNU) -> 0,

(145)

(BNU)fj-

x

iV -> oo

as JV-> oo with JB N (1) ^ ••• ^ i?;v(fe). Bandwidths of estimates of the same order are assumed equal. Then, the estimates are asymptotically jointly normal as TV -> oo, with estimates of different order asymptotically independent, and estimates of the same order having \imN^BNk-1NCov[f^l.ia^1^--^dJ^l..taA^r--,fik)']

(146)

•$-„•••

SW(Tl9.--,Tk)W(TP{l)r--9TP{k^

For a detailed discussion and interpretation of these results, [4] and [5] should be referred to. There are some marked differences between the case of spectral esti mates and probability density estimates. In the higher order spectral case, asymp totic variances have products of 2nd order spectra in their principal term while in the case of probability density estimates, the multivariate density itself (see (34)) itself comes in. The stationarity and the real-valued character of the components of the time series imply that £ f Wj = 0 and f(wl9 vv2, •••, wk) =f( — wl9 —w2, •••, — wfc) so that there are additional symmetries or restraints. This also leads to inhomogeneity on lower dimensional manifolds in w space unless one is careful. Related ideas are used by Brillinger [6] in a discussion of the spectral analysis of point processes. Acknowledgment. I should like to thank colleagues who were kind enough to read the manuscript and send on their comments; and my thanks to Lillian Johnson for typing it.

REFERENCES [1] BARTLETT, M. S. (1950). Periodogram analysis and continuous spectra. Biometrika 37 1-16. [2] BARTLETT, M. S. (1963). Statistical estimation of density functions. Sankhya, Ser. A 25 245-254. [3] BILLINGSLEY, P. (1968). Convergence of Probability Measures. Wiley, New York. [4] BRILLINGER, D. R. and ROSENBLATT, M. (1967). Asymptotic theory of estimates of kth order spectra in Advanced Seminar on Spectral Analysis of Time Series, ed. B. Harris. 153-188. [5] BRILLINGER, D. R. and ROSENBLATT, M. (1967). Computation and interpretation of kth order spectra in Advanced Seminar on Spectral Analysis of Time Series, ed. B. Harris. 189-232. [6] BRILLINGER, D. R. (1970). The spectral analysis of stationary interval functions. Proc. Sixth Berkeley Symp. Math. Statist. Prob. To appear. [7] CACOULLOS, T. (1966). Estimation of a multivariate density. Ann. Inst. Statist. Math. 18 179-189. 280

CURVE ESTIMATES

1841

[8] CARTWRIGHT, D. E. (1968). A unified analysis of tides and surges round North and East Britain. Philos. Trans. Roy. Soc. London Ser. A 263 1-55. [9] CENCOV, N. N . (1962). Evaluation of an unknown distribution density from observations. Soviet Math. 3 1559-1562. [10] COOLEY, J. W. and TUKEY, J. W. (1965). An algorithm for the machine computation of complex Fourier Series. Math. Comp. 19 297-301. [11] CRAMER, H. and LEADBETTER, M. R. (1967). Stationary and Related Processes. Wiley, New York. [12] DANIELS, H. (1964). Saddlepoint approximations in statistics. Ann. Math. Statist. 3 631-650. [13] EPANECHNIKOV, V.A. (1969). Nonparametric estimates of a multivariate probability density. Theor. Probability Appl. 14 153-158. [14] FARREL, R. H. (1967). On the lack of a uniformly consistent sequence of estimators of a density function. Ann. Math. Statist. 38 471-475. [15] FRENKIEL, F. N. and KLEBANOFF, P. S. (1965). Two-dimensional probability distribution in a turbulent field. Phys. Fluids 8 2291-2293. [16] GARSIA, A. M., RODEMICH, E. and RUMSEY, H. Jr. (1970). A Veal variable lemma and the

continuity of paths of some Gaussian processes. Indiana Univ. Math. J. 20 565-578. [17] GRENANDER, U. and ROSENBLATT, M. (1957). Statistical Analysis of Stationary Time Series. Wiley, New York. [18] GRENANDER, U. and SZEGO, G. (1958). Toeplitz Forms and Their Applications. Univ. of California Press. [19] HASSELMAN, K., MUNK, W. and MACDONALD, G. (1963). Bispectra of ocean waves in Time Series Analysis, ed. M.Rosenblatt. Wiley, New York. [20] HAUBRICH, R A. (1965). Earth noise, 5 to 500 millicycles per second. / . Geophys. Res. 70 1415-1427. [21] HODGES, J. L. Jr. and LEHMANN, E. L. (1956). The efficiency of some nonparametric com petitors of the t-test. Ann. Math. Statist. 27 324-335. [22] HUBER, P. J., KLEINER, B., GASSER, T H . and DUMERMUTH, G. (1971). Statistical methods for

[23] [24] [25] [26] [27] [28] [29] [30] [31] [32] [33] [34]

investigating phase relations in stationary stochastic processes. To appear in the IEEE Trans. Audio andElectroacoustics. KOLMOGOROV, A. (1963). On the approximation of distributions of sums of independent summands by infinitely divisible distributions. Sankhya Ser. A 25 159-174. PARZEN, E. (1961). Mathematical considerations in the estimation of spectra. Technometrics 3167-190. PARZEN, E. (1962). On the estimation of a probability density and mode. Ann. Math. Statist. 331065-1076. ROSENBLATT, M. (1956). Remarks on some nonparametric estimates of a density function. Ann. Math. Statist. 27 832-835. ROSENBLATT, M. (1959). Statistical analysis of stochastic processes with stationary residuals in H. Cramer Volume, ed. U. Grenander. 246-275. ROSENBLATT, M. (1969). Conditional probability density and regression estimates, in Multi variate Analysis, ed. Krishnaiah. 25-31. ROSENBLATT, M. (1970). Density estimates and Markov sequences, in Nonparametric Techniques in Statistical Inference, ed. M. Puri. 199-210. ROUSSAS, G. (1969). Nonparametric estimation in Markov processes. Ann. Inst. Statist. Math. 21 73-87. SAZANOV, V. V. (1968). On the multidimensional central limit theorem. Sankhya Ser. A 30 181-204. SZEGO, G.(1959). Orthogonal Polynomials. American Mathematical Society, Providence. TUKEY, J. W. (1959). An introduction to the measurement of spectra, in H. Cramer Volume, ed. U. Grenander. 300-330. VAN ATTA, C. W. and CHEN, W. Y. (1968). Correlation measurements in grid turbulence using digital harmonic analysis. J. Fluid Mech. 34 497-515. 281

1842

MURRAY ROSENBLATT

[35] VAN NESS, J. (1966). Asymptotic normality of bispectral estimates. Ann. Math. Statist. 37 1257-1272. [36] VAN RYZIN, J. (1970). On a histogram method of density estimation. Technical Report No. 226 Dept. of Statistics, Univ. of Wisconsin. [37] WATSON, G. S. (1969). Density estimation by orthogonal series. Ann. Math. Statist. 40 1496-1498. [38] WOOHROOFE, M. (1967). On the maximum deviation of the sample density. Ann. Math. Statist. 3S 415-4S1. [39] YAGLOM, A. M. (1966). The influence of fluctuations in energy dissipation on the shape of turbulence characteristics in the inertial interval. Soviet Physics Dokl. 1126-29.

282

The Annals of Statistics 1973, Vol. 1, No. 6, 1071-1095

ON SOME GLOBAL MEASURES OF THE DEVIATIONS OF DENSITY FUNCTION ESTIMATES BY P. J. BICKEL 1 AND M. ROSENBLATT 2

University of California, Berkeley, University of California, San Diego We consider density estimates of the usual type generated by a weight function. Limt theorems are obtained for the maximum of the normalized deviation of the estimate from its expected value, and for quadratic norms of the same quantity. Using these results we study the behavior of tests of goodness-of-fit and confidence regions based on these statistics. In par ticular, we obtain a procedure which uniformly improves the chi-square goodness-of-fit test when the number of observations and cells is large and yet remains insensitive to the estimation of nuisance parameters. A new limit theorem for the maximum absolute value of a type of nonstationary Gaussian process is also proved.

1. Introduction. Let X19 X2, • • •, Xn be independent and identically distributed random variables with continuous density function f(x). By now there are a goodly number of papers on estimation of the density function (see [13] for a bibliography). The class of estimates fn(x) that we consider are determined by a bounded integrable weight function w,

(1.1)

Mx) =

-4rri:%i"/X~~Xi

nb(n) ^ J \ b(n) 1 dFn(s) . b(n) \ b(n)

In formula (1.1), Fn is the sample distribution function. Also b(n) is a bandwidth that tends to zero as n —> oo but is such that n~l — o(b(n)). The local properties of such estimates have been discussed extensively. Our object will be to get global measures of how good fn(x) is as an estimate of/(x). In particular, the asymptotic distribution of the functional and

m a x 0 ^ : \fn(x) -

f(x)\/(f(x)y

A*) are evaluated under appropriate conditions as n —» oo. Received November 1971; revised March 1973. This work was performed while the author was a fellow of the John Simon Guggenheim Memorial Foundation and supported in part by the Office of Naval Research under Contract N00014-69-A0200-1038. 2 This work was carried out while the author was a fellow of the John Simon Guggenheim Memorial Foundation and supported in part by the Office of Naval Research. Key words and phrases. Density function estimates, weight function, bandwidth, global meas ure of deviation, asymptotic distribution, functional of stochastic process, Gaussian process. 1

1071 R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_27, © Springer Science+Business Media, LLC 2011

283

1072

P . J. BICKEL AND M. ROSENBLATT

We shall state two results, one concerned with absolute deviation of the esti mate fn(x) from/(x) and the other with integrated quadratic deviation. They will give some insight into the type of result that is obtained. However, in order to give the result on absolute deviation it is convenient to introduce at this point certain convenient assumptions which we shall refer to as A l , A2, A3, and A4. A l . The weight function w also assigns mass one to the line and either (a) vanishes outside an interval [ — A, A] and is absolutely continuous on [ — A9 A] with derivative wf or (b) is absolutely continuous on ( — 00, 00) with derivative wf such that \ \w'(t)\k dt < 00, k = 1, 2. A2. The density / i s continuous, positive and bounded. A3. The function /* is absolutely continuous and its derivative \ff\f^ is bounded in absolute value. Moreover, Su^\^Uoglog\z\n\w'(z)\

+ \w(z)\]dz<

oo .

A4. The second derivative / " of / exists and is bounded. Moreover w is symmetric (about 0) and z2w(z) is integrable. We shall simply state a corollary of a main result on absolute deviations which is appealing because it is phrased in a form that is convenient if one wishes to set up a confidence band for the density function. COROLLARY.

Let assumptions Al—A4 be satisfied with b(n) = /r 5 , £ < d < £.

Then lirn n—*<x> __ P (1.2)

nb(ri) I \(251og«) i

^f(x)^fn(x) =

where

e-^z

AWW + lJ^pZL\ I ; T nb(n) I \ (2d log n)

T + <

for all O ^ x ^ l

and dn = (2*log„)» + -^—^V^i^T1)

+ Mlog<5 + log logn]I

if (a) of Al holds and

Kl(w) = ^ > +2"%=4/w

>0 ,

and otherwise

rf, = (25 1og«)*+

l

(251og«)

log.

1 (K2{w) it

with

K^) =

^\-A^mdtix(w).

The following result for a quadratic functional is also of some interest. The function a(x) used in the theorem is assumed to be a bounded piece-wise smooth integrable function. 284

MEASURES OF DENSITY FUNCTION ESTIMATES

1073

Let Al—A4 hold. Then if b(n) = o(n _ i ), Ar*(log n)*(log log A)* = o(b{ri)) as n —> oo, THEOREM.

b(n)-i[nb(n) \ [/„(*) - f(x)Ya(x) dx - \ f(x)a(x) dx \ w\z) dz] is asymptotically normally distributed with mean zero and variance 2

MS w(x + y)w(x)

dx

Y dy S a\x)P(x) dx

as n —> oo. The basic technique in obtaining the results is that of approximating the nor malized and centered sample distribution function by an appropriate Brownian motion process on a convenient probability space by using a Skorohod-like im bedding due to Brillinger and Breiman. The details'of this approximation and remarks on approximation of other functional are given in Section 2. The as ymptotic theory of the maximal deviation and that of quadratic deviations are developed in Sections 3 and 4 respectively. Some computations on the power of these procedures are also carried out. In particular, we show that a goodnessof-fit test based on a quadratic functional is strictly better than the x2 test. There is also an appendix on the asymptotic distribution of the maximal deviation for a type of nonstationary Gaussian process. 2. Approximations. As has been indicated in the introduction our technique is to consider the statistics of interest as functionals of certain stochastic processes on the interval [0, 1] and then to substitute Gaussian processes with the same covariance structure for the latter where possible. It is convenient to introduce Z n °(.) given by

(2.1)

zM

= »h(W)-t)>

o^t^i

where Fn* = ^ ( F " 1 ) is the empirical distribution of F(X1), • • •, F(Xn). This will be approximated by Z°(«), the Brownian bridge, given by (2.2)

Z\t)

= Z(t) -

tZ(\)

where Z is a standard Wiener process on [0, 1]. The process [nb(n)f~\ty\\fn(*) — E(f„(•))) is central to our discussion. It can be written as (2.3)

Yn(t) = b~\n)f-\t)

\~„ " ( l = r i ) dZn\F{s)) .

Approximations 0Yn and 1Yn to this process are obtained by substituting Z°(F(.)) and Z(F(-)) respectively for the random measure in (2.3). The resulting processes are well defined, at least if ^ w\t) dF(t) < oo. Two other processes which also arise naturally are given by (2.4)

,y.(r) = [b(n)f(t)]-i S "

fer^)

285

(/(*))* dZ(s)

1074

P. J. BICKEL AND M. ROSENBLATT

and (2-5)

3Yn(t)

= [b(n)]-i \ w (LZ-^j

dZ(s)

where Z is a two-sided Wiener process on ( — 00, 00) (dZ is Wiener measure). The process 3Yn is well defined if $ w2(t) dt < 00, and all the integrals with respect to dZ\F(.)), dZ(F(.)), dZ(.), dZ\.) are taken in the U sense (cf. Doob [6] page 426). For convenience, suppose all our processes are realized as random elements taking their values in the space D[0, 1] (cf. [3]). For xeD[0, 1] let ||JC|| = sup{|*(r)|: 0 <S t <: 1}. Our approximations rest on the following theorem of Brillinger (1969). (A similar argument appeared simultaneously in Breiman (1969).) There exists a probability space (Q, A, P) on which one can construct versions of Zn° and Z such that THEOREM.

(2.6)

\\z: - Z'H = 0,(/r*(log/i)*(log log/1)*) .

From this we can derive PROPOSITION

2.1. / / the processes Zn°, Z° are constructed as above and Al and

A2 hold, then (2.7) PROOF.

(2.8)

\\Yn -

o y„||

= 0,(6-i(«)/r*(log/!)»(loglog«)*) .

Write, using A l , Yn{q) = [b(n)f(q)]-l{-W(A)Zn°(F(q - Ab{n))) + w(-A)Zn\F(q + Ab(n)))} + b-Kn)f-*{q) $-_ Zn\F(s))w' ( l ^ l ) ds .

(The first two terms inside the curly brackets are taken to be 0 in the event A 1(b) holds but A 1(a) does not.) A similar representation is valid for 0Yn and (2.7) follows. PROPOSITION

2.2. / / A 2 holds then

(2.9)

||„y. - .y.11 = o,(bi(n)).

If A2 and A3 hold then (2.10) PROOF.

(2.11)

\\tY% - 3Yn\\ = Op(A»(n)) . From the representation (2.2), \0YM ~ iYM\ = \Z{\)\[b{n)f{q)V S»(?~')As)ds=\Z(\)\0{»(n)). 286

MEASURES OF DENSITY FUNCTION ESTIMATES

1075

Applying (2.8) and its analogues, if A 1(a) holds, \2Yn(q) -

3Yn(q)\

£ b-*{n){[\Z(Ab{n) + q)\\[f(Ab(n) + q)lf(q)f - 1| (2.12)

+ \Z(-Ab(n)

+ q)\\[f(-Ab(n)

+ q)lf{q)f - 1|] sup( \w{t)\

+ \ \Z(sb(n) + q)\\[f(q + sb{n))lf{q)f - \\\w'{s)\ ds + *(*(«)) S \Z{sb(n) + q)\\f'{q + sb{n))\[f(q)f(q + Sb(n))]-*\w(s)\ds} = 0,(bi(n)) by using A3 and the law of the iterated logarithm for the Wiener process. If A 1(b) holds the first two terms vanish and the same argument applies. To apply these propositions we make the elementary If {gn} is a sequence of functionals on D[0, 1] satisfying Lipschitz conditions such that REMARK.

(2.13)

\gn(X) - gn(y)\ £ Mn\\X - y\\

and An, Bn are stochastic processes realizable in,Z) such that \\An — Bn\\ = op(l/A/w), then gn(An) converges in law if and only if gn(Bn) does, and to the same limit. We shall apply this proposition in the next two sections to the functionals I

(2|log A(n)|)»

maxJ

1^.(01 . o < t < 11 - B(\b(ti)Y')

where B is defined in Theorem Al and, II

b-\n)[ J !!„ Yn\t)f(t)a{t) dt - \1„ w\t) dt]

where a is an integrable weight function. Evidently, since 1Yn and 2Yn have the same joint laws, we can substitute sYn for Yn in I if Al—A3 hold and (2.14)

0

(

l i

^

r

) = „-Mog»(logIog„)»

and 0Yn can be substituted for Yn in II if Al and A2 hold and, (2.15)

o(b{n)) = /r*(log/?)*(loglogrt)± .

Although we do not pursue this it is clear that the same technique can be applied to other functionals, e.g., a normalized version of the total time in [0, 1] spent by Yn above a high level (cf. Berman (1971) [2]). 3. The maximum absolute deviation. The first measure of global deviation that we consider is Mn = max {| Yn(t)\: 0 <^ t <^ 1}. (There is no loss in consider ing [0, 1] rather than any other interval on which the density is bounded away from 0 and oo.) The statistical interest of this functional is twofold as (i) A convenient way of getting a confidence band for / . (ii) A test statistic for the hypothesis H: f = fQ. 287

1076

P. J. BICKEL AND M. ROSENBLATT

Under (ii) we shall also consider the possibility of testing composite hypotheses, for example, t h a t / is Gaussian. The asymptotic theorem we need to discuss (i), and the behavior of (ii) under the null hypothesis is a consequence of our remarks in Section 2 and Theorem Al of the appendix. THEOREM

3.1. Let w satisfy assumptions Al—A3 and

o
b(n) = n- , Then, (3.1)

P "(25 l o g n ) * ( _ ^ _ - < / . ) < * "

where (3.2)

l(w) = $ w\t) dt

and (3-3)

dn = (25 log*)* + _ _ I ^ . { l o g . ^ 1 - tflog* + log log*]}

where Kl(w) =

^ ^ ( -

A

) / l

{ w )

,

ifK1(w)> 0, and otherwise dn

=

<25

where

8 ),+

'° "

K,(w) =

(2W

log

1 K2(w)7T

2

_

M[w'(t)]'dtyX(w).

1. The natural weight function w{t) = -J, \t\ ^ 1, = 0 otherwise, falls under the first case, while the "optimal" weight function of Epanechnikov (1969) w{t) = 3/(4(5)^(1 - (v2/5)) if \v\ ^ 5*, = 0 otherwise, falls under the second. REMARK

REMARK 2. A similar result holds if one considers the maximum deviation (rather than absolute deviation) of a density function estimate as in Rosenblatt (1971). However, since one-sided deviations for density functions are unnatural the present result seems more interesting.

3. The techniques of proof of this result may readily be adapted to prove limit theorems such as that of Woodroofe (1967) for the maximum deviation observed at an increasing finite number of points. REMARK

It follows from Propositions 2.1 and 2.2 and the following remark that the limiting behavior of {2d \ogn)*[{MJ{X{w)y) — dn] is the same as that of (2 logb(n))*(ma.x{\2Yn(t)\/(X(w))t: 0 ^ t ^ 1] - dn). By the similarity transform for the Wiener process, the law PROOF.

(3.4)

H9Yn(t):0£t£\)

= L(sw{.' 288

-

s}dZ(s):0£t£iy

MEASURES OF DENSITY FUNCTION ESTIMATES

1077

Since l/(Z(w))* $ w(t — s) dZ(s) is a stationary Gaussian process with mean 0 and covariance (3.5) r{t)^\w(s+t)w{s)ds^ X(w) Theorem 3.1 follows from Corollary A. 1 provided we show that r satisfies condi tion (v) and (vi) of Theorem Al with a = 1,2. That (v) is satisfied is equivalent to Theorem Bl. Moreover, \ r\t) dt = — $ |r(0| 2 dt =

(3.6)

\\w{t)\* dt

where " denotes Fourier transformation. Since w is integrable and bounded w is square integrable and bounded and (vi) must hold. (i) To obtain a confidence band for / t h a t is simple and ex plicit it is natural to consider 8 such that E(fn) can be replaced by / . This is true if 8 > £ and A4 holds. Then, APPLICATIONS,

(3.7)

J - $ w ( ~ W ) ds = /(0 + 0(*(n)) b{n)

\ b(n) /

with 0 independent of f. If we now define Fw* by replacing E(fn(t)) with /(f) we conclude that (3.8)

\\Yn-Yn*\\

=

0{[nb\n)f).

Using the usual approximations we conclude that max{|Fw*(r)(: 0 ^ t <; 1} be haves like M% if A4 holds and 8 > ^. In this case inverting as usual we obtain the confidence band

(3.9)

/ < / „ + M*-Y c(a)fl + _ ^ L V + J ^ L y

- ■'•

V «6(n) /

W

\

4n6(«)/. /

2nA(«)

where c(a) is given by (3.11). A simpler band is obtained if we further substitute fn f o r / i n the denominator of Yn. The resulting process Yn** (say) has (3 10)

/ ii v *ii2 \\Y * — Y **ll = O ( II " " II/ - -!

= c>

log"

if Al—A4 hold and ^ < 8 < ^. The approximate confidence band obtained by looking at the maximum of |y n **| is given in the introduction (1.2). There is no choice of 8 which asymptotically makes this simple band as thin as possible, i.e. one should choose 8 as small as possible. This of course ignores the obvious—the speed with which bias disappears asymptotically depends on 8 as does the speed of convergence to the asymptote. However, for fixed n there 289

1078

P. J. BICKEL AND M. ROSENBLATT

is an optimal 8(n) (depending on a) > 0 which for moderate n and small a may be the right thing to use if the choice of bandwidth is free. (ii) To test H: f — f0 it is natural to compute Mn with /" = f0 and reject for large values of the statistic. According to the theorem to obtain approximate level a we should use as cutoff point, c(a) = - [ l o g | l o g ( l - a)\ - log2] f^\w))\. + dn(k(w))i . (25 log tip Under some assumptions the same cutoff point may be used for testing composite hypotheses of the form H: / = / 0 ( . , 0) where 0 is an unknown vector parameter by using Mn with an estimate 0 substituted for the unknown parameter 0. We need the following assumption. A5. The estimate 0 is such that if 0 = 0O, for every 0O, (3.11)

(3.12)

sup{|$ [f0(t + sb(n), 6) - f0(t + sb(n)9 0o)]w(s) ds\: 0 ^ t^ =

and

\\fo(;Oo)-M'J)\\

=

1}

o,([nb(n)logb(n)]-i)

o,(\logb(n)n.

Typically for maximum likelihood and method of moments estimates (3.13)

I* - *o| = 0,(*-*) . a

If, moreover, 0 = (0 \ • • •, 0(k)), dfo/d0ij) is bounded for 0 in a neighborhood of 0O, all x, and 1 ^ y ^ A:, it is easy to see that A5 holds. To see that A5 is the needed assumption again introduce a process Yn with Ed(fn) replaced by £g(/tt) and (/(., 0)f replaced in the denominator of Yn by (/(., 0))K Then

(3.14)

( 1 ^ - ^11 = ^([log%)]-*)

and the result follows. To make local power calculations on the test of the simple hypothesis described above we need to consider the behavior of Mn (calculated under f0) for a sequence of alternatives of the form, (3.15)

gn(x)=Mx)

+

rnV(x) +

0(Tn)

where gn satisfy A2—A3 uniformly in n, j n j 0 at a suitable rate, and o(yn) is uniform in x on [0, 1]. (Note that r) must be continuous on [0, 1].) Denote probabilities calculated under gn by Pn. Our basic result is, THEOREM 3.2. Suppose that gn are as above. Let w satisfy Al—A3 and define Mn in terms off0. Let +8/2 [28logn]-t . Tn = n-l Then,

(3.16)

<

251O

S">'(T^ -"-) < *

exp [ — s(r])e-x}

where (3.17)

s(v) = Ji{exp[9(r)/(/-0(0^(M;))*] + exp[-37(/)/(/o(0^(M'))*]} dt 290

1079

MEASURES OF DENSITY FUNCTION ESTIMATES

This result follows from Theorem Al quite readily. One interesting consequence of this formula is that our test is asymptotically strictly unbiased for such alternatives. The reason is that sty) ^ 2 with sty) > 2 unless r] — 0 and the family of distributions e~de~x is an exponential family in 6. Unfortunately these tests are asymptotically inadmissible (have Pitman effi ciency 0) when compared to the test based on the quadratic functional of the next section based on the same w and b(n). The reason is that alternatives there may be permitted to come in t o / 0 at rate n~^+8/i rather than n~*+8/2. However, this test for moderate sample sizes and some alternatives may well be preferable. In analogy to the confidence band situation it would appear that maximum power is achieved by taking d as small as possible. However, consideration of the approximation arguments suggests that styn) is a better measure of the "true shift" than sty) where, (3.18)

Vn

= (gn -f0)(2nb(n)

log*(«))» .

Of course, styn) may well be maximized for d > 0. In all of these questions it would be desirable to have some small sample Monte Carlo explorations. 4. Quadratic functionals. We are interested in the behavior of the functional, (4.1)

Tn = nb(n) $rTO [fn(x) - E(fn(x))fa(x)

dx = J r . Ln\x)a{x) dx ,

where Ln = pYn and a is integrable. We have already remarked that if Al and A2 hold and (say) bn = n~8, d < J, then. (4.2)

\Tn - J QLn\x)a(x) dx\ = o{b\n)) .

Moreover, if a is bounded as well as integrable and w and / a r e bounded, we can replace 0Ln by xLn =p1Yn and hence by 2LM z=fiiYn. To see this note that,

\SW(x)-0Ln\x))a(x)dx\

=

h4K

(4.3)

/(5)

(w) ^)

2

- 2Z(1) $ w ( . L z l ) dZ(F(s)) ] w ( - L z l ) f(s) ds} a(t) dt\ £Z(\yb(n)

sup.

|.f{x)\S\a{t)dt\

+ 2|Z(1)|6(«)| \ (J w{y)c(s + b(n)y)a(s + b(ri)y) dy) dZ{F{s))\

where

But, (4.4)

Z(1Mw

E{\ ($ w(y)c(s + b(n)y)a(s + b(n)y) dy) dZ{F{s))f

= S (S "(y)c(s + H«)y)a(s + bin)y) dyf is bounded. By (4.3) and (4.4), (4.5)

\Tn - S Mx)a(x)

dx\ = Op(b(n)) . 291

dF s

()

1080

P. J. BICKEL AND M. ROSENBLATT

(The infinite range poses no problem since we are approximating Ln rather than the normalized Yn.) The following lemma lets us determine the characteristic function of a quad ratic functional (4.6)

Z = J Y{xfa(x) dx

of a Gaussian process Y(x) under appropriate conditions. 4.1. Let Y(x), EY(x) = 0, be a Gaussian process with bounded, uniformly continuous covariance function r(x, y). Ifa(x) is a piecewise smooth inferrable func tion, the quadratic functional (4.6) has characteristic function formally given by LEMMA

E{e™) = exp {2ZU 2k~\it)kck/k}

(4.7) with

ck = I • • • 5 r(x19 x2)r(x2, x3) • • • r(xk, x1)^(x1)^(x2) • • • a(xk) dx, • • • dxk . The representation (4.6) is valid for \t\ < 1/2M where M = ||r|| $ |a(r)| rff. The quantities (k — 1)! 2A;_1c^ are of course the cumulants of (4.6). The lemma is obtained by considering the form (4.8)

X%iVfr

in jointly Gaussian random variables Yp EYj = 0 with the afs constants. Let R be the covariance matrix of the Y / s with A the diagonal matrix with diagonal entries a.. The characteristic function of (4.8) is then |1 - 2itRA\~l = UU(l

~ 2V0~* = exV{Zk=ilk-1(it)kH(M)k/k}

,

at least if \t\ < l/2tr(RA). Here tr (M) denotes the trace of M, \M\ its determinant and X19 • • •, Xn are the eigenvalues of RA. Lemma 4.1 is then obtained by going through an appropriate limiting operation. The covariance function of the Gaussian process 2Ln(x) can be written (4.9) where

r(x, y) = $ w(z)w(a + z)f(x - b{ri)z) dz = f(x) 5 w(*M« + z)dz + 0(b(n)) a = (y-

x)j(b{n))

and 0(b(n)) is independent of JC if / is bounded and has a uniformly bounded derivative and w2(z)(l + |z() is integrable. Then (4.10)

E{\ 2Ln{xfa(x) dx) = 5 /(*)*(*) dx J w(z)2
Similarly if a is bounded as well as integrable and w is bounded a n d / i s as above, the variance of $ 2Ln\x)a{x) dx is 2b(n) $ [w * w(u)f du $ a\x)p{x) dx to first order as n —> oo, where n>(f) = w( — r) and * denotes convolution. A similar argument shows that under the same conditions the kth cumulant of $ 2Ln\x)a(x) dx equals to first order (k — 1)! 2k-1bk~1(n)[w * w]ik)(0) $ ak(x)fk(x) dx as n -> oo where the 292

MEASURES OF DENSITY FUNCTION ESTIMATES

1081

superscript (k) indicates that w * w is convoluted with itself & times. As a result we have the following theorem which actually holds under the weaker assump tions indicated above. THEOREM 4.1. Let Al—A3 hold and suppose that a is integrable piecewise continuous and bounded. Suppose moreover that (2.16) holds. Then b~^(n)(Tn — ($ f(x)a(x) dx) \ w2(z) dz) is asymptotically normally distributed with mean 0 and variance 2(w * w)(2)(0) \ a\x)f2(x) dx as n —> oo.

A particular case of interest for the application of the theorem is that in which as in Section 3, a(x) vanishes off an interval, say [0, 1], and one sets a(x) = /(x)'1 on [0, 1]. In this case under Al—A3, Tn is asymptotically Gaussian with mean $ w2(z) dz and variance 2b(n)(w * H>)(2)(0). The statistic (4.11)

fn = nb{n) \ [/„(*)

-f(x)fa(x)dx

is probably of greater interest than that considered in Theorem 4.1. However, let us expand fn in the form nb(n){S[fn(x)-Efn(x)fa(x)dx (4.12)

+ 2 $ [/.(*) - Efn(x)][Efn(x) - f(x)]a(x) dx +

\[EUx)-f{x)Ya(x)dx}.

Let w be positive and symmetric about zero with c = $ w(u)u2du < 00 .

(4.13)

Then if n~l = 0(b(n)), b(n) —> 0 as n —> 00, Al holds and / has a continuous bounded second derivative, the second term of (4.12) may, by the usual ap proximation arguments, be shown to be asymptotically normal with mean zero and variance n-lb(nfc2 \ f"(x)*a{x)*f(x) dx

(4.14) to the first order. be shown to be

Also, under the same conditions, the last term of (4.12) can b(nfc2\

(4.15)

f"{xfa(x)dx

to the first order. Then [b(n)]-*[Tn - Tn] = op(\) if and only if b(n) = o(/r*). (The term (4.14) is then negligible.) The theorem quoted in the introduction follows. An explicit confidence band is hard to obtain from Theorem 4.1 and the theorem of the introduction. However we can test H: f = f0 at (approximate) level a by calculating Tn f o r / = fQ and rejecting when Tn ^> d(a) where by Theorem 4.1 APPLICATIONS.

(4.16)

d(a) = [ 5 f0(x)a(x) dx][ J w\z) dz] + b\n)<&-\\ - a)/[2(w * vt>)(2)(0) $ a2(x)f0\x) dx]± . 293

1082

P. J. BICKEL AND M. ROSENBLATT

As in Section 3 it is easy to see that in testing H: / = / 0 ( . , 0) where 0 is an unknown vector parameter we may use Tn with / replaced by f0(., 0) and d(a) with/ 0 replaced b y / 0 ( . , 0), provided that A6 below holds. A6. For each 0O9 (d2f(x, 0)jd0{i)d0^)) is bounded in absolute value for all 0 in a neighborhood of 0O and all x, i, j . Moreover, if 0O is true, (4.17)

\0 - 0O\ = op([nb(n)]-i) .

To see this, taking k — 1 for simplicity, expand as in (4.12) and note that it suffices to show that (4.18)

J [/„(*) - E0o(fn(x))][Effo(fn(x))

- Ei(fn(x))]a(x) dx = op([nb*(n)]-*)

and (4.19)

$ [E0Q(fn(x)) - Ej(fn(x))Ya(x) dx = o,([nb*(n)F) .

Taylor expanding the integral in (4.18) about 0O we obtain a first term

(0 - 0O) $ [AW - £JAM)] 5

df(x + b(n)z, 0)

w(z) dz a(x) dx

which is Op(\0 — 0o\n~^)y and a second term which is Op([nb(n)]-^(0 — #0)2), and (4.18) follows. A similar argument yields (4.19). To make local power calculations we again suppose gn is as in (3.15) with gn satisfying A2—A3 uniformly in n and o(jn) uniform in x and rj is bounded. 4.2. Let gn be as above, w satisfy Al—A4, a be integrable piecewise continuous and bounded, b(n) = n~d, d < J, j n = n~^+d/i. Define Tn in termsoffQ. Then, THEOREM

(4.20)

b-\n){Tn - [ J fQ(x)a(x) dx] J w\z) dz)

is asymptotically normally distributed with mean \ r]2(x)a(x) dx and variance 2(w * wy2)(0) l a\x)f\x)

dx .

The proof is straightforward. As in Section 3 it follows that the test which rejects when Tn is ^> d(a) is locally strictly unbiased if a(x) > 0 for all x. Also as before the asymptotics lead to choosing 8 as large as possible and again this conclusion is shaken if one uses the better approximation to the asymptotic mean, J r]n2(x)a(x) dx where (4.21)

Vn(x)

= J w(z)[gn(x + b(n)z) - f0(x + b(n)z)] dz .

It is also clear that for fixed 8 we can let Xn —> 0 more quickly than for the sup functional and still get power. Thus the Pitman efficiency of the Tn test to the Mn test for the same 8 is oo. Suppose that/ 0 is the uniform density on [0, 1 ] an effect we can always achieve by applying the probability integral transformation to our observations before making the test. Let a(x) = 1 on [0, 1 ] and 0 otherwise, w be the uniform density 294

1083

MEASURES OF DENSITY FUNCTION ESTIMATES

on [ — 1, £]. Neglecting fringe effects we may then write

(4.22)

T% = Si W

~ ^n)'

< + ^(n)] ~

b{ ))2

"

dt

where N[x, y] is the number of observations falling in the interval [x, y]. A related statistic for testing uniformity on the circle was considered by Watson in [15]. This is, of course, very similar to the f statistic for the problem based on the cells [0, b(n)]9 [b(n)9 2b(n)]9 . - . , [ ( * - 1 )£*(*), (K + l)&(n)] given by, (4.23)

Zw

- 2,*.i

^

where (K + \)%b(n) ^ 1 < (K + 2)%b(n) and £ * Now we can write,

(4.24)

t:\K

i s a s u m o v e r o d d in

dex.

= nb(n) JJ (/.(f) - £(A(0))2 ^ . ( 0

where y4% places mass l/K at each of the points £A(/i), • • •, K±b(n). It is easy to see that the arguments leading to Theorem 4.2 apply to functional of this type also and that under the conditions of that theorem, if b(n) = rr\ 8 < £, XnlK is asymptotically normal with the natural parameters E(xn2/K) and Var (xn2/K)' This result is, of course, known. A rigorous proof under milder conditions but using a different method may be found in Steck (1957). Now (4.25)

E(ljp)

= 1 + 1 2 # **(") (* " ^ r -

= l + /i^r.'ls;* (4.26) K }

1

\m\l\l\%Qn(x)dx

C(i + l)6(n)/2 „(x\

(jx

■W

V a r t e ) = 1 Var ( N ^ W) + o(l) \K/ K \ nb(n) I \KI

= A K

+

~* + o{nb(n)Tn%) 0 (i

\K,

Thus if we take j n = n~^+3/4 as in Theorem 4.2, under gn the statistics (4.27)

Wn = b^{n)[^-

l)

have a limiting Gaussian distribution with mean \ J rj\x) dx and variance 2. Under the same circumstances the asymptotic mean of b~\n)(Tn — 1) with Tn given by (4.22) is also \\ rf(x) dx while its asymptotic variance is, (4.28)

2w^(0) = 2 $Lt (1 - M)1 A = f -

The Pitman efficiency of the tests based on Tn to those based on Wn is thus by the usual calculations, (4.29)

e(Tn, Wn) = (f)*-'

and thus at least (§)* = 1.217 on the range 8 > 0. For the Mann-Wald (1942) prescription 8 — \ we get an efficiency of 1.292. 295

1084

P. J. BICKEL AND M. ROSENBLATT

Although as we have seen these asymptotic calculations are to be taken with a grain of salt we feel that the procedure Tn has promise as a competitor to the X2 test, at least for moderate sample sizes. Acknowledgment. We are grateful to S. Berman and J. Pickands, III for providing us with preprints of some of the papers cited in the list of references. 5. Appendix A. On the extrema of some nonstationary Gaussian processes. Let YT(*) be a sequence of separable Gaussian processes with mean ftT(*) such that YT(.) — fiT(*) is stationary. Let r ( . ) be the covariance function of YT, MT = m a x { F r ( r ) : 0 ^ t ^ T] ,

mT = min{Y T (t): 0 ^ t ^

T}.

Let bT(t) = fiT(t)(2 log T)K THEOREM

A l . Suppose that,

(i) bT(t) is uniformly bounded in t and T on [0, T] as T —> oo. (ii) bT(t) —> b(t) uniformly on [0, T] as T —> oo. (iii) r _ 1 ^[f: b(t) ^ x, 0 <^ t ^ T] —> r](x) the cdf of a probability T —> oo. (2 as usual denotes Lebesgue measure.) (iv) £(•) is uniformly continuous on R. (v) r(t) = 1 - C\t\a + o(\t\a), 0 < a ^ 2, as t -> oo. (vi) \~r\t)dt< oo.

measure as

Let

Bit) = (2 log tf + — i X { ( — - i ) l o g l o g r + log(27r)-i(C 1 /«// a 2^-« ) / 2 «)J where Ha = l i m r _ 1 . J~ e s P [ s u p 0 ^ r F(r) > j] ds and Y is a Gaussian process with, (A.l)

E(Y(t)) = -\t\°,

C o v ( r ( r 1 ) , F(r2)) = W" + N * - K -

UT = (2 log r ) * ( M r - B(T))

and

VT = - ( 2 log r)*(m r + * ( r »

are asymptotically independent with, (A.l)

(A.3)

P[UT < z] ->
h=S*z

drj(z) ,

,

f,|a.

P[ F r < z] ->
^2 = \ e~* drj{z) .

An immediate consequence of Theorem A l is, 296

;

MEASURES OF DENSITY FUNCTION ESTIMATES COROLLARY

1085

A l . If MT = ma.x{\YT{f)\: 0 rg t ^ T] then under the conditions

of the theorem, (A.4)

P[(2 log Tf{MT - B(T)) < x] - exp [ - (^ + A2)e~*] .

Note. lx + 12 ;> 2 with strict inequality unless 57 concentrates at 0. COROLLARY A2. Le/ F0(/) — fi(f) be a stationary mean 0 Gaussian process with covariance function r(t) satisfying the conditions of the theorem. Suppose that b(t) = (2 log(r -f 2)hfJt(t) is a bounded uniformly continuous function of t and that b(*) satisfies condition (iii) of the theorem. Then,

(A.5)

P[(2 log r)*(max {Y0(s): 0 ^ s ^ T} - B(T)) < x] -* e~^~x .

Similar assertions hold about the independence of maximum and minimum and the asymptotic distribution of the minimum. This corollary may be viewed as complementing Theorem 4.1 of Quails and Watanabe (1971) which deals with the extrema of a mean 0 process whose covariance function is asymptotically locally approximated by that of a station ary process while we deal with a process'which is stationary when centered and asymptotically stationary. The constants Hx and H2 are the only ones known explicitly. They are given by H,= 1, i/2 = 7r-i(cf. [11]). PROOF OF COROLLARY

(A.6)

YT(t) = Y0(t)

A2. Define, on

[e(T)9 T]

= Y0(t) + ([log(r + 2)/log(r + 2)]* - l)^(r)

otherwise

where e(T) = o(T), \oge(T) ~ log T. Evidently, (2 log TfE(YT(t)) formly and max {Yr(s): 0 ^ s ^ T} < (A.7)

*

max{y 0 (.): 0 ^ s£

T} <

~> b(t) uni

+ B(T) X

+ B{T)

(2 log Tf

< IP max {Y0(s) - E{ Y0(s)): 0 £ s£ e(T)} ^

x

(2 log 7)*

- K + B(T)~\ where K = max {ju(t): 0 <; r <: <:(?)}. Since 5(e(r)) - fi(r) ->• - oo the term on the right of (A.7) tends to 0 by the theorem. Q PROOF OF THEOREM A l . The theorem is argued much as Theorem 3.1 of Pickands (1969). We refer the reader to this paper and Berman (1971) for the details of the argument.

297

1086

P. J. BICKEL AND M. ROSENBLATT

A l . Let is the standard normal density. Let C = I, x = x(T) = B(T) + z,/(2 log T)K Then for a > 0, LEMMA

P[m&x{YT(t + akx~2a), 0 ^ k ^ n] > x] ( A -8)

= (x)) P[min{YT(t + kax~2a), 0 ^ k ^ «} < - x ] = ^ ( x ) e - 6 ' " / / > , a ) + 0 (^(x))

05 7 —> oo uniformly in 0 ^ ( ^ T w/rere (A.9)

//„(«, o) = J ^ e*/>[max {Y{ka): 0 ^ k ^ n} > s]ds .

Moreover, if y = y(T) = B(T) + z2/(2 log Tf then /^max{y r (r + kax~v"): 0 ^ k ^ «} > x, *

(A. 10)

min {r r (? -f kax-%a): 0 ^ it ^ «} < - / ] = 0 (^(x)) = o ( ^ ) ) , uniformly in 0 ^ t <^ T. (Throughout, k may take on integer values only.) PROOF.

AS

in [11] consider the "local" process YT(s) = x(YT(t + sx-2'a) - ftT(t) - x) .

(A. 11) (A. 12)

P[max{YT(t + akx~2 "): 0 ^ k ^ «} < x] = 5r«,rW[max{f 7 ,(te): 0 ^ k ^ «} > -xftT(t)

\ Yr(0) = z]rfz

where ?■ is the density of Yr(0), (A. 13)

r (z)

= l-4>(x + ±) X

\

XI

=
It is easy to see using (ii) and (iv) that the finite dimensional conditional distri butions of YT{s) given ¥r(0) = z converge uniformly in t to those of the process Y(s) + z where Y is given by (A. 1). Arguing as in [11] the first part of (A.8) follows since x/uT(t) —> b(t) uniformly as required. By considering — YT we obtain the second part. To prove (A. 10) let A be the event whose probability is being estimated. Then,

P(A,

rr(o > * - 1 + MO) ^ S-,4 r(z)P[min{fT(ka):

0 ^ k ^ «} - z

^ - z - x(y + x + /iT(f)) | ?T(0) = z] rfz (A. 14)

^ ^(x) J i t e2/>[min{fr(A:a) + z : 0 ^ k ^ «} < z - x(j + x + ^(f)) I ? r (0) = - z ]
MEASURES OF DENSITY FUNCTION ESTIMATES

1087

Applying the usual estimate O(z) <^
P(A, YT(t) - pT(t) < -y + - I ) = o(
Finally, P(A> -y + \ (A. 16)

< yT(t) - MO ^ x - 1 )

^ \z*J j{z)P[mdix{YT{ka):

0 ^ k ^ n) > -x/i r (?) | F(0) = z] rfz

z

^ ^(*) \~e P[m2ix{YT(ka):

0 ^ k <: n} > z]dz

for every ,4 < oo. The final statement of the lemma follows. A2. The assertion of Lemma Al remains valid if a — 1, k is permitted to range over all values in [0, n] and Ha(n, a) is replaced by LEMMA

(A. 17)

Ha(n) = • ^00etP[m3x{Y(s):

0 ^ s ^ n} > t]dt

We prove the analogue of (A.8); the other assertions follow similarly. We need to check that uniformly in 7\ PROOF.

(a) The conditional distributions of the continuous processes YT(t) — z given YT(0) = z converge weakly (in the sense of Prohorov) to that of Y(*)9 (b)

P[max {YT(k): 0 ^ k ^ n) > x/iT(t) | YT(0) = z] ^ g(z)

where $ e~zg(z) dz < oo. To see that (a) holds it suffices to note that, (A. 18)

Var[(? r (j 1 ) - YT(s2)) | ?T(0) = z] £ C\ s, - s2\«

and then apply Billingsley [3] page 95. To see that (b) is valid use the estimate of Fernique (1970) given below on the tails of max {| JV(&)| : 0 ^ k ^ n}. Let Z(*) be a Gaussian process on (0, 1). Let a be such that F[\\Z\\
F[||Z||>z]^exp|-^log3}. A3. Fix t > 0 such that inf{*-*(l - r(s))\ 0 ^ s ^ t] ^ A(t) > 0. Define x and y as before. Let, LEMMA

(A. 19) (A.20)

Ha(a)

0
limn^f{^l. n

=

= lim_ 0 ^ M = l i m _ ^ >

(See the note at the end of the lemma.) 299

.

1088

P. J. BICKEL AND M. ROSENBLATT

Then, max \ YT(v + kax~Va): 0 ^ k ^

(A.21)

= xv"if)(x) H^L a (A.22)

r2/a

La

J

> x

\i+* expb(s) ds + o(x2/«(P(x)) ,

F[max{YT(v + s): 0 ^ s ^ t} > x] = Jc2/^(;c)[5:+i Qxpb(s) ds]Ha + o(x2/«
uniformly inO < v ^ T. Similar assertions hold for P[m'm{YT(v + s): 0 ^ 5 <^ /} < — x] w/7/z — b replacing b. Finally, (A.23)

P[msix{Yr(v + s): 0 ^ 5 ^ r} > x, min{r 7 ,(v %+ 5): 0 ^ 5 <; /} < - j / ] = o(x 2 / ^(x)) .

Note. The correct proof municated to provide yet a

existence of the limit in (A.19) was first proved in [11]. An in of (A.20) was also given. Subsequently, a correct proof was com the author by J. Pickands and another is included in [12]. We third in Appendix B.

We prove (A.22); (A.21) is argued similarly. Begin by bounding the left-hand side of (A.22) from above by, PROOF.

(A.24)

2 X 0 P[mzx{YT(v + knx~2/a + s): 0 ^ s ^ nx~2/a} > x]

where M = [txv"/n]. (A.25)

tffa(n)

By Lemma A2 the expression above is asymptotic to x2/"(p(x)

1

.M + 1

2f = 0 exp6(v + knx-v*)

= **.«("). xVn
2?=*o P(Ar) - Z»^Si,a

P(ArAs)

2/

where Ar = [{max{YT(v + kax~' "), rn ^ k < (r + \)n] > x], Ma = [x-v"tjna]. If we apply Lemma A.l to the first term on the right of (A.26) we obtain that, (A.27)

Sf=% P(AT) ~

Finally, (A.28)

H na

A\\ na

xv"(f>(x)[\l+t
P{ArAs) ^ P{CrCs)

where (A.29)

max {Y,,.(kax-'2/" + v) — fiT(kax-v"): > x

K (2\ogT)K 300

rn <; k < (r + 1)«}

1089

MEASURES OF DENSITY FUNCTION ESTIMATES

where K = sup{(21og7')*|/£7.(f)|: 0 ^ t ^ T}. Now applying Lemma 2.3 of [11] and arguing as in Lemma 2.5 of the same paper we see that, (A.30)

Z P(CrCa) = o{Xv«
Applying (A.20) we see that (A.22) follows. To prove (A.23) it suffices to show that, P{[max{YT(v + s): 0 ^ s ^ /} > x] (A.31)

u [min{YT(v + s): 0 ^ s ^ t} < =

-y]}

x^'(x)Hn^[cxpb(S)]^ + f'"(x)) .

But we can bound the expression on the left of (A.31) from above by /'[max {YT(v + s): 0 ^ j ^ t] > x] + P[min {Yr(v + s): 0 ^ s ^ r) < - 7 ] and from below as in (A.26) where we add A„a+1, ■■-, A2„a+1 with AMa+j [min {Y,,,(v + kax~v"): (j - \)n ^ A: < _/«}} < - y}. Now by (A. 10) (A-32)

\

S & . P(AsAMa+i+1)

=

= o{x^
Finally, again arguing as for the previous case, (A.33)

i-Sosj**^.^^^),

\l>^*kiNa+iP{Aj+MaAk+Ma)

--Zf>n*kiKaP(A}A„^k+l)

are all

and o(x 2 ^(x)) . Q

The rest of the proof goes much as in Berman [1]. Neglecting fringe effects break the interval [0, T] up into 2/V intervals of which half, Wx, ■ ■ ■, WN are of length rand the others Vx, • • •, VN of length e so that Vi follows Wx which follows Vt-i, i = 2, • • •, N. Of course, A' ~ T/(t + s). Define x and j as in Lemma Al and note that, (A.34)

y?"*(x)Ha ~ _L e-'i .

Then, by Lemma A3, P[max{Yr(r): (A.35)

r e U?=i F ;} ^ *] ^ E7=i ^ m a x {^(r): r S F . ) ^ *] -[E^S^exp^)^]^

1

= ^ ( y ) = ^(l) where the O term is independent of e and the Vr A similar assertion holds for min{K 7 ,(r): r e (Jf=1 V)} and hence we need only show that, (A.36)

lim,^0 fimr-0O P[max{Yr(T): r e \JU w3\ ^ *. min {Yr(r): r e [JU W ,\ ^ - y]

= exp - { V * 1 + V * } , 301

1090

P. J. BICKEL AND M. ROSENBLATT

where ^ , 1, are defined in (A.3) and the bars above and below the limit sign indicate lim sup and lim inf respectively. Next choose a > 0. If W5 = [ap ai + t), j=l,...,N. P[max{r 7 .(r): r e Uy=i WJ) ^

x

]

Yr(aj + kax~v") ^ x: 0 ^ k ^ (A.37)

£ZU

P[max{YT(r):

ixVa-

,l^j^N

re Ws} £ x]

max i YT(aj + kax~v"): 0 £ k ^

-[ZUW^xpWWW*)

H,. -

_ a _

"M

[ S ,j

e~*i,

by Lemma A3. A similar argument holds for F[min { ^ ( r ) : r e JJJU W)} ^> —y] and by simple probability manipulations it follows that to prove the theorem we need only show, -y ^ YT(aj + kax~Va) ^ x: 1

lima^0 lim^n \m\T P (A.38)

0 ^ /c ^

^j^N,

' txVn' L

a J_

= exp{-[V-*i + V * 2 ] } Now in view of Lemma A3 it is easy to show that, (A.39)

iim r ZU

V(X (I - P -y ^ YT(aj + kax~ ) <, x:0 ^ k ^

tx2

= ^ lim-' L ^ J l r . {exp[6(5) - z,] + exp -[b{s) + z2]}
lim^0 lim_0 lim7, EU{1

< x: 0 < k <

P Ks) ds\ — °(e)

-y ^ YT(a. + kax~v«)

p

~

ex

(xv«

a

JJ/

Let Ej, j = 1, • • •, N be the events whose probabilities are being summed in (A.40). The assertion (A.38) corresponds to a limiting statement about P(E1- • -EN). If the E- were independent assertion (A.38) would follow readily from (A.40). Let P be the measure which makes the vectors (Y^a^, YT{ax + ax-v«), • • ., YT(ax + ax-v«[tx^/a])), (Yr(a%)9 • • -, YT(a2 + ax^n[t^a/a]))9 • • -, (YT(aN), • • •, YT(aN + ax~Va[tx2/aja]) independent and otherwise agrees with P. 302

MEASURES OF DENSITY FUNCTION ESTIMATES

1091

To conclude the proof of the theorem we need to show that. (A.41)

l i m ^ l i m r \(P - P ) ( ^ . . . ^ ) | = 0 .

To do this apply the following modification of Lemma 4.1 of [1]. LEMMA A4.

Let

Let Zj = \ru\, 2 2 = |^ y | be k x k nonnegative semi-definite matrices with ru = su= I for all i. Let X = (X19 • • •, Xk) be a mean 0 Gaussian vector with covariance matrix I>1 or E2. Let u19 • • •, uk be nonnegative numbers and u = miny u5. Then, (A.43)

\PZi[X3. ^ i/., 1 rg / ^ k] - P^Xi

^ up ikj^

k]\

By the usual argument (see [1] page 931) the left-hand side of (A.43) is bounded by, 4 £*,; |$;;j (ui> UP *) d*V B u t ' bY a n elementary inequality PROOF.

(A.44)

x2 - 2pxy + f ^

(1

~P)

(x + yf .

Thus, (A.45)

f(ui9 up X) (^L+ll

, ?L+JU , j ) rg <J(a, «, X) .

Take X, = YT(aJ - t*T{ax), X2 = -YT(ax) + ^(a,) etc., k = 2N[tx2/«/a], \riS\ corresponding to the distribution of X under F, | ^ | corresponding to P, ux — x - fjiT(a^ u2=y + fxT(a^ etc. Evidently, (A.46)

u = (2 log Tf + 0((log 7*)"*) .

It is clear now that we can apply to the bound of (A.43) exactly the same analysis as that given by Berman on pages 933-936 of [1] to arrive at the conclusion of the theorem. Note. By applying the more refined analysis of Pickands [11] pages 64-72 we can show that the conclusion of the theorem also holds if (vi) is replaced by, (A.47)

l i m ^ r{t) log t = 0 .

Unfortunately, the analysis of Berman appears to only yield the conclusion under the stronger (A.48)

r(t)[\og '] a / a -> ° •

We do not enter into this further since (vi) is what we need for Theorems 1.1 and 1.2. 5. Appendix B. Miscellanea. THEOREM

Bl. Let w be an absolutely continuous square integrable function with 303

Q

1092

P . J. BICKEL AND M. ROSENBLATT

a square inferrable derivative w'.

Let,

(B.l)

r(t) = 5 w(t 4- s)w(s)ds .

Then r is twice differentiate

and r"(t) = - \ wr(t + s)w'(s) ds .

(B.2) PROOF.

W e first show that

(B.3)

r'{t) = I w\t + s)w(s) ds = $ w(s - t)w'(s) ds .

Let H>, vi>' be the Fourier transforms of w, wf. Then by Parseval, r (

( + h)~ *> = 1

(B.4)

5

^ ( t + " * - g"f'") |*(„)|» du .

h 2TT h Applying the dominated convergence theorem we obtain t h e existence of rf given by fit) = — — \ e-ltuu\w(u)\2 du = I w'(t + s)w(s) ds , 2K

Similarly r'i* + h)~ h

r

V)

=

c Hfo - r - A) - w(5 - t) w,(s) 3 w h /

(B.5)

/s>i(t-\-h)u

= A J (?

ds

s,itu\

^—^—) *|tf(")|a ^

_> _ _ L J ^ W W 2 |H>(W)| 2

£/I/

2?r

= - J

W'(J

- t)w'(s) ds .

The theorem follows. Note that r'(O) = 0 from (B4) since \w\ is symmetric. B2. Let w be absolutely continuous on [ — A , A] andO otherwise. r has left and right derivatives at 0 and THEOREM

(B.6) PROOF.

r + '(0) = -r_'(0)

= -^(w\A)

+ w\-A))

Then

.

Write, for h > 0, w(s

+ h)_

w(s)

ds

\s+h w'(z)dz = JlxM — w _ h u

(B.7)

J1 A

L # W'(J)W(J)

J

w (s)

ds - — \AA_h w\s) ds h#

by arguing as in Theorem A l and using Lebesgue's theorem. Since r{ — t) = r(7) the result follows. B3. (Pickands) / / Ha(n, a), Ha(n) are defined as in (A. 17), (A. 19) then (A.20) holds. THEOREM

PROOF.

(B8)

Suppose first that 0 < a < 2. Let for j > 0, #„(/!, r) = S-oo e s [ m a x 0 ^ w 7(f) > s + T] ds = * - ' # a ( / i ) . 304

MEASURES OF DENSITY FUNCTION ESTIMATES

1093

Then — \Ha(n, a) - Ha(na, y)\ n ^ — [S-~ e°P[max0^na n (B.9)

Y(t) > s + r, max0SjfcSm Y(ka) ^ s] ds

+ $!!„ e°P[s < max 0StSlia Y{t) ^ s + r] ds] S — ZtZl S-oo e°P[Y(ka) ^ s, mzxka^(lc+1)a n

Y(t) > s + r) ds

+ 1 [Ha(na) - Ha(na, r)] . n If the summands on the right of the first term of (B.9) are denoted by A(k, j , a) then, (B.10)

A(k, r, a) = $!!«*• $!„, T(Z9 ka) X P[max0^a

Y(t + ka) > 5 + j \ Y(ka) = z] dz ds

where T(Z, ka) is the density of Y(ka). After some manipulation we obtain (B.ll)

A(k,r,a)

= l-<,H^)^esP[m^x0^a(Y(t — w + (ka)a] ds dw .

+ ka)^Y(ka))>s

+

r\Y(ka)

As k —» 00, the finite dimensional conditional distributions of Y(t -f ka) — Y(ka) given Y(ka) — w + (&#)* tend for each w to those of Y(t), 0 <, t <^ a. Arguing as in Lemma Al we conclude that, (B.12)

lim, A(k, r, a) = A(y, a) = $~ esP[mzx0^a

Let Y*(t) = y(r) + |f|a. (B.13)

Y{t) > s +

r]ds.

Then,

^(r, «) ^ $0°° ^ [ m a x 0 ^ a Y*(t) > s + r]ds = $~ ^ F [ m a x 0 ^ ^ y*(r) > ( * + r )a"« /2 ] w]dw .

Applying Fernique's estimate the right-hand side of (B.13) is 0(exp — a~a/2) for every y > 0. We conclude that, (B. 14)

lim sup a lim sup% — \Ha(n, a) - Ha(na, r)\ na <; (1 — e~r) lim supa lim supw —a^na' na

for every y > 0. Since, P[mzx0^n (B.15)

Y{t) > s] ^ 2J=ox P[mzxk^k+1

Y(t) > s]

^ Zt=l {P[Y(k) £ s, max^«* + 1 Y{t) >s] + P[Y(k) > s]}, 305

1094

P. J. BICKEL AND M. ROSENBLATT

it is easy to see that, (B.16)

s u p ^ 1 ^ M < co. x

Hence, (B.17)

lima lim supn i - \Ha(n9 a) - Ha(na)\ = 0 . na But from the argument of Lemma A3 it is clear that for every a > 0, lim supn H*(n> g> ^ lim inf. R^nd) na na

(B.18)

.

The theorem follows for 0 < a < 2. For a = 2 we caiuuse the representation Y(t) = 2*rZ — t2 where Z is a standard normal deviate. Evidently, Z2 max 0 , 8 , M/2 i F(s) = —

(B.19)

if

= naZ —

n a

.2

0< Z < — otherwise .

It follows that,

<E-20>

=W$-"■("■£)! ^ _L j j 2 * 2 es/2P[s^ < Z <(s + a2f] ds ~ 2(1 - *-"*'*)

by standard arguments. The theorem now follows generally. REFERENCES [1] BERMAN, S. N. (1971). Asymptotic independence of the numbers of high and low level crossings of stationary Gaussian processes. Ann. Math. Statist. 42 927-946. [2] BERMAN, S. N. (1971). Maxima and high level excursions of stationary Gaussian processes. Trans. Amer. Math. Soc. To appear. [3] BILLINGSLEY, P. (1968). Convergence of Probability Measures. Wiley, New York. [4] BREIMAN, L. (1969). Probability. Addison-Wesley, Reading. [5] BRILLINGER, D. (1969). An asymptotic representation of the sample distribution function. Bull. Amer. Math. Soc. 75 545-547. [6] DOOB, J. L. (1953). Stochastic Processes. Wiley, New York. [7] EPANECHNIKOV, V. A. (1969). Nonparametric estimates of a multivariate probability density. Theor. Probability AppL 14 153-158. [8] FERNIQUE, X. (1970). Integrabilite des vecteurs gaussiens. C.R. Acad. Sci. Paris 270 A 169899. [9] HAJEK, J. and SIDAK, Z. (1967). Theory of Rank Tests. Academic Press, New York. [10] MANN, H. B. and WALD, A. (1942). On the choice of the number of class intervals in the application of the x2 test. Ann. Math. Statist. 13 306-317. [11] PICKANDS, J. S. Ill (1960). Upcrossing probabilities for Gaussian processes. Trans. Amer. Math. Soc. 145 51-73. [12] Q U A L L S , C . and WATANABE, H. (1971). Asymptotic properties of Gaussian processes. Tech. Report No. 736, Univ. of North Carolina, Chapel Hill.

306

MEASURES OF DENSITY FUNCTION ESTIMATES

1095

[13] ROSENBLATT M. (1971). Curve estimates. Ann. Math. Statist. 42 1815-1842. [14] STECK, G. P. (1957). Limit theorems for conditional distributions. Univ. California PubL Statist. 2 237-284. [15] WATSON, G. S. (1967). Some problems in the statistics of directions. Bull. I.S.I. 17 374385. [16] WOODROOFE, M. (1967). On the maximum deviation of the sample density. Ann. Math. Statist. 38 475-481. D E P A R T M E N T OF STATISTICS

DEPARTMENT OF MATHEMATICS

UNIVERSITY OF CALIFORNIA

UNIVERSITY OF CALIFORNIA

BERKELEY, C A L I F O R N I A 94720

L A J O L L A , CALIFORNIA 92037

307

K. S. Lii and M. Rosenblatt. Asymptotic behavior of a spline estimate of a density function. Comput. Math. Appl., 1(2):223–235, 1975. Reprinted with permission of Elsevier Inc.

J. Appl Prob. 13. 723-732 (1976) Printed in Israel ©Applied Probability Trust 1976

FRACTIONAL INTEGRALS OF STATIONARY PROCESSES AND THE CENTRAL LIMIT THEOREM

M. ROSENBLATT, University of California, San Diego

Abstract A class of limit theorems involving asymptotic normality is derived for stationary processes whose spectral density has a singular behavior near frequency zero. Generally these processes have long-range dependence' but are generated from strongly mixing processes by a fractional integral or derivative transformation. Some related remarks are made about random solutions of the Burgers equation. FRACTIONAL INTEGRAL; STATIONARY PROCESS; CENTRAL LIMIT T H E O R E M ; SINGU LAR SPECTRUM; BURGERS EQUATION

1. Introduction Let X(t), - x < f <°°, be a strictly stationary real-valued separable random process with EX(t) = 0 and EX(t)4<*>. Our interest is in limit theorems for (1)

A(T) = j

X(t)g(t/T)dt

as T—>3c, where g is a real-valued integrable function. The standard central limit theorem is for

(2)

0
and is often proved under a number of possible strong mixing conditions (see [4] and [5]). Suppose that the process X(t) has an absolutely continuous spectral distribution function F with spectral density /. The usual assumption on the variance of (1) for g given by (2) is that it is proportional t o T a s T - ^ o c . In order to arrive at this rate of growth it is usual to assume that (3)

i lim | /(A)

= c>0.

However, there are a number of special results in which asymptotic normality has been obtained where the spectral density has a behavior like Received 2 February 1976. Research carried out while the author was visiting the Australian National University and supported in part by the Office of Naval Research.

723 R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_29, © Springer Science+Business Media, LLC 2011

321

724

M. ROSENBLATT

/(A)HA|-,

«
as | A | —> 0 (see [1], [3] and [7]). If a is not an even integer, such processes are not strongly mixing (see [2]). Our object is to obtain a broader class of such limit theorems in which the basic behavior is laid out in a way that may be more apparent. The argument is based on the assumption that X(t) can be identified as a fractional integral of a strongly mixing process and g as a fractional derivative of a function. We shall want to specify conditions on the process X(t) in terms of an auxiliary stationary process Y(f). The conditions on Y(t) will be those used in the theorem of [5]. Let 3b,(Y) be the Borel field generated by the random variables V(r), r ^ t, and &t(Y) the Borel field generated by the random variables Y(T), r ^ t We shall say that the process Y is strongly mixing if (4)

sup | P(B n F ) - P(B)P(F)\ = g(r - I) i 0

Bern, Fe&T

as r - t —> ». The process Y will be said to satisfy Assumption A if the following conditions are fulfilled. Assumption A. (a) Y is a stationary separable process with EY(t) = 0, £Y(f) 4 <°°, and E\Y(t)Y(T)\4->0 as f - r - > 0 . (b) Y is strongly mixing. (c) The covariance function R(t) of Y, R(t2- U)= E[ Y(f2)Y(f,)], is abso lutely integrable with the spectral density /(A) of Y,

f(\) = j^j e^R(t)dt, positive everywhere. (d) The fourth-order cumulant function of Y, Q(t2- tuh-

tuU- t>)=

EiY^YitJYitJYiu)] -R(t2-

tJR (tA -h)-R

(h ~ U)R (U - t2)

-RiU-t^Rih-h), is absolutely integrable. Notice that under Assumption A the spectral density / is continuous as is the fourth-order cumulant spectral density, 4(A„ A2, A,) = III exp(- i{r,A, + t2k2 + h\,})Q(tu t2y h)dtidt2dt3. 322

Fractional integrals of stationary processes and the central limit theorem

725

The notion of a fractional integral (or derivative), when meaningful, will be of some interest. Because of the concern with stationarity and time translation, the Weyl definition of a fractional integral seems to be most appropriate (see [8], Vol. 2, p. 133). Let us call a V ) ( 0 = F 7 ^ I (t-r)a r(a)J-.

(5)

= l.i.m. L-~

1

l

- P

Y(r)dr

(r-rr'y(T)dT

I(a) ) /

the a t h fractional integral, 0 < a < 1, of Y if it is well-defined. Our definition is understood to be given by the limit in mean square on the right of Equation (5). One can define

(/ 1 y)(r) = l.i.m.(/ o y)(r). a t1

An alternative equivalent definition is given by (J,Y)(0-l.i.m.f P P !.-»»

L

Jo

Y(r)drds.

J-s

Then, recursively, one can define IpY for all /3 > 0 for which it exists by (6)

U^Y

= Ix{hY),

j3>0.

The fractional derivative DaY, 0 < a < 1, is given by DaY = D\IX

(7)

aY)

where by D 1 one understands the derivative in mean square. The derivative Da for all a > 0 would then be given recursively when it exists by

D^aY = D\DaY),

<*>0.

It is then clear that one could just as well make the identification h = Da for all a. These definitions could be extended slightly to that of fractional integrals (derivatives) centred at frequency /x. Thus the a t h fractional integral Infl centred at frequency ji would be given by

(8)

(L*Y)(t)

=| ^ jl

(t - T r V " V ( T ) d T

for 0 < a < 1 with the obvious modification given for other values of a as in the case of an ordinary fractional integral (or derivative). Lemma

1.

Let Y be a separable weakly stationary process (EY(t) = 0) 323

726

M. ROSENBLATT

y( )=

(9)

'

/l

EdZy (\)dZY(fi)=8(\

e dZY

'" W + n )dFY (A)

with corresponding process ZY of orthogonal increments and spectral distribution function FY. The ath ( O i a ) fractional integral (derivative) Ia.»Y (D"*Y) centred at fi exists as a weakly stationary process if and only if [ |A-/i|-2<»(iFv(A)
(10)

(j\\-fi\2°dFY(\)dZy(A) (ID

, r v ((D*-y)(r) = J (i(A -/i))«eft<*-">dZv(A)) .

It will be enough to go through the argument for Ia^Y with 0 < a < 1. Then (7o,My)(0 = li.m. f ^T f

(f-TrV""Y(T)
But f

e"
The following result is a useful formal analogue of integration by parts. Lemma 2. Let Y(t) be a separable weakly stationary process, EY(t) = 0, and g(t) a bounded integrable function. Then if (IaY)(t) exists, 0< a < 1, and if gT(t)

= g(t/T)

it follows that

(12)

j (IaY)(t)gr(t)dt = j Y(t)(Lgr)(t)dt

where

(/.gr)(0 = £ (t + T)-'g (y) dTr(a)(13)

= rf

J-«/T)

(f + T«)°-,g(M)d«r(a)324

Fractional integrals of stationary processes and the central limit theorem

727

Further, if a <{

(14)

j \Lgr(t)\2dt = ^

J |g(A)|2|Ar<*A

where g is the Fourier transform of g. Also (15)

\Lgr(t)\^KT°

for all t with K an absolute constant. The equality (12) follows straightforwardly on using the Parseval relation twice. One also notes that (IagT)(t) is what one obtains on applying the operator Ia to g ( - tjT). The inequality (15) is obtained by using the inequality

j (,/T)

r(a)

j-(„T)+i

r(«)

^ K' f (" ' ' T(f + 7u) a 'dw + T* f |g(u)| du J-UIT)

J

a

^ KT . A corresponding result can be given for fractional derivatives. Lemma 3. Let Y(t) be a separable weakly stationary process, EY(t) = 0, and g(t) a bounded integrable function that satisfies a Lipschitz condition offixedorder ft 0 < j3 < 1, uniformly over ( - «, *>). Then if (DQY)(t) exists, 0 < a < ft and if gr(t)= g(t/T) it follows that (16)

j (D°Y)(t)gT(t)dt

= |

Y(t)(D°gT)(t)dt

where (17)

(D'gT)(t)=

Moreover, if g(\)\\\a

(18)

j_* O + T r g ^ d T r O - a ) - .

E L2

J|J5-gT(0|2*=-^J|$(A)NAr
Fwrf/ier, 09)

\DagT(t)\^KTa

for all t with K an absolute constant. 325

728

M. ROSENBLATT

Relation (16) follows by the Parseval relation as in Lemma 2. Now r(\-a)D"gT(t)=j(jm

(t + T)-°gr(T)dT

-jK"«-£W«-0-Gr<-0>*. where GT is the indefinite integral of gT. On integrating by parts we obtain r(l - a)D°gT(t) = j t £° au-x-°{GT{u - t)-GT{= I Jo

aU-la{gT(u

- t)~

gr(~

t))du t)}du.

The expression

(20) JJV'-'tgTOi-o-gTC-O}^ Notice that the integral on the right-hand side of (20) is bounded in absolute value by

+

+

*r •-'--* /><»>I* ±I«(-T)IThe assumption that g ELip(/3) has been used. K is a constant. The range within which the result of Lemma 3 holds can be broadened somewhat. Let us say that g£A^, /3>0, if it is [/3] times continuously differentiable ([/3] is the greatest integer less than or equal to j3) and its derivative Dl0]g satisfies a Lipschitz condition of order j3 - [j3] uniformly over ( - oo, oo). The result of Lemma 3 then holds for g a bounded integrable function in Aft /3>0, and [ / 3 ] ^ a < 0 . It is clear that results corresponding to Lemmas 2 and 3 for the more general context of fractional integrals (derivatives) centred at \i hold and can be derived by the same type of argument. 2. A central limit theorem We can now obtain a central limit theorem for the sequence A(T) as T—►<» under appropriate conditions by applying the main theorem of [5]. Theorem. Let X(t) be a separable strictly stationary process that is the ath fractional integral for some a,\a\<\of a process Y(t) satisfying Assumption A. Then the spectral density h(k) of the process X(t) is such that (21)

limfc(A)|A| 2 *=/(0)>0 326

Fractional integrals of stationary processes and the central limit theorem

729

where /(O) is the value of the spectral density of the process Y(t) at zero. Consider the sequence of random variables (22)

A(T) = $

X(t)g(-£}dt

as T — ► oo, where g is a bounded integrable function. Ifa
Then

(23)

A(T)/a(T)

is asymptotically normally distributed with mean zero and variance 27r/(0) as T^oc, where
(24)

^j\g(\)\2\\\-2°d\.

The remark on the spectral density h(A) of X{t) near zero follows im mediately from the fact that X(t) is the ath fractional integral of the process Y(t). By Lemmas 2 and 3 A(T) can be written in the form

j Y(t)coT(t)dt with
\g(\)\2\\\-2ad\.

Further, lim W(T)1 I a)T(t + h)(DT(t)dt = l for all h. The bounds obtained on IagT(t), D~agT(t) in Lemmas 2 and 3 together with Assumption A imply that the conditions for the validity of the basic theorem of [5] are satisfied. Here M(A) is a jump function with total mass one at A =0. Notice that if Y(t) satisfies Assumption A then the two derived processes (with | a | < 5) 327

730

M. ROSENBLATT ( C . y x o = f^jjl

(25)

(r-T)-'cos M Ty(T)dT

,, (S...V)(0 = ffcj J_ (< " T)"- sin fir Y(r)dr

are jointly stationary. Of course, these processes are just the real and imaginary parts of (7 O(i y)(0. One can also remark that the processes (Ca,»Y)(t) and (S„,^y)(0 have spectral densities which behave like K|A±M|-2-

in the neighborhood of ± n respectively with K a constant. The following result can be regarded as a corollary of the theorem since it is obtained by essentially the same type of argument. Corollary. Consider the processes (26)

(C.>mY)(t),

(S a ,. y y)(/)

I a, | < i 0 < /i, < n2 < • • • < Mm, / = 1, • • •, m derived from a process Y(t) satisfy ing Assumption A. Let g,(f), / = 1, • • •, m, be bounded integrable functions. If a , < 0 we also assume that g,(f) satisfies a Lipschitz condition of order /3„ 0 < | a , | < / 3 y ^ l . Let

MT) = \(Cai.»lY){t)gl(±)dt (27)

Bi(T) =

j(Sa,,„Y)(t)gl(±)dt.

Then (28)

MTya^T),

B/(T)/oi(T)

j = 1, • • •, m are asymptotically normal and independent with mean zero and variance 7r/(/Lt;), where

(29)

^(r) = ^J|g/(A)|2|Ar^A.

In the context of the theorem, one could consider A(aT)/a(T),

a >0,

as a random process and it is clear that on heuristic grounds one would expect to have weak convergence as T-> oo to a Gaussian random process Z(a) with mean zero and covariance function Cov(Z(n),Z(*)) = 2ir/(0){|a|1+2- + |*|1+2- - |a - 6| I+2 -}, 328

Fractional integrals of stationary processes and the central limit theorem

731

a, b > 0. This type of result has been obtained by Taqqu in [7] for partial sums of instantaneous functions of Gaussian stationary sequences under appropriate conditions. His result corresponds to g(t) given by (2). 3. Random solutions of Burgers equation In Section 6 of [6], a class of random solutions to the Burgers equation was considered. Some doubt was expressed as to the asymptotic normality of the solutions in the one-dimensional case. The question is related to some of the issues brought up in this paper. We shall remark again about the reasons for the doubt in [6], but shall show that under appropriate conditions one does in fact get asymptotic normality. The Burgers equation for u - u(t,x) is (30)

U( + UUX = fJLUxx,

fJL > 0 .

Consider a random solution of (30), u = 2/x^ x (c - ¥) _1 , with c a positive constant and V(t,x) a solution of the heat equation t = fixx, strictly stationary in x with (31)

max |^ 0 (x)|
E^o(x) = 0

where ^ 0 (x) = ^(JC, 0). Let the spectral distribution function of ^ 0 be F(A). Since u()(x)= w(0,jc) = 2/i¥0*(c -V0)~\ we require that J\2dF(\)<*. Notice that (31) implies that max* |^(x, t)\< c for all t. We also assume that ^o satisfies Assumption A and so is in particular strongly mixing. These conditions could be weakened somewhat but they are kept so as to be consistent with [6]. We are interested in the asymptotic distribution of u(x, t) for fixed x as t —>*> when it is appropriately centred and normalized. Since *(x, t) = | * (47rM0-*exp { " ^ ^ 4 ^ } *o(y)dy it follows that maxx |^(JC, r)) —> 0 as f—►» and u(x, t)^ 2/xc_1^x. But (32)

¥,(x, 0 = - 1 (47rMf) *exp { - ^ 4 ^ } *oy(y)dy.

Notice that ^o y (y) is a strongly mixing process with spectral density A2/(A), /(A)= F'(A). In looking at (32) one is narrow band-pass filtering the process ^o y (y) and this is a strongly mixing process whose spectral density has a high order zero at A = 0. Under such circumstances the usual form of central limit theorem need not hold, and this led to possible doubts about the asymptotic normality of (32). We give the following simple example. Consider a homogene ous process £(f), - x < f < x ? with independent increments. In particular, one 329

732

M. ROSENBLATT

could take £(t) to be a Poisson process. Set X(t) = £(f)-2£(f - 1)+ £(t - 2 ) . The process X(t) is then a strongly mixing stationary process with covariance function R(t) =

2 - 31 r | -2+|r| 0

if if

|i|^l, is|r|^2, otherwise.

Notice that (33)

fTX(u)du=fT Jo

{^u)-^u-l)}du-l1{au-l)-au-2)}du, JT-I

JO

so that the sequence (33) has a non-normal limiting distribution as T—►<» if one does not normalize at all. Returning to (32), it is clear that one can write it as *,(*, 0 = / *o(y)(47r/x0-^ exp { - ^ = ^ j dy. Without loss of generality set x = 0. In this form one can see that the theorem of [5] is applicable and so one does have asymptotic normality. The key is that we are dealing with an expression of the form

(34)

X t)dt

\&{l)jt

( >

with X(t) a process satisfying Assumption A. If g is absolutely continuous with lim|,Hoog'(0 = 0 and g,g' absolutely integrable, (34) can be rewritten as -(1/T)/g'(t/T)X(t)dt and the expression is asymptotically normal as T->°°. In the case of (33), the function g is not differentiable. The global asymptotic behavior of the whole process (32) can be determined as T—►<». However, that would be better pursued elsewhere. References [1] DAVYDOV, Y U . A. (1970) The invariance principle for stationary processes. Theor. Prob. Appl. 15, 487-498. [2] HELSON, H. AND SARASON, D. (1967) Past and future. Math. Scand. 21, 5-16. [3] MANDELBROT, B. B. (1975) Limit theorems on the self-normalized range for weakly and strongly dependent processes. Z Wahrscheinlichkeitsth. 31, 271-285. [4] MCLEISH, D. L. (1975) Invariance principles for dependent variables. Z. Wahrscheinlichkeitsth. 32, 165-178. [5] ROSENBLATT, M. (1961) Some comments on narrow band-pass filters. Quart. Appl. Math.lSf 387-393. [6] ROSENBLATT, M. (1968) Remarks on the Burgers equation. J. Math. Phys. 9, 1129-1136. [7] TAQQU, M. (1975) Weak convergence to fractional Brownian motion and to the Rosenblatt process. Z. Wahrscheinlichkeitsth. 31, 287-302. [8] ZYGMUND, A. (1968) Trigonometric Series. Cambridge University Press. 330

J. Appl Prob. 17,265-270 (1980) Printed in Israel 0021-9002/80/010265-06 $00.85 © Applied Probability Trust 1980

LINEAR PROCESSES AND BISPECTRA

M. ROSENBLATT,* University of California, San Diego

Abstract A linear process is generated by applying a linear filter to independent, identically distributed random variables. Only the modulus of the frequency response function can be estimated if only the linear process is observed and if the independent identically distributed random variables are Gaussian. In this case a number of distinct but related problems coalesce and the usual discussion of these problems assumes, for example, in the case of a moving average that the zeros of the polynomial given by the filter have modulus greater than one. However, if the independent identically distributed random variables are non-Gaussian, these problems become distinct and one can estimate the transfer function under appropriate conditions except for a possible linear phase shift by using higher-order spectral estimates. LINEAR PROCESS; BISPECTRUM; NON-GAUSSIAN; FREQUENCY RESPONSE FUNCTION; MODULUS; PHASE; AUTOREGRESSIVE MOVING AVERAGE MODEL; LINEAR AND NONLINEAR PREDICTION; ESTIMATION

Let vty t = • • * , - 1 , 0 , 1 , •• •, be independent identically distributed random variables with mean zero, Evt=0, and variance one, Et?f = l. Consider a sequence of real constants {a,} with

The process (1)

xf = 2

a v

i—°°

i *-i

is the linear process generated by {a,} and {vt}. The frequency response function is

(2)

«(e-iA) = 2«,e" iM

Received 5 December 1978; revision received 23 February 1979. * Postal address: Department of Mathematics, University of California, San Diego, La Jolla, CA 92093, U.S.A. Research partially supported by the Office of Naval Research Contract N00014-75-C-0428.

265 R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_30, © Springer Science+Business Media, LLC 2011

331

266

M. ROSENBLATT

and the spectral density of {xt} is

(3)

f(\) =

±\a(e-*)\2.

We shall actually assume that 2 | a j | < o o (and often more) so that a(e lA) and /(A) are continuous. Of course, if both xt and t>„ t - 1, • • •, T, are observed a(e~lA) can be estimated by estimating the cross-spectral density of {xt} and {vt}. However, if only {*,} is observed a number of distinct but related problems arise. If {vt} (and hence {*,}) is Gaussian, then only /(A) or equivalently the modulus of \a(e~lk)\ can be estimated. It is then usual to assume that if {xt} is an autoregressive moving average process
p

(4)

2 fat-i

=

j=0

2 <*kVt-k9

(b0, q0 ^ 0)

k=0

that the roots of the polynomials k=0

(5) B{z) =

^biz'

are distinct and all have modulus greater than one (see [1] and [4]). It is clear that

(6)

„(«-" )~$Py

In the Gaussian case any zero Zj can be replaced by zjl without changing /(A) so that there are generally 2p+q possible specifications of the zeros without changing the structure of {*,}. However, the specification in which the roots all have modulus greater than one corresponds to the prediction problem in the following sense. Then {xt} can be represented as a one-sided moving average in terms of

V)

*t = 2)Af>f-/,

2lAl<°°-

/=0

vt can be represented as a one-sided moving average in terms of xt

(8)

Elr;l<0°-

i>r = 2 ?/*'-/> /-o

Thus vt is independent of the past of {*,}, that is, xt-uxt-2, one-step predictor of xt in terms of *,_,, JC,_2, - i s

(9)

x!=-££x,_; + i^,- k j = \ Do

k = l Do

with prediction error 332

• • •, and the best

267

Linear processes and bispectra

(10)

xt-x*t=^vh

E\xt-x*\2

b0

=

Oo

b0

It is also true that if one regards an autoregressive scheme p

2

bjXt-j = vt

;-0

as a system characterized by xt being driven by vh then the condition on roots of B(z) having modulus greater than one is equivalent to 'stability' of the system (see [2], [5]). Of course, if one is just interested in estimating the spectral density (ID

/ ( A ) - £ B(e~ix)

of an autoregressive moving average scheme, the roots taken with modulus greater than one may be as convenient a set of parameters for such a finite parameter scheme as any others. The prediction problem for a Gaussian sequence has the same character for forward prediction as for backward prediction. This is decidedly not the case for linear processes with non-Gaussian vt. The simplest case is that of the stationary autoregressive sequence (12)

X, = 2 * r - l + t > r

with r

vt =

1 1 - with probability -z 0 with probability ;r

and vt independent of the past of the xt process. Here the best predictor of xt in terms of the past (in the sense of minimizing mean square error of prediction) is * , -

2

* , - i +

4

.

The fact that vh xt do not have mean zero here should not trouble one. It just accounts for the constant 1 in the expression for the best predictor. Notice that the marginal distribution of xt is uniform [0,1]. The best predictor of xt given the future is x * = 2xt+1 modulo one = xt. Thus the backward predictor is non-linear and has variance of prediction error zero (perfect prediction) while the forward predictor is linear and has positive variance of prediction error. The process (12) is purely non-deterministic going forwards in time and purely deterministic going backwards in time. Of course, this simple example is a bit extreme. If one has an autoregressive moving 333

268

M. ROSENBLATT

average process {JC,} with the u,'s non-Gaussian and by chance the zeros of A (z), B(z) of modulus greater than one, then it is still true that the best predictor of JC, in terms of the past is given by (9) with prediction error (10). But if the zeros of A(z), B(z) do not all have modulus greater than one, then the best predictor of xt given the past will no longer have the form (9) and will generally be non-linear. Also, as we shall see, the function (6) and hence the zeros (as contrasted with the Gaussian case) will generally be identifiable. A simple example is now given of a situation in which there is interest in estimating a(e~lA) or as much of it as one can manage. Suppose {vt} is independent, identically distributed non-Gaussian. It is passed through the linear filter with coefficients {a,} by (1) to get {*,}. The experimenter observes only {xt} (not {vt}) and wishes to estimate a(e~lX). A particular case is that of the moving average (13)

xt = 6vt - 5i?r_i + v,-2

where the roots of A{z) are 2 and 3. The moving average (14)

xt = 3v, - lVt-\ + 2vt~2

has a polynomial A{z) with roots at \ and 3. Both schemes (13) and (14) have the same spectral density but if the independent, identically distributed u,'s are exponential, the marginal distributions of xt for these schemes are different. Assume that one has a linear process where the independent, identically distributed non-Gaussian variables vt have finite moment mk = Ev*(k >2) and corresponding cumulant yk. The fcth-order cumulant spectral density is then 1 ^r~i (2TT) (15) =

2

cum(x„ xt+h9 • • •, xt+ik

/ k_I \ Jexp I - 2 *MJ

1 {2rrf

We shall now indicate circumstances under which a(e except for a linear phase shift.

,A

) can be identified

Theorem. Let {xt} be a linear process generated by independent, identically distributed random variables {vt} with finite third cumulant y = y (v) ^ 0. Further let the coefficients a, be such that

(16)

2I/II«/I<°° i

and a(e~lX)^0 for any A. Then a(e~lX) is identifiable except for a linear phase shift if one observes the process {x,}. The conclusion of the theorem is still valid if the {vt} have finite kth (k > 2 ) cumulant yk = yk(v)^Q. 334

Linear processes and bispectra

269

Obviously |a(e~ i A )| is identifiable since |a(e~ i A )| = (2TT)'[/(A)]'. We thus have only to deal with arg{a(e~iA)}. Let ft(A) = arg{a(e- i A )}-arg{a(l)}. Since the ay's are real, h(- A) = - h(A). The bispectrum of the x process is

HK M) =

(do 1 «(«"I>(«"*><«I
( a ( l ) / | a ( l ) | ) y can be determined since | ^ j

Y

= (2^)Hfc(0,0)}/{/(0)}i.

Also h(A)+h(n)-h(A

+ fi)

can be determined since fc(A)+/t(/i)-/t(A

+ AO

= | log[(2 W )* y - , 6(A,M)V(A)/( / *)/(A + M»"*] " arg{a(l)}. Notice that li'(0)-fc'(A) = Hm^{fc(A)+li(A)-li(A+A)}. Set c = h'(0). Then fc(A) = fA {h'(u)-h\0)}du

+ cA.

Thus a(e" ,A )=w(A)e icA and by the simple argument given above a(e~ix) is identifiable except for sign and for the linear phase shift cA. The shift cA is clearly not identifiable under our assumptions unless one makes additional specifications. Notice that if c = n (integer), this just corresponds to a time shift of n units. The argument given can be immediately elaborated to take care of finite fcth cumulant yk ^ 0 (k > 3) if y3 = 0. The fcth cumulant spectrum of the process is given by (15) and a simple adaptation of the proof of the theorem yields the result. Given that xux2, - - •, xN is observed one can easily suggest a consistent estimate of a(e~ix) up to the unidentifiable phase shift cA. The estimate is suggested by the approximation 335

270

M. ROSENBLATT

/>W-»W*-|>'(;£)-*w}i as m —> oo. Let fN (A), feN (A, \i) be estimates of the spectral density and bispectral density with symmetric weight functions (see [3] for a discussion of such estimates). Let /3N be the linear bandwidth of the estimates so that fiN j 0, JVJSN-* 00 . NOW fN(k)^Trf is a consistent estimate of |a(e~ iA )|. Further (2TT^6 N (0,0)}{/ N (0)}- 3 / 2

is a consistent estimate of (a(l)/|a(l)|)y. This implies that (2.)^(A)i!f([1,.(i,^){fr(^)/»(f)/.(^)}"!]^K(0,0)/N(0)H) will be a consistent estimate of a(e~iX) up to sign and the linear phase shift cA if mp2N^0, m(N/3N)-1—►() as m,N^^>, /3N->0. Of course, this is a complicated and unwieldy estimate. Notice that when dealing with the autoregressive moving average scheme (4), the specification ao,bo^0 implies that the phase shift cA = 0 . References [1] ANDERSON, T. W. (1977) Estimation for autoregressive moving average models in the time and frequency domain, Ann. Statist. 5, 842-865. [2] ASTROM, K. (1970) Introduction to Stochastic Control Theory. Academic Press, New York. [3] BRILLINGER, D. R. AND ROSENBLATT, M. (1967) Asymptotic theory of estimates of fcth order spectra. In Spectral Analysis of Time Series, ed. B. Harris, Wiley, New York, 153-188. [4] HANNAN, E. J. (1970) Multiple Time Series. Wiley, New York. [5] WHITTLE, P. (1963) Prediction and Regulation. English Universities Press, London.

336

Zeitschrift fur

Wahrscheinlichkeitstheorie

Z. Wahrscheinlichkeitstheorie verw. Gebiete 55, 123-132 (1981)

und verwandte Gebiete © Springer-Verlag 1981

Limit Theorems for Fourier Transforms of Functionals of Gaussian Sequences M. Rosenblatt* Dept. of Mathematics* University of California, San Diego La Jolla, California 92093, USA

Summary. Limit theorems with a non-Gaussian (in fact nonstable) limiting distribution have been obtained under suitable conditions for partial sums of instantaneous nonlinear functions of stationary Gaussian sequences with long range dependence. Analogous limit theorems are here obtained for finite Fourier transforms of instantaneous nonlinear functions of stationary Gaussian sequences with long range dependence. Introduction A number of authors (see [2-7]) have considered non-central limit theorems for partial sums derived from nonlinear functionals of Gaussian sequences. One considers a stationary Gaussian sequence Xn9 n = . . . , —1,0,1,..., £ Z n = 0, EX* = 1. Let the correlation function of {Xn} be r(n) = EX0Xn.

(i)

A real function H(x) is considered with 00

/ x2\ J H(x)exp(-—)dx = 0

(2)

and oo

/

%2K

J if(x) 2 exp I—--)dx
The derived sequence H(Xn), n = ..., —1,0,1,... behavior of the sequence 1 l

N

Nn 1

~

j=N(n-l)

n=

(3)

is defined and the limiting -10 1

JV = 1 , 2 , . . .

(4)

Research supported in part by the Office of Naval Research Contract N 00014-75 C-0428.

0044-3719/81/0055/0123/S02.00 R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_31, © Springer Science+Business Media, LLC 2011

337

124

M. Rosenblatt

is determined as iV-*oo, where AN is a suitable positive norming factor. Under appropriate conditions of long-range dependence on the sequence {r(n)} and other conditions on the function H, nonnormal (non-central) limiting distri butions for the sequence {Y^n = ...,—1,0,1,...} have been obtained as N-+oo. The object of this note is to obtain analogous limit theorems for finite Fourier transforms

WH-j- T

mXJe-M.

(5)

We could deal with the Fourier transform in complex form or equivalently the real and imaginary parts under appropriate conditions as iV->oo. The behavior of the Fourier transform is of interest because it is a basic ingredient in the construction of spectral estimates [1]. Various of the ideas used in the paper of Dobrushin and Major [3] are helpful in deriving such a result. We shall assume that the covariance m

r(n) = |n|- a L(|n|)J]s A cosnAj,

s A j >0

for some positive integer m with 0 = X0
(6)

and L(t)9 0 < £ < a o , a

for every *e(0, oo). Let Hj(x) be the j-th Hermite polynomial with leading coefficient 1. The function H(x) is then expanded in terms of the Hermite polynomials H(x)=YcjHj(x)

(8)

where 00

ZcjjKco.

(9)

For convenience, let B% denote B% = {j\jeZ,nN£j<(n

+ l)N}.

(10)

Also let Ak be the set of frequencies obtained by taking sums of any k elements (with repetition allowed) out of the set {0, ±Xl9 ±X2,..., ±Xm}. We also introduce the following complex-valued Gaussian random measures. W0 is the spectral measure of the white noise process so that W0(A)=W0(-A) E\W0(A)\2~\A\ in 338

(11) (110

125

Limit Theorems for Fourier Transforms

for any interval A. Also, for any disjoint intervals Al9...9Aj on the positive axis W0(A1\...>W0(Aj) are independent. Further RQW0(A\ Im W0(A) are independent Gaussian variables with mean zero and equal variances if A is an interval on the positive axis. W for ^ > 0 is a Gaussian random measure with the same properties as W0 with the following exception. We no longer have (11). Also W^A x)9..., W^Aj) are independent for any disjoint intervals Al9 ...9Aj on the real axis. Further the random measure W_ ju>0, is specified so that W^{A)=Wtl(-A)

(12)

for any interval A. Such measures W^ are introduced for fx= ±ll9...? is assumed that Wx%9...9WXrn are independent.

±Xm and it

Asymptotic Distribution of Fourier Transforms We state our result below. Theorem. Let (6) hold with a<- where k is the smallest index in the series (8) for which cfc4=0. Set 1

ka

k

AN = N ~L(N)z.

(13)

Then the finite dimensional distributions of tend to those of

YnN(p\

n = ..., - 1 , 0 , 1 , - . . J e A k ,

Y„*03),

n=...,

-1,0,1,...,0eAk,

given by Y*(Q\—T\-ki2 Y m D

C

" ~

(Jn(xi + ...+xk)

^

a-1 2

■\Xl\

ei(xi

+ ...+xk) _ i

i(x1

+

...+xk)

a-1 ...\xk\

2

(14)

■W^dx^.-.W^dx,) where 00

(0L7l\

D= j exp(ix)|x| a ~ 1 dx = 2r , (a)cos (—-1

(15)

and 27 denotes a sum over k-tuples of \i values where the fijs can only range over 0, ± A 1 , . . . , ± A m . The multiple Wiener integral has to be modified so as to take account of the fact that W0{ — dx) = W0(dx) and W_fl( — dx) = Wfl(dx). See [2] for a discussion of such questions. 339

126

M. Rosenblatt

For convenience let X_j = X,

and s

x-j = sxj>

j=l--,™.

Lemma 1. Let a > 0 be any real number satisfying a<|min|i.f — Xj\. Then there is an infinitely differ entiable symmetric function f satisfying 0£/"(x)£l

for all x

f(x) = l

for all \x\
f(x)=0

for all |x|>a.

(16)

Let X„= J einxdz(x), jXn= j einxf(x-Xj)dz(x)= — oo

—oo

J e^.(x)dz(x), -co

j=-m,...,m,

(17)

and ♦*„=*„- I %

x

jXn=

/

m

^"%(x)dz{x) \

= Je'" i - Z / s W p ( 4

(18)

Let r.{n) be the covariance sequence of jXn,j— — m, ...,m,<£. Then rj(n)=^e^\f}(x)\2dG(x) ^{l+5x)\n\-'L{\n\)sXj^x'(l+o(l))

(19)

*/; = — m, ...,m and ^(") = l ^ | / 4 x ) | 2 r f G ( x ) = 0 (|n|- a L(|n|)).

(20)

—JI

Here G is the spectral measure ofthe stationary Gaussian sequence {Xn}. It is easy to construct a function / having the desired properties. Consider , ,

g(x)=

f

0

MM

if x^O or x ^ l

Let i4 = Jg(w)du and h{x) = A-lSg{u)du. o 340

Limit Theorems for Fourier Transforms

127

Then we can take

,M-»£-2)»(2-£). Consider now r.(n)= \ einxhj(x)dG(x)

(21)

hj{x) = \fj(x)\2.

(22)

with Let Then lei"xhj(x)dG(x)

= ^cj>kr(k

+ n)

k m

= I p= —m

K ( l + ^ ) I c M | n + fc|-aL(|n + /c|) k

However if j= — m, ...,m k

I c M e t t ^ = lIi(A1,) = |/ J (A p )| 2 = 0

for p + j .

Since fy is infinitely differentiable c M =0(|fc|"') where /J>0 can be chosen arbitrarily large. Thus for j — — m, ... ? m k

= I +o(*-2«) \k\
= \n\-«L(\n\)e^(

£

^ ^ ( l +
|i|
=

\n\-«L(\n\)(l+o(l))

and by a similar argument Ic j f f c |n + fe|-aL(|n +

fc|)^(B+k)^==o(|n|-aL(|n|))

if p 4=J. In the same way, one can show that Z ^ i | n + ft|--L(|n + fc|)^"+k>A' = o(|n|--L(|n|)) for all p = — m,..., m. The lemma follows from these observations. 341

128

M. Rosenblatt

Proof of the Theorem. Let I denote the index set { — m, ...,0, ...9m9<j>}. Then

—n

jel

where G is the spectral measure of the Gaussian sequence {Xn} and ZG is the corresponding random spectral measure of the process. Also jXn= ]e**dZGj{x)

(23)

—it

with and

dZGj(x)=fj(x)dZG(x)

(24)

Gj(l)=]\fj(u)\2dG(u).

(25)

—n

Let us consider the case H(x) = Hk(x). Now Hk{Xn) = Hk(ZJXn)

\ein^-+^ZG{dx1)...ZG{dxk)

=

jel

=

Je'"(jCl + - + J * ) Z G j | (dx 1 )...Z G i f c (dx 4 ).

I

(26)

Then ^N V

=

where

j=N(n-l) J _ f e i J V ( n - l ) ( x i + ... + x f c - «

■/cN(x1 + . . . + x ^ ^ ) Z G i i ( d x 1 ) . . . Z G . k ( d x f e ) * * ( * ! +■■■+**) = I

eW*' + - + a * >

i(xi + ... + Xk)_

e'

(27)

I '

We wish to first show that the variance of a term of (27) with one of the subscripts jt = (t> or else with jl9...Jke{ — m9 ...,m} but Ail + ...+A/k=|=J?mod27r tends to zero as iV-*oo. For simplicity in notation the computation is carried out for distinct j i 9 . . . , j f e . The variance is then 1 2 T J|fe N (x 1 + . . . + ^ - i 8 ) | d G j i ( x 1 ) . . . d G i f c ( ^ ) ^4JV 1

N2-k«L(Nfp^„q^ 1

I

YrJi(p-q)...rh(p-q)e-i<»-M

I

ZW-\p\)rh
Limit Theorems for Fourier Transforms

where

129

BN =

{p\-N
If one of the subscripts jt = 9 the estimates (19) and (20) together with an estimation like that given in the proof of Lemma 1 in [3] imply that (28) tends to zero as iV-*oo. The following lemma is helpful in showing that (28) tends to zero as N -► oo if ^.1 + . . . + \ + 0mod27L Lemma 2, Let oe>0. If L(t) is a slowly varying function there exist Lr(t) and L2(t) such that L{t) = Ll(t) + L2{t) L2{t)^o(L^)) (29) and n-'L^n) (30) is monotone decreasing. This lemma follows from Karamata's theorem (for Karamata's theorem refer to the book of Ibragimov and Linnik). By Lemma2 we can estimate (28) when Xjl +... + Xjk^Pmod27i by kc ip N2-klL{Nflp- 'L1(p)e ^

+

-+x^)

(31)

where L1 is a slowly varying function such that n~kccLx(n) is monotone decreas ing. The infinite sum in (31) is convergent since Xh +.„ + Xjk^f}mo&2n and so the whole expression (31) tends to zero as JV->oo. The case in which several subscripts are the same can be carried out similarly but in a more tedious manner. All these terms can therefore be neglected as N-+co. We now have to consider the terms for which Xjt +... + Xjk =ftmod2n. The asymptotic behavior of one such term will be determined as iV->oo but the argument given when trivially elaborated can be applied to the linear combination of any finite number of such terms. Thus the joint asymptotic distribution of the terms can be determined as iV->oo. Let us consider the term

1 J^-DU^...^-/.)*

AN

(

+

_+x flZ

{dx)...Z

Jl

{dx) ( 3 2 )

Jk

of the sum (27) with /.h + ...+Ajh = pmod2n. Let

z

,=

AJa/2

^ wz^+r,yl)

(33)

and

G'Ml-JffiGMj+N-1*)

(34)

with A a Borel set. Our object is to show that (32) converges in distribution to that of

i j-*/2fgi
+ ...+j(t)

i ( * i + •■■+**)

■s^-^...sxJ^WxMx1)...Wlu{dx^ 343

(35)

130

M. Rosenblatt

Of course, the finite dimensional distributions of (5) will converge in distribution to those of (14) and as remarked earlier a simple but notationally tedious elaboration of our argument will yield that result in the case H{x) = Hk(x). Let KN(u)=-kN(^u). Then expression (32) can be rewritten as ^(■-i><*i + . . . + ^ ^

(36)

The measure GNJ(*) corresponding to ZNJ(*) can be shown to converge locally weakly as iV-»oo to

„H«V

where 0G has density £>_1 |xj e _ 1 . Set

= J e ,i«, 1 , 1 +

...+«rt

lKA^

+

+ x X

GNji{dXi)

GNh{dXk)

(37)

where up = \tpN~\, p = l,...,k and I = (yLJiS..., AA). Then (Pn.xitv-'th) 1

Z

2 k

S r(p-q + u1)...r(p-q

N - «L(N)\tB»qtB» ■e

_i(p

~q^

e -{^XJI

+

+ uk)

—+u*xjh)

■r(p + uk)e-ip0e-ii-ua^

+ +UkX

-

^.

(38)

Just as in the proof of Lemma 1 of [3] one can show that l i m ^ i (t!,..., tk) N-XX)

= f U-W);—^—nr-r-^r d x -1 s

x

h '"

l*i+fil" X

3u

\x + tk\'

i + S(Xh) l + W J 2

'"

2

with the limit function (px a continuous function. Also the function
of a finite measure rjNl on Hfe with support on [ — Nn,Nn']k. Since pNfx(0 tends 344

Limit Theorems for Fourier Transforms

131

to a limit function cp^ that is continuous, it follows that the sequence of measures Y\N>1 tends to a finite measure rjx and cpx is the Fourier transform of t]x (see Lemma 2 of [3]). We consider special functions h taking on a finite number of values of the following character. Consider sets AuA2,...,Asi s = l,2,..., Borel sets (of finite and A_S,...,A_U Al7...,As are disjoint Let 0G mass) such that A_i=—Ai /T(zl5 ...Jk) be complex numbers and h{x1,...,xk) = fi(i1,...,ik)

if

xleAh9.„,xkeAiii

where iu...Jk take on the values ± 1 , . . . , + s but with i^iy fe(xl3 ...,xfc) = 0 for all other xx, ...,xfe. One can show that

if J#=/ and

j/ Z (x l5 ...,x fe )Z iVJi (dx 1 )...Z iVJk (dx k )

(40)

asymptotically as iV->oo has the same distribution as [h{x1,...,xk)WXh{dx1)...WXjJ,dxk\ The integral (40) is a polynomial in the random variables ZNJi(B) with the F s Borel sets. Now the joint distribution of the random variables ZN j(B) tends to the joint distribution of the random variables WK (B) and so we have the desired limit behavior. Also KN(xt +... -\-xk) tends to ei(xi

K

^

+

+X )=

- * i(x1

+ ,.. + xk) _ { +

...+Xk)

uniformly on every finite rectangle in Rk. Equations (37) and (39) imply that lim

J

\KN(x1 + ...+xk)\2GNJi(dx1)...GNJk(dxk)^Q

(41)

uniformly for N = 0,1,2,.... An adaptation of Lemma 3 of [3] then implies that expression (36) tends in distribution to expression (35) as JV-»co. For relation (41) implies that for any e>0 one can find a function of type h (depending on e) such that E\l[ei(n-1)ixi + -+Xk)KN(x1 + ...+xk) -h(xl9...,xj] -ZNJi(dx1)...ZNJk(dxk)\2<s for N>N(s) and £|J[^ ( "- 1 ) ( x i + - + X k ) ^ J V (x 1 + ...+xfc) -h(xl9...9xhy]

>wXji(dxx)...whk(dxkr<s.

The same argument as that given on p. 36 of [3] implies the validity of the theorem for general H. The theorem was derived for stationary sequences. However, it is clear that a corresponding result could have been obtained for random fields under con ditions comparable to those given in [3]. The normalization in the theorem is of the form NyL(N) with y>\ and L(N) slowly varying. The greatest interest is most likely in the case of a non-Gaussian limiting process. However, one can obtain nonnormal limiting distributions with a normalization having exponent 345

132

M. Rosenblatt

y<2 by considering processes somewhat like those analyzed in [6] and [7]. A parallel argument carried out in the case of continuous time parameter suggests that the limiting distributions obtained are self-similar. Additional Remarks Taqqu noted orally that one can determine non-Gaussian self-similar processes whose second order properties are the same as the Wiener process by using the techniques of [6] or [7]. This means that the exponent in the self-similarity is 1/2 as it is for the Wiener process. In all of the results discussed in [3] and [4] there is a smallest index in a Hermite expansion that plays a basic role. Given the results of [4] it is easy and of interest to concoct special examples in which one needs much more than one index to characterize a self-similar distribution. Such examples are particular cases of a general class of such processes. Let kW,fc= l,2,..., be independent processes each having the same probability structure as W0. Let Un(k) = ^xplin(xl .\x1 +

+

...+x1)']K0(x1,...9xJ) ...+xk\±\x1\Wk)-K..\xf«4k)->

-kW{dx1)...kW(dxl).

(42)

The discussion in [4] indicates that each of the process {[/n(fc),n=..., —1,0,1,...}, fc= 1,2,... is self-similar with exponent 1/2. Let VH=Y.Un(k)gk fc =

(43)

where the gk are assumed to approach zero sufficiently fast as fc-»oo so that (43) converges in mean square. The resulting process {[/„} is still self-similar with exponent 1/2 but involves polynomial forms in the W processes of all powers. Acknowledgement. I wish to thank a referee who suggested using Lemmas 1 and 2.

References 1. Brillinger, D.: Time series: data analysis and theory. New York: Holt, Rinehart and Winston 1975 2. Dobrushin, R.L.: Gaussian and their subordinated self-similar random generalized fields. Ann. Probab. 7, 1-28 (1979) 3. Dobrushin, R.L., Major, P.: Non-central limit theorems for nonlinear functionals of Gaussian fields. Z. Wahrscheinlichkeitstheorie verw. Gebiete 50, 27-52 (1979) 4. Major, P.: Limit theorems for nonlinear functionals of Gaussian sequences. [To be published in Z. Wahrscheinlichkeitstheorie verw. Gebiete] 5. Rosenblatt, M.: Independence and dependence, Proe. 4th Sympos. Math. Statist. Probab. 431443. Univ. California (1961) 6. Rosenblatt, M.: Some limit theorems for partial sums of quadratic forms in stationary Gaussian variables. Z. Wahrscheinlichkeitstheorie verw. Gebiete, 49, 125-132 (1979) 7. Taqqu, M.S.: Weak convergence to fractional Brownian motion and to the Rosenblatt process. Z. Wahrscheinlichkeitstheorie verw. Gebiete 31, 287-302 (1975) Received March 6, 1980; in revised form September 20, 1980

346

The Annals of Statistics 1982, Vol. 10, No. 4, 1195-1208

DECONVOLUTION AND ESTIMATION OF TRANSFER FUNCTION PHASE AND COEFFICIENTS FOR NONGAUSSIAN LINEAR PROCESSES1 BY K. S. LII AND M. ROSENBLATT

University of California, Riverside and San Diego NonGaussian linear processes are considered. It is shown that the phase of the transfer function can be estimated under broad conditions. This is not true of Gaussian linear processes and in this sense Gaussian linear processes are atypical. The asymptotic behavior of a phase estimate is determined. The phase estimates make use of bispectral estimates. These ideas are applied to a problem of deconvolution which is effective even when the transfer function is not minimum phase. A number of computational illustrations are given. 1. Introduction. Most of the literature on finite parameter time series models is either centered on Gaussian models or the results are motivated by what one can do in the case of Gaussian models. Here we deal with stationary nonGaussian linear processes and show that under broad conditions, aspects of the structure that are completely nonidentifiable in the Gaussian case can be resolved in the nonGaussian case. Assume that the random variables vt, t = • • • , — 1,0,1, • • • are independent and identically distributed with mean zero, Evt = 0, and variance one Evf = 1. Let {a,} be a sequence of real constants with

Consider the linear process generated by {«;} and {vt} (1)

Xt = £ 7 — 0 0 a zi

Let a(z) = YiJ J

OLjVt-j.

e

k the z-transform corresponding to the process {xt}. Then

is called the frequency response function or transfer function. We are concerned with the estimation of a(e~lX) on the basis of observations only on the process {xt}. Further, the linear nonGaussian process serves as a model for types of seismic exploration. The weights (Xj can be regarded as the signature of a disturbance passing through a medium and the random weights Vj as reflectivity of slabs in a layered medium. It has been observed that in many geophysical contexts the observed data (the xt's) are distinctly nonGaussian and a basic object is to deconvolve, estimating the a/s and v/s in the process. A discussion of related questions in the geophysical context can be found in Donoho (1981), Godfrey and Rocca (1981), and Wiggins (1978). The spectral density of {xt} is

f(\)=±.\a(e-*)\2. In the Gaussian case (when {xt} is normally distributed) the full probability structure of {xt} is determined by /(A) or equivalently by the modulus of a (e~lX), \ a (e~lX) |. The phase Received November 1981; revised May 1982. 1 Research supported in part by Office of Naval Research Contract N00014-81-K-0003. AMS 1980 subject classification. Primary 62M15; secondary 62G99. Key words and phrases. Asymptotics, bispectrum, deconvolution, linear process, non-Gaussian, phase. 1195 R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_32, © Springer Science+Business Media, LLC 2011

1196

K. S. LII AND M. ROSENBLATT

information in a(e lX) is not identifiable in the Gaussian case. If a (z) is a rational function i \

a(2)=

with A (z), B(z) Mz)

A ( z )

B(I)

polynomials = Il=o akzk,

ao * 0,

B(z) = JS-o bkzk,

b0 = 1,

the process {xt} is a finite parameter autoregressive moving average process; t h a t is, (2)

Z ! U bjXt-j = Xl=o akvt-k.

If {#*} is a Gaussian process satisfying (2), then any real root z} ¥^ 0 of A(z) or B(z) can be replaced by its inverse and pairs of nonzero conjugate roots by their paired conjugated inverses z'1 without changing the probability structure of {xt} if the process is appropri ately rescaled. This follows since \elX — z0\ = \zo\ \e~lX — zo11. With real distinct roots there are 2p+g ways of specifying the roots without changing the structure of {xt}. To ensure unique determination of the coefficients au and 6, of (2) (since there is a different specification of these coefficients corresponding to each of the possible root specifications) in the Gaussian case, it is the custom to assume t h a t all the roots of A (z) and B(z) are outside the unit circle | z | < 1 in the complex plane. However, for a nonGaussian stationary process satisfying (2) (in which case the inde pendent Vt's are nonGaussian) the different 2p+q specifications of roots mentioned above generally correspond to different probability structures and different stationary processes. As a simple example, consider the moving average xt = 6vt - 5vt-i + Vt-2 with the roots of A (z) being 2 and 3, and the moving average yt = Svt - lvt-i + 2vt-2 having a polynomial A (z) with roots V2 and 3. Both {xt} and [yt] have the same spectral density but if the independent random sequence {vt} is exponentially distributed, the marginal distributions of the {xt} and {y t ) sequences are different. In the problem of deconvolution where one wishes to recover the process {vt} (assumed nonGaussian which is most often the case in applications) in some sense, the proper specification of nonzero roots (which are inside and which are outside) becomes crucial (see Rosenblatt, 1974). There is a discussion concerning the distribution of roots as related to prediction problems in Rosenblatt (1980). T h e assumption t h a t all roots are outside the unit disc is often called the minimum phase condition. There are results on the estimation of the coefficients a, and bk of (2) (corresponding to roots assumed outside the unit circle) in Box and Jenkins (1976). In the Gaussian case these are essentially equivalent asymptotically to maximum likelihood procedures. In the nonGaussian case the computations are carried out as if the process were Gaussian. One has a least squares but not a maximum likelihood solution in the nonGaussian case. T h e coefficients estimated are those corresponding to roots outside the unit circle even though the actual structure of the process may not be one with all the roots outside the unit circle. Thus one will typically not be able to resolve the actual structure using these procedures in the nonGaussian case. Of course, if one knows the actual nonGaussian distribution of the vt9 one can use the maximum likelihood estimate or an asymptotically equivalent procedure to estimate the coefficients in (2) even if the roots Zj are not all outside the unit circle (see Basawa, Feigen and Heyde, 1975). Higher order spectral methods discussed in the next section do not require this knowledge. Our discussion follows t h a t of Rosenblatt (1980). 348

1197

NONGAUSSIAN PHASE AND DECONVOLUTION 2. Higher order spectral m e t h o d .

We prove the following lemma.

LEMMA 1. Let {xt} be a nonGaussian linear process (see (1)) with the random variables {vt} having all moments finite. Assume that

independent

2|y||o>l<<» and a(e~lX) ¥^ 0 for all A. Then the function a(e~lX) is identifiable on the basis of observations on {xt} alone up to the integer a in a factor eiaX and the sign of a (I) = X an. Actually it is enough to have moments of order k > 2 finite with cumulant jk ¥^ 0. Since the i;/s are assumed nonGaussian with all moments finite, there must be a cumulant of vt, jk ¥" 0 of smallest subscript k > 2. T h e kth order cumulant spectral density of the process {xt} is given by bk(Xi, • • • , Xk-i) =

1 ^ - i E y l t . . . ^ - i c u m f e , xi+Jl9 • • • , xt+jk_x) exp(-Xs=i X ij8K)

(3) =

17TTk^^e~1^

~-«(e-iX*-*)a(e^+-+x*->)).

(Z7T)

Let

^(X) = arg{.(e-) I ^L}. Then

[ «(D ]' yk=(27r)k/2-1bk(0,

ll«(l)|J

...,0)/{/(0)}^/2,

and (4)

h(Xi) + . . . + h(\k-i)

r r «(i) r _,bk(Xi,

- h(Xi + • • • + A*_i) = arg|

• • • , A*;-i)

since h(—\) = —h(X). Further (5)

h'(0) - h'(X) = l i m ^ o — i — {h(X) + (* - 2)h(A) - h(X + (k - 2)A)}. (k — 2)A

Now h(X)

[h'(u) - h'(0)}du

+ cX = hx (A) + cX

Jo

where c = h'(0). In particular h(lT) = hi (ir) + C7T.

Since the a / s are real we must have hiir) = air for some integer a. Set hiiir)/^ h(ir) = am = (8 4- c)7r, SO t h a t c = a — 8.

= 6. Then

T h e integer a cannot be determined without further assumptions since it corresponds to reindexing or subscripting the vt's. T h e sign of a ( l ) is also intrinsically undecidable since one can multiply a / s and L>/S by (-1) without changing the observed process {xt}. Thus, under the conditions specified above for a nonGaussian linear process {xt}, a(e~lX) is identifiable up to the integer a and the sign on the basis of observations on {xt} only and is given by a(e- l A ) = |27r/(A)| 1 / 2 exp{^(A)} 349

1198

K. S. LII AND M. ROSENBLATT

with (6)

h(X) = I {hf(u)-h/(0)}du

Jo

+ cX = h1(X)-^^-X

+ aX.

Notice that hi (X) can actually be computed. In Section 3 we will consider an estimate of the identifiable aspects of a(e~lX) suggested by Lemma 1 and the convergence properties of this estimate. Comments on the computations using spectral methods follow in Section 4. A few remarks on other computational methods are made in Section 5. Examples of estimation and deconvolution are given in Section 6. It should be noted that in some seismic investigations, a can be estimated through knowledge of speed of wave (e.g. sound) propagation through the medium. 3. Phase estimation and convergence of estimates. There are many discussions concerned with the estimation of the second order spectral density f(X) (see Anderson 1971, or Jenkins and Watts, 1968). We will concentrate on the estimation of h(X). For simplicity of discussion we will assume that the third order cumulant yz of vt is nonzero. The program in the higher order case can be carried out in a similar manner. Equation (5) becomes fc'(O) - h'(\) = lim A _ 0 ^ {h(X) + /i(A) - h(X + A)} when k = 3. From (4) we find that up to a sign h(X) + h(A) - h(X + A) = arg{63(A, A)}. From this point on we will drop the subscript and understand that we are dealing with the bispectral density 6(A, fi). The following approximation is useful in deriving the theorem of this section. LEMMA 2. Let the assumptions of Lemma 1 be satisfied. Assume that „6(A, \i) is a consistent estimate of 6 (A, p) (as n —> oo) based on a sample of size n. If one considers 0n(X, (i) = arctan(/m„6(A, ii)/Renb(X,

\i))

as an estimate of 0(A,M) = arg&(A,ju) then (7) ft,(A, /*) - 0(A, /*) = ~

Im b(A, fi) \AV 2 {Renbfr, /i) - Re 6(A, /i)} ]u/x \b(\fi)\ Reb(X,ix) {/mn6(A, ji) - Im 6(A, JU)} + Op(n6(A, [i) - 6(A, JU)). |6(X,/*)| S

We note that for a complex number z = x + iy = re10 with r = | z | and 0 = arctan(^/jc) a principal value determination, one has 30

X

~fy~7*'

d0 dx

y r2'

d20

2

2xy

Jx~ ~~'

and d20 _ 1 dxdy r2

2x2 _ r"

1 r2 350

2y2 r2

020

W

2xy

NONGAUSSIAN PHASE AND DECONVOLUTION

1199

The relation (7) follows on using these observations in a Taylor expansion of the arctan function. Let us consider estimating hi(X)

X. IT

Set A = A(n), kk = X, and let A = A(rc) —> 0 as n -^ oo. Assume for convenience that b(0, 0) is positive. A simple modification indicated later takes care of the case in which b(0, 0) is negative. Now hi(X) = h(X) - h'(0)X = h(kA) - ^

A

k&

= -S"=i : {^0"A) + h(A) - h((j 4- 1)A)} = -%kj=l ar g 6(yA, A). This suggests H„(X) = - ^ z 1 1 a r g n 6 0 A , A ) as an appropriate estimate of hi(X). THEOREM.

Assume that b(0, 0) is positive and that EX6t < oo. Consider Hn(X) = - S j - i 1 argn6(7-A, A),

(8) k A = X, as an estimate of

hi(\) = h(X) -

h'(0)\,

where it is understood that the bispectral estimates are weighted averages of 3rd order periodogram values. Ifb(X,n) ^ ^2 and the weight function is symmetric and bandlimited with bandwidth A, then Hn(X) - /i:(X) = Rn(X) + op(Hn(X) -

h^X))

with ERn(X) ~ I - ™b(uUL2

{ADlRe b(u, 0) + 2BDuDvHe b(u, 0) + CDlRe b(u, 0)} dub

fA 1 Re b(u, 0) r A 9 3 , h{u Q ) | 2 [ADllm

b(u, 0) + 2BDuDvlm

b(u, 0) + CZ^Im b(u, 0)} duA + o(A)

and 2
f

Cov{Rn(X), Rn(v)} = T3—2 m i n ( x > M) w 2 (a, *>) ^ <&> A ny3 J /or A(n) -H> 0, A2ra -H> oo as n -H> oo. Here A, B, and C are the second moments of the weight function w of the bispectral estimates (9)

A =

u2w(u, v) du dv,

B =

uvw(u, v) du dv,

C=

v2 w(u, v) du dv

and Du and Dv are partial derivatives with respect to u and v respectively. It has been shown in Brillinger and Rosenblatt (1967) under the assumptions of the theorem that (10)

E{nb(X, ft)} - b(\, n) ~ i I j (uDx + vD,)2b(X, n)w(u, v) du dvL2 + o(A2) 351

K. S. LII AND M. ROSENBLATT

1200 and (11)

Var{ n 6(A, M )} ^ / W 0 W

+ /O f w2(u, v) du dv,

0 * X * /i * TT,

if A 2 JI —> oo as M —» oo, A(n) -» 0. Further estimates n6(A, JU), „6(A', JU'), 0 < jit < A, 0 < JU' < A', are asymptotically uncorrelated as n —» oo if (A, JU) and (A', JU') are distinct. Using (7) and (8) we can write Hn(\) (12)

Rn(\)

= I

- M A ) = Rn(X) + o p <«,(A) - /h(A)),

Im b(jA, A) {Re„60A,A)-Re60A,A)} 60A, A)|2 Re 6Q'A, A) ^ {Im„ 6(7 A, A) - Im 6(/A, A)} I 60A, A)|2

and show by employing (10) that the approximation for ERn given in the statement of the theorem is valid. Further, by using (11) it follows that (9) holds. COROLLARY. Assume that all moments of {xt} exist and that the assumptions of the theorem are satisfied. Then EHn(X) —> hi(X) and Hn(X) is asymptotically normal with variance given by (9). The mean square error ofRn(X) is of order c

2

^

- A^ V

The optimal rate of convergence is n"2/b when A(n) ~ n~1/b. If b(\, ju) E ^3 and the weight function is bandlimited with first and second moments zero, the mean square error ofR„(\) is of the order C

'

A 4 +

The optimal rate of convergence is then n

4/7

^

with A(n) ~ n

1/7

The asymptotic normality with variance given by (9) follows on using (12). Generally we will estimate h\(X) and hence h(X) for a whole range of A values. The sign of 6(0, 0) may not be positive. We estimate it by noting the real part of n 6( A, A). If it is negative we multiply all nb(jA, A) with a minus sign. The estimate Hn(X) is then given by Hn(X) = - S j y a r g { - rt6(>A, A)}. 4. Computations using spectral methods. We remark on the computational as pect of phase estimation of a(e~lX) and give a few illustrative examples to indicate its effectiveness. Given a sample {xt} of size n = kN, we center and normalize it so that it has mean zero and variance one. Break up the sample into k disjoint subsections of equal length N so that the variance of the bispectral estimate from each section is not too large. Then choose a grid of points Ay = / A in (0, 2TT), 7 = 1, • • •, M, A = 27rL/N for an appropriate integer L. Though the symmetry condition h(X) = — h(— A) implies that one need only deal with A in (0,7r), there may be some advantage in considering A G (0, 2TT). We will comment on this point later on. Form the bispectral estimate TV60'A, A) of the type discussed above with a weight function of bandwidth A from each subsection. Average the estimates from the different subsections so as to arrive at a final estimate nb(jA, A). A detailed discussion of this kind of algorithm can be found in Lii and Helland (1981). Compute 0n(j) = arg{ ra 60"A, A)} + 2kir where the integer k is chosen to ensure continuity of Hn(^) = Hn(X/) = — X/=i Qn(j)> t= 2, • • •, M + 1 (neighboring values are as close to each other as 352

NONGAUSSIAN PHASE AND DECONVOLUTION

1201

possible). Since the upper index is /— 1 we start with / = 2. Since h(0) = 0 one sets Hn(0) = 0 and estimates /f n (A) = Hn(Xi) by an interpolation between 0 and Hn(X2), X2 = 2A. Hn(ir) is also computed by an interpolation procedure. This amounts to a complete procedure for estimating h(X). Since

i r

«* = — I

a(e-A)elkxd\

an estimate a* of a* is given by f2*

1 a* = — (13)

277j

a(e~lX)elkx

d\

°

and this computation can be carried out by using the fast Fourier transform. The oik's are real numbers and so the a^'s may or may not be real. If the symmetric property of /(A) and h(X) is used and the integration is carried out from — IT to IT almost real a^'s will be obtained. The imaginary part of the a^'s will only be the size of rounding errors. In practice there is no indication of how good or bad the estimates are apart from asymptotic results. In actual practice { j A } / ^ 1 may not be symmetric about TT. If the estimates Hn(Xj) are reasonably good the estimated a/s from (13) should still be almost real. The size of the imaginary part reflects the level of noise. When the estimates Hn(Xj) are not good the imaginary part of the a^'s becomes comparable to (or larger than) its real part. This can serve as a direct indication of the quality of the estimation. If the linear process is one-sided with a finite number of parameters one has a moving average of order q Xt = 2 / - 0 OLjVt-j,

do ^ 0.

We could estimate a(z) = £/=o oijZJby a(z) =2/=o OLJZK In deconvolution we try to recover the process [vt}9 vt = -7-777 xt(B is the backward shift operator so that BJxt = xt-j) by computing the approximation vt =

A/

_ x xt. If all the roots of a(z) (and a(z)) are outside

01(B)

the unit circle (the frequency function is minimum delay) then a~l(z) has a one-sided expansion J)^=o otjBJ\ In the computation, the series is truncated after a certain number of terms. If some of the roots of a(z) have modulus less than one we can still expand a~l(B) with a Laurent series expansion. Once the roots of a(z) are computed, one can easily get the Laurent series expansion of a~l(B) by partial fractions as described in Rosenblatt (1974) or Henrici (1974). Another way to find the inverse weights in deconvolution is to use a least squares criterion as described in Wiggins (1978). Another general method of deconvolution will be mentioned in the section on computation. A discussion of related questions for autoregressive schemes can be found in Huzii (1981). 5. Other possible computational methods. We briefly discuss two other possible methods of estimating the coefficients of a nonGaussian moving average process of order Q

(14)

** = Z/-oa,i>*-v.

As noted earlier, second order moments will not allow us to determine the location of the roots of (15)

a(z) =

ZUajZJ. 353

K. S. LII AND M. ROSENBLATT

1202

Higher order moments will be used in the first method which makes use of a least procedure. Assume Evt = 0, Ev] = y ¥" 0. Consider (16)

ck = Extx2t+k = y Sy ajaj+k,

squares

k = - q, - q + 1, • • •, q.

Estimate Ck by ck = — £2=i xtx2t+k n and solve the extremal problem (17)

min a , Y?k=-q (c* " 7 S/ a/a/+*) 2 .

There are g + 2 unknowns a0, • • •, aq and y in (17). Due to the homogeneity of the a / s we have to normalize the problem appropriately; all the a / s can be multiplied by a constant c 7^ 0 and y can be divided by c 3 without changing (17). There are a number of ways of carrying out such a normalization. One could set Ev3t = y = 1. Alternatively a 0 = 1 could be the normalization condition. Some comments on the asymptotic distribution of the ck's are given in Appendix 2. T h e second method is a searching procedure. One uses a typical second order method to estimate the roots of a(z), rJyj = 1, • • •, q, assuming all the roots have modulus greater than one. An accurate estimate of the distribution of roots is obtained by taking the conjugated inverse of an appropriate number of the r/s. Suppose all of the r / s are real and distinct. Then there are 2g possible sets of roots that give the same second order structure. Each of these sets yields a distinct set of a / s which in turn lead to a distinct set of the ck'&. Choose the set of a/s which determine the set of ck's minimizing

rk=-Q(ck-ck)2 among all the possible sets of {a,}. If some of the roots r, are complex, the inverse complex conjugates are taken in pairs. If there are multiple roots, the solution of roots in terms of coefficients is unstable. Some comments on this question are made in Appendix 1. T h e initial set of coefficient estimates corresponding to roots all outside the unit circle can be obtained by the method described in Box and Jenkins (1976). Alternatively, one could try to obtain the roots directly by solving for the roots of the polynomial p(z) = z*g(z) where g(z)=a(z)a(z-1)

=

Yt}-qPjZJ

with /3\j\ = ExtXt+\j\ = £ / a / a / + | y | . We estimate /?y by

T h e roots of p(z) with modulus greater than one are the initial set of roots. 6. E x a m p l e s . We will consider a few simple examples generated by Monte Carlo simulation to illustrate the computation and to give a qualitative feeling of the effectiveness of the theory. Details and possible "fine tuning" of the computational method will be considered elsewhere. We generate xt = vt + aiVt-\ + a2vt-2, t = 1, • • •, 640 where vt = vi - 1 and vl's are independent exponentially distributed random deviates with mean one obtained from the GGEXN subroutine in the International Mathematical and Statistical Library (IMSL). 354

NONGAUSSIAN PHASE AND DECONVOLUTION

1203

Then v a r ( ^ ) = Ev2t = 1,

Evt = 0,

Evzt = 2.

We partition {xt}V=\ into five sections, each of which has 128 points. Compute the bispectrum estimate i28&(e)(./A, A), j = 1, • • •, 13; i = 1, • • •, 5 by the algorithm described in Lii and Helland (1981). Here we set A = 187r/128 = 0.442. Our final bispectrum estimate is {i)

woM/A, A) = 1/5 2f-i

12sb

(jA

A).

Compute 0n(j) = a r g { n 6 0 A , A)} = arctan{Im n &(./A, A)/Re„6(yA, A)} by taking the principal value as well as -Hn(jA)

m -Hn(\j)

= lU

6n(i)9

j = 2, . . . , 14.

Let Hn(0) = Hn(\o)

= 0,

Hn(A)=Hn(\1)

= V2Hn(\2),

Hn(w) = % { « , (7A) + ifn(8A)} = %{fln (3.093) + if n (3.534)} and c = —HnM/w

= — 5.

Recall that c is an estimate of c = lim

A

_

^ = /*'(0) A

o

up to an integer. From formula (13) we compute the <Xk's. A standard smoothed periodogram with uniform weights and bandwidth A was used to compute fn (A) as an estimate of the spectrum /(A) of {xt}; fn(0) is obtained by a linear extrapolation. These examples are as follows: M O D E L : xt = vt + a\vt-\

+ ot2vt-2 with four cases specified in Table 1.

Here and from this point on all
«i = -2.247,

a3 = 0.5977.

When we increase the number of parameters to four, we observe qualitatively the same type of result as in the three parameter case. But generally speaking, the instability is increased. We give one example. T h e model is Xt = vt-

4.25^-i + A.lbvt-2 - 0.938i;,-3

with roots %, %, 4 and we get a0 = L106,

&! = -3.572,

a2 = 4.293,

a3 = -0.9762.

These simple examples indicate that one can estimate the unknown coefficients reason ably well and one is able to discriminate between different models even though they have the same spectral structure. 355

o ft ft.

-S"

ft.

oo

5! o

B-

to

00

ft

>^ •^

ft-

Co r-t.

ft-

0 3

>-

3

3

c m z o <

m C

33

T|

ro

4^

-^

-^

01

GO

CD 01

[NO

^1

^

—i.

o o

I GO

ARGUMENT

I 01

o o

I I

I I I I I I I I I I I I I p en en en ^ oo oo oo to to H-I H-> p b
I I I I I 1 1 1 1o o o t o f-» to b p p p p p o p p Q ..- O M O t o b b

o o H O _ c o ^ c n c n c o v j o c n o o c n o o c o O i w i o ^ c n H < i < i a i o i c o < i o o H < l H ^ O 0 0 H 0 0 ( X C 0 H O l 0 0 J i

o b

1 1 1 1 1 1 p1 p O O o p O p f- 00 b 00 b 'to b en CD to CD 4^ -a o 4^ an 00 en On as to On CD en CD to 0 5 C 0 O O C en 0 0 i 0 0 < 00 ICJiC0C0^ai O

I I I I I I I I H H y - ^ ^ h ^ p p p p p ^ b ^ b n b On OS O O Qb ObO b^ Ci O cn -J ^ o o c D O o o c nOt oOi ^—v »] 00 CD 4 K ) ^ . < | H ^ H O l C O oo -a

II I H 00 |>. tO oo

1 1 1 o O *4^ b00 oto

M JO H OS bo 00 H ^
I

I I I I I I I I I I I I p oi ai ^ ^ w co co to to H H H W H < i c o b a t b b k ) b o ^ COCnOOOOOCDCDOOOOCD OiUiv]COUi^.H05lO(OCDH HH
o

O O H W t O C O C O C O C O C O t O H H O O ^ ^ ^ H b o b c o b o a t c o < i b o c o b o ^ ^ W t s 3 0 l O C D C n O t 0 ^ ^ 0 0 H C 0 < l W^OOOtOvlHCOOOOOOOOOl^CO

p p j - 1 t o t o oo oo oo oo po t o »-» o o b b 4 ^ ^ b o 4 ^ b o b b o b o b b b o b o a s

U I W H O O O O I ^ O O H C O ^ O I ^ K )

p u i U i ^ ^ c o w p a t o M H H O p o H ^ w bo ' ^ io bi b CJ iss O05HvJC0C00iO05Kl(Xiiti

r M

3

>*°s cr

3 3

3.

ft

+ a-

w

I >

«N

3

C i-i

3

£3 I

£

p

^±

>

o

Co

I p bo oo

o

P

^ Oi

I CO bo Oo 4^

I JO J-» -J en

4^ h^ O

I CO CD O 4^

as en o Os1 CO i— 4 ^

P

CD *--j

o

CO CO CO CO

O

I O en 00 i—" as

CO

en CD

O CD

o o oo oo bo bo b b

p jo o jo en b en b

as j- 1 p p b on as ^ o as as

Co

o

CO tO H

I I I en po to 'o b\ oo o oo

o

^

a

2

ft*

M

2*

© fD M*

o

a

ft

93

3*

H

GC rK

ft

oo CO

CO

ft

o

cj

5

oo W

o Cd

ftoo

-3 -3

>

W

o &

> a:

4^

to

o

1205

NONGAUSSIAN PHASE AND DECONVOLUTION 7,

(D 0

■ J^jl-l' '••'; I-.'Ji1'. .!..''.. J J » . „. . , / j'l.J.ul^l.il,''

'.i

i

I

i"l

'•I' I

10

42

74

106

10

42

74

hMtAtk 0 42 10

74

I

[

II

138

170

202

234

266

298

106

138

170

202

234

266

?98

106

138

170

202

234

266

298

(2)

(3)

4

k

(4)

10

4 V

^ — ^ ~ / w . A/ " A -

42

74

106

138

170

202

234

266

298

(5) 0

10 42 FIG. 2. Deconvolution for sequence that generates deconvolution of xt using

74 106 138 170 202 234 '266 298 case 2. (1) Moving average process xt,t— 10 to t — 298, (2) exponential vt xt, (3) estimated vt as deconvolved by our procedure, (4) vt — vt, (5) minimum phase assumption.

To illustrate our deconvolution procedure we consider the moving average Xt = vt-

2.333^_i + 0.667^-2

with the v/s independent exponential variables with mean one. T h e vt's are generated as pseudo-random variates. T h e estimated model (using our bispectral techniques) is xt = 0.7164i;, - 2.175^_i + 0.7605^_ 2 . Let a(B) = 0.7164 - 2.1755 - 0.76055 2 . Set (18)

Vt — £jj=-9

&jXt-j

= T

d(B)

xt.

The first line of Figure 2 is the moving average process xt as generated for t = 10 to t = 298. The second line is the independent exponential vt sequence t h a t generates xt for the same t range. Line 3 shows the estimated vt as deconvolved by our procedure. T h e fourth line is the difference vt — vt. T h e last line is a deconvolution of xt using the assumption of minimum phase. T h e sample mean of vt — vt is .00032 and the sample variance .030. Notice 357

K. S. LII AND M. ROSENBLATT

1206

how our deconvolution has generally preserved the shape of the original vt sequence while the minimum phase deconvolution has no resemblance to the original vt sequence. A general way to find the deconvolution weights can be described as follows. We have an estimate «(*"*) = ^fjX) We compute b(e~lX) = a'^e'^).

exp{i(Hn(X)

+ cX)}.

Then 2m

co* = —

2»J„

b(e-nelkX

dX

give the coefficients of the series expansion of a~l(e~lX) which are the weights desired in deconvolution. This method is general. We do not require knowledge of the order of the moving average process and there is no need to compute the roots from the estimated coefficients and how the roots are distributed is irrelevant. APPENDIX 1 In this appendix we consider the relationship between the coefficients of a polynomial and the roots of the polynomial, at least locally. T h e polynomial is

Zf-oa^-ny-i <*-*>) where the roots are zj,j = 1, • • • , p, and the coefficients dj,j = 0, 1, • • • , p , with ap = 1. It is well known t h a t Clp-l = ~ Zij Zj, Clp-3 = — YiJ*k*tZjZkZt,

dp-2 = LiMk ZjZky • • • , do = (~1)P

Z\ • • • Zp.

Let us consider the relationship between the differentials of the coefficients a, and the differentials of the roots Zk. Now dfr-1 OZf

i

da

l,

—

P~2 _ V

— - — = — y.j*k*t OZs

y - -

LJ**ZJ—

OZ{

n

a

ZjZk = —cip-2

p-i

- y z

^

—2ap-\Z^—2z^i

dap-2 = -ap-i

£ , dzj - £ , zj dzj,

dap-3 = -ap-2 S J dzj - 2ap-i £ , z} dzj - 2 £y- zj dzJf . . . and this can be written in matrix notation as fdap-i\ dcip-2

ddp-z

J _ jjy\

dzz

where

u=

-1 -ap-i

0 -1 -2dp-i

358

0 0 -2

0 0 0

NONGAUSSIAN PHASE AND DECONVOLUTION

1207

and

,

/ , , T/

_ I Z\

22

23

\ Z\

22

23

...

£7 is a triangular nonsingular matrix and V is t h e Vander Monde matrix. V is nonsingular as long as t h e roots zt are distinct. APPENDIX 2 The object is to remark on some aspects of the asymptotic behavior of estimates of the c^s in t h e context of a general linear process. Let xt be a nonGaussian linear process (1) with Evt = 0,

E v2t = 1,

cum(uf) = y 4 ,

Evf^ys

cum(y?) = y 6 .

Set

# = xl For convenience we introduce gu = C o v ( ^ , # - „ ) ,

r„ = Cov(x,, xt-u),

hu = Cov(yt9

yt-u).

Consider t h e estimates ga — ~j^ Zjt=l

Xtjt+a

of Extyt+a. Then Cov(ga,

gb)

= —2 S ^ T - I C 0 V f e ^ + a ,

XrJV + ft)

where Cov(*0VHa, ^OV+ft) = rf_Tc,_T+a_& + gt-T-bgt-T+a

+ cum(x^ XT, ^T+6> cum(.y,+ a )

+ c u m ( ^ , xTyyt+a)

cum(yt+b)

+ cum(xt, yt+a,

xT,yT+b).

Now r

gu = y3 Y*J aJaJ-u,

« = Y<J «/«/-«>

hu = y4 2 a 2 a 2 - „ + 2 (£y a/*,-*,) 2 .

Further c u m ( ^ + a ) = H, a 2 , cum(x<, xT, j T + 6) = J4 "ZJ aja]+baj+(t-T) CUm(X/, # T , #

+ a

= k(b,t - T),

) = J4 Y,j ajOlj + aOLj + iT-t) = k(a>T

-

t),

cum(xt, yt+a, XT, yT+b) = ye £ M aMaw+(T-o««+aaa+6+(T-<) + 4 y 4 ( ^ M auau + (r-t)Otu + aOLu + b+(r-t))(Zu + 2 y i ( 2 « «tt«B+(T-t)fl«+fl)(J

OLuau+(T-t)

auOLu+(t-T)+a-b)

+ 2 y l ( 2 n « M « « + « - T ) « M + 6 ) ( 2 ] «««tt+(T-<) + 6-2)

= k(ay by t —

T).

359

+ b-a)

1208

K. S. LII A N D M. R O S E N B L A T T

I t is t h e n c l e a r t h a t limAr^ooiV Cov(ga,

gb)

= %S r8C8+a-b

+

^sgs-bgs+a

+ r 0 £ s { A ! ( M ) +k(a,

s)} +%sk(a,

b,s).

U n d e r t h e a s s u m p t i o n t h a t £ | OLJ \ < oo, w i t h a t r u n c a t i o n a r g u m e n t like t h a t e m p l o y e d i n A n d e r s o n (1970), o n e c a n s h o w t h a t

JN(ga-ga) are asymptotically normally distributed w i t h covariance structure given b y t h e preceding formula. A c k n o w l e d g m e n t . R e s e a r c h of t h e s e c o n d a u t h o r w a s c a r r i e d o u t in p a r t i n t h e U n i v e r s i t y of M a n c h e s t e r I n s t i t u t e of S c i e n c e a n d T e c h n o l o g y . REFERENCES ANDERSON, T. W. (1971). The Spectral Analysis of Time Series. Wiley, New York. BASAWA, I. V., F E I G E N , P. D. and H E Y D E , C. C. (1975). Asymptotic properties of maximum likelihood estimates for stochastic processes. Sankhya, Ser. A 38 259-270. Box, G. E. P . and J E N K I N S , G. M. (1976). Time Series Analysis, Forecasting and Control. HoldenDay, San Francisco. BRILLINGER, D. and ROSENBLATT, M. (1967). Asymptotic theory of estimates of £th order spectra. In Spectral Analysis of Time Series (ed. B. Harris) 153-188. Wiley, New York. DONOHO, D. (1981) On minimum entropy deconvolution. In Applied Time Series Analysis II (ed. D. F. Findley) Academic, New York. 565-608. GODFREY, R. and ROCCA, F. (1981) Zero memory nonlinear deconvolution. Geophys. Prospect. 29 189-228. H u z n , M. (1981) Estimation of coefficients of an autoregressive process by using a higher order moment. J. Time Series Analysis 2(2), 87-93. H E N R I C I , P. (1974). Applied and Computational Complex Analysis, vol. 1. Wiley, New York. J E N K I N S , G. M. and WATTS, D. G. (1968). Spectral Analysis and Its Applications. Holden-Day, San Francisco. L I I , K. S. and HELLAND, K. N. (1981). Cross-bispectrum computation and variance estimation. ACM Trans. Math. Software 7 No. 3. ROSENBLATT, M. (1974). Random Processes. Springer, Berlin. ROSENBLATT, M. (1980). Linear processes and bispectra. J. Appl. Probab. 17 265-270. WIGGINS, R. A. (1978). Minimum entropy deconvolution. Geoexploration 17. D E P A R T M E N T O F STATISTICS U N I V E R S I T Y O F CALIFORNIA R I V E R S I D E , CALIFORNIA 92521

DEPARTMENT OF MATHEMATICS U N I V E R S I T Y O F CALIFORNIA, S A N D I E G O L A J O L L A , CALIFORNIA 92093

360

The Annals of Probability 1984, Vol. 12, No. 4, 1167-1180

ASYMPTOTIC NORMALITY, STRONG MIXING AND SPECTRAL DENSITY ESTIMATES 1 BY M. ROSENBLATT

University of California, San Diego Asymptotic normality is proven for spectral density estimates assuming strong mixing and a limited number of moment conditions for the process analyzed. The result holds for a large class of processes that are not linear and does not require the existence of all moments.

1. Introduction. Strong mixing as a sufficient condition (usually together with some auxiliary moment conditions) for classical limit theorems was initially proposed and discussed in the period from the mid 50's to the early 60's (see Rosenblatt 1956, 1961; Kolmogorov and Rozanov, 1960; and Ibragimov, 1962). There had already been some investigation of the asymptotic properties of spectral density estimates. But from that time to the present day, qiost results on the asymptotic normality of spectral density estimates assume either a very special structure for the process (a linear process in Anderson, 1971, and Hannan, 1970) or else existence of all moments (as in Brillinger, 1975). Approximability by multilinear schemes in independent, identically distributed random variables has been considered in Rosenblatt, 1959. Here we shall show that a direct application of a central limit theorem using strong mixing together with a limited number of moment conditions will imply asymptotic normality for a large class of spectral density estimates. 2. Strong mixing and a triangular central limit theorem. Let X = {Xn} be a strictly stationary process. Let <%n = <%(Xk, k < n) be the (r-field generated by the random variables. Xk, k < n and jFm = &(Xk, k > m) the c-field generated by Xk9 k>m. The process X = {Xn} is said to be strongly mixing if (2.1)

snpB^0,Fs^n I P(BF) - P(B)P(F) | = a(n) -+ 0

as n —> oo. The following central limit theorem can be established by a big blocksmall block argument just as in Rosenblatt (1959, 1961). This result will be assumed. It will be shown that the result can be applied to the case of spectral density estimates. 1. Let {Yjn\j = . . . , - 1 , 0,1, .. •}, EYjn) - 0, n = 1, 2, . . . be a sequence of strictly stationary processes defined on the probability space of the THEOREM

Received March 1983; revised December 1983. 1 Research supported in part by Office of Naval Research Contract N00014-81-K-003 and National Science Foundation Grant DMS 83-12106. AMS 1980 subject classifications. Primary 60F05; secondary 62M15. Key words and phrases. Asymptotic normality, strong mixing, spectral density estimates, cumulants, nonlinear functions of Gaussian processes. 1167 R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_33, © Springer Science+Business Media, LLC 2011

361

1168

M. ROSENBLATT

strongly mixing stationary process X = {Xn}. Let Yjn) be measurable with respect to J?j-cin) n &j+C(n) where c(n) = o(n), c(n) f oo as n —► oo. Set (2.2)

hn(b-a)

E\2^aY(jn)\2.

=

Assume that for any two sequences s(n), m(n) with c(n) = o(m(n))9 m(n) < n and s(n)/m(n) —► 0 one has (o Q\

hn(m(n)) hn(s(n))

Let Fntm{x) be the distribution function of yin) 22Ll

i(n/m)hn(m)}^

with m = m(n) and c(n) = o(m(n))> m(n) = o(n). Assume that (2.4)

-y-

f

?n(n) ^ui>^

* 2 dF n , m ( n ) (x)^0

as n —► oo /or eac/i q > 0. There are then sequences k(n), p(n) —► oo as n —» oo u;itfi k(n)p(n) = n sucfe that (2.5)

2jLx Y,(»)/Vfc(n)Mp(n))

is asymptotically normally distributed with mean zero and variance one. Also if k(n)hn(p(n)) s hn(n) the normalization in (2.5) can be replaced by ylhn(n). Condition (2.4) is like a Lindeberg condition. A corresponding result with this condition replaced by a stronger Liapounov condition is valid. COROLLARY 1. The conclusion of Theorem 1 still holds if all assumptions are the same except for condition (2.4) and condition (2.4) is replaced by the following condition (2.4)'. Let

(2.4)'

\hn(m)}-i2+8)/2E\ 2B-! Yin)\2+5 = 0(1)

for m = m(n) and some 8 > 0. Because conditions in terms of cumulants are used a brief discussion relating to them is given. Let Z = (ZJt j = 1, • • • , k) be a randomfc-vectorwith t = (tj9 j = 1, • • • , k) afc-vectorof coefficients. The characteristic function of the random vector Z is 1^1 = 2 w*- If moments up to order s exist there is a Taylor expansion ;|m|

(m)

nt) = S|m|S8 — ^ — tm + 0( I f |S) 362

SPECTRAL DENSITY ESTIMATES

1169

with

tm = nJLi ip, m! = n^-i ™A and the moments M(m) =

^....^

=

£ ( 1 ] ^ ZfO-

There is a corresponding Taylor expansion for log
log
c(m1,...,mfc)

_

cum

( 2 f i , . . . , Zg**).

There are identities linking the moments fiim\ \m\ < s, up to order s with the cumulants c(m), \m\ < s, up to order s. In fact, the moments up to order s exist if and only if the cumulants up to order s exist. At times one will deal with random variables Wu • • • , Wj given in terms of doubly indexed random variables Uk,/y / = 1, • • ' , fk> k = 1, ' ' • , J by

wk = n i l uK,. The object is to carry out the computation of the cumulant cum(Wi, • • • , Wj) in terms of the cumulants of the Uk,s. Such a computation can be carried out in the following manner. Lay out the following array of the Ukjs in J rows £^2,i f/2,2 • • •

f/2,4

A partition of this array into sets vu • • • , vr is called decomposable if the union of the elements in a nonvacuous proper subcollection of sets of the partition yields exactly the elements in a nonvacuous proper subset of the rows of the table. A partition is indecomposable if it is not decomposable. One can show that cam(Wu • • • , Wj) = £ IU-i c u m ( U U ( j , / ) E p.) where the sum on the right is extended over all indecomposable partitions of the table given above (see Leonov and Shiryaev, 1959). Let us note that if fcth order moments of the process X = \Xn} are finite, that the cumulants eumtX}, Xj+Sl, • • • , Xj+s^) = r ^ . . . , ^ 363

M. ROSENBLATT

1170

are well-defined. Summability conditions on cumulants such as (2.6)

ls1,...,sk_1\r«]...tSkJ
are convenient to make at times. In Brillinger (1975) and Brillinger and Rosen blatt (1967) assumptions of the form

2 ^ , . ^ M ( i + 1 ^ 1 ) 1 ^ . . . ^ ! < oo, j = \f . . . 9 k — 1, are made for all k > 2. We shall only have to assume a limited number of the conditions (2.6) together with strong mixing. Most central limit theorems for dependent processes making use of mixing or mixingale conditions make an assumption on the rate of decay of the mixing coefficient (see Hall and Heyde, 1980). Notice that this is not the case for the limit theorems in this paper. The conditions are basically summability conditions on cumulants. In fact, we shall later give examples of processes that satisfy the assumptions of the theorems but still have a slow rate of decay of the mixing coefficient. 3. Spectral density estimates. The assumption (2.6) for k = 2 is just summability of the covariances Ik I rk\ < oo, rk = cov(x„ xj+k). The spectral density/(X) of the process is then continuous with /(X) = 1/2TT I ,

rke'ik\

Covariance estimates Hn) in terms of a sequence of observations, Xi, • • • , X n , EXk = 0, on X are given by rin) = 1/n 2£i* XjXj+kt

0 < k < n,

with r{-l = rjf}. A large class of spectral density estimates /n(X) are of the form fn{\) = 1/2TT Z f c + i rin)win) cos kX with win) = a(kbn)9 a(0) = l, with a(x) continuous at zero, bounded and satisfying a(x)

= a(—JC).

1

Here bn —> 0 with n" = o(bn) as n —> oo. The following proposition amounts to a simple remark on the asymptotic unbiasedness of a class of estimates of the form specified above. PROPOSITION. Let X = (XJ be a weakly stationary process with EXk = 0, ^ | rk | < oo. Assume that

fn(X) = 1/2TT 2S=in+i r{n)a(kbn) cos kX. Let a(x) be bounded, continuous at zero with a(0) = 1 and symmetric about zero. 364

1171

SPECTRAL DENSITY ESTIMATES

Then Efn(\) - /(A) -+ 0 as bn —> 0, n —► oo. Notice that Efn(X) - /(A) = ^ 2 I * I « rkr~^k{\a(kbn)

- 1] cos fcA

(3.1) ~ 7T 2|*|sn

*•* COS k\ - — 2|*|> n ^ COS &A.

The absolute summability of rk implies that the second and third sums on the right of (3.1) tend to zero as n —» oo. The assumptions ona(-) together with the summability of rk imply that the first sum on the right of (3.1) tends to zero as bn —> oo and n —> oo. The following lemma will be useful in deriving some results on the asymptotic distribution of spectral density estimates. LEMMA.

Let

g(u) = 1/2* 2 gke~iku,

£ \gk\
Further assume that a(-) is piecewise continuous, continuous at 0 with a(0) = 1, symmetric and such that a(x) = 0( | x \ "l/2~e) for some e > 0 as \ x | —»oo. Then if Wn(u) = 1/2* 2 i4 n) e-*" with wfc* = a(kbn) it follows that (3.2)

6„J

W£(u + X)g(i*)du->*(X) J

W2(u)du

as 6n —»0 where W

M

= 27TI <*(")*"*" <*" * */

w/ufe (3.3)

J

Wn(u + X)W„(u + /i)^(u) du = o(6^)

as bn —» 0 i/ A 5* /Lt» I A - \i | < 2*.

Notice that K

Wn{u + X)Wn(u + n)g(u) du j - 2 ^ / e - y " £■ Ik bna(kbn)a((k 2ir

2ir

365

/)bn)eik^xK

1172

M. ROSENBLATT

Clearly | £* bna(kbn)a((k - f)bn) | < 2 fe„| a(kbn)\2

(3.4)

and the right-hand side of (3.4) tends to \a(u) | 2 du

/

as bn -* 0. Also, if fi = X

2* bna(kbn)a((k - /)fe„) -* jJ \a(u) | a(u) | 2 du as bn -* 0 for each fixed /. This yields the limiting relationship (3.2). Let us now consider the case in which X 5* fi, | X - fi \ < 2x. Relation (3.3) will follow if we can show that 2* bna(kbn)a((k - /)6 n )e^" x ) -* 0

(3.5)

asfe„-» 0 for each fixed /. Given e > 0 there is ani(c) such that 2&„I*I>L(*)

bn\ a{kbn) \2 < e.

The function a(x) can be uniformly approximated arbitrarily well by step func tions over the interval | x | < L(e). It is therefore enough to show that (3.5) holds for a step function a(x) of finite support. In fact, this will follow if one can prove (3.5) for a(x) the indicator function of a finite interval, say the interval (au a2), «i < a?2. Then

2* a(kbn)a((k - /)fen)e*<«-x> = 2«1<*6„<«2 e ^ ^ k . + 0(6.) = e ^ " x ) 2o<^
e,(M-x)

_

1

+ 0(bn)

With Cn = fen1. Let /\ \

=

J1 ^ ^ = ^> & integer 10 otherwise.

We shall prove the following theorem. THEOREM 2. Let X = {Xn} be a strictly stationary strongly mixing process with EXj m 0. Assume that the cumulant functions (2.6) of order two and four are summable. Further, let the spectral density estimate fn(X) have weights w™ defined in terms of a function a(-) that is piecewise continuous, continuous at zero with a(0) = 1, symmetric about zero and is such that xa(x) is bounded. Let

n n) (X) = 2 S U . ) XuXu+kwin) cos k\ 366

1173

SPECTRAL DENSITY ESTIMATES with u>in) = a(kb(n))

and c(n) = abn1 for all sufficiently ^n(X) — SCLi

inbn11}) 1 / 2

with m = m(n) and c(n) = o(m(n))9 m(n) function Fnym{x) of Zn(\) and assume that

= o(n).

7T I x2 dFnMn)(x) in) J\x\>n

(3.6)

large fixed a. Set

Consider

the

distribution

is

asymptotically

-* 0

as n - » oo, bn —► 0, nfen —» oo /or eac/i t\ > 0. 7%en / n (X) — Efn{\)

normally distributed with mean zero and variance (3.6)'

2,r(H-„(X))

fw2(a)da

raon

J

Let us first consider the case in which a( •) has finite support. We first consider replacing/ n (X) by

/»(x) = 4 21&U.) - 27-i *^+*i4 n) cos *x. Here the support of the function a ( - ) has been taken as the interval [—1, 1] to simplify notation. The argument goes through in exactly the same way for any other interval. Notice that (3.7)

2 ™ fn(\)

= S2=i YLn)

where Yin) = S l f c U ) XuXu+kwin)

cos *X.

Thus, the expression (3.7) is of the type discussed in Theorem 1. Now, (3.8)

a2{fn{\)

- /„(X)) < - 4 - i *2[2c*(=i I?=n-k XjXJ+kwin)

cos kX]

and the right-hand side of (3.8) is less than 2 2 luhM'^\ ZJ—n—k Zj'^n—k'

7r n

• \\rH.rj-r+k-k.\

+ \rM^rj.r+k\

+ | r£,W + *wll

K ^ ' l

* - 4 - 5 2 $ - i 2* min(*', *){ | r. 11 rs+k,-k | + | rs.k>rs+k | + | r & V |} 7r n ">ln) 11 «#» The assumptions on the weights w^ and the function a( •) imply that the weights are bounded and 1*1 l 4 n ) |

^Lbn1

for all k and some constant L. This together with the summability of (2.6) for 367

1174

M. ROSENBLATT

k = 2, 4 imply that
hn(m(n)) Ziu,u'=l Z)A,ft'=-c(n) vu-u'fu+k-u'-k'

' fu'-u+k'^u'-u-k

(3.9)

• ?k,u'-u,u'-u+k' J

• cosfcXcos fe'Xic^u;^ = (1) + (2) + (3)

where m = m(n) < n with 6„x = o(m(n)) as /i —> oo. The first term on the right of (3.9) is (1)

-JT*

sin2(m/2)(a - ff) /(«)/(£) I E^cCn) " ^ cos uX e'^| 2 da d/8. sin2(l/2)(a - 0)

«/

J-,

si

The sequence of weight functions Wn(u) have the property (3.10)

max|Uis^m-i

jl,

Wn(v)Wn(v + u) dv

J% WIM du

1

as m = m{n) -* ooforeach A > 0 since c(n) = bn1 = o(m(n)). An argument like that given on page 176 of Rosenblatt (1974) implies that

(3.11) (1) = 2xm(l + o(D) £ fdS)

\ Wn(0 + \) + | Wn(f3 - X) dj8

holds. The lemma then yields (1) - x/n(l + n(X) + o(l))/2(X) £

(3.12)

W2(u) dv.

The second term on the right of (3.9) is ,„. (2)

I \ * J 1

sin2 (m/2)(a - 0) ,. v ,.„ v /l „ r , WM sin2 ma - fi) fia)m\2

,. + X) +

1 .„ „ 2 WM "

X)

A j

| Wn{p + X) + | W„(0 - X)j da d/3. Property (3.10) of the family of weight functions as well as the continuity of / imply that relation (3.11) holds for (2) as well as (1). This implies that we get estimate (3.12) valid for (2) as well as (1). A direct estimate shows that

l(3)| < m W 368

kfcVl

SPECTRAL DENSITY ESTIMATES

1175

and this is of smaller order than (3.12) under the assumptions made. Notice that Wn(u)2 du = (1 + odWZ1 J

J

W\u)

du.

The estimates of (1), (2) and (3) imply that a2(fn(X)) and therefore a2{fn{\)) have the same asymptotic behavior as (3.1) as n -» oo. This is obtained by setting ra(ra) = n. Also notice that if m(n) = o(n) and k(n)m(n) = n, that then k(n)hn(m(n))

a hn(n)

because of the summability of the cumulant functions for k = 2 and k = 4. Assumption (3.6) and Theorem 1 now imply the conclusion of this theorem in the case of a function a( •) with finite support. We now indicate how the argument can be modified so as to take care of a function a( •) that is not of finite support. One can replace fn(\) by fn(X)

= l/2x

EIJHSCO.) r ^ W

cos feX

with error term £ n ( X ) = 1 / 2 T ScCnXIAIsn H f ^

COS feX

and c(n) = L6"1 where L is fixed but sufficiently large. The asymptotic normality of fn(\) has already been determined by the argument given since it corresponds to a function a( •) with finite support. We just have to estimate the variance of the error gn(X) and show that it is sufficiently small if L is large. That will establish the asymptotic normality of /n(X) for a(-) not of bounded support. Now

+ C ^ ^ ^ F H ?

cosfcXcos A'A

and this is bounded by A 9n 47r

(3.13)

2i*i,i*'i^c(n) Es {|r,| |r8+*-*'| + \rs-k'Ts+k\

+ |rJSU*|} l*4n)l l«# } lNotice that the terms | wln)| = | a(kbn) | are bounded by an absolute constant. The sum in (3.13) involving fourth cumulants is 0(l/n). The other terms in (3.13) are less than or equal to (3.14)

- V

ZIAUA'I**.)

2 I i w l ^ l *4 n) | I ^ 1

47T M

(2 }

where | rk\ * represents the convolution of the sequence | rk\ with itself. Now (3.15)

| kwin) | = | ka(kbn) | < Mbn1 369

1176

M. ROSENBLATT

with M an absolute constant. This implies that the sum (3.14) is bounded by 1 M2 1 ls\rs\ M*b? Zm*c(n) { | * | ( l * l + |*l I}" * — L^bnbn2 2 . I r. I (2*> n n because of (3.15). The variance
COROLLARY 2. Let all the assumptions of Theorem 2 be satisfied with the exception of (3.6). This is modified as follows. Consider

(3.16)

SJ-i ajZn(Xj)

with the Xj distinct frequencies in [0, w]. Let Fnimin)(x) be the distribution function of (3.16). Let (3.17)

- 7 - I

x2 dFn,m{n){x) -* 0

as n —» oo, bn —* 0, nbn —» 00 /or aW values of the a/s and each rj>0. It then follows that (nbn)1/2\fn(Xj) - Efn(\j)} are jointly asymptotically independent and normally distributed with variances 2x(l + 1?(X,))/2(X,) J W2(a) da, j = 1, . . . , s. The conditions of Corollary 2 imply that every linear combination (nbn)1/2 2U <xj{fn(\j) - Efn(\j)} is asymptotically normal. That implies that the spectral estimates (nbn)1/2{fn(\) — Efn(\j)\, j = 1, • • • , 5, are jointly asymptotically normal. The asymptotic orthogonality and hence asymptotic independence follows from (3.3) of the lemma. The conditions of the form (3.17) are sometimes difficult to verify directly. The following corollary replaces these by stronger cumulant conditions that are sometimes easier to implement and interpret. COROLLARY 3. Let the assumptions of the corollary be satisfied except for (3.17). These are replaced by summability of the cumulant functions up to order eight The conclusion of the Corollary then holds.

It is enough to show this in the case of a single spectral estimate. The argument for several estimates is the same except for the additional notation. In order to apply Corollary 2 in this situation we shall verify condition (3.17) by showing that <x~4 (ZS-i n n ) ) E\ 22.x (Y™ - EYin)) | 2 = 0(1) 370

SPECTRAL DENSITY ESTIMATES

1177

where YLn) = Y(un)(\), for m = m(n) as n -► oo. Now E\ 2S.i (Y - £Y<»>) I4 = ). Because of the multilinear character of the cumulant cunuGUi YLn)) = S!U«3,u4-i cum4(Y<»\ Yjf, ^ , *£?) and cum4(Y<»>, Y%\ Y # , Yki cos kiXJ £„ cum(X„ s G vx) • • • cum(X„ s G vp) where the summation over v = pxU • • • U pp is over all indecomposable partitions (see the discussion in Section 2) of the following table

One of the several indecomposable partitions consisting entirely of pairs leads to the sum ,n - i o \ "ki

^"»

r

"l-"2r"3-"4r"3-"l+^3-^ir"4-"2+^4-*2

w{£ cos kiXwk? cosfe2AWk? cosfc3Au;^ cos k4X. Introduce a = ux - 1*2, 6 = u3 - u4, a = fe3 - ku p = fc4 - ^2. Expression (3.18) is bounded by laAuM.k^.aj \ra\ \rb\ \r^Ul+a\

\r^Ul^b+a^\

| u ; ^ | \w{g+a\ \w(^\

\w(g+0\.

If one first sums over ku k2 the bound is obtained. A sum over a, 0 yields the bound bn Ea,6,ulftt3 I r01 | rb |. The sum over a, 6 and then uu u3 gives us the final bound bn2m2. All the other indecomposable partitions consisting entirely of pairs lead to the same bound. Indecomposable partitions which do not consist entirely of pairs yield smaller bounds. An example is given by the partition leading to the sum (3.19)

2*

n?-! w

cos kM.

Introduce 1*2 - ux = a, u3 - Ui = 6, u4 - ux = c, a = k2 - fei, 0 = k4 - fe3. Expression (3.19) is then bounded by E u W ^ w , \r(X\

\ra+a\ \r0+b\ \wtf\ \w%la\ \w(g\ 371

\w{&0\.

1178

M. ROSENBLATT

Summing first over ku k2 leads to the bound bn

2julta,b,c,a,0 \ 1*a,byc \ \ ^a+a \ \ Fp+b \ •

Then sum over a, 0 and then over a, 6, c to obtain bn2 2-t 1 = mbn2. These estimates imply that (3.17) is valid so that Corollary 2 can be applied to obtain the asymptotic normality of such spectral density estimates. A little reflection shows one that Corollary 3 is still valid if one replaces strict stationarity by stationarity up to eighth order. If one has a vector-valued strongly mixing strictly stationary process, compa rable conditions lead to corresponding results on the asymptotic behavior of cross-spectral estimates (see Brillinger, 1975, and Rosenblatt, 1959). These ideas can be adapted to get results for higher order spectral estimates. 4. Additional remarks. Kolmogorov and Rozanov (1960) have shown a sufficient condition for a Gaussian stationary process to be strongly mixing is that it have a continuous positive spectral density function. In their book, Ibragimov and Rozanov (1978) give an example of a strongly mixing Gaussian process with an unbounded spectral density function. This example is constructed by making use of a result of Helson and Sarason (1967). Non-Gaussian strongly mixing processes can be constructed by taking instantaneous or bandlimited functions of Gaussian strongly mixing processes. Let
v uAd /dz»)
,

v = 0, • • • , the Hermite polynomials. Let Yu • • • , Ym be jointly Gaussian random variables with means zero, variances one, and covariance matrix 9 = (piji hi

= 1> • • • > m)-

Slepian (1972) has shown that the moment E\HSl(Y.)

• • • HSm(Ym)\ = * ! . . . sj E ' £ Vij.

where the primed summation is over all m X m symmetric matrices v = (vi/, i,j = 1, • • • , m) with nonnegative integer entries such that Sk = 2/#* Vjk, k = 1, • • • , m. This result leads directly to the following result. COROLLARY 4. Let X = (Xk) be a strictly stationary Gaussian sequence with mean zero and summable covariance function. Assume that yp{x)isa function with 372

1179

SPECTRAL DENSITY ESTIMATES

a Fourier-Hermite expansion that has nonnegatiue coefficients. Further let E\$(x)\2k/2

=

^

U

Hk{x)tk/kl

For any positive c

~lk-°

k\ J-J6

dt

-2»-°

2kk\ C

•

With c sufficiently large the (2fe)th moment of \p(x) will be finite but not higher order moments. If X = (Xn) is a stationary Gaussian sequence with summable covariance sequence and positive spectral density, it will be strongly mixing. The non-Gaussian process \l/(Xn) will be strongly mixing and have cumulant sequences summable up to order 2k. If k = 4 the assumptions of Corollary 3 will be satisfied. We now show how one can construct examples satisfying the assumptions of Corollary 3 but have a slow rate of decay of the mixing coefficient. A result of Ibragimov and Rozanov (1978, page 181) states that the mixing coefficient of a stationary Gaussian sequence a(n) = 0(n^) for a fixed 0, 0 < 0 < 1, if and only if the process has a spectral density of the form /(X)= \P(eix)\2w(X) where P is a polynomial with zeros on | z \ = 1 and w is strictly positive and satisfies a Holder condition of order 0. One can show that if {Yn}, EYn = 0, is a stationary Gaussian sequence with covariances for | n | > 1, its spectral density is of the form /(X) = c | X | ' + * ( X ) where c > 0 is a constant and g(X) an infinitely differentiable function for | X | < 7r (see Zygmund, 1968, vol. 2, page 135). Notice that for ft in this range E|rn|<°o 373

1180

M. ROSENBLATT

but not I \n\ | r „ | < o o . By choosing ft sufficiently close to zero, we can effect as slow an inverse polynomial rate of decay of a(n) as may be desired. Consider the process Xn = Yn

, n = 1, 2, • • • ,

for k some integer greater than two. By (4.1) it is clear that all cumulant functions are absolutely summable. Corollary 3 is therefore applicable to the process {Xn\. However, \Xn} has exactly the same mixing coefficient sequence as the process One can ask whether absolute summability of cumulants up to fourth order is enough to get the conclusion of Corollary 3. The examples constructed by Herrndorf (1983), indicating that moment conditions up to second order are not enough for a standard version of a central limit theorem, suggest that one may need more than summability of cumulants up to fourth order in the context of spectral density estimates. REFERENCES ANDERSON, T. W. (1971). Statistical Analysis of Time Series. Wiley, New York. BRILLINGER, D. R. and ROSENBLATT, M. (1967). Asymptotic theory offcthorder

spectra. Spectral Analysis of Time Series (ed. B. Harris), 153-188. Wiley, New York. BRILLINGER, D. R. (1975). Time Series, Data Analysis and Theory, Holt, New York. HALL, P. and HEYDE, C. C. (1980). Martingale Limit Theory and Its Application. Academic, New York. HANNAN, E. J. (1970). Multiple Time Series. Wiley, New York. HELSON, H. and SARASON, D. (1967). Past and future. Math. Scand. 21 5-16. HERRNDORF, N. (1983). Stationary strongly mixing sequences not satisfying the central limit theorem. Ann. Probab. 11 809-813. IBRAGIMOV, I. A. (1962). Some limit theorems for stationary processes. Theory Probab. Appl. 7 349382. IBRAGIMOV, I. A. and ROZANOV, Y. A. (1978). Gaussian Random Processes. Springer, Berlin. KOLMOGOROV, A. and ROZANOV, Y. A. (1960). On a strong mixing condition for stationary Gaussian processes. Theory Probab. Appl. 5 204-208. LEONOV, V. P. and SHIRYAEV, A. N. (1959). On a method of calculation of semi-invariants. Theory Probab. Appl. 4 319-329. ROSENBLATT, M. (1956). A central limit theorem and a strong mixing condition. Proc. Nat. Acad. Sci. U.S.A. 42 43-47. ROSENBLATT, M. (1959). Statistical analysis of stochastic processes with stationary residuals. In Probability and Statistics (ed. U. Grenander) 246-275. Wiley, New York. ROSENBLATT, M. (1961). Some comments on narrow band-pass filters. Quart. Appl. Math. 18 387393. ROSENBLATT, M. (1974). Random Processes. Springer, Berlin. SLEPIAN, D. (1972). On the symmetrized Kronecker power of a matrix and extensions of Mehler's formula for Hermite polynomials. SIAM J. Math. Anal. 3 606-616. MATHEMATICS DEPARTMENT UNIVERSITY OF CALIFORNIA, SAN DIEGO LA JOLLA, CALIFORNIA 92093

374

Proc. Natl. Acad. Sci. USA Vol. 83, pp. 199-200, January 1986 Applied Mathematical Sciences

Deconvolution of non-Gaussian linear processes yrith vanishing spectral values (non-minimum phase/cumulant spectra/filter/noise) K E H - S H I N L I I * A N D MURRAY ROSENBLATT*

•Department of Statistics, University of California, Riverside, CA 92521; and tDepartment of Mathematics, University of California at San Diego, La Jolla, CA 92093 Contributed by Murray Rosenblatt, September 9, 1985

ABSTRACT We consider the problem of estimating the filter generating a non-Gaussian linear process and the deconvolution of that process when the spectral density of the process has zeros. Without using a minimum phase assumption we show that often if there are only finitely many zeros there are procedures to effect such an estimation and deconvolution.

by making use of the relation

Linear processes X, are generated by an independent identi cally distributed input sequence % convolved with a linear filter at

which is derived from thefcth-ordercumulant spectral densi ty of the process {Xr}

Xr = Z, ajTjt-j.

*(Xi) + • • • + *(X*-i) - MXi + k

arg

\a(X)]

ll«(i)IJ

-1

yk

WXr

+ x*_i) ., x*-i)

[2]

6*(\i, • • •» k*-i)

[1]

= 7T*W a(*-,Xl) ' • • a(*-'x*-1)a(*'(Xl+-+x*-«).

[3]

\LTT)

This model is of considerable interest in geoexploration (14). Given observations on X, only, a main interest is to esti mate the filter coefficients at and the input sequence TJ, by deconvolving X,. Let a(z) = %ajZJ be the z-transform of the filter at. Then the spectral density of {Xf} is /(A) = o%\a{e~iK)\2/2TT, which is completely determined by the second-order properties of {X,} and does not contain the phase information of a(z). If TJ, has a Gaussian distribution then the process {X,} is complete ly determined by second-order properties and one can only determine a(z) up to \a(z)\. There is the indeterminancy of the phase of a(z). Often this is resolved by making the ad hoc assumption that a(z) is minimum phase. If a(z) is polynomial this means the roots are outside the unit disc. Often, 77, appears to have a non-Gaussian distribution and then one need not assume a(z) minimum phase. By using information provided by the higher-order (greater than 2) cumulants, the phase of a(z) can be estimated. Results using higher-order cumulant spectral techniques have been given by Rosenblatt (5) and by Lii and Rosenblatt (3, 6). A main result is summarized below. LEMMA. Consider a non-Gaussian linear process Xt given in Eq. 1 withtyhaving all their moments finite. Let 2|joj| < » and assume a(e~lX) ^ 0 for all A. The transfer function a(e~lX) can then be identified in terms of observations only on {Xt} up to an indeterminate integer a in a factor eiaX that corresponds to a shift of index of Xt and an indeterminate sign of a(l) = Xak. For this result it is sufficient to have some finite moment of order k > 2 with cumulant y^ ^ 0. The basic idea is to obtain the modulus of a(e~ik) using the spectral density /(A) of {X,} [since \a(e~ik)\ = \2
Given an estimate bk of bk one can obtain an estimate h(\) of h(\). a{e~ik) = |dU?~/x)|e*fc> is an estimate of a(e~ik). One can deconvolve X, and obtain estimates of bk by using 6Tl * X,.

[4]

THE RESULTS In many contexts the spectral density /(A) does have zeros and it is of some interest to see whether one can carry out deconvolution and in the case of this model recover the origi nal non-Gaussian white noise sequence 17,,. We have already remarked that, under the assumption of the lemma, Condition A is valid. Condition A. The function a(e~,x) can be consistently es timated up to a factor etak with a an integer and the sign ± 1. In the results we mention, the important details of how to estimate /(A) and a(e~ix) will not be dealt with in any detail here. Some of the aspects of estimation of a(e~,x) can be found in the papers by Lii and Rosenblatt (3, 6). One should note that Condition A will be satisfied under the following assumption concerning a(e~'k). PROPOSITION. Ifa(e~lk) is the restriction to the unit circle of a function a(z) # 0 analytic in an annulus about the unit circle, Condition A is satisfied. Such a function can have at most a finite number of zeros on |z| = 1. Under this assumption the procedure suggested via equa tions 2 and 3 of the paper for the estimation of a(e~lk) will still be asymptotically valid (see additional remark at end of paper). THEOREM 1. Under Condition A the process Xn can be deconvolved so as to obtain the process Tjn up to sign and a time shift by making use of the filter corresponding to {f„(X) + an}"1/2 exp{-\ arg an(e" K)},

[5]

lK

where fn(A) (5* 0), an(e ) are appropriate consistent estimates o/f(A), a(e _lX ) based on a sample size n and an an appropriate sequence of positive numbers with limit zero.

199

R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_34, © Springer Science+Business Media, LLC 2011

375

200

Applied Mathematical Sciences: Lii and Rosenblatt

Details, particularly those concerned with estimation and derivation of results, will be provided later.

We shall also consider the case in which the process xn has added to it Gaussian white noise & with small variance a > 0. We thus have the observed process £, = *„ + & = / _ e

+ f

J-ir

irik

einkdzt(\),

Proc. Natl Acad Set USA 83 (1986)

This research was partially supported by Office of Naval Research Contract N00014-81-K0003 and National Science Foundation Grant DMS83-12106.

ik

a(e- )dzv(k)

1. 2. 3. 4.

Wiggins, R. (1978) Geoexploration 17, 21-35. Jurkevics, A. & Wiggins, R. (1984) Geophysics 49, 2109-2116. Lii, K. S. & Rosenblatt, M. (1982) Ann. Stat. 10, 1195-1208. Matsuoka, T. & Ulrych, T. J. (1984) Proc. IEEE 72, 14031411. 5. Rosenblatt, M. (1980) J. Appi Probab. 17, 265-270. 6. Lii, K. S. & Rosenblatt, M. (1985) Multivariate Analysis (Elsevier, New York), Vol. 6, pp. 395-410. 7. Priestley, M. B. (1981) Spectral Analysis and Time Series (Academic, New York).

M

with z„(A), Z|(A) random spectral processes corresponding to the processes 17 and £ THEOREM 2. Under Condition A and 2[jaj| < °°, the process X n can be deconvolved so as to obtain the process r^ up to sign and a time shift by applying the filter corresponding to Eq. 5 to the process £„ and letting n -> ».

376

/ . AppL Prob. 24, 328-338 (1987) Printed in Israel ® Applied Probability Trust 1987

SCALE RENORMALIZATION AND RANDOM SOLUTIONS OF THE BURGERS EQUATION M. ROSENBLATT,* University of California, San Diego

Abstract Solutions of the Burgers equation with a stationary (spatially) stochastic initial condition are considered. A class of limit laws for the solution which correspond to a scale renormalization is considered. STATIONARY INITIAL CONDITION; SHORT-RANGE DEPENDENCE; LONG-RANGE DEPENDENCE

Introduction The Burgers equation (1)

ut + uux = fiu^,

(i>0,

u = u(x91) is a simple example of a non-linear partial differential equation that has been suggested as a simple model for turbulence (see Burgers (1948)) as well as for shock waves when //10 (see Whitham (1974)). Spatially stochastic solutions of the Navier-Stokes equation have been used in discussions of turbulence (see Batchelor (1953)) and Vishik et al., (1979)). We shall consider solutions of the Burgers equation with spatially stationary stochastic initial conditions. Both Hopf (1950) and Cole (1951) have shown how the discussion of the Burgers equation can be reduced to that of the diffusion equation by a simple non-linear transformation. We shall give Hopf s formulation in the following theorem. Theorem 0 (Hopf). Let u(x,t)bea solution of Equation (1) with continuous

Received 25 November 1985; revision received 22 April 1986. * Postal address: Department of Mathematics, University of California, San Diego, La Jolla, CA 92093, USA. This research is supported in part by Office of Naval Research Contract N00014-81-K-003 and National Science Foundation Grant DMS 83-12106.

328 R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_35, © Springer Science+Business Media, LLC 2011

377

329

Scale renormalization and random solutions of the Burgers equation

initial condition uQ(x) = u(x, 0). / /

f* Uo(Z)dt = o(x2)

(2) as \x\ -*oo, then

u(x, t) =

i:

x-y

r.

^-F(x,y,t)

exp

exp

2fi

dy

F(x, y, t) dy

with {

F(x,y,t) =

^-r^-\ It

J\(Z)dZ.

Notice that if we set 1 cy Tu4y)dy 2u Jo

and

J

oo

(* - y)2

dy

- 00

then (3)

u(x, t) = — 2/z — log (p(x, t). ox

Random solutions. We shall be interested in the case in which UQ(X) is a continuous and stationary process in x. To use Hopfs representation of the solution and its consequences, we shall require that the stationary process UQ( •) satisfy (2). For the moment assume that this is the case. Simple conditions on the process w0( *) that imply (2) will be given a bit later. Our interest is in the asymptotics of

(4)

I u(x, t)dx = — 2/ilog

P(0, 0

as R -* oo with t > 0 fixed. Notice that
with 378

_

rfv

330

M. ROSENBLATT

1 CR 1 1 CR+y
u0(y)dy

so that log
1 CR -— Uo(y)dy 2/i Jo

(5) f

Too

1

rR+v

+ log^(47r//0- 1/2 exp - [ J -oo L 2fi JR The following simple remark can be made.

Uo(y)dy-

v2

— dv\ 4/it j

Proposition 1. Ifu0(x) is a continuous stationary process in x that satisfies (2), then the process u(x, t) is stationary in xfor each fixed t>0. The proposition can be seen to be valid in the following way. If (2) is satisfied by w0( • )> the expression (6)

J

00

J

p*+V

yl

dv

exp - — ujyydy- — -00 III Jxthat is jointly stationary 4fit is well defined and determines a process in x with w0( •). On differentiating log (p{x, t) with respect to x and making use of (4) and (5), one sees that W(JC, /) is stationary in x. As an immediate consequence of the ergodic theorem we have the following remark. Proposition 2. Ifu0 is a stationary process with E\u0(x)\

then

(7)

f\(y)dy

= 0(x)

as \x | -* oo and consequently (2) is satisfied. Under the assumptions of this proposition x Jo with probability 1 as | JC | -* oo where J is the tr-field of invariant sets for the process w0( • )• However, this immediately yields (7). Let us now consider stationary processes w0( •) with finite second moment 379

Scale renormalization and random solutions of the Burgers equation

331

Eu0(x)2. For most such processes the variance a2(R) = var( J Q u0(y) will diverge as R -* oo though it is of course easy to construct processes for which this variance remains bounded as i?-»oo. Nonetheless, the most interesting processes are those for which the variance diverges and we shall assume that this is the case. Consider the scale renormalization Ry

a(R)~l f "u(r\,t)dr\ Jo

as R -* oo. Given a process z(x) consider >Ry

h(R)~l f y{z(x)-c}dx

= TR GO

where h(R)^ oo as R -* oo. We shall say that TR(y) converges weakly (to some process W{y)) if for any given continuous functional y/ on C[0, 1] the distribution of V(TR(y),0
(8)

e(R)~l

(9)

(j(R)-1 f

both converge weakly asR^cc, process.

{u0(x)-Eu0(x)}dx, *{u(x,t)-Eu0(x)}dx they must converge weakly to the same random

The theorem follows from the stationarity of (6) in x for each t > 0, together with relations (4) and (5). Theorem 1 is of interest because it implies that the low frequency behavior of the spatial spectrum of u(x, t) in the immediate neighborhood of 0 is the same independently of / > 0 and is the same as that of u0(x). Remarks suggesting something like this have been made in the case of homogeneous turbulence (see Batchelor (1953), Section 3.3). 380

332

M. ROSENBLATT

We shall mention a number of conditions on the process UQ( •) (the initial condition) that allow one to determine the asymptotic behavior of (8) and (9) as R -* oo (but with fi > 0 and /fixed).Let

^

= ^{W 0 (HO,H>^.X},

be the a-fields of events generated by UQ(W\ W ^ X , and UQ(W\ W^X, respectively. Thus, Mx and &x correspond to the spatial information to the left of Jt, and the spatial information to the right of x for the process HQ( •). Let a(d\ d > 0 be the coefficient a(d)= sup

Aeax Berx+d

\P(AnB)-P(A)P(B)\.

Notice that a(d) is independent of x because of the assumption that HQ(X) is stationary in x. The process u0( •) is said to be strongly mixing if a(d)^0 as d -* oo. The strong mixing property is one of the many non-equivalent ways of formulating a concept of short-range dependence. Let UQ( •) be stationary with Euo(x) = 0, (10)

E\uo(x)\2+S
with 0 < d ^ 1, and strongly mixing with

a(rf) = Or ( 1 + i X 1 + ^) for some 0 < e ^ 1. If
sup

f^L\m x) 2 geL (^x+d)

|corr(/,£)|

with corr(/,g) the correlation of/and g. The process WQ(-) is said to be asymptotically uncorrelated iffi(d)-* 0 as d -* oo. The asymptotically uncorre381

Scale renormalization and random solutions of the Burgers equation

333

lated condition is an even stronger formulation of a concept of short-range dependence than is that of strong mixing. Theorem 2. Let u0( -)bea stationary process that is asymptotically uncorrelatedx satisfies (10), and is such that a(R )-»ccasR-+cc.It then follows that (8) converges weakly to a Brownian motion process B{y) asR-*oo. Theorem 2 follows directly from our Theorem 1 and a result of Ibragimov (1975) concerning asymptotic normality for asymptotically uncorrelated processes. It is easy to see how to modify these results if the mean of u0( •) is non-zero. Let us now consider a special class of processesty>(•) as initial condition which imply a weak limiting process different from the Brownian motion. These processes all exhibit a long-range dependence as contrasted with those processes dealt with earlier. This long-range dependence shows itself in terms of a singular mass accumulation in the neighborhood of 0 for the spectrum of the process w0(-). AH the processes w0(-) we exhibit are obtained by an instantaneous function of a Gaussian process. Let G(x) be a stationary Gaussian process with mean zero and covariance function (11)

r(x) =

EG(0)G(x)^x-DL(x)

as x -* oo, L( •) slowly varying and 0 < D < 1 For convenience, assume that r(0) = 1. Let h( •) be a function with Eh(G(x)) = 0, (12)

Eh(G(x))2
Our object is to take the initial condition for the Burgers equation u(x, 0) = uQ(x) a process of the form (13)

u0(x) = h(G(x)).

Consider the Fourier-Hermite expansion of h{ •) 00

h(u)= I

J(a) ^fhj(u)

and let m be the first index with non-zero coefficient 7(m). The Hermite polynomials hj(u) have leading coefficient 1. We shall be interested in the range 0
334

M. ROSENBLATT

the asymptotic behavior of (8) will be the same as that of (9). Now

O\R) = E( JRh(G(x))dxY \ m\ ) ( 1 - ml>)(2 - mD)

R2~mDLm(R)

as R —■ oo. Let W{ •) be the complex Gaussian process with EW(A)=0 W(A) - W{ - A) for A an interval, 71

1-1

for disjoint intervals A/, and covariance £(^(A0^^))=|A1nA2| (| A | is the length of A). Let the symbolJ^ denote integration over all of m-dimensional Euclidean space except for A,- = ± A,-, i ¥*j, i,j = 1,- • •, m. We introduce the process

Zm(x) = C(m,D) f

' exp (i(Xx + • • • + Am).x) - 1

Jn

X lA,^-^. • • lAJ^-^JWi)- • • *Wm) with a

im

m '

j

f 0 " ^ X 2 - m D ) ]1/2 l2(m!X2r(D)cosDw/2rJ *

Notice that the process is Gaussian only in the case m = 1. Theorem 3. Lef w0( •) 6e f/*e proem (13) satisfying (11) and (12). 7%ew rRy

l

a(R)~

Jo

w/// converge weakly to the process zjy) as i?-*oo. 383

u(x,t)dx

Scale renormalization and random solutions of the Burgers equation

335

Theorem 3 follows directly from our Theorem 1 and results of Taqqu (1981) (his Theorem 3.2) and Dobrushin and Major (1979) (their Theorem 1). The limiting process Zm{y) is Gaussian when m = 1 and is then a fractional Brownian motion. A counterexample for a growth conditon. The growth condition (2) has played an important part in our discussion. It is of some interest to give a simple example of a stationary process u0( •) that does not satisfy the condition. The process will understandably not have moments in view of Proposition 2. It will be defined in terms of a stable symmetric process £( •) with coefficient a and with stationary independent increments. Let F(y) be the distribution function of £(1). It is well known that y as y -» oo with some constant c> 0 and 0 < a < 2. The characteristic function of £(1) is exp [ - | u |"]. Since the characteristic function of £(r), r > 0, is exp [ - r | u T] it follows that the distribution function £(T) is F{x~my) with l-F{x-my)*-a y as y -* oc. Let #(0 be an increasing continuous non-negative function defined for t > 0. Now g(t) is called an upper function for £( •) if (14)

P\ lim sup£(Ote(0
pjlim sup£(0/£(0>U=lt-*

oo

There are sufficient conditions for #(0 to be an upper or lower function for £( •) (see Gikhman and Skorokhod (1969)). A sufficient condition for g( •) to be an upper function for £(«) is (15)

j ' " -tP{Z(t)>g{t)}dt<

oo.

A sufficient condition for g( •) to be a lower function for £( •) is that (16)

|

P{$(ak)>g(ak)}

diverge for all a > 1. By using (14) and (15) it is clear that 384

336

M. ROSENBLATT

g(t)-tVaQ.Ogt)Ua+' or g(t) = f1/o(log 0,/adog log t)Ua+< are upper functions for 0. Similarly (14) and (16) imply that g(t) = tl/a(loglogt)Ua or £(0 = f""(log f)""(log logO""

are lower functions for <^( •). We shall now generate a stationary process «o( •) with continuous sample functions from the stable process
Uo(x) = £

+ l

[$(w + 1) - Z{w)]dw.

Now JO

JO Jy

(18)

= fy[Z(y + D-Z(y)]dy+ f JO

(x-y)[$(y + 1)-Z(y))dy

JJC-1

+ f*~i[t(y + D-Z(y)]dy while

(19)

f J 1

' K(y + i) - t(y)Viy = f JJC-1

Z(y)*y - FS(y)dy. J1

From (18) and (19) it follows that the functions g(x) mentioned above as upper and lower functions for £(x) as x -* oo are also upper and lower functions for J o Uo(y)dy with w0(-) given by (17). Notice that this implies that (17) satisfies the growth condition (2) if a > \ but does not if a ^ \. Varying \i. In the discussion of asymptotics in the earlier part of the paper one considered letting R -* oo in (4) but with JU and /fixed.It's clearly also of interest to look at the case in which i? -* oo and //10 but with tfixedor at least bounded. A simple estimate will show us that the asymptotics obtained earlier 385

337

Scale renormalization and random solutions of the Burgers equation

are still valid in such a circumstance if one can assume that for the initial condition one has the finite bound |w0(x)| ^c for all x with probability 1. For then — cv + — i f v ^ O It (20) J Uo(y)dy + 2 v It cv + — ifv^O. It This implies that an upper bound for (6) is given by f+x + v

1 >/An/it

(c2t exp —

exp \

\ 4//

(v-ct)2\, r° —— ) dv + j exp Ant

(v + ct)2 dv Apt

(21)

A lower bound for (6) can be obtained by noting that „2

Cx+V

(20)'

Jx

v

cv + — It

ifv^O

cv+ — It

if v < 0 .

u0(y)dy+—< 1

The lower bound for (6) is then y/4~njui

r.

%xp/_
\WLJO

exp f_(^iO! lrfv

Ant

ifc, l/2>I < i

i)

C\1tt

(c2t

exp {-J [I -OKD]

* ' ( £ ) % I-

The upper and lower bounds (21) and (22) for (6) together with the relations (4) and (5) imply that for a bounded initial condition w0( •) the results already obtained for R -* oo but with ji and tfixedare still valid when R -» oo, JLL 10 and t is bounded. 386

338

M. ROSENBLATT

Acknowledgment I should like to thank S. Karlin and the Mathematics Department at Stanford University for their hospitality during a sabbatical when this paper was written. References BATCHELOR, G. (1953) The Theory of Homogeneous Turbulence. Cambridge University Press. BURGERS, J. M. (1948) A mathematical model illustrating the theory of turbulence. Adv. Appl. Mech.h 171-199. COLE, J. D. (1951) On a quasilinear parabolic equation occurring in aerodynamics. Q. Appl. Math. 9, 225-236. DEHLING, H. AND PHILIPP, W. (1982) Almost sure invariance principles for weakly dependent vector-valued random variables. Ann. Prob. 10, 689-701. DOBRUSHIN, R. AND MAJOR, P. (1979) Non-central limit theorems for non-linear functionals of Gaussian fields. Z. Wahrscheinlichkeitsth. 50, 27-52. GIKHMAN, I. AND SKOROKHOD, A. V. (1969) Introduction to the Theory of Random Processes. Saunders, Philadelphia, PA. HOPF, E. (1950) The partial differential equation ut + uux ^fiu^. Comm. Pure Appl. Math. 3, 201-230. IBRAGIMOV, I. (1975) A note on the central limit theorem for dependent random variables. Theory Prob. Appl. 20, 135-141. TAQQU, M. (1981) Self-similar processes and related ultraviolet and infrared catastrophes. In Random Fields, ed. J. Fritz, J. Lebowitz, and D. Szasz, North-Holland, Amsterdam, 1057-1096. VISHIK, M., KOMECH, A. AND FURSIKOV, A. (1979) Some mathematical problems of statistical hydrodynamics. Russian Math. Surveys 34, 149-234. WHITHAM, G. B. (1974) Linear and Nonlinear Waves. Wiley, New York.

387

Biometrika (198S), 75, 3, pp. 477-84 Printed in Great Britain

On frequency estimation BY JOHN A. RICE AND MURRAY ROSENBLATT Department of Mathematics, University of California, San Diego, La Jolla, California 92093, US.A SUMMARY

/

This paper discusses a least-squares procedure and the use of the periodogram for isolating a discrete harmonic of a time series. It is shown that the usual asymptotics on estimation of frequency, amplitude and phase of such a harmonic have to be used with great caution from a moderate sample perspective. Computational issues are discussed and some illustrations are provided. Bolt & Brillinger (1979) make use of these asymptotic results. Some key words: Frequency estimation; Harmonic analysis; Periodogram; Time series. 1.

INTRODUCTION

We consider a time series model of the form Xt = a cos (wt + ) + et, where et is a stationary noise sequence, and one is interested in estimating the amplitude, frequency and phase of the harmonic component. The asymptotic theory of the least-squares estimates of these parameters has a long history. Whittle (1951,1953) obtained some of the earliest results. More recent results are by Hasan (1982), Hannan (1973) and Walker (1971), who formalize and extend Whittle's results. In these works it is shown that the asymptotic variance of the frequency estimate is of order n - 3 and that the asymptotic variances of the other two components are of the more usual order n"1. These results extend when there are several harmonic components. The rate for the estimate of
388

478

JOHN A. RICE AND MURRAY ROSENBLATT

especially severe if one is estimating the parameters of several harmonic components simultaneously since then the iteration is taking place in a high-dimensional space with many local minima. Walker (1971) suggests finding the maximum of the periodogram and then computing the amplitude and phase estimates. However, it turns out that this approach has some drawbacks. A bias can arise for moderate sample sizes that is appreciable compared to the standard deviation suggested by asymptotic theory. These issues are further discussed in § 3. There we also report the results of some modest simulations which support the qualms about asymptotic theory raised in § 2. The bulk of the paper deals with a general context where the procedures are suggested by the likelihood function computed for Gaussian residuals. A discussion of the likelihood function with residuals a Gaussian stationary sequence is given in the Appendix. 2. ASYMPTOTIC THEORY

We initially assume that X, = A0 cos ((o0t) + B0 sin (
-\ log \Rn\ - 3 £ {Xt - AQ cos (o>o0 - B0 sin (o>0f M ^ P ^ r - A) cos (CO0T) - B0 sin (O>0T)}, f,T-l

where Rn is the covariance matrix of the X's and r£ 1} is the (f, r) element of the inverse of the Toeplitz matrix Rn. Under a number of assumptions on the spectral density /(A) of the residuals eti one can give a heuristic argument showing that the part of the log likelihood above depending on (o0, after fitting A0 and B0, can be approximated by a constant multiplied by 1 1 In f(o)0) n\t-x

I2 |

We shall give a brief heuristic discussion of this in the Appendix. In the main body of the paper we shall restrict ourselves to the case in which the e, are independent and identically distributed N(0, v). In this case the log likelihood is -{n log {2irv)-\Sn(A, B9 a>)/v, with Sn(A, fl, t)}2. The maximum likelihood estimates of A, B9 o> are determined by minimizing the sum of squares S„. The maximum likelihood estimate of v is given by the minimum sum of squares divided by n. Suppose that one replaces Sn(A>fl,to) by t/n(A,B,o>)= £ X]-2

£ Xt{Acos(a>t) +

Bsin(a)t)}+ln(A2+B2).

The difference between Sn(A, B9 ) is O(l) as n-»oo if w +0, TT and this suggests that if Un is minimized instead of Sn the difference will be small for large

389

On frequency estimation

479

n if the true value w0 of a> is not zero or TT. The estimates A, B, n such that In(£n)=

max {/„(*>)}.

Following the notation of Walker (1971), the periodogram In() = £ e*"'{A0 cos (
(1)

At first the analysis will be carried out for A$ and B0 assumed constant. Later a corresponding computation will be considered where A0 and B0 may depend on the sample size n. Walker (1971) considers estimating the unknown frequency a>0first.In the asymptotic theory it is usual to assume that a>0 is estimated with an error o(n^). We shall finally see what happens to the estimates of the unknown parameters if a)0 is not estimated with this precision. First consider A 0 , B0 to be independent of n. Let D0 = \{AQ- iB0) and x* ( \

v

tut fexp{5i(n + l)M}sin(lmi)/sin(iii)

(0
Equation (1) can thus be rewritten 2

I r» !*/„(") = X e,eto' + 2 $ X ^e'^DoAf^w + oioJ + DoAfn^-oio)} r-l

Lf«l

+ \DoMn{c0 + wQ) + DoMn{a>-a>o)\2. (2) Let 0o<7r. Following Walker (1971) it can be shown that when a) = a)0 expression (2) is given to the first order by \D0Mn{Q)\2 =

W{Al+Bl)

which arises from the last term on the right of (2). It is clear that max {|Mn(
max {\Mn(
Under assumptions of short range dependence on the sequence {e,} (Brillinger, 1981; Turkman & Walker, 1984) one can show that

/I max \\y

n

eteitot\

\2\ = 0(n log n)*.

It then follows that max {|/ n ( w )-^" 1 (A 2 + B20)|Mn(^-W0)|2|} = O(nlogn)i.

(3) (4)

The argument given by Walker (1971) shows that if 5 > 0 is chosen small enough so that {sin (|5)/(^)} 2 > 1/TT2 then K(M)=

max., {/„(«)}

390

480

JOHN A. RICE AND MURRAY ROSENBLATT

using estimate (3). Since limpr{K(n, 5) } = 1 it follows that lim pr {n\mn - o>0| < 5) = 1,

rt-*ao

where a>n is the location of the absolute maximum of In(^ir. This itself suggests that if one wishes effectively to estimate the maximum one would have to sample the periodogram deterministically by a grid mesh with the mesh length o(n _1 ) as n-*oo. Because of the many local maxima referred to in § 1, it is clear that the use of a nonlinear least-squares procedure implemented by a recursive local computation has great disadvantages. Walker (1971) goes on to show that An9 B„, a>n are asymptotically normally distributed. This result is established by a linearization. It is part of the point of the present paper to note that this linearization only holds in a very narrow range. To examine the situation in which the amplitude of the harmonic component is small, let us now consider the analysis again where A$ and B0 can depend on n. Estimate (3) implies again that the first term on the right-hand side of (2) is 0(n log n)i We shall estimate the remaining terms on theright-handside of (2) over the range \ h9 where h decreases to zero as n -> oo. The second term on theright-handside of (2) is of order of magnitude 0(n log w)*|D0|{l +min (nh, l)//i} and the last term is 0{\D0\mm(nh9l)/h}2. These estimates imply that if |D 0 | 2 »(log n)/n asymptotically as n-»oo one will be able to estimate o>0 with an error o(n~l). Suppose that one had obtained an estimate a>n of u>Q by searching along a grid mesh of mesh length h which was not o(n~l) or that an iterative procedure had converged to a local maximum which was not within o(n~l) away from o>0. One would still be tempted to use the estimates An, Bn of the coefficients A0 and B0

Then Am-A0+i(Bn-B0)=l[D0MH(a>H

+ uo) +

^

Assuming that n-w0)-n}+O{n-*(\ogn)*}. n If (on-w0-cn~p,

for O < 0 < 1 , c + 0, then An-A0+i{Bn-'B0)

= -2Do+0{n-x+fi

391

+

n-\\ogn)-i}

On frequency estimation -A

481 A

-A

.A

and this suggests that instead of using the conjugate of D n , Dc„ = A„ + iBn as the appropriate estimate one ought to use —Dcn, However, if a>„-a>0~cn~\ one would obtain D 5 - D 0 = 2Do{cxp(|ic)sin(|c)/(ic)-l} + o(l). Since one could not determine how close con is to a>0 such relations are meaningless in obtaining an estimate of D 0 . We thus see that, if
In this section we consider some computational aspects and the results of some modest simulations for the simple model x, = a 0 cos (0) + ety where the noise sequence et is Gaussian with mean 0 and variance 1. We will see that certain difficulties arise already in fitting this simple model. As mentioned in the introduction, computation of the least-squares estimates is complicated by the presence of many local minima. Figure 1 illustrates this problem. It shows minus the residual sum of squares for one realization of the model above with n = 100, Oo= 1, a>0 = i, an(* o = 0 as a function of the variables a, 9 a>, for a = 1, -TT^^^TT and 0-3 ^ co ^0-7. A Newton-type algorithm iterating in a three-dimensional space runs a risk of converging to a local extrema. The situation is not as bad as it could be since the ridges corresponding to local extrema in frequency make a small angle with the axis and since the estimate of a is asymptotically uncorrelated with the estimates of frequency and phase. Nevertheless, it is clear that real problems can arise, especially if one were fitting two or more peaks simultaneously. The initial values provided to the search algorithm are thus critical; in particular initial values suggested by complex demodulation may be unsuitable because of the low pass filter employed in this process. We have found it necessary to use a large number of random starting points in our computations.

Fig. 1. Minus residual sum of squares; n = 100, OQ= 1,

A direct search of the periodogram at a fine grid of points substantially finer than that given by the frequencies 2nj/n used by the fast Fourier transform is thus appealing, but unfortunately has its drawbacks as well. In the previous section it was shown that the contribution to the expected value of the periodogram by the harmonic component is proportional to |Mn(o> + 6>0) + Mn(a> -
392

482

JOHN A. RICE AND MURRAY ROSENBLATT

contribution. For example, for n = 100 and a>0 = 0-5, the maximum of this expression occurs at co = 0-5013 rather than at 0-5. According to our simulations, if a = 8, the bias of co is consequently more than twice its standard error, and similarly for the estimate of phase. While this is noticeable, it is not as large as that caused by using a local rather than the global minimum of the sum of squares. As we will see below, the least-squares estimate does not appear to be similarly affected. We now turn to a discussion of some simulations. For the simple model above we took n = 100 and varied a as 0-5,1,2,4, 8. For each such model, 100 realizations were constructed, and the least-squares estimates were calculated by a modified LevenbergMarquardt algorithm, SNLSE from MINPACK. Because of the many local minima alluded to above we found it necessary to use many starting points to locate the global minimum. Thus 50 starting values for the algorithm were generated on each run as a + aiUi—l), co + 87r( U2—i)n, and 27rt/3, where the U{ were pseudo random uniform variates on [0,1]. The choice of 50 may have been unduly conservative, but the only penalty can have been in computer time. The proper choice of a range for the random starting points is more problematical. The fast Fourier transform computes the periodogram at frequencies which are multiples of 27r/fi, and such a periodogram would often be the starting point of an investigation of periodicities; we thus generate the starting values for co in an interval of width iir/n around co0 assuming that a crude guess is obtained from the periodogram. We assume that some knowledge, but not much, about amplitude is obtained from the periodogram, and that there is no information about phase. To compare the results to the asymptotic theory, we standardized the least-squares estimates by subtracting the true values and dividing by the asymptotic standard deviations. Table 1 gives the means and standard deviations of the standardized variables for each model. Table 1 has been supplemented by graphical analysis of the frequency distributions. Estimates of amplitude are positively biased and the bias becomes smaller with increasing amplitude. Table 1. Least-squares estimates of a, o>, minus true value divided by asymptotic standard error. Mean and standard deviation of resulting standardized variable a=\ a

0-94,

a=l 103

& -0-71, 6-98 4> 005, 1*34

0-45, 0-02, -005,

a= 2 1-30 1 26 1-31

0-31, 0-05, -0-08,

1-33 1-25 1-28

fl=4 0-24, 0-06, -010,

a=8 1-34 1-25 1-28

0-20, 007, -010,

1-34 1-25 1-28

When a = \ the frequency is not well resolved, and the asymptotics fail to give a reasonable indication of variability. Direct examination of periodograms shows that in this case the periodogram frequently has a local maximum near the true value, but larger maxima at other locations. In this case the contribution of the noise sequence to the expected value of the periodogram, the first term in (2), is 2 and the contribution from the harmonic component, the last term in (2), is {no1 = 12*5. At co = 0-5 the contribution to the variance of the periodogram from the product of the noise sequence and the harmonic component, the middle term in (2), is about (7-07)2, which is large enough to occasionally result in a false dominant peak. For larger amplitudes, the frequency resolution is in rough accord with the asymptotics, although there are occasional outlying values. Apart from some inflation of the standard deviation, the distribution of the phase estimates is about as predicted by the asymptotic theory.

393

On frequency estimation

483

APPENDIX

An approximation relating to the likelihood function for a Gaussian process We consider a Gaussian stationary process and note that if the spectral density /(A) is continuously differentiable and bounded away from zero the following approximations appear to be appropriate. The basic approximation is that of replacing r\~l) (/, r = 1 , . . . , n) by (Al) Aside from the term —5 log |i?„| the log likelihood can be written as £ X,XTr{-Tx)-2

f

X,{Acos(a>T) + Bsin(wT)}r(-l) + A2 £ cos (cut) cos (C^T)^;"

f,T-l

+ B2 £ sin(o>r)sin(ct)r)r^,) + 2AB £ cos (wt) sin (wT)r["r°. The third term, on using the approximation (Al) mentioned above, is to first order

r- 1 X cos (art) e* 4ir 2 .L/(A) A2

<*A

and if 0 < a> < ir this is approximated by \mT~lA2/f(o>). The same argument shows that the fourth term is approximated by \mr~lB2/f((o). The fifth term can be shown to be o(n) if 0<7r, since it is to thefirstorder

SDiite-*" 0 •"}{!, ,h<"') •""}*■ If we insert these approximations and minimize the resulting expressions as a function of A B> <*> the estimates of A, B9 o> resulting are A=

X X ^ c o s ^ r ) , fi = n

n

I^T-I

X X r rl T ,) sm(«T) >

i.T-i

where <*> maximizes 1

^[{iiX,r<;;'cos(Wr)}%{liX,r<;'>sin(a>r)}2].

(A2)

Again using the approximation (Al), it would appear to be the case that (A2) can be approximated by "/(<3)

I X,e'

From the same approximation, the estimates of AQ and B0 are given approximately by A = - £ Xt cos (a>t)9 B = - f X,sin(a5f).

ACKNOWLEDGEMENT

This research is supported in part by the Office of Naval Research and National Science Foundation of the U.S.A.

394

484

JOHN A.

RICE A N D M U R R A Y

ROSENBLATT

REFERENCES

BOLT, B. A, & BRILLINGER, D. R. (1979). Estimation of uncertainties in eigenspectral estimates from decaying geophysical time series. Geophysics. J. R. Astr. Soc 59, 593-603. BRILLINGER, D. R. (1981). Time Series: Data Analysis and Theory. Oakland, CA: Holden-Day. HANNAN, E. J. (1973). The estimation of frequency. /. AppL Prob. 10, 510-9. HASAN, T. (1982). Nonlinear time series regression for a class of amplitude modulated cosinusoids. J. Time Series Anal 3, 109-22. TURKMAN, K. & WALKER, A. M. (1984). On the asymptotic distributions of maxima of trigonometric polynomials with random coefficients. Adv. AppL Prob. 16, 819-42. WALKER, A. M. (1971). On the estimation of a harmonic component in a time series with stationary independent residuals. Biometrika 58, 21-36. WHITTLE, P. (1951). Hypothesis Testing in Time Series Analysis. Uppsala: Almqvist and Wiksells Boktryckeri. WHITTLE, P. (1953). The simultaneous estimation of a time series* harmonic components and covariance structure. Trab. Estad. 3, 43-57.

[Received August 1987. Revised January 1988]

395

F. J. Breidt, R. A. Davis, K.-S. Lii, and M. Rosenblatt. Maximum likelihood estimation for noncausal autoregressive processes. J. Multivariate Anal., 36(2): 175–198, 1991. Reprinted with permission of Elsevier Inc.

The Annals of Statistics 2002, Vol. 30, No. 1, 258-297

SPECTRAL ANALYSIS FOR HARMONIZABLE PROCESSES B Y K E H - S H I N L I I AND MURRAY ROSENBLATT

University of California, Riverside and University of California, San Diego Spectral estimation of nonstationary but harmonizable processes is considered. Given a single realization of the process, periodogram-like and consistent estimators are proposed for spectral mass estimation when the spectral support of the process consists of lines. Such a process can arise in signals of a moving source from array data or multipath signals with Doppler stretch from a single receiver. Such processes also include periodically correlated (or cyclostationary) and almost periodically correlated processes as special cases. We give detailed analysis on aliasing, bias and covariances of various estimators. It is shown that dividing a single long realization of the process into nonoverlapping subsections and then averaging periodogramlike estimates formed from each subsection will not yield meaningful results if one is estimating spectral mass with support on lines with slope not equal to 1. If the slope of a spectral support line is irrational, then spectral masses do not fold on top of each other in estimation even if the data are equally spaced. Simulation examples are given to illustrate various theoretical results.

1. Introduction. Spectral analysis of stationary processes has a long history with interest in both theory and applications. Wide-ranging applications in various practical problems in engineering, economics, science and medicine are well documented [see Yaglom (1987)]. These developments have been closely coupled with the stationarity assumption. In recent years there has been growing interest in the spectral analysis of various specific models of nonstationarity. A class of models of this type that have generated considerable interest are the periodically correlated (or cyclostationary, periodically stationary) or almost periodically correlated processes [Alekseev (1988), Hurd (1989), Gardner (1991), Gerr and Allen (1994a, 1994b), Dandawate and Giannakis (1994) and Leskow and Weron (1992)]. The stationary and almost periodically correlated processes are proper subsets of the class of harmonizable processes. A harmonizable continuous-timeparameter process X(t), EX(t) = 0, can be represented as a Fourier-Stieltjes integral [Loeve (1963)] oo

/

eitkdZ(k)

-00

Received December 1999; revised August 2001. AMS 2000 subject classifications. Primary, 62M15; secondary, 62G07, 60G12, 62F12. Key words and phrases. Spectral density function, harmonizable processes, estimation, bias, covariance, aliasing, cyclostationary, consistency, Doppler, multipath. 258 R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-l-4419-8339-8_38, © Springer Science+Business Media, LLC 2011

420

HARMONIZABLE SPECTRAL ANALYSIS

259

of a process Z(X) and the covariance function rttX = ( 1 2)

cov(X(t),X(T))

= / / ° ° eia~iXiX cov(dZ(l), = //

dZ(fi))

eitX-itlxdH{k,ix),

J J—oo

with the spectral function (13)

r f\dH(u,v)\ <

J — OO J

oo

of bounded variation. A readable discussion of the properties of harmonizable processes with many references is given in Yaglom (1987). In the case of stationary processes, one has a representation of the form (1.1) with Z(A) a process of orthogonal increments, E{dZ{X)dZ{ii)) = Sx^dF(k) and F a bounded nondecreasing function. Major interest focuses on the estimation of the spectral density f(k) = F\X) when F is differentiable. The concept of harmonizability in the sense of Loeve is a natural generalization that includes a large class of nonstationary processes. The spectral mass may not concentrate on the diagonal line k = JJL because the process Z{X) in (1.1) may not have orthogonal increments. A natural question concerns what one can estimate consistently about the spectral function H(k, /x) of a harmonizable process from a single realization X(t)9 \t\ 0) cov(X(0, X(r)) = R(t, r) = cov(X(t + T), X(r + T)). All discrete time para meter processes of this type are harmonizable but they are not locally stationary if they are periodic but not stationary. It should be noted that there are continuous-time-parameter periodic processes that are not harmonizable. Gladyshev (1963) gives the example R(t, r) = g(t)g(r) All

260

K.-S. LII AND M. ROSENBLATT

with g continuous periodic with period T but having Fourier coefficients that are not absolutely summable. If the Fourier coefficients are absolutely summable, the process with this covariance function is, of course, harmonizable. Let the number r be a translation number of the complex continuous function / belonging to 8 > 0 if s u p . ^ ^ , ^ \f(x + r) — f(x)\ < s. If for any e > 0 there is an 1(e) > 0 such that any interval of length 1(e) contains a translation number of / belonging to £, then / is almost periodic in the sense of Bohr ( / e B) [Bohr (1951)]. Any finite trigonometric polynomial J2%=i an oxp(iXnx) with the Vs real is in B. Furthermore, B is an algebra closed under uniform convergence. A process X(t) is said to have an almost periodic covariance function B(t,r) if B(t + s,r + s) is almost periodic as a function of s for each t,r. Most processes with almost periodic covariance functions that are not stationary are harmonizable but not locally stationary. The class of processes we consider is qualitatively broader than the processes with almost periodic covariance function and the nonstationary subclass is not locally stationary in general. In this paper we consider the estimation of spectra of harmonizable processes with spectral mass concentrated on lines on the basis of a time-ordered sequence of observations. Estimation of spectra on lines with slopes not parallel to the diagonal has been of certain interest in signal processing [see Allen and Hobbs (1992)]. Some preliminary results were announced in Lii and Rosenblatt (1998). Further remarks on where such problems arise are given after (2.7), where specific models are discussed. In Section 2 we lay out some basic properties relevant to the estimation problem and motivate the consideration of harmonizable processes with spectral mass concentrated on lines. In Section 3 a periodogram-like estimator is proposed and its bias is examined. The covariance properties of this estimator are given in Section 4. A consistent estimator is considered in Section 5. Remarks are made about the multivariate case. Section 6 has simulated examples and remarks on estimation and computation. Section 7 has most of the proofs. We note that, while the basic form of the estimator is a smoothing of a periodogram-like form, the analysis is quite different from that of the usual periodogram-based estimators. It is shown that averaging periodograms of nonoverlapping subsections of a long section of data will not produce a meaningful result when the slope of the line of the spectral support is not 1 in contrast to what happens in the case of stationary or almost periodically correlated processes. Results obtained include periodically correlated or almost periodically correlated processes as special cases for which much more detailed analysis and results are presented here than in the existing literature. In particular, the effects of aliasing are given. Extensive applications of these processes are well documented in Gardner (1994). 2. Preliminaries. Consider a zero-mean harmonizable process with the representation (1.1). If the process is real, then rtyX — rtit and is real. This implies that (2.1)

dH(u, v) = dH(-v, -u). All

HARMONIZABLE SPECTRAL ANALYSIS

261

This is symmetry with respect to u = — v. Also, dH(u, v) = dH(—u, —v) = dH(v, u). This is complex conjugate symmetry with respect to the point (0,0) and the line u = v. From this we see that information of H{u, v) in the quadrant given by {(u,v) | v > \u\} determines H(u,v) everywhere. If the process Xt = X(t) is stationary, the spectral mass is concentrated on the diagonal line dH(u, v) = Sv-UioxdF(u). If the process Xt is periodically or almost periodically correlated, then the spectral mass is concentrated on at most countably many lines which are parallel to the diagonal [Gladyshev (1963) and Dehay and Hurd (1993)]

(2.2)

dH(u,v) = J^ j

Su-V-W j ?o d Fj (u).

In this paper we assume the continuous-time harmonizable process X(t) has all its spectral mass concentrated on a finite number of lines of the form u = av + w, a > 0. We will discuss cases in which there is spectral mass on a set of twodimensional positive measure, on a curve which is not a straight line, as well as when a < 0. We are interested in estimation of the spectral density / a , w (/x) on a line X = a/ut + w: (2.3) dH(X,

H)\x=an+w = Sx-aii-wyodFa,w(tM)

=Sx-a(i-w,ofa,w(l^)dfM.

The special case with a = 1 was considered in Hurd (1989) and Dandawate and Giannakis (1994). Because of the symmetries in the case of a real-valued process, if u = a i ; + £isalineof spectral support, then so are u = a~lv — a~lb, u—av — b and u = a~lv +a~lb with fa,b(v)

= afa-K-a~lb(aV

+ b) = fat-b(~v)

= 0/ f l -l, f l -lfc(-<™ ~ b).

Sampling at the integers n, m, we have einldZ(k)=

einX J2 J n

~

dZ(X + 2jrk),

k=-oo

with r„,m = cov(Xn, Xm) = EXnXm = Jj*

e

^ ~ ^ dH(X, fi)

and (2.4)

dH(k, fi) = ^dH{X

+ 2jtk, /x + 2itj).

kj

From this it is seen that an aliasing problem (in the sense of spectral mass folding onto [—7r, 7r]2) occurs if the harmonizable process is broadband. It is well known that one generally does not have consistent estimates of spectral mass for a harmonizable process when H is absolutely continuous with a 423

262

K.-S. LII AND M. ROSENBLATT

spectral density h, dH(X, JJL) = h(X, fi) dXdfi with h(X, JJL) ^ 0 on a set of twodimensional Lebesgue measure, if the sample is X-n, ...,Xn, letting n -> oo. The simplest example is given by Xo ~ N(0,1) with Xk = 0 for k ^ 0, where dH{X, fi) = C dXd/ii for some constant C. The most one could possibly hope for is to estimate spectral mass whose support has two-dimensional Lebesgue measure 0. The simplest such support is a finite collection of curves and, in particular, lines. This motivates the consideration of estimation of spectral mass concentrated on lines given a single realization. A simple example of a harmonizable process with spectral mass on lines is given by k

(2.5)

Xt = Yt + YtPsYa,t,

with Yt stationary and fis and as real and positive numbers, respectively. In such case the spectral support lines are X = //,, X = asfi, X = a~l[i and X = asaj, \x for all s, s; = 1, ...,&. In Section 6 we give a slightly more complicated example where it is shown that the spectral support lines have the given form. A small variant of this model is given by Xt = Y^s=\fisYas(t-rs) • Such processes can be encountered in contexts when signals are of a multipath character and contain time delays rs and Doppler stretches as due to different propagation speed along different paths for a single receiver. For a multiple-receiver example, consider an acoustic signal that emanates from a moving point source and propagates through an acoustic medium, before being received at two spatially separated sensors. If, for instance, the source is in motion while the sensors are stationary, then the actual waveform observed at the two sensor locations can be modeled by [Ferguson (1999), page 261].

xi(t) = s(Pit) + vi(t), x2(t) = as(P2(t-d))

+ V2(t),

where v\(t) and V2(t) represent additive noise observed in the absence of signal, d is the time delay, a is the relative attenuation and f}\, ^2 are the respective receiver time scales that are introduced to account for the differential Doppler effect. In the present case, the relative motion between the source and the sensors results in the radial velocity component of the source at any given time being different for each receiver. An example of aircraft transits overhead illustrating this model is given in Ferguson (1999), page 261. Alternatively, the received waveforms can be modeled by

x(t)=s(t)+nx(t), y(t)=als(P(t-r))

+ ny(t),

where /J = /J1//J2 [Ferguson (1999), (16)]. It is also demonstrated in Ferguson that if the Doppler effect is ignored then the estimated delay will not be correct. 424

HARMONIZABLE SPECTRAL ANALYSIS

263

Equation (16) of Ferguson (1999) is a version of a special case of the model we dealt with (see the remarks at the end of Section 5 on multivariate harmonizable processes). Additional examples can be found in Jin, Wong and Luo (1995), page 904, for a wideband communication example and in Munk, Worcester and Wunsch (1995), page 204, for one in ocean acoustic tomography. A related example can be found in Chiu (1986) for seismic applications. Throughout the paper we shall assume that X(t) is a continuous-time-parameter harmonizable process with: ASSUMPTION 1. All spectral support is on a finite number of lines given by X = {(ai,bi)\u = a(V + b(, i = 1 , . . . , K} with positive slopes a; > 0.

2. The spectral mass on the line u = av + b is given by a continuously differentiable spectral density fa,b(v),(a,b) e X. The spectral densities fa,b(v) a n ^ their first derivatives are continuous and bounded in absolute value by a function g(v) that is a monotonic decreasing function of |t>| that decreases to 0 as \v\ -> oo and that is integrable as a function of v. ASSUMPTION

The following simple remark indicates how a harmonizable process with line spectra may differ from a stationary or almost periodically correlated process in terms of aliasing. The aliasing then can have a more complicated character. We say that a line u = av + b is a line of spectral support if the spectral density fa,b(v) on the line is nonzero on a set of positive one-dimensional Lebesgue measure. PROPOSITION 2.1. Let X(t) be a continuous-time-parameter process con tinuous in mean square satisfying Assumptions 1 and 2. The discretely observed process Xn then has a countably infinite number of lines of support in (—n, n] if and only if one of the lines of spectral support ofX{t) has irrational slope a and its spectral density is nonzero at an infinite number of points Vj, with \VJ\ -> oo. Furthermore, if there is a line of spectral support with irrational slope with spec tral density nonzero at all v, \v\ > s for some s > 0, then the discretely observed process has a countably dense set of lines of support in (—it, n] . Before we prove the proposition, we set the following convention which will be followed in the paper. When we refer to w = zmod27r, it is understood that —it <w<7t. Let {u} be the integer / such that —1/2 < u — I < 1/2. Our version of zmod27r is then (zmod27r) =z — {z/(27r)}2jr. 2.1. The proposition follows from the following simple remarks. Let u = av + b be a line with nonzero spectral density for the continuous-time process. The aliased line segments of this line, in the discretely sampled case, in (—7t, n]2 all have slope a. Let us look at the intersections of these line segments with v = vo <E {—kit, kit], —IT o, — 7t < u < it. If a is rational there are at most a finite number of values {litna + CIVQ + b) mod 2TT for n integer. Notice that if a = 1 there are at most two such aliased line segments. If a is irrational there are an infinite number of values {litna + avo + b) mod27r for n integer and they are dense on (—kit, kit]. Since VQ is an arbitrary point in (—kit, kjt], the aliased line segments in (—JT, JT]2 are dense. Now the proposition follows using Assumptions 1

and 2.

□

We note that the previous argument also implies that spectral mass with its support on a line with irrational slope never folds back on its own aliased line segments. Equally spaced sampling of a continuous-time process does produce aliasing. In the stationary case the "Nyquist" frequency is the frequency such that spectral mass beyond that frequency folds back on spectral mass of a lower frequency. In the harmonizable case aliasing does not produce a Nyquist frequency in the conventional sense for a spectral density whose spectral line support has irrational slope. Spectral masses of two different frequencies do not overlap due to aliasing in the case of a spectral density with support line having irrational slope. In this sense there is no overlap "aliasing problem" for spectral density estimation if the support line has irrational slope. The effect of aliasing on estimation in almost periodically correlated processes has not appeared in the literature and is given as a special case of our analysis in Sections 3-5. 3. A periodogram and its bias. In the classical case of a stationary process, an estimate proposed by Daniell (1946) was obtained by dividing a sequence of observations Xo, . . . , X # - i into m disjoint sections of length n, N = mn, computing a periodogram for each section and averaging the resulting periodograms. On letting m, n —► oo as iV -> oo, a consistent estimate is obtained under appropriate conditions. If a harmonizable process has mass on a line of slope a / 1, one object of our discussion is to show that a Daniell-like estimate cannot be used to estimate the spectral mass on that line. To show the effect relative to a Daniell-like estimate in sampling from the continuous-time harmonizable process X(t), —oc < t < oo, let us consider observations Xt+u with t — 0 , . . . , n — 1 with J,n > 0 integers and / an index value. Let n-i

(3.1)

Y/Xt+lJe-i^lJ)\

Fl,n(k) = t=o

with corresponding periodogram (3.2)

//,„(*, /x) = - — Lixn

FUn{X)Fln{^).

Notice that, for J = n, F/)W(A.) uses nonoverlapping subsections of X(t) for different integers /. When / < /i, we have overlapping sections [Zurbenko (1986), 426

265

HARMONIZABLE SPECTRAL ANALYSIS

page 214]. Given a sample XQ, . . . , X#_i, N = mn, a Daniell-like estimate is, when / = n, the average of periodograms of nonoverlapping subsections of the data, m—1 1 (3.3) 1=0

Another concern is to show the effect of aliasing that is the consequence of discretely sampling a continuous-time-parameter process. In the case of a stationary process, one tries to correct the aliasing by either narrowband filtering or discretely sampling more frequently. Similar methods can be used for the processes we deal with, but it should be noted that the effects of aliasing are more complicated than they are for a stationary process. Given (k,/i) e (—7r,7r]2, let (a, w) be a fixed pair with a > 0 such that k = a\i + w. Then set y= (3.4)

y(k',a,b;k,fi)

= ((2irka + (a — a)fi + b — w) mod27r) = {{Inka + a/jL + b — k)mod27z)

for (a,b) e X with k an integer. The meaning of some aspects of y will be remarked on later. One can then obtain the following theorem. THEOREM 3.1. Assume Assumptions 1 and 2 hold. Then

EIi9n(k9fi)=

J^ (a,b)e£

(3.5)

Yl

HJy(k;a,b;k,[A)

\y(k;a9b;X.,(i)\
x faMt* + 2nk)G(n, /, 7, y(k; a, b\ k, /x))

+0

logrc"

n uniformly in (k, ji) with a fixed v e (0, 7t] and G = G(n, /, J, y(k; a, b\ k, //)) 1 f°° = - I

ho,a](t)l[-U(a-l)/n,l-U(a-l)/n](t)einyt'a

dt

a J—oo (3.6)

U{a-\) — exp( my mini a, 1 a (iny ( U{a-l)\\ — exp I — max 10, ) I if (0, a) n

0,

U(a-1) , -,1

otherwise,

where 0 is the empty set. 427

U{a-\)\ ~)^0,

266

K.-S. LII AND M. ROSENBLATT

We note that for an error term like that in (3.5) we need v fixed and cannot let V|0.

The theorem gives a detailed estimate for the expectation of this type of periodogram. It is worthwhile making a few remarks about the theorem. First, notice that if (a, b) e X and y(k\ a, b\ X, /x) ^ 0 for a fixed k the contribution to the expectation from that term in the double sum on the right-hand side of (3.5) [see (3.6)] is 0{\/n). For an {a, b) e X, the condition y = {{Inka — X + {a\i + b)) mod27r) = 0 for some integer k means that the line of spectral support u = av + b or an aliased version of it passes through the point (A,, /x). From (3.4) we also see that y = 0 implies 2nka + a\i + b — (a/x + w) = 0mod27r. Notice that if (a, w) e X with (a,b) = (a, w) and k = 0 then y = 0 is satisfied. This means that if (A, /x) is a point on a spectral support line then there is a contribution to (3.5) in the summation. If (a, w) £ X then y = 0 in (3.4) implies there exists (a,b) e X such that at least one of its aliased lines (i.e., at least one k) passes through the point (X, /x). So for a given point (X, /x) if it is not on any spectral support lines or their aliased versions then EIitn(X, /x) = 0(logn/n) + G(Af), where

G(M)=

J2

eilJyfa,b(ii

E

+ 27Tk)G{n, l,J,y)

(a,b)eX \y\M

for any fixed integer M from (3.4) and (3.5). However, \G(M)\ -+ 0 as M -+ oo by Assumptions 1 and 2. The rate at which G(M) -> 0 depends on the rate of g(v) —► 0 in Assumption 2. As a result we have: COROLLARY 3.1. Under Assumptions 1 and 2, if (X,fi) G [—it, it]2 is not on any lines given by (a, b) e X or their aliased lines, then EI^n(X, /x) —> 0 as n —> oo. For the term with a = 1, we have

G=—[einy-l]. iny Then, if y ^ 0, y = ((a/x + b — A.)mod27r) is fixed and independent of k and G = 0(l/n) whatever the values of / and / . If a ^ 1, then G -> 0 if /7/w -► oo as n —> oo. Also, if y(£; a, Zr, X, (JL) = 0, then, for a > 0, (3.7)

G = - | [ 0 , a] n [ - / / ( a - l)/n, 1 - / / ( a - l)/n]||,

with || • || denoting length. Notice that, if y = 0, then G 7^ 0 if and only if either 1—a 428

n

I—a

HARMONIZABLE SPECTRAL ANALYSIS

267

or a > 1 and

a IJ 1 < — < a—1 n a— 1

To guarantee that G ^ 0 for arbitrary a > 0, we need — 1 < /7/n < 0. Also notice that, from (3.7), 1 G = -min(a, 1) a if (i) a — 1 or / = 0 for any / or (ii) / e [—1,0] and J = n. This suggests that, for J = n and y = 0, any / e [—1,0] can be used in (3.1) and in these cases G terms in (3.5) have a simple form. However, for computational purposes, the following three forms of a finite Fourier transform, which correspond to the cases when J =n and / = 0, —1 and ~\, respectively, in (3.1), can be conveniently used, for any a > 0, n

(a)

Fn(k) =

^2xte-ia,

f=0

o (b) Fn(k)= Y,

(3.8)

x

te~if\

t=-n

n/2

(c)

Fn{X)=

J2

X

teitX,

t=-n/2

with a corresponding periodogram (3.9)

1

In(k, fi) =

Fn(k)Fnfa).

27t(n + 1) We will use the form (c) given by (3.8) later and in such cases we take n to be even. COROLLARY 3.2. Assume Assumptions 1 and 2 hold and J — n with I € [—1, 0]. Then, as n —> oo, (3.10)

EIi9n(k9ti)^

J2

J2

(a,b)e£

y(k;atb;k,tJL)=0

/ f l f ^ + 27rfc)-min(fl,l).

Notice that if there is no aliasing the right-hand side of (3.10) becomes Y) (a,b)eX

Y)

/ fl ,&(/x)-mn(a, 1).

y(k;a,b;k,fji)=0 429

268

K.-S. LII AND M. ROSENBLATT

Since we assume only a finite number of lines of spectral support, there are at most a finite number of points of intersection of these lines. Aside from these points, the mean value tends to /«,u;(M)-min(a, 1) a if X — aix + w is a line of spectral support. In the case of a Daniell-like estimate (3.3) (note J = n), where / takes on integer index values, if a / 1 then, except for a fixed number of terms, \l(a — 1)| > 1 and so the corresponding G [see (3.7)] value is 0. Alternatively, we saw earlier that, to guarantee G ^ 0 for arbitrary a > 0, we needed / e [—1,0] when J—n. Consequently, the mean of the Daniell estimate (3.3) which averages over periodograms of nonoverlapping subsections of the data will generally not tend to fa,w(') &sm,n ^ oc even when a < 1. If a process is almost periodically correlated, the spectral mass is located on lines of slope 1 and y(k; 1, b; /x + w, /i) = (b — w)mod2n. The following corollary holds. COROLLARY 3.3. If the process is almost periodically correlated, under Assumptions 1 and 2 one has EIi9n(k9ti)=

J^

E

jUyWMiL+w)

(l,b)eX \y(k;l,b;ix+w,fM)\
(3.11)

xfi9b(fi

+

27tk)^[einy-l] iny

As n -^ oo, the expected value tends to

(3.12)

J2 (l,b)eX

J2

/!,*(/* +2TT*).

y(k;l,b;fi+w,fi)=0

Notice that y(k; 1, b\ \± + w, /x) = (b — w) mod2jr from (3.4). From the previous discussions we see that the expectations of the periodogram from an equally spaced discrete sample [see (3.1)] asymptotically converge to the "aliased" version of the spectral density function on the support lines. This is analogous to the stationary case where the expectation of the periodogram converges asymptotically to the aliased version of the spectral density function. The difference is that the aliasing is more complicated for the harmonizable processes than for the stationary processes. Bias and consistency properties of an estimator based on discrete samples are properties relative to these aliased spectral densities. We also note here that when the spectral support line has slope a < 0, previous and subsequent discussions and results still hold if we use the 430

269

HARMONIZABLE SPECTRAL ANALYSIS

symmetrized version (c) in (3.8) with very minor modifications in the derivations. We have degenerate cases when the slope of the spectral support line is a = 0 which is symmetric to the line with slope a - 1 = oc. It is also plausible, based on the discussions in this section, that if the spectral mass is concentrated on a smooth curve, similar results on the expectations and covariances should hold. These details are not pursued here. 4. Periodogram covariance. Our objective is to now give the asymptotic behavior of the covariance of the periodogram. Given fixed (A, /x), (A7, /x'), assume that the fixed pair (a, w), (a\ wf) with a > 0, a' > 0 is such that A = a/ui + w and A; = a''jAf + u/. Let (a, b), (a'\ b;) e £. To state the result in a convenient form, the following expressions are introduced: (4.1)

y(l) = (y(k; a,b\X, A')) = ((2nka + al! + b - A) mod2jr),

(4.2)

y(2) = (y(k;; a\ b;; -/x, -/x ; )) = ((2TT&V - a'p' + /x + &;)mod27r),

(4.3)

y(3) = (y(k\ a, b\ A, — /x;)) = ((2jtka — ay! + b — k) mod2jr),

(4.4)

y(4) = (y(kf; a, bf; -/x, Xf)) = {{2nkfaf + a;Xf + [i + bf) mod2jr).

This additional notation is introduced so that formulas later on do not become too cluttered. Notice that in (4.1)-(4.4) two lines X = a/x + w, X1 = a!yJ + w; are dealt with. The notation in this case is introduced to take care of the covariance computation. In formula (3.4) the bias of a point on a single line was involved. The covariance properties of the periodogram are given in the following theorem. THEOREM 4.1. Let X(t) be a normal harmonizable process satisfying Assumptions 1 and 2. Then cov(//,w(A., /x), Ii,n(X\ /x;)) V

■(a,b)e£\y(l)\
n

(4.5) X ■(a',b')e£ \y(2)\
V »

E E faM-^ + ^k)eilJy^G(nJ,J,y(3)) + + [(a,b)eX\y(3)\
X

E

E

fa',b^' + 2nk')eiUy^G{nA,J,y(A)) +

(a',b')eX |y(4)|
/

0^)] \

n

n

o{^^) V "

/

with —it < X,X.', fz, /A'
270

K.-S. LII AND M. ROSENBLATT

Theorem 4.1 will be useful in deriving the covariance properties of locally smoothed estimates of the spectral density on lines of spectral support. COROLLARY 4.1. The result of Theorem 4.1 holds for non-Gaussian harmonizable processes with finite fourth-order moments if the fourth-order cumulants satisfy \ sup^ Ylz^,^ |cum(Xf, XT, Xt/, XTf)\ -> 0 as n -> oo. This condition is satisfied if supf J2r,tf,rf |cum(Xr, Xz,Xtf, XTt)\ < oo. 5. Consistent estimates. The estimates that we consider are smoothed versions of the periodogram and for that reason we introduce the following additional assumption. 3. Let K(rj) be a continuous nonnegative symmetric weight function of finite support with/ K(x)dx = 1. Also, Kn(r]) = b~lK(b~lr]), where bn | 0 and nbn -> oo as n -> oo. ASSUMPTION

The periodogram has the doubly periodic property (5.1)

In(k + Ink, ix + lit]) = In(k, fi)

for all A, /x and all integers k, j . This suggests that (5.2)

r*+

fa,w(rl)= /

In(<xv> +

w,fi)Kn(iJ,-ri)dti,

where 7t+ = n + 8 (8 > 0) is a plausible estimate of f^wiv) with ~IC < *7> arl + w < 7t. For convenience in exposition, form (c) of (3.8) of the finite Fourier transform will be used in (3.9). Let us then note that we have, from (3.5) and (4.5), Theorem 5.1. THEOREM 5.1. Under Assumptions 1 and 2 and /W(A, /x) given by (3.9) and (3.8)(c), wring (3.4), EIn(k,ii)=

Yl J2 /fl,&(^ + 2jrfc)sin( ——jmin(a, 1)) / ( — — y) V Za (a,b)eX\y\
(5.3) + 0* 432

271

HARMONIZABLE SPECTRAL ANALYSIS

uniformly in (A,/x) with y given in (3.4) with the sum over k through the expression y and cov (In(afi + W,/JL), In(a'v<' + w\ MO)

£ x (5-4)

L

/^(A / + 2/:7r)sinc(a,);(l)) + o ( ^ ) l

£

£ £ / / (fl ,ft )€c£ |y(2)|
+

/

£

x

£

£ L

f l

^ ( V + 2*V)sinc(^y(2)) +

o ( ^ ) l V ^ /J

/ fl , fc (V + 27rfc)sinc(a,y(3)) + o ( ^ ) l /fl^(A., + 2jr^sinc(^y(4)) + o ( — ) 1

£

(a',&')€.£ |y(4)|
V ^

/ J

/or —7T < A, A/, /x, /x; < 7T, w/z^re y ( l ) , . . . , ;y(4) are given in (4.1)-(4.4) and

fn + 1

sine (a, y) — sinl ymin(a, 1) V 2a / n+1 = sinc| ;W(tf) U(#),

(5.5) with

1(a) =

min(a,1)

£

£

-y

sinc(x) = sinx

a Then, by (5.2), (5.3) and (5.5), Efa,w(rj)=

n+1

I fa9b(fi

+

27tk)smc(a9y)Kn(ti-ri)dfi

(a,fc)€<£|;yl
(n h\cz P h , | oo with £ = Under Assumptions 1-3 when Efa,w(ri) = o(bn) +

/'logrc log/A

0(^r) \ nbn ) if a is not a slope of a spectrum support line. However, vt (5.7)

, ,

„ x v- f kX*X)eX

+ o(bn) + 0

,

,o

y>n(((n

+ l)/2)/(«)/) «n + l)/2)/l(a)

logn

n if a is a slope of a spectrum support line with y' = ((27tka + b — w) mod27r). 433

272

K.-S. LII AND M. ROSENBLATT

Equation (5.7) says, asymptotically, if (£, rj) is not on a spectral support line or, alternatively, if it is on a spectral support line but the smoothing in the estimate is not on a spectral support line, then Efa^w(rj) —> 0. If (£, rj) is on a spectral support line and the smoothing is on the spectral support line given by (a, w) which passes through (£, 77) (there could be more than one support line passing through the same point), then EfatW(rf) converges to 1(a) times the sum of all spectral masses on spectral support lines which have the same slope a and which are aliased onto (£, rj) through the condition yf = 0. In the almost periodically correlated case, a simpler estimate leads to the following proposition. PROPOSITION 5.1. If the process X(t) is almost periodically correlated, then under Assumptions 1-3, bn ^ 0, nbn —> 00,

Ef

T

(meXk

Tsin^

+ 1)/2

((n +

)((fc"^)mod27r))

D/2)((b-w)mod2jr)

f x / f\,b(iz + 27Tk)Kn(ix-ri)dfi+

01

/logn\ 1.

If the functions f\,b are continuously differentiable up to second order, then I f\,b(^ + 27ik)Kn(ii - T}) dii = fhb(ri

+ 2nk) + b-^flb(r) + 2irk) f z2K(z) dz + o(b\)

and Efi,w(ri) =

J2 (\,b)e£

^ s i n c f — — ((b - w)mod27t))fhb(r]

+ 2nk)

k

.2

+ ^fz2K(z)dz

J2 (l,b)eX

X;sinc(^((^-u;)mod27r)) k

xf{[b(r, + 27tk) N

logft +.(«) +o(*5f

We note that if ((b — w) mod2jr) / 0, then the first term on the right-hand side of the last equation is 0(1/n). If ((b — w) mod2jr) = 0, then the sine function is 1. We now consider the asymptotic behavior of the covariance of spectral estimates. 434

HARMONIZABLE SPECTRAL ANALYSIS

273

5.3. Let (a, w), (a', u/) be fixed with a > 0, a' > 0. Also, let (a, b), (a', b') range over the finite set £. If —TT < %,%',r),r}' < TT with § = at] + w, %' = a'n' + w', then THEOREM

cov (/«,«,(??),/«/,„/(?/)) 2TT

(n + \)bn

E

^aaf-afa^{y{k\aM^Hr))^{y^'\a\bf\-r],-r]'))fi

k,kf,a,a' b,bf

x fajia'ri' + w; + 2nk)fa>#(-rjf

+ 2itkf)

x / K(a x)K\x) ax mm a — ——, 1 J \ min(a% 1) / Sa-afa,a,oS(y(k/;af,b/;-r],^)),oS(y(k;a,b;^-r]f))fi

+

(5.8)

x faM-rf x

+ 2nk)fa>#{a!ri' + w + Ink!)

r . . . / 1 min(<2,1) \ K(—aax)K(x)dxmm[ ,1 J \af mm(a\ 1) / X

+

°(iH

ifn- ■> oo,bn

^logn v, n

I0,nbn-■>

-— aa'

min(a;, 1)

z log2, n 2 n bl

00.

In Theorem 5.3 the covariance properties of the smoothed spectral estimates are given. Thus Theorems 5.2 and 5.3 show that the smoothed spectral estimates give consistent estimates of the spectral mass on the lines of concentration if there is no aliasing. The possible effects of aliasing can be read off from Theorem 5.2. Also consider formula (5.8) for the case of the variance when there is no aliasing. Then, except for a finite number of points, the expression on the right-hand side is lit

-flMv

(n + \)bn

+ ^fuoi-y)

/

K(x)2dx,

J

the contribution from the diagonal a = 1, b = 0. The additional terms that can arise at this finite number of points correspond to the doubling at 0, TT in the case of spectral estimates for a stationary process. If the process is almost periodically correlated, all the spectral lines have slope 19 a = a' = a = a' = 1. The following corollary is an immediate consequence of the theorem. 435

274

K.-S. LII AND M. ROSENBLATT

COROLLARY 5.1. If the process is almost periodically correlated, then, under the conditions of Theorem 5.3, cov(fhw(r)),fhw,(rf)) =

(n + \)bn ft

%?? / -?7+w / -w+£)mod27r), O^i^-^+b^modlTt),

0

by

x fhb(ri;

+ wf + 27Tk)fhb?(-r]/ + 27tk;) I

8((-r}/+b-r]-w)mod2n),0$((ri/+w'+7]+b0mod27T),0

+

x fiA-rt

1 \ nbn)

K(xfdx

+ 2nk)fhV{ti'

+ wf + 2itk!)

x !

K(-x)K(x)dx

/log 2 /? \ n2b%

_/log^\ \ n )

Now we discuss briefly the estimation of the cross-spectrum of harmonizable processes. DEFINITION. Stochastic processes {X^\t)\j nizable if, for /, j = 1 , . . . , / , XU)

_

X0')W =

= 1,...,/} are jointly harmo

[ei*dZj(),)

and cov(X(l\t),Xu\r))

eia-iT^cov(dZi(k),dZj(iJi))

= f f°°

J J—oo

f r

=

eitx-ir„dHi

. (A

ttl)t

J J—oo -00

with / /

\dHi9j(\,fi)\dkdii
J J—oo

If cross-spectral mass is concentrated on lines in the set X = {(a,b) | A = afz + b] and spectral densities exist, then dHij(X, IX) =

]T

Sx-afjL-bfifafil^dll.

(a,b)e£ 436

275

HARMONIZABLE SPECTRAL ANALYSIS

An obvious periodogram is Inn with n-\

Under assumptions similar to Assumptions 1-3 analogs of the previous results in the univariate case hold for the multivariate case. 6. Simulation examples. In this section we give an example of a harmonizable process with spectral support on lines and carry out spectral density estima tion. Let eitXdZ(X)

Yt= I

(6.1)

be a stationary process with spectral density /(•) and spectral distribution F(-). Let (6.2)

Xt =

Yt+cos(wt)Yat,

with WJ,CKE(0,OO). This is the multipath problem of (2.5) with k = 1 where we have made the example more complicated by including a "carrier" cosOO for the Doppler stretched signal. Then

Xt = feitXdZ(X)+1-f{eiwt

= f

+ e-iwt)ei0ltk

dZ{X)

eiadZi(X),

with dZ\(X) = dZ(X) + \ dZ(\(X -w)) + \ dZ(\(X + w)). Then the spectrum of Xt is given by dH(X, /x) = E dZ\ (X) dZ\(/x). Using the orthogonal increment property of the Z(X) process and the fact that E dZ(X) dZ(X) — f(X) dX, we have E dZ(X) dZ(\x) = /(/x)5x,/x d/x, El-dz(-(X - w)Y-dz(-(lJL ~ w) 2 \a J2 \a 1 f 1 = -E\ lim - Z[ —(fz + h — w) - Z( a a 4 h^oh 437

-(fi-w)

8x,Lid[i

276

K.-S. LII AND M. ROSENBLATT

= - hm - F[ -(/x — w) + -) — F\ -(/x - w) h,u,dn 4h^oh a a) \a

7~/f-(^-^))^,/x^/ E dZ

2 \(k

" w^)dZ<jU^

=

=

2EdZ^dZ^h{lla){x~w)^dl1

jf(V<)h,aiJL+wdfl,

l

E dZ(X)-dZ[ 2 2

71

— (/x — w) \a

J Z ( — (fi — if) Sk,(l/a)(fjL-w)dfl

= — / ( " O - UO J ^,(l/a)(ju,-w) d/X, with corresponding expressions for all other possible terms. Therefore dH(k,ii)

h,u.dfj,

=

+ 2~f(~^

~

+ ^-fi-(^

+

W

))h,(\/a){lJi-w)dlX

(6.3)

w))8x,dM(fi+w)dfi

X = /JL, k = a/ji + w9 X = afi — w, k = —(/JL — W), The spectral mass of Xt is supported on the seven lines given a by (6.4)

X = —(/x + w), a

X — \x — 2w

and X = fx + 2w,

which we shall refer to as lines (l)-(7), respectively. These lines are shown in Figure 1 for (a, w) — (0.5,0.5). 438

HARMONIZABLE SPECTRAL ANALYSIS

277

FIG. 1. Spectral support lines for yt = xt + cos((jot)xat with (a, co) = (0.5, 0.5).

As an example, consider (6.5)

Yt =

et+0.5et-u

with^i.i.d. 7V(0,1). Five thousand independent Yt with t = - 1 2 8 , . . . , 128 are generated. Shannon interpolation is used to obtain YaU t = - 1 2 8 , . . . , 128, for a = 0.5. Using (6.2), Xt, f = —128, , 128, is obtained. Periodograms are formed from (3.9) on each of the seven lines given by (6.4). Five thousand independent such periodograms are averaged and for each line both real and imaginary parts of the corresponding periodogram are plotted against the corresponding theoretical values given by (6.3). These plots are shown in Figure 2 in order starting from term 1. Relative to term 1, the sum of the three terms which gives the theoretical spectral mass on the diagonal line [see line 1 from (6.3)] is plotted as a dashed curve. The averaged periodogram of Xt is plotted with it as the solid line and we see that it is roughly unbiased with some variability. The theoretical spectral mass and its estimate are almost superimposed. The theoretical spectral mass is also plotted as the dashed curve slightly below its estimate with an offset. The two spikes located at ±1 are 439

278

K.-S. LII AND M. ROSENBLATT

-4

-2

1.2 • • :

1 0.810.6 0.4 0.2 0

-

■: j[;. ■ CD

£

CD

^^^r...?,,.........

\ •:

-0.2

FIG. 2. Averaged periodogram spectral estimates on different lines; (a,(o) = (0.5,0.5); sample size, 257, each averaged 5000 times; —, estimated real part; • • •, estimated imaginary part; — , theoretical real part; theoretical imaginary parts are all 0 for this example and are not plotted. Theoretical real parts are almost superimposed on their estimates so they are also plotted with an arbitrary offset.

due to the intersection of the diagonal line with two other spectral support lines. Lines 2 and 4 intersect line 1 at +1 while lines 3 and 5 intersect line 1 at —1 (see Figure 1). There are extra spectral masses contributing to the diagonal lines at these intersection points. Similar remarks can be made for all other graphs where there are spikes. These graphs are consistent with the properties of the periodogram given by (3.10) or (5.4). Since the theoretical spectral functions are almost the same as their estimates, we also plot a shift of the real part of the theoretical spectral functions to better indicate their character. Terms 2, 4 and 6 correspond to lines 2, 4 and 6 from (6.3). Terms 3, 5 and 7 are not plotted since they are symmetric versions of terms 2, 4 and 6, respectively. Overall, we see that the procedure is effective in estimating the spectral mass supported on lines. One way to remove the spikes is to use the average of neighboring spectral mass on the line. From the graphs in Figure 2, we also see the effects of the band-limitness of Yt [see (6.5)] in the sudden drop-offs. 440

HARMONIZABLE SPECTRAL ANALYSIS f(lit) on X = \i

f(jLL) o n X = a

279

f(JLX) o n X = ajii + co

(\i - co)

f(ji) o n X = \i - 2co

FIG. 3. Spectral estimates on different lines', (a, co) = (0.5, 0.5); sample size, 16385; —, estimated real part; • •, estimated imaginary part; — , theoretical real part; theoretical imaginary parts are all Ofor this example and are not plotted.

In reality, we do not have independent realizations with which to average the periodograms. Using the same model, we generated one realization of Xt for t — — 2 1 3 , . . . , 2 1 3 . Then a smoothed periodogram of the form in (5.2) is used to estimate the spectral mass on each of the support lines. Results are given in Figure 3. Symmetry properties given in (2.5) make some graphs redundant. Consequently, some graphs are not plotted. We see that the estimates are generally very close to the theoretical ones. These computations confirm the results of Theorems 5.2 and 5.3. Spikes at the intersection of spectral support lines (see Figure 2) are removed. The values at such intersection points are determined by the direction of smoothing. If the lines of spectral support are not known a priori, the results of Theorems 5.2 and 5.3 imply that one can search for these using a grid of points, by computing smoothed periodograms in a grid of directions at each of these points and recording the largest magnitude among these directions at each point. Set an appropriate threshold level to remove all those points whose spectral 441

280

K.-S. LII AND M. ROSENBLATT

magnitude is below the threshold. Then by connecting the remaining points one obtains information about the locations of the spectral support lines. Alternatively, one can search for (a, w) in that if certain statistics from the spectral estimate along the line represented by (a, w) are significantly different from 0 then (a, w) represents a line of spectral support. Spectral mass on these lines of spectral support can be estimated. The spectral function /(•) of the unobserved signal Y(t) can be estimated from estimates of off-diagonal spectral lines in (6.3). These are obviously computationally intensive and are not pursued here. 7. Proofs. We need the following lemmas in the derivations of the results and they are used often. LEMMA 7.1.

Let n—l D„W

(7.1)

=

^inx pinx

I V«

=

f=0

i

e

and Jnx __ i

Dn(x) =

IX

Then D;n(x) == Dn(x) — Dn(x) is uniformly bounded in [—2JT + c, lit — c]for a fixed c > 0. In particular, Dfn{x) is uniformly bounded in [—7T, it} with Dfn{x) = {einx-l)iX~elX } " {elx

+

l

-X)ix

.

The derivation is straightforward, LEMMA 7.2.

For any real a, | a | > 0 , |y|<7r,

Dn(ax+ y)Dn(x)dx

= /

+ OQogn).

J — IX

-TV

PROOF.

Dn(ax + y)Dn(x)dx

Since Dn{x) = Dfn(x) + Dn(x), Dn(ax + y)Dn{x)dx

—I

Dn(ax +

y)Dn(x)dx

J—71

-71

PTt

TV

/

D'n(ax + y)D'n(x)dx+ / D'n(ax + y)Dn(x)dx + ^ Dn(ax + y)D'n(x)dx = (1) + (2) + (3). 442

HARMONIZABLE SPECTRAL ANALYSIS

281

We first note that, for a fixed M > 0,

-M

\Dn(x)\dx = I

\Dn(x)\dx + I

J\x\
JM>\x\>\/n2

\Dn(x)\dx

2 <- + O(logn) + O(logM) = O(logft). n Using Lemma 7.1, for a constant C°, n

(2°

/

ran+y nan+y -an+y

\D'n(ax + y)\dx = — \D'n(x)\dx. -n df > 0, by Lemma 7.1, \d\ J—an-\-y For a fixed d e [df, n) with \ J—d

\Dfn(x)\dx = 0(l).

For k ^ 0, «2kn+d rztcn-\-a

/ J2kn-d

1 ix — elx + 1 dx |D;(x)|Jx=/ e'* 1 zx J2kn-d 2kn+d sin|x 2\ r2*n+, dx < i+ TC ) J2kn-d sm 2* sin|x dx < 0 (log ft). = i+ 2nk+d /»/7T/c-h

sin j j c

It is clear that JA \ Dfn (x) \ dx = O (1) for a closed interval A not containing 2JTk for integer k. Therefore |(1)| = 0(logn). A similar argument applied to Dn(ax + y) shows that |(3)| = 0(logft). Finally, |(2)|= I J\x\
\Dfn(ax + y)Dn(x)\dx+

I Jn>\x\>d

\Dfn(ax + y)Dn(x)\ dx = (4) + (5).

For fixed a, we can choose a d such that \ax + y\ < 2n — c for a fixed c > 0 for all |JC|
\Dn(x)\dx = 0(logn).

For the same d, |Dn(jc)| < 2 d - 1 for ix > \x\ > d. Hence (5) < 2d

Jni i>\x\>d

Dfn(ax + y)\dx = O(logft).

This completes the proof of Lemma 7.2. 443

□

282

K.-S. LII AND M. ROSENBLATT

3.1.

PROOF OF THEOREM

Given {Xt}*L_N, A/ =

CC/JL'

+ w. Let [using (3.1)

and (3.2)] e1^-^-^-^

Go ss EI,,n {X',lx') = -^£ 27tn J J-oo ~n.

x eilJ{x-il)dH{X,

(7.2) = e-iiJix>-S)

J2

/x)
_/x}

Y.Gi(k,a,b),

(a,b)eX

k

where G\(k, a, b) = G\(k, a, b, A/, jxf\ n, /, / ) = G\ I

rti'+n+2kn pii'+n+lkTt

-

Dn(k-X')Dn(-fi

2nn '.n „ Jii'-jz+2kJT

X

Let x = jx — IJL' — 2nk, X' — a/x' + w,

+ fi')e

-U(X-ii)

fa,b(n)h,aii+bdll.

/x = x + fi' + Ink. Then, using X = a/x + b and

1 f* n D {ax + (a — a)/jLf + b — w + 2itka)Dn{—x) 2rcn / -n n

x ema~l)x

fa,b(x

+ At' +

2itk)dxeilJ[b+{a-l){lx'+2jtm

since -/x + /x' = -x -Ink,

Dn(-/x + IJL') = Dn(-x

-Ink)

=

Dn(-x),

X - ix = (a - l)x + b + (a - l)(/x' + Ink). Notice that e-HJ^-^)eUJ[b+(a-iW+27tk)] w)}. Then, using (3.4),

(7.3)

+ (a - a)/x' + b-

J2eiU-y(k'a'b'X'^G2,

G0 = Y, (a,b)€X

= eXp{ilJ(27tka

k

with G2= —

2itn

I" Dn(ax + y)Dn(-x)eiU{a-1)xfa,b(x

J-7T

+ fi' + 27tk)dx.

Because of the continuous differentiability of fa,b, we have fa,b(x + ix' + 2nk) = fatb(ji'

+ 2nk)+xtitb(x*

with x* between JX' and /x' + x. Then (7.4)

G2 = G3 + G 4 , 444

+ 2itk),

283

HARMONIZABLE SPECTRAL ANALYSIS

with GA = ^~ r Dn(ax + y)Dn(-x)xeilJia-1)xf^b(x* Inn J-n (7.5) log ft \GA\
+ 27tk)dx,

by Assumption 2 and G3 = fa,b(^ + ^k)^— (7.6)

=fa^

r

+ 27t*)[—

y)Dn(-x)eilJ(a-l)xdx

Dn(ax +

Dn(ax+y)Dn(-x)eilJ(a-l)xdx

r

\_l7tn

J-jt

+0

'log ft

using Lemma 7.2. Consider I

G5 =

(7.7)

rn rn

ein(ax+y) e^

_ j e~inx

Inn J-7T i {ax + y)

_ j

—ix

eilJ{a~l)xdx.

Let ft* = z. Then I

G5

rnn ei(az+ny) nn f.

Inn / I

_ j e~iz

J-nn +ny)/n -nn i(az i roo

ei(az+ny)

L

_ \ e~iz

2itfJ\ \>nn

ei(az+ny)

_ ^

7ilJ{a

— \)z/n

,HJ(a—\)z/n

-iz _ j e~iz

i(az+ny)

z

lit JU

j

dz/n

—iz/n

= - / in j - oo i(az + ny)

(7.8)

_

_ j

dz

eilJ{a-\)z/ndz

-u

— G^- G 7 . To evaluate the second term G7, we first note that \y \ < it and

L

z\>nn

\ei(az+ny)

_ j e~iz

i(az+ny)

_ j

-iz

\dz ei{az+ny)

1 JzeAif\A heAiHA

+ Jz€A / ,r\Ai + J\z c JzeAclf)A2

J \z\>nn+ \nn / a\

_ j e~iz

i(z + ny/a)

_ \

-iz

dz,

where A\ = [—ny/a — c, —ny/a + c] with fixed c > 0 and A = [z\ \z\ > rut], A2 = [z I nix < \z\ < nix + \nn/a\}. For the first integral, z € Ai n A and |(gi(az+*30 _ i)/(/( z + ny/ fl ))| < \a\, \{e~iz - \)/{-iz)\ < 1/ft. Therefore ei(az+ny)

/

JzeAiHA

_ j e~iz

i(z + ny/a)

_ j

-iz 445

dz

2\a\c n

(\ \n

284

K.-S. LII AND M. ROSENBLATT

The second integral is bounded by log ft 1 dz = 0 17T Jzc nn JzeA\r\A2 z + ny/a n

The third integral is bounded by

cf

\dz=o(-)

J\z\>njT zl

From these we see

ei{az+ny)

(7.9) G 7

In J\z lit ,

_ j e~iz _ j -lzlefina-l)z/ndz=onW.\

-iz

(az + ny) To evaluate G& we note that ei(az+ny)

(7.10) (7.11)

__ j

leilJ{a-\)z/ ~ZZ

\ n

)

f°°OO I1

i(az + ny) £

\n)

/ -oo

■]*-/.

Cl

J— oo

J

h-lJ(a-\)/n,\-lJ(a-\)/n}{t)e

dt

and, using the Parseval relation, we have G6=~

00 f00

11

a J—oo

1

r

Mo,a](t)l[-U(a-l)/n,l-U{a-l)/n](t)einyt/adt

(-ny\

, ,

U{a-\)

— exp [i — min(a, 1 — inyL V a L max(0, - / 7 ( a - l ) ■ ) — exp

(7.12)

if [0, a] n I0,

Z/(a-l)

U(a-l)

^0,

otherwise.

Successively using (7.2)-(7.9) and (7.12), we obtain Theorem 3.1. PROOF OF THEOREM

□

4.1. Let dn = l/(2nn). Then

COv(ll,n(X,fl),Il
= dn

22 t,T,t',t'=0

\-rt+u,t'+ijrT+u,r'+u

+rt+u,x'+ijrx+ij,t'+ij'\

x exp{i[-(/ + U)k + (T + / J > + (*' + Z7)A/ - (T' + Z/)^']}

=4 E

[£ f gi(f+//)M ^ Z (") f e-iit>+lJ)v dZjvj xE

ir+U)r

/ '

dZ(r) I e-iir'+lJ)s 446

dZ(s)

HARMONIZABLE SPECTRAL ANALYSIS

+

f ei(t+U)u

E

dZ(u)

285

t e-i(r'+U)vdZ(v)

x E f ei{r+lJ)r dZ(r) f e-i(t'+lJ)s dZ^jl x exp{i[-(t + U)k + (T + U)fi + (t' + U)X' - (r' + lJ)fi']} Yl

J2 [exp i f [ f (M - A ) + r ( >+^)+*'(*' - y )

=di IHI

— x1Xii + s)]} exp{HJ[u — v + r — s]} + exp{i[t(u - k) + x{r + /x) + t'(k' - s)

- r V + i))]) x exp{z//[w — v + r — s]}\ x exp{///[—X + ix + kf — //]} dH(u, v)dH(r, s) = d^JJJj[Dn(u-X)Dn(X/-v)Dn(fi

+

r)Dn(-lx/-s)

x exp{z7/[w — v + r — s]} + Dn(u- X)Dn(-ix; - v)Dn(fji + r)Dn(kf - s) x exp{z//(w — v +r — s)}\ x exp{z7/(—k + fju + kf — [if))dH(u,

v)dH(r,s)

= c\ +C2,

where c\

\\dn

Yl

/ Dn(av + b-an-w)Dn(af[i/

+ wf -

v)fa,b(v)

(a,b)eX

x exp{///(tft; + b — v))dv

exp{—HJ(k — A/)}

(a'^eX'

x exp{///(a ; ^ + // — s)] ds exp{—ilJiji — \i)\ 447

286

K.-S. LII AND M. ROSENBLATT c

2 = j \dn

Yl

/

D

n(av + b-a/LL~ w)Dn(-fjif -

v)fa,b(v)

(a,b)eX

x exp[ilJ(av + b — v)} dv exp{-UJ(k-k')}\ dn

x

/ Dn(fi + afs + b/)Dn(a/fi/

J2

+

wf-s)fa^^(s)

(a,b)eX

x exp{HJ(a's +b — s)}ds Qxp{—ilJ(ixf — /x)} |, with u = av + b, r = a!s + b\ X = a\x + w, A/ = a''\A! + wr. Now each term in the outer brackets has exactly the same form as the expectation of the periodogram [see (7.2)]. Therefore the covariance of the periodogram has the form of the product of the expectations of the periodogram. □ PROOF OF THEOREM

5.2.

We now take the symmetrized version. From (5.7),

(5.3), (5.5) and (3.4), Efa,w(fl)=

2Z Yl

(7.13)

+0 Let OO

/ -oo OO

(7.14)

/ -oo

fa,b(^ +

m

(a,b)e£

2nk)§mc(a,y)Kn(ii-ri)dii

k

faMv + 2nk) sinc(a, y)Kn(n - rj) dfi fa,b{r] + bnz + 2nk) x sine] —-—l(a)[[2nka + (a — a)r] + b — w + (a — a)bnz]mod2n]

\l(a)K(z)dz.

ByAssumption2,wehave/07 +fcnz+ 27rfc) = /^^ for r)lz between r\ + 2nk and rj + 2nk + bnz. Then Eo = E\+ Ei with' E\ = l(a)fa,b(rj + 2nk)E?>,

E2 = bnfzf^b(vtz+2nk) (7.15) E3=

x sine

fft + 1

Ka)[[yri + (a - a)bnz] mod2n] \l{a)K(z) dz,

sincj——— l(a)[[yrj + (a -

a)bnz]mod2n]\K(z)dz 448

287

HARMONIZABLE SPECTRAL ANALYSIS and yv=^2jTka + (a —a)r] + b — w. We evaluate £2 first and then £3. E2 = bnl(a)f^b(n

+ Ink) £

ZK(z)

x sincj

l(a)[iyr) + (a — a)bnz] mod27r] | dz + o(bn)

by the continuity of / ' with [—c, c] the support of K for a finite c. For a = a,

n+ 1

')£

£ 2 = W («)/«> (^ + 27r£:) sine j — - ^ / ( a ) ( ^ m o d 2 ^ r ) | /

zK(z)dz

+o(bn)

= o(bn) by Assumption 3. For fl^a, £2 =
+ Ink) f

w

*m

zK{z) . fn + 1 K«)[[j^ + (« — «)& n z] mod27r]} dz x smc

= bnl{a)f'ab{j]

+ Ink)

\

zK(z) sine

n+ l

l(a)[(ynhn

+ (a - a)Z>„z] [ dz,

where (j^h^r = (J?? mod 2TT). Therefore

\E'2\
sine

n+ l

Cyi))2ff+(«-a)*«c

(7.16)

= C° f k.(yr})2n-(fl-a)b„c

+(

f(fl)[0'ij)2w + ( a - a ) f c » z ]

smej

>[JC|>I/«

l(a)x I

' \—^

dz

dx {a — a ) l(a)x\ dx

U\x\
where C°, C ; are constants. To evaluate £ 3 , we see that, by a similar argument, for a / a, |2?31 = nbn). If a=a, then sincj

/(a)(j^mod27r)

f/7 + 1

I

= sine] ———/(a)(j^mod2;r) 449

\K(z)dz

0(logn/

288

K.-S. LII AND M. ROSENBLATT

Substituting these results on £3, £2, E\ into £b using (7.13)—(7.16), we have the results. □ PROOF OF THEOREM

= [[* JJ ~^

(7.17)

5.3.

We evaluate the covariance

KniiL-rftKntjL'-ri') x cov(/n(a/x + w9 /x), In(a'ij/ + wf,

^^d/xd^

/logn\ with C5 = J £ ^ + KnQl - T))Kn(lJLf - rf^tffafiQ,' + 2lTk)faf^{-tJLf + 2ltkf) (7.18) x sinc(a, j(l))sinc(a ; , y(2))dfjLdfi\ C6 = jjn+ (7.19)

Knin - n)Kn(jif - ? / ) £ 3 £ V < a ( - V + 27tk)fa^(kf

+ 2ixkf)

x sinc(a, j(3))sinc(a / , y(4)) dfidyd

using (5.5). The sum S 1 is over |y(l)| < v and |y(2)| < v, E 2 and E 4 are over (a, b)eX and (a\ bf) e X, while S 3 is over |y(3)| < v and |y(4)| < v. Now consider a typical summand of (7.18), C7 = y y Kn(n - ri)KnQi' - rf)fa,b&

+ 27rk)fa,j(-fi'

+ 2jtk;)

(7.20) x sinc(a, y(l))sinc(a\

y(2))d/jidix\

For fixed k, kf, (a,fc)€ dC, (a ; , &') e <£, we evaluate C7 in the following cases: (i) If aa' 7^ a! a, because of the form of y(l) and j(2), we introduce the change of variables a! y! — /x = xf,

aa [i! — afi = x, with the Jacobian aa a!

—a = —aa + a!a ^ 0 -1

and \x —J

{aax —a!x),

y! = / 450

( a x ; - x).

HARMONIZABLE SPECTRAL ANALYSIS

289

We have C7= ff

Kn(J~\aafxf x

-afx)-r])Kn{J-l{axf

fa,b(aJ~l(°ixf

-x)-r]f)

~x) + w; + 27ik)faf^(—J~l(axf

— x)

J

r2jtk!)

f

x sinc(a, (2nka + x + aw + b — w) mod27r) x sinc(a ; , (2irkfd — xf + bf)mod2n)J~

dxdx\

where Q is a bounded region. Using Assumption 3 and the fact that K is bounded, there is a constant c such that |C 7 | < ~g(xi

+ 27tk)g(x2 + 2ixk!)

x / | sine (a', (27tka +x +aw; + b — w)mod27r) JQ

(7.21)

x sinc(a;, (2nkfa — x1 + bf)mod2n)\

dxdx/

+ 27Tk/),

= o(-^^)g(xi+27tk)g(x2

with x\, X2 some fixed values. Using Assumptions 1 and 2, the contribution to C5 is shown to be O(log2 n/(n2bl)). (ii) If aa' = a'a, then, by a change of variable, x — aa! n! — a /x, C7 =

r

f Kn(

a J-7T+ JA

\

a f

/x = —{aan! — x) — a! \JL a a

x + affM/-r])Kn(fif-i1f)fa^(af^

x fa &(-^

)

x, + wf + 27tk)

+ 2itk!) sinc(a, (x + 3^,1)2*)

x sinc(
\lAQjL')(x)dxdiif,

where yk^ =2irka +awf + b — w,

3^2 = 2jxk!d + b\

with A = \—{aaf + a)7t+, (aaf + a)7T+],

AGu/) = [aa'/JL' — an^, aa1 \x' + an^].

Let x! = (/// — r)f)b~l, fif = rf + bnxf. Then C7 =

f K(afxf + --(afrif - ri -xYjKix') abn JAI JA \ bn\ a )) x fa.biptr] + a'bnx + wf + 2itk)faf^{-Y]r /

451

- bnx + 2nkf)

290

K.-S. LII AND M. ROSENBLATT

x sinc(a, (x + ^,l)2^)sinc(a / , -I - ( x -ayk',i)\ x

j

IA(n'+bnX')(x)dxdx',

with Ai = [(-7T+ - rj')b~l, (TT+ - r]r)b~1]. Let z = x + yk,i, x = z- y*,i [we note that since (x + yt,i)2n = (x + (yicihnhn we can replace yky\ by {y^xhn in the following] Ci = ^ i f K(a'x'' + ^ ( W ' - V + <xbn JAiJA' \ bn\ a

-yk,i--z))K(x') a JJ

x fa,b(<x'v' + w> + 27T& + bnu'x')faitf(—r)'

+ Ink' — bnx')

x s i n c ( a , z 2 ^ ) s i n c f a / , - ( - ( z - ^ , i -ay^,i)\ x IA(r,'+bnx')(z -

J

yk,i)dzdx',

where A' = A + y^, 1. Let £' = {l\ litl e A', I integers} which is a finite set. For each / e £', let A1 = [-n + 2nl, n + 2nl\ n A'. Then abn ^JAIJA1

\

bn\

a

a

))

x fa,b(oi'r)' + a'fcnx' + u/ + 2jTk)faiib>(-r)' - &nx' + 2^A;') x sinc(a, z2^) sine fa', -l-(zx IA(ni+bnX')(z -

yt,i ~ ayk',2))

yk,i)dzdx'.

Letting z' = z — 2TTI, C

(7.22)

7=E^T

G

''

with Gi= f

K(O!X! + — ( W - r\ + -ykA

f 1

JAi JA -2TZI

V

bn\

a

- -2TVI - - / ) a

x /«,*(«'// + u/ + 2TT/; + bnafxf)fa^y{-r]f

a

Wx')

JJ

+ Ink' - bnxf)

x sinc(a,z / )sinc(a / ,-(-(z / + 2 7 r / - ^ , i - a % / , 2 ) j

452

J

HARMONIZABLE SPECTRAL ANALYSIS

291

We note that Alc = A1 — 2nl is centered on 0 with length at most 2rc. For each / e X\ rewriting this last expression with q\ = a!r\! — r\ + ^yk,\ — ^27i/ ^ 0 and / ( / , z\ xf) = lA{r)f+bnx>)(zf + 2itl - yk,i), we have

x faA^rf

+ wf + 2rck + bnufxf)faf,b'(-ilf

x sinc(a, z) sincla , -l-(z x

+ 2xkf - bnxf)

+2itl - yky\ - « j ^ 2 ) )

)

J(l,z\x)dz!dx'.

Let en be a sequence such that n~l = o(en),

en = o(bn),

n~l = o(bn),

c$, c\ constants.

Then there exist constants co, x\ and X2 such that \Gi\ < c 0 g(xi + 2itk)g(x2 + 27tkf) (7.23) .Mi L^l^l^ci^

Jn>\z'\>c\eni

|G(z',x')|dz',

where G(z',x') = G ' C Z ' . J C O G ' V ) , G'(z',*') = K(a'x' + (l/bn)(qi - (l/a)zO) K(x'), G"(z') - sinc(a, z) smcia', -( ~(z' + 2TT/ - yfej - ory/t',2) J

J.

We see that the first double integral in (7.23) is !\z>\ 00. (We can also use the fact that x' has finite range or K has finite support.) Note that f\zn\<„(\G"(.z')\fAl \G'{x',x')\dx') x dz'. But I \G'(z',x')\dx'<\a'\-y2

j

by the Schwarz inequality and

f

\G"{z')\dz'

>c\en<\z'\
sin(((/i + l)/2)z7(a)) L

\n )

^ici x^en^<\z'\ lz'!^

£ 453

K2(y)dy

292

K.-S. LII AND M. ROSENBLATT

sin{((n + l / 2 ) [ - ( l / a ) ( z ' + 2nl - yk,i - ayk>,2)hJ(a')}

N><

1

( - ( l / a ) ( z ' + 2JTI - yk>\ -

1

, flg

aykttl))2a

Therefore, when qi^O, |G/| = o(l/n)g(xi + 2jtk)g(x2 + 2nkf). In this case, by Assumptions 1 and 2, the contribution from Ci to C5 is shown to be o(l/nbn).

If 3 / = a y-»7 + £y*,i-£ 2 * z = 0 > Gi= [

[1

JAI JA

(7.24)

K(a,x/--z//bn)K(x/) a J

\

C

J{Uz'>xf)G"(z')dzfdxf

x

L M i ^{k/|
/ A J -/{l*'!^**

1G''V,*')G''(Z')

x J(l,z ,x)dz

dxf,

where G'"(z',*') = * Y a Y - -z/bn\K(xf)fa,b(ufr}f x fa\bf(-rf

+ 2nkf -

+ wf + Ink + bnaxf)

bnx).

Using the same argument as before, we see that the second double integral in (7.24) is o(l/n)g(x\ + 2nk)g(x2 + Ink!). Its total contribution to C5 is o(\/nbn). The first double integral can be approximated by G/j given below using the fact that \z'\
l

K(axf)K(xf)fab(ar)f

f

J{\z'\
+ w' + 2jtk + bnaxf)

x faf,b'(-rf + 2nk' - bnx'VAtf+bnx'^z' + 2TT/ - yk,\)G"(z) dxf dz . The weight function K has finite support AK and by Assumption 2 we have Gi 1 = f

K(a'x')K(x')

JAK

x /

J{\z'\
dxffa b(arjf + wf + 2ixk)fa> v(-rf

smc(a,z')smc(a',--(--(z'+ \

2jrl-yk,\

\<X

x IAw+bnxf)(z +2nl - yk,\)dz 454

dxf

+ 2nk!) - ayk^2))

) Jin)

HARMONIZABLE SPECTRAL ANALYSIS

+ 0{bn)g{af^

+ wf + 27tk)g(-^

293

+ 2izkf) f

\G;\z;)\dz'

J{\z'\
It is clear that | G ^ | < 0(bn)g(afr]f* + w' + 2nk)g(-r]^ total contribution to C5 is O(^). To evaluate G[ 1? let

+ 27tkf)0^)

and its

A3 = \aavl — a7T+ — 2nl + y^^.aa'r] + a7t+ — 2nl + y&,i]. Note that lA(ri'+bnx')(z'

+ 2TT/ -ykfi)

=

IA'3(z),

with A3 = [aa'(rj' + bnxf) — an^ — 2nl + y^^.aa' {r] + bnxf) + a 7 r + — 2nl + 3^,2]The condition qi —0 means a'arf — arj + y^i — 2nl = 0. This means aa'r}' — ar 1 + ykt\ — 2TT/ = 0 by the condition a! a = aaf. Now r\ e [—n, n], TT+ > it, x1 is bounded. Therefore 0 e A3, the interior of A3, and then IA3(Z') = 1 when \zr\ < c\en for large n. Also qi — 0 implies 0 e Alc, so, for large n, the inner integral of G' u is G4=

sincia, zf) sinc( a\ -( -(zf + 2TTI - yk,\ - ayk? 2)) \dzf. J\z'\
If q[ = (±yhf! + yk,f2 - \2nl)l7Z G4 = /

± 0, then G 4 = o(±). If q[ = 0, then, for large n, sinc(a,z / ) s i n c(^ / ,

J\z'\
\

— —a /

n ^ Oi )

sinc(a, —ax) sinc(V, x)dx.

J\x\
Following an argument like that leading to (7.7)-(7.12) and using the Parseval identity, we have r , 2TZ , . (\\ /]\x\
J\x\
n + 1

/(a, a ,a,a)—

\nj

,min(a, 1) \min(a', 1) ; mini a — : min(a , 1) , 1/ aa'

In this case, G U = fa,b(a'r}' + wf + 27rk)fa^v(-r)f '—a2it

n+ 1

+2irkf) f

. . /1\" /(a, a , a, a ) + o -

\n/

455

JAK

K{afxf)K(xf)dxf

294

K.-S. LII AND M. ROSENBLATT

In summary, under the assumption aa' — a'a, C

/«,&(flV + ">' + 27tk)fa\b>{-ri + Ink!)

7 = r- E bn

le£"

(7.25) X

/

K{a!xf)K(xr)dxf

n+l

l(a,a',a,d)

+

o\-\ \nj.

where, for each k, k\ a, b, a\ b', X" = X1 n X\ n X'2 with dC; = {/ \2nl e Af = [ — (at/ + a)7T+, (aa + a)7r + ] + J ^ , I } , ykl — 2JT ka + awf + b — w, X\ = ll\qi=

afr)' -n + ~(ykfi - In I) = ol,

A = {l \q[ = (-(y*,i

-2TT/)

+ w , 2 ) =oJ

and ykf^2 = Ink! a + b!. However, we note that if there is an / e X[ then I eXf and the condition q\ — 0 is 0 = aa!r] — at] + 2rcka + awr + b — w — 2TCI = 2jxka + b + aipdr] + wr) — (arj + w) — 2TXI by the condition aa' = aa;. This is the same as y(k,a,b;^,^f) = 0mod27r. Substituting qi =0 into q[ — 0 in X'2, we have (—aV + r) + yv ,2)2n = 0- This is the same as y(k\ a\ b\ —rj, —r]f) — 0mod27r. Therefore the sum over / e Xn in (7.25) can be replaced by the restriction ^aar-a'a^{y{k;aM^^f))^{y{kf\af,bf\-r],-r]f))fi'

Substituting the modified (7.25) into (7.20) and (7.18), we have C5 in (7.17) and the first term in (5.8). By a similar argument for Ce in (7.19), we obtain the second term in (5.8). This completes the proof. □ LEMMA

7.3.

IfK(x) e L^ n L\ and K(x) —> 0 as \x\ —> oo,

then, for e ^ 0, j (K(ey + z) - K(ey))K(y)g(y, for a bounded function g. 456

z)dy-+0

asz-^0

HARMONIZABLE SPECTRAL ANALYSIS

295

Let \g\
PROOF.

j (K(ey +z)-

K(ey))g(y, z)K(y) dy

= \ f + f 1 (*(30*(3>> z)(K(ey + z) - K(ey))) dy. U\y\>L J\y\ 0, since K(x) e L\ and K(x) -» 0 as |x| -> oo there exists an L > 0 such that f

g(y, z)K(y)(K(ey

+ z) - tf(ey)) dy <e/2.

By Lusin's theorem, there exists a continuous function KCyG(y) such that 2 36MZ

JlvKL

where M2 = c2f K2(y)dy. Then g(y,z)^(y)(A-(e3;)-A'(ey + z)) dy

I /" \J\y\
-.1/2 <\

f g2(y,z)K2(y)dy lJ\y\
= M\ f

f {K(ey) J\y\
- K(ey + z)f

[K(ey) - Kc,e(ey) + Kc>e(ey) - Kc,e(ey + z)

[J\y\
+ Kc,£(ey + z)-K(ey < V3M( [

[K(ey) - Kc,e(ey)f

+ f r

since

dy

+ / [KCte(ey + z)-K(ey J\y\
36M2

pi

+

+

[Kc,s(ey)-Kc,E(ey+z)fdy

J\y\
[

dy

36M2

pi

+

36M2

2l1/2

+ z)Y\ J e 2

for z sufficiently small by the continuity of Kc^£(u) for \u\ < 2L. 457

z)fdy^

296

K.-S. LII AND M. ROSENBLATT

Therefore, for any s > 0, one can choose z sufficiently small such that | f g(y, z)K(y){K(ey) and the lemma is proved.

- K(ey + z)) dy

<£

□ REFERENCES

V. G. (1988). Estimating the spectral densities of a Gaussian periodically correlated stochastic process. Problems Inform. Transmission 24 109-115. ALLEN, J. and HOBBS, S. (1992). Detecting target motion by frequency-plane smoothing. In Asilomar Conference on Signals, Systems and Computers (A. Singh, ed.) 1042-1047. BOHR, H. A. (1951). Almost Periodic Functions. Chelsea, New York. CHIU, S.-T. (1986). Statistical estimation of the parameters of a moving source from array data. Ann. Statist. 14 559-578. DAHLHAUS, R. (1997). Fitting time series models to nonstationary processes. Ann. Statist. 25 1-37. DANDAWATE, A. V. and GIANNAKIS, G. B. (1994). Nonparametric polyspectral estimators for fcthorder (almost) cyclostationary processes. IEEE Trans. Inform. Theory 40 67-84. DANIELL, P. J. (1946). Discussion of paper by M. S. Bartlett. J. Roy. Statist. Soc. (Suppl.) 8 88-90. DEHAY, D. and HURD, H. L. (1993). Representation and estimation for periodically and almost periodically correlated random processes. In Cyclostationarity in Communications and Signal Processing (W. A. Gardner, ed.). IEEE Press, New York. FERGUSON, B. G. (1999). Time-delay estimation techniques applied to the acoustic detection of jet aircraft transits. /. Acoust. Soc. Amer. 106 255-264. GARDNER, W. A. (1991). Exploitation of spectral redundancy in cyclostationary signals. IEEE Signal Processing Magazine 8 14-36. GARDNER, W. A. (1994). Cyclostationarity in Communications and Signal Processing. IEEE Press, New York. GERR, N. and ALLEN, J. (1994a). The generalized spectrum and spectral coherence of a harmonizable time series. Digital Signal Processing 4 222-238. GERR, N. and ALLEN, J. (1994b). Time-delay estimation for harmonizable signals. Digital Signal Processing 4 49-62. GLADYSHEV, E. G. (1963). Periodically and almost periodically correlated random processes with continuous time parameters. Theory Probab. Appl. 8 173-177. HURD, H. (1989). Nonparametric time series analysis for periodically correlated processes. IEEE Trans. Inform. Theory 35 350-359. HURD, H. and GERR, N. (1991). Graphical methods for determining the presence of periodic correlation. J. Time Ser. Anal. 12 337-350. JIN, Q., WONG, K. M. and Luo, Z. Q. (1995). The estimation of time delay and Doppler stretch of wideband signals. IEEE Trans. Signal Processing 43 904-916. LESKOW, J. and WERON, A. (1992). Ergodic behavior and estimation for periodically correlated processes. Statist. Probab. Lett. 15 299-304. Lll, K. S. and ROSENBLATT, M. (1998). Line spectral analysis for harmonizable processes. Proc. Nat. Acad. Sci. USA 95 4800-4803. LOEVE, M. (1963). Probability Theory, 3rd ed. Van Nostrand, Princeton, NJ. MALLAT, S., PAPANICOLAOU, G. and ZHANG, Z. (1998). Adaptive covariance estimation of locally stationary processes. Ann. Statist 26 1-47. MUNK, W., WORCESTER, P. and WUNSCH, C. (1995). Ocean Acoustic Tomography. Cambridge Univ. Press. ALEKSEEV,

458

HARMONIZABLE SPECTRAL ANALYSIS

297

N E U M A N N , M. H. and V O N S A C H S , R. (1997). Wavelet thresholding in anisotropic function classes and application to adaptive estimation of evolutionary spectra. Ann. Statist. 25 38-76. YAGLOM, A. M . (1987). Correlation Theory of Stationary and Related Random Functions 1, 2. Springer, New York. ZURBENKO, I. G. (1986). The Spectral Analysis of Time Series. North-Holland, New York. DEPARTMENT OF STATISTICS

DEPARTMENT OF MATHEMATICS

UNIVERSITY OF CALIFORNIA, RIVERSIDE

UNIVERSITY OF CALIFORNIA, SAN DIEGO

RIVERSIDE, CALIFORNIA 92521

L A JOLLA, CALIFORNIA 92093

E-MAIL: [email protected]

E-MAIL: [email protected]

459

The Annals of Statistics 2003, Vol. 31, No. 5, 1693 © Institute of Mathematical Statistics, 2003

CORRECTION SPECTRAL ANALYSIS FOR HARMONIZABLE PROCESSES BY KEH-SHIN LII AND MURRAY ROSENBLATT

The Annals of Statistics (2002) 30 258-297 In the statement of Theorem 5.3, page 273, fa,b(afrif + <*>' + Ink) should be replaced by fa,b(ot'ri' + co' + Ink) and fa^b'ia'rf + <*)' + Ink') by fawWrf + co' + Ink'), The theorem holds in the non-Gaussian case if sup J^ l cum (*r> XT, Xt', XT>)\ < oo. T,tf,Tf

On the same page /i,o(?? + co) should be replaced by f\,o(ar] + co)l(a). From page 292 on, wherever a'r\' + co' occurs it should be replaced by a'r]' + co'.

DEPARTMENT OF STATISTICS UNIVERSITY OF CALIFORNIA, RIVERSIDE RIVERSIDE, CALIFORNIA 92521

DEPARTMENT OF MATHEMATICS UNIVERSITY OF CALIFORNIA, SAN DIEGO LA JOLLA, CALIFORNIA 92093

E-MAIL: [email protected]

E-MAIL: [email protected]

Received December 2002. 1693 460

The Annals of Statistics 2006, Vol. 34, No. 3, 1115-1139 DOP. 10.1214/009053606000000218 © Institute of Mathematical Statistics, 2006

ESTIMATION FOR ALMOST PERIODIC PROCESSES B Y K E H - S H I N LII AND MURRAY ROSENBLATT

University of California, Riverside and University of California, San Diego Processes with almost periodic covariance functions have spectral mass on lines parallel to the diagonal in the two-dimensional spectral plane. Meth ods have been given for estimation of spectral mass on the lines of spectral concentration if the locations of the lines are known. Here methods for es timating the intercepts of the lines of spectral concentration in the Gaussian case are given under appropriate conditions. The methods determine rates of convergence sufficiently fast as the sample size n -> oo so that the spec tral estimation on the estimated lines can then proceed effectively. This task involves bounding the maximum of an interesting class of non-Gaussian pos sibly nonstationary processes.

1. Introduction. The main objective of this paper is to present a constructive method to determine the lines of support of spectra or, equivalently, the frequencies or periods of Gaussian harmonizable processes with almost periodic covariances under appropriate conditions. The processes considered provide an interesting set of random processes that are generally not transient and not stationary and for which Fourier methods of analysis are helpful and meaningful. The lines of support of spectra are lines parallel to the diagonal in the two-dimensional spectral plane. A number of papers have discussed spectral estimation in this context but they all assume knowledge of the lines of support of the spectra. The novelty of this paper is in the presentation of constructive methods for estimating the lines of support under appropriate conditions with a rate of convergence good enough to imply bias and covariance of spectral estimation using the estimated support lines with the same asymptotic properties as if the actual lines of support were known precisely. Almost periodically correlated processes have been considered by Alekseev [1], Hurd [10], Gardner [5], Gladyshev [9], Gerr and Allen [7, 8], Dandawate and Giannakis [3], Leskow and Weron [14] and Lii and Rosenblatt [15]. Exten sive application of periodically correlated (or cyclostationary) processes is noted in [6] and references therein. For the sake of simplicity and economy, we limit our selves to a discrete time parameter process [Xn, n = . . . , —1,0,1,...} with mean EXn = 0. The process {Xn} with covariance function rn^m = EXnXm is said to be Received February 2004; revised May 2005. AMS 2000 subject classifications. Primary 62M15, 62M10; secondary 62G05, 62M99. Key words and phrases. Almost periodic covariance, spectral estimation, periodogram, process maximum, Gaussian and non-Gaussian process, frequency detection and estimation. 1115 R.A. Davis et al. (eds.), Selected Works of Murray Rosenblatt, Selected Works in Probability and Statistics, DOI 10.1007/978-1-4419-8339-8_39, © Springer Science+Business Media, LLC 2011

461

1116

K.-S. LII AND M. ROSENBLATT

periodically correlated with period T if (i.i)

r

n+T ,m+T — rn,h

We shall call {Xn} a process with almost periodically correlated covariance function if for each (s, r), there are functions otj(s, r) and real Xj(s,r) with Y,j \otj(s, r ) | < 00 such that, for each s, r, r 5+r , T+r = J2jaj(s, r)elkJ^9T^ with the convergence uniform in s,r. We shall call a sequence {^(n)} almost peri odic if there are sequences {ctj}, real {fij} such that a(n) = J2jaje^jn with Ejla/I
X„ =

(1.2)

einXdZ(k),

with the random spectral function having a covariance cov(Z(A.), Z(fx)) = F(X, ix),

(1.3) r(n,m)=

/

einX~imfM dF(X, fi),

/

/

|
[16, 20]. Almost periodicity of the covariance function of a harmonizable process implies that F has mass (complex) on at most a countable number of diagonal lines X — ix + b,

b = bj

] = ..., - 1 , 0 , 1 ,

If the process is real-valued (as we assume to be the case) and X = JJL + b is a line of spectral support, then so is X = JJL — b. If the spectra on the lines of support of F are absolutely continuous with respect to Lebesgue measure with spectral densities fb(f^), the assumption that {Xn} is real-valued implies that (1.4)

Mil) = f-biv

+ b) = f-b(-n)

= fb(-V> ~ b).

Also, (1.5)

\Mn)f<Mii)Mv>

+ b).

Let us call X the collection of b's corresponding to lines with nontrivial spec tral support. In [3], the spectra fb(ti) are estimated under appropriate conditions assuming that the elements of X are known. We consider estimation of b e X under the assumption that X is finite and $1n fbWdk ^ 0 and see whether the final estimates of fb(li) based on these estimated elements of X produce an effect. At this point, a few additional remarks are made on related papers. Tian [18] considers time domain estimation of the integral period of a sequence with period ically correlated covariance function observed at integral time. Hurd and Gerr [11] 462

1117

ALMOST PERIODICITY AND ESTIMATION

and Gerr and Allen [7] use the coherence between spectral components to test for a specific period of a periodically correlated covariance function against the null hypothesis of stationarity. They used the integrated coherence along diagonal spec tral support lines. Lund et al. [17] use an averaged squared coherence statistic to test existence of periodic correlation against stationarity. The case of a continuous parameter process {X(-)} is considered in [4] and [12]. Before the main discussion we describe a simple but interesting class of processes with almost periodic covariance function. Additional examples can be found in [6] and [20]. Set (1.6)

Xn = YjC°^WJn^Yn-rr

0 < Wj < 7T,

;=i

,0)

where the processes {Yn\J } are jointly stationary with means zero and the crossspectral density of Y^\*) and F w ( ) is //&(•)• This commonly used model in communication is a mixture of amplitude modulated (AM) signals with carrier frequency Wj modulated by the signal Yn with possible relative delay r ; . If the stationary processes {Yn } have the spectral representation (1.7)

Y^ =

j\inXdZj(X),

the process {Xn} is given by Xn = f*n einX dZ(X) with (1.8) dZ(r]) = \ YXe~iTj{T]~Wi)dZJ^

- w^ + e-W'+^dZjiri

+ wj)].

Since the vector of processes {Z; (X)} has orthogonal increments, the spectral mass of the process {Xn} is given by

iJ^^^-^)^--/)/7,(x-^) (1.9) EdZ(X)dZ(fi)

+ J2/e-iiTj-Tk)a+Wj)fMX

=

+ wj)\ dk,

where the first sum is over (j, k) such that X — /l = W j ± Wfc — T) ^ 0

and the second primed sum over (j, k) such that X - fl = -Wj ±Wk = T]^0,

0,

otherwise. 463

1118

K.-S. LII AND M. ROSENBLATT

The estimation method proposed in this paper can be used to estimate the frequen cies Wj, the spectral densities of the Y^'s and the r / s under certain assumptions. We note that the remarks for almost periodic sequences imply that a moving aver age process m 7=0

with the sequences (in n) {aj (n)}, j = 0, 1 , . . . , m, almost periodic and the process {£„} a white noise process, is a process with almost periodic covariance func tion. That is also true of the autoregressive scheme Xn — anXn-\ = %n if {an = Y,j ctjelPjn} is an almost periodic sequence with J^j \aj\ < 1 ( s e e a l s o [13]). We note that in meteorological, astronomical and engineering applications questions of periodic or almost periodic covariances are often of interest (see [17] and [20] for examples). The organization of the rest of the paper is as follows. In the next section notation and assumptions are given. The main result on the es timation of intercepts of almost periodic spectral support lines using an integrated periodogram [see (2.6)] is presented after Corollary 4. The estimation method is based on Theorem 4. Theorems 5 and 7 show that the asymptotic bias and variance of spectral estimates on the estimated lines are the same as those when the support lines are known. Section 3 contains a simulated example to demonstrate the effec tiveness of the methods. The Appendix contains most of the proofs of the results. 2. Results. In this section we introduce notation and assumptions, and state the main results. The principal thread of the derivation will be indicated. However, the detailed proofs of the results will be given in the Appendix. By (2.1)

W = Zmod27r,

it is to be understood that —n < W < n and if {u} is the integer I with —1/2 < u - I < 1/2, then (Zmod27r) = Z - {Z/(27r)}2;r. Consider the finite Fourier transform /i/2

(2.2)

Fn{X)= J2

t=-n/2

X e -itX

t~

and the periodogram (2.3)

In(X,ix) =

■Fn(k)Fn(fi).

2-izin + 1) Here we assume that n is even and t is summed over integers. Let K(t]) be a continuous nonnegative symmetric weight function with finite support and such that /f^ K(r))dt] = 1. Set Kn(n) = b-lK(b-lr}) with b„ I 0 and nbn -► oo as n -* oo. A plausible estimate of fw(v) i s given by (2.4)

fwin)

,(/z + w, (i)Kn(fx - 7])dix. 464

1119

ALMOST PERIODICITY AND ESTIMATION

ASSUMPTION 1. {X^}, EXk = 0, is a harmonizable Gaussian sequence with almost periodic covariance function. The set X of b's with nontrivial support is finite, and the fb(p), b e X, are continuously differentiate on the curve X = (b + /x)mod27r on the torus of points (A, p) e [—or, 7r]2, with (—7r, /X) identified with {it, p) and (X, —7t) with (A., 7r). Now

rn,m = £ X„Xm = £

f

e^+^e-WfbWdn

= £

beXJ~K

(2.5) cb(n) := r

J—71

***Q,(n - m),

beX

einkfb(X)dX.

2. There is a function c(j) > 0 on the integers with \cb(j)\ < c(j) for all b e X and J^j CU) < °°- Unless otherwise noted, the summation over j is on the integers. ASSUMPTION

ASSUMPTION

3. The fb(p),

beX,

are such that f*n fb(p) dp / 0.

In general, it is possible that there may be spectral lines b with f*n fb(p) dp = 0 (see [11], page 347). In that case let X consist only of those b with f*n fb(p) dp ^ 0 and our arguments will be valid for the restricted X. The local extrema of the absolute value of the integrated periodogram (2.6)

Un(b):= T In(b + p,p)dp J n

= -^— H + l

~

T

X\e-ihk

k=-n/2

will be used to estimate the b e X under Assumption 3. The fact that (2.6) is a periodogram computed for X\ is due to the uniform weighting in integrat ing In(b + /x, /x). With nonuniform weighting, one would get more complicated quadratic forms that might be useful if f*n fb(p) dp = 0. Theorems 1 and 2 give estimates for EIn(X, p) and cow (In(p + u;, /x), ln{pf + u/, pf)) and it is crucial that they are given with error terms uniform in the u/s and /x's. Let (2.7)

y(b, w) — {b~

w)mod2n.

It will also be convenient to make use of the following expressions for b, bf e X when X = p + w, Xf = p! + wf: y{\) = (Xf + b-X)mo&27T, (2.8)

y(2) = {-p

+p, + b') mod2jr,

y(3) = (-p' + b - X)mod27T, y(4) = (X' + p + bf)mod27t. 465

1120

K.-S. LII AND M. ROSENBLATT

Also let

(2.9)

si„c W=sin (i±i y y^±i y y

THEOREM 1.

ifx

= /n + w,

(2.10) EIM. ») = E W ^ y

T" + 0(—)

^ ((#i + l)/2)y(fc,u;) vWf/i uniformity inw,ix when Assumption 1 is satisfied.

V n

/

2. Assumption 1 araj? A = // + w, k' = p' + w' imply that

THEOREM

cov(ln(p + w, fi), I„(p' + w', p'))

(2-11)

x J X ; M-^)sinc(y(2)) +

o ( ^ ) J

x f E/vOi' + ^ s i n c ^ ^ + O ^ ^ ) ! . Theorem 2 almost immediately implies Corollary 1. COROLLARY 1. Assumption 1 implies that cov(fw(ri)9fw'(ri')) = E

pit

rn

/

J nJ n

/fc(M/ + ^ / )A'(-/)sinc( ) ;(l))sinc();(2))^„( A i-/ 7 )

/

b,b'€£ - -

x Kn(p' — n')dpdp' + E

/

/

/fc(-M / )/fe'(^ + w/)sinc(3;(3))sinc(y(4)) x Kn(p-n)Kn(p'

/loen\ o(fe). 466

-t)')dpdp

1121

ALMOST PERIODICITY AND ESTIMATION

/ 1 \

c o v ( / ^ ) , Mr,')) Bounds on sup^ \Un(w) — EUn(w)\ rem 3. Let (2.14)

= 0{—)

/logn

+

0{-2-

will be obtained by making use of Theo

Vn(w) = (* + l){t/„(u;) -

EUn(w)}.

Cumulants of Vn(w) will be estimated and an argument akin to Brillinger [2] will be used to gauge sup^ | Vn (w) |. THEOREM 3. Let {X^} be a Gaussian sequence with almost periodic covariance function satisfying Assumptions 1 and 2. Then (2.15)

limsupsup|V„(u;)|/(nlogn)1/2<25/2? Q > 0 )

,

with probability 1 where q is the number of elements in X. The arguments used to prove Theorem 3 will imply the following two corollar ies. COROLLARY 2. Let c > 0. Then under the conditions of Theorem 3, there is a constant C > 0 such that (2.16)

sup IUn(w) — EUn(w)\

COROLLARY

> c <exp[-Oz1/2(logn)1/2].

3. If c > 0 isfixedbut large, 2cn'{/2(logn)l/2 < e x p { - - l o g n

(2.17) P sup\U n (w) - EUn(w)\ >

An additional corollary indicates that sup w | Vn(w)\ diverges as n —► oo. COROLLARY 4. Assume that fo(-) is not identically zero. Then under Assump tions 1-3, there are a countable number of distinct W( in the neighborhood of zero such that, for each finite subcollection C, (2.18)

— _ ^:Vn(wi),

Wi eC

are jointly asymptotically normal and independent with variances each greater than a constant £ > 0. Further, (2.19)

1 = 0 sup|V w (u;)

in probability as n -> oo. 467

1122

K.-S. LII AND M. ROSENBLATT

We now describe the estimation procedure. Fix a given level 8 > 0. Consider the number ofbeX with (2.20)

/

Ml*) dp >8.

The following procedure will yield estimates wn(b) of b e X satisfying (2.20) which are such that n\wn(b) — b\ < an = n~1^5 except for a set of probability less than 0(\ogn/n2) ifn is sufficiently large {i.e., as n -> oo). Divide the range (it can be taken as —n < w < it) ofw into blocks of length \/n and compute the global maximum of \ f*n In(/jL + w, ii)d\x\ in each block. Throw out the blocks with maxima that fall less than or equal to 8 + (- 5 f^) 1/4 . Also, if there are two maxima located within \/n of each other, throw out the block with the smaller maximum. As n —> oo, the locations wn(b) at which the remaining maxima oc cur satisfying (2.20) are such that n\wn(b) — b\ < an = n~l/5 except for a set of probability less than 0(\ogn/n2) as n -> oo. We sketch the argument for this. Using (2.10), it is seen that

£ M^d^+°(^y

T?TT , x ^ s i n ( ( n + l)/20>-u;)) EUn(w)= 2^ beX

(n +

l)/2(b-w)

The maximum of sin((n + l)/2)(fc-u0| II Mii)dfi * I ((n + l)/2)(b-w) is clearly at w = b given Assumption 1. The contribution from the other terms in \EUn(w)\ with b' i^b,b! eX is asymptotically 0 ( i ) in a neighborhood {w:\bw\ < mirib'^b^ex \W - b\} ofbeX. First note that, uniformly in w, (2 21)

(2.22)

*<¥)

\Un(w)\-\EUn(w)\

1/2

2

except on a set of probability less than 0(n~ ) by Corollary 3. We shall show that the local maximum wn(b) converging to b must be such that \n(wn(b) - b)\ < n~l/5 except for a set of probability less than 0(logn/n2). First note that II rn | I f* sin((fi + l)/2)(fc-u;)| / In{ti + w,ii)dii\ - / fb(li)dii((n + l)/2)(b - w) II J—K I I J— 7T

^(T)

1/2

in a fixed nontrivial interval I(b, s) = {w: \b - w\ < e], 0 < e < 5, containing b but excluding the other points bf e X except for a set of probability 0(logn/n2). The value of \fln(fi + w, /x) d^\ at w = b is at least I f71

I 468

/logn\1/2

1123

ALMOST PERIODICITY AND ESTIMATION 1//5

However, the value outside \n(w — b)\ < n w, fx)d)x\, is at most

smn 1/5

©:= IT M^)dfi

1/5

I J—n

, but inside I(b, s), of \ f In(ix +

+ c2

and since sinn

-1/5 1/5

,-2/5

= 1-

3!

m

/WB\l/2

+ 0(n- 4 / 5 ),

0 i s larger than ©for n sufficiently large. Consequently, we have the following theorem. THEOREM 4. Let 8 > 0 and Assumptions 1-3 hold. There are then estimates wn(b) of values b € X with \ f*K ft(p)dp\ > 8 such that n\wn(b) — b\ < n~1^5 except for a set of probability 0(\ogn/n2) ifn is sufficiently large. A similar but more detailed argument yields a better estimate for \wn{b) — b\. C O R O L L A R Y 5. Under the assumptions of Theorem 4, (n + \){b — wn(b)) = 0 ( n - 1 / 4 ( l o g n ) 1 / 4 ) and [(n + \){b - wn(b))]2 = 0(supw \Un(w) - EUn(w)\) ex cept for a set of probability 0(logn/n2).

The following results show that if one uses these estimates wn (b) of b to esti mate the spectral densities at b, the bias and variance are asymptotically as good as for the estimates with b e X given. THEOREM

5.

If\fH.„ fb(l^)d/x\

> 8 and Assumptions 1-3 hold,

Efwn(b)(ri) _rsm((n+l)/2)y(b,wn(b)) L «n +

f*

l)/2)y(b,wn{b))

£* fb(n)K„(ji - V)dij] + o(^jp)

and the right-hand side is Mfi)

+ jfb(v)

f^x2K(x)dx

+ 0{b2n) + Oin-^ilogn)1'2)

Corollaries 2 and 3 are used in the derivation of this result. THEOREM

6.

LetS>0

and Assumptions 1-3 hold. If

r MH) dp.

I/

>«,

J — 7Z

J—n

469

fi/(ti)dfi >8

+

o(-±-\

1124

forb,bf

K.-S. LII AND M. ROSENBLATT

e X, then cow(fWnib)(rih fWnm(rif))

=° ( ^ )

+ °(^P)

as n -> oo, bn ], 0, nbn —> oo. THEOREM 7. If\f-n fb(l^) dyi\ > 5 and Assumptions 1-3 /*o/
nbn

Mri + b)fo(-ri)

+

^y(-2r]

+

b'-b)\fb,(-r])\7

f°° K2(x)dx +

o(-^).

J-oo

bfe£

V n

)

A comparison of Theorems 5 and 7 with Proposition 5.1 and Corollary 5.1 in [15] indicate that the bias and variance of the spectral estimates using estimates wn (b) of b e X have the same behavior asymptotically as spectral estimates as suming complete knowledge of b e X. 3. Examples. In this section we give a simulated example. Consider the am plitude modulated (AM) signal model in communication (3.1)

y(t) = cos(coit)Zi(t) + cos(a)2t)Z2(t),

EZ|(f)=0,

i = 1,2,

where Z\(t) and Z2(t) are independent stationary processes with spectral den sities f\{X) and /2(A) at frequency A, respectively. Then {y(t)} is harmonizable with almost periodic covariance function and spectral support lines on A = /z, A = IJL ± 2o)\ and A = /x ± 2&>2- We consider (3.1) with co\ = -^=, &>2 = 77^, and Zi(r) = £r + 0.5st-\, Z2(t) = r]t+ 0.3rjr_i, where st and *}r are independent standard Gaussian sequences. In this case, y(t) = f eltXdZy{X) with dZy{X) = \[dz\(X zp a>i) + dzi{X ^ G>2)] and Zt(t) = feitkdzi(X) for i = 1,2. Therefore, the spectrum of {y(t)} is given by E[dZy(X)dZy(iM)]

+ f\(X

=F CWl)5x,/x±2(wi + /2(A- T <02)8k,ii±2a>2}'

Given a realization of {y(t)}9 we are interested in estimating co\ and &>2 and the spectral densities on the lines given by X = \x ± 2a>i and A = /x ± 2<^2As an example, a simulated realization of {y(t)} for f = -512 to 512 is gen erated. A stretch of {Z\(t)} and {y(t)} are given in Figure 1. The periodogram 470

F I G . 1.

Simulated time series with z\(t),Z2(t) All

independent stationary time series.

1126

K.-S. LII AND M. ROSENBLATT

of {y(t)} is given in Figure 1 also. There is no apparent information on co\ and 0)2 in the periodogram. The Un(b) statistic of {y(t)} is computed for a range of 6's. For b = bj = ^ with 7 = 1,2 512, Un(b) and E(Un(b)) are given in Figure 2(a)-(f). Two spikes are apparent at frequencies near 2&>i and 2&>2. The largest two b's in |£/„(6)| occurred at a>\ = 0.45406 and cb2 = 0.73938. When the £/„(£) are cal culated at a more refined grid with b = bj = 4 ^ , j = 1,2,..., 2048, the largest I [/„(&) I occurred at co\ = 0.45329 and a>2 = 0.74015. These latter values are closer to the true values co\ = 0.45345 and a>2 = 0.74048. These are plotted in Figure 2(g) and (h). The scale in figures (g) and (h) differs from that in figures (e) and (f) since there is leakage at frequencies very close to zero because the computation at (e) and (f) is at a more refined set of frequencies. The zero frequency corre sponds to the diagonal line which is the support line of the "power spectrum" of the process. Therefore, the value of \Un(0)\ is relatively large. It is this mass that leaks. A large value of \Un(b)\ suggests that X = fi + b is a spectral support line. Figure 3 gives the plots of spectral estimates on these estimated support lines. The dashed line gives the theoretical spectral density if the intercept b is known exactly. In these cases the imaginary part is zero. For the diagonal line X = /x, there is no need to estimate the intercept. The estimated spectral densities on each line when the intercepts are exactly known are given by the solid lines. The dotted lines are spectral estimates when the intercepts are estimated with a crude search, while the dash-dot lines are spectral estimates when the intercepts are estimated by the more refined search. When the sample size of {y(t)} is increased to t = -2048 to 2048, we have more resolution as expected. These graphs show that intercepts of these support lines or frequencies or "pe riods" of the covariance function can be effectively estimated using Un(b). These intercepts can be estimated sufficiently accurately to give accurate estimates of the spectral densities on the spectral support lines. APPENDIX For the proofs, the following will be useful. Let r w x sin((n + l)/2)* Dn(x) = . , sin(;c/2)

(A.1) LEMMA

(A.2)

r J—TC

A.1.

~ sin((* + l)/2)s Dn(x) = — . x/2

If\y\<7t9

Dn(x + y)Dn(-x)dx=

I J—7T

Dn(x + y)Dn(-x)dx

with O uniform in y. All

+ 0(log/i),

473

474

ALMOST PERIODICITY AND ESTIMATION

11

The argument given in [15] for Lemma 7.2 holds for this case. 1. Given X,, t = -n/2,...,n/2,

PROOF OF THEOREM

EIn(k,n)

=—

V

/

then

Dn{v + b-ti-w)Dn(-v

+

fi)fb(v)dv

beX w i t h G

l = 2^^J-nDn(v-^

+ y(b,w))Dn{-V

+ ^)fb^^^

differentiability of fb implies that (A.4)

fb(v) = Mix) + (v-

n)ft(v%

with v* between /x and v. Then G\ = G2 + G3 with [denoting y(b, w) by y] G2 = —

f

2n(n + 1) J-iT

G3 = —-!-—fb(li)

Dn(v-fi ^

27t(n + l)

J-7T

+ y)Dn(-v

Dn(x +

+

fi)(v-ii)fj,(v*)dv,

y)Dn(-x)dx.

\ogn

It is clear that \G2\ = O ( ^ ) . Also,

using Lemma A. 1. To simplify the notation, we define

\LjA\f^dx=iijA.f^dxl/=l

Now G4

x)dx

27r(« 27r(« ++ 1) J-jr 1 2n(n + 1)

(A.5)

If"/

sin(l/2)(z + (n + l)y) sin(l/2)z

dz

(z + (n + l)y)/2(n + 1) z/2(n + 1) n + 1

= G5 — G6, with G5 and G6 the two integrals of the previous line. Notice that

|G6|

-y-{(

271 [J(n+l)n + \>\z\>{n+l)7T

+

(

^2(«+l)jr>|z|>(n+l)7r + l

sin(l/2)(z + (n + \)y) sin(l/2)z dz, (z + (n + \)y)/2 z/2 475

+

f

1

J\z\>2(n+l)7t i

K.-S. LII AND M. ROSENBLATT

with the integrand less than or equal to \/({n + \)n) in the first integral, less than or equal to ,n+l)n (z + (n + l)y)~l in the second integral and less than 2/z2 in the third integral (because |y|
The Parseval relation implies that G5 = sinc(y). Applying the above bounds for the Gt 's proves the theorem. D 2. By a direct calculation, we have four expressions of the same type as that analyzed in Theorem 1. Using (2.8) and (2.10), formula (2.11) is obtained. □ PROOF OF THEOREM

PROOF OF COROLLARY 1. The first equation follows directly from Theo rem 2. The order of magnitude statement is obtained by making use of the Schwarz inequality. □ PROOF OF THEOREM

3.

First notice that, for distinct indices in the range

-n/2 to n/2, (A.7)

c u m ( 4 , X\,...,

X)s) = 2s~l ]Tr j u j a i r j a 2 j^

... rjasJl,

where v is a listing of the (s — 1)! irreducible sequences 1 -> o?2 -> > ots - > 1 with c*2, •.., OLS distinct choices of j'2,. ♦., j s . The case in which the indices are not distinct is obtained from (A.7) by making the appropriate identification of identical indices. By (2.5), (A.8)

IOW 2 |'S are uniformly bounded by a summable c(-) > 0, (A.9) \rjlJa2,rja2ja3 . ..rjasJl \ < qsc{jx - ja2)c(ja2 ~ ja3)' "c(jas - 71). We shall carry out an argument for Re{Vn(b)} alone since that for Im{Vn(w)} is similar. Let an be the variance of Re{Vn(w)}. Then (A. 10)

log£exp[aRe{V^(u;)}] 00

-a2on/2 /

<(B+l)^)V s=3

00

\

5

£

C(j)\ JS

^J=~oo

/

since the ^th cumulant of Vn(b) is less than or equal to (s - l)\(n + l)(2q)s x (Hj CU))S' If ^ > 0 is sufficiently small, (A.ll)

logEcxp[aRe{Vn(b)}]

<(n + l)q2\Tc(j)\ 476

(1 +e)a2

sup|Re{V„(M;)}|>2fl W

for some constant C. By (A. 14) and Chebyshev's inequality, the probability on the 477

478

1133

ALMOST PERIODICITY AND ESTIMATION

nlogn and a = n~ 1 / 2 (log«) 1 / 2 . Using (A. 16), we see that then there is a C > 0 such that z /z <exp(-Cn l / n, (logn) J1/2^ ).

sup\Re{Un(b)-EUn(b)}\>

The same type of inequality holds for Im{Un(b) - EU„(b)}. Thus, the corollary holds. □ 3. The arguments are similar but now set a = , R = nlogn and a = n~ 1 / 2 (logn) 1 / 2 with c > 0 fixed but large.

PROOF OF COROLLARY

cn

1/2

(logn)

1//2

□

PROOF OF COROLLARY

4.

Trivially,

sup\Un(w)-EUn(w)\2>—

f

EUn(w)\2dw

\Un(w) -

n/2

= E tf-EXJ) n + 1 2x2

j=-n/2

The covariance of j^(Un(w)

- EUn(w)) and j^(Un(r))

- EU„(rj)) is

n/2

£ («+l)2 . /

cov(X2j,X2k)e-iJw+ik\

..

'2 v 2 ^ with cov(Xy, Xp = 2rjk since {Xt} is a Gaussian process. But then

(1):=

n/2 n/2

E

1

tl

(« + D2 j,k=-n/2 =-n/2ib€X

E^u-*)

\ \2 -

-ijw+ikr]

J

(n + l) 2 7T

/ b,b'e£

sin ((« + l)/2)(£ + // - w + a) sin((» + X)/2){r) - a) sm(\/2)(b + b'-w + a) sin(l/2)(i;-a) x fh*

fb'(a)da,

where "*" denotes convolution. If r\ = w — b — b', we essentially get a Fejer kernel centered at this point and so

0) = -X7 E S(r,-w + b + b')fb*Mr,) +

o(\)

if there are a finite number of b in =£. Notice that, for ?j = w, we get (

2TT

1) = — ^ E /* * f-bW + 0(-A n+1^ V«2/ 479

1134

K.-S. LII AND M. ROSENBLATT

We first show joint asymptotic normality of Un(w) — EUn(w) for any finite num ber of w's. Also, there are an infinite number of wt in a neighborhood of zero such that wt - Wj +b + bf ^ 0 for any b,bf e X. For joint asymptotic normality, we have to deal with m

(A.23)

£[t/„(u>;) -

EU„(wj)]aj.

The argument for estimates of the cumulants of Vn (b) leading to Theorem 3 still holds for the 5th cumulant of (A.23) with the additional factor (2~2\aj\)s- The asymptotic variance is

(A.24) -JL-J2aJaj'

£

S(wj-wr

f

+ b + b')fb*Mwj)

+

o(-2).

b,b eX

JJ

We now consider conditions for positivity of the coefficient of -^. We trivially get asymptotic normality for y/n Y!J=\ aj Wn (wj) — EUn(WJ)] but with a possibly singular distribution because the 5th cumulants with s > 2 all tend to zero. The asymptotic variance of Un(wj) — EUn(wj) is -^ Y^beX fb * f-b(wj) + 0 ( £ ) . Now /o * / 0 (0) = fHn fo(w)2dw > 0 if /o is not identically zero. Further, for b e X,

/**/_*(())= r Mw)f-b(-w)dw= J—TC

r J—7T

\fb(w)\2dw>0.

Thus, J^beX fb * f-b(0) > 0 and this implies J^beX fb * f-b(ri) > £ > 0 for T? in some neighborhood of zero. But there are a countable number of wt ^ 0 in this neighborhood such that u;,- — Wj■ + b — bf ^ 0 for any b,bf e X. Therefore, the y/n[Un(Wj) — EUn(u)j)] are asymptotically jointly normal and independent with variance greater than 8 > 0. Therefore, l=o

sup\Vn[Un(Wi) - EUn(wi)]\

. Wi

in probability. We have the corollary. PROOF OF COROLLARY

omitted.

J

=o\ sup\y/n[Un(w) - EUn(w)]\ L W

□

5. The proof is similar to that of Theorem 4 and is

□

PROOF OF THEOREM

EIn(k9/i)=

5.

J

If X = \x + w, Theorem 1 gives

J2 MM beX \y(b,w)\<7T

sin(((#i + l)/2)y(b, w)) «n + l)/2)y(b,w) + 0

Since fwn(ri)= /

In(^ +

J — 7T

480

Wn,v)Kn(n-r})du,

m

1135

ALMOST PERIODICITY AND ESTIMATION

it follows that

■sm((n + |"sin(0

EfWn(b)(ri) = 2^E[—+ fn

l)/2)y(b\wn(b)) l)/2)yibfj

Wn(b))

/logn\ fv(ji)Kn(V>-ri)dn+0\—-\,

x

sm((n + l)/2)y(b,wn(b)) = (l) + (2), «n + l)/2)y(b,wn(b)) J where (1):=£L

sin((n + l)/2)y(b,wn(b)) \n{b-wn(b))\
L ((n + l)/2)y(b,w„(b)) with £ [ / ( X ) ; A] := fA f(x) dP(x) where P(-) is the distribution of X. Now (l) = E [ l - [ n ( & - i M k ) ) ] 2 / 3 ! + o([n(fc - wn(b))f); \n(b - u;„(fc))| < ^ - 1 / 4 ( l o g n ) 1 / 4 ] . By Corollary 3, 1 > (1) > (1 - ^ " ^ ( l o g / O ^ X l - ±), while |(2)| < x- if £ is large enough. Therefore, sin((n + l)/2)y(&, «>»(&))' «n + l)/2)y(b,wn(b)) Now xfb'^b

1/2 1/2 1
consider Jsm{(n + \)/2)y{b',wnQ>)) = (!') +(2'), L ((/i + l)/2)y(fc',w„(fc)) J

where , rsin(( sin((iH-l)/2)y(fc',u;n(fc)) ; |/i(fc-«;„(&))| > c (1' ) : = £ — L ((« + l)/2)y(fc',u;„(W ,

(2

rsin(( sin((/H-l)/2)y(*»', ">«(&)) ; |n(& — w„(b))\ < c L ((« + l)/2)j(^,u; n (&))

By Corollary 2 |(1')| < exp(-Cn 1/2 (log«) 1/2 ) + 0{n-2), |(2')| < C/n. Notice that | ( ^ ) 0 - w n O))l 2 < k sup | t / „ 0 ) - £{/„(u;)| except for a set of probability 0(n~2), which implies P[$n + $(b - wn(b))\ < crt~ 1/2 (logn) 1/2 ] > 1 - exp[-c'logn] for some c' and so P[$n + ${b - w„(b))\ > cn~'/2(logn)1/2] < exp[-c'logn]. These estimates imply that, if fa is twice continuously differentiable, EfWn(b){ji) — fa(ri+b-ffb'(ri)fx2K(x)dx+o(b2n)

+ 0(n-]/2(logn)V2). 481

□

1136

K.-S. LII AND M. ROSENBLATT

PROOF OF THEOREM

cov{fWn(b)(ri),

6.

First notice that

fWn(b/)(t]r))

= E[E[fWnib)(r])fWnm(T]')\wn(b)

= wn, wn(b') = w'n]]

-EfWn(b)(r))EfWn(b>}(r}') (A.25)

= E[co$fWn{b)(r)), fWn(b')(r]')\wn(b) = w„, wn(b') = w'n)] + E[E(fWn{b)(ri)\wn(b)

= w„, w„(b') = w'n)

x E{fWn{y){r)')\wn{b) = wn, wn(bf) = w'n)] -Efw„(b)(ri)EfWn{V)(ri'). The estimate for the first term on the right-hand side of (A.25) is obtained by making use of Corollary 1. The remaining terms on the right-hand side of (A.25) can be seen to be ^ [sinan + l)/2)y(b'f,wn(b)) sin((/i + $/2)y(b>", wn(b')) > cov , ^ L ((n + l)/2)y(b",wn(b)) ((n + \)/2)y{b"',wn{b>)) b",b'"eX

x T fb(n)Kn(n-n)dn J-jt

logn fv{n)Kn{n-ri')dn+oC^

^ J-7T

\

n

Of course, | cov(sinc >>(/?", wn(b)), smcy(b'", wn(b')))\ < {var(sincy(Z?", w„(b)))var(smcy(b'",

wn(b')))}1/2.

Notice that \ar(sinc y(b', wn(b))) is E[(smcy(b", w„(b)))2] - (E[smcy(b",

wn(b))])2

= E[(l + {(sine >>(*/', wn{b))) - l}) 2 ] - [E(l + {sine y(b", wn{b)) -

\))f

= E[{smcy{b", wn(b)) - l} 2 ] - (E[{sincy(b", wn(b)) - l}]) 2 . One has to consider the case in which b = b" and that in which b ^ b". Also, one should note that the expression above is less than or equal to E[{sine y(b", wn(b))-\}2]. If b = b", E[{sincy(b, wn(b)) - l} 2 ; \n(b - wn(b))\ = E

«n + l)/2)(b-wn(b))y

+

+

2\ 2 l)/2)(b-wn(b))\y

3!

\n{b - wn(b))\ < itn- 1 / 4 (logn) 1 / 4 l <

Clogn/n, 482

1137

ALMOST PERIODICITY AND ESTIMATION

while, by Corollaries 3 and 4, E[{sincy{b, wn(b)) - l} 2 ; \n(b - wn(b))\ > fcn~1/4(logn)1/4] < c/n if k is large enough. If b ^ b", l} 2 ; \n(b - wn(b))\

E[{smcy(b",wn(b))-

because wn(b) is close to b and b" — wn(b) is large, while £[{sinc >>(//', wn(b)) - 1| 2 ; \n(b - wn(b))\ > c] <e-CnW2(^n)]/2

+

0(n-2)

follows by Corollary 2 as n —► oo. The proof of Theorem 6 is complete. D PROOF OF THEOREM 7.

v**{fwn(b)(r\j) = £(var(/WnW(?7)|u;„(fc) = io„)) (A.26)

+ E[E{fWn{b){n)\xun{b) = wnf] -

{EfWn(b)(r]))2.

We first consider the first term on the right-hand side: var(fWn{b)(r])\wn(b) = wn) = f*n M

E Mn' + wn)sim(y(l))

+ o(^jp)

x

E

+

E /*(-/"') sinc(y (3)) + o ( ^ p ) x

A'(-^)sinc(j(2)) + o ( ^ j p )

£

/ ^ ' + w»)sinc(y(4)) + C > ( ^ p

x K„(ix- r])Kn(ix' - x])d[id\x . Making use of the symmetry of K about zero on taking the expectation, we obtain var(/W(1(fc)Oj)|wn(fc) = w„) lit

nb„

Mri +

b)M-r,)

+ YtY(~2ri

+

b,-b)\M-r,)\1

b'e£ 483

£

Kz(x)dx

+ 0

m

K.-S. LII AND M. ROSENBLATT

E[E(fWn(b)(ri)\wn(b) = wn)f -

{EfWn(b)(t])Y

.logn

can be shown to be 0 ( - * - ) just as in the discussion of the corresponding term in the proof of Theorem 6. □ REFERENCES V. G. (1988). Estimating the spectral densities of a Gaussian periodically corre lated stochastic process. Problems Inform. Transmission 24 109-115. MR0955986 BRILLINGER, D. (1975). Time Series: Data Analysis and Theory. Holt, Rinehart and Winston, New York. MR0443257 DANDAWATE, A. V. and GIANNAKIS, G. B. (1994). Nonparametric polyspectral estimators for &th order (almost) cyclostationary processes. IEEE Trans. Inform. Theory 40 67-84. DEHAY, D. and LESKOW, J. (1996). Functional limit theory for the spectral covariance estima tor. J. Appl Probab. 33 1077-1092. MR 1416228 GARDNER, W. A. (1991). Exploitation of spectral redundancy in cyclostationary signals. IEEE Signal Processing Magazine 8 (2) 14-36. GARDNER, W. A., ed. (1994). Cyclostationarity in Communications and Signal Processing. IEEE Press, New York. GERR, N. and ALLEN, J. (1994). The generalized spectrum and spectral coherence of a harmonizable time series. Digital Signal Processing 4 222-238. GERR, N. and ALLEN, J. (1994). Time-delay estimation for harmonizable signals. Digital Signal Processing 4 49-62. GLADYS HE v, E. G. (1963). Periodically and almost-periodically correlated random processes with a continuous time parameter. Theory Probab. Appl. 8 173-177. HURD, H. (1989). Nonparametric time series analysis for periodically correlated processes. IEEE Trans. Inform. Theory 35 350-359. MR0999650 HURD, H. and GERR, N. (1991). Graphical methods for determining the presence of periodic correlation. J. Time Ser. Anal. 12 337-350. MR1131006 HURD, H. and LESKOW, J. (1992). Strongly consistent and asymptotically normal estima tion of the covariance for almost periodically correlated processes. Statist. Decisions 10 201-225. MR1183203 HURD, H., MAKAGON, A. and MIAMEE, A. G. (2002). On AR(1) models with periodic and almost periodic coefficients. Stochastic Process. Appl. 100 167-185. MR1919612 LESKOW, J. and WERON, A. (1992). Ergodic behavior and estimation for periodically corre lated processes. Statist. Probab. Lett. 15 299-304. MR1192541 Lll, K.-S. and ROSENBLATT, M. (2002). Spectral analysis for harmonizable processes. Ann. Statist. 30 258-297. MR 1892664 LOEVE, M. (1963). Probability Theory, 3rd ed. Van Nostrand, Princeton, NJ. MR0203748 LUND, R., HURD, H., BLOOMFIELD, P. and SMITH, R. (1995). Climatological time series with periodic correlation. J. Climate 8 2787-2809. TIAN, C. J. (1988). A limiting property of sample autocovariances of periodically corre lated processes with application to period determination. J. Time Ser. Anal. 9 411-417. MR0995617 [19 WOODROOFE, M. B. and VAN NESS, J. W. (1967). The maximum deviation of sample spec tral densities. Ann. Math. Statist. 38 1558-1569. MR0216717 ALEKSEEV,

484

ALMOST PERIODICITY AND ESTIMATION

1139

[20] YAGLOM, A. M. (1987). Correlation Theory of Stationary and Related Random Functions 1, 2. Springer, Berlin. MR0893393, MR0915557 [21] ZYGMUND, A. (1959). Trigonometric Series 1, 2, 2nd ed. Cambridge Univ. Press. MR0107776 DEPARTMENT OF STATISTICS

DEPARTMENT OF MATHEMATICS

UNIVERSITY OF CALIFORNIA, RIVERSIDE

UNIVERSITY OF CALIFORNIA, SAN DIEGO

RIVERSIDE, CALIFORNIA 92521

LA JOLLA, CALIFORNIA 92093

USA E-MAIL: [email protected]

USA E-MAIL: [email protected]

485

486

K. S. Lii and M. Rosenblatt. Prolate spheroidal spectral estimates. Statist. Probab. Lett., 78(11):1339–1348, 2008. Reprinted with permission of Elsevier Inc.

Selected Works of S.L.Sobolev

Read more

Selected works

Read more

Selected Works

Read more

Selected works

Read more

Selected works of A.I. Shirshov

Read more

Selected Works of Debabrata Basu

Read more

Selected works of Emil Wolf

Read more

Selected works. Vol.1

Read more

Selected Short Works

Read more

Foundational Studies, Selected Works

Read more

Selected works. Vol.2

Read more

Selected Works, Vol. IV

Read more

Selected Works, Vol. III

Read more

Selected Works, Vol. I

Read more

Selected works. Vol.3

Read more

Selected works. Vol.4

Read more

Foundational Studies, Selected Works

Read more

Selected Short Works

Read more

Selected Works, Vol. II

Read more

Foundational studies: Selected works,

Read more

Kengo Kuma: Selected Works

Read more

Murray Leinster: The Life and Works

Read more

Selected works. - Mathematics and mechanics

Read more

Selected Works Of Professor Herbert Kroemer

Read more

SELECTED WORKS OF WEN-TSUN WU

Read more

Selected Works Of Wen-Tsun Wu

Read more

Reason and Violence: Selected Works of R.D. Laing (Selected Works of R.D. Laing, 3)

Read more

Architects (Selected & Current Works)

Read more

Selected works by Vinod Mishra

Read more

Selected Works of A.I. Shirshov (Contemporary Mathematicians)

Read more

Recommend Documents

Selected Works of S.L.Sobolev

SELECTED WORKS OF S.L. SOBOLEV Volume I: Mathematical Physics, Computational Mathematics, and Cubature Formulas SELECT...

Selected works

Nail H. Ibragimov SELECTED WORKS Volume II MSc and PhD theses Nonlocal symmetries Approximate symmetries Preliminary g...

Selected Works

...

Selected works

Nail H. Ibragimov SELECTED WORKS Volume I Lie group analysis, Differential equations, Riemannian geometry, Lie–B¨ ackl...

Selected works of A.I. Shirshov

A.I. Shirshov Contemporary Mathematicians Gian-Carlo Rota† Joseph P.S. Kung Editors Selected Works of A.I. Shirshov...

Selected Works of Debabrata Basu

Selected Works in Probability and Statistics For other titles published in this series, go to www.springer.com/series/...

Selected works of Emil Wolf

Selected works. Vol.1

NAIL H. IBRAGIMOV SELECTED WORKS I ALGA Publications Nail H. Ibragimov SELECTED WORKS Volume I Lie group analysis,...

Selected Short Works

Foundational Studies, Selected Works

STUDIES IN LOGIC AND THE FOUNDATIONS OF MATHEMATICS VOLUME 93 Editors J. BARWISE, Madison D. KAPLAN, Los Angeles H. J....