PROBABILISTIC INEQUALITIES
SERIES ON CONCRETE AND APPLICABLE MATHEMATICS Series Editor: Professor George A. Anastassiou Department of Mathematical Sciences The University of Memphis Memphis, TN 38152, USA
Published Vol. 1
Long Time Behaviour of Classical and Quantum Systems edited by S. Graffi & A. Martinez
Vol. 2
Problems in Probability by T. M. Mills
Vol. 3
Introduction to Matrix Theory: With Applications to Business and Economics by F. Szidarovszky & S. Molnár
Vol. 4
Stochastic Models with Applications to Genetics, Cancers, Aids and Other Biomedical Systems by Tan Wai-Yuan
Vol. 5
Defects of Properties in Mathematics: Quantitative Characterizations by Adrian I. Ban & Sorin G. Gal
Vol. 6
Topics on Stability and Periodicity in Abstract Differential Equations by James H. Liu, Gaston M. N’Guérékata & Nguyen Van Minh
Vol. 7
Probabilistic Inequalities by George A. Anastassiou
ZhangJi - Probabilistic Inequalities.pmd
2
6/24/2009, 4:38 PM
Series on Concrete and Applicable Mathematics – Vol. 7
PROBABILISTIC INEQUALITIES George A Anastassiou University of Memphis, USA
World Scientific NEW JERSEY
•
LONDON
•
SINGAPORE
•
BEIJING
•
SHANGHAI
•
HONG KONG
•
TA I P E I
•
CHENNAI
Published by World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link, Singapore 596224 USA office: 27 Warren Street, Suite 401-402, Hackensack, NJ 07601 UK office: 57 Shelton Street, Covent Garden, London WC2H 9HE
British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library.
PROBABILISTIC INEQUALITIES Series on Concrete and Applicable Mathematics — Vol. 7 Copyright © 2010 by World Scientific Publishing Co. Pte. Ltd. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the Publisher.
For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher.
ISBN-13 978-981-4280-78-5 ISBN-10 981-4280-78-X
Printed in Singapore.
ZhangJi - Probabilistic Inequalities.pmd
1
6/24/2009, 4:38 PM
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Dedicated to my wife Koula
v
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
This page intentionally left blank
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Preface
In this monograph we present univariate and multivariate probabilistic inequalities regarding basic probabilistic entities like expectation, variance, moment generating function and covariance. These are built on recent classical form real analysis inequalities also given here in full details. This treatise relies on author’s last twenty one years related research work, more precisely see [18]-[90], and it is a natural outgrowth of it. Chapters are self-contained and several advanced courses can be taught out of this book. Extensive background and motivations are given per chapter. A very extensive list of references is given at the end. The topics covered are very diverse. Initially we present probabilistic Ostrowski type inequalities, other various related ones, and Grothendieck type probabilistic inequalities. A great bulk of the book is about Information theory inequalities, regarding the Csiszar’s f -Divergence between probability measures. Another great bulk of the book is regarding applications in various directions of Geometry Moment Theory, in particular to estimate the rate of weak convergence of probability measures to the unit measure, to maximize profits in the stock market, to estimating the difference of integral means, and applications to political science in electorial systems. Also we develop Gr¨ uss type and Chebyshev-Gr¨ uss type inequalities for Stieltjes integrals and show their applications to probability. Our results are optimal or close to optimal. We end with important real analysis methods with potential applications to stochastic. The exposed theory is destined to find applications to all applied sciences and related subjects, furthermore has its own theoretical merit and interest from the point of view of Inequalities in Pure Mathematics. As such is suitable for researchers, graduate students, and seminars of the above subjects, also to be in all science libraries. The final preparation of book took place during 2008 in Memphis, Tennessee, USA. I would like to thank my family for their dedication and love to me, which was the strongest support during the writing of this monograph.
vii
Book
22nd June 2009
18:37
viii
World Scientific Book - 9.75in x 6.5in
PROBABILISTIC INEQUALITIES
I am also indebted and thankful to Rodica Gal and Razvan Mezei for their heroic and great typing preparation of the manuscript in a very short time. George A. Anastassiou Department of Mathematical Sciences The University of Memphis, TN U.S.A. January 1, 2009
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Contents
Preface
vii
1. Introduction
1
2. Basic Stochastic Ostrowski Inequalities
3
2.1 2.2 2.3 2.4 2.5
. . . . .
3 4 7 10 12
3. Multidimensional Montgomery Identities and Ostrowski type Inequalities
15
3.1 3.2 3.3
Introduction . . . . . . . One Dimensional Results Multidimensional Results Applications . . . . . . . Addendum . . . . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Application to Probability Theory . . . . . . . . . . . . . . . . . .
4. General Probabilistic Inequalities 4.1 4.2 4.3 4.4
Introduction . . . . . . . . Make Applications . . . . Remarks on an Inequality Lq , q > 1, Related Theory
31 . . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
5. About Grothendieck Inequalities 5.1 5.2
31 31 35 36 39
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6. Basic Optimal Estimation of Csiszar’s f -Divergence 6.1 6.2
15 16 29
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
39 40 45 45 46
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
x
7. Approximations via Representations of Csiszar’s f -Divergence 7.1 7.2
55
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . All Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8. Sharp High Degree Estimation of Csiszar’s f -Divergence 8.1 8.2 8.3 8.4
Background . . . . . . . . . . . . . . . . . . . . . . . . Results Based on Taylor’s Formula . . . . . . . . . . . Results Based on Generalized Taylor–Widder Formula Results Based on an Alternative Expansion Formula .
73 . . . .
9. Csiszar’s f -Divergence as a Measure of Dependence 9.1 9.2
55 56
. . . .
. . . .
. . . .
. . . .
. . . .
73 74 84 92 101
Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
10. Optimal Estimation of Discrete Csiszar f -Divergence
117
10.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 10.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 11. About a General Discrete Measure of Dependence
135
11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 11.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 12. H¨ older-Like Csiszar’s f -Divergence Inequalities
155
12.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 12.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 13. Csiszar’s Discrimination and Ostrowski Inequalities via Eulertype and Fink Identities
161
13.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 13.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 14. Taylor-Widder Representations and Gr¨ uss, Means, Ostrowski and Csiszar’s Inequalities
173
14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 14.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 14.3 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 15. Representations of Functions and Csiszar’s f -Divergence
193
15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 15.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Contents
xi
16. About General Moment Theory 16.1 16.2 16.3 16.4
215
The Standard Moment Problem . The Convex Moment Problem . . Infinite Many Conditions Moment Applications – Discussion . . . . .
. . . . . . . . . . . . Problem . . . . . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
17. Extreme Bounds on the Average of a Rounded off Observation under a Moment Condition
215 220 224 226
227
17.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 17.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 18. Moment Theory of Random Rounding Rules Subject to One Moment Condition
239
18.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 18.2 Bounds for Random Rounding Rules . . . . . . . . . . . . . . . . 240 18.3 Comparison of Bounds for Random and Deterministic Rounding Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 19. Moment Theory on Random Rounding Rules Using Two Moment Conditions
249
19.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 19.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251 19.3 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 20. Prokhorov Radius Around Zero Using Three Moment Constraints 20.1 20.2 20.3 20.4
Introduction and Main Result Auxiliary Moment Problem . Proof of Theorem 20.1. . . . . Concluding Remarks . . . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
263 . . . .
. . . .
. . . .
. . . .
21. Precise Rates of Prokhorov Convergence Using Three Moment Conditions
263 265 267 268
269
21.1 Main Result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269 21.2 Outline of Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 22. On Prokhorov Convergence of Probability Measures to the Unit under Three Moments 22.1 22.2 22.3 22.4
Main Result . . . . . . . . . . . An Auxiliary Moment Problem Further Auxiliary Results . . . Minimizing Solutions in Boxes .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
277 . . . .
. . . .
. . . .
. . . .
. . . .
277 282 288 291
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
xii
22.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296 23. Geometric Moment Methods Applied to Optimal Portfolio Management
301
23.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301 23.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 23.3 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 24. Discrepancies Between General Integral Means
331
24.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 24.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 25. Gr¨ uss Type Inequalities Using the Stieltjes Integral
345
25.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 25.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 25.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352 26. Chebyshev-Gr¨ uss Type and Difference of Integral Means Inequalities Using the Stieltjes Integral
355
26.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355 26.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356 26.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368 27. An Expansion Formula
373
27.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373 27.2 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377 28. Integration by Parts on the Multidimensional Domain
379
28.1 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379 Bibliography
395
List of Symbols
413
Index
415
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Chapter 1
Introduction
This monograph is about probabilistic inequalities from the theoretical point of view, however with lots of implications to applied problems. Many examples are presented. Many inequalities are proved to be sharp and attained. In general the presented estimates are very tight. This monograph is a rare of its kind and all the results presented here appear for the first time in a book format. It is a natural outgrowth of the author’s related research work, more precisely see [18]-[90], over the last twenty one years. Several types of probabilistic inequalities are discussed and the monograph is very diverse regarding the research on the topic. In Chapters 2-4 are discussed univariate and multivariate Ostrowski type and other general probabilistic inequalities, involving the expectation, variance, moment generating function, and covariance of the engaged random variables. These are applications of presented general mathematical analysis classical inequalities. In Chapter 5 we give results related to the probabilistic version of Grothendieck inequality. These inequalities regard the positive bilinear forms. Next, in Chapters 6-15 we study and present results regarding the Csiszar’s f -Divergence in all possible directions. The Csiszar’s discrimination is the most essential and general measure for the comparison between two probability measures. Many other divergences are special cases of Csiszar’s f -divergence which is an essential component of information theory. The probabilistic results derive as applications of deep mathematical analysis methods. Many Csiszar’s f -divergence results are applications of developed here Ostrowski, Gr¨ uss and integral means inequalities. Plenty of examples are given. In Chapter 16 we present the basic moment theory, especially we give the geometric moment theory method. This basically follows [229]. It has to do with finding optimal lower and upper bounds to probabilistic integrals subject to prescribed moment conditions. So in Chapters 17-24 we apply the above described moment methods in various studies. In Chapters 17-19 we deal with optimal lower and upper bounds to the expected convex combinations of the Adams and Jefferson rounding rules of a nonnegative random variable subject to given moment conditions. This theory has 1
Book
22nd June 2009
2
18:37
World Scientific Book - 9.75in x 6.5in
PROBABILISTIC INEQUALITIES
applications to political science. In Chapters 20-22, we estimate and find the Prokhorov radius of a family of distributions close to Dirac measure at a point, subject to some basic moment constraints. The last expresses the rate of weak convergence of a sequence of probability measures to the unit measure at a point. In Chapter 23 we apply the geometric moment theory methods to optimal portfolio management regarding maximizing profits from stocks and bonds and we give examples. The developed theory provides more general optimization formulae with applications to stock market. In Chapter 24 we find, among others, estimates for the differences of general integral means subject to a simple moment condition. The moment theory part of the monograph relies on [28], [23], [89], [85], [88], [90], [86], [44] and [48]. In Chapter 25 we develop Gr¨ uss type inequalities related to Stieltjes integral with applications to expectations. In Chapter 26 we present Chebyshev-Gr¨ uss type and difference of integral means inequalities for the Stieltjes integral with applications to expectation and variance. Chapters 27-28 work as an Appendix of this book. In Chapter 27 we discuss new expansion formula, while in last Chapter 28 we study the integration by parts on the multivariate domain. Both final Chapters have potential important applications to probability. The writing of the monograph was made to help the reader the most. The chapters are self-contained and several courses can be taught out of this book. All background needed to understand each chapter is usually found there. Also are given, per chapter, strong motivations and inspirations to write it. We finish with a rich list of 349 related references. The exposed results are expected to find applications in most applied fields like: probability, statistics, physics, economics, engineering and information theory, etc.
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Chapter 2
Basic Stochastic Ostrowski Inequalities
Very general univariate and multivariate stochastic Ostrowski type inequalities are given, involving k · k∞ and k · kp , p ≥ 1 norms of probability density functions. Some of these inequalities provide pointwise estimates to the error of probability distribution function from the expectation of some simple function of the involved random variable. Other inequalities give upper bounds for the expectation and variance of a random variable. All are presented over finite domains. At the end of chapter are presented applications, in particular for the Beta random variable. This treatment relies on [31]. 2.1
Introduction
In 1938, A. Ostrowski [277] proved a very important integral inequality that made the foundation and motivation of a lot of further research in that direction. Ostrowski inequalities have a lot of applications, among others, to probability and numerical analysis. This chapter presents a group of new general probabilistic Ostrowski type inequalities in the univariate and multivariate cases over finite domains. For related work see S.S. Dragomir et al. ([174]). We mention the main result from that article. Theorem 2.1. ([174]) Let X be a random variable with the probability density function f : [a, b] ⊂ R → R+ and with cumulative distribution function F (x) = P r(X ≤ x). If f ∈ Lp [a, b], p > 1, then we have the inequality: P r(X ≤ x) − b − E(X) b−a " 1+q 1+q # q x−a q b−x q 1/q ≤ kf kp (b − a) + q+1 b−a b−a q 1 1 ≤ kf kp (b − a)1/q , for all x ∈ [a, b], where + = 1, (2.1) q+1 p q E stands for expectation. 3
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
4
Later Brneti´c and Peˇcari´c improved Theorem 2.1, see [117]. This is another related work here. We would like to mention Theorem 2.2 ([117]). Let the assumptions of Theorem 2.1 be fulfilled. Then 1/q q+1 + (b − x)q+1 P r(X ≤ x) − b − E(X) ≤ (x − a) · kf kp , (2.2) b−a (q + 1)(b − a)q
for all x ∈ [a, b]. 2.2
One Dimensional Results
We mention Lemma 2.3. Let X be a random variable taking values in [a, b] ⊂ R, and F its probability distribution function. Let f be the probability density function of X. Let also g : [a, b] → R be a function of bounded variation, such that g(a) 6= g(b). Let x ∈ [a, b]. We define g(t) − g(a) , a ≤ t ≤ x, g(b) − g(a) P (g(x), g(t)) = (2.3) g(t) − g(b) , x < t ≤ b. g(b) − g(a) Then Z b Z b 1 F (x) = P (g(x), g(t))f (t)dt. (2.4) F (t)dg(t) + (g(b) − g(a)) a a Proof. Here notice that F 0 = f and apply Proposition 1 of [32]. Remark 2.4 (on Lemma 2.3). i) We see that the expectation Z b Z b E(g(X)) = g(t)dF (t) = g(b) − F (t)dg(t), a
a
proved by using integration by parts. ii) By (2.4) we obtain
! Z b g(b) 1 − F (x) = g(b) − F (t)dg(t) (g(b) − g(a)) (g(b) − g(a)) a Z b − P (g(x), g(t))f (t)dt. a
Hence by (2.5) we get
g(b) 1 − F (x) = E(g(X)) (g(b) − g(a)) (g(b) − g(a)) Z b − P (g(x), g(t))f (t)dt. a
(2.5)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Basic Stochastic Ostrowski Inequalities
5
And we have proved that
F (x) −
g(b) − E(g(X)) g(b) − g(a)
=
Z
b
P (g(x), g(t))f (t)dt.
(2.6)
a
Now we are ready to present Theorem 2.5. Let X be a random variable taking values in [a, b] ⊂ R, and F its probability distribution function. Let f be the probability density function of X. Let also g : [a, b] → R be a function of bounded variation, such that g(a) 6= g(b). Let x ∈ [a, b], and the kernel P defined as in (2.3). Then we get F (x) − g(b) − E(g(X)) g(b) − g(a)
! Z b |P (g(x), g(t))|dt · kf k∞ , a sup |P (g(x), g(t))|, t∈[a,b] !1/q Z b |P (g(x), g(t))|q dt kf kp , a
if f ∈ L∞ ([a, b]), (2.7) if f ∈ Lp ([a, b]), p, q > 1 :
1 p
+
1 q
= 1.
Proof. From (2.6) we obtain Z b F (x) − g(b) − E(g(X)) ≤ |P (g(x), g(t))|f (t)dt. g(b) − g(a) a
The rest are obvious.
(2.8)
In the following we consider the moment generating function MX (x) := E(exX ),
x ∈ [a, b];
and we take g(t) := ext , all t ∈ [a, b]. Theorem 2.6. Let X be a random variable taking values in [a, b] : 0 < a < b. Let F be its probability distribution function and f its probability density function of X.
22nd June 2009
6
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
Let x ∈ [a, b]. Then xb F (x) − e − MX (x) exb − exa 1 kf k∞ x2 xa xb xa xb (exb − exa ) · x (2e − e − e ) + e (a − x) + e (b − x) , if f ∈ L∞ ([a, b]), 2 x2 max{(e − exa ), (exb − ex )} , (exb − exa ) "Z #1/q Z b x kf kp xt xa q xb xt q · (e − e ) dt + (e − e ) dt , exb − exa ≤ a x if f ∈ Lp ∈ ([a, b]), p, q > 1 : p1 + q1 = 1, andin particular, kf k 3 2xa 2 · e2xb (b − x) + e2xa (x − a) + e xb − exa ) (e 2x # 1/2 2 2 3 2xb 2exa+x 2exb+x , if f ∈ L2 ([a, b]). + − 2x e − x x Proof. Application of Theorem 2.5.
(2.9)
Another important result comes: Theorem 2.7. Let X be a random variable taking values in [a, b] ⊂ R and F its probability distribution function. Let f be the probability density function of X. Let x ∈ [a, b] and m ∈ N. Then m m F (x) − b − E(X ) m m b −a kf k∞ (2xm+1 − am+1 − bm+1 ) m+1 m+1 m m · + a + b − a x − b x , bm − a m m+1 if f ∈ L∞ ([a, b]), max(xm − am , bm − xm ) , bm − a"m #1/q Z Z b x kf k p · (tm − am )q dt + (bm − tm )q dt , bm − a m ≤ a x if f ∈ Lp ([a, b]); p, q > 1 : p1 + 1q = 1, (2.10) and in particular, m+1 kf k2 2x m m 2m 2m bm − am · m + 1 (b − a ) + x(a − b ) 1/2 2m2 2m+1 2m+1 + (b −a ) , if f ∈ L2 ([a, b]). (m + 1)(2m + 1)
Proof. Application of Theorem 2.5, when g(t) = tm . An interesting case follows.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Basic Stochastic Ostrowski Inequalities
Book
7
Theorem 2.8. Let f be the probability density function of a random variable X. Here X takes values in [−a, a], a > 0. We assume that f is 3-times differentiable on [−a, a], and that f 00 is bounded on [−a, a]. Then it holds 0 0 E(X 2 ) + 2(f (a) + f (−a)) + a(f (a) − f (−a)) · a3 6 4 41a · k6f 0 (t) + 6tf 00 (t) + t2 f 000 (t)k∞ . (2.11) ≤ 120 Proof. Application of Theorem 4 from [32].
The next result is regarding the variance Var(X). Theorem 2.9. Let X be a random variable that takes values in [a, b] with probability density function f . We assume that f is 3-times differentiable on [a, b]. Let EX = µ. Define s − a, a ≤ s ≤ r p(r, s) := s − b, r < s ≤ b, r, s ∈ [a, b]. Then
Var(X) + (b − µ)2 f (b) − (µ − a)2 f (a) · µ − (a + b) 2 1 2 0 2 0 + (b − µ)f (b) + (µ − a)f (a) + [(b − µ) f (b) − (µ − a) f (a)] 2 2 5a + 5b2 + 8ab 2 2 · µ + (a + b) − (a + b)µ − 6 Z Z b b kf (3) kp · |p(µ, s1 )| |p(s1 , s2 )| · kp(s2 , ·)kq ds1 ds2 , 2 (b − a) a a if f (3) ∈ Lp ([a, b]), p, q > 1 : p1 + q1 = 1, ≤ (2.12) Z bZ b kf (3) k1 · |p(µ, s )| |p(s , s )| · kp(s , ·)k ds ds , 1 1 2 2 ∞ 1 2 (b − a)2 a a (3) if f ∈ L ([a, b]). 1
Proof. Application of Lemma 1([32]), for (t − µ)2 f (t) in place of f , and x = µ ∈ [a, b]. 2.3
Multidimensional Results
Background 2.10. Here we consider random variables Xi ≥ 0, i = 1, . . . , n, n ∈ N, taking values in [0, bi ], bi > 0. Let F be the joint probability distribution function of (X1 , . . . , Xn ) assumed to be a continuous function. Let f be the probability density
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
8
n
F function. We suppose that ∂x1∂···∂x (= f ) exists on ×ni=1 [0, bi ] and is integrable. n Clearly, the expectation ! Z Z Z bn n b1 b2 Y E Xi = ··· x1 · · · xn f (x1 , x2 , . . . , xn )dx1 · · · dxn . (2.13) 0
i=1
0
0
Consider on ×ni=1 [0, bi ] the function n X Gn (x1 , . . . , xn ) := 1 − F (b1 , b2 , . . . , bj−1 , xj , bj+1 , . . . , bn ) j=1
(X ) + F (b1 , b2 , . . . , bi−1 , xi , . . . , xj , bj+1 , . . . , bn )` n 2
`=1 i<j
(n3 ) X F (b1 , b2 , . . . , xi , . . . , xj , . . . , xk , bk+1 , . . . , bn )`∗ + (−1)3 `∗ =1 i<j
+ · · · + (−1) F (x1 , . . . , xn ).
(2.14)
Above ` counts the (i, j)’s, while `∗ counts the (i, j, k)’s, etc. One can easily establish that ∂ n Gn (x1 , . . . , xn ) = (−1)n f (x1 , . . . , xn ). ∂x1 · · · ∂xn
We give
(2.15)
Theorem 2.11. According to the above assumptions and notations we have ! Z Z bn n b1 Y Gn (x1 , . . . , xn )dx1 · · · dxn . (2.16) ··· E Xi = i=1
0
0
Proof. Integrate by parts Gn with respect to xn , taking into account that Gn (x1 , x2 , . . . , xn−1 , bn ) = 0. Then again integrate by parts with respect to variable xn−1 the outcome of the first integration. Then repeat, successively, the same procedure by integrating by parts with respect to variables xn−2 , xn−3 , . . . , x1 . Thus we have recovered formula (2.13). That is deriving (2.16). Let (x1 , . . . , xn ) ∈ ×ni=1 [0, bi ] be fixed. We define the kernels pi : [0, bi ]2 → R: si , si ∈ [0, xi ], pi (xi , si ) := (2.17) si − bi , si ∈ (xi , bi ], for all i = 1, . . . , n.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Basic Stochastic Ostrowski Inequalities
9
Theorem 2.12. Based on the above assumptions and notations we derive that Z n Y θ˜1,n := (−1)n · pi (xi , si )f (s1 , s2 , . . . , sn )ds1 ds2 · · · dsn =
(
×n i=1 [0,bi ] i=1
n Y
i=1
bi
!
· Gn (x1 , . . . , xn )
n (X ) Z n 1 Y − bj i=1
j=1 j6=i
bi 0
k=1 k6=i,j
Gn (x1 , . . . , si , . . . , xn )dsi
n (X Z n 2) Y + bk `=1
)
bi
0
Z
bj
Gn (x1 , . . . , si , . . . , sj , . . . , xn )dsi dsj 0
− + · · · − + · · · + (−1)n−1 n (X Z n−1) cj · · · dsn bj ×n [0,b ] Gn (s1 , . . . , xj , . . . , sn )ds1 · · · ds j=1
+ (−1)n · E
!
(`)
i
i=1
i6=j
n Y
Xi
i=1
!
=: θ˜2,n .
(2.18)
cj means that The above ` counts all the (i, j)’s: i < j and i, j = 1, . . . , n. Also ds dsj is missing.
Proof. Application of Theorem 2 ([30]) for f = Gn , also taking into account (2.15) and (2.16). As a related result we give Theorem 2.13. Under the notations and assumptions of this Section 2.3, additionally we assume that kf (s1 , . . . , sn )k∞ < +∞. Then (n ) Y kf (s1 , . . . , sn )k∞ 2 2 ˜ (2.19) · [xi + (bi − xi ) ] , |θ2,n | ≤ 2n i=1 for any (x1 , . . . , xn ) ∈ ×ni=1 [0, bi ], n ∈ N.
Proof. Based on Theorem 2.12, and Theorem 4 ([30]).
The Lp analogue follows Theorem 2.14. Under the notations and assumptions of this Section 2.3, additionally we suppose that f ∈ Lp (×ni=1 [0, bi ]), i.e., kf kp < +∞, where p, q > 1: 1 1 p + q = 1. Then (n )1/q Y q+1 kf k p q+1 |θ˜2,n | ≤ · [xi + (bi − xi ) ] , (2.20) (q + 1)n/q i=1
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
10
for any (x1 , . . . , xn ) ∈ ×ni=1 [0, bi ]. Proof. Based on Theorem 2.12, and Theorem 6 ([30]). 2.4
Applications
i) On Theorem 2.6: Let X be a random variable uniformly distributed over [a, b], 0 < a < b. Here the probability density function is f (t) =
1 . b−a
(2.21)
And the probability distribution function is F (x) = P (X ≤ x) =
x−a , b−a
a ≤ x ≤ b.
(2.22)
Furthermore the moment generating function is MX (x) = Here kf k∞ = is
1 b−a ,
ebx − eax , (b − a)x
a ≤ x ≤ b.
(2.23)
1
and kf kp = (b − a) p −1 , p > 1. Finally the left-hand side of (1.9) bx ax −e ebx − ex(b−a) x−a . − b−a bx ax e −e
(2.24)
Then one finds interesting comparisons via inequality (2.9). ii) On Theorem 2.7 (see also [105], [107], [117], [174]): Let X be a Beta random variable taking values in [0, 1] with parameters s > 0, t > 0. It has probability density function f (x; s, t) :=
xs−1 (1 − x)t−1 ; B(s, t)
where B(s, t) :=
Z
1 0
s s+t ,
see [105]. As noted in [174] for
1 · [B(p(s − 1) + 1, p(t − 1) + 1)]1/p . B(s, t)
Also from [107] for s, t ≥ 1 we have kf (·, s, t)k∞ =
(2.25)
τ s−1 (1 − τ )t−1 dτ,
the Beta function. We know that E(X) = p > 1, s > 1 − p1 , and t > 1 − p1 we find that kf (·; s, t)kp =
0 ≤ x ≤ 1,
(s − 1)s−1 (t − 1)t−1 . B(s, t)(s + t − 2)s+t−2
(2.26)
(2.27)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Basic Stochastic Ostrowski Inequalities
Book
11
One can find easily for m ∈ N that
m Q
j=1
E(X m ) = Q m
j=1
(s + m − j)
.
(2.28)
(s + t + m − j)
Here F (x) = P (X ≤ x), P stands for probability. So we present Theorem 2.15. Let X be a Beta random variable taking values in [0,1] with parameters s,t. Let x ∈ [0, 1], m ∈ N. Then we obtain Q m (s + m − j) j=1 P (X ≥ x) − m Q (s + t + m − j) j=1
≤
(s − 1)s−1 (t − 1)t−1 (2xm+1 − 1) · + 1 − x , when s, t ≥ 1, B(s, t)(s + t − 2)s+t−2 m+1 max(xm , 1 − xm ), when s, t > 0, mq+1 1/q Z 1 1 x 1/p m q , · + (1 − t ) dt B(s, t) · [B(p(s − 1) + 1, p(t − 1) + 1)] mq + 1 x 1 1 1 when p, q > 1 : + = 1, and s, t > 1 − , p q p 1 · [B(2s − 1, 2t − 1)]1/2 B(s, t) m+1 1/2 2x 2m2 1 · −x+ , when s, t ≥ . m+1 (m + 1)(2m + 1) 2
The case of x =
1 2
(2.29)
is especially interesting. We find
Theorem 2.16. Let X be a Beta random variable taking values in [0, 1] with
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
12
parameters s, t. Let m ∈ N. Then Q m (s + m − j) j=1 1 P X ≥ − Q m 2 (s + t + m − j) j=1 ) 1 s−1 t−1 (s − 1) (t − 1) 1 m − 1 2 · + , s+t−2 m+1 2 B(s, t)(s + t − 2) when s, t ≥ 1, 1 1 − m , when s, t > 0, (2.30) 2 q1 Z 1 1 1 1 m q p ≤ B(s, t) · [B(p(s − 1) + 1, p(t − 1) + 1)] · (mq + 1)2mq+1 + 1 (1 − t ) dt , 2 1 1 1 when p, q > 1 : + = 1, and s, t > 1 − , p q p 12 1 1 1 2m2 1 2 , · [B(2s − 1, 2t − 1)] · − + B(s, t) (m + 1) · 2m 2 (m + 1)(2m + 1) 1 when s, t > . 2 2.5
Addendum
We present here a detailed proof of Theorem 2.11, when n = 3. Q3 Theorem 2.17. Let the random vector (X, Y, Z) taking values on i=1 [0, bi ], where bi > 0, i = 1, 2, 3. Let F be the cumulative distribution function of (X,Y,Z), asQ ∂3F sumed to be continuous on 3i=1 [0, bi ]. Assume that there exists f = ∂x∂y∂z ≥0 Q3 a.e, and is integrable on i=1 [0, bi ]. Then f is a joint probability density function of (X, Y, Z). Furthermore we have the representation E (XY Z) =
Z
b1 0
Z
b2 0
Z
b3
G3 (x, y, z)dxdydz,
(2.31)
0
where G3 (x, y, z) := 1 − F (x, b2 , b3 ) − F (b1 , y, b3 ) + F (x, y, b3 ) −F (b1 , b2 , z) + F (x, b2 , z) + F (b1 , y, z) − F (x, y, z).
(2.32)
Proof. For the definition of the multidimensional cumulative distribution see [235], p. 97.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Basic Stochastic Ostrowski Inequalities
We observe that Z b3 Z b2 Z 0
= =
Z
b3 0
Z
b3 0
Z
0
Z
b2 0
Z
b2 0
Z
!
! ! ∂ 3 F (x, y, z) dx dy dz ∂x∂y dz
∂2 (F (b1 , y, z) − F (o, y, z)) dy ∂y∂z
!
dz
(2.33)
b3
∂ [F (b1 , b2 , z) − F (0, b2 , z) − F (b1 , 0, z) + F (0, 0, z)] dz ∂z 0 = F (b1 , b2 , b3 ) − F (0, b2 , b3 ) − F (b1 , 0, b3 ) + F (0, 0, b3 ) − F (b1 , b2 , 0)
=
13
f (x, y, z)dx dy dz 0
b1 0
!
b1
Book
+ F (0, b2 , 0) + F (b1 , 0, 0) − F (0, 0, 0) = F (b1 , b2 , b3 ) = 1,
(2.34)
proving that f is a probability density function. We know that Z b1 Z b2 Z b3 E (XY Z) = xyzf (x, y, z)dxdydz. 0
0
0
We notice that G3 (x, y, b3 ) = 0. So me have Z b3 Z b3 b3 G3 (x, y, z)dz = zG3 (x, y, z) 0 − zdz G3 (x, y, z) 0 0 =−
where α1 (x, y, z) := Next take Z
where
Z
b3
z dz G3 (x, y, z) =
0
Z
b3
xα1 (x, y, z)dz,
(2.35)
0
∂F (b1 , b2 , z) ∂F (x, b2 , z) ∂F (b1 , y, z) ∂F (x, y, z) − − + . ∂z ∂z ∂z ∂z
(2.36)
b2
α1 (x, y, z)dy Z b2 ∂F (b1 , y, z) ∂F (x, y, z) ∂F (b1 , b2 , z) ∂F (x, b2 , z) − − + dy = ∂z ∂z ∂z ∂z 0 ∂F (b1 , b2 , z) ∂F (x, b2 , z) ∂F (b1 , y, z) ∂F (x, y, z) b2 =y − − + 0 ∂z ∂z ∂z ∂z Z b2 Z b2 2 −∂ F (b1 , y, z) ∂ 2 F (x, y, z) yα2 (x, y, z)dy, (2.37) y + dy = − ∂y∂z ∂y∂z 0 0 0
α2 (x, y, z) :=
∂ 2 F (b1 , y, z) ∂ 2 F (x, y, z) − . ∂y∂z ∂y∂z
(2.38)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
14
Finally we see that Z b1 α2 (x, y, z)dx 0
∂ 2 F (b1 , y, z) ∂ 2 F (x, y, z) − dx ∂y∂z ∂y∂z 0 2 Z b1 3 ∂ F (x, y, z) ∂ F (b1 , y, z) ∂ 2 F (x, y, z) x − =x − · b01 − dx ∂y∂z ∂y∂z ∂x∂y ∂z 0 Z b1 Z b1 ∂ 3 F (x, y, z) x f (x, y, z)dx. (2.39) x = dx = ∂x∂y∂z 0 0 Putting things together we get ! ! Z b1 Z b2 Z b3 G3 (x, y, z)dz dy dx =
Z
b1
= =
=
= proving the claim.
Z
0
0
Z
Z
Z
Z
=
=
0
Z
b3
z 0
z 0 b3
z 0 b3
z 0
b1 0
0
b3
Z
Z
b1
Z
b2 0
b2 0
Z
Z
Z
0
0 b2
y 0 b2
y 0
zα1 (x, y, z)dz 0
Z
Z
b1
Z
Z
b1
b3
b2
!
!
dy dx
!
!
α1 (x, y, z)dy dx dz 0 b2
y α2 (x, y, z)dy 0
Z
Z
b1
!
!
dx dz
α2 (x, y, z)dx dy 0 b1
!
!
!
dz
!
x f (x, y, z)dx dy dz 0
b3
xyz f (x, y, z)dxdydz = E(XY Z),
(2.40)
0
Remark 2.18. Notice that ∂G3 (x, y, z) ∂F (b1 , b2 , z) ∂F (x, b2 , z) =− + ∂z ∂z ∂z ∂F (b1 , y, z) ∂F (x, y, z) + − , ∂z ∂z ∂ 2 G3 (x, y, z) ∂ 2 F (b1 , y, z) ∂ 2 F (x, y, z) = − , ∂y∂z ∂y∂z ∂y∂z and ∂ 3 G3 (x, y, z) ∂ 3 F (x, y, z) =− = −f (x, y, z). ∂x∂y∂z ∂x∂y∂z That is giving us ∂ 3 G3 (x, y, z) = −f (x, y, z). ∂x∂y∂z
(2.41)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Chapter 3
Multidimensional Montgomery Identities and Ostrowski type Inequalities
Multidimensional Montgomery type identities are presented involving, among others, integrals of mixed partial derivatives. These lead to interesting multidimensional Ostrowski type inequalities. The inequalities in their right-hand sides involve Lp -norms (1 ≤ p ≤ ∞) of the engaged mixed partials. An application is given also to Probability. This treatment is based on [33]. 3.1
Introduction
In 1938, A. Ostrowski [277] proved the following inequality: Theorem 3.1. Let f : [a, b] → R be continuous on [a, b] and differentiable on (a, b) whose derivative f 0 : (a, b) → R is bounded on (a, b), i.e., kf 0 k∞ = sup |f 0 (t)| < t∈(a,b)
+∞. Then " # 1 Z b 2 ) 1 (x − a+b 2 f (t)dt − f (x) ≤ + · (b − a)kf 0 k∞ , b − a a 4 (b − a)2
for any x ∈ [a, b]. The constant
1 4
(3.1)
is the best possible.
Since then there has been a lot of activity about these inequalities with interesting applications to Probability and Numerical Analysis. This chapter has been greatly motivated by the following result of S. Dragomir et al. (2000), see [176] and [99], p. 263, Theorem 5.18. It gives a multidimensional Montgomery type identity. Theorem 3.2. Let f : [a, b] × [c, d] → R be such that the partial derivatives 2
∂f (t,s) ∂ f (t,s) ∂s , ∂t∂s
∂f (t,s) ∂t ,
exist and are continuous on [a, b] × [c, d]. Then for all (x, y) ∈ [a, b] × [c, d], we have the representation Z b Z d Z bZ d 1 ∂f (t, s) f (x, y) = dsdt f (t, s)dsdt + p(x, t) (b − a)(d − c) a c ∂t a c Z bZ d Z bZ d ∂f (t, s) ∂ 2 f (t, s) + q(y, s) dsdt + p(x, t)q(y, s) dsdt , (3.2) ∂s ∂t∂s a c a c 15
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
16
where p : [a, b]2 → R, q : [c, d]2 → R and are given by t − a, if t ∈ [a, x] p(x, t) := t − b, if t ∈ (x, b] and q(y, s) :=
s − c, s − d,
if s ∈ [c, y] if s ∈ (y, d].
In this chapter we give more general multidimensional Montgomery type identities and derive related new multidimensional Ostrowski type inequalities involving k · kp (1 ≤ p ≤ ∞) norms of the engaged mixed partial derivatives. At the end we give an application to Probability. That is finding bounds for the difference of the expectation of a product of three random variables from an appropriate linear combination of values of the related joint probability distribution function. 3.2
Results
We present a multidimensional Montgomery type identity which is a generalization of Theorem 5.18, p. 263 in [99], see also [176]. We give the representation. Theorem 3.3. Let f : ×3i=1 [ai , bi ] → R such that the partials
∂f (s1 ,s2 ,s3 ) , ∂sj
j =
3
2
(s1 ,s2 ,s3 ) 1 ,s2 ,s3 ) , j < k, j, k ∈ {1, 2, 3}; ∂ ∂sf (s exist and are continuous on 1, 2, 3; ∂ f∂s 3 ∂s2 ∂s1 k ∂sj ×3i=1 [ai , bi ]. Let (x1 , x2 , x3 ) ∈ ×3i=1 [ai , bi ]. Define the kernels pi : [ai , bi ]2 → R: si − ai , si ∈ [ai , xi ] pi (xi , si ) = si − bi , si ∈ (xi , bi ],
for i = 1, 2, 3. Then f (x1 , x2 , x3 ) Z b1 Z b2 Z b3 1 f (s1 , s2 , s3 )ds3 ds2 ds1 = 3 Q (bi − ai ) a1 a2 a3 +
i=1 3 X j=1
+
Z
`=1 j
+
b1 a1
a1
Z
3 X
Z
b1
Z
Z
b1 a1
b2 a2
Z
b2
a2
Z
b2 a2
b3 a3
Z
b3
∂f (s1 , s2 , s3 ) pj (xj , sj ) ds3 ds2 ds1 ∂sj
a3
Z
b3 a3
3 Y
i=1
pj (xj , sj )pk (xk , sk ) ·
pi (xi , si )
!
!
∂ 2 f (s1 , s2 , s3 ) ds3 ds2 ds1 (`) ∂sk ∂sj
∂ 3 f (s1 , s2 , s3 ) ds3 ds2 ds1 . ∂s3 ∂s2 ∂s1
Above ` counts (j, k): j < k; j, k ∈ {1, 2, 3}.
(3.3)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Multidimensional Montgomery Identities and Ostrowski type Inequalities
Book
17
Proof. Here we use repeatedly the following identity due to Montgomery (see [265], p. 565), which can be easily proved by using integration by parts Z β Z β 1 1 g(z)dz + k(u, z)g 0 (z)dz, g(u) = β−α α β−α α
where k : [α, β]2 → R is defined by k(u, z) :=
z − α, z − β,
if z ∈ [α, u] if z ∈ (u, β],
and g is absolutely continuous on [α, β]. First we see that f (x1 , x2 , x3 ) = A0 + B0 , where 1 A0 := b1 − a 1 and 1 B0 := b1 − a 1
Z
Z
b1
f (s1 , x2 , x3 )ds1 , a1
b1
p1 (x1 , s1 ) a1
∂f (s1 , x2 , x3 ) ds1 . ∂s1
Furthermore we have f (s1 , x2 , x3 ) = A1 + B1 , where A1 :=
1 b2 − a 2
and B1 :=
1 b2 − a 2
Z
b2
f (s1 , s2 , x3 )ds2 , a2
b2
p2 (x2 , s2 ) a2
Also we find that 1 f (s1 , s2 , x3 ) = b3 − a 3 +
Z
1 b3 − a 3
Z
∂f (s1 , s2 , x3 ) ds2 . ∂s2
b3
f (s1 , s2 , s3 )ds3
a3 Z b3 a3
p3 (x3 , s3 )
∂f (s1 , s2 , s3 ) ds3 . ∂s3
Next we put things together, and we derive Z b2 Z b3 1 f (s1 , s2 , s3 )ds3 ds2 A1 = (b2 − a2 )(b3 − a3 ) a2 a3 Z b2 Z b3 1 ∂f (s1 , s2 , s3 ) + p3 (x3 , s3 ) ds3 ds2 . (b2 − a2 )(b3 − a3 ) a2 a3 ∂s3
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
PROBABILISTIC INEQUALITIES
18
And 1
A0 =
3 Q
i=1
+
3 Q
i=1
(bi − ai ) 1 (bi − ai )
Z Z
b1 a1
b1 a1
Z Z
a2
Z
b2 a2
1 (b1 − a1 )(b2 − a2 )
+
Z
b2
Z
b3
f (s1 , s2 , s3 )ds3 ds2 ds1 a3
b3
p3 (x3 , s3 ) a3
Z
b1 a1
∂f (s1 , s2 , s3 ) ds3 ds2 ds1 ∂s3
b2
p2 (x2 , s2 ) a2
∂f (s1 , s2 , x3 ) ds2 ds1 . ∂s2
Also we obtain 1 ∂f (s1 , s2 , x3 ) = ∂s2 b3 − a 3 +
1 b3 − a 3
Z Z
b3 a3
∂f (s1 , s2 , s3 ) ds3 ∂s2
b3
p3 (x3 , s3 ) a3
∂ 2 f (s1 , s2 , s3 ) ds3 . ∂s3 ∂s2
Therefore we get
1
A0 =
3 Q
i=1
+
(bi − ai )
3 Q
(bi − ai ) 1 (bi − ai )
3 Q
(bi − ai )
i=1
·
a1
3 Q
i=1
+
b1
1
i=1
+
Z
1
Z Z Z
Z
b2 a2
b1 a1
b1 a1
b1 a1
Z Z Z
Z
b3
f (s1 , s2 , s3 )ds3 ds2 ds1 a3
b2 a2
b2 a2
b2 a2
Z Z Z
b3
p3 (x3 , s3 )
∂f (s1 , s2 , s3 ) ds3 ds2 ds1 ∂s3
p2 (x2 , s2 )
∂f (s1 , s2 , s3 ) ds3 ds2 ds1 ∂s2
a3
b3 a3
b3
p2 (x2 , s2 )p3 (x3 , s3 ) a3
∂ 2 f (s1 , s2 , s3 ) ds3 ds2 ds1 . ∂s3 ∂s2
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Multidimensional Montgomery Identities and Ostrowski type Inequalities
19
Similarly we obtain that
1
B0 =
3 Q
i=1
+
1
3 Q
i=1
+
i=1
3 Q
(bi − ai ) 1
i=1
·
(bi − ai ) 1
3 Q
+
(bi − ai )
(bi − ai )
Z Z Z Z
b1 a1
b1 a1
b1 a1
b1 a1
Z Z Z Z
b2 a2
b2 a2
b2 a2
b2 a2
Z Z Z Z
b3
p1 (x1 , s1 ) a3
∂f (s1 , s2 , s3 ) ds3 ds2 ds1 ∂s1
b3
p1 (x1 , s1 )p3 (x3 , s3 )
∂ 2 f (s1 , s2 , s3 ) ds3 ds2 ds1 ∂s3 ∂s1
p1 (x1 , s1 )p2 (x2 , s2 )
∂ 2 f (s1 , s2 , s3 ) ds3 ds2 ds1 ∂s2 ∂s1
a3
b3 a3
b3
p1 (x1 , s1 )p2 (x2 , s2 )p3 (x3 , s3 ) a3
∂ 3 f (s1 , s2 , s3 ) ds3 ds2 ds1 . ∂s3 ∂s2 ∂s1
We have proved (3.3).
A generalization of Theorem 3.3 to any n ∈ N follows: Theorem 3.4. Let f : ×ni=1 [ai , bi ] → R such that the partials ∂2f ∂f (s1 , s2 , . . . , sn ), j = 1, . . . , n; , j < k, ∂sj ∂sk ∂sj j, k ∈ {1, . . . , n};
∂3f , j < k < r, ∂sr ∂sk ∂sj
j, k, r ∈ {1, . . . , n}; . . . ;
∂ n−1 f ∂ nf , ` ∈ {1, . . . , n}; ∂sn · · · ∂ˆ s` · · · ∂s1 ∂sn ∂sn−1 · · · ∂s1
all exist and are continuous on ×ni=1 [ai , bi ], n ∈ N. Above ∂ˆ s` means ∂s` is missing. Let (x1 , . . . , xn ) ∈ ×ni=1 [ai , bi ]. Define the kernels pi : [ai , bi ]2 → R:
pi (xi , si ) :=
(
si − a i , si − b i ,
si ∈ [ai , xi ], si ∈ (xi , bi ],
for i = 1, 2, . . . , n.
(3.4)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
20
Then f (x1 , x2 , . . . , xn ) =
·
(Z
+
i=1
×n i=1 [ai ,bi ]
Z n X j=1
+
(n2 ) X
(n3 ) X
`2 =1 j
·
(bi − ai )
f (s1 , s2 , . . . , sn )dsn dsn−1 · · · ds1
!
∂ 2 f (s1 , s2 , . . . , sn ) dsn · · · ds1 pj (xj , sj )pk (xk , sk ) ∂sk ∂sj ×n i=1 [ai ,bi ] Z
×n i=1 [ai ,bi ]
n (X Z n−1)
`=1
!
(`1 )
pj (xj , sj )pk (xk , sk )pr (xr , sr )
∂ 3 f (s1 , . . . , sn ) dsn · · · ds1 ∂sr ∂sk ∂sj
+···+
∂f (s1 , s2 , . . . , sn ) pj (xj , sj ) dsn · · · ds1 ∂sj ×n i=1 [ai ,bi ]
Z
`1 =1 j
+
1 n Q
×n i=1 [ai ,bi ]
(`2 )
p1 (x1 , s1 ) · · · p`\ (x` , s` ) · · · pn (xn , sn )
∂ n−1 f (s1 , . . . , sn ) b · dsn · · · ds` · · · ds1 ∂sn · · · ∂ˆ s` · · · ∂s1 ! ) Z n Y ∂ n f (s1 , . . . , sn ) + pi (xi , si ) · dsn · · · ds1 . ∂sn · · · ∂s1 ×n i=1 [ai ,bi ] i=1
(3.5)
Above `1 counts (j, k) : j < k; j, k ∈ {1, 2, . . . , n}, also `2 counts (j, k, r) : j < c` means that p` (x` , s` ) and k < r; j, k, r ∈ {1, 2, . . . , n}, etc. Also p`\ (x` , s` ) and ds ds` are missing, respectively. Proof. Similar to Theorem 3.3.
Next we present some Ostrowski type [277] inequalities. In particular the result that follows generalizes Theorem 5.21, p. 266 from [99], also the corresponding result from [176]. Theorem 3.5. Let f : ×3i=1 [ai , bi ] → R as in Theorem 3.3. Then Z b1 Z b2 Z b3 1 f (x1 , x2 , x3 ) − f (s1 , s2 , s3 )ds3 ds2 ds1 3 Q (bi − ai ) a1 a2 a3 i=1 ! 3 Y ≤ (M1,3 + M2,3 + M3,3 )/ (bi − ai ) . i=1
(3.6)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Multidimensional Montgomery Identities and Ostrowski type Inequalities
Here we have
M1,3 :=
3 P ∂f
∂sj ∞ j=1 3 2 2 Q (xj −aj ) +(bj −xj ) , · (bi − ai ) · 2 i=1 i6=j ∂f if ∈ L∞ ×3i=1 [ai , bi ] , j = 1, 2, 3; ∂sj P
3 Q ∂f 3
· (bi − ai )1/qj
∂s j=1 j pj
i=1
i6=j h i1/qj (xj −aj )qj +1 +(bj −xj )qj +1 , · qj+1 ∂f if ∂s ∈ Lpj ×3i=1 [ai , bi ] , pj > 1, j 1 1 pj + qj = 1, for j = 1, 2, 3;
3 bj −aj aj +bj ∂f P · + x − ,
j ∂sj 1 2 2 j=1 ∂f if ∂s ∈ L1 ×3i=1 [ai , bi ] , for j = 1, 2, 3. j
Also we have
2 3 3 X Y
∂ f (bi − ai )
· ∂s ∂s k j ∞ i=1 `=1 i6=j,k j
3 3 2 X ∂ f Y
· (bi − ai )1/qkj
∂s ∂s k j pkj i=1 `=1 j
1, + = 1, if ∂s ∂s p q k j kj kj if j, k ∈ {1, 2, 3}; 3
X
∂2f bj − aj aj + bj
+ xj −
∂sk ∂sj · 2 2 1 `=1 j
Book
21
(3.7)
(3.8)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
22
And finally,
M3,3
3 Q ((xj − aj )2 + (bj − xj )2 )
3
∂ f (s , s , s ) j=1 1 2 3 ,
∂s3 ∂s2 ∂s1 · 8 ∞ ∂ 3 f (s1 , s2 , s3 ) if ∈ L∞ ×3i=1 [ai , bi ] ; ∂s3 ∂s2 ∂s1
3
3 Y
∂ f (s1 , s2 , s3 )
∂s3 ∂s2 ∂s1 p123 j=1 1/q123 q123 +1 (xj − aj ) + (bj − xj )q123 +1 · , := q123 + 1 3 ∂ f (s1 , s2 , s3 ) ∈ Lp123 ×3i=1 [ai , bi ] , p123 > 1, if ∂s ∂s ∂s 3 2 1 1 1 + = 1; p123 q123
3 Y ∂ 3 f (s1 , s2 , s3 ) bj − aj aj + bj
, + xj −
∂s3 ∂s2 ∂s1 · 2 2 1 j=1 ∂ 3 f (s1 , s2 , s3 ) if ∈ L1 ×3i=1 [ai , bi ] . ∂s3 ∂s2 ∂s1
(3.9)
Inequality (3.6) is valid for any (x1 , x2 , x3 ) ∈ ×3i=1 [ai , bi ], where k · kp (1 ≤ p ≤ ∞) are the usual Lp -norms on ×3i=1 [ai , bi ]. Proof. We have by (3.3) that Z b1 Z b2 Z b3 1 f (x1 , x2 , x3 ) − f (s1 , s2 , s3 )ds3 ds2 ds1 3 Q (bi − ai ) a1 a2 a3 i=1 Z b1 Z b2 Z b3 3 X 1 |pj (xj , sj )| ≤ 3 Q a3 a2 a1 (bi − ai ) j=1 i=1 ∂f (s1 , s2 , s3 ) ds3 ds2 ds1 · ∂sj Z b1 Z b2 Z b3 3 X + |pj (xj , sj )| · |pk (xk , sk )| `=1 j
a1
a2
a3
2 ∂ f (s1 , s2 , s3 ) · ds3 ds2 ds1 ∂sk ∂sj +
Z
b1 Z
a1
b2 Z
a2
b3
a3
3 Y
i=1
(`)
! ! ∂ 3 f (s1 , s2 , s3 ) ds3 ds2 ds1 . |pi (xi , si )| · ∂s3 ∂s2 ∂s1
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Multidimensional Montgomery Identities and Ostrowski type Inequalities
23
We notice the following (j = 1, 2, 3) Z
∂f (s1 , s2 , s3 ) ds3 ds2 ds1 |pj (xj , sj )| · ∂sj a3 a2 a1
Z b1 Z b2 Z b3
∂f
|pj (xj , sj )|ds3 ds2 ds1 ,
∂sj · a3 a2 a1 ∞ ∂f if ∈ L∞ ×3i=1 [ai , bi ] ; ∂sj !1/qj
Z b1 Z b2 Z b3
∂f q j
≤ , |pj (xj , sj )| ds3 ds2 ds1 ∂sj pj a1 a2 a3 1 1 ∂f ∈ Lpj ×3i=1 [ai , bi ] , pj > 1, + = 1; if ∂s p q j j
j
∂f 3
∂f ∂s · sup |pj (xj , sj )|, if ∂s ∈ L1 ×i=1 [ai , bi ] . j 1 sj ∈[aj ,bj ] j b1
Z
b2
Z
b3
Furthermore we observe that Z
b1 a1
Z
b2 a2
Z
b3 a3
|pj (xj , sj )|ds3 ds2 ds1
= (b1 − a1 )(b\ j − aj )(b3 − a3 )
(xj − aj )2 + (bj − xj )2 2
;
b\ j − aj means (bj − aj ) is missing, for j = 1, 2, 3. Also we find Z
b1 a1
Z
b2 a2
Z
b3 a3
|pj (xj , sj )|qj ds3 ds2 ds1
!1/qj
1/qj (b3 − a3 )1/qj = (b1 − a1 )1/qj (b\ j − aj ) 1/qj (xj − aj )qj +1 + (bj − xj )qj +1 · ; qj + 1 1/qj means (bj − aj )1/qj is missing, for j = 1, 2, 3. Also (b\ j − aj )
bj − aj aj + bj sup |pj (xj , sj )| = max{xj − aj , bj − xj } = + xj − , 2 2 sj ∈[aj ,bj ]
for j = 1, 2, 3. Putting things together we get Z 3 X j=1
b1 a1
Z
b2 a2
Z
b3 a3
! ∂f (s1 , s2 , s3 ) |pj (xj , sj )| · ds3 ds2 ds1 ∂sj
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
24
3 3 X Y
∂f (xj − aj )2 + (bj − xj )2
(bi − ai ) · ,
∂sj · 2 ∞ j=1 i=1 i6=j ∂f if ∈ L∞ ×3i=1 [ai , bi ] , j = 1, 2, 3; ∂sj
3 3 Y X
∂f (bi − ai )1/qj
∂sj j=1 pj i=1 i6=j ≤ 1/qj qj +1 (x − a ) + (bj − xj )qj +1 j j · , qj + 1 1 ∂f 1 ∈ Lpj ×3i=1 [ai , bi ] , pj > 1, + = 1, for j = 1, 2, 3; if ∂s p q j j j
3 X
∂f bj − aj aj + bj
+ xj − ,
∂sj · 2 2 1 j=1 ∂f ∈ L1 ×3i=1 [ai , bi ] , for j = 1, 2, 3. if ∂sj
(3.10)
Similarly working, next we find that 3 X `=1 j
Z
b1 a1
Z
b2 a2
Z
b3 a3
! 2 ∂ f (s1 , s2 , s3 ) ds3 ds2 ds1 |pj (xj , sj )| · |pk (xk , sk )| · ∂sk ∂sj
3 3 2 X ∂ f Y
(bi − ai )
∂sk ∂sj · ∞ i=1 `=1 i6=j,k j
2 3 3 X Y ∂ f
(bi − ai )1/qkj
∂sk ∂sj · ≤ `=1 pkj i=1 j 1, p i i i=1 kj ∂sk ∂sj 1 1 + = 1, for j, k ∈ {1, 2, 3}; pkj qkj
(`)
(3.11a)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Multidimensional Montgomery Identities and Ostrowski type Inequalities 3 X `=1 j
Z
b1 a1
Z
b2 a2
Z
b3 a3
25
! 2 ∂ f (s1 , s2 , s3 ) ds3 ds2 ds1 |pj (xj , sj )| · |pk (xk , sk )| · ∂sk ∂sj
3 X
∂2f bj − aj aj + bj
+ xj −
∂sk ∂sj · 2 2 1 `=1 j
(`)
(3.11b)
Finally we obtain that ! Z b1 Z b2 Z b3 Y 3 ∂ 3 f (s1 , s2 , s3 ) ds3 ds2 ds1 |pi (xi , si )| · ∂s ∂s ∂s 3 2 1 a1 a2 a3 i=1 3 Q ((xj − aj )2 + (bj − xj )2 ) 3
∂ f (s1 , s2 , s3 ) j=1
,
∂s3 ∂s2 ∂s1 · 8 ∞ ∂ 3 f (s1 , s2 , s3 ) ∈ L∞ ×3i=1 [ai , bi ] ; if ∂s3 ∂s2 ∂s1
3
1/q123 3 Y
∂ f (s (xj − aj )q123 +1 + (bj − xj )q123 +1 1 , s2 , s3 )
·
∂s3 ∂s2 ∂s1 q123 + 1 (3.12) ≤ p123 j=1 3 1 1 ∂ f (s1 , s2 , s3 ) ∈ Lp123 ×3i=1 [ai , bi ] , p123 > 1, + = 1; if ∂s ∂s ∂s p q 3 2 1 123 123
3
∂ 3 f (s1 , s2 , s3 ) Y bj − aj aj + bj
· + xj − ,
∂s3 ∂s2 ∂s1 1 j=1 2 2 ∂ 3 f (s1 , s2 , s3 ) if ∈ L1 ×3i=1 [ai , bi ] . ∂s3 ∂s2 ∂s1 Taking into account (3.10), (3.11a), (3.11b), (3.12) we have completed the proof of (3.6). A generalization of Theorem 3.5 follows: Theorem 3.6. Let f : ×ni=1 [ai , bi ] → R as in Theorem 3.4, n ∈ N, and (x1 , . . . , xn ) ∈ ×ni=1 [ai , bi ]. Here k · kp (1 ≤ p ≤ ∞) is the usual Lp -norm on ×ni=1 [ai , bi ]. Then Z 1 f (x1 , x2 , . . . , xn ) − f (s , s , . . . , s )ds · · · ds 1 2 n n 1 n Q n [a ,b ] × i i i=1 (bi − ai ) i=1 ! ! Y n n X ≤ Mi,n (bi − ai ) . (3.13) i=1
i=1
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
26
Here we have
M1,n
n n Y X ∂f
(bi − ai )
∂sj · ∞ i=1 j=1 i6=j 2 (xj − aj ) + (bj − xj )2 , · 2 ∂f ∈ L∞ (×ni=1 [ai , bi ]) , j = 1, 2, . . . , n; if ∂s j
n n X Y
∂f · (bi − ai )1/q
∂s j p j=1 i=1 i6=j := 1/q ! q+1 (x − a ) + (bj − xj )q+1 j j , · q+1 ∂f ∈ Lp (×ni=1 [ai , bi ]) , p > 1, if ∂s j 1 1 + = 1, for j = 1, 2, . . . , n; p q n X ∂f bj − aj aj + bj
· + x − , j
∂sj 2 2 1 j=1 ∂f ∈ L1 (×ni=1 [ai , bi ]) , for j = 1, 2, . . . , n. if ∂sj
(3.14)
Also we have n (2) 2 n Y X ∂ f
· (bi − ai )
∂sk ∂sj ∞ i=1 `1 =1 i6=j,k j
(bi − ai )1/q
∂sk ∂sj · M2,n := (3.15a) p i=1 `1 =1 i6 = j,k j 1, if ∂sk ∂sj 1 1 + = 1, for j, k ∈ {1, 2, . . . , n}; p q
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Multidimensional Montgomery Identities and Ostrowski type Inequalities
M2,n
And,
M3,n
n ( 2 ) 2 X
∂ f aj + bj bj − aj
+ xj −
∂sk ∂sj · 2 2 1 `1 =1 j
n
( n 3) 3 X
∂ f Y
(bi − ai )
∂sr ∂sk ∂sj · ∞ i=1 `2 =1 i6=j,k,r j
( 3) n 3
Y ∂ f X
· (bi − ai )1/q
∂sr ∂sk ∂sj p i=1 `2 =1 i6=j,k,r j 1, ∂s ∂s ∂s r k j 1 1 + = 1, for j, k, r ∈ {1, 2, . . . , n}; p q n
(3) Y X
∂3f
∂sr ∂sk ∂sj · 1 m∈{j,k,r} `2 =1 j
Book
27
(3.15b)
(3.16)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
28
And,
Mn−1,n :=
n (n−1) X `=1
∂ n−1 f
(b − a` ) c ` · · · ∂s1 ∞ ` ∂sn · · · ∂s n Q ((xm − am )2 + (bm − xm )2 ) m=1,m6=` , · 2n−1
∂ n−1 f if ∈ L∞ (×ni=1 [ai , bi ]) , ` = 1, . . . , n; c` · · · ∂s1 ∂sn · · · ∂s n
(X n−1)
∂ n−1 f
· (b` − a` )1/q
∂s · · · ∂ˆ s · · · ∂s n ` 1 p `=1 n q+1 q+1 1/q Y (xm − am ) + (bm − xm ) · , q + 1 m=1 m6=` ∂ n−1 f ∈ Lp (×ni=1 [ai , bi ]) , p > 1, if ∂s · · · ∂ˆ s · · · ∂s n ` 1 1 1 + = 1, ` = 1, . . . , n; Mn−1,n := p q n
(n−1)
∂ n−1 f X
∂sn · · · ∂ˆ s` · · · ∂s1 1 `=1 n Y bm − a m am + b m · + xm − , 2 2 m=1 m6=` ∂ n−1 f if ∈ L1 (×ni=1 [ai , bi ]) , ` = 1, . . . , n. ∂sn · · · ∂ˆ s` · · · ∂s1 Finally we have n Q ((xj − aj )2 + (bj − xj )2 )
n
∂ f j=1
,
∂sn · · · ∂s1 · 2n ∞ ∂nf ∈ L∞ (×ni=1 [ai , bi ]) ; if ∂s · · · ∂s n 1
1/q n
X ∂nf (xj − aj )q+1 + (bj − xj )q+1
· , q+1 Mn,n := ∂sn · · · ∂s1 p j=1 ∂nf 1 1 if ∈ Lp (×ni=1 [ai , bi ]) , p > 1, + = 1; ∂s · · · ∂s p q n
1 n n
Y bj − a j ∂ f a + bj j
+ xj − ,
∂sn · · · ∂s1 · 2 2 1 j=1 ∂nf if ∈ L1 (×ni=1 [ai , bi ]) . ∂sn · · · ∂s1
Proof. Similar to Theorem 3.5.
(3.17a)
(3.17b)
(3.18)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Multidimensional Montgomery Identities and Ostrowski type Inequalities
3.3
29
Application to Probability Theory
Here we consider the random variables Xi ≥ 0, i = 1, 2, 3, taking values in [0, bi ], bi > 0. Let F be the joint probability distribution function of (X1 , X2 , X3 ) assumed to be in C 3 (×3i=1 [0, bi ]). Let f be the corresponding probability density function, 3 that is f = ∂x1∂∂xF2 ∂x3 . Clearly the expectation Z b1 Z b2 Z b3 E(X1 X2 X3 ) = x1 x2 x3 f (x1 , x2 , x3 )dx1 dx2 dx3 . (3.19) 0
0
0
Consider on ×3i=1 [0, bi ] the function
G3 (x1 , x2 , x3 ) := 1 − F (x1 , b2 , b3 ) − F (b1 , x2 , b3 ) − F (b1 , b2 , x3 ) + F (x1 , x2 , b3 ) We see that
+ F (x1 , b2 , x3 ) + F (b1 , x2 , x3 ) − F (x1 , x2 , x3 ).
(3.20)
∂ 3 G3 (x1 , x2 , x3 ) = −f (x1 , x2 , x3 ). ∂x1 ∂x2 ∂x3 In [31], Theorem 6, we established that Z b1 Z b2 Z b3 G3 (x1 , x2 , x3 )dx1 dx2 dx3 . E(X1 X2 X3 ) = 0
0
(3.21)
(3.22)
0
Obviously here we have F (x1 , b2 , b3 ) = FX1 (x1 ), F (b1 , x2 , b3 ) = FX2 (x2 ), F (b1 , b2 , x3 ) = FX3 (x3 ), where FX1 , FX2 , FX3 are the probability distribution functions of X1 , X2 , X3 . Furthermore, F (x1 , x2 , b3 ) = FX1 ,X2 (x1 , x2 ), F (x1 , b2 , x3 ) = FX1 ,X3 (x1 , x3 ), F (b1 , x2 , x3 ) = FX2 ,X3 (x2 , x3 ), where FX1 ,X2 , FX1 ,X3 , FX2 ,X3 are the joint probability distribution functions of (X1 , X2 ), (X1 , X3 ), (X2 , X3 ), respectively. That is G3 (x1 , x2 , x3 ) = 1 − FX1 (x1 ) − FX2 (x2 ) − FX3 (x3 ) + FX1 ,X2 (x1 , x2 ) + FX1 ,X3 (x1 , x3 ) + FX2 ,X3 (x2 , x3 ) − F (x1 , x2 , x3 ).
We set by f1 = ∂ 2 FX2 ,X3 ∂x2 ∂x3
0 , FX 1
f2 =
0 , FX 2
f3 =
0 , FX 3
f12 =
∂ 2 FX1 ,X2 ∂x1 ∂x2
, f13 =
(3.23)
∂ 2 FX1 ,X3 ∂x1 ∂x3
, the corresponding existing probability density functions. f23 = Then we have ∂G3 (x1 , x2 , x3 ) ∂FX1 ,X2 (x1 , x2 ) = −f1 (x1 ) + ∂x1 ∂x1 ∂FX1 ,X3 (x1 , x3 ) ∂F (x1 , x2 , x3 ) + − , ∂x1 ∂x1 ∂G3 (x1 , x2 , x3 ) ∂FX1 ,X2 (x1 , x2 ) = −f2 (x2 ) + ∂x2 ∂x2 ∂FX2 ,X3 (x2 , x3 ) ∂F (x1 , x2 , x3 ) + − , ∂x2 ∂x2 ∂G3 (x1 , x2 , x3 ) ∂FX1 ,X3 (x1 , x3 ) = −f3 (x3 ) + ∂x3 ∂x3 ∂FX2 ,X3 (x2 , x3 ) ∂F (x1 , x2 , x3 ) + − . ∂x3 ∂x3
, and
(3.24)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
30
Furthermore we find that ∂G3 (x1 , x2 , x3 ) ∂ 2 F (x1 , x2 , x3 ) = f12 (x1 , x2 ) − , ∂x1 ∂x2 ∂x1 ∂x2 ∂G3 (x1 , x2 , x3 ) ∂ 2 F (x1 , x2 , x3 ) = f23 (x2 , x3 ) − , ∂x2 ∂x3 ∂x2 ∂x3
(3.25)
and ∂G3 (x1 , x2 , x3 ) ∂ 2 F (x1 , x2 , x3 ) = f13 (x1 , x3 ) − . ∂x1 ∂x3 ∂x1 ∂x3 Then one can apply (3.6) for a1 = a2 = a3 = 0 and f = G3 . The left-hand side of (3.6) will be Z b1 Z b2 Z b3 1 G3 (s1 , s2 , s3 )ds1 ds2 ds3 G3 (x1 , x2 , x3 ) − b1 b2 b3 0 0 0 = 1 − F (x1 , b2 , b3 ) − F (b1 , x2 , b3 )
− F (b1 , b2 , x3 ) + F (x1 , x2 , b3 ) + F (x1 , b2 , x3 ) 1 + F (b1 , x2 , x3 ) − F (x1 , x2 , x3 ) − E(X1 X2 X3 ) , b1 b2 b3
(3.26)
by using (3.20) and (3.21). Here (x1 , x2 , x3 ) ∈ ×3i=1 [0, bi ]. Next one can apply (3.21), (3.24), (3.25) in the right side of (3.6).
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Chapter 4
General Probabilistic Inequalities
In this chapter we use an important integral inequality from [79] to obtain bounds for the covariance of two random variables (1) in a general setup and (2) for a class of special joint distributions. The same inequality is also used to estimate the difference of the expectations of two random variables. Also we present about the attainability of a related inequality. This chapter is based on [80].
4.1
Introduction
In [79] was proved the following result: Theorem 4.1. Let f ∈ C n (B), n ∈ N, where B = [a1 , b1 ] × · · · × [an , bn ], aj , bj ∈ R, with aj < bj , j = 1, ..., n. Denote by ∂B the boundary of the box B. Assume that f (x) = 0, for all x = (x1 , ..., xn ) ∈ ∂B (in other words we assume that f (· · ·, aj , · · ·) = f (· · ·, bj , · · ·) = 0, for all j = 1, ..., n). Then Z n Z ∂ f (x1 , ..., xn ) m(B) (4.1) |f (x1 , ..., xn )| dx1 · · · dxn ≤ ∂x1 · · · ∂xn dx1 · · · dxn , 2n B B Qn where m(B) = j=1 (bj − aj ) is the n-th dimensional volume (i.e. the Lebesgue measure) of B. In the chapter we give probabilistic applications of the above inequality and related remarks.
4.2
Make Applications
Let X ≥ 0 be a random variable. Given a sequence of positive numbers bn → ∞, we introduce the events Bn = {X > bn } .
Then (as in the proof of Chebychev’s inequality) Z X d P ≥ bn P (Bn ) = bn P (X > bn ), E[X 1Bn ] = Bn
31
(4.2)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
32
where, as usual, P is the corresponding probability measure and E its associated expectation. If E [X] < ∞, then monotone convergence implies (since X1Bnc % X) lim E X1Bnc = E [X] . n
Thus
Hence by (4.2)
E [X1Bn ] = E [X] − E X1Bnc → 0. bn P (X > bn ) → 0,
(4.3)
for any sequence bn → ∞. The following proposition is well-known, but we include it here for the sake of completeness: Proposition 4.2. Let X ≥ 0 be a random variable with (cumulative) distribution function F (x). Then Z ∞ E[X] = [1 − F (x)]dx (4.4) 0
(notice that, if the integral in the right hand side diverges, it should diverge to +∞, and the equality still makes sense). Proof. Integration by parts gives Z ∞ 0
lim x [1 − F (x)] +
x→∞
Z
∞
[1 − F (x)] dx =
xdF (x) = lim xP (X > x) + E [X] . x→∞
0
If E [X] < ∞, then (4.4) follows by (4.3). If E [X] = ∞, then the above equation gives that Z ∞ [1 − F (x)] dx = ∞, 0
since xP (X > x) ≥ 0, whenever x ≥ 0.
Remark 4.3. If X ≥ a is a random variable with distribution function F (x), we can apply Proposition 4.2 to Y = X − a and obtain Z ∞ E [X] = a + [1 − F (x)] dx. (4.5) a
We now present some applications of Theorem 4.1.
Proposition 4.4. Consider two random variables X and Y taking values in the interval [a, b]. Let FX (x) and FY (x) respectively be their distribution functions,
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
General Probabilistic Inequalities
33
which are assumed absolutely continuous. Their corresponding densities are f X (x) and fY (x) respectively. Then Z b−a b |fX (x) − fY (x)| dx. (4.6) |E [X] − E [Y ]| ≤ 2 a Proof. We have that FX (b) = FY (b) = 1, thus formula (4.5) gives us E[X] = a + It follows that
Z
b a
[1 − FX (x)]dx
and
|E [X] − E [Y ]| ≤
Z
E[Y ] = a +
Z
b a
[1 − FY (x)]dx.
b a
|FX (x) − FY (x)| dx.
Since FX (x) and FY (x) are absolutely continuous with densities fX (x) and fY (x) respectively, and since FX (a) − FY (a) = FX (b) − FY (b) = 0, Theorem 4.1, together with the above inequality, imply (4.6). Theorem 4.5. Let X and Y be two random variables with joint distribution function F (x, y). We assume that a ≤ X ≤ b and c ≤ Y ≤ d. We also assume that F (x, y) possesses a (joint) probability density function f (x, y). Then Z Z (b − a) (d − c) d b |cov (X, Y )| ≤ |f (x, y) − fX (x)fY (y)| dxdy, (4.7) 4 c a
where
cov (X, Y ) = E [XY ] − E [X] E [Y ] is the covariance of X and Y , while fX (x) and fY (y) are the marginal densities of X and Y respectively. Proof. Integration by parts implies Z dZ E [XY ] = aE [Y ]+cE [X]−ac+ c
b a
[1 − F (x, d) − F (b, y) + F (x, y)] dxdy (4.8)
(in fact, this formula generalizes to n random variables). Notice that F (x, d) is the (marginal) distribution function of X alone, and similarly F (b, y) is the (marginal) distribution function of Y alone. Thus, by (4.5) Z b Z d E [X] = a + [1 − F (x, d)] dx and E [Y ] = c + [1 − F (b, y)] dy, a
c
which derives
(E [X] − a) (E [Y ] − c) = or
Z
c
dZ b a
[1 − F (x, d)] [1 − F (b, y)] dxdy,
E [X] E [Y ] = aE [Y ] + cE [X] − ac+
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
34
Z
c
dZ b a
[1 − F (x, b) − F (a, y) + F (x, d)F (b, y)] dxdy.
Subtracting (4.9) from (4.8) we get cov (X, Y ) = E [XY ] − E [X] E [Y ] = Hence |cov (X, Y )| ≤
Z
c
dZ b a
Z
d c
Z
(4.9)
b a
[F (x, y) − F (x, d)F (a, y)] dxdy.
|F (x, y) − F (x, d)F (b, y)| dxdy.
Now F (a, y) = F (x, c) = 0 and F (b, d) = 1. It follows that F (x, y) − F (x, d)F (b, y) vanishes on the boundary of B = (a, b) × (c, d). Thus Theorem 4.1, together with the above inequality, imply (4.7). In Feller’s classical book [194], par. 5, page 165, we find the following fact: Let F (x) and G(y) be distribution functions on R, and put U (x, y) = F (x)G(y) {1 − α [1 − F (x)] [1 − G(y)]} ,
(4.10)
2
where −1 ≤ α ≤ 1. Then U (x, y) is a distribution function on R , with marginal distributions F (x) and G(y). Furthermore U (x, y) possesses a density if and only if F (x) and G(y) do. Theorem 4.6. Let X and Y be two random variables with joint distribution function U (x, y), as given by (4.10). For the marginal distributions F (x) and G(y), of X and Y respectively, we assume that they posses densities f (x) and g(y) (respectively). Furthermore, we assume that a ≤ X ≤ b and c ≤ Y ≤ d. Then (b − a) (d − c) |cov (X, Y )| ≤ |α| . (4.11) 16 Proof. The (joint) density u(x, y) of U (x, y) is u(x, y) =
∂ 2 U (x, y) = f (x)g(y) {1 − α [1 − 2F (x)] [1 − 2G(y)]} . ∂x∂y
Thus u(x, y) − f (x)g(y) = −α [1 − 2F (x)] [1 − 2G(y)] f (x)g(y)
and Theorem 4.5 implies (b − a) (d − c) |α| 4 or
"Z
b a
|cov (X, Y )| ≤ |1 − 2F (x)| f (x)dx
# "Z
d c
#
|1 − 2G(y)| g(y)dy ,
(b − a) (d − c) E [|1 − 2F (X)|] E [|1 − 2G (Y )|] . (4.12) 4 Now, F (x) and G(y) have density functions, hence F (X) and G (Y ) are uniformly distributed on (0, 1). Thus 1 E [|1 − 2F (X)|] = E [|1 − 2G (Y )|] = 2 and the proof is done by using the above equalities in (4.12). |cov (X, Y )| ≤ |α|
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
General Probabilistic Inequalities
4.3
35
Remarks on an Inequality
The basic ingredient in the proof of Theorem 4.1 is the following inequality (also shown in [79]): Let B = (a1 , b1 ) × · · · × (an , bn ) ⊂ Rn , with aj < bj , j = 1, ..., n. Denote by B and ∂B respectively the (topological) closure and the boundary of the open box B. Consider the functions f ∈ C0 (B) ∩ C n (B), namely the functions that are continuous on B, have n continuous derivatives in B, and vanish on ∂B. Then for such functions we have Z n ∂ f (x1 , ..., xn ) 1 dx1 · · · dxn , |f (x1 , ..., xn )| ≤ n 2 B ∂x1 · · · ∂xn true for all (x1 , ..., xn ) ∈ B. In other words Z n ∂ f (x1 , ..., xn ) 1 kf k∞ ≤ n ∂x1 · · · ∂xn dx1 · · · dxn , 2
(4.13)
B
where k · k∞ is the supnorm of C0 (B).
Remark 4.7. Suppose we have that aj = −∞ for some (or all) j and/or bj = b = B ∪ {∞} be the one-point compactification of ∞ for some (or all) j. Let B b ∩ C n (B). This means that, if aj = −∞, then B and assume that f ∈ C0 (B) limsj →−∞ f (x1 , ..., sj , ..., xn ) = 0, for all xk ∈ (ak , bk ), k 6= j, and also that, if bj = ∞, then limsj →∞ f (x1 , ..., sj , ..., xn ) = 0, for all xk ∈ (ak , bk ), k 6= j. Then the proof of (4.13), as given in [79], remains valid. Remark 4.8. In the case of an unbounded domain, as described in the previous remark, although kf k∞ < ∞, it is possible that Z n ∂ f (x1 , ..., xn ) ∂x1 · · · ∂xn dx1 · · · dxn = ∞. B
For example, take B = (0, ∞) and √ Z x π iπ/4 −ξ iξ 2 f (x) = e − e e dξ. 2 0
Before discussing sharpness and attainability for (4.13) we need some notation. For c = (c1 , ..., cn ) ∈ B, we introduce the intervals Ij,0 (c) = (aj , cj )
and
Ij,1 (c) = (cj , bj ),
j = 1, ..., n.
n
Theorem 4.9. For f ∈ C0 (B) ∩ C (B), inequality (4.13) is sharp. Equality is attained if and only if there is a c = (c1 , ..., cn ) ∈ B such that the n-th mixed derivative ∂ n f (x1 , ..., xn )/∂x1 · · · ∂xn does not change sign in each of the 2n “subboxes” I1,ε1 (c) × · · · × In,εn (c). Proof. (⇐) For an arbitrary c = (c1 , ..., cn ) ∈ B we have Z Z ∂ n f (x1 , ..., xn ) ε1 +···+εn dx1 · · · dxn , (4.14) f (c1 , ..., cn ) = (−1) ··· I1,ε1 (c) In,εn (c) ∂x1 · · · ∂xn
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
36
where each εj can be either 0 or 1. If c is as in the statement of the theorem, then n Z Z ∂ f (x1 , ..., xn ) |f (c1 , ..., cn )| = (4.15) ··· ∂x1 · · · ∂xn dx1 · · · dxn . I1,ε1 (c)
In,εn (c)
Adding up (4.15) for all 2n choices of (ε1 , ..., εn ) we find n Z X Z ∂ f (x1 , ..., xn ) n 2 |f (c1 , ..., cn )| = ··· ∂x1 · · · ∂xn dx1 · · · dxn , (4.16) I (c) I (c) 1,ε n,ε n ε ,...,ε 1 1
n
or
1 |f (c1 , ..., cn )| = n 2
Z n ∂ f (x1 , ..., xn ) ∂x1 · · · ∂xn dx1 · · · dxn . B
This forces (4.13) to become an equality. In fact we also must have |f (c1 , ..., cn )| = kf k∞ , i.e. |f (x)| attains its maximum at x = c. (⇒) Conversely, let c ∈ B be a point at which |f (c)| = kf k∞ (such a c exists since f is continuous and vanishes on ∂B and at infinity). If for this c, the sign of the mixed derivative ∂ n f (x1 , ..., xn )/∂x1 · · · ∂xn changes inside some “sub-box” I1,ε1 (c) × · · · × In,εn (c), then (4.14) implies n Z Z ∂ f (x1 , ..., xn ) ··· kf k∞ = |f (c1 , ..., cn )| < ∂x1 · · · ∂xn dx1 · · · dxn . In,εn (c) I1,ε1 (c) Thus forces f, (4.13) become a strict inequality. 4.4
Lq , q > 1, Related Theory
We give Theorem 4.10. Let f ∈ AC([a, b]) (absolutely continuous): f (a) = f (b) = 0, and p, q > 1 : p1 + 1q = 1. Then Z
b a
Proof. Call c =
1
(b − a)1+ p |f (x)|dx ≤ 2(p + 1)1/p
a+b 2 .
Z
b a
|f 0 (y)|q dy
Then f (x) =
Z
and f (x) = −
x
f 0 (y)dy,
a
Z
b x
f 0 (y)dy,
a ≤ x ≤ c,
c ≤ x ≤ b.
!1/q
.
(4.17)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
General Probabilistic Inequalities
For a ≤ x ≤ c we observe,
and
Z
|f (x)| ≤
Z
|f (x)|dx ≤
Z
c a
x a
a
37
|f 0 (y)|dy,
cZ x
|f 0 (y)|dydx
a
(by Fubini’s theorem) Z
=
c a
Z ≤
c y
Z c |f 0 (y)|dx dy = (c − y)|f 0 (y)|dy a
Z
c
p
(c − y) dy
a
1/p
kf 0 kLq (a,c) .
That is, Z
1
c
|f (x)|dx ≤
a
(b − a)1+ p 1
21+ p (p + 1)1/p
kf 0 kLq (a,c) .
(4.18)
Similarly we obtain
and
Z
|f (x)| ≤
Z
|f (x)|dx ≤
Z
b c
b
|f 0 (y)|dy
x
b c
Z
b x
|f 0 (y)|dydx
(by Fubini’s theorem) Z
=
b c
Z
≤
y c
|f 0 (y)|dxdy = Z
b p
c
(y − c) dy
Z
b c
!1/p
(y − c)|f 0 (y)|dy kf 0 kLq (c,b) .
That is, we get Z
1
b c
|f (x)|dx ≤
Next we see that Z
b a
|f (x)|dx
!q
=
(b − a)1+ p 1
21+ p (p + 1)1/p "Z
kf 0 kLq (c,b) .
c a
|f (x)|dx +
Z
b c
|f (x)|dx
(4.19) #q
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
38
!q
1
≤ ≤
1 1+ p
(p + 1)1/p !q 1 (b − a)1+ p 2
2
(b − a)1+ p
1 1+ p
(p + 1)1/p
kf 0 kLq (a,c) + kf 0 kLq (c,b)
2q−1 kf 0 kqLq (a,c) + kf 0 kqLq (c,b) 1
=
2
(b − a)1+ p
1 1+ p
(p +
q
1)1/p
!q
2q−1 kf 0 kqLq (a,b) .
That is we proved Z
1
b a
|f (x)|dx ≤
2
(b − a)1+ p
1 1+ p
(p +
1
1)1/p
21− q kf 0 kLq (a,b) ,
(4.20)
which gives
proving (4.17).
Z
b a
1
(b − a)1+ p 0 |f (x)|dx ≤ ( kf kLq (a,b) , 2 p + 1)1/p
(4.21)
Remark 4.11. We notice that lim ln(p + 1)
1/p
p→∞
= 0, that is lim (p + 1) p→∞
1/p
= 1.
0
Let f ∈ C ([a, b]), f (a) = f (b) = 0. Then by Theorem 4.1 we obtain Z Z b b−a b 0 |f (x)|dx ≤ |f (y)|dy, 2 a a
(4.22)
which is (4.17) when p = ∞ and q = 1. When p = q = 2, by Theorem 4.10, we get !1/2 Z b Z b (b − a)3/2 0 2 √ |f (x)|dx ≤ (f (y)) dy . (4.23) 12 a a The Lq , q > 1, analog of Proposition 4.4 follows Proposition 4.12. Consider two random variables X and Y taking values in the interval [a, b]. Let FX (x) and FY (x) respectively be their distribution functions which are assumed absolutely continuous. Call their corresponding densities f X (x), fY (x), respectively. Let p, q > 1 : p1 + 1q = 1. Then !1/q 1 Z b (b − a)1+ p q |E(X) − E(Y )| ≤ |fX (y) − fY (y)| dy . (4.24) 2(p + 1)1/p a Proof. Notice that |E(X) − E(Y )| ≤
Z
b a
|FX (x) − FY (x)|dx.
(4.25)
Then use Theorem 4.10 for f = FX − FY , which fulfills the assumptions there. Observe that (4.6) is the case of (4.24) when p = ∞ and q = 1.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Chapter 5
About Grothendieck Inequalities
We give here the analog of Grothendieck inequality for positive linear forms. We find upper and lower bounds of Lp 0 < p ≤ ∞, type, which all lead to sharp inequalities.This treatments is based on [68].
5.1
Introduction
We mention the famous Grothendieck inequality [204]. Theorem 5.1. (Grothendieck 1956) Let K1 and K2 be compact spaces. Let u : C (K1 ) × C (K2 ) → R be a bounded bilinear form. Then there exist probability measures µ1 and µ2 on K1 and K2 , respectively, such that 1/2 1/2 Z Z 2 2 R g dµ2 , (5.1) f dµ1 |u (f, g)| ≤ KG kuk K2
K1
R for all f ∈ C (K1 ) and g ∈ C (K2 ) , where KG is a universal constant. We have that π π R √ = 1.782 . . . . 1.57 . . . = ≤ KG ≤ 2 2 ln 1 + 2
(5.2)
See [204], [240], for the left hand side and right hand side bounds, respectively. R Still KG precise value is unknown. The complex case is studied separately, see [209]. Here we present analogs of Theorem 5.1 for the case of a positive linear form, which is of course a bounded linear one. We use Theorem 5.2. (W. Adamski (1991), [3]) Let X be a pseudocompact space and Y be an arbitrary topological space, then, for any positive bilinear form Φ : C (X) × C (Y ) → R, there exists a uniquely determined measure µ on the product of the Baire σ− algebras of X and Y such that Z Φ (f, g) = f ⊗ gdµ (5.3) X×Y
39
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
40
holds for all f ∈ C (X) and all g ∈ C (Y ) . Note. Above the tensor product (f ⊗ g) (x, y) = f (x) · g (y) , X pseudocompact means all continuous functions f : X → R are bounded. Positivity of Φ means that for f ≥ 0, g ≥ 0 we have Φ (f, g) ≥ 0. From (5.3) we see that 0 ≤ µ (X × Y ) = Φ (1, 1) < ∞. If Φ (1, 1) = 0, then Φ ≡ 0, the trivial case. Also |Φ (f, g)| ≤ Φ (1, 1) kf k∞ kgk∞ , with kΦk = Φ (1, 1) , in the case that X, Y are both pseudocompact, so that Φ is a bounded bilinear form. 5.2
Main Results
We give Theorem 5.3. Let X, Y be pseudocompact spaces, Φ : C (X) × C (Y ) → R be a positive bilinear form and p, q > 1 : p1 + 1q = 1. Then there exist uniquely determined probability measures µ1 , µ2 on the Baire σ−algebras of X and Y, respectively, such that 1/q 1/p Z Z q p |g (y)| dµ2 , (5.4) |f (x)| dµ1 |Φ (f, g)| ≤ kΦk Y
X
for all f ∈ C (X)and all g ∈ C (Y ) . Inequality (5.4) is sharp, namely it is attained, and the constant 1 is the best possible. Proof. By Theorem 5.2 we have Φ (f, g) =
Z
f (x) g (y) dµ (x, y) ,
(5.5)
X×Y
and µ is a unique measure on the product of the Baire σ− algebras of X and Y. Without loss of generality we may suppose that Φ 6≡ 0. Denote by m := µ (X × Y ) , so it is 0 < m < ∞, and consider the measures µ∗ (A) := µ (A × Y ) , for any A in the Baire σ−algebra of X, and µ∗∗ (B) := µ (X × B) , for any B in the Baire σ−algebra of Y. Here µ∗ , µ∗∗ are uniquely defined. Notice that µ∗ (X) = µ (X × Y ) = µ∗∗ (Y ) = m. Denote the probability measures µ1 := Also it holds Z and
p
X×Y
∗
µ m
, µ2 :=
|f (x)| dµ =
Z
∗∗
µ m
.
p
X
(5.6)
|f (x)| dµ∗ ,
(5.7)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
About Grothendieck Inequalities
Z
q
|g (y)| dµ =
X×Y
So from (5.5) we get |Φ (f, g)| ≤
Z
X×Y
Z
41
q
|g (y)| dµ∗∗ .
Y
(5.8)
|f (x)| |g (y)| dµ (x, y)
(5.9)
(by H¨ older’s inequality) ≤
Z
p
X×Y
=
Z
=m
|f (x)| dµ (x, y) p
X
Z
|f (x)| dµ∗ (x)
X
1/p Z
1/p Z
|f (x)|p dµ1 (x)
q
X×Y
|g (y)| dµ (x, y) q
Y
1/p Z
|g (y)| dµ∗∗ (y)
Y
1/q
1/q
|g (y)|q dµ2 (y)
1/q
(5.10)
(5.11)
,
proving (5.4) . Next we derive the sharpness of (5.4) . Let us assume f (x) = c1 > 0, g (y) = c2 > 0. Then the left hand side of (5.4) equals c1 c2 m, equal to the right hand side of (5.4) . That is proving attainability of (5.4) and that 1 is the best constant. We give Corollary 5.4. All as in Theorem 5.3 with p = q = 2. Then Z 1/2 Z 1/2 2 2 |Φ (f, g)| ≤ kΦk (f (x)) dµ1 (g (y)) dµ2 , X
(5.12)
Y
for all f ∈ C (X) and all g ∈ C (Y ) . Inequality (5.12) is sharp and the best constant is 1. So when Φ is positive we improve Theorem 5.1. Corollary 5.5. All as in Theorem 5.3. Then Z 1/p Z 1/q p q |Φ (f, g)| ≤ kΦk inf |f (x)| dµ1 |g (y)| dµ2 , (5.13) {p,q>1: p1 + 1q =1} X Y for all f ∈ C (X) and all g ∈ C (Y ) . Inequality (5.13) is sharp and the best constant is 1. Corollary 5.6. All as in Theorem 5.3, but p = 1, q = ∞. Then
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
42
|Φ (f, g)| ≤ kΦk kgk∞
Z
X
|f (x)| dµ1 ,
(5.14)
for all f ∈ C (X)and all g ∈ C (Y ) . Inequality (5.14)is attained when f (x) ≥ 0 and g (y) = c > 0, so it is sharp. Proof. We observe that |Φ (f, g)| ≤ kgk∞ = kgk∞
Z
Z
= m kgk∞
|f (x)| dµ (x, y) ∗
X
Z
= kΦk kgk∞
X×Y
|f (x)| dµ (x)
X
|f (x)| dµ1 (x)
Z
X
(5.15)
(5.16)
|f (x)| dµ1 .
Sharpness of (5.14) is obvious.
Corollary 5.7. Let X, Y be pseudocompact spaces, Φ : C (X) × C (Y ) → R be a positive bilinear form. Then there exist uniquely determined probability measures µ1 , µ2 on the Baire σ− algebras of X and Y, respectively, such that Z Z |g (y)| dµ2 , (5.17) |f (x)| dµ1 , kf k∞ |Φ (f, g)| ≤ kΦk min kgk∞ X
Y
for all f ∈ C (X) and all g ∈ C (Y ) . Proof. Clear from Corollary 5.6.
Next we give some converse results. We need Definition 5.8. Let X, Y be pseudocompact spaces. We define C + (X) := {f : X → R+ ∪ {0} continuous and bounded} ,
(5.18)
and C ++ (Y ) := {g : Y → R+ continuous and bounded} , where R+ := {x ∈ R : x > 0} . Clearly C + (X) ⊂ C (X) , and C ++ (Y ) ⊂ C (Y ) . So C ++ (Y ) is the positive cone of C (Y ) . Theorem 5.9. Let X, Y be pseudocompact spaces, Φ : C + (X) × C ++ (Y ) → R be a positive bilinear form and 0 < p < 1, q < 0 : p1 + q1 = 1. Then there exist
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
About Grothendieck Inequalities
43
uniquely determined probability measures µ1 , µ2 on the Baire σ− algebras of X and Y, respectively, such that
Φ (f, g) ≥ kΦk
Z
p
(f (x)) dµ1 X
1/p Z
q
(g (y)) dµ2 Y
1/q
,
(5.19)
for all f ∈ C + (X) and all g ∈ C ++ (Y ) . Inequality (5.19) is sharp, namely it is attained, and the constant 1 is the best possible. Proof. We use the same notations as in Theorem 5.3. By Theorem 5.2 and reverse H¨ older’s inequality we obtain Z Φ (f, g) = f (x) g (y) dµ (x, y) ≥ X×Y
Z
p
(f (x)) dµ (x, y) X×Y
=
Z
=m
p
∗
1/p Z
(f (x)) dµ (x) X
Z
p
(g (y)) dµ (x, y) X×Y
1/p Z
(f (x)) dµ1 (x) X
q
q
(g (y)) dµ Y
1/p Z
q
∗∗
(y)
(5.20)
1/q
(g (y)) dµ2 (y) Y
1/q
1/q
,
(5.21)
proving (5.19) . For the sharpness of (5.19) , take f (x) = c1 > 0, g (y) = c2 > 0. Then L.H.S (5.19) = R.H.S (5.19) = mc1 c2 . We need Definition 5.10. Let X, Y be pseudocompact spaces. We define C − (X) := {f : X → R− ∪ {0} continuous and bounded} ,
(5.22)
and C −− (Y ) := {g : Y → R− continuous and bounded} ,
where R− := {x ∈ R : x < 0} . Clearly C − (X) ⊂ C (X) , and C −− (Y ) ⊂ C (Y ) . So C −− (Y ) is the negative cone of C (Y ) .
Theorem 5.11. Let X, Y be pseudocompact spaces, Φ : C − (X) × C −− (Y ) → R be a positive bilinear form and 0 < p < 1, q < 0 : p1 + q1 = 1. Then there exist uniquely determined probability measures µ1 , µ2 on the Baire σ− algebras of X and Y, respectively, such that Z 1/p Z 1/q Φ (f, g) ≥ kΦk |f (x)|p dµ1 |g (y)|q dµ2 , (5.23) X
Y
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
44
for all f ∈ C − (X) and all g ∈ C −− (Y ) . Inequality (5.23) is sharp, namely it is attained, and the constant 1 is the best possible. Proof. Acting as in Theorem 5.9, we have that Φ (f, g) ≥ 0 and Z Z Φ (f, g) = f (x) g (y) dµ (x, y) = |f (x)| |g (y)| dµ (x, y)
(5.24)
1/q
(5.25)
X×Y
≥
Z
X×Y
p
X×Y
=
Z
=m
|f (x)| dµ (x, y) p
X
Z
∗
|f (x)| dµ (x) p
X
1/p Z
1/p Z
|f (x)| dµ1 (x)
q
X×Y
|g (y)| dµ (x, y) q
|g (y)| dµ
Y
1/p Z
∗∗
(y)
q
Y
1/q
|g (y)| dµ2 (y)
1/q
,
(5.26)
establishing (5.23) . For the sharpness of (5.23) , take f (x) = c1 < 0, g (y) = c2 < 0. Then L.H.S (5.23) = mc1 c2 = R.H.S (5.23) . Hence optimality in (5.23) is established. We finish this chapter with Corollary 5.12. (to Theorem 5.9) We have Φ (f, g) ≥ kΦk "Z
p
(f (x)) dµ1 X
sup {p,q:
1 0
1/p Z
q
(g (y)) dµ2 Y
1/q #
,
(5.27)
for all f ∈ C + (X) and all g ∈ C ++ (Y ) . Inequality (5.27)is sharp, namely it is attained, and the constant 1 is the best possible. And analogously we find Corollary 5.13. (to Theorem 5.11) We have Φ (f, g) ≥ kΦk "Z
p
X
|f (x)| dµ1
sup {p,q:
1 + 1q =1} 0
1/p Z
q
Y
|g (y)| dµ2
1/q #
,
(5.28)
for all f ∈ C − (X) and all g ∈ C −− (Y ) . Inequality (5.28) is sharp, namely it is attained, and the constant 1 is the best possible.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Chapter 6
Basic Optimal Estimation of Csiszar’s f -Divergence
In this chapter are established basic sharp probabilistic inequalities that give best approximation for the Csiszar’s f -divergence, which is the most essential and general measure for the discrimination between two probability measures.This treatment relies on [37]. 6.1
Background
Throughout this chapter we use the following. Let f be a convex function from (0, +∞) into R which is strictly convex at 1 with f (1) = 0. Let (X, A, λ) be a measure space, where λ is a finite or a σ-finite measure on (X, A). And let µ1 , µ2 be two probability measures on (X, A) such that µ1 λ, µ2 λ (absolutely continuous), e.g. λ = µ1 + µ2 . dµ2 1 Denote by p = dµ dλ , q = dλ the (densities) Radon–Nikodym derivatives of µ1 , µ2 with respect to λ. Here we suppose that p 0 < a ≤ ≤ b, a.e. on X and a ≤ 1 ≤ b. q The quantity Γf (µ1 , µ2 ) =
Z
q(x)f X
p(x) q(x)
dλ(x),
(6.1)
was introduced by I. Csiszar in 1967, see [140], and is called f -divergence of the probability measures µ1 and µ2 . By Lemma 1.1 of [140], the integral (6.1) is well-defined and Γf (µ1 , µ2 ) ≥ 0 with equality only when µ1 = µ2 , see also our Theorem 6.8, etc, at the end of the chapter. Furthermore Γf (µ1 , µ2 ) does not depend on the choice of λ. The concept of f -divergence was introduced first in [139] as a generalization of Kullback’s “information for discrimination” or I-divergence (generalized entropy) [244], [243] and of R´enyi’s “information gain” (I-divergence) of order α [306]. In fact the I-divergence of order 1 equals Γu log2 u (µ1 , µ2 ). 45
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
46
The choice f (u) = (u − 1)2 produces again a known measure of difference of distributions that is called χ2 -divergence. Of course the total variation distance Z |µ1 − µ2 | = |p(x) − q(x)| dλ(x) X
is equal to Γ|u−1| (µ1 , µ2 ). Here by supposing f (1) = 0 we can consider Γf (µ1 , µ2 ) the f -divergence as a measure of the difference between the probability measures µ1 , µ2 . The f -divergence is in general asymmetric in µ1 and µ2 . But since f is convex and strictly convex at 1 so is 1 f ∗ (u) = uf (6.2) u and as in [140] we obtain Γf (µ2 , µ1 ) = Γf ∗ (µ1 , µ2 ).
(6.3)
In Information Theory and Statistics many other divergences are used which are special cases of the above general Csiszar f -divergence, for example Hellinger distance DH , α-divergence Da , Bhattacharyya distance DB , Harmonic distance DHa , Jeffrey’s distance DJ , triangular discrimination D∆ , for all these see e.g. [102], [152]. The problem of finding and estimating the proper distance (or difference or discrimination) of two probability distributions is one of the major ones in Probability Theory. The above f -divergence measures in their various forms have been also applied to Anthropology, Genetics, Finance, Economics, Political science, Biology, Approximation of Probability distributions, Signal Processing and Pattern Recognition. A great motivation for this chapter has been the very important monograph on the matter by S. Dragomir [152]. 6.2
Main Results
We give the following. Theorem 6.1. Let f ∈ C 1 ([a, b]). Then 0
Γf (µ1 , µ2 ) ≤ kf k∞,[a,b]
Z
X
|p(x) − q(x)| dλ(x).
(6.4)
Inequality (6.4) is sharp. The optimal function is f ∗ (y) = |y − 1|α , α > 1 under the condition, either i)
max(b − 1, 1 − a) < 1 and C := esssup(q) < +∞,
(6.5)
or ii)
|p − q| ≤ q ≤ 1, a.e. on X.
(6.6)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Basic Optimal Estimation of Csiszar’s f -Divergence
Proof. We see that Γf (µ1 , µ2 ) = = = = ≤ ≤ =
Book
47
p(x) dλ q(x)f q(x) X Z p(x) q(x)f dλ − f (1) q(x) X Z Z p(x) q(x)f dλ − q(x)f (1) dλ q(x) X X Z p(x) − f (1) dλ q(x) f q(x) X Z p(x) q(x) f − f (1) dλ q(x) X Z p(x) 0 − 1 dλ kf k∞,[a,b] q(x) q(x) ZX 0 kf k∞,[a,b] |p(x) − q(x)| dλ, Z
X
proving (6.4). Clearly here f ∗ is convex, f ∗ (1) = 0 and strictly convex at 1. Also we have f ∗0 (y) = α|y − 1|α−1 sign(y − 1) and kf ∗0 k∞,[a,b] = α max(b − 1, 1 − a) We have L.H.S.(6.4) = and
Z
X
.
(q(x))1−α |p(x) − q(x)|α dλ,
R.H.S.(6.4) = α max(b − 1, 1 − a) Hence lim R.H.S.(6.4) =
α→1
α−1
Z
X
α−1
Z
X
|p(x) − q(x)| dλ.
|p(x) − q(x)| dλ.
(6.7)
(6.8)
(6.9)
Based on Lemma 6.3 (6.22) following we find lim L.H.S.(6.4) = lim R.H.S.(6.4),
α→1
establishing that inequality (6.4) is sharp.
α→1
(6.10)
Next we give the more general Theorem 6.2. Let f ∈ C n+1 ([a, b]), n ∈ N, such that f (k) (1) = 0, k = 0, 2, . . . , n. Then Z kf (n+1) k∞,[a,b] Γf (µ1 , µ2 ) ≤ (q(x))−n |p(x) − q(x)|n+1 dλ(x). (6.11) (n + 1)! X
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
48
Inequality (6.11) is sharp. The optimal function is f˜(y) := |y − 1|n+α , α > 1 under the condition (i) or (ii) of Theorem 6.1. Proof. Let any y ∈ [a, b], then f (y) − f (1) = where Rn (1, y) :=
Z
y 1
n X f (k) (1) (y − 1)k + Rn (1, y), k!
(6.12)
k=1
f (n) (t) − f (n) (1)
here y ≥ 1 or y ≤ 1. As in [27], p. 500 we get |Rn (1, y)| ≤
(y − t)n−1 dt, (n − 1)!
(6.13)
kf (n+1) k∞,[a,b] |y − 1|n+1 , (n + 1)!
(6.14)
for all y ∈ [a, b]. By theorem’s assumption we have f (y) = f 0 (1)(y − 1) + Rn (1, y),
all y ∈ [a, b].
(6.15)
Consequently we obtain Z Z p(x) p(x) q(x)f dλ(x) = q(x)Rn 1, dλ(x). q(x) q(x) X X Therefore
(6.16)
p(x) Γf (µ1 , µ2 ) ≤ q(x) Rn 1, dλ(x) q(x) X n+1 Z (by (6.14)) kf (n+1) k p(x) ∞,[a,b] dλ(x) q(x) − 1 ≤ (n + 1)! q(x) X Z kf (n+1) k∞,[a,b] = (q(x))−n |p(x) − q(x)|n+1 dλ(x), (n + 1)! X Z
proving (6.11). Clearly here f˜ is convex, f˜(1) = 0 and strictly convex at 1, also f˜ ∈ C n+1 ([a, b]). Furthermore f˜(k) (1) = 0,
k = 1, 2, . . . , n,
and f˜(n+1) (y) = (n + α)(n + α − 1) · · · (α + 1)α|y − 1|α−1 sign(y − 1) Thus
|f˜(n+1) (y)| =
n Y
j=0
(n + α − j) |y − 1|α−1 ,
n+1
.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Basic Optimal Estimation of Csiszar’s f -Divergence
and
kf˜(n+1) k∞,[a,b] =
n Y
j=0
(n + α − j) max(b − 1, 1 − a)
49
α−1
.
Applying f˜ into (6.11) we derive n+α Z p(x) − 1 dλ L.H.S.(6.11) = q(x) q(x) X Z = (q(x))1−n−α |p(x) − q(x)|n+α dλ(x),
(6.17)
(6.18)
X
and
R.H.S.(6.11) =
n Q
j=0
· Then
Z
(n + α − j)(max(b − 1, 1 − a))α−1 (n + 1)!
X
lim R.H.S.(6.11) =
α→1
(q(x))−n |p(x) − q(x)|n+1 dλ(x).
Z
X
(q(x))−n |p(x) − q(x)|n+1 dλ(x).
(6.19)
(6.20)
By applying Lemma 6.3 here, see next, we find Z lim L.H.S.(11) = lim (q(x))1−n−α |p(x) − q(x)|n+α dλ(x) α→1
α→1
=
Z
X
X
(q(x))−n |p(x) − q(x)|n+1 dλ(x),
proving the sharpness of (6.11).
Lemma 6.3. Here α > 1, n ∈ Z+ and condition i) or ii) of Theorem 6.1 is valid. Then Z lim (q(x))1−n−α |p(x) − q(x)|n+α dλ(x) α→1
X
=
Z
X
(q(x))−n |p(x) − q(x)|n+1 dλ(x).
When n = 0 we have Z Z 1−α α lim (q(x)) |p(x) − q(x)| dλ(x) = |p(x) − q(x)| dλ(x). α→1
X
X
Proof. i) Let the condition (i) of Theorem 6.1 hold. We have a−1≤
p(x) − 1 ≤ b − 1, q(x)
a.e. on X.
(6.21)
(6.22)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
PROBABILISTIC INEQUALITIES
50
Then
Hence
and
p(x) q(x) − 1 ≤ max(b − 1, 1 − a) < 1,
a.e. on X.
n+α n+1 p(x) p(x) p(x) ≤ − 1 ≤ − 1 < 1, a.e. on X, q(x) − 1 q(x) q(x) n+α n+1 p(x) p(x) lim − 1 = − 1 , α→1 q(x) q(x)
a.e. on X.
By Dominated convergence theorem we obtain n+α n+1 Z Z p(x) p(x) − 1 dλ(x) = − 1 dλ(x). lim α→1 X q(x) X q(x)
Then notice that Z 0≤ q(x) X
≤C
Z
X
Consequently, lim
α→1
Z
n+1 n+α ! p(x) p(x) − − 1 dλ q(x) − 1 q(x) n+1 n+α ! p(x) p(x) − − 1 dλ → 0, as α → 1. q(x) − 1 q(x)
n+α n+1 Z p(x) p(x) q(x) q(x) − 1 dλ = − 1 dλ, q(x) q(x) X X
implying (6.21), etc. ii) By q ≤ 1 a.e. on X we get q n+α ≤ q n+α−1 a.e. on X. |p − q|n+α ≤ q n+α−1 a.e. on X.
Here by assumptions obviously q > 0, q 1−n−α > 0 a.e. on X. Set o1 := x ∈ X : |p(x) − q(x)|n+α > (q(x))n+α−1 ,
then λ(o1 ) = 0, then λ(o2 ) = 0,
o2 := x ∈ X : q(x) = 0 ,
o3 := x ∈ X : (q(x))1−n−α |p(x) − q(x)|n+α > 1 .
Let x0 ∈ o3 , then
(q(x0 ))1−n−α |p(x0 ) − q(x0 )|n+α > 1.
If q(x0 ) > 0, i.e. (q(x0 ))n+α−1 > 0 and (q(x0 ))1−n−α < +∞ then |p(x0 ) − q(x0 )|n+α > (q(x0 ))n+α−1 . Hence x0 ∈ o1 .
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Basic Optimal Estimation of Csiszar’s f -Divergence
51
If q(x0 ) = 0, then x0 ∈ o2 . Clearly o3 ⊆ o1 ∪ o2 and since λ(o1 ∪ o2 ) = 0 we get that λ(o3 ) = 0. In the case of q(x0 ) = 0 we get from above ∞(p(x0 ))n+α > 1, which is only possible if p(x0 ) > 0. That is, we established that (q(x))1−n−α |p(x) − q(x)|n+α ≤ 1,
a.e. on X.
Furthermore lim (q(x))1−n−α |p(x) − q(x)|n+α = (q(x))−n |p(x) − q(x)|n+1 , a.e. on X.
α→1
Therefore by Dominated convergence theorem we derive Z Z 1−n−α n+α (q(x)) |p(x) − q(x)| dλ → (q(x))−n |p(x) − q(x)|n+1 dλ, X
X
as α → 1, proving (6.21), etc.
Note 6.4. See that |p − q| ≤ q a.e. on X is equivalent to p ≤ 2q a.e. on X. Corollary 6.5. (to Theorem 6.2) Let f ∈ C 2 ([a, b]), such that f (1) = 0. Then Z kf (2) k∞,[a,b] Γf (µ1 , µ2 ) ≤ · (q(x))−1 (p(x) − q(x))2 dλ(x). (6.23) 2 X ˜ := |y − 1|1+α , α > 1 under Inequality (6.23) is sharp. The optimal function is f(y) the condition (i) or (ii) of Theorem 6.1.
Note 6.6. Inequalities (6.4) and (6.11) have some similarity to corresponding ones from those of N.S. Barnett et al. in Theorem 1, p. 3 of [102], though there the setting is slightly different and the important matter of sharpness is not discussed at all. Next we connect Csiszar’s f -divergence to the usual first modulus of continuity ω1 . Theorem 6.7. Suppose that Z 0 < h := |p(x) − q(x)| dλ(x) ≤ min(1 − a, b − 1).
(6.24)
X
Then
Γf (µ1 , µ2 ) ≤ ω1 (f, h).
(6.25)
Inequality (6.25) is sharp, namely it is attained by f ∗ (x) = |x − 1|. Proof. By Lemma 8.1.1, p. 243 of [20] we obtain ω1 (f, δ) |x − 1|, δ for any 0 < δ ≤ min(1 − a, b − 1). Therefore Z p(x) ω1 (f, δ) Γf (µ1 , µ2 ) ≤ q(x) − 1 dλ(x) δ q(x) X Z ω1 (f, δ) = |p(x) − q(x)| dλ(x). δ X f (x) ≤
(6.26)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
52
By choosing δ = h we establish (6.25). Notice ω1 (|x − 1|, δ) = δ, for any δ > 0. Consequently Z Z p(x) Γ|x−1| (µ1 , µ2 ) = q(x) − 1 dλ(x) = |p(x) − q(x)| dλ(x) q(x) X X Z = ω1 |x − 1|, |p(x) − q(x)| dλ(x) , X
proving attainability of (6.25).
Finally we reproduce the essence of Lemma 1.1. of [140] in a much simpler way. Theorem 6.8. Let f : (0 + ∞) → R convex which is strictly convex at 1, 0 < a ≤ 1 ≤ b, a ≤ pq ≤ b , a.e on X. Then Γf (µ1 , µ2 ) ≥ f (1).
(6.27)
f 0 (1)+f 0 (1)
0 Proof Set b := − 2 + ∈ R, because by f convexity both f± exist. Since f is convex we get in general that
f (u) ≥ f (1) + b(u − 1), We prove this last fact: Let
1 < u 1 < u2 ,
∀u ∈ (0, +∞). then
f (u2 ) − f (1) f (u1 ) − f (1) ≤ , u1 − 1 u2 − 1
hence
0 f+ (1) ≤
That is
f (u) − f (1) u−1
for
u > 1.
0 f (u) − f (1) ≥ f+ (1)(u − 1) for u > 1.
Let now u1 < u2 < 1, then
hence
f (u1 ) − f (1) f (u2 ) − f (1) 0 ≤ ≤ f− (1), u1 − 1 u2 − 1 0 f (u) − f (1) ≥ f− (1)(u − 1)
0 0 Also by convexity f− (1) ≤ f+ (1), and 0 f− (1) ≤
for u < 1.
0 0 f− (1) + f+ (1) 0 ≤ f+ (1). 2
That is, we have 0 0 f− (1) ≤ b ≤ f+ (1).
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Basic Optimal Estimation of Csiszar’s f -Divergence
Book
53
0 Let u > 1 then f (u) ≥ f (1) + f+ (1)(u − 1) ≥ f (1) + b(u − 1), that is
f (u) ≥ f (1) + b(u − 1)
for
u > 1.
0 Let u < 1 then f (u) ≥ f (1) + f− (1)(u − 1) ≥ f (1) + b(u − 1), that is
f (u) ≥ f (1) + b(u − 1) for u < 1. Consequently it holds f (u) ≥ f (1) + b(n − 1),
∀u ∈ (0, +∞),
proving this important feature of f . Therefore lim
u→+∞
We do have f and
p p ≥ f (1) + b −1 , q q qf
The last gives
p ≥ f (1)q + b(p − q), q Z
which is
qf X
f (u) u
≥ b.
a.e.on X,
a.e.on X.
p dλ ≥ f (1), q
Γf (µ1 , µ2 ) ≥ f (1).
Note 6.9 (to Theorem 6.8). Since f is strictly convex at 1 we get f (u) > f (1) + b(u − 1), ∀u ∈ (0, ∞) − {1}, with equality at u = 1. Consequently f
p p > f (1) + b −1 , q q
p 6= q,
with equality if p = q; a.e. on X. Thus p qf ≥ f (1)q + b(p − q), whenever p 6= q; a.e.onX, q and
Γf (µ1 , µ2 ) > f (1), if µ1 6= µ2 .
(6.28)
Γf (µ1 , µ2 ) = f (1), if µ1 6= µ2 .
(6.29)
Clearly Note 6.10 (to Theorem 6.8). If f (1) = 0, Γf (µ1 , µ2 ) = 0, if µ1 = µ2 .
then Γf (µ1 , µ2 ) > 0, for µ1 6= µ2 , and
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
This page intentionally left blank
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Chapter 7
Approximations via Representations of Csiszar’s f -Divergence
The Csiszar’s discrimination is the most essential and general measure for the comparison between two probability measures. Here we provide probabilistic representation formulae for it on various very general settings. Then we present tight estimates for their remainders involving a variety of norms of the engaged functions. Also are given some direct general approximations for the Csiszar’s f -divergence leading to very close probabilistic inequalities. We give also many important applications. This treatment relies a lot on [43].
7.1
Background
Throughout this chapter we use the following. Let f be a convex function from (0, +∞) into R which is strictly convex at 1 with f (1) = 0. Let (X, A, λ) be a measure space, where λ is a finite or a σ-finite measure on (X, A). And let µ1 , µ2 be two probability measures on (X, A) such that µ1 λ, µ2 λ (absolutely continuous), e.g. λ = µ1 + µ2 . dµ2 1 Denote by p = dµ dλ , q = dλ the (densities) Radon-Nikodym derivatives of µ1 , µ2 with respect to λ. Here we suppose that p 0 < a ≤ ≤ b, a.e. on X and a ≤ 1 ≤ b. q The quantity Z p(x) Γf (µ1 , µ2 ) = q(x)f dλ(x), (7.1) q(x) X was introduced by I. Csiszar in 1967, see [140], and is called f -divergence of the probability measures µ1 and µ2 . By Lemma 1.1 of [140], the integral (7.1) is well-defined and Γf (µ1 , µ2 ) ≥ 0 with equality only when µ1 = µ2 . Furthermore Γf (µ1 , µ2 ) does not depend on the choice of λ. The concept of f -divergence was introduced first in [139] as a generalization of Kullback’s “information for discrimination” or I-divergence (generalized entropy) [244], [243] and of R´enyi’s “information gain” (I-divergence of order α) [306]. In fact the I-divergence of order 1 equals Γu log2 u (µ1 , µ2 ). 55
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
56
The choice f (u) = (u − 1)2 produces again a known measure of difference of distributionsR that is called χ2 -divergence. Of course the total variation distance |µ1 − µ2 | = X |p(x) − q(x)| dλ(x) is equal to Γ|u−1| (µ1 , µ2 ). Here by supposing f (1) = 0 we can consider Γf (µ1 , µ2 ), the f -divergence as a measure of the difference between the probability measures µ1 , µ2 . The f -divergence is in general asymmetric in µ1 and µ2 . But since f is convex and strictly convex at 1 so is 1 ∗ (7.2) f (u) = uf u and as in [140] we obtain Γf (µ2 , µ1 ) = Γf ∗ (µ1 , µ2 ).
(7.3)
In Information Theory and Statistics many other divergences are used which are special cases of the above general Csiszar f -divergence, e.g. Hellinger distance D H , α-divergence Dα , Bhattacharyya distance DB , Harmonic distance DHa , Jeffrey’s distance DJ , triangular discrimination D∆ , for all these see, e.g. [102], [152]. The problem of finding and estimating the proper distance (or difference or discrimination) of two probability distributions is one of the major ones in Probability Theory. The above f -divergence measures in their various forms have been also applied to Anthropology, Genetics, Finance, Economics, Political Science, Biology, Approximation of Probability distributions, Signal Processing and Pattern Recognition. A great motivation for this chapter has been the very important monograph on the topic by S. Dragomir [152]. 7.2
All Results
We present the following Theorem 7.1. Let f, g ∈ C 1 ([a, b]) where f as in this chapter, g 0 6= 0 over [a, b]. Then
0 Z
f
g p(x) − g(1) dλ. q(x) (7.4) Γf (µ1 , µ2 ) ≤
g0 q(x) ∞,[a,b]
X
Proof. From [112], p. 197, Theorem 27.8, Cauchy’s Mean Value Theorem, we have that f 0 (c)(g(b) − g(a)) = g 0 (c)(f (b) − f (a)), where c ∈ (a, b). Then
0 f (c) |f (b) − f (a)| = 0 |g(b) − g(a)|, g (c)
(7.5)
(7.6)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Approximations via Representations of Csiszar’s f -Divergence
and
0
f
|g(z) − g(y)|, |f (z) − f (y)| ≤
g0 ∞,[a,b]
for any y, z ∈ [a, b]. Clearly it holds 0 f p(x)
q(x) f − f (1) ≤
g0 q(x)
a.e. on X. The last gives (7.4).
Examples 7.2 (to Theorem 7.1). 1) Let g(x) = x1 . Then 2 0
Γf (µ1 , µ2 ) ≤ kx f k∞,[a,b] 2) Let g(x) = ex . Then
p(x) q(x) g − g(1) , q(x) ∞,[a,b]
Z
X
Γf (µ1 , µ2 ) ≤ ke−x f 0 k∞,[a,b] 3) Let g(x) = e−x . Then x 0
Γf (µ1 , µ2 ) ≤ ke f k∞,[a,b] 4) Let g(x) = ln x. Then
Z
Γf (µ1 , µ2 ) ≤ kxf 0 k∞,[a,b]
Z
X
Z
q(x) |p(x) − q(x)| dλ(x). p(x)
X
p(x) q(x) e q(x) − e dλ(x).
p(x) q(x) e− q(x) − e−1 dλ(x).
p(x) q(x) ln dλ(x). q(x) X
5) Let g(x) = x ln x, with a > e−1 . Then
Z p(x)
f0
dλ(x).
p(x) ln Γf (µ1 , µ2 ) ≤ 1 + ln x q(x) 6) Let g(x) =
∞,[a,b]
√
Book
57
(7.7)
(7.8)
(7.9)
(7.10)
(7.11)
(7.12)
(7.13)
X
x. Then Z p p p √ Γf (µ1 , µ2 ) ≤ 2k xf 0 k∞,[a,b] q(x)| p(x) − q(x)| dλ(x).
(7.14)
X
7) Let g(x) = xα , α > 1. Then
0 Z 1 f
Γf (µ1 , µ2 ) ≤ α−1 q(x)1−α |(p(x))α − (q(x))α | dλ(x). α x ∞,[a,b] X
(7.15)
Next we give
Theorem 7.3. Let f ∈ C 1 ([a, b]). Then Z b 1 f (y) dy Γf (µ1 , µ2 ) − b−a a 2 Z 0 kf k∞,[a,b] a + b2 ≤ − (a + b) + q −1 p2 dλ(x) . (7.16) (b − a) 2 X
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
58
Proof. Let z ∈ [a, b], then by Theorem 22.1, p. 498, [27] we have 1 Z b (z − a)2 + (b − z)2 f (y) dy − f (z) ≤ kf 0 k∞,[a,b] =: A(z), b − a a 2(b − a) which is the basic Ostrowski inequality [277]. That is Z b 1 −A(z) ≤ f (z) − f (y)dy ≤ A(z), b−a a and
−qA
Z b p q p p ≤ qf − f (y)dy ≤ qA , q q b−a a q
a.e. on X. Integrating (7.19) against λ we find Z Z b Z p 1 p − qA dλ ≤ Γf (µ1 , µ2 ) − f (y)dy ≤ qA dλ. q b−a a q X X
That is,
proving (7.16).
Z Z b p 1 qA f (y)dy ≤ dλ Γf (µ1 , µ2 ) − b−a a q X 2 2 ! Z kf 0 k∞,[a,b] p p = q −a + b− dλ 2(b − a) X q q Z kf 0 k∞,[a,b] = q −1 (p − aq)2 + (bq − p)2 dλ, 2(b − a) X
Note 7.4. In general we have, if |A(z)| ≤ B(z),then Z Z A(z)dλ ≤ B(z)dλ. X
(7.17)
(7.18)
(7.19)
(7.20)
(7.21)
(7.22)
X
Remark 7.5. Let f as in this chapter and f ∈ C 1 ([a, b]). Then we have Montogemery’s identity, see p. 69 of [169], Z b Z b 1 1 f (z) = f (t)dt + P ∗ (z, t)f 0 (t)dt, (7.23) b−a a b−a a for all z ∈ [a, b], and
∗
P (z, t) := Hence
a.e. on X.
t − a, t − b,
t ∈ [a, z] t ∈ (z, b].
Z b Z b q q p p ∗ qf = f (t)dt + P , t f 0 (t)dt, q b−a a b−a a q
(7.24)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Approximations via Representations of Csiszar’s f -Divergence
We get the following representation formula ! Z b 1 Γf (µ1 , µ2 ) = f (t)dt + R1 , b−a a where R1 :=
Z
q(x) X
Z
b
P a
∗
! p(x) 0 , t f (t)dt dλ(x). q(x)
Book
59
(7.25)
(7.26)
Remark 7.6. Let f be as in this chapter and f ∈ C (2) [a, b]. Then according to (2.2), p. 71 of [169], or (4.104), p. 192 of [126] we have Z b f (b) − f (a) 1 a+b f (t)dt + z− f (z) = b−a a b−a 2 Z bZ b 1 + P ∗ (z, t)P ∗ (t, s)f 00 (s)dsdt, (7.27) (b − a)2 a a for all z ∈ [a, b], where P ∗ is defined by (7.24). So that Z b p q f (b) − f (a) p a+b qf = f (t)dt + q − q b−a a b−a q 2 Z bZ b q p + P∗ , t P ∗ (t, s)f 00 (s)dsdt, (7.28) (b − a)2 a a q a.e. on X. By integration against λ we obtain the representation formula Z b 1 f (b) − f (a) a+b R2 Γf (µ1 , µ2 ) = f (t)dt + 1− + , (7.29) b−a a b−a 2 (b − a)2 where ! Z Z bZ b p ∗ 00 ∗ R2 := q P , t P (t, s)f (s)dsdt dλ. (7.30) q X a a Again from Theorem 2.1, p. 70 of [169], or Theorem 4.17, p. 191 of [126] we see that Z b 1 f (b) − f (a) a + b f (t)dt − z− f (z) − b−a a b−a 2 " # 2 2 1 1 1 z − a+b 2 ≤ + + (b − a)2 kf 00 k∞,[a,b] , (7.31) 2 (b − a)2 4 12 for all z ∈ [a, b]. Therefore Z b p q f (b) − f (a) (a + b) − f (t)dt − p− q qf q b−a a b−a 2 2 p a+b 2 1 q 1 q − 2 ≤ q + + (b − a)2 kf 00 k∞,[a,b] , 2 (b − a)2 4 12
(7.32)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
60
a.e. on X. Using (7.22) we have established that Z b 1 f (b) − f (a) a + b f (t)dt − 1− Γf (µ1 , µ2 ) − b−a a b−a 2 2 Z p a+b 2 2 (b − a) 1 1 q − 2 00 kf k∞,[a,b] q + dλ + . ≤ 2 (b − a)2 4 12 X
(7.33)
From p. 183 of [32] we have
Lemma 7.7. Let f : [a, b] → R be 3-times differentiable on [a, b]. Assume that f 000 is integrable on [a, b]. Let z ∈ [a, b]. Define s−a , a≤s≤r b−a P (r, s) := (7.34) s−b , r < s ≤ b; r, s ∈ [a, b]. b−a Then Z b Z b 1 f (b) − f (a) f (z) = f (s1 )ds1 + P (z, s1 )ds1 b−a a b−a a Z bZ b 0 f (b) − f 0 (a) + P (z, s1 )P (s1 , s2 )ds1 ds2 b−a a a Z bZ bZ b + P (z, s1 )P (s1 , s2 )P (s2 , s3 )f 000 (s3 )ds1 ds2 ds3 . (7.35) a
a
a
Remark 7.8. Let f as in this chapter and f ∈ C (3) ([a, b]). Then from (7.35) we obtain Z b Z b q f (b) − f (a) p p = f (s1 )ds1 + q P , s1 ds1 qf q b−a a b−a q a 0 Z bZ b f (b) − f 0 (a) p + q P , s1 P (s1 , s2 )ds1 ds2 b−a q a a Z bZ bZ b p +q P , s1 P (s1 , s2 )P (s2 , s3 )f 000 (s3 )ds1 ds2 ds3 , (7.36) q a a a a.e. on X. We have derived the representation formula ! Z Z b Z b 1 f (b) − f (a) p Γf (µ1 , µ2 ) = f (s1 )ds1 + , s1 ds1 dλ q P b−a a b−a q X a ! 0 Z Z bZ b f (b) − f 0 (a) p + , s1 P (s1 , s2 )ds1 ds2 dλ + R3 , (7.37) q P b−a q X a a where
R3 :=
Z
q X
Z
b a
Z
b a
Z
b
P a
! p 000 , s1 P (s1 , s2 )P (s2 , s3 )f (s3 )ds1 ds2 ds3 dλ. q (7.38)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Approximations via Representations of Csiszar’s f -Divergence
Book
61
We use also Theorem 7.9. ([32]). Let f : [a, b] → R be 3-times differentiable on [a, b]. Assume that f 000 is measurable and bounded on [a, b]. Let z ∈ [a, b]. Then Z b 1 f (b) − f (a) a+b f (s1 )ds1 − z− f (z) − (b − a) a b−a 2 0 f (b) − f 0 (a) (a2 + b2 + 4ab) 2 − z − (a + b)z + 2(b − a) 6 ≤
where
kf 000 k∞,[a,b] · A∗ (z), (b − a)3
(7.39)
1 1 1 1 A (z) := abz 4 − a2 b3 z + a3 bz 2 − ab2 z 3 − a3 b2 z + ab3 z 2 + a2 b2 z 2 3 3 3 3 1 1 1 3 3 1 − a2 bz 3 − az 5 − bz 5 + z 6 + a2 z 4 + b2 z 4 + b2 a4 2 2 6 4 4 3 2 3 3 2 3 3 1 3 3 5 4 2 5 4 2 1 4 2 − a z − b z − b a + a z + b z + b a 3 3 3 12 12 3 2 5 1 5 1 5 a6 b6 2 5 ba − ab − a z − b z + + . (7.40) − 15 15 6 6 20 20 Inequality (7.39) is attained by ∗
f (z) = (z − a)3 + (b − z)3 ;
(7.41)
in that case both sides of inequality equal zero.
Remark 7.10. Let f be as in this chapter and f ∈ C (3) ([a, b]). Then from (7.39) we find Z b p q f (b) − f (a) a+b − f (s1 )ds1 − p−q qf q b−a a b−a 2 0 f (b) − f 0 (a) p2 (a2 + b2 + 4ab)q − − (a + b)p + 2(b − a) q 6 000 kf k∞,[a,b] ∗ p ≤ qA , (7.42) (b − a)3 q a.e. on X, A∗ is as in (7.40). We have proved that (via (7.22)) that Z b 1 f (b) − f (a) a+b f (s1 )ds1 − 1− Γf (µ1 , µ2 ) − b−a a b−a 2 0 Z f (b) − f 0 (a) (a2 + b2 + 4ab) −1 2 · − q p dλ − (a + b) + 2(b − a) 6 X Z 000 kf k∞,[a,b] p ≤ qA∗ dλ, (7.43) 3 (b − a) q X
22nd June 2009
62
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
where A∗ is as in (7.40). We need a generalization of Montgomery identity. Proposition 7.11. ([32]). Let f : [a, b] → R be differentiable on [a, b], f 0 : [a, b] → R is integrable on [a, b]. Let g : [a, b] → R be of bounded variation and such that g(a) 6= g(b). Let z ∈ [a, b]. We define g(t) − g(a) , a ≤ t ≤ z, g(b) − g(a) P (g(z), g(t)) := (7.44) g(t) − g(b) , z < t ≤ b. g(b) − g(a) Then it holds Z b Z b 1 f (t)dg(t) + P (g(z), g(t))f 0 (t)dt. (7.45) f (z) = (g(b) − g(a)) a a Remark 7.12. Let f ∈ C 1 ([a, b]), g ∈ C([a, b]) and of bounded variation, g(b). The rest are as in this chapter. Then by (7.44) and (7.45) we find Z b Z b p q p qf = f (t)dg(t) + q P g , g(t) f 0 (t)dt, q g(b) − g(a) a q a a.e. on X. Finally we obtain the representation formula Z b 1 Γf (µ1 , µ2 ) = f (t)dg(t) + R4 , g(b) − g(a) a where ! Z Z b p 0 R4 := q P g , g(t) f (t)dt dλ. q X a
g(a) 6= (7.46)
(7.47)
(7.48)
Based on the last we have Theorem 7.13. In this chapter’s setting, f ∈ C 1 ([a, b]) and g ∈ C([a, b]) of bounded variation, g(a) 6= g(b). Then Z b 1 f (t)dg(t) Γf (µ1 , µ2 ) − g(b) − g(a) a (Z ! ) Z b p 0 P g ≤ kf k∞,[a,b] q , g(t) dt dλ . (7.49) q X a Examples 7.14. (to Theorem 7.13). Here f ∈ C 1 ([a, b]) and as in this chapter. √ We take g1 (x) = ex , g2 (x) = ln x, g3 (x) = x, g4 (x) = xα , α > 1; x > 0. We elaborate on gi (t) − gi (a) , a ≤ t ≤ x, gi (b) − gi (a) P (gi (x), gi (t)) := i = 1, 2, 3, 4. (7.50) g (t) − g (b) i i , x < t ≤ b, gi (b) − gi (a)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Approximations via Representations of Csiszar’s f -Divergence
63
1 Clearly dg1 (x) = ex dx, dg2 (x) = x1 dx, dg3 (x) = 2√ dx, dg4 (x) = αxα−1 dx. x We have 1) t e − ea eb − ea , a ≤ t ≤ x, x t P (e , e ) = (7.51) t b e − e , x < t ≤ b, eb − e a
2)
P (ln x, ln t) =
3)
√
√
P ( x, t) =
4)
α
α
P (x , t ) =
Furthermore we derive 1)
|P (e
2)
p/q
t
, e )| =
ln t − ln a , ln b − ln a
a ≤ t ≤ x,
ln t − ln b , ln b − ln a
x < t ≤ b,
√ √ t− a √ √ , b− a
a ≤ t ≤ x, (7.53)
√ √ t− b √ , √ b− a α t − aα , bα − a α α α t −b , α b − aα t e − ea , eb − e a eb − e t , b e − ea
(7.52)
x < t ≤ b,
a ≤ t ≤ x,
(7.54)
x < t ≤ b.
a ≤ t ≤ p/q, (7.55) p < t ≤ b, q
ln t − ln a , ln b − ln a p P ln , ln t = q ln b − ln t , ln b − ln a
a≤t≤
p , q (7.56)
p < t ≤ b. q
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
64
3) √ √ t− a √ √ , b− a
r p √ P , t = √ √ q b− t √ , √ b− a
4)
tα − a α , bα − a α
Next we get 1) Z
b a
α P p , t α = α α q α b −t , bα − a α
|P (ep/q , et )|dt =
1 eb − e a
2ep/q − ea
a≤t≤
p , q (7.57)
p < t ≤ b, q
a≤t≤
p , q (7.58)
p < t ≤ b. q
p p − a + 1 + eb b − − 1 , (7.59) q q
2) Z
b a
−1 ( 2p p p P ln , ln t dt = ln b ln − 1 − a(ln a − 1) q a q q ) p p − (ln a) − a + (ln b) b − − b(ln b − 1) , (7.60) q q
3) 3/2 Z b r √ √ p (a3/2 + b3/2 ) √ 1 p 4p P , t dt = √ , − ( a + b) + √ q q 3 b − a 3 q 3/2 a (7.61) 4) ( α+1 Z b α p 2 p 1 α P dt = , t α α − aα q b (α + 1) q α+1 a ) α(aα+1 + bα+1 ) α α p − (a + b ) + . (7.62) q (α + 1) Clearly we find 1) Z b q |P (ep/q , et )|dt = a
n o 1 p/q a b 2qe − e (p − aq + q) + e (bq − p − q) , (7.63) eb − e a
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Approximations via Representations of Csiszar’s f -Divergence
2) Z
q
−1 ( p p P ln , ln t dt = ln b 2p ln − 1 q a q
b a
65
)
− a(ln a − 1)q − (ln a)(p − aq) + (ln b)(bq − p) − b(ln b − 1)q ,
(7.64)
3) q
Z
b a
4)
r √ √ p √ (a3/2 + b3/2 ) 4 −1/2 3/2 1 P , t dt = √ q q , p − ( a + b)p + √ q 3 b− a 3 (7.65) q
Z
( α 2 1 p α P , t dt = α (q −α pα+1 ) α α q b −a (α + 1) ) α (aα+1 + bα+1 )q . (7.66) − (aα + bα )p + α+1
b a
Finally we obtain 1) ! Z Z b p/q t q |P (e , e )|dt dλ = X
a
2) Z
X
Z 1 p/q a b 2 qe dλ − e (2 − a) + e (b − 2) , eb − e a X (7.67)
! p q P ln q , ln t dt dλ a −1 Z p b p ln dλ − ln(ab) + a + b − 2 , = ln 2 a q X Z
b
(7.68)
3) Z
4) Z
q X
q X
r p √ P , t dt q a Z √ √ 1 4 (a3/2 + b3/2 ) −1/2 3/2 = √ q p dλ − ( a + b) + , √ 3 b− a 3 X Z
Z
b
b a
(7.69)
α Z 2 1 P p , tα dt = q −α pα+1 dλ − (aα + bα ) qα bα − aα (α + 1) X α α+1 α+1 + (a +b ) . (7.70) α+1
Applying (7.49) we derive
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
66
1) Z b 1 t f (t)e dt Γf (µ1 , µ2 ) − b e − ea a Z 0 kf k∞,[a,b] p/q a b 2 qe dλ − e (2 − a) + e (b − 2) , ≤ (eb − ea ) X 2) −1 Z b b f (t) dt Γf (µ1 , µ2 ) − ln a t a −1 Z b p 0 ≤ kf k∞,[a,b] ln 2 p ln dλ − ln(ab) + (a + b − 2) , a q X
(7.71)
(7.72)
3) Z b f (t) 1 √ dt Γf (µ1 , µ2 ) − √ √ t 2( b − a) a Z 0 √ √ kf k∞,[a,b] 4 (a3/2 + b3/2 ) ≤ √ q −1/2 p3/2 dλ + − ( a + b) , √ 3 ( b − a) 3 X
(7.73)
4) (α > 1) Z b α α−1 f (t)t dt Γf (µ1 , µ2 ) − α (b − aα ) a ( Z kf 0 k∞,[a,b] 2 −α α+1 ≤ q p dλ (bα − aα ) (α + 1) X ) α α+1 α+1 α α (a +b ) − (a + b ) . + α+1
(7.74)
We need to mention Proposition 7.15. ([32]). Let f : [a, b] → R be twice differentiable on [a, b]. The second derivative f 00 : [a, b] → R is integrable on [a, b]. Let also g : [a, b] → R be of bounded variation, such that g(a) 6= g(b). Let z ∈ [a, b]. The kernel P is defined as in Proposition 7.11 (7.44). Then we get Z b 1 f (z) = f (t)dg(t) (g(b) − g(a)) a ! Z ! Z b b 1 0 + P (g(z), g(t))dt f (t1 )dg(t1 ) (g(b) − g(a)) a a Z bZ b + P (g(z), g(t))P (g(t), g(t1 ))f 00 (t1 )dt1 dt. (7.75) a
a
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Approximations via Representations of Csiszar’s f -Divergence
Book
67
Remark 7.16. Let f ∈ C (2) ([a, b]), g ∈ C([a, b]) and of bounded variation, g(a) 6= g(b). The rest are as in this chapter. We have the representation formula ! Z b Z b 1 1 0 Γf (µ1 , µ2 ) = f (t)dg(t) + f (t1 )dg(t1 ) (g(b) − g(a)) a (g(b) − g(a)) a ! ! Z Z b p , g(t) dt dλ + R5 , (7.76) · q P g q X a where ! Z Z bZ b p q P g R5 := , g(t) P (g(t), g(t1 ))f 00 (t1 )dt1 dt dλ. (7.77) q a a X Clearly |R5 | ≤ kf 00 k∞,[a,b] ! ! Z Z bZ b p · q (7.78) P g q , g(t) |P (g(t), g(t1 ))|dt1 dt dλ . X
a
a
Therefore we get Z b 1 Γf (µ1 , µ2 ) − f (t)dg(t) (g(b) − g(a)) a ! Z Z b 1 0 − f (t1 )dg(t1 ) (g(b) − g(a)) a X 00
q
Z
b a
! ! p P g , g(t) dt dλ q
≤ kf k∞,[a,b] ! ! Z Z bZ b p P g q · , g(t) |P (g(t), g(t1 ))|dt1 dt dλ . q X a a We further need to use
(7.79)
Theorem 7.17. ([32]). Let f : [a, b] → R be n-times differentiable on [a, b], n ∈ N. The nth derivative f (n) : [a, b] → R is integrable on [a, b]. Let z ∈ [a, b]. Define the kernel s−a , a ≤ s ≤ r, b−a (7.80) P (r, s) := s−b , r < s ≤ b, b−a where r, s ∈ [a, b]. Then Z b n−2 X f (k) (b) − f (k) (a) 1 f (z) = f (s1 )ds1 + b−a a b−a k=0 Z b Z b k Y · ··· P (z, s1 ) P (si , si+1 )ds1 ds2 · · · dsk+1 a a i=1 | {z } (k+1)th−integral
+
Z
b
a
···
Z
b
P (z, s1 )
a
n−1 Y i=1
P (si , si+1 )f (n) (sn )ds1 ds2 · · · dsn .
(7.81)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
68
Here and later we agree that
−1 Q
k=0
• = 0,
0 Q
i=1
• = 1.
Remark 7.18. Let f ∈ C (n) ([a, b]), n ∈ N and as in this chapter. Then by Theorem 1 ([32]) we obtain the representation formula Z b n−2 X f (k) (b) − f (k) (a) 1 f (s1 )ds1 + b−a a b−a k=0 ! ! Y Z Z b Z b k p , s1 P (si , si+1 )ds1 · · · dsk+1 dλ · q ··· P q X a a i=1
Γf (µ1 , µ2 ) =
+ R6 ,
(7.82)
where R6 :=
Z
Z
q X
b a
···
Z
b a
p P , s1 q
n−1 Y
P (si , si+1 )f
(n)
i=1
(sn )ds1 · · · dsn
!
dλ.
(7.83)
Clearly we have that
|R6 | ≤ kf
(n)
k∞,[a,b]
Z
q X
Z
b a
···
Z
We also need
b a
! n−1 Y P p , s 1 |P (si , si+1 )|ds1 · · · dsn dλ. q i=1
(7.84)
Theorem 7.19. ([32]). Let f : [a, b] → R be n-times differentiable on [a, b], n ∈ N. The nth derivative f (n) : [a, b] → R is integrable on [a, b]. Let g : [a, b] → R be of bounded variation and g(a) 6= g(b). Let z ∈ [a, b]. The kernel P is defined as in Proposition 1 ([32]). Then 1 f (z) = (g(b) − g(a)) Z b Z · ··· a
+
Z
b a
···
Z
b
! n−2 X Z b 1 (k+1) f (s1 )dg(s1 ) + f (s1 )dg(s1 ) (g(b) − g(a)) a a k=0 ! k b Y P (g(z), g(s1 )) P (g(si ), g(si+1 ))ds1 ds2 · · · dsk+1
Z a
b
P (g(z), g(s1 )) a
i=1 n−1 Y i=1
P (g(si ), g(si+1 ))f (n) (sn )ds1 · · · dsn . (7.85)
Remark 7.20. Let f ∈ C (n) ([a, b]), n ∈ N and g ∈ C([a, b]) and of bounded variation, g(a) 6= g(b). The rest are as in this chapter. Then by Theorem 2 ([32])
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Approximations via Representations of Csiszar’s f -Divergence
we obtain the representation formula Z b 1 f (s1 )dg(s1 ) Γf (µ1 , µ2 ) = (g(b) − g(a)) a ! n−2 X Z b 1 (k+1) + f (s1 )dg(s1 ) (g(b) − g(a)) a k=0 Z b Z Z b p ··· P g · q , g(s1 ) q a a X k Y
· where R7 =
Z
Z
q X
· Clearly we get |R7 | ≤ kf
i=1
P (g(si ), g(si+1 ))ds1 ds2 · · · dsk+1 )dλ
b a
n−1 Y
···
Z
b a
p , g(s1 ) P g q
P (g(si ), g(si+1 ))f
(n)
i=1
(n)
k∞,[a,b]
·
n−1 Y i=1
Z
q X
Z
b a
···
Z
b a
!
69
+ R7 , (7.86)
!
(sn )ds1 · · · dsn dλ.
P g p , g(s1 ) q !
|P (g(si ), g(si+1 ))|ds1 · · · dsn dλ.
(7.87)
(7.88)
Next we give some Lα results, α > 1. Remark 7.21. We repeat from Remark 7.5 the remainder(7.26), ! Z Z b p(x) , t f 0 (t)dt dλ(x), R1 := q(x) P∗ q(x) X a where f is as in this chapter and f ∈ C 1 ([a, b]), P ∗ is given by (7.24). Let α, β > 1: 1 1 α + β = 1. Then
Z
∗ p(x)
dλ(x) (7.89) |R1 | ≤ kf 0 kα,[a,b] q(x) P , ·
q(x) X β,[a,b] and
|R1 | ≤ kf 0 k1,[a,b]
Z
∗ p(x)
dλ(x). q(x) P , ·
q(x) X ∞,[a,b]
(7.90)
Similarly we obtain for R2 (see (7.30)) of Remark 7.6, here f ∈ C (2) ([a, b]) and as in this chapter that ! Z Z b ∗ p ∗ 00 P |R2 | ≤ kf kα,[a,b] q , t kP (t, ·)kβ,[a,b] dt dλ, (7.91) q X
a
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
70
and 00
|R2 | ≤ kf k1,[a,b]
Z
! Z b ∗ ∗ p P , t kP (t, ·)k∞,[a,b] dt dλ. q
q X
(7.92)
a
Similarly we get for R3 (see (7.38)) of Remark 7.8, here f ∈ C (3) ([a, b]) and as in this chapter, P as in (7.34), that ! Z Z b Z b p P |R3 | ≤ kf 000 kα,[a,b] q , s1 |P (s1 , s2 )| kP (s2 , ·)kβ,[a,b] ds1 ds2 dλ q X a a (7.93) and ! Z Z b Z b p 000 P |R3 | ≤ kf k1,[α,β] q , s1 |P (s1 , s2 )| kP (s2 , ·)k∞,[a,b] ds1 ds2 dλ. q X a a (7.94) Next let things be as in Remark 7.12 and R4 as in (7.48). Then
Z
p(x) 0
dλ(x) (7.95) |R4 | ≤ kf kα,[a,b] q(x) P g , g(·)
q(x) X β,[a,b]
and
|R4 | ≤ kf 0 k1,[a,b]
Z
p(x)
q(x) P g , g(·) dλ(x).
q(x) X ∞,[a,b]
Let things be as in Remark 7.16 and R5 as in (7.77). Then ! Z Z b p 00 P g , g(t) kP (g(t), g(·))kβ,[a,b] dt dλ |R5 | ≤ kf kα,[a,b] q q X
|R5 | ≤ kf k1,[a,b]
Z
q X
(7.97)
a
and
00
(7.96)
Z
b a
! P g p , g(t) kP (g(t), g(·))k∞,[a,b] dt dλ. (7.98) q
Let things be as in Remark 7.18 and R6 as in (7.83). Then Z Z b Z b P p , s1 |R6 | ≤ kf (n) kα,[a,b] q ··· q X a a ! ! n−2 Y · |P (si , si+1 )| kP (sn−1 , ·)kβ,[a,b] ds1 · · · dsn−1 dλ,
(7.99)
i=1
and
|R6 | ≤ kf ·
(n)
k1,[a,b]
n−2 Y i=1
Z
q X
Z
b a
!
···
Z
b a
P p , s 1 q
!
|P (si , si+1 )| kP (sn−1 , ·)k∞,[a,b] ds1 · · · dsn−1 dλ.
(7.100)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Approximations via Representations of Csiszar’s f -Divergence
71
Finally let things be as in Remark 7.20 and R7 as in (7.87). Then Z Z b Z b P g p(x) , g(s1 ) |R7 | ≤ kf (n) kα,[a,b] q(x) ··· q(x) X a a ! n−2 Y · |P (g(si ), g(si+1 ))| i=1
!
kP (g(sn−1 ), g(·))kβ,[a,b] ds1 · · · dsn−1 dλ(x), and |R7 | ≤ kf ·
(n)
k1,[a,b]
n−2 Y i=1
Z
Z
q(x) X
b a
|P (g(si ), g(si+1 ))|
··· !
Z
b a
(7.101)
P g p(x) , g(s1 ) q(x) !
kP (g(sn−1 ), g(·))k∞,[a,b] ds1 · · · dsn−1 dλ(x).
(7.102)
At the end we simplify and expand results of Remark 7.21 in the following Remark 7.22. Here 0 < a < b with 1 ∈ [a, b], and pq ∈ [a, b] a.e. on X. i) We simplify (7.90). We notice that p t − a, t ∈ a, q ∗ p P = , t , q p ,b b − t, t ∈ q here P ∗ is as in (7.24). Then
∗ p(x)
P ,·
q(x) Hence it holds
|R1 | ≤ kf 0 k1,[a,b]
Z
= max ∞,[a,b]
q(x) max X
p(x) p(x) − a, b − q(x) q(x)
p(x) p(x) − a, b − q(x) q(x)
(7.103)
.
(7.104)
dλ(x).
(7.105)
Here f is as in this chapter and f ∈ C 1 ([a, b]). ii) Next we simplify (7.96). Take here g strictly increasing and continuous over √ [a, b], e.g. g(x) = ex , ln x, x, xα with α > 1. Also f ∈ C 1 ([a, b]) and as in this chapter. Then clearly we derive Z kf 0 k1,[a,b] p(x) p(x) q(x) max g − g(a), g(b) − g dλ(x). |R4 | ≤ (g(b) − g(a)) X q(x) q(x) (7.106) In particular we obtain
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
72
1) |R4 | ≤ 2)
Z
kf 0 k1,[a,b] (eb − ea )
kf 0 k1,[a,b] |R4 | ≤ ln b − ln a
Z
X
p(x) p(x) q(x) max e q(x) − ea , eb − e q(x) dλ(x),
q(x) max ln X
p(x) q(x)
− ln a, ln b − ln
p(x) q(x)
(7.107)
dλ(x), (7.108)
3) kf 0 k1,[a,b] |R4 | ≤ √ √ b− a
Z
q(x) max X
s
p(x) √ √ − a, b − q(x)
s
p(x) q(x)
!
dλ(x),
and finally for α > 1 we get 4) α Z kf 0 k1,[a,b] p (x) pα (x) α α |R4 | ≤ α q(x) max −a ,b − α dλ(x). b − aα X q α (x) q (x)
(7.109)
(7.110)
iii) Finally we simplify (7.89). Let α, β > 1 such that α1 + β1 = 1. Also f ∈ C 1 ([a, b]) and as in this chapter. We obtain that p β (t − a) , t ∈ a, β q ∗ p P = , t (7.111) , q p (b − t)β , t ∈ ,b q
and
Z
b a
Thus
∗ p(x)
P
, ·
q(x)
β ∗ p dt = P , t q =
β,[a,b]
s β
p q
−a
β+1
+ b−
β+1
p β+1 q
.
(p(x) − aq(x))β+1 + (bq(x) − p(x))β+1 . (β + 1)q(x)β+1
(7.112)
(7.113)
Here notice q = 0 a.e. on X. So we need only work with q(x) > 0. So all in (iii) make sense. Therefore s
β+1 + (bq(x) − p(x))β+1
∗ p(x)
β (p(x) − aq(x))
, · , (7.114) q(x) P =
q(x) (β + 1)q(x) β,[a,b] a.e. on X. So we have established that s ! Z β+1 + (bq(x) − p(x))β+1 (p(x) − aq(x)) β |R1 | ≤ kf 0 kα,[a,b] dλ(x). (7.115) (β + 1)q(x) X
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Chapter 8
Sharp High Degree Estimation of Csiszar’s f -Divergence
In this chapter are presented various sharp and nearly optimal probabilistic inequalities giving the high order approximation of Csiszar’s f -divergence between two probability measures, which is the most essential and general tool for their comparison. The above are done through Taylor’s formula, generalized Taylor-Widder’s formula, an alternative recent expansion formula. Based on these we give many representation formulae of Csiszar’s distance, then we estimate in all directions their remainders by using either the norms approach or the modulus of continuity way. Most of the last probabilistic estimates are sharp or nearly sharp, attained by basic simple functions. This treatment relies on [41]. 8.1
Background
Throughout this chapter we use the following. Let f be a convex function from (0, +∞) into R which is strictly convex at 1 with f (1) = 0. Let (X, A, λ) be a measure space, where λ is a finite or a σ-finite measure on (X, A). And let µ1 , µ2 be two probability measures on (X, A) such that µ1 λ, µ2 λ (absolutely continuous), e.g. λ = µ1 + µ2 . dµ2 1 Denote by p = dµ dλ , q = dλ the (densities) Radon-Nikodym derivatives of µ1 , µ2 with respect to λ. Here we suppose that p 0 < a ≤ ≤ b, a.e. on X and a ≤ 1 ≤ b. q The quantity Z p(x) dλ(x), (8.1) Γf (µ1 , µ2 ) = q(x)f q(x) X
was introduced by I. Csiszar in 1967, see [140], and is called f -divergence of the probability measures µ1 and µ2 . By Lemma 1.1 of [140], the integral (8.1) is well-defined and Γf (µ1 , µ2 ) ≥ 0 with equality only when µ1 = µ2 . Furthermore Γf (µ1 , µ2 ) does not depend on the choice of λ.The concept of f -divergence was introduced first in [139] as a generalization of Kullback’s information for discrimination or I-divergence “(generalized entropy )” 73
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
74
[244], [243] and of R´enyi’s information gain (I-divergence of order α) [306]. In fact the I-divergence of order 1 equals Γu log2 u (µ1 , µ2 ). The choice f (u) = (u − 1)2 produces again a known measure of difference of distributions that is called χ2 -divergence. Of course the total variation distance Z |µ1 − µ2 | = |p(x) − q(x)| dλ(x) X
is equal to Γ|u−1| (µ1 , µ2 ). Here by assuming f (1) = 0 we can consider Γf (µ1 , µ2 ), the f -divergence as a measure of the difference between the probability measures µ1 , µ2 . The f -divergence is in general asymmetric in µ1 and µ2 . But since f is convex and strictly convex at 1 so is 1 (8.2) f ∗ (u) = uf u and as in [140] we find Γf (µ2 , µ1 ) = Γf ∗ (µ1 , µ2 ).
(8.3)
In Information Theory and Statistics many other divergences are used which are special cases of the above general Csiszar f -divergence, e.g. Hellinger distance D H , α-divergence Dα , Bhattacharyya distance DB , Harmonic distance DHa , Jeffrey’s distance DJ , triangular discrimination D∆ , for all these see, e.g. [102], [152]. The problem of finding and estimating the proper distance (or difference or discrimination ) of two probability distributions is one of the major ones in Probability Theory. The above f -divergence measures in their various forms have been also applied to Anthropology, Genetics, Finance, Economics, Political Science, Biology, Approximation of Probability distributions, Signal Processing and Pattern Recognition. A great motivation for this chapter has been the very important monograph on the topic by S. Dragomir [152]. 8.2
Results Based on Taylor’s Formula
We give the following Theorem 8.1. Let 0 < a < 1 < b, f as in this chapter and f ∈ C n ([a, b]), n ≥ 1 with |f (n) (t) − f (n) (1)| be a convex function in t. Let 0 < h < min(1 − a, b − 1) be fixed. Then Z Z n X ω1 (f (n) , h) |f (k) (1)| 1−k k + Γf (µ1 , µ2 ) ≤ q (p − q) dλ q −n |p − q|n+1 dλ. k! (n + 1)!h X X k=2 (8.4)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Sharp High Degree Estimation of Csiszar’s f -Divergence
75
Here ω1 is the usual first modulus of continuity, ω1 (f (n) , h) := sup{|f (n)(x) − f (n) (y)|; x, y ∈ [a, b], |x − y| ≤ h}. If f (k) (1) = 0, k = 2, . . . , n, then Z ω1 (f (n) , h) q −n |p − q|n+1 dλ. (8.5) Γf (µ1 , µ2 ) ≤ (n + 1)!h X Inequalities (8.4) and (8.5) when n is even are attained by |t − 1|n+1 , f˜(t) := (n + 1)!
a ≤ t ≤ b.
(8.6)
n ∈ N, α > 1
(8.7)
Example 8.2. The function g(x) := |x − 1|n+α ,
is convex and strictly convex at 1, also g (k) (1) = 0, k = 0, 1, 2, . . . , n. Furthermore (g(x))(n) = (n + α)(n + α − 1) · · · (α + 1)|x − 1|α (sign(x − 1))n with |g(x)(n) | = (n + α)(n + α − 1) · · · (α + 1)|x − 1|α being convex. That is g fulfills all the assumptions of Theorem 8.1. Proof. Here we have (f (1) = 0) f (t) =
n X f (k) (1) k=1
where It :=
Z t Z 1
t1 1
···
Z
tn−1 1
k!
(t − 1)k + It ,
(f (n) (tn ) − f (n) (1))dtn · · · dt1
(8.8)
.
(8.9)
By Lemma 8.1.1 of [20], p. 243 we have that |f (n) (t) − f (n) (1)| ≤
ω1 (f (n) , h) |t − 1|, h
(8.10)
and |It | ≤
ω1 (f (n) , h) |t − 1|n+1 . h (n + 1)!
We observe by (8.8) that n X f (k) (1) 1−k p qf = f 0 (1)(p − q) + q (p − q)k + qIp/q , q k!
(8.11)
(8.12)
k=2
true a.e. on X. Here q 6= 0 a.e. on X. Integrating (8.12) against λ and using (8.11) we derive (8.4). Notice that for n even f˜(n) (t) = |t − 1| and f˜(k) (1) = 0 for all k = 0, 1, . . . , n, with kf˜(n+1) k∞ = 1. Furthermore, ω1 (f˜(n) , h) = h.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
76
In both cases in (8.4) and (8.5) their two sides equal Z 1 q −n |p − q|n+1 dλ, (n + 1)! X proving attainability and sharpness of (8.4) and (8.5).
(8.13)
We continue with the general Theorem 8.3. Let 0 < a ≤ 1 ≤ b, f as in this chapter and f ∈ C n ([a, b]), n ≥ 1. Suppose that ω1 (f (n) , δ) ≤ w, where 0 < δ ≤ b − a, w > 0. Let x ∈ R and denote by Z |x| t (|x| − t)n−1 dt, (8.14) φn (x) := δ (n − 1)! 0
where d·e is the ceiling of the number. Then Z Z n X |f (k) (1)| p−q 1−k k Γf (µ1 , µ2 ) ≤ (p − q) dλ + w qφn dλ. q k! q X X
(8.15)
k=2
Inequality (8.15) is sharp, namely it is attained by the function f˜n (t) := wφn (t − 1),
a ≤ t ≤ b,
(8.16)
when n is even. Proof. One has that f (x) =
n X f (k) (1)
k=1
for all x ∈ [a, b], where
Z
x
k!
(x − 1)k + R(x),
(8.17)
(x − t)n−1 dt. (n − 1)!
(8.18)
|R(x)| ≤ ω1 (f (n) , δ)φn (x − 1) ≤ wφn (x − 1).
(8.19)
R(x) :=
1
(f (n) (t) − f (n) (1))
From p. 3, see (8) of [29] we obtain
By (8.17) we get qf
X k n p f (k) (1) p p = q − 1 + qR , q k! q q
(8.20)
k=1
a.e. on X. Furthermore we find Z Z n X f (k) (1) p Γf (µ1 , µ2 ) = q 1−k (p − q)k dλ + qR dλ. k! q X X
(8.21)
k=2
Using now (8.19) and taking absolute value over (8.21) we derive (8.15). Next we prove the sharpness of (8.15). According to Remark 7.1.3 of [20], pp. 210–211 we have that Z x φn (x) = φn−1 (t)dt, x ∈ R+ , n ≥ 1, (8.22) 0
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Sharp High Degree Estimation of Csiszar’s f -Divergence
where
Clearly
|t| , φ0 (t) := δ φn(k) (x) = (sign(x))k φn−k (x),
t ∈ R. k = 1, 2, . . . , n, x ∈ R.
Book
77
(8.23)
(8.24)
That is f˜n(k) (x) = (sign(x − 1))k f˜n−k (x),
x ∈ R, k = 1, . . . , n.
(8.25)
Furthermore f˜n(k) (1) = 0,
k = 0, 1, . . . , n.
Since the function φn is even and convex over R, then we have that f˜n is convex over R, n ≥ 1. Also since φn is strictly increasing over R+ , then f˜n is strictly increasing over [1, ∞) and strictly decreasing over (−∞, 1]. Thus, f˜n is strictly convex at 1, n ≥ 1. Here since n is even |x − 1| (n) ˜ fn (x) = w δ and by Lemma 7.1.1, see p. 208 of [20] we find that ω1 (f˜n(n) , δ) ≤ w.
So f˜n fulfills basically all the assumptions of Theorem 8.3. Clearly then both sides of (8.15) equal to Z p−q w qφn dλ, q X
establishing that (8.15)lis attained. m can be arbitrarily close approximated by a sequence Next we see that w |t−1| δ of functions that fulfill the assumptions of Theorem 8.3. More precisely l continuous m |t−1| is a step function over R which takes the values j = 1, 2, . . ., respectively, δ over the intervals [1 − (j + 1)δ, 1 − jδ) and (1 + (j − 1)δ, 1 + jδ] with upward jumps of size 1, therefore |t − 1| f˜n(n) (t) = w δ
has finitely many discontinuities over [a, b] and is approximated by continuous functions as follows. Here for N ∈ N we define the continuous functions over R, N w|t| N 2 + kw 1 − , if kδ ≤ |t| ≤ k + δ, 2 N 2δ f0N (t) := (8.26) 2 (k + 1)w, if k + δ < |t| ≤ (k + 1)δ, N
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
78
where k ∈ Z+ . Clearly f0N (|t|) > 0 with f0N (0) = 0 and f0N is increasing over R+ and decreasing over R− . Also we define Z tn−1 Z |t| Z t1 f0N (tn )dtn · · · dt1 , (8.27) ··· fnN (t) := w 0
0
0
n
for t ∈ R. Clearly fnN ∈ C (R), is even and convex over R and is strictly increasing on R+ . According to [20], pp. 218–219 we obtain |t − 1| lim f0N (|t − 1|) = w , for t ∈ [a, b], N ∈ N, (8.28) N →+∞ δ with ω1 (f0N (| · −1|), δ) = ω1 (f0N (| · |), δ) ≤ w.
(8.29)
Furthermore it invalid that lim fnN (|t − 1|) = f˜n (t),
N →+∞
for t > 0.
(8.30)
Let g(x) := fnN (x − 1), x ∈ R, then g (k) (x) = (sign(x − 1))k f(n−k)N (x − 1), x ∈ R, k = 1, 2, . . . , n, g (k) (1) = 0 for k = 0, 1, . . . , n, and for all x ∈ R it holds g (n) (x) = f0N (x − 1) since n is even. Furthermore g ∈ C n (R), g convex on R, g strictly increasing on [1, +∞) and g strictly decreasing on (−∞, 1], therefore g is strictly convex at 1. That is g with all the above properties and (8.29), is consistent with the assumptions of Theorem 8.3. That is, f˜n (t) is arbitrarily close approximated consistently with the assumptions of Theorem 8.3. Next we have Corollary 8.4. (to Theorem 8.3). It holds (0 < δ ≤ b − a) Z n X |f (k) (1)| 1−k k Γf (µ1 , µ2 ) ≤ q (p − q) dλ k! X k=2 ( Z Z 1 1 −n n+1 (n) + ω1 (f , δ) q |p − q| dλ + q 1−n |p − q|n dλ (n + 1)!δ X 2n! X ) Z δ 2−n n−1 + q |p − q| dλ , (8.31) 8(n − 1)! X and
Z n X |f (k) (1)| 1−k k Γf (µ1 , µ2 ) ≤ q (p − q) dλ (8.32) k! X k=2 ( Z ) Z 1 1 (n) 1−n n −n n+1 + ω1 (f , δ) q |p − q| dλ + q |p − q| dλ . n! X n!(n + 1)δ X
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Sharp High Degree Estimation of Csiszar’s f -Divergence
In particular we have
(
1 Γf (µ1 , µ2 ) ≤ ω1 (f 0 , δ) 2δ and Γf (µ1 , µ2 ) ≤ ω1 (f 0 , δ)
Z
1 q −1 (p − q)2 dλ + 2 X
Z
X
|p − q|dλ +
1 2δ
Z
X
Z
) δ |p − q|dλ + , 8 X
q −1 (p − q)2 dλ .
Proof. From [20], p. 210, inequality (7.1.18) we have |x|n+1 |x|n δ|x|n−1 φn (x) ≤ , x ∈ R, + + (n + 1)!δ 2n! 8(n − 1)! with equality only at x = 0. We obtain that (see (8.18) and (8.19)) |x − 1|n+1 |x − 1|n δ|x − 1|n−1 (n) + + ≤ w(· · · ), |R(x)| ≤ ω1 (f , δ) (n + 1)!δ 2n! 8(n − 1)! with equality at x = 1. Also from [20], p. 217, inequality (7.2.9) we get |x − 1|n |x − 1| |R(x)| ≤ ω1 (f (n) , δ) 1+ , n! (n + 1)δ ≤ w(· · · ),
Book
79
(8.33)
(8.34)
(8.35)
(8.36)
(8.37)
with equality at x = 1. Hence by (8.17), (8.18) and (8.36) we find Z Z −n n X q |p − q|n+1 |f (k) (1)| 1−k k (n) Γf (µ1 , µ2 ) ≤ q (p − q) dλ + ω (f , δ) 1 k! (n + 1)!δ X X k=2 ! q 1−n |p − q|n δq 2−n |p − q|n−1 + + dλ , (8.38) 2n! 8(n − 1)!
etc. Also by (8.17), (8.18) and (8.37) we derive Z n X |f (k) (1)| 1−k k Γf (µ1 , µ2 ) ≤ q (p − q) dλ k! X k=2 Z q −n |p − q| |p − q|n + ω1 (f (n) , δ) q 1−n + dλ , (8.39) n! (n + 1)δ X etc. We give Corollary 8.5. (to Theorem 8.3). Suppose here that f (n) is of Lipschitz type of order α, 0 < α ≤ 1, i.e. ω1 (f (n) , δ) ≤ Kδ α ,
K > 0,
(8.40)
for any 0 < δ ≤ b − a. Then Z Z n X |f (k) (1)| K 1−k k + Γf (µ1 , µ2 ) ≤ q (p − q) dλ q 1−n−α |p − q|n+α dλ. n Q k! X X k=2 (α + i) i=1
(8.41)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
80
When n is even (8.41) is attained by f ∗ (x) = c|x − 1|n+α , where c := K/ i) > 0.
n Q
(α +
i=1
Proof. We have again that
f (x) =
n X f (k) (1)
k!
k=1
where R(x) :=
Z
x 1
Z
x1
···
1
From [29], p. 6 see (29)∗ , we get
Z
xn−1 1
(x − 1)k + R(x),
(f (n) (xn ) − f (n) (1))dxn · · · dx1 .
|x − 1|n+α . |R(x)| ≤ K · Q n (α + i)
(8.42)
(8.43)
i=1
n+1
(n+1)
(E.g. if f ∈ C ([a, b]), then K = kf k∞ and α = 1). Then n+α p K p Kq q − 1 = Q q 1−n−α |p − q|n+α , q R ≤ n n Q q (α + i) (α + i) i=1
a.e. on X, and Z
(8.44)
i=1
Z p K q R q 1−n−α |p − q|n+α dλ. dλ ≤ Q n q X X (α + i)
(8.45)
i=1
Thus we have established (8.41). Clearly (8.41) is attained by f ∗ which is acceptable to Theorem 8.3 requirements. Notice that f ∗(i) (1) = 0, i = 0, 1, . . . , n and f ∗(n) (x) = K|x − 1|α fulfills (8.40) in here by (105) of [29], p. 28. See also from [29], p. 28 at the bottom that f ∗ (x) = |R(x)|, etc. Corollary 8.6. (to Theorem 8.3). Suppose that Z b−a≥ q −1 p2 dλ − 1 > 0.
(8.46)
X
Then
Z Z 1 Γf (µ1 , µ2 ) ≤ ω1 f 0 , . q −1 p2 dλ − 1 |p − q|dλ + 2 X X
Proof. From (8.34) by choosing Z Z δ := q −1 (p − q)2 dλ = q −1 p2 dλ − 1. X
X
(8.47)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Sharp High Degree Estimation of Csiszar’s f -Divergence
81
Corollary 8.7. (to Theorem 8.3). Suppose (1.46), that is Z b−a≥ q −1 p2 dλ − 1 > 0, X
|p − q| ≤ 1 a.e. on X,
(8.48)
λ(X) < ∞.
(8.49)
and
Then Γf (µ1 , µ2 ) ≤
1 λ(X) + 2
0
ω1 f ,
Z
q
−1 2
X
p dλ − 1
.
(8.50)
Proof. From (8.47) and (8.48).
Note. In our setting observe that always Z q −1 p2 dλ ≥ 1,
(8.51)
X
and of course
a≤ We present
Z
X
q −1 p2 dλ ≤ b.
(8.52)
Corollary 8.8. (to Theorem 8.3). Suppose that Z |p − q|dλ > 0.
(8.53)
X
Then
Z
Z
b−a Γf (µ1 , µ2 ) ≤ ω1 f , |p − q|dλ |p − q|dλ + 2 X X Z 3 ≤ (b − a)ω1 f 0 , |p − q|dλ . 2 X Proof. Notice that q Hence
−1
2
(p − q) = q
−1
Z
0
(8.54)
p |p − q| = |p − q| − 1 ≤ (b − a)|p − q|, a.e. on X. q
X
2
q −1 (p − q)2 dλ ≤ (b − a)
Z
X
|p − q|dλ.
We use (8.34), (8.53), (8.55) and we choose Z δ := |p − q|dλ ≤ b − a,
(8.55)
(8.56)
X
etc.
Notice that (8.54) is good only for small b − a > 0.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
82
Next we derive Corollary 8.9. (to Theorem 8.3). Assume that r > 0 and (8.46)- (8.49), respectively, Z b−a≥r q −1 p2 dλ − 1 > 0, X
|p − q| ≤ 1 a.e. on X, and λ(X) < ∞. Then Γf (µ1 , µ2 ) ≤
λ(X) +
1 2r
Z q −1 p2 dλ − 1 . ω1 f 0 , r
(8.57)
X
Proof. Here we choose in (8.34) that Z δ := r q −1 (p − q)2 dλ,
(8.58)
Remark 8.10. (on Theorem 8.1). From (8.5) for n = 1 we obtain Z ω1 (f 0 , h) Γf (µ1 , µ2 ) ≤ q −1 (p − q)2 dλ, 2h X
(8.59)
X
etc.
which is attained by
(t−1)2 . 2
0 < h := we derive the inequality
Z
Assuming that X
q −1 (p − q)2 dλ < min(1 − a, b − 1),
Z 1 0 −1 2 Γf (µ1 , µ2 ) ≤ ω1 f , q p dλ − 1 . 2 X
(8.60)
(8.61)
The counterpart of the above is
Proposition 8.11. Let f and the rest as in this chapter, also assume f ∈ C([a, b]). i) Suppose (8.53), Z |p − q|dλ > 0. X
Then
Γf (µ1 , µ2 ) ≤ 2ω1 f, ii) Let r > 0 and b−a≥r
Z
X
Z
X
|p − q|dλ .
|p − q|dλ > 0.
(8.62)
(8.63)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Sharp High Degree Estimation of Csiszar’s f -Divergence
Then Γf (µ1 , µ2 ) ≤
1+
1 r
Z · ω1 f, r |p − q|dλ .
83
(8.64)
X
Proof. We observe that Z Z p p qf Γf (µ1 , µ2 ) = dλ = qf dλ − f (1) q q X X Z Z p p = q f − f (1) dλ ≤ q f − f (1) dλ q q X X ' & p Z − 1 q ≤ (by Lemma 7.1.1 of [20], p. 208) qω1 (f, h) dλ h X ! ! p Z Z − 1 1 q dλ = ω1 (f, h) 1 + |p − q|dλ . ≤ ω1 (f, h) q 1+ h h X X That is, we obtain
Setting
Z 1 Γf (µ1 , µ2 ) ≤ ω1 (f, h) 1 + |p − q|dλ . h X h :=
we find proving (8.62). By setting
X
|p − q|dλ
(8.66)
Γf (µ1 , µ2 ) ≤ 2ω1 (f, h),
h := r we derive
Z
(8.65)
Z
X
|p − q|dλ,
r>0
1 Γf (µ1 , µ2 ) ≤ ω1 (f, h) 1 + . r
(8.67)
(8.68)
Also we give Proposition 8.12. Let f as in this setting and f is a Lipschitz function of order α, 0 < α ≤ 1, i.e. there exists K > 0 such that Then
|f (x) − f (y)| ≤ K|x − y|α , Γf (µ1 , µ2 ) ≤ K
Z
X
all x, y ∈ [a, b].
q 1−α |p − q|α dλ.
(8.69) (8.70)
Proof. As in the proof of Proposition 8.11 we have α Z Z Z p p Γf (µ1 , µ2 ) ≤ q f − f (1) dλ ≤ qK − 1 dλ = K q 1−α |p − q|α dλ. q q X X X
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
PROBABILISTIC INEQUALITIES
84
8.3
Book
Results Based on Generalized Taylor–Widder Formula
(see also [339], [29]) Let f , u0 , u1 , . . . , un ∈ C n+1 ([a, b]), n ∈ Z+ , and let the Wronskians Wi (x) := W [u0 (x), u1 (x), . . . , ui (x)] : u0 (x) u1 (x) · · · ui (x) u0 (x) u0 (x) · · · u0 (x) 0 1 i , = . .. (i) (i) u(i) (x) u (x) · · · u (x) 0 1 i
i = 0, 1, . . . , n,
(8.71)
and suppose that Wi (x) are positive over [a, b]. Clearly the functions W1 (x) ,... φ0 (x) := W0 (x) := u0 (x), φ1 (x) := (W0 (x))2 Wi (x)Wi−2 (x) φi (x) := , i = 2, 3, . . . , n, (8.72) (Wi−1 (x))2 are positive everywhere on [a, b]. Consider the linear differential operator of order i≥0 W [u0 (x), u1 (x), . . . , ui−1 (x), f (x)] (8.73) Li f (x) := Wi−1 (x) for i = 1, . . . , n + 1,where L0 f (x) := f (x) all x ∈ [a, b]. Here W [u0 (x), u1 (x), . . . , ui−1 (x), f (x)] denotes the Wronskian of u0 , u1 , . . . , ui−1 , f . Notice that for i = 1, . . . , n + 1, we obtain d 1 d 1 d d 1 d f (x) Li f (x) = φ0 (x)φ1 (x) · · · φi−1 (x) ··· . dx φi−1 (x) dx φi−2 (x) dx dx φ1 (x) dx φ0 (x) (8.74) Consider also the functions u0 (t) u1 (t) ··· ui (t) u0 (t) u01 (t) ··· u0i (t) 0 1 .. gi (x, t) := (8.75) · . Wi (x) (i−1) (i−1) (i−1) u0 (t) u1 (t) · · · ui (t) u0 (x) u1 (x) ··· ui (x)
for i = 1, 2, . . . , n, where g0 (x, t) := uu00(x) (t) all x, t ∈ [a, b]. Notice that gi (x, t), as a function of x, is a linear combination of u0 (x), u1 (x), . . . , ui (x) and, furthermore, it holds Z x Z x1 φ0 (x) gi (x, t) = φ1 (x1 ) φ0 (t) · · · φi (t) t Z xi−2 Z xi−1t ··· φi−1 (xi−1 ) φi (xi )dxi dxi−1 · · · dx1 t t Z x 1 = φ0 (s) · · · φi (s)gi−1 (x, s)ds (8.76) φ0 (t) · · · φi (t) t
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Sharp High Degree Estimation of Csiszar’s f -Divergence
all i = 1, 2, . . . , n. Put Nn (x, t) :=
Z
Book
85
x
gn (x, s)ds.
(8.77)
t
Example 8.13. Let ui (x) = xi , i = 0, 1, . . . , n, be defined on [a, b]. Then Li f (t) = i f (i) (t), i = 1, . . . , n + 1 and gi (x, t) = (x−t) for all x, t ∈ [a, b] and i = 0, 1, . . . , n. i! From [339, Theorem II, p. 138], we obtain Z x n X f (x) = Li f (t) · gi (x, t) + gn (x, s) · Ln+1 f (s)ds (8.78) t
i=0
all x, t ∈ [a, b], n ∈ Z+ . This is the Taylor–Widder formula. Thus we have f (x) =
n X i=0
where Rn∗ (x, t) :=
Z
Li f (t)gi (x, t) + Ln+1 f (t) · Nn (x, t) + Rn∗ (x, t),
(8.79)
x
gn (x, s)(Ln+1 f (s) − Ln+1 f (t))ds,
t
n ∈ Z+ .
(8.80)
Here we use a lot the above concepts. From the above definitions, we observe the following gn (x, t) > 0, x > t, n ≥ 1, gn (x, t) < 0, x < t, n odd, (8.81) g (x, t) > 0, x < t, n even, n gn (t, t) = 0, n ≥ 1, g0 (x, t) > 0 for all x, t ∈ [a, b].
Since f (1) = 0 we have f (x) =
n X i=1
where R∗n (x, 1) := and
Z
Li f (1)gi (x, 1) + Ln+1 f (1)Nn (x, 1) + R∗n (x, 1),
x 1
gn (x, s)(Ln+1 f (s) − Ln+1 f (1))ds,
Nn (x, 1) := Let
(8.82)
Z
n ∈ Z+ ,
(8.83)
x
gn (x, s)ds, 1
ω1 (Ln+1 f, h) ≤ w,
x ∈ [a, b].
0 < h ≤ b − a,
w > 0.
Then by Theorem 23, p. 18 of [29] we get that |x − 1| |x − 1| |Nn (x, 1)| ≤ w 1 + |Nn (x, 1)|. |R∗n (x, 1)| ≤ w h h
(8.84)
(8.85)
(8.86)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
86
Working in (8.82) we observe that X n p p p p ∗ qf = Li f (1)qgi , 1 + Ln+1 f (1)qNn , 1 + qRn , 1 , a.e. on X. q q q q i=1 (8.87) That is we have established the representation formula Z Z n X p p Γf (µ1 , µ2 ) = Li f (1) qgi , 1 dλ + Ln+1 f (1) qNn , 1 dλ + T1 , q q X X i=1 (8.88) where Z p ∗ , 1 dλ. (8.89) T1 := qRn q X From (8.86) we get ' & p Z Z − 1 ∗ p q Nn p , 1 dλ q |T1 | ≤ q Rn , 1 dλ ≤ w q h q X X Z |p − q| p ≤w q+ , 1 dλ. Nn h q X
(8.90)
Based on the above we have now established
Theorem 8.14. It holds Γf (µ1 , µ2 ) ≤
Z Z p p , 1 dλ + |Ln+1 f (1)| qNn , 1 dλ |Li f (1)| qgi q q X X i=1 Z p |p − q| Nn (8.91) +w q+ , 1 dλ. h q X
n X
Next let
ω1 (Ln+1 f, δ) ≤ Aδ α ,
(8.92)
0 < δ ≤ b − a, A > 0, 0 ≤ α ≤ 1. Then from Theorem 25, p. 19 of [29] we obtain |R∗n (x, t)| ≤ A|x − t|α |Nn (x, t)|, for all x, t ∈ [a, b] and n ≥ 0. Hence p p q R∗n , 1 ≤ Aq 1−α |p − q|α Nn , 1 , q q
a.e. on X, and Z Z p p |T1 | ≤ q R∗n , 1 dλ ≤ A , 1 dλ. q 1−α |p − q|α Nn q q X
Thus we have proved
X
(8.93)
(8.94)
(8.95)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Sharp High Degree Estimation of Csiszar’s f -Divergence
Book
87
Theorem 8.15.. It holds Z Z n X p p , 1 dλ + |Ln+1 f (1)| qNn , 1 dλ Γf (µ1 , µ2 ) ≤ |Li f (1)| qgi q q X X i=1 Z p +A q 1−α |p − q|α Nn , 1 dλ. (8.96) q X From Theorem 26, p. 19 of [29] we have: let 1 ∈ (a, b) and choose 0 < h ≤ min(1 − a, b − 1), assuming that |Ln+1 f (x) − Ln+1 f (1)| is convex in x ∈ [a, b] and calling w := ω1 (Ln+1 f, h) we get w |R∗n (x, 1)| ≤ |x − 1| |Nn (x, 1)|, (8.97) h for all x ∈ [a, b] and n ≥ 0. Therefore w p p (8.98) , 1 ≤ |p − q| Nn , 1 , a.e. on X. q R∗n q h q So that
w |T1 | ≤ h
We have established
Z
X
p , 1 dλ. |p − q| Nn q
(8.99)
Theorem 8.16. It holds i) Z Z p p Γf (µ1 , µ2 ) ≤ |Li f (1)| qgi , 1 dλ + |Ln+1 f (1)| qNn , 1 dλ q q X X i=1 Z w p (8.100) + |p − q| Nn , 1 dλ, n ≥ 0. h X q n X
ii) Choosing and supposing Z p 0 < h := |p − q| Nn , 1 dλ ≤ min(1 − a, b − 1), q X
(8.101)
we derive
Z Z p p Γf (µ1 , µ2 ) ≤ |Li f (1)| qgi , 1 dλ + |Ln+1 f (1)| qNn , 1 dλ q q X X i=1 Z p (8.102) + ω1 Ln+1 f, |p − q| Nn , 1 dλ . q X n X
Next by Lemmas 11.1.5, 11.1.6 of [20], p. 345 we have: Let a < 1 < b, u0 (x) := c > 0, u1 (x) concave for x ≤ 1, and u1 (x) convex for x ≥ 1. Let ˜ n (x, 1) := |Gn (x, 1)|, G
(8.103)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
88
where Gn (x, 1) :=
Z
x
gn (x, s) 1
|s − 1| ds, h
all x ∈ [a, b], 0 < h ≤ b − a, n ≥ 0. Then ( ) ˜ n (b, 1) G ˜ n (a, 1) G ˜ Gn (x, 1) ≤ max , |x − 1| =: θ|x − 1|, b−1 1−a
(8.104)
(8.105)
all x ∈ [a, b], n ≥ 1. But from Theorem 21, p. 16 of [29] we get ˜ n (x, 1), |R∗n (x, 1)| ≤ wG
(8.106)
all x ∈ [a, b], n ∈ Z+ , with ω1 (Ln+1 f, h) ≤ w, 0 < h ≤ b − a, w > 0. Consequently we obtain |R∗n (x, 1)| ≤ wθ|x − 1|,
(8.107)
all x ∈ [a, b], n ≥ 1. Furthermore it holds Z Z p |T1 | ≤ q R∗n , 1 dλ ≤ wθ |p − q|dλ, q X
n ≥ 1.
X
(8.108)
We have established
Theorem 8.17. It holds Z n X p , 1 dλ Γf (µ1 , µ2 ) ≤ |Li f (1)| qgi q X i=1 Z Z p + |Ln+1 f (1)| qN , 1 dλ + wθ |p − q|dλ, n ≥ 1, q X X where
θ := max
(
˜ n (b, 1) G ˜ n (a, 1) G , b−1 1−a
)
,
(8.109)
(8.110)
with 1 ∈ (a, b). It follows
Corollary 8.18. (to Theorem 8.17). It holds Z Z p p Γf (µ1 , µ2 ) ≤ |L1 f (1)| qg1 , 1 dλ + |L2 f (1)| qN1 , 1 dλ q q X X Z Z + ω1 L2 f, |p − q|dλ · |p − q|dλ X )!X ( ˜ 1 (a, 1) ˜ 1 (b, 1) G G , , (8.111) · max b−1 1−a
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Sharp High Degree Estimation of Csiszar’s f -Divergence
under the assumption that
Z
89
|p − q|dλ > 0.
X
From (8.78) we have f (x) =
n X i=1
where
Rw (x) := all x ∈ [a, b], n ∈ Z+ . Set also
Li f (1)gi (x, 1) + Rw (x),
(8.112)
Z
(8.113)
x
gn (x, s)Ln+1 f (s)ds, 1
Nn∗ (x, 1) Then we observe that
:=
Z
x 1
|gn (x, s)|ds.
(8.114)
|Rw (x)| ≤ kLn+1 f k∞,[a,b] |Nn∗ (x, 1)|. Hence
and
Z
(8.115)
∗ p p q Rw , 1 ≤ kLn+1 f k∞,[a,b] q Nn q q
X
Z ∗ p p dλ. dλ ≤ kL f k q N , 1 q Rw n+1 ∞,[a,b] n q q
(8.116)
X
We have proved
Theorem 8.19. It holds Γf (µ1 , µ2 ) ≤
Z p |Li f (1)| qgi , 1 dλ q X i=1 Z p + kLn+1 f k∞,[a,b] q Nn∗ , 1 dλ. q X
n X
Next from (8.113) we obtain Z x |Rw (x)| ≤ |(Ln+1 f )(s)|ds sup |gn (x, ·)|. 1
(8.117)
(8.118)
s∈[a,b]
Furthermore it is true that
|Rw (x)| ≤ kLn+1 f k1,[a,b] kgn (x, ·)k∞,[a,b] . Thus
p p q Rw ≤ kLn+1 f k1,[a,b] q gn ,·
q q
(8.119)
, ∞,[a,b]
(8.120)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
90
and
Z
Z
p p dλ ≤ kLn+1 f k1,[a,b] q gn dλ. ,· q Rw
q q X X ∞,[a,b]
(8.121)
We have established
Theorem 8.20. It holds
Z p Γf (µ1 , µ2 ) ≤ |Li f (1)| qgi , 1 dλ q X i=1
Z
p
dλ. + kLn+1 f k1,[a,b] q
g n q , · X ∞,[a,b] n X
Next we observe that for α, β > 1 such that
1 α
+
1 β
(8.122)
= 1 we have
|Rw (x)| ≤ kLn+1 f kα,[a,b]kgn (x, ·)kβ,[a,b] .
(8.123)
We derive Theorem 8.21. It holds Z n X p Γf (µ1 , µ2 ) ≤ |Li f (1)| qgi , 1 dλ q X i=1 !
Z
p
+ kLn+1 f kα,[a,b] q dλ .
g n q , · X β,[a,b]
(8.124)
In the following we use Lemmas 12.1.4 and 12.1.5, p. 368 of [20]. Here u0 (x) = c > 0, u1 (x) concave for x ≤ 1 and convex for x ≥ 1. We put ˜n (x, 1) := |Nn (x, 1)|, N
(8.125)
Z
(8.126)
where Nn (x, 1) := all x ∈ [a, b], n ≥ 0. Let 1 ∈ (a, b) then
x
gn (x, s)ds, 1
˜n (x, 1) ≤ ψ|x − 1|, N
all x ∈ [a, b], n ≥ 1 where ψ := max
(
˜n (a, 1) ˜n (b, 1) N N , b−1 1−a
(8.127) )
.
(8.128)
Here we take n even then gn (x, s) ≥ 0, all x, s ∈ [a, b]. Clearly then we have Z x ˜n (x, 1) |Rw (x)| ≤ kLn+1 f k∞,[a,b] gn (x, s)ds = kLn+1 f k∞,[a,b] N 1
≤ ψkLn+1 f k∞,[a,b] |x − 1|, all x ∈ [a, b].
We have proved
(8.129)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Sharp High Degree Estimation of Csiszar’s f -Divergence
Theorem 8.22. Here 1 ∈ (a, b), n even. It holds Z n X p , 1 dλ Γf (µ1 , µ2 ) ≤ |Li f (1)| qgi q X i=1 Z |p − q|dλ . + ψkLn+1 f k∞,[a,b]
Book
91
(8.130)
X
Remark 8.23. Here again we use (8.78). Let x ≥ t then by (8.81) gn (x, t) ≥ 0, n ∈ Z+ . Let Ln+1 f ≥ 0 (≤ 0) then Z x gn (x, s)Ln+1 f (s)ds ≥ 0 (≤ 0) (8.131) t
and
f (x) ≥ (≤)
n X
Li f (t)gi (x, t),
i=0
x ≥ t, n ∈ Z+ .
(8.132)
Let n be even then gn (x, s) ≥ 0 for any x, s ∈ [a, b]. If Ln+1 f ≥ 0 (≤ 0) and x ≤ t, then Z x gn (x, s)Ln+1 f (s)ds ≤ 0 (≥ 0), (8.133) t
and
f (x) ≤ (≥)
n X
Li f (t)gi (x, t).
(8.134)
i=0
If n is odd, then for x ≤ t by (8.81) we get gn (x, t) ≤ 0. If Ln+1 f ≥ 0 (≤ 0) and x ≤ t, then (8.131) and (8.132) are again true. Conclusion. If n is odd and Ln+1 f ≥ 0 (≤ 0), then n X f (x) ≥ (≤) Li f (t)gi (x, t),
(8.135)
i=0
for any x, t ∈ [a, b], n ∈ Z+ . In particular, by f (1) = 0, a ≤ 1 ≤ b, a ≤ pq ≤ b, a.e. on X we obtain: If n is odd and Ln+1 f ≥ 0 (≤ 0), then n X f (x) ≥ (≤) Li f (1)gi (x, 1), any x ∈ [a, b].
(8.136)
i=1
Hence we have
qf
n X p p ≥ (≤) Li f (1)qgi ,1 , q q i=1
a.e. on X.
Conclusion. We have established that Z n X p Γf (µ1 , µ2 ) ≥ (≤) Li f (1) qgi , 1 dλ, q X i=1
when n is odd and Ln+1 f ≥ 0 (≤ 0).
(8.137)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
92
8.4
Results Based on an Alternative Expansion Formula
We need the following basic general result (see also (3.53), p. 89 of [125] and [36]). We give our own proof. Theorem 8.24. Let f ∈ C n ([a, b]), [a, b] ⊂ R, n ∈ N, x, z ∈ [a, b]. Then Z x Z n−1 X (x − z)k+1 (−1)n x f (t)dt = (−1)k f (k) (x) + (t − z)n f (n) (t)dt. (k + 1)! n! z z
(8.138)
k=0
Proof. From [327], p. 98 we use the generalized integration by parts formula Z Z (n) (n−1) 0 (n−2) 00 (n−3) n uv dt = uv −uv +u v − · · · + (−1) u(n) vdt, (8.139)
where here
u(t) = f (t),
v(t) =
(t − z)n . n!
(8.140)
n−k
(n) Notice u(k) (t) = (t−z) (t) = 1, and u(k) (z) = 0, (n−k)! , k = 1, 2, . . . , n − 1 and v k = 1, 2, . . . , n − 1. Using (8.139) we have Z (t − z)2 (t − z)3 f (t)dt = f (t)(t − z) − f 0 (t) + f 00 (t) 2 3! 4 n Z (t − z) (−1) − f (3) (t) ··· f (n) (t)(t − z)n dt. (8.141) 4! n!
Thus Z x z
(x − z)3 (x − z)4 (x − z)2 f (t)dt = f (x)(x − z) − f 0 (x) + f 00 (x) − f (3) (x) ··· 2 3! 4! Z (−1)n x + (t − z)n f (n) (t)dt n! z Z n−1 X (x − z)k+1 (−1)n x k (k) = (−1) f (x) + (t − z)n f (n) (t)dt, (8.142) (k + 1)! n! z k=0
proving (8.138).
p q
Here again 0 < a ≤ 1 ≤ b, f (1) = 0, a ≤ ≤ b, a.e. on X and all the rest as in this chapter and f ∈ C n+1 ([a, b]), n ∈ N. We plug into (8.138) instead of f , f 0 and z = 1, we obtain Z n−1 X (x − 1)k+1 (−1)n x f (x) = (−1)k f (k+1) (x) + (t − 1)n f (n+1) (t)dt, (8.143) (k + 1)! n! 1 k=0
all x ∈ [a, b], see also (2.13), p. 5 of [101]. We call the remainder of (8.143) Z (−1)n x ψ1 (x) := (t − 1)n f (n+1) (t)dt. (8.144) n! 1
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Sharp High Degree Estimation of Csiszar’s f -Divergence
Book
93
Consequently we have k+1 n−1 p Z p/q X q q −1 p (−1)n k (k+1) p qf = (−1) f + q (t − 1)n f (n+1) (t)dt, q q (k + 1)! n! 1 k=0 (8.145) a.e. on X. We have proved our way the following representation formula (see also (2.10), p. 4, of [101]). Theorem 8.25. All elements involved here as in this chapter and f ∈ C n+1 ([a, b]), n ∈ N, 0 < a ≤ 1 ≤ b. Then n−1 X (−1)k Z p f (k+1) q −k (p − q)k+1 dλ + ψ2 , (8.146) Γf (µ1 , µ2 ) = (k + 1)! X q k=0
where (−1)n ψ2 := n!
Z
q x
Z
p/q 1
!
(8.147)
all x ∈ [a, b].
(8.148)
(t − 1)n f (n+1) (t)dt dλ.
Next we estimate ψ2 . We see that |ψ1 (x)| ≤
kf (n+1) k∞,[a,b] |x − 1|n+1 , (n + 1)!
Also |ψ1 (x)| ≤ Furthermore for α, β > 1: |ψ1 (x)| ≤
kf (n+1) k1,[a,b] |x − 1|n , n!
1 α
+
1 β
all x ∈ [a, b].
(8.149)
= 1 we derive
kf (n+1) kα,[a,b] 1 |x − 1|n+ β , n!(nβ + 1)1/β
all x ∈ [a, b].
(8.150)
Then (n+1) kf k∞,[a,b] −n q |p − q|n+1 , (n + 1)! (n+1) p kf k1,[a,b] 1−n ≤ min of q ψ1 q |p − q|n , q n! kf (n+1) kα,[a,b] 1−n− 1 1 1 1 β |p − q|n+ β , α, β > 1 : q + = 1. α β n!(nβ + 1)1/β (8.151) All the above are true a.e. on X. We have proved (see also the similar Corollary 1, p. 10, of [101]).
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
PROBABILISTIC INEQUALITIES
94
Theorem 8.26. All elements are as in this chapter and f ∈ C n+1 ([a, b]), n ∈ N. Here ψ2 is as in (8.147). Then Z kf (n+1) k∞,[a,b] q −n |p − q|n+1 dλ, B := 1 (n + 1)! X Z kf (n+1) k1,[a,b] 1 1 |ψ2 | ≤ min of B2 := q 1−n |p − q|n dλ, for α, β > 1 : + = 1, n! α β X Z kf (n+1) kα,[a,b] 1 1 B3 := q 1−n− β |p − q|n+ β dλ. 1/β n!(nβ + 1) X (8.152) Also it holds Z n−1 X 1 f (k+1) p q −k (p − q)k+1 dλ + min(B1 , B2 , B3 ). Γf (µ1 , µ2 ) ≤ (k + 1)! X q k=0 (8.153) Corollary 8.27. (to Theorem 8.26). Case of n = 1. It holds Z kf 00 k∞,[a,b] B1,1 := q −1 (p − q)2 dλ, 2 X Z 1 1 |p − q|dλ, for α, β > 1 : + = 1, |ψ2 | ≤ min of B2,1 := kf 00 k1,[a,b] α β X Z kf 00 kα,[a,b] 1 1 B3,1 := q − β |p − q|1+ β dλ. (β + 1)1/β X (8.154) Also it holds Z p 0 Γf (µ1 , µ2 ) ≤ f (p − q)dλ + min(B1,1 , B2,1 , B3,1 ). (8.155) q X
Remark 8.28. (to Theorem 8.25). Notice that Z Z (−1)n x (−1)n x n (n+1) (t − 1) f (t)dt = (t − 1)n [f (n+1) (t) − f (n+1) (1)]dt n! n! 1 1 (x − 1)n+1 . (8.156) + (−1)n f (n+1) (1) (n + 1)!
Therefore from (8.143) we obtain f (x) =
n−1 X k=0
(−1)k f (k+1) (x)
(x − 1)k+1 (−1)n (n+1) + f (1)(x − 1)n+1 + ψ3 (x), (k + 1)! (n + 1)! (8.157)
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Sharp High Degree Estimation of Csiszar’s f -Divergence
where ψ3 (x) :=
(−1)n n!
Z
95
x 1
(t − 1)n (f (n+1) (t) − f (n+1) (1))dt.
(8.158)
Then for 0 < h ≤ b − a we obtain Z 1 x n (n+1) (n+1) |ψ3 (x)| ≤ |t − 1| |f (t) − f (1)|dt n! 1 (by Lemma 7.1.1, p. 208, of [20]) Z ω1 (f (n+1) , h) x n |t − 1| |t − 1| dt ≤ n! h 1 Z x (n+1) ω1 (f , h) |t − 1| n ≤ |t − 1| 1 + dt n! h 1 Z x Z x (n+1) ω1 (f , h) 1 n n+1 |t − 1| dt + |t − 1| dt = n! h 1 1 Z Z x (n+1) 1 x ω1 (f , h) n+1 n ≤ |t − 1| dt . |t − 1| dt + n! h 1 1 We have proved for 0 < h ≤ b − a that Z Z 1 x ω1 (f (n+1) , h) x n n+1 |t − 1| dt + |ψ3 (x)| ≤ |t − 1| dt , n! h 1 1
(8.159)
all x ∈ [a, b]. Next suppose that |f (n+1) (t) − f (n+1) (1)| is convex in t and 0 < h ≤ min(1 − a, b − 1) then Z 1 x n (n+1) (n+1) |ψ3 (x)| ≤ |t − 1| |f (t) − f (1)|dt n! 1 (by Lemma 8.1.1, p. 243, of [20]) Z ω1 (f (n+1) , h) x n+1 ≤ |t − 1| dt . (8.160) n!h 1
By
Z
x
1
|x − 1|m+1 |t − 1|m dt = , m+1
we get the following For 0 < h ≤ b − a we have |ψ3 (x)| ≤
ω1 (f (n+1) , h) n!
m 6= −1,
|x − 1|n+1 1 |x − 1|n+2 + n+1 h (n + 2)
,
(8.161)
and for 0 < h ≤ min(1 − a, b − 1) and |f (n+1) (t) − f (n+1) (1)| convex in t we obtain |ψ3 (x)| ≤
ω1 (f (n+1) , h) |x − 1|n+2 , n!(n + 2)h
(8.162)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
96
n ∈ N, all x ∈ [a, b]. The last inequality (8.162) is attained when n is even by (t − 1)n+2 fˆ(t) := , a ≤ t ≤ b. (n + 2)!
(8.163)
Really fˆ ∈ C n+1 ([a, b]), it is convex and strictly convex at 1. We get fˆ(k) (1) = 0, k = 0, 1, . . . , n + 1 where fˆ(n+1) (t) = t − 1 with obviously |fˆ(t)(n+1) | convex and ω1 (fˆ(n+1) , h) = h. For fˆ the corresponding (x − 1)n+2 , ψˆ3 (x) = n!(n + 2)
x ∈ [a, b],
(8.164)
proving (8.162) attained. Here again n−1 X (−1)k p p −k (k+1) qf = f q (p − q)k+1 q (k + 1)! q k=0
(−1)n (n+1) + f (1)q −n (p − q)n+1 + qψ3 (n + 1)! a.e. on X. We have established
p , q
(8.165)
Theorem 8.29. All elements involved here as in this chapter and f ∈ C n+1 ([a, b]), n ∈ N, 0 < a ≤ 1 ≤ b. Then n−1 X (−1)k Z p (k+1) Γf (µ1 , µ2 ) = f q −k (p − q)k+1 dλ (k + 1)! X q k=0
(−1)n (n+1) + f (1) (n + 1)! where
Z
X
q −n (p − q)n+1 dλ + ψ4 ,
p ψ4 := qψ3 dλ. q X
(8.166)
Z
(8.167)
Let 0 < h ≤ b − a, then from (8.161) we have 1 q −n−1 |p − q|n+2 p ω1 (f (n+1) , h) q −n |p − q|n+1 q ψ3 + , ≤ q n! n+1 h (n + 2)
(8.168)
Next we estimate ψ4 .
therefore
|ψ4 | ≤ +
ω1 (f (n+1) , h) n!
1 (n + 2)h
Z
X
1 n+1
Z
X
q −n |p − q|n+1 dλ
q −n−1 |p − q|n+2 dλ , n ∈ N.
(8.169)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Sharp High Degree Estimation of Csiszar’s f -Divergence
97
When 0 < h ≤ min(1 − a, b − 1) and |f (n+1) (t) − f (n+1) (1)| convex in t we obtain (n+1) , h) −n−1 ψ3 p ≤ ω1 (f q |p − q|n+2 , (8.170) q n!(n + 2)h
and
|ψ4 | ≤
ω1 (f (n+1) , h) n!(n + 2)h
Z
X
q −n−1 |p − q|n+2 dλ,
n ∈ N.
(8.171)
For n even the last inequality (8.171) is attained by fˆ by (8.163). We have proved Theorem 8.30. All elements involved here as in this chapter and f ∈ C n+1 ([a, b]), n ∈ N, 0 < a ≤ 1 ≤ b and Z 1 0< q −n−1 |p − q|n+2 dλ ≤ b − a. (8.172) (n + 2) X
Then (8.166) is again valid and
|ψ4 | ≤ Z 1 1 ω1 f (n+1) , q −n−1 |p − q|n+2 dλ n! (n + 2) X Z 1 1+ q −n |p − q|n+1 dλ . n+1 X
(8.173)
Also we have Theorem 8.31. All elements involved here as in this chapter and f ∈ C n+1 ([a, b]), n ∈ N, 0 < a ≤ 1 ≤ b. Furthermore we suppose that |f (n+1) (t) − f (n+1) (1)| is convex in t and Z 1 q −n−1 |p − q|n+2 dλ ≤ min(1 − a, b − 1). (8.174) 0< n!(n + 2) X Then (8.166) is again valid and |ψ4 | ≤ ω1 f (n+1) ,
1 n!(n + 2)
Z
q X
−n−1
|p − q|
n+2
The last (8.175) is attained by fˆ of (8.163) when n is even.
dλ .
(8.175)
When n = 1 we derive Corollary 8.32. (to Theorem 8.30). All elements involved here as in this chapter and f ∈ C 2 ([a, b]), 0 < a ≤ 1 ≤ b and Z 1 0< q −2 |p − q|3 dλ ≤ b − a. (8.176) 3 X Then Z Z p f 00 (1) Γf (µ1 , µ2 ) = f0 (p − q)dλ − q −1 (p − q)2 dλ + ψ4,1 , (8.177) q 2 X X
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
98
where ψ4,1 with ψ3,1 (x) := −
Z
x 1
p dλ, := qψ3,1 q X Z
(8.178)
(t − 1)(f 00 (t) − f 00 (1))dt, x ∈ [a, b].
Furthermore it holds Z Z 1 00 1 −2 3 −1 2 q |p − q| dλ · |ψ4,1 | ≤ ω1 f , q p dλ + 1 . 3 X 2 X
(8.179)
(8.180)
Corollary 8.33. (to Theorem 8.31). All elements involved here as in this chapter and f ∈ C 2 ([a, b]), 0 < a ≤ 1 ≤ b. Furthermore we suppose that |f 00 (t) − f 00 (1)| is convex in t and Z 1 q −2 |p − q|3 dλ ≤ min(1 − a, b − 1). (8.181) 0< 3 X Then again (8.177) is valid and
|ψ4,1 | ≤ ω1
1 f , 3 00
Z
q X
−2
3
|p − q| dλ .
(8.182)
Remark 8.34 (to Theorem 8.29). Here all as in Theorem 8.29 are valid especially the conclusions (8.166) and (8.167). Additionally assume that |f (n+1) (x) − f (n+1) (y)| ≤ K|x − y|α , K > 0, 0 < α ≤ 1, all x, y ∈ [a, b] with (x − 1)(y − 1) ≥ 0. Then Z K x K n+α n+α+1 |ψ3 | ≤ |t − 1| dt , = n!(n + α + 1) |x − 1| n!
(8.183)
(8.184)
1
all x ∈ [a, b], n ∈ N. Therefore it holds p K q ψ3 q −n−α |p − q|n+α+1 , a.e. on X. ≤ q n!(n + α + 1)
(8.185)
We have established
Theorem 8.35. All elements involved here as in this chapter and f ∈ C n+1 ([a, b]), n ∈ N, 0 < a ≤ 1 ≤ b. Additionally assume (8.183), |f (n+1) (x) − f (n+1) (y)| ≤ K|x − y|α ,
K > 0, 0 < α ≤ 1, all x, y ∈ [a, b] with (x − 1)(y − 1) ≥ 0. Then we get (8.166), n−1 X (−1)k Z p f (k+1) q −k (p − q)k+1 dλ Γf (µ1 , µ2 ) = (k + 1)! X q k=0
+
(−1)n (n+1) f (1) (n + 1)!
Z
X
q −n (p − q)n+1 dλ + ψ4 .
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Sharp High Degree Estimation of Csiszar’s f -Divergence
99
Here we obtain that
Z K q −n−α |p − q|n+α+1 dλ. n!(n + α + 1) X Inequality (8.186) is attained when n is even by |ψ4 | ≤
f ∗ (x) = c˜|x − 1|n+α+1 ,
where
c˜ := Q n
j=0
x ∈ [a, b]
K
.
(8.186)
(8.187)
(8.188)
(n + α + 1 − j)
Proof of sharpness of (8.186). Here f ∗ ∈ C n+1 ([a, b]), is convex and strictly convex at x = 1, f ∗ (1) = 0. We find that ∗(k)
f ∗(n+1) (x) = K|x − 1|α sign (x − 1)
(8.189)
∗(n+1)
with f (1) = 0, k = 1, 2, . . . , n + 1. Furthermore f clearly fulfills (8.183), see in p. 28 of [29] inequality (105). Additionally we obtain the corresponding K|x − 1|n+α+1 ψ3 (x) = , x ∈ [a, b]. (8.190) n!(n + α + 1) Then the corresponding Z Z K p q −n−α |p − q|n+α+1 dλ, (8.191) dλ = ψ4 = q ψ3 q n!(n + α + 1) X X proving that (8.186) is attained. Finally we present
Corollary 8.36 (to Theorem 8.35). All elements involved here as in this chapter and f ∈ C 2 ([a, b]), 0 < a ≤ 1 ≤ b. Additionally assume that |f 00 (x) − f 00 (y)| ≤ K|x − y|α ,
(8.192)
K > 0, 0 < α ≤ 1, all x, y ∈ [a, b] with (x − 1)(y − 1) ≥ 0. Then we get (8.177), Z Z f 00 (1) p 0 (p − q)dλ − q −1 (p − q)2 dλ + ψ4,1 , Γf (µ1 , µ2 ) = f q 2 X X where Z p ψ4,1 := qψ3,1 dλ, q X with Z x ψ3,1 (x) := − (t − 1)(f 00 (t) − f 00 (1))dt, x ∈ [a, b]. 1
It holds that
|ψ4,1 | ≤
K (α + 2)
Z
X
q −1−α |p − q|α+2 dλ.
(8.193)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
This page intentionally left blank
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Chapter 9
Csiszar’s f -Divergence as a Measure of Dependence
In this chapter the Csiszar’s f -Divergence or Csiszar’s Discrimination, the most general distance among probability measures, is established as the most general measure of Dependence between two random variables which is a very important aspect of stochastics. Many estimates of it are given, leading to optimal or nearly optimal probabilistic inequalities. That is we are comparing and connecting this general measure of dependence to known other specific entities by involving basic parameters of our setting. This treatment relies on [39]. 9.1
Background
Throughout this chapter we use the following. Let f by a convex function from (0, +∞) into R which is strictly convex at 1 with f (1) = 0. Let (R2 , B 2 , λ) be the measure space, where λ is the product Lebesgue measure on (R2 , B 2 ) with B being the Borel σ-field. And let X, Y : Ω → R be random variables on the probability space (Ω, P ). Consider the probability distributions µXY and µX × µY on R2 , where µXY , µX , µY stand for the joint distribution of X and Y and their marginal distributions, respectively. Here we assume as existing the following probability density functions, the joint pdf of µXY to be t(x, y), x, y ∈ R, the pdf of µX to be p(x) and the pdf of µY to be q(y). Clearly µX × µY has pdf p(x)q(y). Here we further assume that 0 < a ≤ t 2 pq ≤ b, a.e. on R and a ≤ 1 ≤ b. The quantity Z t(x, y) Γf (µXY , µX × µY ) = p(x)q(y)f dλ(x, y), (9.1) p(x)q(y) R2
is the Csiszar’s distance or f -divergence between µXY and µX × µY . Γf is the most general measure for comparing probability measures and it was first introduced by I. Csiszar in 1963, [139], see also [140], [152], [244], [243], [305], [306], and [101], [102], [125], [126], [37], [42], [43], [41]. In Information Theory and Statistics many various divergences are used which are special cases of the above Γf divergence. Γf has many applications also in 101
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
102
Anthropology, Genetics, Finance, Economics, Political Science, Biology, Approximation of Probability Distributions, Signal Processing and Pattern Recognition. Here X, Y are less dependent the closer the distributions µXY and µX × µY are, thus Γf (µXY , µX × µY ) can be considered as a measure of dependence of X and Y . For f (u) = u log2 u we obtain the mutual information of X, and Y , I(X, Y ) = I(µXY k µX × µY ) = Γu log2 u (µXY , µX × µY ),
see [140]. For f (u) = (u − 1)2 we get the mean square contingency : ϕ2 (X, Y ) = Γ(u−1)2 (µXY , µX × µY ),
see [305]. In the last we need µXY µX ×µY , where denotes absolute continuity, but to cover the case of µXY 6 µX × µY we set ϕ2 (X, Y ) = +∞, then the last formula is always valid. Clearly here µXY , µX × µY λ, also Γf (µXY , µX × µY ) ≥ 0 with equality only when µXY = µX × µY , i.e. when X, Y are independent r.v.’s. In this chapter we present applications of author’s articles ([37]), ([42]), ([43]), ([41]). 9.2
Results
Part I. Here we apply results from [37]. We present the following. Theorem 9.1. Let f ∈ C 1 ([a, b]). Then Γf (µXY , µX × µY ) ≤ kf 0 k∞,[a,b]
Z
R2
|t(x, y) − p(x)q(y)| dλ(x, y).
(9.2)
Inequality (9.2) is sharp. The optimal function is f ∗ (y) = |y − 1|α , α > 1 under the condition, either i) max(b − 1, 1 − a) < 1
and
C := esssup(pq) < +∞,
(9.3)
or ii) |t − pq| ≤ pq ≤ 1,
a.e. on R2 .
Proof. Based on Theorem 1 of [37].
(9.4)
Next we give the more general Theorem 9.2. Let f ∈ C n+1 ([a, b]), n ∈ N, such that f (k) (1) = 0, k = 0, 2, 3, . . . , n. Then Γf (µXY , µX × µY ) Z kf (n+1) k∞,[a,b] (p(x))−n (q(y))−n |t(x, y) − p(x)q(y)|n+1 dλ(x, y). ≤ (n + 1)! R2
(9.5)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Csiszar’s f -Divergence as a Measure of Dependence
Book
103
Inequality (9.5) is sharp. The optimal function is f˜(y) := |y − 1|n+α , α > 1 under the condition (i) or (ii) of Theorem 9.1. Proof. As in Theorem 2 of [37].
As a special case we have Corollary 9.3 (to Theorem 9.2). Let f ∈ C 2 ([a, b]), such that f (1) = 0. Then Z kf (2) k∞,[a,b] (p(x))−1 (q(y))−1 (t(x, y)−p(x)q(y))2 dλ(x, y). Γf (µXY , µX ×µY ) ≤ 2 R2 (9.6) ˜ := |y − 1|1+α , α > 1 under Inequality (9.6) is sharp. The optimal function is f(y) the condition (i) or (ii) of Theorem 9.1. Proof. By Corollary 1 of [37].
Next we connect Csiszar’s f -divergence to the usual first modulus of continuity ω1 . Theorem 9.4. Assume that Z 0 < h := |t(x, y) − p(x)q(y)| dλ(x, y) ≤ min(1 − a, b − 1).
(9.7)
R2
Then
Γf (µXY , µX × µY ) ≤ ω1 (f, h).
(9.8)
Inequality (9.8) is sharp, namely it is attained by f ∗ (x) = |x − 1|. Proof. Based on Theorem 3 of [37].
Part II. Here we apply results from [42]. But first we need some basic background from [26], [122]. Let ν > 0, n := [ν] and α = ν−n (0 < α < 1). Let x, x0 ∈ [a, b] ⊆ R such that x ≥ x0 , where x0 is fixed. Let f ∈ C([a, b]) and define Z x 1 x0 (Jν f )(x) := (x − s)ν−1 f (s) ds, x0 ≤ x ≤ b, (9.9) Γ(ν) x0 the generalized Riemann-Liouville integral, where Γ stands for the gamma function. We define the subspace x0 Dn f ∈ C 1 ([x0 , b])}. Cxν0 ([a, b]) := {f ∈ C n ([a, b]) : J1−α
(9.10)
For f ∈ Cxν0 ([a, b]) we define the generalized ν-fractional derivative off over [x 0 , b] as d x0 (n) ν . (9.11) D := Dx0 f := DJ1−α f dx We present the following.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
104
Theorem 9.5. Let a < b, 1 ≤ ν < 2, f ∈ Caν ([a, b]), and 0 < a ≤ R2 . Then
t pq
≤ b, a.e. on
Γf (µXY , µX × µY ) Z kDaν f k∞,[a,b] ≤ (p(x))1−ν (q(y))1−ν (t(x, y) − ap(x)q(y))ν dλ(x, y). Γ(ν + 1) 2 R Proof. It follows from Theorem 2 of [42].
(9.12)
The counterpart of the previous result comes next. Theorem 9.6. Let a < b, ν ≥ 2, n := [ν], f ∈ Caν ([a, b]), f (i) (a) = 0, i = t(x,y) 0, 1, . . . , n − 1, and a ≤ p(x)q(y) ≤ b, a.e. on R2 . Then Γf (µXY , µX × µY ) Z kDaν f k∞,[a,b] (p(x))1−ν (q(y))1−ν (t(x, y) − ap(x)q(y))ν dλ(x, y). ≤ Γ(ν + 1) R2 Proof. Based on Theorem 3 of [42].
(9.13)
Next we give an Lα˜ estimate. Theorem 9.7. Let a < b, ν ≥ 1, n := [ν], f ∈ Caν ([a, b]), f (i) (a) = 0, i = t 0, 1, . . . , n − 1, and a ≤ pq ≤ b, a.e. on R2 . Let α ˜ , β > 1 : α1˜ + β1 = 1. Then kDaν f kα,[a,b] ˜ Γf (µXY , µX × µY ) ≤ Γ(ν)(β(ν − 1) + 1)1/β Z 1 1 · (p(x)q(y))2−ν− β (t(x, y) − ap(x)q(y))ν−1+ β dλ(x, y).
(9.14)
R2
Proof. Based on Theorem 4 of [42].
It follows an L∞ estimate. Theorem 9.8. Let a < b, ν ≥ 1, n := [ν], f ∈ Caν ([a, b]), f (i) (a) = 0, i = t ≤ b, a.e. on R2 . Then 0, 1, . . . , n − 1, and a ≤ pq Z kDaν f k1,[a,b] Γf (µXY , µX × µY ) ≤ (p(x)q(y))2−ν (t(x, y) − ap(x)q(y))ν−1 dλ(x, y). Γ(ν) R2 (9.15) Proof. Based on Theorem 5 of [42].
We continue with Theorem 9.9. Let f ∈ C 1 ([a, b]), a 6= b. Then Z b 1 f (x)dx Γf (µXY , µX × µY ) ≤ b−a a Z t(x, y) p(x)q(y)(a + b) − f0 − t(x, y) dλ(x, y). p(x)q(y) 2 R2
(9.16)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Csiszar’s f -Divergence as a Measure of Dependence
Proof. Based on Theorem 6 of [42].
Book
105
Furthermore we obtain Theorem 9.10. Let n odd and f ∈ C n+1 ([a, b]), such that f (n+1) ≥ 0 (≤ 0), t ≤ b, a.e. on R2 . Then 0 < a ≤ pq Z b 1 f (x)dx (9.17) Γf (µXY , µX × µY ) ≤ (≥) b−a a ! n i Z X X 1 t (i) − f (pq)1−i (bpq − t)(i−k) (apq − t)k dλ . (i + 1)! pq 2 R i=1 k=0
Proof. Based on Theorem 7 of [42].
Part III. Here we apply results from [43]. We start with Theorem 9.11. Let f, g ∈ C 1 ([a, b]) where f as in this chapter, g 0 6= 0 over [a, b]. Then
0 Z
f t(x, y)
Γf (µXY , µX × µY ) ≤ 0 p(x)q(y) g − g(1) dλ(x, y). g p(x)q(y) 2 R
∞,[a,b]
(9.18)
Proof. Based on Theorem 1 of [43].
We give Examples 9.12 (to Theorem 9.11). 1) Let g(x) = x1 . Then Z 2 0 Γf (µXY , µX × µY ) ≤ kx f k∞,[a,b] 2. Let g(x) = ex . Then
Γf (µXY , µX × µY ) ≤ ke−x f 0 k∞,[a,b] 3) Let g(x) = e−x . Then Γf (µXY , µX × µY ) ≤ kex f 0 k∞,[a,b]
Z
4) Let g(x) = `nx. Then Γf (µXY , µX × µY ) ≤ kxf 0 k∞,[a,b]
p(x)q(y) |t(x, y) − p(x)q(y)|dλ(x, y). (9.19) t(x, y)
R2
Z
Z
R2
R2
t(x,y) p(x)q(y) e p(x)q(y) − e dλ(x, y).
(9.20)
t(x,y) p(x)q(y) e− p(x)q(y) − e−1 dλ(x, y). (9.21)
t(x, y) p(x)q(y) `n dλ(x, y). p(x)q(y) R2
(9.22)
5) Let g(x) = x `n x, with a > e−1 . Then
Z
f0
t(x, y)
Γf (µXY , µX ×µY ) ≤ dλ(x, y). (9.23) t(x, y) `n 1 + `n x ∞,[a,b] R2 p(x)q(y)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
106
√ x. Then Z √ Γf (µXY , µX × µY ) ≤ 2k xf 0 k∞,[a,b] 6) Let g(x) =
R2
p p p p(x)q(y) | t(x, y) − p(x)q(y)|dλ(x, y).
(9.24)
7) Let g(x) = xα , α > 1. Then Γf (µXY , µX × µY )
Z 1 f0
≤ (p(x)q(y))1−α |(t(x, y))α − (p(x)q(y))α |dλ(x, y). α xα−1 ∞,[a,b] R2
(9.25)
Next we give
Theorem 9.13. Let f ∈ C 1 ([a, b]), a 6= b. Then " Z b kf 0 k 1 a2 + b 2 ∞,[a,b] f (y)dy ≤ − (a + b) Γf (µXY , µX × µY ) − b−a a (b − a) 2 # Z −1 2 + (p(x)q(y)) t (x, y)dλ(x, y) . (9.26) R2
Proof. Based on Theorem 2 of [43].
It follows Theorem 9.14. Let f ∈ C (2) ([a, b]), a 6= b. Then Z b 1 f (b) − f (a) a + b f (s)ds − 1− (9.27) Γf (µXY , µX × µY ) − b−a a b−a 2 2 (Z ) t(x,y) a+b 2 (b − a)2 00 1 1 p(x)q(y) − 2 ≤ kf k∞,[a,b] p(x)q(y) + dλ(x, y) + . 2 (b − a)2 4 12 R2
Proof. Based on (33) of [43].
Working more generally we have Theorem 9.15. Let f ∈ C 1 ([a, b]) and g ∈ C([a, b]) of bounded variation, g(a) 6= g(b). Then Z b 1 f (s)dg(s) Γf (µXY , µX × µY ) − g(b) − g(a) a (Z ! ) Z b t 00 P g ≤ kf k∞,[a,b] pq , g(s) ds dλ , (9.28) pq R2 a where
P (g(z), g(s)) :=
g(s) − g(a) , g(b) − g(a) g(s) − g(b) , g(b) − g(a)
a ≤ s ≤ z, (9.29) z < s ≤ b.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Csiszar’s f -Divergence as a Measure of Dependence
107
Proof. By Theorem 5 of [43].
We give Examples 9.16 (to Theorem 9.15). Let f ∈ C 1 ([a, b]) and g(x) = ex , `n xα , α > 1; x > 0. Then by (71)–(74) of [43] we obtain 1) Z b 1 s f (s)e ds Γf (µXY , µX × µY ) − b a e −e a Z kf 0 k∞,[a,b] t pq dλ − ea (2 − a) + eb (b − 2) , 2 pqe ≤ (eb − ea ) R2 2) −1 Z b b f (s) ds Γf (µXY , µX × µY ) − `n a s a −1 Z b t ≤ kf 0 k∞,[a,b] `n 2 t `n dλ − `n(ab) + (a + b − 2) , a pq R2 3) Z b 1 f (s) √ ds Γf (µXY , µX × µY ) − √ √ s 2( b − a) a Z √ √ kf 0 k∞,[a,b] 4 (a3/2 + b3/2 ) −1/2 3/2 ≤ √ (pq) t dλ + − ( a + b) , √ 3 ( b − a) 3 R2 4) (α > 1) Z b α α−1 f (s)s ds Γf (µXY , µX × µY ) − α (b − aα ) a Z kf 0 k∞,[a,b] 2 ≤ (pq)−α tα+1 dλ (bα − aα ) (α + 1) R2 α α+1 α+1 α α (a +b ) − (a + b ) . + α+1
x,
√
x,
(9.30)
(9.31)
(9.32)
(9.33)
We continue with
Theorem 9.17. Let f ∈ C (2) ([a, b]), g ∈ C([a, b]) and of bounded variation, g(a) 6= g(b). Then Z b 1 f (s)dg(s) Γf (µXY , µX × µY ) − (g(b) − g(a)) a ! ! ! ! Z b Z Z b 1 t 0 f (s1 )dg(s1 ) · pq P g , g(s) ds dλ − (g(b) − g(a)) pq a R2 a ! ! Z Z b Z b t , g(s) P (g(s), g(s1 ))ds1 ds dλ . ≤ kf 00 k∞,[a,b] pq P g pq 2 R a a
(9.34)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
108
Proof. Apply (79) of [43].
Remark 9.18. Next we define ∗
P (z, s) :=
s − a, s − b,
s ∈ [a, z] s ∈ (z, b].
(9.35)
Let f ∈ C 1 ([a, b]). Then by (25) of [43] we get the representation ! Z b 1 Γf (µXY , µX × µY ) = f (s)ds + R1 , b−a a where according to (26) of [43] we have ! Z Z b t(x, y) ∗ 0 , s f (s)ds dλ(x, y). R1 = p(x)q(y) P p(x)q(y) R2 a
(9.36)
Let again f ∈ C 1 ([a, b]), g ∈ C([a, b]) and of bounded variation, g(a) 6= g(b). Then by (47) of [43] we get the representation Z b 1 Γf (µXY , µX × µY ) = f (s)dg(s) + R4 , (9.37) (g(b) − g(a)) a
where by (48) of [43] we obtain Z Z R4 = pq R2
Let α ˜, β > 1 :
1 α ˜
+
b a
! t 0 , g(s) f (s)ds dλ. P g pq
1 β
= 1, then by (89) of [43] we have
Z
∗ t(x, y)
dλ(x, y), |R1 | ≤ kf 0 kα,[a,b] p(x)q(y) P , · ˜
p(x)q(y) R2 β,[a,b]
and by (90) of [43] we get
Z
∗
t(x, y) 0
dλ(x, y). |R1 | ≤ kf k1,[a,b] p(x)q(y) P ,·
p(x)q(y) R2 ∞,[a,b] Furthermore by (95) and (96) of [43] we find
! Z
t(x, y)
0 |R4 | ≤ kf kα,[a,b] p(x)q(y) P g , g(·) ˜
p(x)q(y) 2 R
and
0
(9.38)
|R4 | ≤ kf k1,[a,b]
Z
!
t(x, y)
, g(·) p(x)q(y) P g
p(x)q(y) 2 R
(9.39)
(9.40)
dλ(x, y),
(9.41)
dλ(x, y).
(9.42)
β,[a,b]
∞,[a,b]
t Remark 9.19. Here 0 < a < b with 1 ∈ [a, b], and pq ∈ [a, b] a.e. on R2 . 1 (i) By (105) of [43] for f ∈ C ([a, b]) we obtain Z t(x, y) t(x, y) |R1 | ≤ kf 0 k1,[a,b] p(x)q(y) max − a, b − dλ(x, y). (9.43) p(x)q(y) p(x)q(y) R2
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Csiszar’s f -Divergence as a Measure of Dependence
109
(ii) Let g be strictly increasing and continuous over [a, b], e.g., g(x) = ex , `n x, x, xα with α > 1, and x > 0 whenever is needed. Also f ∈ C 1 ([a, b]) and as in this chapter. Then by (106) of [43] we find Z kf 0 k1,[a,b] t(x, y) p(x)q(y) max g |R4 | ≤ (g(b) − g(a)) R2 p(x)q(y) ! t(x, y) − g(a), g(b) − g dλ(x, y). (9.44) p(x)q(y)
√
In particular via (107)–(110) of [43] we obtain 1) Z t(x,y) t(x,y) kf 0 k1,[a,b] |R4 | ≤ b p(x)q(y) max e p(x)q(y) − ea , eb − e p(x)q(y) dλ(x, y), a (e − e ) R2
(9.45)
2)
|R4 | ≤
kf 0 k1,[a,b] (`n b − `n a)
Z
p(x)q(y) max `n R2
− `n a, `n b − `n
t(x, y) p(x)q(y)
!
3) kf 0 k1,[a,b] |R4 | ≤ √ √ b− a
Z
p(x)q(y) max R2
√ √ − a, b −
s
t(x, y) p(x)q(y)
dλ(x, y),
s
(9.46)
t(x, y) p(x)q(y)
! t(x, y) dλ(x, y), p(x)q(y)
(9.47)
and finally for α > 1 we obtain 4) α Z kf 0 k1,[a,b] t (x, y) tα (x, y) α α |R4 | ≤ α p(x)q(y) max −a ,b − dλ(x, y). b − a α R2 (p(x)q(y))α (p(x)q(y))α (9.48) At last iii) Let α, ˜ β > 1 such that α1˜ + β1 = 1, then by (115) of [43] we have |R1 | ≤ kf 0 kα,[a,b] ˜ s ! Z β+1 + (bp(x)q(y) − t(x, y))β+1 β (t(x, y) − ap(x)q(y)) · dλ(x, y). (β + 1)p(x)q(y) R2 (9.49) Part IV. Here we apply results from [41]. We start with
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
110
Theorem 9.20. Let 0 < a < 1 < b, f as in this chapter and f ∈ C n ([a, b]), n ≥ 1 with |f (n) (s) − f (n) (1)| be a convex function in s. Let 0 < h < min(1 − a, b − 1) be fixed. Then Z n X |f (k) (1)| 1−k k Γf (µXY , µX × µY ) ≤ (pq) (t − pq) dλ k! R2 k=2 Z ω1 (f (n) , h) + (pq)−n |t − pq|n+1 dλ. (9.50) (n + 1)!h R2 Here ω1 is the usual first modulus of continuity. If f (k) (1) = 0, k = 2, . . . , n, then Z ω1 (f (n) , h) (pq)−n |t − pq|n+1 dλ. (9.51) Γf (µXY , µX × µY ) ≤ (n + 1)!h R2 Inequalities (9.50) and (9.51) when n is even are attained by |s − 1|n+1 f˜(s) = , a ≤ s ≤ b. (9.52) (n + 1)! Proof. Apply Theorem 1 of [41].
We continue with the general Theorem 9.21. Let 0 < a ≤ 1 ≤ b, f as in this chapter and f ∈ C n ([a, b]), n ≥ 1. Assume that ω1 (f (n) , δ) ≤ w, where 0 < δ ≤ b − a, w > 0. Let x ∈ R and denote by Z |x| l m s (|x| − s)n−1 ds, (9.53) φn (x) := δ (n − 1)! 0 where d·e is the ceiling of the number. Then Z n X |f (k) (1)| 1−k k (pq) (t − pq) dλ Γf (µXY , µX × µY ) ≤ k! R2 k=2 Z t − pq +w pqφn dλ. (9.54) pq R2 Inequality (9.54) is sharp, namely it is attained by the function f˜n (s) := wφn (s − 1), a ≤ s ≤ b, (9.55) when n is even. Proof. Apply Theorem 2 of [41].
It follows Corollary 9.22 (to Theorem 9.21). It holds (0 < δ ≤ b − a) Z n X |f (k) (1)| 1−k k Γf (µXY , µX × µY ) ≤ (t − pq) dλ 2 (pq) k! R k=2 ( Z 1 + ω1 (f (n) , δ) (pq)−n |t − pq|n+1 dλ (n + 1)!δ R2 Z 1 + (pq)1−n |t − pq|n dλ 2n! R2 ) Z δ 2−n n−1 + (pq) |t − pq| dλ , (9.56) 8(n − 1)! R2
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Csiszar’s f -Divergence as a Measure of Dependence
and Γf (µXY , µX × µY ) ≤
Z n X |f (k) (1)| 1−k k (pq) (t − pq) dλ k! R2 k=2 ( Z 1 (n) (pq)1−n |t − pq|n dλ + ω1 (f , δ) n! R2 ) Z 1 −n n+1 (pq) |t − pq| dλ . + n!(n + 1)δ R2
In particular we have 0
Γf (µXY , µX × µY ) ≤ ω1 (f , δ) 1 + 2 and 0
Γf (µXY , µX × µY ) ≤ ω1 (f , δ)
111
(Z
Z
(
1 2δ
Z
R2
(9.57)
(pq)−1 (t − pq)2 dλ
) δ |t − pq|dλ + , 8 R2
1 |t − pq|dλ + 2δ R2
Z
(pq) R2
−1
(9.58)
2
)
(t − pq) dλ . (9.59)
Proof. By Corollary 1 of [41].
Also we have Corollary 9.23 (to Theorem 9.21). Assume here that f (n) is of Lipschitz type of order α, 0 < α ≤ 1, i.e. ω1 (f (n) , δ) ≤ Kδ α ,
for any 0 < δ ≤ b − a. Then Γf (µXY , µX × µY ) ≤
K > 0,
Z n X |f (k) (1)| 1−k k (pq) (t − pq) dλ k! R2 k=2 Z K + Q (pq)1−n−α |t − pq|n+α dλ. n 2 R (α + i)
(9.60)
(9.61)
i=1
When n is even (9.61) is attained by f ∗ (x) = c|x − 1|n+α , where c := K/ i) > 0. Proof. Application of Corollary 2 of [41].
n Q
(α +
i=1
Next comes Corollary 9.24 (to Theorem 9.21). Assume that Z b−a≥ (pq)−1 t2 dλ − 1 > 0. R2
(9.62)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
112
then
0
Γf (µXY , µX × µY ) ≤ ω1 f ,
Z
(pq)
−1 2
t dλ − 1
R2
Z
1 |t − pq|dλ + 2 R2
.
(9.63)
Proof. Application of Corollary 3 of [41].
Corollary 9.25 (to Theorem 9.21). Assume that Z |t − pq|dλ > 0.
(9.64)
R2
Then
Z Γf (µXY , µX × µY ) ≤ ω1 f 0 ,
Z b−a |t − pq|dλ + |t − pq|dλ 2 R2 R2 Z 3 ≤ (b − a)ω1 f 0 , |t − pq|dλ . (9.65) 2 R2
Proof. Application of Corollary 5 of [41].
We present Proposition 9.26. Let f as in this chapter and f ∈ C([a, b]). i) Assume that Z |t − pq|dλ > 0.
(9.66)
R2
Then
Z Γf (µXY , µX × µY ) ≤ 2ω1 f,
ii) Let r > 0 and
b−a≥r Then Γf (µXY , µX × µY ) ≤
Z
R2
R2
|t − pq|dλ .
(9.67)
|t − pq|dλ > 0.
1 1+ r
ω1 f, r
Z
R2
(9.68)
|t − pq|dλ .
Proof. Application of Proposition 1 of [41].
(9.69)
Also we give Proposition 9.27. Let f as in this setting and f is a Lipschitz function of order α, 0 < α ≤ 1, i.e., there exists K > 0 such that |f (x) − f (y)| ≤ K|x − y|α ,
Then Γf (µXY , µX × µY ) ≤ K
Z
R2
all x, y ∈ [a, b].
(9.70)
(pq)1−α |t − pq|α dλ.
(9.71)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Csiszar’s f -Divergence as a Measure of Dependence
Proof. By Proposition 2 of [41].
113
Next we present some alternative type of results. We start with Theorem 9.28. Let f ∈ C n+1 ([a, b]), n ∈ N, then we get the representation n−1 X (−1)k Z t (k+1) f (pq)−k (t−pq)k+1 dλ+ψ2 , (9.72) Γf (µXY , µX ×µY ) = (k + 1)! R2 pq k=0
where
(−1)n ψ2 = n!
Z
(pq) R2
Z
t pq
1
n (n+1)
(s − 1) f
!
(s)ds dλ.
Proof. By Theorem 12 of [41].
(9.73)
Next we estimate ψ2 . We give n+1 Theorem 9.29. Let ([a, b]), n ∈ N. Then f ∈ C Z (n+1) kf k∞,[a,b] (pq)−n |t − pq|n+1 dλ, B1 := (n + 1)! R2 Z kf (n+1) k1,[a,b] B := (pq)1−n |t − pq|n dλ, 2 n! R2 (9.74) |ψ2 | ≤ min of 1 1 and for α ˜ , β > 1 : + = 1, α ˜ β Z kf (n+1) kα,[a,b] 1 1 ˜ B := (pq)1−n− β |t − pq|n+ β dλ. 3 n!(nβ + 1)1/β R2 Also it holds Z n−1 X 1 t (k+1) −k k+1 Γf (µXY , µX × µY ) ≤ f (pq) (t − pq) dλ (k + 1)! R2 pq k=0
+ min(B1 , B2 , B3 ).
Proof. See Theorem 13 of [41].
(9.75)
We have Corollary 9.30 (to Theorem 9.29). Case of n = 1. It holds Z kf 00 k∞,[a,b] B := (pq)−1 (t − pq)2 dλ, 1,1 2 2 R Z B2,1 := kf 00 k1,[a,b] |t − pq|dλ, R2 |ψ2 | ≤ min of 1 1 and for α ˜ , β > 1, + = 1, α ˜ β Z 00 1 ˜ B3,1 := kf kα,[a,b] (pq)−1/β |t − pq|1+ β dλ. 1/β (β + 1) R2
(9.76)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
114
Also it holds
Z Γf (µXY , µX × µY ) ≤
f
0
R2
Proof. See Corollary 8 of [41].
t (t − pq)dλ + min(B1,1 , B2,1 , B3,1 ). pq
(9.77)
We further present Theorem 9.31. Let f ∈ C n+1 ([a, b]), n ∈ N, then we get the representation n−1 X (−1)k Z t (k+1) Γf (µXY , µX × µY ) = f (pq)−k (t − pq)k+1 dλ (k + 1)! R2 pq k=0 Z (−1)n (n+1) + f (pq)−n (t − pq)n+1 dλ + ψ4 , (9.78) (1) (n + 1)! 2 R where
Z
ψ4 = with ψ3 (x) :=
(−1)n n!
Z
x 1
pqψ3 R2
t pq
dλ,
(s − 1)n (f (n+1) (s) − f (n+1) (1))ds.
Proof. Based on Theorem 14 of [41].
(9.79)
(9.80)
Next we present estimations of ψ4 . Theorem 9.32. Let f ∈ C n+1 ([a, b]), n ∈ N, and Z 1 0< (pq)−n−1 |t − pq|n+2 dλ ≤ b − a. (n + 2) R2
Then (9.78) is again valid and Z 1 1 |ψ4 | ≤ ω1 f (n+1) , (pq)−n−1 |t − pq|n+2 dλ n! n + 2 R2 Z 1 · 1+ (pq)−n |t − pq|n+1 dλ . n + 1 R2
Proof. By Theorem 15 of [41].
(9.81)
(9.82)
Also we have Theorem 9.33. Let f ∈ C n+1 ([a, b]), n ∈ N, and |f (n+1) (s) − f (n+1) (1)| is convex in s and Z 1 0< (pq)−n−1 |t − pq|n+2 dλ ≤ min(1 − a, b − 1). (9.83) n!(n + 2) R2 Then (9.78) is again valid and |ψ4 | ≤ ω1 f (n+1) ,
1 n!(n + 2)
Z
R2
(pq)−n−1 |t − pq|n+2 dλ .
(9.84)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Csiszar’s f -Divergence as a Measure of Dependence
Book
115
The last (9.84) is attained by n+2 ˆ := (s − 1) f(s) , (n + 2)!
a≤s≤b
(9.85)
when n is even. Proof. By Theorem 16 of [41].
When n = 1 we obtain Corollary 9.34 (to Theorem 9.32). Let f ∈ C 2 ([a, b]) and Z 1 (pq)−2 |t − pq|3 dλ ≤ b − a. 0< 3 R2 Then
Γf (µXY , µX × µY ) =
Z
f R2
0
Z t f 00 (1) (t − pq)dλ − (pq)−1 (t − pq)2 dλ + ψ4,1 , pq 2 R2 (9.87)
where ψ4,1 := with ψ3,1 (x) := −
Z
(9.86)
x 1
Z
pqψ3,1 R2
t pq
dλ,
(9.88)
(s − 1)(f 00 (s) − f 00 (1))ds,
x ∈ [a, b].
Furthermore it holds Z Z 1 1 |ψ4,1 | ≤ ω1 f 00 , (pq)−2 |t − pq|3 dλ (pq)−1 t2 dλ + 1 . 3 R2 2 R2 Proof. See Corollary 9 of [41].
(9.89)
(9.90)
Also we have Corollary 9.35 (to Theorem 9.33). Let f ∈ C 2 ([a, b]), and |f (2) (s) − f (2) (1)| is convex in s and Z 1 (pq)−2 |t − pq|3 dλ ≤ min(1 − a, b − 1). (9.91) 0< 3 R2 Then again (9.87) is true and |ψ4,1 | ≤ ω1
1 f , 3 00
Z
(pq)
−2
R2
3
|t − pq| dλ .
Proof. See Corollary 10 of [41].
(9.92)
The last main result of the chapter follows. Theorem 9.36. Let f ∈ C n+1 ([a, b]), n ∈ N. Additionally assume that |f (n+1) (x) − f (n+1) (y)| ≤ K|x − y|α ,
(9.93)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
116
K > 0, 0 < α ≤ 1, all x, y ∈ [a, b] with (x − 1)(y − 1) ≥ 0. Then n−1 X (−1)k Z t (k+1) Γf (µXY , µX × µY ) = f (pq)−k (t − pq)k+1 dλ (k + 1)! R2 pq k=0 Z (−1)n (n+1) f (1) (pq)−n (t − pq)n+1 dλ + ψ4 . (9.94) + (n + 1)! 2 R
Here we find that
|ψ4 | ≤
K n!(n + α + 1)
Z
R2
(pq)−n−α |t − pq|n+α+1 dλ.
(9.95)
Inequality (9.95) is attained when n is even by f ∗ (x) = c˜|x − 1|n+α+1 ,
x ∈ [a, b]
(9.96)
where c˜ := Q n
j=0
K
.
(9.97)
(n + α + 1 − j)
Proof. By Theorem 17 of [41].
Finally we give Corollary 9.37 (to Theorem 9.36). Let f ∈ C 2 ([a, b]). Additionally assume that |f 00 (x) − f 00 (y)| ≤ K|x − y|α , K > 0, 0 < α ≤ 1, all x, y ∈ [a, b]
(9.98)
with (x − 1)(y − 1) ≥ 0. Then Z Z t f 00 (1) 0 Γf (µXY , µX × µY ) = f (t − pq)dλ − (pq)−1 (t − pq)2 dλ + ψ4,1 . pq 2 R2 R2 (9.99) Here Z t ψ4,1 := pqψ3,1 dλ (9.100) pq 2 R with
ψ3,1 (x) := − It holds that
Z
x 1
(s − 1)(f 00 (s) − f 00 (1))ds,
K |ψ4,1 | ≤ (α + 2) Proof. See Corollary 11 of [41].
Z
R2
x ∈ [a, b].
(pq)−1−α |t − pq|α+2 dλ.
(9.101)
(9.102)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Chapter 10
Optimal Estimation of Discrete Csiszar f -Divergence
In this chapter are established a number of sharp and nearly optimal discrete probabilistic inequalities giving various types of order of approximation of Discrete Csiszar f -divergence between two discrete probability measures. This distance is the most essential and general tool for the comparison of these measures. We give also various types of representation of this discrete Csiszar distance and then we estimate tight their remainders, that is leading to very close discrete probabilistic inequalities involving various norms. We give also plenty of interesting applications. This treatment follows [45]. 10.1
Background
The basic general background here has as follows. Let f by a convex function from (0, +∞) into R which is strictly convex at 1 with f (1) = 0. Let (X, A, λ) be a measure space, where λ is a finite or a σ-finite measure on (X, A). Let µ1 , µ2 be two probability measures on (X, A) such that µ1 λ, µ2 λ (absolutely continuous), e.g. λ = µ1 + µ2 . dµ2 1 Denote by p = dµ dλ , q = dλ the (densities) Radon–Nikodym derivatives of µ1 , µ2 with respect to λ Typically we assume that p 0 < a ≤ ≤ b, a.e. on X and a ≤ 1 ≤ b. q The quantity Z p(x) dλ(x), (10.1) Γf (µ1 , µ2 ) = q(x)f q(x) X was introduced by I. Csiszar in 1967, see [140], and is called f -divergence of the probability measures µ1 and µ2 . By Lemma 1.1 of [140], the integral (10.1) is well-defined and Γf (µ1 , µ2 ) ≥ 0 with equality only when µ1 = µ2 . Furthermore by [140], [42] Γf (µ1 , µ2 ) does not depend on the choice of λ. The concept of f -divergence was introduced first in [139] as a generalization of Kullback’s “information for discrimination” or I-divergence 117
Book
22nd June 2009
18:37
118
World Scientific Book - 9.75in x 6.5in
PROBABILISTIC INEQUALITIES
(generalized entropy) [244], [243], and of R´enyi’s “information gain” (I-divergence) of order α [306]. In fact the I-divergence of order 1 equals Γu log2 u (µ1 , µ2 ). The choice f (u) = (u − 1)2 produces again a known measure of difference of distributions that is called χ2 -divergence. Of course the total variation distance Z |µ1 − µ2 | = |p(x) − q(x)| dλ(x) X
is equal to Γ|u−1| (µ1 , µ2 ). Here by assuming f (1) = 0 we can consider Γf (µ1 , µ2 ) the f -divergence as a measure of the difference between the probability measures µ1 , µ2 . This is a chapter where we apply to the discrete case a great variety of interesting results of the above continuous case coming from the author’s earlier but recent articles [37], [42], [43], [41]. Throughout this chapter we use the following. Let again f be a convex function from (0, +∞) into R which is strictly convex at 1 with f (1) = 0. Let X = {x1 , x2 , . . .} be a countable arbitrary set and A be the set of all subsets of X. Then any probability distribution µ on (X, A) is uniquely determined by the sequence P = {p1 , p2 , . . .}, pi = µ({xi }), i = 1, . . . . We call M the set of all distributions on (X, A) such that all pi , i = 1, . . ., are positive. Denote here by µ1 , µ2 ∈ M the distributions with corresponding sequences P = {p1 , . . .}, Q = {q1 , q2 , . . .} (pi > 0, qi > 0, i = 1, 2, . . .). We consider here only the set M ∗ ⊂ M of distributions µ1 , µ2 such that pi 0
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Optimal Estimation of Discrete Csiszar f -Divergence
Book
119
The problem of finding and estimating the proper distance (or difference or discrimination) of two discrete probability distributions is one of the major ones in Probability Theory. The above discrete f -divergence measure (10.3) for various specific f ’s, of finite or infinite sums have been applied a lot to Anthropology, Genetics, Finance, Economics, Political Science, Biology, Approximation of Probability distributions, Signal Processing and Pattern Recognition. In the following the proofs of all formulated and presented results are based a lot on this Section 10.1, especially on the transfer from the continuous to the discrete case. 10.2
Results
Part I. Here we apply results from [37]. We present Theorem 10.1. Let f ∈ C 1 ([a, b]) and µ1 , µ2 ∈ M ∗ . Then ! ∞ X 0 Γf (µ1 , µ2 ) ≤ kf k∞,[a,b] |pi − qi | .
(10.4)
i=1
Inequality (10.4) is sharp. The optimal function is f ∗ (y) = |y − 1|α , α > 1 under the condition, either i) max(b − 1, 1 − a) < 1 and C := sup(Q) < +∞,
(10.5)
or ii) |pi − qi | ≤ qi ≤ 1, Proof. Based on Theorem 1 of [37].
i = 1, 2, . . . .
(10.6)
Next we give the more general Theorem 10.2. Let f ∈ C n+1 ([a, b]), n ∈ N, such that f (k) (1) = 0, k = 0, 2, . . . , n, and µ1 , µ2 ∈ M ∗ . Then ! ∞ kf (n+1) k∞,[a,b] X −n Γf (µ1 , µ2 ) ≤ qi |pi − qi |n+1 . (10.7) (n + 1)! i=1 Inequality (10.7) is sharp. The optimal function is f˜(y) := |y − 1|n+α , α > 1 under the condition (i) or (ii) of Theorem 10.1. Proof. Based on Theorem 2 of [37]. As a special case we have
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
120
Corollary 10.3 (to Theorem 10.2). Let f ∈ C 2 ([a, b]) and µ1 , µ2 ∈ M ∗ . Then ! ∞ kf (2) k∞,[a,b] X −1 2 Γf (µ1 , µ2 ) ≤ qi (pi − qi ) . (10.8) 2 i=1
˜ := |y − 1|1+α , α > 1 under Inequality (10.8) is sharp. The optimal function is f(y) the condition (i) or (ii) of Theorem 10.1. Proof. By Corollary 1 of [37].
Next we connect discrete Csiszar f -divergence to the usual first modulus of continuity ω1 . Theorem 10.4. Assume that 0 < h :=
∞ X i=1
Let µ1 , µ2 ∈ M ∗ . Then
|pi − qi | ≤ min(1 − a, b − 1).
Γf (µ1 , µ2 ) ≤ ω1 (f, h).
(10.9)
(10.10)
Inequality (10.10) is sharp, namely it is attained by f ∗ (x) := |x − 1|. Proof. By Theorem 3 of [37].
Part II. Here we apply results from [42]. But first we need some basic concepts from [122], [26]. Let ν > 0, n := [ν] and α = ν − n (0 < α < 1). Let x, x0 ∈ [a, b] ⊆ R such that x ≥ x0 , where x0 is fixed. Let f ∈ C([a, b]) and define Z x 1 (x − s)ν−1 f (s)ds, x0 ≤ x ≤ b, (10.11) (Jνx0 f )(x) := Γ(ν) x0
the generalized Riemann–Liouville integral, where Γ stands for the gamma function. We define the subspace x0 Dn f ∈ C 1 ([x0 , b]) . (10.12) Cxν0 ([a, b]) := f ∈ C n ([a, b]) : J1−α For f ∈ Cxν0 ([a, b]) we define the generalized ν-fractional derivative of f over [x 0 , b] as d x0 (n) ν D := Dx0 f := DJ1−α f . (10.13) dx We present the following.
Theorem 10.5. Let a < b, 1 ≤ ν < 2, f ∈ Caν ([a, b]), µ1 , µ2 ∈ M ∗ . Then ! ∞ kDaν f k∞,[a,b] X 1−ν Γf (µ1 , µ2 ) ≤ qi (pi − aqi )ν . Γ(ν + 1) i=1 Proof. By Theorem 2 of [42]. The counterpart of the previous result follows.
(10.14)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Optimal Estimation of Discrete Csiszar f -Divergence
Book
121
Theorem 10.6. Let a < b, ν ≥ 2, n := [ν], f ∈ Caν ([a, b]), f (i) (a) = 0, i = 0, 1, . . . , n − 1, µ1 , µ2 ∈ M ∗ . Then ! ∞ kDaν f k∞,[a,b] X 1−ν ν qi (pi − aqi ) . (10.15) Γf (µ1 , µ2 ) ≤ Γ(ν + 1) i=1 Proof. By Theorem 3 of [42].
Next we give an Lα˜ estimate. Theorem 10.7. Let a < b, ν ≥ 1, n := [ν], f ∈ Caν ([a, b]), f (i) (a) = 0, i = 0, 1, . . . , n − 1, µ1 , µ2 ∈ M ∗ . Let α ˜ , β > 1 : α1˜ + β1 = 1. Then ! ∞ X kDaν f kα,[a,b] 1 2−ν− β1 ˜ ν−1+ β q . (10.16) Γf (µ1 , µ2 ) ≤ (pi − aqi ) Γ(ν)(β(ν − 1) + 1)1/β i=1 i Proof. By Theorem 4 of [42].
It follows an L∞ estimate. Theorem 10.8. Let a < b, ν ≥ 1, n := [ν], f ∈ Caν ([a, b]), f (i) (a) = 0, i = 0, 1, . . . , n − 1, µ1 , µ2 ∈ M ∗ . Then ! ∞ kDaν f k1,[a,b] X 2−ν ν−1 Γf (µ1 , µ2 ) ≤ . (10.17) qi (pi − aqi ) Γ(ν) i=1 Proof. Based on Theorem 5 of [42].
We continue with Theorem 10.9. Let f ∈ C 1 ([a, b]), a 6= b, µ1 , µ2 ∈ M ∗ . Then Z b ∞ X 1 pi qi (a + b) Γf (µ1 , µ2 ) ≤ f (x)dx − f0 − pi . b−a a qi 2 i=1 Proof. Based on Theorem 6 of [42].
(10.18)
Furthermore we obtain Theorem 10.10. Let n odd and f ∈ C n+1 ([a, b]), such that f (n+1) ≥ 0 (≤ 0), µ1 , µ2 ∈ M ∗ . Then Z b n i X ∞ X X 1 1 pj (i) f (x)dx − f qj1−i Γf (µ1 , µ2 ) ≤ (≥) b−a a (i + 1)! q j i=1 k=0 j=1 ! · (bqj − pj )(i−k) (aqj − pj )k . (10.19) Proof. Based on Theorem 7 of [42]. Part III. Here we apply results from [43]. We begin with
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
122
Theorem 10.11. Let f, g ∈ C 1 ([a, b]) where f as in this chapter, g 0 6= 0 over [a, b], µ1 , µ2 ∈ M ∗ . Then
0 ! ∞ X
f pi
Γf (µ1 , µ2 ) ≤ qi g − g(1) . (10.20)
g0 qi ∞,[a,b] i=1
Proof. Based on Theorem 1 of [43].
Examples 10.12 (to Theorem 10.11). 1) Let g(x) = x1 . Then
! ∞ X qi |pi − qi | . p i=1 i
2 0
Γf (µ1 , µ2 ) ≤ kx f k∞,[a,b] 2) Let g(x) = ex . Then Γf (µ1 , µ2 ) ≤ ke
f k∞,[a,b]
i=1
3) Let g(x) = e−x . Then x 0
Γf (µ1 , µ2 ) ≤ ke f k∞,[a,b] 4) Let g(x) = ln x. Then
∞ X i=1
0
Γf (µ1 , µ2 ) ≤ kxf k∞,[a,b]
7) Let g(x) = xα , α > 1. Then
0 1 f
Γf (µ1 , µ2 ) ≤
α−1 α x ∞,[a,b]
Next we present
(10.22)
! pi − qi −1 −e . qi e
∞ X i=1
5) Let g(x) = x ln x, with a > e−1 . Then
f0
Γf (µ1 , µ2 ) ≤ 1 + ln x ∞,[a,b] √ 6) Let g(x) = x. Then √ Γf (µ1 , µ2 ) ≤ 2k xf 0 k∞,[a,b]
! pi qi q i e − e .
∞ X
−x 0
(10.21)
(10.23)
! pi qi ln . qi
∞ X i=1
∞ X √ i=1
∞ X i=1
(10.24)
! pi . pi ln qi √
qi | pi −
qi1−α |pα i
√
−
(10.25)
!
qi | .
(10.26)
!
(10.27)
qiα |
.
Theorem 10.13. Let f ∈ C 1 ([a, b]), a 6= b, µ1 , µ2 ∈ M ∗ . Then " Z b kf 0 k∞,[a,b] 1 a2 + b 2 f (x)dx ≤ Γf (µ1 , µ2 ) − b−a a (b − a) 2 # ∞ X −1 2 − (a + b) + qi p i . i=1
(10.28)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Optimal Estimation of Discrete Csiszar f -Divergence
123
Proof. Based on Theorem 2 of [43].
It follows Theorem 10.14. Let f ∈ C (2) ([a, b]), a 6= b, µ1 , µ2 ∈ M ∗ . Then Z b 1 f (b) − f (a) a + b f (x)dx − 1− Γf (µ1 , µ2 ) − b−a a b−a 2 pi 2 a+b 2 ∞ X (b − a)2 00 1 1 qi − 2 ≤ + + . kf k∞,[a,b] qi 2 (b − a)2 4 12
(10.29)
i=1
Proof. Based on (33) of [43].
Working more generally we have Theorem 10.15. Let f ∈ C 1 ([a, b]) and g ∈ C([a, b]) of bounded variation, g(a) 6= g(b), µ1 , µ2 ∈ M ∗ . Then Z b 1 f (x)dg(x) Γf (µ1 , µ2 ) − g(b) − g(a) a (∞ !) Z b X p i 00 P g ≤ kf k∞,[a,b] qi , q(t) dt , (10.30) qi a i=1 where,
P (g(z), g(t)) :=
Proof. By Theorem 5 of [43].
g(t) − g(a) , g(b) − g(a) g(t) − g(b) , g(b) − g(a)
a ≤ t ≤ z, (10.31) z < t ≤ b.
1
x
Example 10.16 (to Theorem 10.15). Let f ∈ C ([a, b]), a 6= b, and g(x) = e , ln x, √ x, xα , α > 1; x > 0. Then by (71)-(74) of [43] we derive 1) Z b 1 x f (x) − e dx Γf (µ1 , µ2 ) − b e − ea a ( ∞ ) X pi kf 0 k∞,[a,b] a b ≤ 2 qi e qi − e (2 − a) + e (b − 2) , (10.32) (eb − ea ) i=1 2)
−1 Z b b f (x) dx Γf (µ1 , µ2 ) − ln a x a ) −1 ( X ∞ b pi 0 ≤ kf k∞,[a,b] ln 2 pi ln − ln(ab) + (a + b − 2) , a qi i=1
(10.33)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
124
3)
Z b f (x) 1 √ dx Γf (µ1 , µ2 ) − √ √ x 2( b − a) a ( ∞ ) √ √ kf 0 k∞,[a,b] 4 X −1/2 3/2 (a3/2 + b3/2 ) qi pi + − ( a + b) , ≤ √ √ 3 ( b − a) 3 i=1
4) (α > 1)
Z b α α−1 f (x)x dx Γf (µ1 , µ2 ) − α (b − aα ) a ( ! ∞ X kf 0 k∞,[a,b] 2 ≤ q −α pα+1 (bα − aα ) (α + 1) i=1 i i α + (aα+1 + bα+1 ) − (aα + bα ) . α+1
(10.34)
(10.35)
We continue with Theorem 10.17. Let f ∈ C (2) ([a, b]), g ∈ C([a, b]) and of bounded variation, g(a) 6= g(b), µ1 , µ2 ∈ M ∗ . Then Z b 1 f (x)dg(x) Γf (µ1 , µ2 ) − (g(b) − g(a)) a !" ∞ Z b !!# Z b X pi 1 0 P g − qi f (x)dg(x) , g(x) dx (g(b) − g(a)) q i a a i=1 !!! Z Z ∞ b b X p i P g ≤ kf 00 k∞,[a,b] qi . , g(x) |P (g(x), g(x1 ))|dx1 dx q i a a i=1
(10.36)
Proof. Apply (79) of [43].
Remark 10.18. Next we define ∗
P (z, s) :=
s − a, s − b,
s ∈ [a, z] s ∈ (z, b].
(10.37)
Let f ∈ C 1 ([a, b]), a 6= b, µ1 , µ2 ∈ M ∗ . Then by (25) of [43] we get the representation ! Z b 1 Γf (µ1 , µ2 ) = f (x)dx + R1 , (10.38) b−a a where by (26) of [43] we have R1 :=
∞ X i=1
qi
Z
b
P a
∗
! pi 0 , x f (x)dx . qi
(10.39)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Optimal Estimation of Discrete Csiszar f -Divergence
125
Let again f ∈ C 1 ([a, b]), g ∈ C([a, b]) and of bounded variation, g(a) 6= g(b), µ1 , µ2 ∈ M ∗ . Then by (47) of [43] we get the representation formula Z b 1 Γf (µ1 , µ2 ) = f (x)dg(x) + R4 , (10.40) g(b) − g(a) a where by (48) of [43] we have R4 := Let α ˜ , β > 1:
1 α ˜
+
1 β
∞ X
qi
i=1
Z
b a
! pi 0 , g(x) f (x)dx . P g qi
(10.41)
= 1, then by (89) of [43] we have !
∞ X
∗ pi 0
|R1 | ≤ kf kα,[a,b] qi ˜
P q i , · β,[a,b] i=1
and by (90) of [43] we obtain
∞ X
0
|R1 | ≤ kf k1,[a,b]
i=1
∗ pi
, · qi P
qi
∞,[a,b]
!
(10.42)
.
(10.43)
Furthermore by (95) and (96) of [43] we derive !
∞ X
p i
|R4 | ≤ kf 0 kα,[a,b] qi ˜
P g qi , g(·) β,[a,b] i=1
and
0
|R4 | ≤ kf k1,[a,b] Remark 10.19. Let a < b. (i) By (105) of [43] we get 0
|R1 | ≤ kf k1,[a,b]
∞ X
!
p i
qi .
P g qi , g(·) ∞,[a,b] i=1
∞ X
qi max
i=1
pi pi − a, b − qi qi
!
, f ∈ C 1 ([a, b]).
(10.44)
(10.45)
(ii) Let g be strictly increasing and continuous over [a, b], e.g. g(x) = ex , ln x, √ x, xα with α > 1, and x > 0 whenever is needed. Also let f ∈ C 1 ([a, b]) and as in this chapter always. Then by (106) of [43] we find ! ∞ X kf 0 k1,[a,b] pi pi qi max g − g(a), g(b) − g . (10.46) |R4 | ≤ (g(b) − g(a)) i=1 qi qi In particular via (107)-(110) of [43] we obtain 1) kf 0 k1,[a,b] |R4 | ≤ b (e − ea )
∞ X i=1
qi max e
pi qi
a
b
−e ,e −e
pi qi
!
,
(10.47)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
126
2) kf 0 k1,[a,b] |R4 | ≤ ln b − ln a 3)
! pi pi qi max ln − ln a, ln b − ln , qi qi i=1
∞ X
kf 0 k1,[a,b] |R4 | ≤ √ √ b− a
∞ X
qi max
i=1
r
pi √ √ − a, b − qi
r
pi qi
!
and finally for α > 1 we derive 4)
kf 0 k1,[a,b] |R4 | ≤ α b − aα
∞ X
qi max
i=1
pα pα i α α i − a , b − qiα qiα
,
!
(10.48)
(10.49)
.
(10.50)
iii) At last let α ˜ , β > 1 such that α1˜ + β1 = 1, then by (115) of [43] we have s !! ∞ β+1 + (bq − p )β+1 X (p − aq ) i i i i β |R1 | ≤ kf 0 kα,[a,b] . (10.51) ˜ (β + 1)qi i=1 Part IV. Here we apply results from [41]. We start with Theorem 10.20. Let 0 < a < 1 < b, f as in this chapter and f ∈ C n ([a, b]), n ≥ 1 with |f (n) (t) − f (n) (1)| be a convex function in t. Let 0 < h < min(1 − a, b − 1) be fixed, µ1 , µ2 ∈ M ∗ . Then n ∞ X |f (k) (1)| X 1−k Γf (µ1 , µ2 ) ≤ qi (pi − qi )k k! i=1 k=2 ! ∞ ω1 (f (n) , h) X −n n+1 . (10.52) q |pi − qi | + (n + 1)!h i=1 i
Here ω1 is the usual first modulus of continuity. If f (k) (1) = 0, k = 2, . . . , n, then ! ∞ ω1 (f (n) , h) X −n n+1 Γf (µ1 , µ2 ) ≤ . (10.53) q |pi − qi | (n + 1)!h i=1 i
Inequalities (10.52) and (10.53) when n is even are attained by |t − 1|n+1 f˜(t) := , (n + 1)! Proof. By Theorem 1 of [41].
a ≤ t ≤ b.
(10.54)
We continue with the general Theorem 10.21. Let f ∈ C n ([a, b]), n ≥ 1, µ1 , µ2 ∈ M ∗ . Assume that ω1 (f (n) , δ) ≤ w, where 0 < δ ≤ b − a, w > 0. Let x ∈ R and denote by Z |x| t (|x| − t)n−1 φn (x) := dt, (10.55) δ (n − 1)! 0
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Optimal Estimation of Discrete Csiszar f -Divergence
127
where d·e is the ceiling of the number. Then ∞ ! ∞ n X X pi − q i |f (k) (1)| X 1−k k qi (pi − qi ) + w q i φn . (10.56) Γf (µ1 , µ2 ) ≤ k! qi i=1
k=2
i=1
Inequality (10.56) is sharp, namely it is attained by the function f˜n (t) := wφn (t − 1),
a ≤ t ≤ b,
(10.57)
when n is even. Proof. By Theorem 2 of [41].
It follows Corollary 10.22 (to Theorem 10.21). Let µ1 , µ2 ∈ M ∗ . It holds (0 < δ ≤ b − a) ! n ∞ X |f (k) (1)| X 1−k k Γf (µ1 , µ2 ) ≤ qi (pi − qi ) k! i=1 k=2 ( ! ∞ X 1 + ω1 (f (n) , δ) q −n |pi − qi |n+1 (n + 1)!δ i=1 i ! ∞ X 1 1−n n + q |pi − qi | 2n! i=1 i !) ∞ X δ , (10.58) + q 2−n |pi − qi |n−1 8(n − 1)! i=1 i
and
∞ n X |f (k) (1)| X 1−k k Γf (µ1 , µ2 ) ≤ qi (pi − qi ) k! i=1 k=2 ! ( ∞ X 1 q 1−n |pi − qi |n + ω1 (f (n) , δ) n! i=1 i !) ∞ X 1 −n n+1 + q |pi − qi | . n!(n + 1)δ i=1 i
In particular we have 0
Γf (µ1 , µ2 ) ≤ ω1 (f , δ) and 0
(
) ∞ ∞ 1 X −1 1 X δ 2 q (pi − qi ) + |pi − qi | + , (10.60) 2δ i=1 i 2 i=1 8
Γf (µ1 , µ2 ) ≤ ω1 (f , δ)
(
∞ X i=1
|pi − qi |
Proof. Based on Corollary 1 of [41]. Also we have
(10.59)
!
1 + 2δ
∞ X i=1
qi−1 (pi
− qi )
2
!)
.
(10.61)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
128
Corollary 10.23 (to Theorem 10.21). Let µ1 , µ2 ∈ M ∗ . Assume here that f (n) is of Lipschitz type of order α, 0 < α ≤ 1, i.e. ω1 (f (n) , δ) ≤ Kδ α ,
K > 0,
for any 0 < δ ≤ b − a. Then ∞ n X |f (k) (1)| X 1−k K k Γf (µ1 , µ2 ) ≤ qi (pi − qi ) + Q n k! i=1 k=2 (α + i) i=1
∗
When n is even (10.63) is attained by f (x) = c|x − 1| i) > 0.
(10.62)
∞ X i=1
n+α
qi1−n−α |pi
− qi |
n+α
!
(10.63) n Q , where c := K/ (α + i=1
Proof. Based on Corollary 2 of [41].
Corollary 10.24 (to Theorem 10.21). Let µ1 , µ2 ∈ M ∗ . Suppose that b−a≥ Then Γf (µ1 , µ2 ) ≤ ω1
0
f,
∞ X
∞ X i=1
qi−1 p2i
i=1
qi−1 p2i − 1 > 0.
−1
(10.64)
!! ( ∞ X
1 |pi − qi | + 2 i=1
)
.
(10.65)
Proof. Based on Corollary 3 of [41].
Corollary 10.25 (to Theorem 10.21). Let µ1 , µ2 ∈ M ∗ . Suppose that ∞ X i=1
Then Γf (µ1 , µ2 ) ≤ ω1
0
f,
∞ X i=1
3 ≤ (b − a)ω1 2
|pi − qi | > 0.
|pi − qi | f 0,
!! (
∞ X i=1
(10.66)
∞ X i=1
|pi − qi |
|pi − qi |
!!
Proof. Based on Corollary 5 of [41].
.
!
b−a + 2
) (10.67)
We give Proposition 10.26. Let f ∈ C([a, b]), µ1 , µ2 ∈ M ∗ . i) Assume that ∞ X i=1
|pi − qi | > 0.
(10.68)
.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Optimal Estimation of Discrete Csiszar f -Divergence
Then Γf (µ1 , µ2 ) ≤ 2ω1
∞ X
f,
i=1
|pi − qi |
!!
129
.
(10.69)
ii) Let r > 0 and ∞ X
b−a≥r
i=1
|pi − qi |
!
> 0.
(10.70)
Then Γf (µ1 , µ2 ) ≤
1 1+ r
ω1
f, r
∞ X i=1
|pi − qi |
!!
.
(10.71)
Proof. Based on Proposition 1 of [41].
Also we give Proposition 10.27. Let f as in this setting and f is a Lipschitz function of order α, 0 < α ≤ 1, i.e. there exists K > 0 such that |f (x) − f (y)| ≤ K|x − y|α , all x, y ∈ [a, b]. Let µ1 , µ2 ∈ M ∗ . Then ∞ X
Γf (µ1 , µ2 ) ≤ K
i=1
qi1−α |pi
− qi |
α
!
(10.72)
.
(10.73)
Proof. Based on Proposition 2 of [41].
Next we present some alternative type of results. We start with Theorem 10.28. All elements involved here as in this chapter and f ∈ C n+1 ([a, b]), n ∈ N. Let µ1 , µ2 ∈ M ∗ . Then ! ∞ n−1 X X (−1)k −k (k+1) pi k+1 f q (pi − qi ) + ψ2 , (10.74) Γf (µ1 , µ2 ) = (k + 1)! i=1 qi i k=0
where (−1)n ψ2 := n!
∞ X i=1
Proof. By Theorem 12 of [41]. Next we estimate ψ2 . We give
qi
Z
pi qi
1
n (n+1)
(t − 1) f
(t)dt
!!
.
(10.75)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
130
Theorem 10.29. All here as in Theorem 10.28. Then ! ∞ kf (n+1) k∞,[a,b] X −n n+1 B1 := qi |pi − qi | , (n + 1)! i=1 ! ∞ (n+1) X kf k 1,[a,b] qi1−n |pi − qi |n , B := 2 n! i=1 |ψ2 | ≤ min of 1 1 and for α, ˜ β > 1 : + = 1, α ˜ β ! ∞ X 1 kf (n+1) kα,[a,b] 1 1−n− β ˜ n+ β qi . |pi − qi | B3 := n!(nβ + 1)1/β i=1
(10.76)
Also it holds
Γf (µ1 , µ2 ) ≤
n−1 X k=0
∞ X 1 −k k+1 (k+1) pi qi (pi − qi ) f + min(B1 , B2 , B3 ). (k + 1)! i=1 qi (10.77)
Proof. By Theorem 13 of [41].
Corollary 10.30 (to Theorem 10.29). Case of n = 1. It holds ! ∞ kf 00 k∞,[a,b] X −1 2 B1,1 := qi (pi − qi ) , 2 i=1 ! ∞ X 00 B := kf k1,[a,b] |pi − qi | , 2,1 i=1 |ψ2 | ≤ min of 1 1 and for α ˜ , β > 1 : + = 1, α ˜ β ! ∞ X 1 kf 00 kα,[a,b] 1 −β ˜ 1+ β B3,1 := q |pi − qi | . (β + 1)1/β i=1 i
(10.78)
Also it holds
∞ X p i Γf (µ1 , µ2 ) ≤ f0 (pi − qi ) + min(B1,1 , B2,1 , B3,1 ). qi
(10.79)
i=1
Proof. By Corollary 8 of [41]. We further give
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Optimal Estimation of Discrete Csiszar f -Divergence
131
Theorem 10.31. Let all elements involved as in this chapter, f ∈ C n+1 ([a, b]), n ∈ N and µ1 , µ2 ∈ M ∗ . Then ! n−1 ∞ X (−1)k X −k (k+1) pi k+1 Γf (µ1 , µ2 ) = f q (pi − qi ) (k + 1)! i=1 qi i k=0 ! ∞ X (−1)n (n+1) −n n+1 f (1) qi (pi − qi ) + ψ4 , (10.80) + (n + 1)! i=1 where
ψ4 :=
∞ X
q i ψ3
ψ3 (x) :=
(−1)n n!
Z
x 1
pi qi
,
(10.81)
(s − 1)n (f (n+1) (s) − f (n+1) (1))ds.
(10.82)
i=1
with
Proof. Based on Theorem 14 of [41].
Next we present estimations of ψ4 . Theorem 10.32. Let all elements involved here as in this chapter and f ∈ C n+1 ([a, b]), n ∈ N, µ1 , µ2 ∈ M ∗ and ! ∞ X 1 −n−1 n+2 0< ≤ b − a. (10.83) q |pi − qi | (n + 2) i=1 i Then (10.80) is again valid and 1 |ψ4 | ≤ ω1 n!
f
(n+1)
∞ X
1 , (n + 2)
1 · 1+ (n + 1)
i=1
∞ X i=1
qi−n−1 |pi
qi−n |pi
− qi |
− qi |
n+1
n+2
!!
!!
.
(10.84)
Proof. Based on Theorem 15 of [41].
Also we have Theorem 10.33. All elements involved here as in this chapter and f ∈ C n+1 ([a, b]), n ∈ N, µ1 , µ2 ∈ M ∗ . Furthermore we assume that |f (n+1) (t) − f (n+1) (1)| is convex in t and ! ∞ X 1 −n−1 n+2 0< q |pi − qi | ≤ min(1 − a, b − 1). (10.85) n!(n + 2) i=1 i Then (10.80) is again valid and |ψ4 | ≤ ω1
f
(n+1)
1 , n!(n + 2)
∞ X i=1
qi−n−1 |pi
− qi |
n+2
!!
.
(10.86)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
132
The last (10.86) is attained by (s − 1)n+2 fˆ(s) := , (n + 2)!
a ≤ s ≤ b,
(10.87)
when n is even. Proof. By Theorem 16 of [41].
When n = 1 we obtain Corollary 10.34 (to Theorem 10.32). Let f ∈ C 2 ([a, b]), µ1 , µ2 ∈ M ∗ and ! ∞ 1 X −2 3 q |pi − qi | ≤ b − a. (10.88) 0< 3 i=1 i Then
Γf (µ1 , µ2 ) =
∞ X
f
i=1
where
0
f 00 (1) pi (pi − qi ) − qi 2
ψ4,1 :=
∞ X
∞ X i=1
qi ψ3,1
i=1
with ψ3,1 (x) := − Furthermore it holds |ψ4,1 | ≤ ω1
1 f 00 , 3
Z
x 1
∞ X i=1
qi−1 (pi
pi qi
− q i |3
!!
1 2
!
+ ψ4,1 ,
,
∞ X
(10.89)
(10.90)
(t − 1)(f 00 (t) − f 00 (1))dt,
qi−2 |pi
− qi )
2
x ∈ [a, b].
qi−1 p2i
i=1
!
Proof. Based on Corollary 9 of [41].
!
+1 .
(10.91)
(10.92)
Also we have Corollary 10.35 (to Theorem 10.33). Let f ∈ C 2 ([a, b]) and |f 00 (t) − f 00 (1)| is convex in t. Here µ1 , µ2 ∈ M ∗ such that ! ∞ 1 X −2 0< q |pi − qi |3 ≤ min(1 − a, b − 1). (10.93) 3 i=1 i Then again (10.89) is valid and |ψ4,1 | ≤ ω1
1 f , 3 00
∞ X i=1
qi−2 |pi
Proof. By Corollary 10 of [41]. The last main result of the chapter follows.
− qi |
3
!!
.
(10.94)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Optimal Estimation of Discrete Csiszar f -Divergence
133
Theorem 10.36. All elements involved here in this chapter and f ∈ C n+1 ([a, b]), n ∈ N, µ1 , µ2 ∈ M ∗ . Additionally we assume that |f (n+1) (x) − f (n+1) (y)| ≤ K|x − y|α ,
(10.95)
K > 0, 0 < α ≤ 1, all x, y ∈ [a, b] with (x − 1)(y − 1) ≥ 0. Then ! ∞ n−1 X X (−1)k pi −k (k+1) k+1 f qi (pi − qi ) Γf (µ1 , µ2 ) = (k + 1)! i=1 qi k=0 ! ∞ X (−1)n (n+1) −n n+1 + f (1) qi (pi − qi ) + ψ4 . (n + 1)! i=1
Here it holds
K |ψ4 | ≤ n!(n + α + 1)
∞ X i=1
qi−n−α |pi
− qi |
n+α+1
!
.
(10.96)
(10.97)
Inequality (10.97) is attained when n is even by f ∗ (x) = c˜|x − 1|n+α+1 ,
where
c˜ := Q n
j=0
x ∈ [a, b]
K
(10.98)
.
(10.99)
(n + α + 1 − j)
Proof. By Theorem 17 of [41].
Next we give Corollary 10.37 (to Theorem 10.36). Let f ∈ C 2 ([a, b]), µ1 , µ2 ∈ M ∗ . Additionally we suppose that |f 00 (x) − f 00 (y)| ≤ K|x − y|α ,
(10.100)
K > 0, 0 < α ≤ 1, all x, y ∈ [a, b] with (x − 1)(y − 1) ≥ 0. Then ! ∞ ∞ 00 X X p f (1) i −1 Γf (µ1 , µ2 ) = f0 (pi − qi ) − qi (pi − qi )2 + ψ4,1 . qi 2 i=1 i=1 Here
ψ4,1 :=
∞ X
qi ψ3,1
i=1
with ψ3,1 (x) := − It holds that
Z
1
x
pi qi
(t − 1)(f 00 (t) − f 00 (1))dt,
K |ψ4,1 | ≤ (α + 2) Proof. By Corollary 11 of [41].
∞ X i=1
qi−1−α |pi
(10.101)
(10.102)
x ∈ [a, b].
− qi |
α+2
!
.
(10.103)
(10.104)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
This page intentionally left blank
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Chapter 11
About a General Discrete Measure of Dependence
In this chapter the discrete Csiszar’s f -Divergence or discrete Csiszar’s Discrimination, the most general distance among discrete probability measures, is established as the most general measure of Dependence between two discrete random variables which is a very important matter of stochastics. Many discrete estimates of it are given, leading to optimal or nearly optimal discrete probabilistic inequalities. That is we are comparing and connecting this general discrete measure of dependence to known other specific discrete entities by involving basic parameters of our setting. This treatment is based on [38]. 11.1
Background
The basic general background here has as follows. Let f by a convex function from (0, +∞) into R which is strictly convex at 1 with f (1) = 0. Let (Ω, A, λ) be a measure space, where λ is a finite or a σ-finite measure on (Ω, A). Let µ1 , µ2 be two probability measures on (Ω, A) such that µ1 λ, µ2 λ (absolutely continuous), e.g. λ = µ1 + µ2 . dµ2 1 Denote by p = dµ dλ , q = dλ the (densities) Radon–Nikodym derivatives of µ1 , µ2 with respect to λ. Typically we assume that p 0 < a ≤ ≤ b, a.e. on Ω and a ≤ 1 ≤ b. q The quantity Γf (µ1 , µ2 ) =
Z
q(x)f Ω
p(x) q(x)
dλ(x),
(11.1)
was introduced by I. Csiszar in 1967, see [140], and is called f -divergence of the probability measures µ1 and µ2 . By Lemma 1.1 of [140], the integral (11.1) is well-defined and Γf (µ1 , µ2 ) ≥ 0 with equality only when µ1 = µ2 . Furthermore by [140], [42] Γf (µ1 , µ2 ) does not depend on the choice of λ. The concept of f -divergence was introduced first in [139] as a 135
22nd June 2009
18:37
136
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
generalization of Kullback’s “information for discrimination” or I-divergence (generalized entropy) [244], [243], and of R´enyi’s “information gain” (I-divergence) of order α [306]. In fact the I-divergence of order 1 equals Γu log2 u (µ1 , µ2 ). The choice f (u) = (u − 1)2 produces again a known measure of difference of distributions that is called χ2 -divergence. Of course the total variation distance Z |µ1 − µ2 | = |p(x) − q(x)| dλ(x) Ω
is equal to Γ|u−1| (µ1 , µ2 ). Here by assuming f (1) = 0 we can consider Γf (µ1 , µ2 ) the f -divergence as a measure of the difference between the probability measures µ1 , µ2 . This is an expository, survey and applications chapter where we apply to the discrete case a large variety of interesting results of the above continuous case coming from the author’s papers [37], [42], [43], [41], [39]. Namely here we present a complete study of discrete Csiszar’s f -divergence as a measure of dependence of two discrete random variables. Throughout this chapter we use the following. Let again f be a convex function from (0, +∞) into R which is strictly convex at 1 with f (1) = 0. Let the probability space (Ω, P ) and X, Y be discrete random variables from Ω into R such that their ranges RX = {x1 , x2 , . . .}, RY = {y1 , y2 , . . .}, respectively, are finite or countable subsets of R. Let the probability functions of X and Y be pi = p(xi ) = P (X = xi ),
i = 1, 2, . . . ,
qi = q(yj ) = P (Y = yj ),
j = 1, 2, . . . ,
(11.2)
respectively. Without loss of generality we assume that all pi , qj > 0, i, j = 1, 2, . . . . Consider the probability distributions µXY and µX × µY on RX × RY , where µXY , µX , µY stand for the joint probability distribution of X and Y and their marginal distributions, respectively, while µX × µY is the product probability distribution of X and Y . Here µXY is uniquely determined by the double sequence tij = t(xi , yj ) = P (X = xi , Y = yj ) = µXY ({(xi , yj )}),
i, j = 1, 2, . . . ,
(11.3)
which t is the probability function of (X, Y ). Again without loss of generality we assume that tij > 0 for all i, j = 1, 2, . . . . Clearly also here µX , µY are uniquely determined by pi = µX ({xi }), qj = µY ({yj }), respectively, i, j = 1, 2, . . . . It is obvious that the product probability distribution µX × µY has associated probability function pq. In this article we assume that tij 0
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
About a General Discrete Measure of Dependence
137
where a, b are fixed. Let P be the set of all subsets of RX × RY . We define λ to be a finite or σ-finite measure on (RX × RY , P) such that λ({(xi , yi )}) = 1, for all i, j = 1, 2, . . . . We notice that µXY and µX × µY are discrete probability measures on (RX × RY , P) and are absolutely continuous ( ) with respect to λ. By applying (11.1) we obtain the special case ∞ X ∞ X tij Γf (µXY , µX × µY ) = p i qj f . (11.5) p i qj i=1 j=1 This is the discrete Csiszar’s distance or f -divergence between µXY and µX × µY . The random variables X, Y are the less dependent the closer the distributions µXY and µX × µY are, thus Γf (µXY , µX × µY ) can be considered as a measure of dependence of X and Y , see also [305], [140], [39]. From the above derives that Γf (µXY , µX × µY ) ≥ 0 with equality only when µXY = µX × µY , i.e. when X, Y are independent discrete random variables. As related references see here [139], [140], [141], [152], [244], [243], [305], [306], [37], [42], [43], [41], [39]. For f (u) = u log2 u we obtain the mutual information of X and Y , I(X, Y ) = I(µXY k µX × µY ) = Γu log2 u (µXY , µX × µY ), see [140]. For f (u) = (u − 1)2 we get the mean square contingency: ϕ2 (X, Y ) = Γ(u−1)2 (µXY , µX × µY ), see [140]. In Information Theory and Statistics many various divergences are used which are special cases of the above Γf divergence. Γf has many applications also in Anthropology, Genetics, Finance, Economics, Political Science, Biology, Approximation of Probability Distributions, Signal Processing and Pattern Recognition. In the next the proofs of all formulated and presented results are based a lot on this Section 11.1 especially on the transfer from the continuous to the discrete case. 11.2
Results
Part I. Here we apply results from [37], [39]. We present the following Theorem 11.1. Let f ∈ C 1 ([a, b]). Then
Γf (µXY , µX × µY ) ≤ kf 0 k∞,[a,b]
∞ X ∞ X i=1 j=1
|tij − pi qj | .
(11.6)
Inequality (11.6) is sharp. The optimal function is f ∗ (y) = |y − 1|α , α > 1 under the condition, either
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
138
i) max(b − 1, 1 − a) < 1 and C := sup (pi qj ) < +∞,
(11.7)
i,j∈N
or ii) |tij − pi qj | ≤ pi qj ≤ 1,
all i, j = 1, 2, . . . .
Proof. Based on Theorem 1 of [37] and Theorem 1 of [39].
(11.8)
Next we give the more general Theorem 11.2. Let f ∈ C n+1 ([a, b]), n ∈ N, such that f (k) (1) = 0, k = 0, 2, 3, . . . , n. Then ∞ ∞ kf (n+1) k∞,[a,b] X X −n −n Γf (µXY , µX × µY ) ≤ pi qj |tij − pi qj |n+1 . (11.9) (n + 1)! i=1 j=1
Inequality (11.9) is sharp. The optimal function is f˜(y) := |y − 1|n+α , α > 1 under the condition (i) or (ii) of Theorem 11.1. Proof. As in Theorems 2 of [37], [39], respectively.
As a special case we have Corollary 11.3 (to Theorem 11.2). Let f ∈ C 2 ([a, b]). Then ∞ X ∞ kf (2) k∞,[a,b] X −1 2 p−1 . Γf (µXY , µX × µY ) ≤ i qj (tij − pi qj ) 2 i=1 j=1
(11.10)
Inequality (11.10) is sharp. The optimal function is f˜(y) := |y −1|1+α , α > 1 under the condition (i) or (ii) of Theorem 11.1.
Proof. By Corollaries 1 of [37] and [39].
Next we connect Csiszar’s f -divergence to the usual first modulus of continuity ω1 . Theorem 11.4. Suppose that ∞ X ∞ X 0 < h := |tij − pi qj | ≤ min(1 − a, b − 1).
(11.11)
i=1 j=1
Then
Γf (µXY , µX × µY ) ≤ ω1 (f, h).
(11.12)
Inequality (11.12) is sharp, i.e. attained by f ∗ (x) = |x − 1|. Proof. By Theorems 3 of [37] and [39].
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
About a General Discrete Measure of Dependence
Book
139
Part II. Here we apply results from [42] and [39]. But first we need to give some basic background from [122], [26]. Let ν > 0, n := [ν] and α = ν − n (0 < α < 1). Let x, x0 ∈ [a, b] ⊆ R such that x ≥ x0 , where x0 is fixed and f ∈ C([a, b]) and define Z x 1 (x − s)ν−1 f (s)ds, x0 ≤ x ≤ b, (11.13) (Jνx0 f )(x) := Γ(ν) x0 the generalized Riemann–Liouville integral, where Γ stands for the gamma function. We define the subspace x0 f (n) ∈ C 1 ([x0 , b]) . (11.14) Cxν0 ([a, b]) := f ∈ C n ([a, b]) : J1−α For f ∈ Cxν0 ([a, b]) we define the generalized ν-fractional derivative of f over [x 0 , b] as d x0 (n) ν D := Dx0 f = DJ1−α f . (11.15) dx We present the following Theorem 11.5. Let a < b, 1 ≤ ν < 2, f ∈ Caν ([a, b]). Then ∞ ν X kDa f k∞,[a,b] Γf (µXY , µX × µY ) ≤ (pi qj )1−ν (tij − api qj )ν . Γ(ν + 1) i,j=1 Proof. It follows from Theorem 2 of [42] and Theorem 4 of [39].
(11.16)
The counterpart of the previous result comes next. Theorem 11.6. Let a < b, ν ≥ 2, n := [ν], f ∈ Caν ([a, b]), f (i) (a) = 0, i = 0, 1, . . . , n − 1. Then ∞ kDaν f k∞,[a,b] X Γf (µXY , µX × µY ) ≤ (pi qj )1−ν (tij − api qj )ν . (11.17) Γ(ν + 1) i,j=1 Proof. By Theorem 3 of [42] and Theorem 5 of [39].
Next we give an Lα˜ estimate. Theorem 11.7. Let a < b, ν ≥ 1, n := [ν], f ∈ Caν ([a, b]), f (i) (a) = 0, i = 0, 1, . . . , n − 1 and let α ˜ , β > 1 : α1˜ + β1 = 1. Then ∞ X kDaν f kα,[a,b] 1 1 ˜ Γf (µXY , µX ×µY ) ≤ (pi qj )2−ν− β (tij − api qj )ν−1+ β . Γ(ν)(β(ν − 1) + 1)1/β i,j=1 (11.18)
Proof. Based on Theorem 4 of [42] and Theorem 6 of [39]. It follows an L∞ estimate.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
140
Theorem 11.8. Let a < b, ν ≥ 1, n := [ν], f ∈ Caν ([a, b]), f (i) (a) = 0, i = 0, 1, . . . , n − 1. Then ∞ kDaν f k1,[a,b] X (pi qj )2−ν (tij − api qj )ν−1 . (11.19) Γf (µXY , µX × µY ) ≤ Γ(ν) i,j=1 Proof. Based on Theorem 5 of [42] and Theorem 7 of [39].
We continue with Theorem 11.9. Let f ∈ C 1 ([a, b]), a < b. Then Z b ∞ X 1 tij pi qj (a + b) Γf (µXY , µX × µY ) ≤ f (x)dx − f0 − tij . b−a a p q 2 i j i,j=1
(11.20)
Proof. Based on Theorem 6 of [42] and Theorem 8 of [39].
Furthermore we get Theorem 11.10. Let n odd and f ∈ C n+1 ([a, b]), a < b, such that f (n+1) ≥ 0 (≤ 0). Then Z b 1 Γf (µXY , µX × µY ) ≤ (≥) f (x)dx b−a a " i n ∞ X ∞ X X X 1 t`j (i) f − (i + 1)! p ` qj i=1 k=0 `=1 j=1 !# · (p` qj )1−i (bp` qj − t`j )(i−k) (ap` qj − t`j )k
Proof. Based on Theorem 7 of [42] and Theorem 9 of [39].
.
(11.21)
Part III. Here we apply results from [43] and [39]. We start with Theorem 11.11. Let f, g ∈ C 1 ([a, b]) where f as in this chapter, g 0 6= 0 over [a, b]. Then
0 ∞ X
f t ij
Γf (µXY , µX × µY ) ≤ pi qj g − g(1) . (11.22)
g0 p q i j ∞,[a,b] i,j=1
Proof. Based on Theorem 1 of [43] and Theorem 10 of [39]. We give Examples 11.12 (to Theorem 11.11).
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
About a General Discrete Measure of Dependence
1) Let g(x) = x1 . Then
∞ X p q i j Γf (µXY , µX × µY ) ≤ kx2 f 0 k∞,[a,b] |tij − pi qj | . t i,j=1 ij
2) Let g(x) = ex . Then
Γf (µXY , µX × µY ) ≤ ke−xf 0 k∞,[a,b]
3) Let g(x) = e−x . Then
Γf (µXY , µX × µY ) ≤ kex f 0 k∞[a,b]
4) Let g(x) = ln x. Then
∞ X
i,j=1
∞ X
i,j=1
Γf (µXY , µX × µY ) ≤ kxf 0 k∞,[a,b]
t /p q pi qj e ij i j − e .
−t /p q pi qj e ij i j − e−1 .
∞ X
i,j=1
t ij . pi qj ln p i qj
5) Let g(x) = x ln x, with a > e−1 . Then
∞ X
f0 t ij
. Γf (µXY , µX × µY ) ≤ tij ln
1 + ln x p q i j ∞,[a,b] i,j=1 6) Let g(x) =
√
Book
141
(11.23)
(11.24)
(11.25)
(11.26)
(11.27)
x. Then
∞ X p √ 0 √ √ Γf (µXY , µX × µY ) ≤ 2k xf k∞,[a,b] pi qj tij − pi qj .
(11.28)
i,j=1
7) Let g(x) = xα , α > 1. Then
0 ∞ X
1 f α
Γf (µXY , µX × µY ) ≤ (pi qj )1−α |tα ij − (pi qj ) | . α xα−1 ∞,[a,b] i,j=1
(11.29)
Next we give
Theorem 11.13. Let f ∈ C 1 ([a, b]), a < b. Then Z b 1 f (x)dx Γf (µXY , µX × µY ) − b−a a 2 ∞ 2 X kf 0 k∞,[a,b] a + b ≤ − (a + b) + (pi qj )−1 t2ij . (b − a) 2 i,j=1 Proof. By Theorem 2 of [43] and Theorem 11 of [39]. It follows
(11.30)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
142
Theorem 11.14. Let f ∈ C (2) ([a, b]), a < b. Then Z b 1 f (b) − f (a) a + b f (s)ds − 1− Γf (µXY , µX × µY ) − b−a a b−a 2 2 tij a+b 2 ∞ X (b − a)2 00 1 1 pi q j − 2 kf k∞,[a,b] p i qj + + . ≤ 2 (b − a)2 4 i,j=1 12
(11.31)
Proof. Based on (33) of [43] and Theorem 12 of [39].
Working more generally we have Theorem 11.15. Let f ∈ C 1 ([a, b]) and g ∈ C([a, b]) of bounded variation, g(a) 6= g(b). Then Z b 1 f (s)dg(s) Γf (µXY , µX × µY ) − g(b) − g(a) a ! Z b ∞ X P g tij , g(s) ds ≤ kf 00 k∞,[a,b] p i qj , (11.32) p i qj i,j=1
where
P (g(z), g(s)) :=
a
g(s) − g(a) , g(b) − g(a) g(s) − g(b) , g(b) − g(a)
a ≤ s ≤ z,
(11.33) z < s ≤ b.
Proof. By Theorem 5 of [43] and Theorem 13 of [39].
We give
√ Examples 11.16 (to Theorem 11.15). Let f ∈ C 1 ([a, b]) and g(x) = ex , ln x, x, xα , α > 1; x > 0. Then by (71)–(74) of [43] and (30)–(33) of [39] we obtain 1) Z b 1 s f (s)e ds Γf (µXY , µX × µY ) − b e − ea a ∞ kf 0 k∞,[a,b] X tij /pi qj a b ≤ 2 p q e − e (2 − a) + e (b − 2) , (11.34) i j (eb − ea ) i,j=1
2) −1 Z b f (s) b ds Γf (µXY , µX × µY ) − ln a s a −1 ∞ X t b ij − ln(ab) + (a + b − 2) , ≤ kf 0 k∞,[a,b] ln 2 tij ln a p i qj i,j=1
(11.35)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
About a General Discrete Measure of Dependence
3)
143
Z b f (s) 1 √ ds Γf (µXY , µX × µY ) − √ √ s 2( b − a) a ∞ 0 3/2 3/2 X √ √ kf k∞[a,b] 4 (a + b ) 3/2 ≤ √ (pi qj )−1/2 tij + − ( a + b) , √ 3 ( b − a) 3 i,j=1
(11.36)
4) (α > 1)
Z b α α−1 f (s)s ds Γf (µXY , µX × µY ) − α (b − aα ) a ( ∞ X kf 0 k∞,[a,b] 2 ≤ (pi qj )−α tα+1 ij (bα − aα ) (α + 1) i,j=1 +
) α α+1 α+1 α α (a +b ) − (a + b ) . α+1
(11.37)
We continue with Theorem 11.17. Let f ∈ C (2) ([a, b]), g ∈ C([a, b]) and of bounded variation, g(a) 6= g(b). Then Γf (µXY , µX × µY ) ! Z b Z b 1 1 0 − f (s)dg(s) − f (s1 )dg(s1 ) (g(b) − g(a)) a (g(b) − g(a)) a ( ∞ ! Z b X tij · p i qj P g , g(s) ds p q i j a i,j=1 !!) ( ∞ Z Z b b X tij 00 . ≤ kf k∞,[a,b] p i qj P g pi qj , g(s) |P (g(s), g(s1 ))|ds1 ds a a i,j=1
(11.38)
Proof. Apply (79) of [43] and Theorem 14 of [39]. Remark 11.18. Next we define ∗
P (z, s) :=
s − a, s − b,
s ∈ [a, z], s ∈ (z, b].
Let f ∈ C 1 ([a, b]). Then by (25) of [43] we get the representation ! Z b 1 Γf (µXY , µX × µY ) = f (s)ds + R1 , b−a a
(11.39)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
144
where according to (26) of [43] and (36) of [39] we have ! Z b ∞ X tij ∗ 0 , s f (s)ds . R1 = p i qj P p i qj a i,j=1
(11.40)
where by (48) of [43] and (38) of [39] we obtain ! Z b ∞ X tij 0 , g(s) f (s)ds . R4 = p i qj P g p i qj a i,j=1
(11.42)
Let again f ∈ C 1 ([a, b]), g ∈ C([a, b]) and of bounded variation, g(a) 6= g(b). Then by (47) of [43] we get the representation Z b 1 Γf (µXY , µX × µY ) = f (s)dg(s) + R4 , (11.41) (g(b) − g(a)) a
Let α ˜, β > 1 :
1 α ˜
+
1 β
= 1, then by (89) of [43] and (39) of [39] we have
∞ X
tij ∗
, |R1 | ≤ kf 0 kα,[a,b] p i qj ˜
P p i q j , · β,[a,b] i,j=1
and by (90) of [43] and (40) of [39] we get
∞ X
tij ∗
. |R1 | ≤ kf 0 k1,[a,b] p i qj
P p i q j , · ∞,[a,b] i,j=1 Furthermore by (95), (96) of [43] and (41), (42) of [39] we find
∞ X
tij
, |R4 | ≤ kf 0 kα,[a,b] p i qj ˜
P g pi qj , g(·) β,[a,b] i,j=1
and
t ij
, |R4 | ≤ kf 0 k1,[a,b] p i qj
P g pi qj , g(·) ∞,[a,b] i,j=1 ∞ X
Remark 11.19. Let a < b. (i) Here by (105) of [43] and (43) of [39] we obtain ∞ X t t ij ij , − a, b − |R1 | ≤ kf 0 k1,[a,b] pi qj max p q p qj i j i i,j=1
(11.43)
(11.44)
(11.45)
(11.46)
f ∈ C 1 ([a, b]). (11.47)
(ii) Let g be strictly increasing and continuous over [a, b], e.g., g(x) = ex , ln x, √ x, xα with α > 1, and x > 0 whenever is needed. Also f ∈ C 1 ([a, b]). Then by (106) of [43] and (44) of [39] we find ∞ X kf 0 k1,[a,b] t t ij ij . |R4 | ≤ pi qj max g − g(a), g(b) − g (g(b) − g(a)) i,j=1 p i qj p i qj
(11.48)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
About a General Discrete Measure of Dependence
In particular via (107)–(110) of [43] and (45)–(48) of [39] we obtain 1) ∞ 0 X kf k1,[a,b] |R4 | ≤ b pi qj max etij /pi qj − ea , eb − etij /pi qj , (e − ea ) i,j=1
145
(11.49)
2)
∞ X kf 0 k1,[a,b] t t ij ij , pi qj max ln − ln a, ln b − ln |R4 | ≤ (ln b − ln a) i,j=1 p i qj p i qj
(11.50)
3)
! s s ∞ √ √ kf 0 k1,[a,b] X t t ij ij , |R4 | ≤ √ − a, b − pi qj max √ p i qj p i qj b − a i,j=1
and finally for α > 1 we obtain 4) α ∞ α tij t kf 0 k1,[a,b] X ij . |R4 | ≤ α pi qj max − a α , bα − α α b − aα (p q ) (p q i j i j) i,j=1
(11.51)
(11.52)
At last iii) Let α ˜ , β > 1 such that α1˜ + β1 = 1, then by (115) of [43] and (49) of [39] we have s ! ∞ β+1 + (bp q − t )β+1 X (t − ap q ) ij i j i j ij β . |R1 | ≤ kf 0 kα,[a,b] (11.53) ˜ (β + 1)pi qj i,j=1
Part IV. Here we apply results from [41] and [39]. We start with Theorem 11.20. Let a < b, f ∈ C n ([a, b]), n ≥ 1 with |f (n) (s) − f (n) (1)| be a convex function in s. Let 0 < h < min(1 − a, b − 1) be fixed. Then ∞ n X |f (k) (1)| X 1−k k Γf (µXY , µX × µY ) ≤ (pi qj ) (tij − pi qj ) k! i,j=1 k=2 ∞ ω1 (f (n) , h) X + (pi qj )−n |tij − pi qj |n+1 . (11.54) (n + 1)!h i,j=1
Here ω1 is the usual first modulus of continuity. If f (k) (1) = 0, k = 2, . . . , n, then ∞ ω1 (f (n) , h) X (pi qj )−n |tij − pi qj |n+1 . (11.55) Γf (µXY , µX × µY ) ≤ (n + 1)!h i,j=1
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
146
Inequalities (11.54) and (11.55) when n is even are attained by |s − 1|n+1 f˜(s) = , (n + 1)!
a ≤ s ≤ b.
(11.56)
Proof. Apply Theorem 1 of [41] and Theorem 15 of [39].
We continue with the general Theorem 11.21. Let f ∈ C n ([a, b]), n ≥ 1. Assume that ω1 (f (n) , δ) ≤ w, where 0 < δ ≤ b − a, w > 0. Let x ∈ R and denote by Z |x| l m s (|x| − s)n−1 ds, (11.57) φn (x) := δ (n − 1)! 0 where d·e is the ceiling of the number. Then
∞ n X |f (k) (1)| X 1−k k Γf (µXY , µX × µY ) ≤ (pi qj ) (tij − pi qj ) k! i,j=1 k=2 ∞ X tij − pi qj +w p i q j φn . p i qj i,j=1
(11.58)
Inequality (11.58) is sharp, namely it is attained by the function f˜n (s) := wφn (s − 1),
a ≤ s ≤ b,
(11.59)
when n is even. Proof. Based on Theorem 2 of [41] and Theorem 16 of [39].
It follows Corollary 11.22 (to Theorem 11.21). It holds (0 < δ ≤ b − a) ∞ n X |f (k) (1)| X 1−k k Γf (µXY , µX × µY ) ≤ (pi qj ) (tij − pi qj ) k! i,j=1 k=2 ( ∞ X 1 + ω1 (f (n) , δ) (pi qj )−n |tij − pi qj |n+1 (n + 1)!δ i,j=1 ∞ 1 X + (pi qj )1−n |tij − pi qj |n 2n! i,j=1 ) ∞ X δ + (pi qj )2−n |tij − pi qj |n−1 , (11.60) 8(n − 1)! i,j=1
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
About a General Discrete Measure of Dependence
and
147
X n ∞ (k) X |f (1)| 1−k k Γf (µXY , µX × µY ) ≤ (p q ) (t − p q ) i j ij i j k! i,j=1 k=2 ( ∞ X 1 (pi qj )1−n |tij − pi qj |n + ω1 (f (n) , δ) n! i,j=1 ) ∞ X 1 + (pi qj )−n |tij − pi qj |n+1 . (11.61) n!(n + 1)δ i,j=1
In particular we have
∞ X 1 (pi qj )−1 (tij − pi qj )2 Γf (µXY , µX × µY ) ≤ ω1 (f 0 , δ) 2δ i,j=1 ) ∞ 1X δ + |tij − pi qj | + , 2 i,j=1 8 (
and
( ∞ X Γf (µXY , µX × µY ) ≤ ω1 (f , δ) |tij − pi qj |
(11.62)
0
+
i,j=1
∞ X
)
1 (pi qj )−1 (tij − pi qj )2 . 2δ i,j=1
(11.63)
Proof. By Corollary 1 of [41] and Corollary 2 of [39].
Also we have Corollary 11.23 (to Theorem 11.21). Assume here that f (n) is of Lipschitz type of order α, 0 < α ≤ 1, i.e. ω1 (f (n) , δ) ≤ Kδ α ,
K > 0,
(11.64)
for any 0 < δ ≤ b − a. Then
∞ n (k) X X |f (1)| 1−k k Γf (µXY , µX × µY ) ≤ (p q ) (t − p q ) i j ij i j k! i,j=1 k=2 ∞ X K + Q (pi qj )1−n−α |tij − pi qj |n+α . n (α + i) i,j=1
(11.65)
i=1
When n is even (11.65) is attained by f ∗ (x) = c|x − 1|n+α , where c := K/ i) > 0.
n Q
i=1
(α +
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
148
Proof. Application of Corollary 2 of [41] and Corollary 3 of [39].
Next comes Corollary 11.24 (to Theorem 11.21). Assume that ∞ X b−a≥ (pi qj )−1 t2ij − 1 > 0.
(11.66)
i,j=1
Then
Γf (µXY , µX × µY ) ≤ ω1 f 0 ,
X ∞
i,j=1
∞ X −1 2 (pi qj ) tij − 1 |tij − pi qj | +
i,j=1
1 . 2
(11.67)
Proof. Application of Corollary 3 of [41] and Corollary 4 of [39].
Corollary 11.25 (to Theorem 11.21). Assume that ∞ X
i,j=1
Then
|tij − pi qj | > 0.
(11.68)
∞ X b − a Γf (µXY , µX × µY ) ≤ ω1 f 0 , |tij − pi qj | |tij − pi qj | + 2 i,j=1 i,j=1 ∞ X 3 ≤ (b − a)ω1 f 0 , |tij − pi qj | . (11.69) 2 i,j=1
∞ X
Proof. Application of Corollary 5 of [41] and Corollary 5 of [39].
We present Proposition 11.26. Let f ∈ C([a, b]). i) Suppose that ∞ X |tij − pi qj | > 0.
(11.70)
i,j=1
Then
Γf (µXY , µX × µY ) ≤ 2ω1 f,
ii) Let r > 0 and
b−a ≥ r
∞ X
i,j=1
∞ X
i,j=1
|tij − pi qj | .
|tij − pi qj | > 0.
(11.71)
(11.72)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
About a General Discrete Measure of Dependence
Then Γf (µXY , µX × µY ) ≤
1+
1 r
ω1 f, r
Proof. By Propositions 1 of [41] and [39].
∞ X
i,j=1
149
|tij − pi qj | .
(11.73)
Also we give Proposition 11.27. Let f be a Lipschitz function of order α, 0 < α ≤ 1, i.e. there exists K > 0 such that |f (x) − f (y)| ≤ K|x − y|α , all x, y ∈ [a, b]. (11.74) Then ∞ X Γf (µXY , µX × µY ) ≤ K (pi qj )1−α |tij − pi qj |α . (11.75) i,j=1
Proof. By Propositions 2 of [41] and [39].
Next we present some alternative type of results. We start with Theorem 11.28 Let f ∈ C n+1 ([a, b]), n ∈ N, then we get the representation n−1 ∞ X (−1)k X t ij Γf (µXY , µX ×µY ) = f (k+1) (pi qj )−k (tij − pi qj )k+1 +ψ2 , (k + 1)! i,j=1 p i qj k=0
(11.76)
where
Z ∞ (−1)n X ψ2 = (pi qj ) n! 1 i,j=1
tij p i qj
(s − 1)n f (n+1) (s)ds .
(11.77)
Proof. By Theorem 12 of [41] and Theorem 17 of [39].
Next we estimate ψ2 . We present Theorem 11.29. Let f ∈ C n+1 ([a, b]), n ∈ N. Then ∞ (n+1) X kf k∞,[a,b] (pi qj )−n |tij − pi qj |n+1 , B1 := (n + 1)! i,j=1 ∞ (n+1) X kf k 1,[a,b] B := (pi qj )1−n |tij − pi qj |n , 2 n! i,j=1 |ψ2 | ≤ min of 1 1 and for α ˜ , β > 1 : + = 1, α ˜ β ∞ (n+1) X kf k α,[a,b] ˜ B3 := (pi qj )1−n−1/β |tij − pi qj |n+1/β . n!(nβ + 1)1/β i,j=1
(11.78)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
150
Also it holds Γf (µXY , µX × µY ) ≤
n−1 X k=0
∞ X 1 tij (k+1) −k k+1 f (p q ) (t − p q ) i j ij i j (k + 1)! i,j=1 p i qj
+ min(B1 , B2 , B3 ).
(11.79)
Proof. See Theorem 13 of [41] and Theorem 18 of [39].
We have Corollary 11.30 (to Theorem 11.29). Case of n = 1. It holds ∞ 00 X kf k∞,[a,b] B1,1 := (pi qj )−1 |tij − pi qj |2 , 2 i,j=1 ∞ X B := kf 00 k1,[a,b] |tij − pi qj |, 2,1 i,j=1 |ψ2 | ≤ min of 1 1 and for α ˜ , β > 1, + = 1, α ˜ β ∞ 00 X kf k α,[a,b] ˜ B := (pi qj )−1/β |tij − pi qj |1+1/β . 3,1 (β + 1)1/β i,j=1
(11.80)
Also it holds
∞ X 0 tij Γf (µXY , µX × µY ) ≤ f (tij − pi qj ) + min(B1,1 , B2,1 , B3,1 ). (11.81) p i qj i,j=1
Proof. See Corollary 8 of [41] and Corollary 6 of [39].
We further give Theorem 11.31 Let f ∈ C n+1 ([a, b]), n ∈ N, then we get the representation n−1 ∞ X (−1)k X tij Γf (µXY , µX × µY ) = f (k+1) (pi qj )−k (tij − pi qj )k+1 (k + 1)! i,j=1 p i qj k=0 ∞ n X (−1) + f (n+1) (1) (pi qj )−n (tij − pi qj )n+1 + ψ4 , (n + 1)! i,j=1
(11.82)
where ψ4 =
∞ X
i,j=1
p i q j ψ3
tij p i qj
,
(11.83)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
About a General Discrete Measure of Dependence
with (−1)n ψ3 (x) := n!
Z
x 1
(s − 1)n f (n+1) (s) − f (n+1) (1) ds.
Proof. Based on Theorem 14 of [41] and Theorem 19 of [39].
Book
151
(11.84)
Next we present estimations of ψ4 . Theorem 11.32. Let f ∈ C n+1 ([a, b]), n ∈ N, and ∞ X 1 (pi qj )−n−1 |tij − pi qj |n+2 ≤ b − a. 0< (n + 2) i,j=1
(11.85)
Then (11.82) is again valid and
! ∞ X 1 1 |ψ4 | ≤ ω1 f (n+1) , (pi qj )−n−1 |tij − pi qj |n+2 n! (n + 2) i,j=1 ∞ X 1 · 1 + (pi qj )−n |tij − pi qj |n+1 . (n + 1) i,j=1
Proof. By Theorem 15 of [41] and Theorem 20 of [39].
(11.86)
Also we have Theorem 11.33. Let f ∈ C n+1 ([a, b]), n ∈ N, and |f (n+1) (s) − f (n+1 (1)| is convex in s and ∞ X 1 0< (pi qj )−n−1 |tij − pi qj |n+2 ≤ min(1 − a, b − 1). (11.87) n!(n + 2) i,j=1
Then (11.82) is again valid and ∞ X 1 |ψ4 | ≤ ω1 f (n+1) , (pi qj )−n−1 |tij − pi qj |n+2 . n!(n + 2) i,j=1
(11.88)
(s − 1)n+2 fˆ(s) := , (n + 2)!
(11.89)
The last (11.88) is attained by
a≤s≤b
when n is even. Proof. By Theorem 16 of [41] and Theorem 21 of [39].
When n = 1 we obtain Corollary 11.34 (to Theorem 11.32). Let f ∈ C 2 ([a, b]) and ∞ 1 X 0< (pi qj )−2 |tij − pi qj |3 ≤ b − a. 3 i,j=1
(11.90)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
152
Then
where
tij (tij − pi qj ) Γf (µXY , µX × µY ) = f p i qj i,j=1 ∞ 00 X f (1) − (pi qj )−1 (tij − pi qj )2 + ψ4,1 , 2 i,j=1 ∞ X
ψ4,1 :=
0
∞ X
pi qj ψ3,1
i,j=1
with ψ3,1 (x) := − Furthermore it holds
Z
x 1
tij p i qj
,
(s − 1)(f 00 (s) − f 00 (1))ds,
(11.91)
(11.92)
x ∈ [a, b].
(11.93)
∞ ∞ X X 1 1 |ψ4,1 | ≤ ω1 f 00 , (pi qj )−2 |tij − pi qj |3 (pi qj )−1 t2ij + 1 . 3 i,j=1 2 i,j=1
(11.94)
Proof. See Corollary 9 of [41] and Corollary 7 of [39].
Also we have Corollary 11.35 (to Theorem 11.33). Let f ∈ C 2 ([a, b]), and |f (2) (s) − f (2) (1)| is convex in s and ∞ 1 X 0< (pi qj )−2 |tij − pi qj |3 ≤ min(1 − a, b − 1). (11.95) 3 i,j=1 Then again (11.91) is true and
∞ X 1 |ψ4,1 | ≤ ω1 f 00 , (pi qj )−2 |tij − pi qj |3 . 3 i,j=1
Proof. See Corollary 10 of [41] and Corollary 8 of [39].
(11.96)
The last main result of the chapter follows. Theorem 11.36. Let f ∈ C n+1 ([a, b]), n ∈ N. Additionally assume that |f (n+1) (x) − f (n+1) (y)| ≤ K|x − y|α ,
(11.97)
K > 0, 0 < α ≤ 1, all x, y ∈ [a, b] with (x − 1)(y − 1) ≥ 0. Then n−1 ∞ X (−1)k X t ij Γf (µXY , µX × µY ) = f (k+1) (pi qj )−k (tij − pi qj )k+1 (k + 1)! i,j=1 p i qj k=0 ∞ (−1)n (n+1) X + f (1) (pi qj )−n (tij − pi qj )n+1 + ψ4 . (n + 1)! i,j=1
(11.98)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
About a General Discrete Measure of Dependence
Book
153
Here we find that
∞ X K (pi qj )−n−α |tij − pi qj |n+α+1 . |ψ4 | ≤ n!(n + α + 1) i,j=1
(11.99)
Inequality (11.99) is attained when n is even by f ∗ (x) = c˜|x − 1|n+α+1 ,
x ∈ [a, b]
(11.100)
where c˜ := Q n
j=0
K
.
(11.101)
(n + α + 1 − j)
Proof. By Theorem 17 of [41] and Theorem 22 of [39].
Finally we give Corollary 11.37 (to Theorem 11.36). Let f ∈ C 2 ([a, b]). Additionally assume that |f 00 (x) − f 00 (y)| ≤ K|x − y|α ,
K > 0, 0 < α ≤ 1,
all x, y ∈ [a, b] with (x − 1)(y − 1) ≥ 0. Then ∞ X tij 0 (tij − pi qj ) Γf (µXY , µX × µY ) = f p i qj i,j=1 ∞ 00 X f (1) − (pi qj )−1 (tij − pi qj )2 + ψ4,1 . 2 i,j=1
Here
ψ4,1 :=
∞ X
i,j=1
with ψ3,1 (x) := − It holds that |ψ4,1 | ≤
Z
x 1
pi qj ψ3,1
tij p i qj
(s − 1)(f 00 (s) − f 00 (1))ds,
K (pi qj )−1−α |tij − pi qj |α+2 . (α + 2) i,j=1
Proof. See Corollary 11 of [41] and Corollary 9 of [39].
(11.103)
(11.104)
x ∈ [a, b].
∞ X
(11.102)
(11.105)
(11.106)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
This page intentionally left blank
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Chapter 12
H¨ older-Like Csiszar’s f -Divergence Inequalities
In this chapter we present a set of a large variety of probabilistic inequalities regarding Csiszar’s f -divergence of two probability measures. These are in parallel to H¨ older’s type inequalities and some of them are attained. This treatment is based on [40]. 12.1
Background
Here we follow [140]. Let f be an arbitrary convex function from (0, +∞) into R which is strictly convex at 1. We agree with the following notational conventions: f (0) = lim f (u), u→0+
0 = 0, 0·f 0 0f
a 0
a
= a lim
pi (x) =
µi (dx) , λ(dx)
= lim εf ε→0+
ε
u→+∞
(12.1)
f (u) , u
(0 < a < +∞).
Let (X, A, λ) be an arbitrary measure space with λ being a finite or σ-finite measure. Let also µ1 , µ2 probability measures on X such that µ1 , µ2 λ (absolutely continuous). Denote the Radon–Nikodym derivatives (densities) of µi with respect to λ by pi (x):
The quantity (see [140]). Γf (µ1 , µ2 ) =
Z
p2 (x)f X
i = 1, 2.
p1 (x) p2 (x)
λ(dx)
(12.2)
is called the f -divergence of the probability measures µ1 and µ2 . The function f is called the base function. By Lemma 1.1 of [140] Γf (µ1 , µ2 ) is always well-defined 155
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
156
and Γf (µ1 , µ2 ) ≥ f (1) with equality only for µ1 = µ2 . Also from [140] it derives that Γf (µ1 , µ2 ) does not depend on the choice of λ. When assuming f (1) = 0 then Γf can be considered as the most general measure of difference between probability measures. The Csiszar’s f -divergence Γf incorporated most of special cases of probability measure distances, e.g. the variation distance, χ2 -divergence, information for discrimination or generalized entropy, information gain, mutual information, mean square contingency, etc. Γf has tremendous applications to almost all applied sciences where stochastics enters. For more see [37], [42], [43], [41], [139], [140], [141], [152]. In this chapter all the base functions f appearing in the function Γf are assumed to have all the above properties of f . Notice that Γf (µ1 , µ2 ) ≤ Γ|f | (µ1 , µ2 ). For arbitrary functions f , g we notice that f r ◦ g = (f ◦ g)r ,
r ∈ R and |f ◦ g| = |f | ◦ g.
Clearly one can have widely products of (strictly) convex functions to be (strictly) convex. 12.2
Results
We start with 1 p
p
+
1 q
= 1. Then 1/q 1/p . Γ|f2 |q (µ1 , µ2 ) Γ|f1 f2 | (µ1 , µ2 ) ≤ Γ|f1 |p (µ1 , µ2 )
Theorem 12.1. Let p, q > 1 such that
(12.3)
q
Equality in (12.3) is true when |f1 | = c|f2 | for c > 0. More precisely, inequality (12.3) holds as equality iff there are a, b ∈ R not both zero such that p1 p q p2 · a|f1 | − b|f2 | = 0, a.e. [λ]. p2 Proof. Here we use the H¨ older’s inequality. We see that Z p1 p1 Γ|f1 f2 | (µ1 , µ2 ) = p2 |f1 | · |f2 | dλ p2 p2 X Z p1 p1 1/q 1/p p2 |f2 | dλ = p2 |f1 | p2 p2 X Z 1/p Z 1/q p1 p1 p q ≤ dλ dλ p2 |f1 | p2 |f2 | p2 p2 X X 1/q 1/p . Γ|f2 |q (µ1 , µ2 ) = Γ|f1 |p (µ1 , µ2 )
Corollary 12.2. It holds Γ|f1 f2 | (µ1 , µ2 ) ≤ Γf12 (µ1 , µ2 )
1/2
Γf22 (µ1 , µ2 )
1/2
.
(12.4)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
H¨ older-Like Csiszar’s f -Divergence Inequalities
157
Proof. By (12.3) setting p = q = 2.
We continue with a generalization of Theorem 12.1. n P 1 Theorem 12.3. Let a1 , a2 , . . . , an > 1, n ∈ N : ai = 1. Then i=1
n Γ Q
i=1
(µ1 , µ2 ) ≤
fi
n Y
i=1
Γ|fi |ai (µ1 , µ2 )
1/ai
.
(12.5)
Proof. Here we use the generalized H¨ older’s inequality, see p. 186, [235] and p. 200, [214]. We observe that n Z Z Y n Y p p1 1 1/ai n (µ1 , µ2 ) = p2 fi Γ Q p2 fi dλ dλ = fi p p2 2 X X i=1 i=1 i=1 1/ai Y n n Z Y 1/ai p1 . p2 |fi |αi dλ = Γ|fi |αi (µ1 , µ2 ) ≤ p2 X i=1 i=1 It follows the counterpart of Theorem 12.1.
Theorem 12.4. Let 0 < p < 1 and q < 0 such that p1 + 1q = 1. We suppose that p2 > 0 a.e. [λ]. Then we have 1/p (12.6) Γ|f2 |q (µ1 , µ2 )1/q Γ|f1 f2 | (µ1 , µ2 ) ≥ Γ|f1 |p (µ1 , µ2 ) unless Γ|f2 |q (µ1 , µ2 ) = 0. Inequality (12.6) holds as an equality iff there exist A, B ∈ R+ (A + B 6= 0) such that 1/p−1 p1 p1 1/p 1/q Ap2 f1 = B p 2 f 2 , a.e. [λ]. p2 p2
Proof. It is based on H¨ older’s inequality for 0 < p < 1, see p. 191 of [214]. Indeed we have Z p1 p1 Γ|f1 f2 | (µ1 , µ2 ) = p2 f1 f2 dλ p p2 2 X Z p1 p1 1/p 1/q = p2 |f1 | p2 |f2 | dλ p p2 2 X Z 1/p Z 1/q p1 p1 ≥ dλ dλ p2 |f1 |p p2 |f2 |q p p 2 2 X X 1/p 1/q = Γ|f1 |p (µ1 , µ2 ) Γ|f2 |q (µ1 , µ2 ) . Next we have
p1 p2
∈ L∞ (X). Then p1 . Γ|f1 f2 | (µ1 , µ2 ) ≤ Γ|f1 | (µ1 , µ2 ) · esssup |f2 | p2
Proposition 12.5. Let f2
(12.7)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
158
Proof. We notice that Z Z p1 p1 p1 Γ|f1 f2 | (µ1 , µ2 ) = p2 · |f1 · f2 | dλ = p2 |f1 | |f2 | dλ p p p2 2 2 X X Z p1 p1 p2 |f1 | dλ · esssup |f2 | ≤ p p2 2 X p1 = Γ|f1 | (µ1 , µ2 ) · esssup |f2 | . p2
In the following we use from [185], p. 120, see also [309], Theorem 12.6. Let f and g be positive, measurable functions on X for a measure space (X, F, µ). Let 0 < t < r < m < ∞. Then (i) The Rogers inequality r−t m−r Z m−t Z Z m t r f g dµ (12.8) f g dµ f g dµ ≤ X
X
X
provided that the integrals on the right are finite. (ii) For m > 1, the “historical H¨ older’s inequality” m−1 Z m Z Z f g m dµ f dµ f g dµ ≤
(12.9)
X
X
X
provided that the integrals on the right are finite. Using the last results we present the final result of the chapter. Theorem 12.7. Let f1 , f2 with |f1 |, |f2 | > 0 and p2 (x) > 0 a.e. [λ]. Let 0 < t < r < m < ∞. Then it holds i) r−t m−r m−t . (12.10) Γ|f1 | |f2 |m (µ1 , µ2 ) ≤ Γ|f1 | |f2 |t (µ1 , µ2 ) Γ|f1 | |f2 |r (µ1 , µ2 ) ii) For m > 1,
Γ|f1 | |f2 | (µ1 , µ2 )
m
≤ Γ|f1 | (µ1 , µ2 )
m−1
Γ|f1 | |f2 |m (µ1 , µ2 ).
(12.11)
Proof. i) By applying (12.8) we obtain Z m−t m−t p1 r p1 = p2 |f1 | Γ|f1 | |f2 |r (µ1 , µ2 ) |f2 | dλ p2 p2 X Z m−r p1 p1 |f2 |t dλ ≤ p2 |f1 | p2 p2 X Z r−t p1 m p1 · |f2 | dλ p2 |f1 | p2 p2 X m−r r−t = Γ|f1 | |f2 |t (µ1 , µ2 ) Γ|f1 | |f2 |m (µ1 , µ2 ) .
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
H¨ older-Like Csiszar’s f -Divergence Inequalities
Book
159
ii) By applying (12.9) we derive Z m m p1 p1 Γ|f1 | |f2 | (µ1 , µ2 ) = |f2 | dλ p2 |f1 | p2 p2 X m−1 Z Z p1 p1 p1 p2 |f1 | p2 |f1 | ≤ dλ |f2 |m dλ p p p 2 2 2 X X m−1 = Γ|f1 | (µ1 , µ2 ) Γ|f1 | |f2 |m (µ1 , µ2 ).
Note 12.8. Let X be a finite or countable discrete set, A be its power set P(X) and λ has mass 1 for each x ∈ X, then Γf in (12.2) becomes a finite or infinite sum, respectively. As a result all the above presented integral inequalities are discretized and become summation inequalities.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
This page intentionally left blank
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Chapter 13
Csiszar’s Discrimination and Ostrowski Inequalities via Euler-type and Fink Identities The Csiszar’s discrimination is the most essential and general measure for the comparison between two probability measures. In this chapter we provide probabilistic representation formulae for it via a generalized Euler-type identity and Fink identity. Then we give a large variety of tight estimates for their remainders in various cases. We finish with some generalized Ostrowski type inequalities involving k · k p , p > 1, using the Euler-type identity. This treatment follows [46]. 13.1
Background
(I) Throughout Part A we use the following. Let f be a convex function from (0, +∞) into R which is strictly convex at 1 with f (1) = 0. Let (X, A, λ) be a measure space, where λ is a finite or a σ-finite measure on (X, A). And let µ1 , µ2 be two probability measures on (X, A) such that µ1 λ, µ2 λ (absolutely continuous), e.g. λ = µ1 + µ2 . dµ2 1 Denote by p = dµ dλ , q = dλ the (densities) Radon–Nikodym derivatives of µ1 , µ2 with respect to λ. Here we suppose that p 0 < a ≤ ≤ b, a.e. on X and a ≤ 1 ≤ b. q The quantity Γf (µ1 , µ2 ) =
Z
q(x)f X
p(x) q(x)
dλ(x),
(13.1)
was introduced by I. Csiszar in 1967, see [140], and is called f -divergence of the probability measures µ1 and µ2 . By Lemma 1.1 of [140], the integral (13.1) is well-defined and Γf (µ1 , µ2 ) ≥ 0 with equality only when µ1 = µ2 . Furthermore Γf (µ1 , µ2 ) does not depend on the choice of λ. The concept of f -divergence was introduced first in [139] as a generalization of Kullback’s “information for discrimination” or I-divergence (generalized entropy) [244], [243] and of R´enyi’s “information gain” (I-divergence of order α) [306]. In fact the I-divergence of order 1 equals Γu log2 u (µ1 , µ2 ). 161
22nd June 2009
18:37
162
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
The choice f (u) = (u − 1)2 produces again a known measure of difference of distributionsR that is called χ2 -divergence. Of course the total variation distance |µ1 − µ2 | = X |p(x) − q(x)|dλ(x) is equal to Γ|u−1| (µ1 , µ2 ). Here by assuming f (1) = 0 we can consider Γf (µ1 , µ2 ), the f -divergence as a measure of the difference between the probability measures µ1 , µ2 . The f -divergence is in general asymmetric in µ1 and µ2 . But since f is convex and strictly convex at 1 so is 1 (13.2) f ∗ (u) = uf u and as in [140] we find Γf (µ2 , µ1 ) = Γf ∗ (µ1 , µ2 ).
(13.3)
In Information Theory and Statistics many other divergences are used which are special cases of the above general Csiszar f -divergence, e.g. Hellinger distance D H , α-divergence Dα , Bhattacharyya distance DB , Harmonic distance DHa , Jeffrey’s distance DJ , triangular discrimination D∆ , for all these see, e.g. [102], [152]. The problem of finding and estimating the proper distance (or difference or discrimination) of two probability distributions is one of the major ones in Probability Theory. The above f -divergence measures in their various forms have been also applied to Anthropology, Genetics, Finance, Economics, Political Science, Biology, Approximation of Probability distributions, Signal Processing and Pattern Recognition. A great inspiration for this chapter has been the very important monograph on the topic by S. Dragomir [152]. In Part A we find representations and estimates for Γf (µ1 , µ2 ) via the generalized Euler type and Fink identities. We use here the sequence {Bk (t), k ≥ 0} of Bernoulli polynomials which is uniquely determined by the following identities: and
Bk0 (t) = kBk−1 (t), k ≥ 1, B0 (t) = 1 Bk (t + 1) − Bk (t) = ktk−1 ,
k ≥ 0.
The values Bk = Bk (0), k ≥ 0 are the known Bernoulli numbers. We need to mention 1 1 B0 (t) = 1, B1 (t) = t − , B2 (t) = t2 − t + , 2 6 3 2 1 1 3 4 3 2 B3 (t) = t − t + t, B4 (t) = t − 2t + t − , 2 2 30 5 5 1 5 t2 1 . B5 (t) = t5 − t4 + t3 − t, and B6 (t) = t6 − 3t5 + t4 − + 2 3 6 2 2 42 ∗ Let {Bk (t), k ≥ 0} be a sequence of periodic functions of period 1, related to Bernoulli polynomials as Bk∗ (t) = Bk (t),
0 ≤ t < 1,
Bk∗ (t + 1) = Bk∗ (t), t ∈ R.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Csiszar’s Discrimination and Ostrowski Inequalities via Euler-type and Fink Identities
163
We have that B0∗ (t) = 1, B1∗ is a discontinuous function with a jump of −1 at each integer, while Bk∗ , k ≥ 2 are continuous functions. Notice that Bk (0) = Bk (1) = Bk , k ≥ 2. We use the following result, see [53], [143]. Theorem 13.1. Let any a, b ∈ R, f : [a, b] → R be such that f (n−1) , n ≥ 1, is a continuous function and f (n) (x) exists and is finite for all but a countable set of x in (a, b) and that f (n) ∈ L1 ([a, b]). Then for every x ∈ [a, b] we have f (x) =
1 b−a +
Z
(b − a)k−1 x−a [f (k−1) (b) − f (k−1) (a)] Bk k! b−a k=1 " # Z x−a ∗ x−t Bn − Bn f (n) (t)dt. (13.4) b−a b−a [a,b]
b
f (t)dt + a
(b − a)n−1 n!
n−1 X
The sum in (13.4) when n = 1 is zero. If f (n−1) is absolutely continuous then (13.4) is valid again. We need also Theorem 13.2 (see [195]). Let any a, b ∈ R, f : [a, b] → R, n ≥ 1, f (n−1) is absolutely continuous on [a, b]. Then f (x) =
n b−a +
where Fk (x) :=
Z
b a
f (t)dt −
1 (n − 1)!(b − a)
n−k k!
n−1 X
Fk (x)
k=1 Z b a
(x − t)n−1 k(t, x)f (n) (t)dt,
(13.5)
(13.6)
f (k−1) (a)(x − a)k − f (k−1) (b)(x − b)k b−a
and k(t, x) :=
t − a, a ≤ t ≤ x ≤ b, t − b, a ≤ x < t ≤ b.
,
(13.7)
(II) In Part B we give Lp form, 1 < p < ∞, Ostrowski type inequalities via the generalized Euler type identity (13.4). We mention as inspiration writing this chapter the famous Ostrowski inequality [277]: " 2 # Z b x − a+b 1 1 2 f (t)dt ≤ + (b − a)kf 0 k∞ , f (x) − b−a a 4 (b − a)2
where f ∈ C 0 ([a, b]), x ∈ [a, b], which is a sharp inequality, see [21]. Other motivations come from [53], [32], [21], [143]. Here our inequalities involve higher order derivatives of f .
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
164
13.2
Main Results
Part A We give Theorem 13.3. Let f : [a, b] → R as in Section 13.1 (I) and be such that f (n−1) , n ≥ 1, is a continuous function and f (n) (x) exists and is finite for all but a countable set of x in (a, b) and that f (n) ∈ Lp1 ([a, b]), for 1 ≤ p1 ≤ ∞. Then Z b 1 Γf (µ1 , µ2 ) = f (t)dt (13.8) b−a a ! Z p n−1 X (b − a)k−1 q −a f (k−1) (b) − f (k−1) (a) qBk dλ + Rn , + k! b−a X k=1
where
(b − a)n−1 Rn = n!
Z
Z
q X
Bn
Z
q X
Z
−a
b−a
[a,b]
One has (b − a)n−1 |Rn | ≤ n!
p q
B n [a,b]
p q
!
−
−a
b−a
Bn∗
!
−
p q
−t
b−a
Bn∗
!!
f
(n)
!
(t)dt dλ. (13.9)
! ! − t (n) |f (t)|dt dλ. b−a (13.10) p q
Proof. In (13.4), set x = pq , multiply it by q and integrate against λ.
Next we present related estimates to Γf . Theorem 13.4. All assumptions as in Theorem 13.3. Then (i) ! ! ! Z Z 1 p (b − a)n q −a |Rn | ≤ q dt dλ kf (n) k∞ , Bn (t) − Bn n! b − a X 0
(13.11)
if f (n) ∈ L∞ ([a, b]), (ii)
(b − a)n |Rn | ≤ n!
Z
Z 1 q Bn (t) − Bn X 0 for p1 , q1 > 1 :
if f (n) ∈ Lp1 ([a, b]), (iii) (b − a)n−1 |Rn | ≤ n!
Z
q B n X
p q
−a
b−a
! q1 !1/q1 ! − a dλ kf (n) kp1 , dt b−a p q
1 1 + = 1, p1 q1
!
− Bn (t)
∞,[0,1]
(13.12)
!
dλ kf (n) k1 ,
(13.13)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Csiszar’s Discrimination and Ostrowski Inequalities via Euler-type and Fink Identities
165
if f (n) ∈ L1 ([a, b]). Proof. We are using (13.10). Part (iii) is obvious by [143], p. 347. For Part (i) we notice that ! ! ! ! Z Z p p (13.10) (b − a)n−1 q −a q −t ∗ q − Bn |Rn | ≤ dt dλ kf (n) k∞ B n n! b − a b − a X [a,b]
and by the change of variable method the claim is proved. For part (ii) we apply H¨ older’s inequality on (13.10) and we have ! Z Z p (b − a)n−1 q −a |Rn | ≤ q Bn n! b−a X [a,b] 1 ! ! q p q1 1 q −t − Bn∗ dλ kf (n) kp1 , dt b−a
again by change of variable claim is established.
Next we derive Theorem 13.5. Let f as in Section 13.1, (I) such that f (n−1) is absolutely continuous function on [a, b], n ≥ 1 and that f (n) ∈ Lp1 ([a, b]), where 1 ≤ p1 ≤ ∞. Then Z b n−1 X n − k n Γf (µ1 , µ2 ) = f (t)dt − (13.14) b−a a k! k=1 ! R R f (k−1) (a) X q 1−k (p − aq)k dλ − f (k−1) (b) X q 1−k (p − bq)k dλ × + Gn , b−a where 1 Gn := (n − 1)!(b − a)
Z
q
2−n
X
Z
b a
and 1 |Gn | ≤ (n − 1)!(b − a)
Z
q 2−n X
Z
b a
(p − tq)
n−1
p k t, q
f
(n)
!
(t)dt dλ,
(13.15)
! p (n) n−1 |p − tq| |f (t)|dt dλ. (13.16) k t, q
Proof. In (13.5) set x = pq , multiply it by q and integrate against λ.
We give the estimates Theorem 13.6. All suppositions as in Theorem 13.5. Then (i) Z 1 |Gn | ≤ q −n (p − aq)n+1 + (bq − p)n+1 dλ kf (n) k∞ , (13.17) (n + 1)!(b − a) X
if f (n) ∈ L∞ ([a, b]),
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
166
(ii) |Gn | ≤
(B(q1 + 1, q1 (n − 1) + 1))1/q1 (13.18) (n − 1)!(b − a) ! Z 1/q1 1 dλ kf (n) kp1 , × q 1−n− q1 (p − aq)q1 n+1 + (bq − p)q1 n+1 X
for p1 , q1 > 1 : and (iii)
1 p1
+
1 q1
= 1, if f (n) ∈ Lp1 ([a, b]),
1 |Gn | ≤ (n − 1)!(b − a)
Z
p p q max − a, b − q q X
n
dλ kf (n) k1 ,
(13.19)
if f (n) ∈ L1 ([a, b]).
Proof. (i) We have by (13.16) that Z b ! Z 1 p 2−n (n) n−1 |Gn | ≤ q |p − tq| K t, q dt dλ kf k∞ , (n − 1)!(b − a) X a if f (n) ∈ L∞ ([a, b]). Here
K t,
p q
=
t − a, t − b,
a≤t≤ a≤
p ≤ b, q
p < t ≤ b. q
Then by [338], p. 256 we obtain Z b p q −2 n−1 |p − tq| K t, (p − aq)n+1 + (bq − p)n+1 , dt = q n(n + 1) a which proves the claim. (ii) Again by (13.16) we derive
Z Z b 1 2−n |Gn | ≤ q |p − tq|q1 (n−1) (n − 1)!(b − a) X a q1 !1/q1 p dt K t, dλ kf (n) kp1 , q 1 p1
1 q1
= 1, if f (n) ∈ Lp1 ([a, b]). Again by [338], p. 256 we find q !1/q1 Z b p 1 q1 (n−1) |p − tq| K t, q dt a 1 1/q1 1 = q − 1+ q1 B(q1 + 1, q1 (n − 1) + 1) q1 (p − aq)q1 n+1 + (bq − p)q1 n+1 ,
for p1 , q1 > 1 :
+
proving the claim.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Csiszar’s Discrimination and Ostrowski Inequalities via Euler-type and Fink Identities
(iii) From (13.16) we obtain
Z
1 p
n−1 2−n q kp − tqk∞,[a,b] K t, |Gn | ≤
(n − 1)!(b − a) q X
∞,[a,b]
167
dλ kf (n) k1 ,
if f (n) ∈ L1 ([a, b]). We notice that
p
p p p
= − t = max − a, b − .
K t,
q
q q q ∞,[a,b] ∞,[a,b]
The last establishes the claim.
Next we study some important cases. For that we need
˜ := x−a . It holds Theorem 13.7. Let any a, b ∈ R, a ≤ x ≤ b, call λ b−a (i) 2 Z 1 x − a+b 1 1 2 ˜ ˜ ≤ , ∀x ∈ [a, b], I1 (λ) := |B1 (t) − B1 (λ)|dt = + 2 4 (b − a) 2 0
with equality at x = a, b, (ii) ([143], p. 343) Z 1 8 1 1 ˜ := ˜ I2 (λ) |B2 (t) − B2 (λ)|dt = δ 3 (x) − δ 2 (x) + ≤ , ∀x ∈ [a, b], 3 12 6 0
(13.20)
(13.21)
with equality at x = a, b, where
(iii) ([13])
where
x − a+b 2 δ(x) := , b−a
x ∈ [a, b],
Z 1 ˜ := ˜ I3 (λ) |B3 (t) − B3 (λ)|dt = 0 ˜ 3 t2 3 ˜ 4 ˜3 ˜2 λ − t41 + 2t31 − 1 + λ −λ −λ + , 2 2 2 2 ˜ t21 3 ˜4 3 4 3 ˜3 ˜2 λ 2 t1 − 2t1 + 2 − 2 λ + 3λ − 2λ + 2 , ˜ 3 4 t22 3 ˜4 ˜ 3 ˜ 2 λ 3 t − 2t + − λ + λ + λ − , 2 2 2 2 2 2 ˜ 3 t2 3 ˜4 ˜ 3 + 2λ ˜2 − λ , − 3λ − t42 + 2t32 − 2 + λ 2 2 2 2
√ 3− 3 ˜ λ ∈ 0, , 6 √ 3− 3 1 ˜ λ∈ , , 6 2 √ ˜ ∈ 1, 3 + 3 , λ 2 6 √ ˜ ∈ 3 + 3, 1 , λ 6
r ˜ 1 1 3 λ ˜ − 3λ ˜2 , t1 := − − + 3λ 4 2 2 4 r ˜ 1 1 3 λ ˜ − 3λ ˜2 , + 3λ t2 := − + 4 2 2 4
(13.22)
(13.23)
(13.24)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
168
and
˜= with equality at λ (iv) ([53]) ˜ := I4 (λ)
with
√ ˜ ≤ 3, I3 (λ) 36
√ 3± 3 6 .
Z
1 0
(13.25)
˜ |B4 (t) − B4 (λ)|dt
˜5 16λ ˜3 − λ ˜2 + 1 , ˜ 4 + 14 λ − 7λ 5 3 30 = 5 3 ˜ ˜ − 16λ + 9λ ˜ 4 − 26λ + 3λ ˜2 − 1 , 5 3 10 ˜ ≤ I4 (λ)
˜ ≤ 1, 0≤λ 2 1 2
(13.26)
˜ ≤ 1, ≤λ
1 , 30
(13.27)
˜ = 0, 1, i.e. when x = a, b, and attained at λ (v) ([143]) for n ≥ 1 we have Z 1 1/2 Z 1 2 ˜ ˜ ˜ In (λ) := |Bn (t) − Bn (λ)|dt ≤ (Bn (t) − Bn (λ)) dt 0 0 s (n!)2 ˜ . |B2n | + Bn2 (λ) = (2n)!
(13.28)
Consequently by (13.11) and Theorem 13.7 we derive the precise estimates. Theorem 13.8. All assumptions as in Theorem 13.3 with f (n) ∈ L∞ ([a, b]), n ≥ 1. Here ˜ := λ(x) ˜ λ :=
p(x) q(x)
˜ ∈ [0, 1], a.e. on X. Then i.e. λ
−a
b−a
(b − a)n |Rn | ≤ n!
and
Z
X
p q
−a
x ∈ X,
(13.29)
˜ qIn (λ)dλ kf (n) k∞ ,
(13.30)
=
b−a
,
(b − a) 0 kf k∞ , 2 2 (b − a) |R2 | ≤ kf 00 k∞ , 12 √ 3(b − a)3 000 |R3 | ≤ kf k∞ , 216
|R1 | ≤
(13.31) (13.32) (13.33)
and |R4 | ≤
(b − a)4 (4) kf k∞ . 720
(13.34)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Csiszar’s Discrimination and Ostrowski Inequalities via Euler-type and Fink Identities
169
We have also by (13.12) and (13.28) that Corollary 13.9. All assumptions as in Theorem 13.3 with f (n) ∈ L2 ([a, b]). Then s p Z n 2 − a (b − a) (n!) q dλ kf (n) k2 , n ≥ 1. q |B2n | + Bn2 |Rn | ≤ n! (2n)! b − a X
(13.35)
We need Note 13.10. From [143], p. 347 for r ∈ N, x ∈ [a, b] we have that −2r x − a x − a −2r = (1−2 )|B2r |+ 2 B2r −B2r . (13.36) max B2r (t)−B2r b−a b−a t∈[0,1]
And from [143], p. 348 we obtain x − a max B2r+1 (t) − B2r+1 t∈[0,1] b−a 2(2r + 1)! B2r+1 x − a , ≤ + 2r+1 −2r (2π) (1 − 2 ) b−a along with
a + b x − a 1 = x − + . max B1 (t) − B1 b−a 2 2 t∈[0,1]
(13.37)
(13.38)
Using (13.13) and Note 13.10 we find
Theorem 13.11. All assumptions as in Theorem 13.3 with f (n) ∈ L1 ([a, b]), r, n ≥ 1. Then Z 1 (a + b) |R1 | ≤ + q dλ kf 0 k1 , (13.39) p − 2 2 X when n = 2r we have
( (b − a)2r−1 |R2r | ≤ (1 − 2−2r )|B2r | (2r)! p ) Z −2r q −a + q 2 B2r − B2r dλ kf (2r) k1 , b−a X
(13.40)
and for n = 2r + 1 we obtain
( (b − a)2r 2(2r + 1)! |R2r+1 | ≤ (2r + 1)! (2π)2r+1 (1 − 2−2r ) ) p Z q −a + q B2r+1 dλ kf (2r+1) k1 . b−a X
(13.41)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
170
Part B Remark 13.12. Let f be as in Theorem 13.1, we denote Z b 1 f (t)dt (13.42) ∆n (x) := f (x) − b−a a n−1 X (b − a)k−1 x − a − Bk [f (k−1) (b) − f (k−1) (a)], x ∈ [a, b], n ≥ 1. k! b−a k=1
We have by (13.4) that
(b − a)n−1 ∆n (x) = n!
Z
[a,b]
Bn
x−a b−a
−
Bn∗
x−t b−a
f (n) (t)dt.
(13.43)
Let here 1 < p < ∞, we see by H¨ older’s inequality that q !1/q Z b (b − a)n−1 x − a x − t dt Bn kf (n) kp =: (∗), |∆n (x)| ≤ − Bn∗ n! b − a b − a a (13.44) where q > 1 such that p1 + 1q = 1, given that f (n) ∈ Lp ([a, b]). By change of variable we find that q 1/q 1 Z 1 (b − a)n−1+ q Bn (t) − Bn x − a dt (∗) = kf (n) kp . (13.45) n! b − a 0 We have established the generalized Ostrowski type inequality.
Theorem 13.13. Let any a, b ∈ R, f : [a, b] → R be such that f (n−1) , n ≥ 1, is a continuous function and f (n) (x) exists and is finite for all but a countable set of x in (a, b) and that f (n) ∈ Lp ([a, b]), 1 < p, q < ∞: p1 + 1q = 1. Then for every x ∈ [a, b] we have q 1/q 1 Z 1 (b − a)n−1+ q Bn (t) − Bn x − a dt |∆n (x)| ≤ kf (n) kp . (13.46) n! b − a 0 The last theorem appeared first in [143] as Theorem 9 there, but with missing two of the assumptions only by assuming f (n) ∈ Lp ([a, b]). We give
Corollary 13.14. Let any a, b ∈ R, f : [a, b] → R be such that f (n−1) , n ≥ 1, is a continuous function and f (n) (x) exists and is finite for all but a countable set of x in (a, b) and that f (n) ∈ L2 ([a, b]). Then for any x ∈ [a, b] we have s ! 1 (n!)2 x − a (b − a)n− 2 |B2n | + Bn2 kf (n) k2 . (13.47) |∆n (x)| ≤ n! (2n)! b−a Proof. By Theorem 13.13, p = 2 and that 2 Z 1 x−a (n!)2 x−a 2 Bn (t) − Bn dt = |B2n | + Bn , b−a (2n)! b−a 0 see [143], p. 352.
(13.48)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Csiszar’s Discrimination and Ostrowski Inequalities via Euler-type and Fink Identities
171
Here again, (13.47) appeared first in [143], p. 351, as Corollary 5 there, erroneously under the only assumption f (n) ∈ L2 ([a, b]). Remark 13.15. We find the trapezoidal formula Z b f (a) + f (b) (b − a) 0 1 0 − (f (b)−f (a))− f (t)dt, (13.49) ∆4 (a) = ∆4 (b) = 2 12 b−a a and the midpoint formula Z b a+b a+b (b − a) 0 1 ∆4 =f + (f (b) − f 0 (a)) − f (t)dt. 2 2 24 b−a a
(13.50)
Furthermore we obtain the trapezoidal formula (f (a) + f (b)) (b − a) 0 − (f (b) − f 0 (a)) 2 12 Z b (b − a)3 000 1 + (f (b) − f 000 (a)) − f (t)dt. (13.51) 720 b−a a
∆5 (a) = ∆5 (b) = ∆6 (a) = ∆6 (b) =
We also derive the midpoint formula a+b a+b a+b (b − a) 0 ∆5 = ∆6 =f + (f (b) − f 0 (a)) 2 2 2 24 Z b 7(b − a)3 000 1 − (f (b) − f 000 (a)) − f (t)dt. (13.52) 5760 b−a a Using Mathematica 4 and (13.47) we establish Theorem 13.16. Let any a, b ∈ R, f : [a, b] → R be such that f (3) is a continuous function and f (4) (x) exists and is finite for all but a countable set of x in (a, b) and that f (4) ∈ L2 ([a, b]). Then (b − a)3.5 (4) √ kf k2 , 72 70 r 107 3.5 (4) ∆4 a + b ≤ 1 1152 35 (b − a) kf k2 . 2 |∆4 (a)| ≤
(13.53) (13.54)
We continue with
Theorem 13.17. Let any a, b ∈ R, f : [a, b] → R be such that f (4) is a continuous function and f (5) (x) exists and is finite for all but a countable set of x in (a, b) and that f (5) ∈ L2 ([a, b]). Then
We finish with
(b − a)4.5 (5) √ |∆5 (a)| ≤ kf k2 , 144 2310 4.5 ∆5 a + b ≤ (b −√a) kf (5) k2 . 2 144 2310
(13.55) (13.56)
22nd June 2009
18:37
172
World Scientific Book - 9.75in x 6.5in
PROBABILISTIC INEQUALITIES
Theorem 13.18. Let any a, b ∈ R, f : [a, b] → R be such that f (5) is a continuous function and f (6) (x) exists and is finite for all but a countable set of x in (a, b) and that f (6) ∈ L2 ([a, b]). Then r 101 1 (b − a)5.5 kf (6) k2 , (13.57) |∆6 (a)| ≤ 1440 30030 and r 7081 5.5 (6) ∆6 a + b ≤ 1 (13.58) 46080 2145 (b − a) kf k2 . 2
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Chapter 14
Taylor-Widder Representations and Gr¨ uss, Means, Ostrowski and Csiszar’s Inequalities In this chapter based on the very general Taylor–Widder formula several representation formulae are developed. Applying these are established very general inequalities of types: Gr¨ uss, comparison of means,Ostrowski, Csiszar f -divergence. The approximations involve Lp norms, any 1 ≤ p ≤ ∞. This treatment follows [56]. 14.1
Introduction
The article that motivates to most this chapter is of D. Widder (1928), see [339], where he generalizes the Taylor formula and series, see also Section 14.2. Based on that approach we give several representation formulae for the integral of a function f over [a, b] ⊆ R, and the most important of these involve also f (x), for any x ∈ [a, b], see Theorems 14.6, 14.8, 14.9, 14.10 and Corollary 14.12. Then being motivated by the works of Ostrowski [277], Gr¨ uss [205], of the author [48] and Csiszar [140] we give our related generalized results. We would like to mention the following inspiring results. Theorem 14.1 (Ostrowski, 1938, [277]). Let f : [a, b] → R be continuous on [a, b] and differentiable on (a, b) whose derivative f 0 : (a, b) → R is bounded on (a, b), i.e., kf 0 k∞ = sup |f 0 (t)| < +∞. Then t∈(a,b)
" 2 # 1 Z b x − a+b 1 2 f (t)dt − f (x) ≤ + (b − a)kf 0 k∞ , b − a a 4 (b − a)2
for any x ∈ [a, b]. The constant
1 4
is the best possible.
Theorem 14.2 (Gr¨ uss, 1935, [205]). Let f, g integrable functions from [a, b] into R, such that m ≤ f (x) ≤ M , ρ ≤ g(x) ≤ σ, for all x ∈ [a, b], where m, M, ρ, σ ∈ R. Then Z b Z b 1 Z b 1 1 1 f (x)g(x)dx − f (x)dx g(x)dx ≤ (M −m)(σ−ρ). b − a a 4 b−a a b−a a 173
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
PROBABILISTIC INEQUALITIES
174
So our generalizations here are over an extended complete Tschebyshev system {ui }ni=0 ∈ C n+1 ([a, b]), n ≥ 0, see Theorems 14.3, 14.4, and [223]. Thus our Ostrowski type general inequality is presented in Theorem 14.14 and is sharp. Next we generalize Gr¨ uss inequality in Theorems 14.15, 14.16, 14.17 and Corollary 14.18. It follows the generalized comparison of integral averages in Theorem 14.19, where we compare the Riemann integral to arbitrary general integrals involving finite measures. We finish with generalized representation and estimates for the Csiszar f -divergence, see Theorems 14.21, 14.22. The last is the most general and best measure for the difference between probability measures. 14.2
Background
The following are taken from [339]. Let f , u0 , u1 , . . . , un ∈ C n+1 ([a, b]), n ≥ 0, and the Wronskians u0 (x) u1 (x) ··· ui (x) 0 u01 (x) ··· u0i (x) u0 (x) Wi (x) := W [u0 (x), u1 (x), . . . , ui (x)] := . .. (i) u (x) u(i) (x) · · · u(i) (x) 0 1 i i = 0, 1, . . . , n. Suppose Wi (x) > 0 over [a, b]. Clearly then φ0 (x) := W0 (x) = u0 (x), φ1 (x) :=
W1 (x) Wi (x)Wi−2 (x) , . . . , φi (x) := , 2 (W0 (x)) (Wi−1 (x))2
i = 2, 3, . . . , n are positive on [a, b]. For i ≥ 0, the linear differentiable operator of order i: W [u0 (x), u1 (x), . . . , ui−1 (x), f (x)] , Wi−1 (x) L0 f (x) := f (x), ∀x ∈ [a, b]. Li f (x) :=
i = 1, . . . , n + 1,
Then for i = 1, . . . , n + 1 we have Li f (x) = φ0 (x)φ1 (x) · · · φi−1 (x)
d 1 d 1 d d 1 d f (x) ··· . dx φi−1 (x) dx φi−2 (x) dx dx φ1 (x) dx φ0 (x)
Consider also
u0 (t) 0 u0 (t) 1 .. gi (x, t) := . Wi (t) (i−1) u (t) 0 u0 (x)
i = 1, 2, . . . , n; g0 (x, t) :=
u0 (x) u0 (t) ,
u1 (t) u01 (t) .. . (i−1) u1 (t) u1 (x)
∀x, t ∈ [a, b].
···
··· ···
···
ui (t) u0i (t) .. , . (i−1) ui (t) ui (x)
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Taylor-Widder Representations and Gr¨ uss, Means, Ostrowski and Csiszar’s Inequalities
175
Note that gi (x, t) as a function of x is a linear combination of u0 (x), u1 (x), . . . , ui (x) and it holds gi (x, t) Z xi−1 Z xi−2 Z x Z x1 φ0 (x) φi (xi )dxi dxi−1 · · · dx1 φi−1 (xi−1 ) ··· = φ1 (x1 ) φ0 (t) · · · φi (t) t t t t Z x 1 = φ0 (s) · · · φi (s)gi−1 (x, s)ds, i = 1, 2, . . . , n. φ0 (t) · · · φi (t) t Example [339]. The sets {1, x, x2 , . . . , xn }, {1, sin x, − cos x, − sin 2x, cos 2x, . . . , (−1)n−1 sin nx, (−1)n cos nx} fulfill the above theory. We mention Theorem 14.3 (Karlin and Studden (1966), see p. 376, [223]). Let u0 , u1 , . . . , un ∈ C n ([a, b]), n ≥ 0. Then {ui }ni=0 is an extended complete Tschebyshev system on [a, b] iff Wi (x) > 0 on [a, b], i = 0, 1, . . . , n. We also mention Theorem 14.4 (D. Widder, p. 138, [339]). Let the functions f (x), u0 (x), u1 (x), . . . , un (x) ∈ C n+1 ([a, b]), and the Wronskians W0 (x), W1 (x), . . . , Wn (x) > 0 on [a, b], x ∈ [a, b]. Then for t ∈ [a, b] we have f (x) = f (t)
u0 (x) + L1 f (t)g1 (x, t) + · · · + Ln f (t)gn (x, t) + Rn (x), u0 (t)
where Rn (x) :=
Z
x
gn (x, s)Ln+1 f (s)ds. t
E.g. one could take u0 (x) = c > 0. If ui (x) = xi , i = 0, 1, . . . , n, defined on [a, b], then Li f (t) = f (i) (t) and
gi (x, t) =
(x − t)i , t ∈ [a, b]. i!
So under the assumptions of Theorem 14.4 it holds n
f (x) = f (y)
u0 (x) X + Li f (y)gi (x, y) + u0 (y) i=1
Z
x y
gn (x, t)Ln+1 f (t)dt, ∀x, y ∈ [a, b]. (14.1)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
176
14.3
Main Results
Define the Kernel K(t, x) :=
t − a, t − b,
a ≤ t ≤ x ≤ b; a ≤ x < t ≤ b.
(14.2)
Integrating (14.1) with respect to y we find Z b n Z b X f (y) dy + Li f (y)gi (x, y)dy f (x)(b − a) = u0 (x) a u0 (y) i=1 a Z b Z x gn (x, t)Ln+1 f (t)dt dy, ∀x ∈ [a, b]. + We notice that (see also [195]) Z b Z x Z x Z x Z dy dt + dy dt = a
y
=
Z
a
x
dt a
y Z t a
(14.3)
y
a
dy −
b
dy
x Z b
dy
Z
x
dt
y Z y
Z
dt =
x
x
Z
x
dt a
t
dy −
a
Z
b
dt x
Z
b
dy. (14.4) t
By letting ∗ := gn (x, t)Ln+1 f (t) we better notice that Z b Z x Z x Z x Z b Z x ∗ dt dy = ∗ dt dy + ∗ dt dy a
y
a
= = = Above we have that
and
Z x Z
Z
Z
a
x
∗
a x a
y
t
a
Z
∗ dy dt − t a
x
y y
x
x
Z b Z
Z dy dt − Z
∗ (t − a)dt +
a≤y≤x y≤t≤x
x≤y≤b x≤t≤y
b
x
b
x
∗
∗ dt dy
Z
b t
dy dt
∗ (t − b)dt =
Z
b
a
∗ k(t, x)dt.
(14.5)
⇔
a≤t≤x a≤y≤t
,
⇔
x≤t≤b t≤y≤b
.
(14.6)
gn (x, t)Ln+1 f (t)K(t, x)dt.
(14.7)
So we obtain Lemma 14.5. It holds Z b Z x Z gn (x, t)Ln+1 f (t)dt dy = a
y
b a
We give the general representation result, see also [195], as a special case.
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Taylor-Widder Representations and Gr¨ uss, Means, Ostrowski and Csiszar’s Inequalities
177
Theorem 14.6. Let f (x), u0 (x), u1 (x), . . . , un (x) ∈ C n+1 ([a, b]), n ≥ 0, the Wronskians Wi (x) > 0, i = 0, 1, . . . , n, over [a, b], x ∈ [a, b]. Then ! Z n Z b X u0 (x) b f (y) 1 f (x) = dy + Li f (y)gi (x, y)dy b − a a u0 (y) b − a i=1 a Z b 1 + gn (x, t)Ln+1 f (t)K(t, x)dt, ∀x ∈ [a, b]. (14.8) b−a a
When u0 (x) = c > 0, then ! Z b n Z b X 1 1 f (x) = f (y)dy + Li f (y)gi (x, y)dy b−a a b − a i=1 a Z b 1 + gn (x, t)Ln+1 f (t)K(t, x)dt, ∀x ∈ [a, b]. b−a a
(14.9)
Proof. Based on (14.3) and Lemma 14.5.
We also need Lemma 14.7. Let f , {ui }ni=0 ∈ C n ([a, b]), and Wi (x) > 0, i = 0, 1, . . . , n, n ≥ 0, over [a, b]. Then ( n−1 n Z b X X (n − i) Li f (b)gi+1 (x, b) − Li f (a)gi+1 (x, a) Li f (y)gi (x, y)dy = a
i=1
+
Z
a
b
Li f (y)gi+1 (x, y) φi+1 (y)
i=0
φ0i+1 (y)dy
)
+ nu0 (x)
Z
b
a
f (y) dy, ∀x ∈ [a, b]. (14.10) u0 (y)
In case of u0 (x) = c > 0 then we have ( n Z b n−1 X X Li f (y)gi (x, y)dy = (n − i) Li f (b)gi+1 (x, b) − Li f (a)gi+1 (x, a) a
i=1
+
Z
b
a
Li f (y)gi+1 (x, y) φi+1 (y)
i=0
φ0i+1 (y)dy
)
+n
Z
b
a
f (y)dy, ∀x ∈ [a, b].
(14.11)
Proof. By [339] for i = 0, 1, . . . , n − 1, we get ! ! Z y i+1 i+1 Y Y φk (y) gi+1 (x, y) = − φk (s) gi (x, s)ds. x
k=0
Thus
i+1 Y
k=0
and −d
i+1 Y
k=0
!0
!
φk (y) gi+1 (x, y)
φk (y) gi+1 (x, y)
=−
=
k=0
i+1 Y
i+1 Y
k=0
φk (y) gi (x, y)
k=0
!
φk (y) gi (x, y)dy.
!
(14.12)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
178
Consequently for i = 0, 1, . . . , n − 1 we have Z b Z b Li f (y) Ji (x) := Li f (y)gi (x, y)dy = i+1 Q a a φk (y) k=0 (14.12)
=
−
Z
b
a
i+1 Y
!
φk (y) gi (x, y)dy
k=0
! i+1 L f (y) Y i φk (y) gi+1 (x, y) i+1 d Q k=0 φk (y) k=0
=
b Z −Li f (y)gi+1 (x, y) + a
b a
!
L f (y) i φk (y) gi+1 (x, y)d i+1 . Q k=0 φk (y) i+1 Y
k=0
(14.13)
So we find for i = 0, 1, . . . , n − 1 that Ji (x) = Li f (a)gi+1 (x, a) − Li f (b)gi+1 (x, b) + θi (x), where
θi (x) :=
Z
b a
!
L f (y) i φk (y) gi+1 (x, y)d i+1 . Q k=0 φk (y) i+1 Y
(14.14)
(14.15)
k=0
Again from [339], for i = 0, 1, . . . , n − 1, we have Li f (y) i−1 Q
=
φk (y)
d 1 d 1 d d 1 d f (y) ··· , dy φi−1 (y) dy φi−2 (y) dy dy φ1 (y) dy φ0 (y)
(14.16)
k=0
and
Li+1 f (y) d 1 d 1 d d 1 d f (y) = ··· . i Q dy φi (y) dy φi−1 (y) dy dy φ1 (y) dy φ0 (y) φk (y)
(14.17)
k=0
That is, for i = 0, 1, . . . , n − 1 we derive Li+1 f (y) d = i Q dy φk (y) k=0
L f (y)φ (y) i i+1 i+1 Q φk (y) k=0
d L f (y) L f (y) i 0 i = φ (y) + i+1 φi+1 (y). i+1 Q dy i+1 Q φk (y) φk (y) k=0
k=0
(14.18)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Taylor-Widder Representations and Gr¨ uss, Means, Ostrowski and Csiszar’s Inequalities
Hence we find
d L f (y) i = Q dy i+1 φk (y)
Li+1 f (y) i Q φk (y)
−
Li f (y) i+1 Q
φk (y)
k=0
k=0
φi+1 (y)
179
φ0i+1 (y) ,
(14.19)
k=0
for i = 0, 1, . . . , n − 1. That is
L f (y) L f (y) L f (y) φ0 (y) i i+1 i i+1 d i+1 − i+1 = i+1 dy, i = 0, 1, . . . , n − 1. Q Q Q φi+1 (y) φk (y) φk (y) φk (y) k=0
k=0
k=0
(14.20)
Applying (14.20) into (14.15) we get Z b φ0i+1 (y) θi (x) = gi+1 (x, y) Li+1 f (y) − Li f (y) dy φi+1 (y) a Z b Z b dφi+1 (y) = gi+1 (x, y)Li+1 f (y)dy − Li f (y)gi+1 (x, y) . φi+1 (y) a a That is, we proved that Z b dφi+1 (y) θi (x) = Ji+1 (x) − Li f (y)gi+1 (x, y) , for i = 0, 1, . . . , n − 1. (14.21) φi+1 (y) a Therefore it holds Ji (x) = Li f (a)gi+1 (x, a)−Li f (b)gi+1 (x, b)+Ji+1 (x)−
Z
b
Li f (y)gi+1 (x, y) a
dφi+1 (y) , φi+1 (y)
and Ji+1 (x) = Ji (x) + Li f (b)gi+1 (x, b) − Li f (a)gi+1 (x, a) Z b Li f (y)gi+1 (x, y) + dφi+1 (y), φi+1 (y) a for i = 0, 1, . . . , n − 1, with J0 (x) = u0 (x)
Z
b a
f (y) dy. u0 (y)
(14.22)
(14.23)
Thus we obtain n−1 X i=0
(n − i)(Ji+1 (x) − Ji (x)) = +
n−1 X
i=0 n−1 X i=0
(n − i) Li f (b)gi+1 (x, b) − Li f (a)gi+1 (x, a) (n − i)
Z
b
a
Li f (y)gi+1 (x, y) φi+1 (y)
dφi+1 (y). (14.24)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
180
Clearly, finally we have n−1 X i=0
(n − i)(Ji+1 (x) − Ji (x)) =
n X i=1
Ji (x) − nJ0 (x),
(14.25)
proving the claim.
We give Theorem 14.8. Let f, {ui }ni=0 ∈ C n ([a, b]), and Wi (x) > 0, i = 0, 1, . . . , n, n ≥ 0, over [a, b]. Then ! Z b n Z f (y) 1 X b dy = Li f (y)gi (x, y)dy u0 (x) n i=1 a a u0 (y) n−1 X i + 1− Li f (a)gi+1 (x, a) − Li f (b)gi+1 (x, b) n i=0 Z b n−1 X i Li f (y)gi+1 (x, y) − 1− φ0i+1 (y)dy, ∀x ∈ [a, b]. n φ (y) i+1 a i=0
(14.26)
When u0 (x) = c > 0, then we obtain the representation ! Z b n Z 1 X b f (y)dy = Li f (y)gi (x, y)dy n i=1 a a n−1 X i + 1− Li f (a)gi+1 (x, a) − Li f (b)gi+1 (x, b) n i=0 Z b n−1 X Li f (y)gi+1 (x, y) i − 1− φ0i+1 (y)dy, ∀x ∈ [a, b]. n φ (y) i+1 a i=0
(14.27)
Proof. By Lemma 14.7.
We give also the alternative result Theorem 14.9. Let f, {ui }ni=0 ∈ C n ([a, b]), n ≥ 0, x ∈ [a, b], all Wi (x) > 0 on [a, b], i = 0, 1, . . . , n. Then Z b Z b f (y) u0 (x) dy = Ln f (y)gn (x, y)dy a u0 (y) a ( n−1 X + Li f (a)gi+1 (x, a) − Li f (b)gi+1 (x, b) −
Z
i=0
b
a
) Li f (y)gi+1 (x, y) 0 φi+1 (y)dy , φi+1 (y)
∀x ∈ [a, b].
(14.28)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Taylor-Widder Representations and Gr¨ uss, Means, Ostrowski and Csiszar’s Inequalities
When u0 (x) = c > 0, then Z b Z f (y)dy =
b
Ln f (y)gn (x, y)dy ( n−1 X + Li f (a)gi+1 (x, a) − Li f (b)gi+1 (x, b)
a
− Proof. Set
181
"
Z
a
i=0
b
a
) Li f (y)gi+1 (x, y) 0 φi+1 (y)dy , φi+1 (y)
Ai (x) := Li f (b)gi+1 (x, b) − Li f (a)gi+1 (x, a) +
Z
b a
# Li f (y)gi+1 (x, y) 0 φi+1 (y)dy , φi+1 (y)
∀x ∈ [a, b].
(14.29)
i = 0, 1, . . . , n − 1.
(14.30)
From (14.22) we have Ji+1 = Ji + Ai ,
i = 0, 1, . . . , n − 1.
(14.31)
That is J1 = J 0 + A 0 , J2 = J 1 + A 1 = J 0 + A 0 + A 1 J3 = J 2 + A 2 = J 0 + A 0 + A 1 + A 2 .. . n−1 X Jn = Jn−1 + An−1 = J0 + Ai , i=0
i.e.
Jn (x) = J0 (x) +
n−1 X i=0
Ai (x),
∀x ∈ [a, b].
(14.32)
Hence we established Z b Z b f (y) Ln f (y)gn (x, y)dy = u0 (x) dy u 0 (y) a a ( n−1 X + Li f (b)gi+1 (x, b) − Li f (a)gi+1 (x, a) i=0
+
Z
b
a
) Li f (y)gi+1 (x, y) 0 φi+1 (y)dy , ∀x ∈ [a, b], φi+1 (y)
proving (14.28). At last we give the following general representation formulae.
(14.33)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
182
Theorem 14.10. Let f, {ui }ni=0 ∈ C n+1 ([a, b]), n ≥ 0, x ∈ [a, b], all Wi (x) > 0 on [a, b], i = 0, 1, . . . , n. Then ( Z n−1 X u0 (x) b f (y) [Li f (b)gi+1 (x, b) − Li f (a)gi+1 (x, a)] f (x) = (n + 1) dy + (n − i) b − a a u0 (y) b−a i=0 ) Z b Li f (y)gi+1 (x, y)φ0i+1 (y) 1 dy + b−a a φi+1 (y) Z b 1 + gn (x, t)Ln+1 f (t)K(t, x)dt, ∀x ∈ [a, b]. (14.34) b−a a If u0 (x) = c > 0, then Z
b
n−1 X
(
[Li f (b)gi+1 (x, b) − Li f (a)gi+1 (x, a)] b−a a i=0 ) Z b Li f (y)gi+1 (x, y)φ0i+1 (y) 1 + dy b−a a φi+1 (y) Z b 1 + gn (x, t)Ln+1 f (t)K(t, x)dt, ∀x ∈ [a, b]. (14.35) b−a a
(n + 1) f (x) = b−a
f (y)dy +
(n − i)
Proof. By Theorem 14.6 and Lemma 14.7.
Note 14.11. Clearly (14.8),(14.9),(14.34) and (14.35) generalize the Fink identity (see [195]). We have Corollary 14.12. Let f, {ui }ni=0 ∈ C n+1 ([a, b]), n ≥ 0, x ∈ [a, b], all Wi (x) > 0 on [a, b], i = 0, 1, . . . , n. Then f (x) u0 (x) Ebn+1 (x) := − (n + 1) b−a
Z
b
f (y) dy u 0 (y) a ( n−1 X 1 [Li f (a)gi+1 (x, a) − Li f (b)gi+1 (x, b)] + (n − i) (n + 1) i=0 b−a ) Z b Li f (y)gi+1 (x, y)φ0i+1 (y) 1 − dy b−a a φi+1 (y) Z b 1 = gn (x, t)Ln+1 f (t)K(t, x)dt, ∀x ∈ [a, b]. (14.36) (n + 1)(b − a) a
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Taylor-Widder Representations and Gr¨ uss, Means, Ostrowski and Csiszar’s Inequalities
If u0 (x) = c > 0, then f (x) 1 − En+1 (x) := (n + 1) b − a
Z
183
b
f (y)dy ( n−1 X 1 [Li f (a)gi+1 (x, a) − Li f (b)gi+1 (x, b)] (n − i) + (n + 1) i=0 b−a ) Z b Li f (y)gi+1 (x, y)φ0i+1 (y) 1 − dy b−a a φi+1 (y) Z b 1 = gn (x, t)Ln+1 f (t)K(t, x)dt, ∀x ∈ [a, b]. (14.37) (n + 1)(b − a) a
Proof. By Theorem 14.10.
a
Note 14.13. If φ0i+1 (y) = 0, i = 0, 1, . . . , n − 1, then Ebn+1 (x), En+1 (x) simplify a lot, e.g. case of ui (x) = xi , i = 0, 1, . . . , n, where φi+1 (y) = i + 1. Next we estimate Ebn+1 (x), En+1 (x) via the following Ostrowski type inequality:
Theorem 14.14. Let f , {ui }ni=0 ∈ C n+1 ([a, b]), n ≥ 0, x ∈ [a, b], all Wi (x) > 0 on [a, b], i = 0, 1, . . . , n. Then ( (Z x 1 |Ebn+1 (x)|, |gn (x, t)|(t − a)dt ≤ min (n + 1)(b − a) |En+1 (x)| a Z b + |gn (x, t)|(b − t)dt kLn+1 f k∞ , kgn (x, ·)k∞ x ) max (x − a), (b − x) kLn+1 f k1 , kgn (x, ·)K(·, x)kq kLn+1 f kp ,
(14.38)
1 p
1 q
= 1, any x ∈ [a, b]. The last inequality (14.38) is sharp, namely it is attained by an fˆ ∈ C n+1 [a, b], such that ˆ |Ln+1 f(t)| = A|gn (x, t)K(t, x)|q/p , where p, q > 1 :
+
when [Ln+1 fˆ(t)gn (x, t)K(t, x)] is of fixed sign, where A > 0, ∀t ∈ [a, b], for a fixed x ∈ [a, b]. Proof. We have at first that Z b 1 b |En+1 (x)| ≤ |gn (x, t)| |Ln+1 f (t)| |K(t, x)|dt (n + 1)(b − a) a Z b kLn+1 f k∞ ≤ |gn (x, t)| K(t, x)|dt (n + 1)(b − a) a Z x Z b kLn+1 f k∞ |gn (x, t)|(t − a)dt + |gn (x, t)|(b − t)dt . = (n + 1)(b − a) a x (14.39)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
184
Next we see that |Ebn+1 (x)| ≤
kLn+1 f k1 kgn (x, ·)k∞ max (x − a), (b − x) . (n + 1)(b − a)
Finally we observe for p, q > 1 with |Ebn+1 (x)| ≤
The last is equality when
1 p
+
1 q
(14.40)
= 1, that
kLn+1 f kp kgn (x, ·)k(·, x)kq . (n + 1)(b − a)
|Ln+1 f (t)|p = B|gn (x, t)k(t, x)|q , where B > 0. The claim now is obvious.
(14.41)
(14.42)
Next we present a basic general Gr¨ uss type inequality. Theorem 14.15. Let f , h, {ui }ni=0 ∈ C n+1 ([a, b]), n ≥ 0, all Wi (x) > 0 on [a, b], i = 0, 1, . . . , n. Then Z b Z b 1 Z b 1 1 D := f (x)h(x)dx − f (x)dx h(x)dx b − a a b−a a b−a a X Z Z n b b 1 h(x)Li f (y) + f (x)Li h(y) gi (x, y)dydx − 2 2(b − a) i=1 a a Z b Z b 1 |h(x)| kLn+1 f k∞ ≤ 2(b − a)2 a a + |f (x)| kLn+1 hk∞ |gn (x, t)| |k(t, x)|dtdx =: M1 . (14.43)
Proof. We have by (14.9) that X Z b n Z b 1 1 f (x) = Li f (y)gi (x, y)dy f (y)dy + b−a a b − a i=1 a Z b 1 + gn (x, t)Ln+1 f (t)K(t, x)dt, ∀x ∈ [a, b], b−a a
and
X n Z b 1 h(y)dy + Li h(y)gi (x, y)dy b − a i=1 a a Z b 1 + gn (x, t)Ln+1 h(t)K(t, x)dt, ∀x ∈ [a, b]. b−a a
1 h(x) = b−a
Therefore it holds f (x)h(x) =
h(x) b−a +
Z
Z
b
b
f (x)dx + a
1 b−a
(14.44)
Z
b
X n Z b 1 h(x)Li f (y)gi (x, y)dy b − a i=1 a
h(x)Ln+1 f (t)gn (x, t)K(t, x)dt, a
(14.45)
(14.46)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Taylor-Widder Representations and Gr¨ uss, Means, Ostrowski and Csiszar’s Inequalities
185
and X n Z b 1 h(x)dx + f (x)Li h(y)gi (x, y)dy b − a i=1 a a Z b 1 + f (x)Ln+1 h(t)gn (x, t)K(t, x)dt, ∀x ∈ [a, b]. b−a a
f (x) f (x)h(x) = b−a
Z
b
(14.47)
Integrating (14.46) and (14.47) we have 1 (b − a)
Z
b
f (x)h(x)dx = a
Rb a
Rb f (x)dx a h(x)dx (b − a)2
X n Z bZ b 1 + h(x)Li f (y)gi (x, y)dydx (b − a)2 i=1 a a Z bZ b 1 h(x)Ln+1 f (t)gn (x, t)K(t, x)dtdx, + (b − a)2 a a
(14.48)
and 1 (b − a)
Z
b
f (x)h(x)dx = a
Rb a
Rb f (x)dx a h(x)dx (b − a)2
X n Z bZ b 1 f (x)L h(y)g (x, y)dydx i i (b − a)2 i=1 a a Z bZ b 1 + f (x)Ln+1 h(t)gn (x, t)K(t, x)dtdx. (b − a)2 a a +
(14.49)
Consequently from (14.48) and (14.49) we derive Z b Z b 1 f (x)h(x)dx − f (x)dx h(x)dx (b − a)2 a a a X n Z bZ b 1 h(x)Li f (y)gi (x, y)dydx − (b − a)2 i=1 a a Z bZ b 1 = h(x)Ln+1 f (t)gn (x, t)K(t, x)dtdx, (b − a)2 a a
1 (b − a)
Z
b
(14.50)
and Z b Z b 1 f (x)h(x)dx − f (x)dx h(x)dx (b − a)2 a a a X n Z bZ b 1 f (x)L h(y)g (x, y)dydx − i i (b − a)2 i=1 a a Z bZ b 1 = f (x)Ln+1 h(t)gn (x, t)K(t, x)dtdx. (b − a)2 a a
1 (b − a)
Z
b
(14.51)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
186
By adding and dividing by 2 the equalities (14.50) and (14.51) we obtain
∆ :=
1 (b − a)
Z
b a
f (x)h(x)dx −
1 (b − a)2
Z
b
f (x)dx a
Z
b
h(x)dx a
−
X n Z bZ b 1 h(x)L f (y) + f (x)L h(y) g (x, y)dydx i i i 2(b − a)2 i=1 a a
=
1 2(b − a)2
Z
b a
Z
b a
h(x)Ln+1 f (t) + f (x)Ln+1 h(t) gn (x, t)K(t, x)dtdx .
(14.52)
Therefore we conclude that
1 |∆| ≤ 2(b − a)2
Z
b a
Z
b a
|h(x)| kLn+1 f k∞
+ |f (x)| kLn+1 hk∞ |gn (x, t)| |K(t, x)|dtdx ,
(14.53)
proving the claim.
The related Gr¨ uss type L1 result follows. Theorem 14.16. Let f , h, {ui }ni=0 ∈ C n+1 ([a, b]), n ≥ 0, all Wi (x) > 0 on [a, b], i = 0, 1, . . . , n. Then Z b Z b 1 Z b 1 1 D= f (x)h(x)dx − f (x)dx h(x)dx b − a a b−a a b−a a
≤
X n Z bZ b 1 − h(x)Li f (y) + f (x)Li h(y) gi (x, y)dydx 2(b − a)2 i=1 a a 1 kgn (x, t)k∞,[a,b]2 khk∞ kLn+1 f k1 + kf k∞ kLn+1 hk1 =: M2 . 2
(14.54)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Taylor-Widder Representations and Gr¨ uss, Means, Ostrowski and Csiszar’s Inequalities
187
Proof. We obtain Z Z b b 1 h(x)Ln+1 f (t)gn (x, t)K(t, x)dtdx |∆| = 2(b − a)2 a a Z bZ b + f (x)Ln+1 h(t)gn (x, t)K(t, x)dtdx a a (Z Z b b 1 |h(x)| |Ln+1 f (t)| |gn (x, t)|dtdx ≤ 2(b − a) a a ) Z bZ b + |f (x)| |Ln+1 h(t)| |gn (x, t)|dtdx a
a
( Z Z ! b b 1 ≤ |Ln+1 f (t)|dt dx khk∞ kgn (x, t)k∞,[a,b]2 2(b − a) a a ) ! Z b Z b |Ln+1 h(t)|dt dx kf k∞ kgn (x, t)k∞,[a,b]2 + =
(
a
a
1 kLn+1 f k1 khk∞ kgn (x, t)k∞,[a,b]2 2 )
+ kLn+1 hk1 kf k∞ kgn (x, t)k∞,[a,b]2 =
1 kgn (x, t)k∞,[a,b]2 khk∞ kLn+1 f k1 + kf k∞ kLn+1 hk1 , 2
establishing the claim.
The related Gr¨ uss type Lp result follows. Theorem 14.17. Let f , h, {ui }ni=0 ∈ C n+1 ([a, b]), n ≥ 0, all Wi (x) > 0 on [a, b], i = 0, 1, . . . , n. Let also p, q, r > 0 such that p1 + 1q + r1 = 1. Then Z b Z b 1 Z b 1 1 D= f (x)h(x)dx − f (x)dx h(x)dx b − a a b−a a b−a a X n Z bZ b 1 − h(x)Li f (y) + f (x)Li h(y) gi (x, y)dydx 2(b − a)2 i=1 a a 1 ≤ 2−1 (b − a)−(1+ r ) kgn (x, t)K(t, x)kr,[a,b]2 khkp,[a,b] kLn+1 f kq,[a,b] # + kf kp,[a,b]kLn+1 hkq,[a,b] =: M3 .
(14.55)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
188
Proof. We derive by generalized H¨ older’s inequality that
|∆| ≤ +
Z
1 2(b − a)2 Z
b a
Z
Z
b a
b
|h(x)| |Ln+1 f (t)| |gn (x, t)K(t, x)|dtdx
a
b
|f (x)| |Ln+1 h(t)| |gn (x, t)K(t, x)|dtdx
a
1 ≤ 2(b − a)2
"Z
b a
Z
b a
|h(x)|p dtdx
1/p Z
b a
Z
b a
! |Ln+1 f (t)|q dtdx
· kgn (x, t)K(t, x)kr,[a,b]2 +
Z
b a
Z
b a
p
|f (x)| dtdx
1/p Z
b a
Z
b a
q
|Ln+1 h(t)| dtdx
1/q
1/q
kgn (x, t)K(t, x)kr,[a,b]2
#
" 1 (b − a)1/p khkp,[a,b] (b − a)1/q kLn+1 f kq,[a,b] kgn (x, t)K(t, x)kr,[a,b]2 = 2(b − a)2 + (b − a)
1/p
kf kp,[a,b](b − a)
1/q
kLn+1 hkq,[a,b] kgn (x, t)K(t, x)kr,[a,b]2
#
1 (b − a)1− r khkp,[a,b] kLn+1 f kq,[a,b] 2(b − a)2 + kf kp,[a,b]kLn+1 hkq,[a,b] kgn (x, t)K(t, x)kr,[a,b]2 ,
=
proving the claim.
We conclude the Gr¨ uss type results with Corollary 14.18 (on Theorems 14.15, 14.16, 14.17). Let f , h, {ui }ni=0 ∈ C n+1 ([a, b]), n ≥ 0, all Wi (x) > 0 on [a, b], i = 0, 1, . . . , n. Let also p, q, r > 0 such that 1p + 1q + 1r = 1. Then D ≤ min{M1 , M2 , M3 }.
(14.56)
The related comparison of integral means follows, see also [48]. Theorem 14.19. Let f , {ui }ni=0 ∈ C n+1 ([a, b]), with u0 (x) = c > 0, n ≥ 0, all Wi (x) > 0 on [a, b], i = 0, 1, . . . , n. Let µ be a finite measure of mass m > 0 on
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Taylor-Widder Representations and Gr¨ uss, Means, Ostrowski and Csiszar’s Inequalities
189
([c, d], P([c, d])), [c, d] ⊆ [a, b] ⊆ R, where P stands for the power set. Then Z Z b 1 1 f (x)dµ(x) − f (x)dx (n + 1)m [c,d] b−a a n−1
+
·
X 1 (n − i) (n + 1)m i=0 i h R R Li f (a) [c,d] gi+1 (x, a)dµ(x) − Li f (b) [c,d] gi+1 (x, b)dµ(x)
(Z
)) # Li f (y)gi+1 (x, y)φ0i+1 (y) dy dµ(x) φi+1 (y) [c,d] a ( Z Z x 1 ≤ |gn (x, t)|(t − a)dt min (n + 1)(b − a)m a [c,d] # ! Z
1 − b−a
"Z
b−a
b
b
|gn (x, t)|(b − t)dt dµ(x) kLn+1 f k∞ ,
+
x
Z
[c,d]
Z
where p, q > 1 :
[c,d] 1 p
!
(kgn (x, ·)k∞ max(x − a, b − x)) dµ(x) kLn+1 f k1 , !
kgn (x, ·)K(·, x)kq dµ(x) kLn+1 f kp
+
1 q
)
,
(14.57)
= 1.
Proof. By Theorem 14.14.
Assumptions 14.20. Next we follow [140]. Let f be a convex function from (0, +∞) into R which is strictly convex at 1 with f (1) = 0. Let (X, A, λ) be a measure space, where λ is a finite or a σ-finite measure on (X, A). And let µ1 , µ2 be two probability measures on (X, A) such that 1 µ1 λ, µ2 λ (absolutely continuous), e.g. λ = µ1 + µ2 . Denote by p = dµ dλ , dµ2 q = dλ , the (densities) Radon–Nikodym derivatives of µ1 , µ2 with respect to λ. Here we assume that p 0 < a ≤ ≤ b, a.e. on X and a ≤ 1 ≤ b. q The quantity Γf (µ1 , µ2 ) =
Z
q(x)f X
p(x) q(x)
dλ(x)
(14.58)
was introduced by I. Csiszar in 1967, see [140], and is called f -divergence of the probability measures µ1 and µ2 . By Lemma 1.1 of [140], the integral (14.58) is well-defined and Γf (µ1 , µ2 ) ≥ 0 with equality only when µ1 = µ2 . Furthermore Γf (µ1 , µ2 ) does not depend on the choice of λ. Here by assuming f (1) = 0 we can
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
190
consider Γf (µ1 , µ2 ), the f -divergence, as a measure of the difference between the probability measures µ1 , µ2 . Here we give a representation and estimates for Γf (µ1 , µ2 ) via formula (14.35). Theorem 14.21. Let f and Γf as in Assumptions 14.20. Additionally assume that f , {ui }ni=0 ∈ C n+1 ([a, b]), with u0 (x) = c > 0, n ≥ 0, all Wi (t) > 0 on [a, b], i = 0, 1, . . . , n. Then Γf (µ1 , µ2 )
Z (n + 1) b f (y)dy b−a a ( n−1 X Li f (b)Γgi+1 (·,b) (µ1 , µ2 ) − Li f (a)Γgi+1 (·,a) (µ1 , µ2 ) + (n − i) b−a i=0 !) ! p(x) Z Z b L f (y)g 0 i i+1 q(x) , y)φi+1 (y) 1 + q(x) dy dλ(x) + Rn+1 , b−a X φi+1 (y) a =
(14.59) where Rn+1
1 := b−a
Z
q(x) X
Z
b
gn a
! p(x) p(x) , t)Ln+1 f (t)K t, dt dλ(x). q(x) q(x)
(14.60)
Proof. In (14.35), set x = pq , multiply it by q and integrate against λ.
The estimate for Γf (µ1 , µ2 ) follows. Theorem 14.22. Let all as in Theorem 14.21. Then |Rn+1 |
# ("Z Z b ! p(x) p(x) 1 g n dt dλ(x) kLn+1 f k∞,[a,b] , ≤ min q(x) , t K t, b−a q(x) q(x) a X !
Z
p(x) p(x)
q(x) gn dλ(x) kLn+1 f kp1 ,[a,b] , , · K ·, q(x) q(x) p2 ,[a,b] X "Z # )
p(x) p(x) p(x)
q(x) − a, b − dλ(x) kLn+1 f k1,[a,b] , max
gn q(x) , · q(x) q(x) X ∞,[a,b]
(14.61)
where p1 , p2 > 1 such that
1 p1
+
1 p2
= 1.
Proof. From (14.60) we obtain ! Z Z b p(x) p(x) 1 dt dλ(x) g n q(x) , t |Ln+1 f (t)| K t, |Rn+1 | ≤ b−a X q(x) q(x) a =: Λ.
(14.62)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Taylor-Widder Representations and Gr¨ uss, Means, Ostrowski and Csiszar’s Inequalities
We see that 1 Λ≤ b−a
"Z
X
Book
191
# ! Z b p(x) p(x) q(x) gn q(x) , t K t, q(x) dt dλ(x) kLn+1 f k∞,[a,b] . a
(14.63) Also we have true that !
Z
1 p(x) p(x)
dλ(x) kLn+1 f kp1 ,[a,b] . Λ≤ q(x)
gn q(x) , · K ·, q(x) b−a X p2 ,[a,b] (14.64) Finally we derive
Z
1 p(x)
Λ≤ q(x) g , · n
b−a X q(x) ∞,[a,b] ! p(x) p(x) − a, b − dλ(x) kLn+1 f k1,[a,b] . (14.65) max q(x) q(x)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
This page intentionally left blank
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Chapter 15
Representations of Functions and Csiszar’s f -Divergence
In this chapter we present very general Montgomery type weighted representation identities. We give applications to Csiszar’s f -Divergence. This treatment follows [62].
15.1
Introduction
This chapter is motivated by Montgomery’s identity [265]: Theorem A. Let f : [a, b] → R be differentiable on [a, b] and f 0 : [a, b] → R integrable on [a, b]. Then the Montgomery identity holds Z b Z b 1 1 f (t)dt + P ∗ (x, t)f 0 (t)dt, b−a a b−a a where P ∗ (x, t) is the Pe´ ano Kernel defined by f (x) =
∗
P (x, t) :=
(
t−a b−a , t−b b−a ,
a ≤ t ≤ x, x < t ≤ b.
We are also motivated by the following weighted Montgomery type identity: Theorem B (see [294] Pecaric). Let w : [a, b] → R+ Ris some probability b density function, that is, an integrable function satisfying a w(t)dt = 1, and Rt W (t) = a w(x)dx for t ∈ [a, b], W (t) = 0 for t < a, and W (t) = 1 for t > b. Then Z b Z b f (x) = w(t)f (t)dt + PW (x, t)f 0 (t)dt, a
a
where the weighted Pe´ ano Kernel is W (t), a ≤ t ≤ x, PW (x, t) := W (t) − 1, x < t ≤ b. 193
Book
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
194
The aim of this chapter is to present a set of very general representation formulae to functions. They are 1- weighted and 2- weighted very diverse representations. At the end we give several applications to representing Csiszar’s f -Divergence. 15.2
Main Results
We need Proposition 15.1 ([32]). Let f : [a, b] → R be differentiable on [a, b]. The derivative f 0 : [a, b] → R is integrable on [a, b]. Let g : [a, b] → R be of bounded variation and such that g(a) 6= g(b). Let x ∈ [a, b]. We define g(t)−g(a) g(b)−g(a) , a ≤ t ≤ x, PW (g(x), g(t)) := (15.1) g(t)−g(b) , x < t ≤ b. g(b)−g(a) Then
1 f (x) = g(b) − g(a)
Z
b
f (t)dg(t) + a
Z
b
P (g(x), g(t))f 0 (t)dt.
(15.2)
a
We also need Theorem 15.2 ([241], p. 17). Let f ∈ C n ([a, b]), n ∈ N, x ∈ [a, b]. Then Z b 1 f (x) = f (t)dt + Φn−1 (x) + Rn (x), b−a a
where for m ∈ N we call Φm (x) :=
m k−1 X (b − a) k=1
k!
Bk
x−a b−a
h
f (k−1) (b) − f (k−1) (a)
(15.3)
i
with the convention Φ0 (x) = 0, and n−1 Z b (b − a) x−a x−t ∗ Rn (x) := − Bn − Bn f (n) (t)dt. n! b−a b−a a
(15.4)
(15.5)
Here Bk (x), k ≥ 0 are the Bernoulli polynomials, Bk = Bk (0), k ≥ 0, the Bernoulli numbers, and Bk∗ , k ≥ 0, are periodic functions of period one related to the Bernoulli polynomials as Bk∗ (x) = Bk (x), 0 ≤ x < 1,
(15.6)
Bk∗ (x + 1) = Bk∗ (x), x ∈ R.
(15.7)
and
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Representations of Functions and Csiszar’s f -Divergence
Book
195
Some basic properties of Bernoulli polynomials follow (see [1], 23.1). We have 1 1 (15.8) B0 (x) = 1, B1 (x) = x − , B2 (x) = x2 − x + , 2 6 and
B0∗
B1∗
Bk0 (x) = kBk−1 (x), k ∈ N,
(15.9)
Bk (x + 1) − Bk (x) = kxk−1 , k ≥ 0.
(15.10)
Clearly = 1, is a discontinuous function with a jump of -1 at each integer, and Bk∗ , k ≥ 2, is a continuous function. Notice that Bk (0) = Bk (1) = Bk , k ≥ 2. We need also the more general Theorem 15.3 (see [53]). Let f : [a, b] → R be such that f (n−1) , n ≥ 1, is a continuous function and f (n) exists and is finite for all but a countable set of x in (a, b) and that f (n) ∈ L1 ([a, b]). Then for every x ∈ [a, b] we have Z b n−1 i X (b − a)k−1 1 x − a h (k−1) f (x) = f (t)dt + Bk f (b) − f (k−1) (a) b−a a k! b−a k=1
(b − a)n−1 + n!
Z
[a,b]
Bn
x−a b−a
−
Bn∗
x−t b−a
f (n) (t)dt.
(15.11)
The sum in (15.11) when n = 1 is zero. If f (n−1) is just absolutely continuous then (15.11) is valid again. Formula (15.11) is a generalized Euler type identity, see also [143]. We give Remark 15.4. Let f, g be as in Proposition 15.1, then we have Z b Z b 1 f (z) = f (x)dg(x) + P (g(z), g(x))f 0 (x)dx. g(b) − g(a) a a
(15.12)
Further assume that f 0 be such that f (n−1) , n ≥ 2, is a continuous function and f (n) (x) exists and is finite for all but a countable set of x in (a, b) and that f (n) ∈ L1 ([a, b]). Then, by Theorem 15.3, we have f (b) − f (a) f 0 (x) = + Tn−2 (x) + Rn−1 (x), b−a where n−2 i X (b − a)k−1 x − a h (k) Tn−2 (x) := Bk f (b) − f (k) (a) , k! b−a
(15.13)
(15.14)
k=1
and
n−2
Rn−1 (x) := −
(b − a) (n − 1)!
Z
b a
x−t x−a ∗ Bn−1 − Bn−1 f (n) (t)dt. (15.15) b−a b−a
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
196
The sum in (15.14) when n = 2 is zero. If f (n−1) is just absolutely continuous then (15.13) is valid again. By plugging (15.13) into (15.12) we obtain our first result. Theorem 15.5. Let f : [a, b] → R be such that f (n−1) , n ≥ 2, is a continuous function and f (n) exists and is finite for all but a countable set of x in (a, b) and that f (n) ∈ L1 ([a, b]). Let g : [a, b] → R be of bounded variation and g(a) 6= g(b). Let z ∈ [a, b]. Then Z b Z 1 f (b) − f (a) b f (z) = f (x)dg(x) + P (g(z), g(x))dx g(b) − g(a) a b−a a n−2 Z b X (b − a)k−1 x−a (k) (k) f (b) − f (a) P (g(z), g(x))Bk dx + k! b−a a k=1 Z b n−2 Z b (b − a) x−a + P (g(z), g(x)) Bn−1 (n − 1)! a b−a a x−t ∗ − Bn−1 f (n) (t)dt dx. (15.16) b−a Here, P (g(z), g(t)) :=
g(t)−g(a) g(b)−g(a) , a ≤ t ≤ z,
g(t)−g(b) g(b)−g(a) ,
(15.17)
z < t ≤ b.
The sum in (15.16) is zero when n = 2. We need Fink’s identity. Theorem 15.6 (Fink, [195]). Let any a, b ∈ R, a 6= b, f : [a, b] → R, n ≥ 1, f (n−1) is absolutely continuous on [a, b]. Then Z b n−1 X n − k f (k−1) (a)(x − a)k − f (k−1) (b)(x − b)k n f (x) = f (t)dt − b−a a k! b−a k=1
+
1 (n − 1)! (b − a)
Z
K(t, x) :=
where
When n = 1 the sum We make
Pn−1 k=1
b a
(x − t)n−1 K(t, x)f (n) (t)dt,
t − a, a ≤ t ≤ x ≤ b, t − b, a ≤ x < t ≤ b.
in (15.18) is zero.
(15.18)
(15.19)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Representations of Functions and Csiszar’s f -Divergence
Remark 15.7. Let f, g be as in Proposition 15.1, then we have Z b Z b 1 f (z) = f (x)dg(x) + P (g(z), g(x))f 0 (x)dx. g(b) − g(a) a a
Book
197
(15.20)
Further suppose that f 0 be such that f (n−1) , n ≥ 2, is absolutely continuous on [a, b]. Then (n − 1)(f (b) − f (a)) f 0 (x) = b−a n−2 X n − k − 1 f (k) (b)(x − b)k − f (k) (a)(x − a)k + k! b−a k=1 Z b 1 + (x − t)n−2 K(t, x)f (n) (t)dt. (15.21) (n − 2)! (b − a) a P When n = 2 the sum n−2 k=1 in (15.21) is zero. By plugging (15.21) into (15.20) we obtain
Theorem 15.8. Let f : [a, b] → R be such that f (n−1) , n ≥ 2, is absolutely continuous on [a, b]. Let g : [a, b] → R be of bounded variation and a 6= b, g(a) 6= g(b). Let z ∈ [a, b]. Then Z b Z b (n − 1)(f (b) − f (a)) 1 f (x)dg(x) + + P (g(z), g(x))dx f (z) = g(b) − g(a) a b−a a n−2 X n − k − 1Z b P (g(z), g(x)) f (k) (b)(x − b)k − f (k) (a)(x − a)k dx + (b − a) k! a k=1 ! Z b Z b 1 n−2 (n) + P (g(z), g(x)) (x − t) K(t, x)f (t)dt dx. (n − 2)! (b − a) a a (15.22) When n = 2 the sum We need to mention
Pn−2 k=1
in (15.22) is zero.
Theorem 15.9 (Aljinovic- Pecaric, [14]). Let’s suppose f (n−1) , n > 1, is a continuous function of bounded variation on [a, b]. If w : [a, R bb] → R+ is some probability density function, i.e. integrable function satisfying a w(t)dt = 1, and Z t W (t) = w(x)dx f or t ∈ [a, b], a
W (t) = 0 for t < a and W (t) = 1 for t > b, the weighted Pe´ ano Kernel is W (t), a ≤ t ≤ x, PW (x, t) := W (t) − 1, x < t ≤ b.
(15.23)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
198
Then the following identity holds f (x) =
+
n−1 X k=1
Z
k−2
(b − a) (k − 1)!
n−2
(b − a) − (n − 1)!
Z
b a
Z
Z
b
w(t)f (t)dt a
b
PW (x, t)Bk−1 a
b
PW (x, s) a
∗ Bn−1
t−a b−a
s−t b−a
dt
!
f (k−1) (b) − f (k−1) (a)
− Bn−1
s−a b−a
!
ds df (n−1) (t). (15.24)
We make Assumption 15.10. Let f : [a, b] → R be such that f (n−1) , n > 1, is a continuous function and f (n) (x) exists and is finite for all but a countable set of x in (a, b) and that f (n) ∈ L1 ([a, b]). We give Comment 15.11. Let f as in Assumption 15.10. Then f (n−1) (x) absolutely continuous and thus of bounded variation. Furthermore, we can write in the last integral of (15.24) df (n−1) (t) = f (n) (t)dt. So, if we assume in Theorem 15.9 for f that f have that (15.24) and (15.25) are valid. We make
(15.25)
(n−1)
is absolutely continuous we
Remark 15.12. Let f, g be as in Proposition 15.1 , then we have Z b Z b 1 f (z) = f (x)dg(x) + P (g(z), g(x))f 0 (x)dx. g(b) − g(a) a a
(15.26)
Further assume that f be such that f (n−1) , n > 2, is a continuous function and f (n) (x) exists and is finite for all but a countable set of x in (a, b) and that f (n) ∈ L1 ([a, b]). Then using terms of Theorem 15.9 we find Z b 0 f (x) = w(t)f 0 (t)dt+ a
n−2 X k=1
Z
k−2
(b − a) (k − 1)!
(b − a)n−3 − (n − 2)!
Z
b a
Z
b
PW (x, t)Bk−1 a
b
PW (x, s) a
∗ Bn−2
t−a b−a
s−t b−a
dt
!
f (k) (b) − f (k) (a)
− Bn−2
s−a b−a
!
ds f (n) (t)dt. (15.27)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Representations of Functions and Csiszar’s f -Divergence
199
Plugging (15.27) into (15.26) we get Theorem 15.13. Let f : [a, b] → R be such that f (n−1) , n > 2, is a continuous function and f (n) (x) exists and is finite for all but a countable set of x in (a, b) and that f (n) ∈ L1 ([a, b]). Let g : [a, b] → R be of bounded variation and g(a) 6= g(b). Let w, W and PW as in Theorem 15.9, and P as in (15.1). Let z ∈ [a, b]. Then ! Z ! Z b Z b b 1 0 f (x)dg(x) + P (g(z), g(x))dx w(t)f (t)dt f (z) = g(b) − g(a) a a a
k−2 (b − a) f (k) (b) − f (k) (a) (k − 1)! k=1 ! Z b Z b t−a (b − a)n−3 · P (g(z), g(x)) PW (x, t)Bk−1 dt dx − b−a (n − 2)! a a " Z b Z b Z b s−t ∗ · P (g(z), g(x)) PW (x, s) Bn−2 b−a a a a s−a − Bn−2 ds f (n) (t)dt dx. (15.28) b−a
+
n−2 X
We need Theorem 15.14 (see [16], Aljinovic et al) Let f : [a, b] → R be such that f (n−1) , is an absolutely continuous function on [a, b] for some n > 1. If w : [a, b] → R+ is some probability density function, then the following identity holds: Z b n−1 n−1 XZ b X f (x) = w(t)f (t)dt − Fk (x) + w(t)Fk (t)dt a
+
1 (n − 1)! (b − a)
1 − (n − 1)! (b − a) Here Fk (x) :=
n−k k!
k=1 b
Z Z
a
(x − y)n−1 K(y, x)f (n) (y)dy
a b a
k=1
Z
b a
w(t)(t − y)
n−1
!
K(y, t)dt f (n) (y)dy.
f (k−1) (a)(x − a)k − f (k−1) (b)(x − b)k b−a
K(t, x) :=
t − a, a ≤ t ≤ x ≤ b,
t − b, a ≤ x < t ≤ b.
,
(15.29)
(15.30)
(15.31)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
200
We make Remark 15.15. Same assumptions as in Theorem 15.14 and n > 2. Here put (k) n−k−1 f (a)(x − a)k − f (k) (b)(x − b)k . F k (x) := k! b−a
(15.32)
Thus by (15.29) we obtain 0
f (x) = +
Z
b a
0
w(t)f (t)dt −
1 (n − 2)! (b − a)
1 − (n − 2)! (b − a)
n−2 X
k=1 Z b
Z
F k (x) +
n−2 XZ b k=1
w(t)F k (t)dt
a
(x − y)n−2 K(y, x)f (n) (y)dy
a
b
a
Z
b a
!
w(t)(t − y)n−2 K(y, t)dt f (n) (y)dy. (15.33)
Let g : [a, b] → R be of bounded variation and a 6= b, g(a) 6= g(b). Then, by Proposition 15.1, it holds (15.26). By plugging in (15.33) into (15.26) we obtain Theorem 15.16. Let f : [a, b] → R be such that f (n−1) is an absolutely continuous function on [a, b] for n > 2. Here w : [a, b] → R+ is some probability density function, and z ∈ [a, b]. Let g : [a, b] → R be of bounded variation with a 6= b, g(a) 6= g(b). Then ! Z ! Z b Z b b 1 f (z) = f (x)dg(x) + P (g(z), g(x))dx w(x)f 0 (x)dx g(b) − g(a) a a a −
n−2 XZ b k=1
P (g(z), g(x))F k (x)dx +
a
1 + (n − 2)! (b − a)
Z
Z
b
P (g(z), g(x))dx a
b
P (g(z), g(x)) a
Z
b a
(x − y)
! Z b 1 − P (g(z), g(x))dx (n − 2)! (b − a) a ! Z b Z b n−2 · w(t)(t − y) K(y, t)dt f (n) (y)dy. a
a
Here F k (x) is as in (15.32). We mention
!
n−2
n−2 XZ b k=1
w(x)F k (x)dx
a
K(y, x)f
(n)
(y)dy
!
!
dx
(15.34)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
Representations of Functions and Csiszar’s f -Divergence
201
Background 15.17 (see [144]). Harmonic representation formulae. Assume that (Pk (t), k ≥ 0) is a harmonic sequence of polynomials i.e. the sequence of polynomials satisfying
Define
Pk∗ (t),
as
Pk0 (t) = Pk−1 (t), k ≥ 1; P0 (t) = 1.
(15.35)
k ≥ 0, to be a periodic function of period 1, related to Pk (t), k ≥ 0, Pk∗ (t) = Pk (t), 0 ≤ t < 1,
Thus,
P0∗ (t)
Pk∗ (t + 1) = Pk∗ (t), t ∈ R. Pk∗ (t)
= 1, while for k ≥ 1,
(15.36)
is continuous on R \ Z and has a jump of
αk = Pk (0) − Pk (1)
(15.37)
at every integer t, whenever αk 6= 0. Note that α1 = −1, since P1 (t) = t + c, for some c ∈ R. Also, note that from the definition it follows ∗ (t), k ≥ 1 , t ∈ R \ Z. Pk0∗ (t) = Pk−1
(15.38)
Let f : [a, b] → R be such that f (n−1) is a continuous function of bounded variation on [a, b] for some n ≥ 1. In [144] the following two identities have been proved: Z b 1 ˜ 1 (x) f (x) = f (t)dt + T˜n (x) + τn (x) + R (15.39) n b−a a and
f (x) = where T˜m (x) :=
m X
k=1
1 b−a
Z
b a
˜ n2 (x), f (t)dt + T˜n−1 (x) + τn (x) + R
(b − a)k−1 Pk
for 1 ≤ m ≤ n, and
τm (x) :=
x−a b−a
m X
k=2
h
i f (k−1) (b) − f (k−1) (a) ,
(b − a)k−1 αk f (k−1) (x),
˜ 2 (x) := −(b − a)n−1 R n
Z
[a,b]
Pn∗
x−t b−a
− Pn
x−a b−a
The last two integrals are of Riemann- Stieltjes type.
(15.41)
(15.42)
with convention T˜0 (x) = 0, τ1 (x) = 0, while Z x−t 1 n−1 ∗ ˜ Rn (x) := −(b − a) Pn df (n−1) (t) b−a [a,b]
and
(15.40)
df (n−1) (t).
(15.43)
(15.44)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
202
We make Remark 15.18. Let f : [a, b] → R be such that f (n−1) is a continuous function of bounded variation on [a, b] for some n ≥ 2. Then we apply (15.40) for f 0 and for n − 1 to obtain f 0 (x) = where T˜n−2 (x) :=
f (b) − f (a) 2 ˜ n−1 + T˜n−2 (x) + τn−1 (x) + R (x), b−a
n−2 X k=1
(b − a)k−1 Pk
τn−1 (x) :=
n−1 X k=2
x−a b−a
(15.45)
f (k) (b) − f (k) (a) ,
(15.46)
(b − a)k−1 αk f (k) (x)
(15.47)
with convention T˜0 (x) = 0, τ1 (x) = 0, while 2 ˜ n−1 R (x) := −(b − a)n−2
Z
[a,b]
∗ Pn−1
x−t b−a
(15.48)
− Pn−1
x−a b−a
df (n−1) (t). (15.49)
Let g : [a, b] → R be of bounded variation with a 6= b, g(a) 6= g(b). Then by plugging (15.49) into (15.26) we obtain Theorem 15.19. Let f : [a, b] → R be such that f (n−1) is a continuous function of bounded variation on [a, b] for some n ≥ 2. Let g : [a, b] → R be of bounded variation with a 6= b, g(a) 6= g(b). The rest of the terms as in Background 15.17 and Remark 15.18. Let z ∈ [a, b]. Then ! Z b Z b 1 f (b) − f (a) f (z) = f (x)dg(x) + P (g(z), g(x))dx g(b) − g(a) a b−a a n−2 X
+
k=1
+
(b − a)
k−1
n−1 X k=2
(b − a)k−1 αk Z
[a,b]
Pn−1
Z
f
b a
(k)
(b) − f
(k)
(a)
Z
b
P (g(z), g(x))Pk a
P (g(z), g(x))f (k) (x)dx + (b − a)n−2
x−a b−a
−
∗ Pn−1
x−t b−a
df
(n−1)
Z
!
x−a b−a
dx
b
P (g(z), g(x)) a
(t) dx.
(15.50)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Representations of Functions and Csiszar’s f -Divergence
Book
203
If f (n) exists and is integrable on [a, b], then in the last integral of (15.50) one can write df (n−1) (t) = f (n) (t)dt. We need Background 15.20 (see [12]). General Weighted Euler harmonic identities. For a, b ∈ R, a < b, let w : [a, b] → [0, ∞) be a probability density function i.e. integrable function satisfying Z b w(t)dt = 1. (15.51) a
For n ≥ 1 and t ∈ [a, b] let 1 wn (t) := (n − 1)!
Z
t a
(t − s)n−1 w(s)ds.
(15.52)
Also, let w0 (t) = w(t), t ∈ [a, b].
(15.53)
It is well known that wn is equal to the n- th indefinite integral of w, being equal to zero at a, i.e. wnn (t) = w(t) and wn (a) = 0, for every n ≥ 1. A sequence of functions Hn : [a, b] → R, n ≥ 0, is called w- harmonic sequence of functions on [a, b] if Hn0 (t) = Hn−1 (t), t 6= 1; H0 (t) = w(t), t ∈ [a, b].
(15.54)
The sequence (wn (t), t 6= 0) is an example of w- harmonic sequence of functions on [a, b]. Assume that (Hn (t), n ≥ 0) is a w- harmonic sequence of functions on [a, b]. Define Hn∗ (t), for n ≥ 0, to be a periodic function of period 1, related to Hn (t) as Hn∗ (t) =
Hn (a + (b − a)t) , 0 ≤ t < 1, (b − a)n
Hn∗ (t + 1) = Hn∗ (t), t ∈ R.
(15.55) (15.56)
Thus, for n ≥ 1, Hn∗ (t) is continuous on R \ Z and has a jump of βn = at every t ∈ Z, whenever βn 6= 0. Note that
Hn (a) − Hn (b) (b − a)n
β1 = −
1 , b−a
(15.57)
(15.58)
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Book
PROBABILISTIC INEQUALITIES
204
since H1 (t) = c + w1 (t) = c + for some c ∈ R. Also, note that
Z
t
w(s)ds,
(15.59)
a
∗ (t), n ≥ 1, t ∈ R \ Z. H ∗ 0n (t) = Hn−1
(15.60)
(n−1)
Let f : [a, b] → R be such that f exists on [a, b] for some n ≥ 1. For every x ∈ [a, b] and 1 ≤ m ≤ n we introduce the following notation m m h i X X [Hk (a) − Hk (b)] f (k−1) (x), Sm (x) := Hk (x) f (k−1) (b) − f (k−1) (a) + k=2
k=1
(15.61)
with convention S1 (x) = H1 (x) [f (b) − f (a)]. Theorem 15.21 (see [12]). Let (Hk , k ≥ 0) be w- harmonic sequence of functions on [a, b] and f : [a, b] → R such that f (n−1) is a continuous function of bounded variation on [a, b] for some n ≥ 1. Then for every x ∈ [a, b] Z b f (t)Wx (t)dt + Sn (x) + Rn1 (x), f (x) = (15.62) a
where
Rn1 (x)
:= −(b − a)
with Wx (t) :=
n
Z
[a,b]
Hn∗
x−t b−a
df (n−1) (t),
w(a + x − t), a ≤ t ≤ x, w(b + x − t), x < t ≤ b.
(15.63)
(15.64)
Also we have Theorem 15.22 (see [12]). Let (Hk , k ≥ 0) be w- harmonic sequence of functions on [a, b] and f : [a, b] → R such that f (n−1) is a continuous function of bounded variation on [a, b] for some n ≥ 1. Then for every x ∈ [a, b] and n ≥ 2 Z b f (x) = f (t)Wx (t)dt + Sn−1 (x) + [Hn (a) − Hn (b)] f (n−1) (x) + Rn2 (x), (15.65) a
while for n = 1
f (x) = Rn2 (x)
:= −(b − a)
n
Z
[a,b]
Z
b a
Hn∗
f (t)Wx (t)dt + R12 (x),
(15.66)
(15.67)
x−t b−a
Hn (x) − df (n−1) (t), (b − a)n
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
Representations of Functions and Csiszar’s f -Divergence
Book
205
for n ≥ 1. Comment 15.23 (see [12]). In the case when ϕ : [a, b] → R isR such that ϕ0 exists and is integrable on [a, b], then the Riemann- Stieltjes integral [a,b] g(t)dϕ(t) Rb is equal to the Riemann integral a g(t)ϕ0 (t)dt. Therefore, if f : [a, b] → R is such that f (n) exists and is integrable on [a, b], for some n ≥ 1, then Theorem 15.21 and Theorem 15.22 hold with Z x−t f (n) (t)dt, (15.68) Rn1 (x) := −(b − a)n Hn∗ b − a [a,b] and Rn2 (x)
:= −(b − a)
n
Z
[a,b]
Hn∗
x−t b−a
Hn (x) − f (n) (t)dt. (b − a)n
(15.69)
We make Remark 15.24. We use the assumptions of Theorem 15.22, n ≥ 2. We apply first (15.65) for f 0 and n − 1 to get for n ≥ 3, Z b 0 ˜ 2 (x), f (x) = f 0 (t)Wx (t)dt + S˜n−2 (x) + [Hn−1 (a) − Hn−1 (b)] f (n−1) (x) + R n−1 a
(15.70)
where S˜n−2 (x) =
n−2 X k=1
and
n−2 X Hk (x) f (k) (b) − f (k) (a) + (Hk (a) − Hk (b)) f (k) (x), (15.71) k=2
S˜1 (x) = H1 (x)(f 0 (b) − f 0 (a)), while for n = 2, by (15.66), we obtain Z b 0 ˜ 2 (x), f (x) = f 0 (t)Wx (t)dt + R 1
(15.72)
(15.73)
a
here, by (15.67), we have 2 ˜ n−1 R (x) = −(b − a)n−1
Z
[a,b]
∗ Hn−1
x−t b−a
Hn−1 (x) − df (n−1) (t), (b − a)n−1
(15.74)
for n ≥ 2. Plugging (15.70), (15.73) into (15.26) we derive Theorem 15.25. Let (Hk , k ≥ 0) be w- harmonic sequence of functions on [a, b] and f : [a, b] → R such that f (n−1) is a continuous function of bounded variation on [a, b] for some n ≥ 2. Let g : [a, b] → R be of bounded variation on [a, b] such that g(a) 6= g(b). Here Wx is as in (15.64). Let z ∈ [a, b].
22nd June 2009
18:37
World Scientific Book - 9.75in x 6.5in
