This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
0}. These classes of functions turn out to be useful in studying well-conditioned convex functions, uniformly convex and uniformly smooth convex functions, as well as in the study of the geometry of normed spaces. As an illustration of the use of these classes of functions we study the differentiability of convex functions with
XV111
Introduction
respect to arbitrary bornologies. Using one of the characterizations and the Br0ndsted-Rockafellar theorem one obtains the following interesting result of Asplund and Rockafellar: Let X be a Banach space and / : I - > l a proper lower semicontinuous convex function; if / * is Frechet differentiable at x* e int(dom/*) then V/*(x*) € X. In Section 3.4 we introduce the well-conditioned convex functions and give several characterizations of this notion using the conjugate and the subdifferential of the function. An important special case is that of wellconditioning with linear rate. This situation is studied in Section 3.10. In Section 3.5 we study uniformly convex and uniformly smooth convex functions, respectively. To any convex function / one associates the gages pj and 07 of uniform convexity and uniform smoothness, respectively. The gage pf has an important property: the mapping 0 < t i-» t~2pf(t) is nondecreasing. Because a/* = ( p / ) * and a/ = (p/*) # f° r a n v proper lower semicontinuous convex function / , the mapping 0 < i 4 t~2crf(t) is nonincreasing for such a function; moreover, one obtains that for such an / , / is uniformly convex if and only if / * is uniformly smooth and / is uniformly smooth if and only if / * is uniformly convex. Then one establishes many characterizations of uniformly convex functions and of uniformly smooth convex functions. In these characterizations appear functions (gages or moduli) belonging to different subclasses of A introduced in Section 3.3. These gages and moduli are sharp enough in order to obtain that / is c-strongly convex if and only if /* is Frechet differentiable on X* and V / * is c _1 -Lipschitz. Even if the results are established in general Banach spaces the natural framework for uniformly convex and uniformly smooth convex function is that of reflexive Banach spaces. This is due to the fact that when there exists a proper lower semicontinuous and uniformly convex function on a Banach space whose domain has nonempty interior, the space is necessarily reflexive. Section 3.6 is dedicated to the study of those convex functions which are uniformly convex on bounded sets and uniformly smooth on bounded sets, respectively. Under strong coercivity of the function one shows that these notions are dual. In Section 3.7 we study the function / v : X ->• E, f^x) = / „ " tp(t) dt, where
E,
x such that {0} x Vo C {{x,Ax)
\xeB}-[F<
Ao];
(ii) for every U £ Nx there exist A > 0 and V £ Ny0 such that {0} x V C {(x, Ax)\xe
At/} ~[F<
A];
(iii) there exists xo £ X such that (:ro,j4a;o) £ d o m F and F(xo,-) continuous at AXQ;
is
(iv) X and Y are metrizable, 0 E tbD and either F is cs-closed and gr A is cs-complete or F is cs-complete; (v) X is a Frechet space, Y is metrizable, F is li-convex and 0 £ lhD; (vi) X is a Frechet space, F is Isc and 0 £ lbD; (vii) X and Y are Frechet spaces, F is Isc and 0 £ %CD; (viii) dimy 0 < oo and 0 £ 'D,
Convex analysis in locally convex spaces
122
(ix) there exists XQ £ X such that F(xo, •) is quasi-continuous and {0} and D are unitedThen for x* £ X*, x £ domip and e > 0 we have: tp*(x*) = mm{F*(x* - A*y\y*) dM*) Proof.
= {A*y*+x*
\ y* £ Y*},
(2.56)
| (x*,y*) € dEF(x, Ax)}.
(2.57)
It is easy to verify that Vx* £ X* :
0 and V € Ky0 such that V C [g < A] - e(XU); (ii) there exist XQ € R, B € 3x X] - G(B);
and V0 € Ny0 such that V0 C [g <
(iii) there exists j/o € dorngfllmC such that g is continuous at yo; (iv) X, Y are metrizable, g has proper conjugate, either g is cs-closed and C is cs-complete, or g is cs-complete and C verifies (Hz), and 0 £ lbD; (v) X, Y are Frechet spaces, g, C are li-convex and 0 6 lbD, (vi) dim Y0 < oo and 0 6 ' D . Then Vi'el*
: ip*{x*)=mm{g*{y*)
\ x* G C*(*)}.
Moreover, ifxE domtp = C - 1 (domg) is such that
0 then de(p(x) C C* {deg(y)) (with equality i/grC is a linear sub space). Proof. Then
Let x* £ X*. Consider F : X x Y -)• I , F(x,y)
0. Let x* € dE(p(x). Since
• (ii) =» (i) and (iv) V (v) => (i). Important situations when gr C is a linear subspace are: C = A"1 with A £ &(X, Y) and 6 = A with A a densely defined closed operator {i.e. A : D(A) -> Y, D(A) being a dense linear subspace of X, A being a linear operator and gr A a closed subset of X x Y; see also Exercise 1.11). T h e o r e m 2.8.7 Let / , € A(X). Assume that d o m / D domg ^ 0 and let Xo := lin (dom/ — doing). Assume that one of the following conditions holds: (i) there exist Ao £ K, B £ S ^ and VQ £ 3sfx0 SMc/t ^ f l £ VoC[/ 0, £ = £x + e 2 . Then f{x) + (y* o H){x) + (f + y*o H)* {x*) < (x, x*) + ex, g(H(x))+g*(y*)<(H(x),y*)+e2. Adding them side by side we get ip(x) + g*(y*) + (/ + y* ° H)* (x*) < {x,x*) + e. Using Eq. (2.68) we obtain that x* € deip(x). Therefore Eq. (2.69) holds. Let F(x, y) := f(x) + g(y) and G : X =t Y with gr C := epi H; then F £ A(X x Y) and 6 is convex. Since g is increasing on H(dom H) + Q, it follows that (p(x) = int{F(x,y) + iep\H{x-,y) \ y 6 Y] for every x € X. Hence ip is the marginal function associated to the convex function F + iepi H ', hence ip is convex. If one of the conditions (i) — (vi) is verified, then F and 6 satisfy the corresponding condition of Theorem 2.7.4. (When / and g are cs-complete and have proper conjugates, similarly to the proof of Proposition 2.2.17, we get that F is cs-complete.) As noticed after the proof of that theorem, also the perturbed function F, F(x,y) = F(x,y) — {x,x*), satisfies the same condition. Therefore there exists z* £ Y* such that a : = p*{x*) = min{(Ai/i + • • • + A n / n )*(x*) | ( A i , . . . , An) e A„}, E, >p := m a x { / i , / 2 } . Suppose that one of following conditions is verified: (i) there exists XQ G dorafa n d o m / 2 such that fa is continuous at XQ; (ii) X is a Frechet space, fa, fa are li-convex and 0 G l 6 (dom/i — dom/2); (iii) dim X < 00 and ^dom fa) n *(dom fa) ^ 0. £ T(E) and > is continuous on domy D [0, ||a; — j/||]. It is obvious that >'(t) = g'(y + tu,u) for every £ £ domyj. By Theorem 2.1.5(v) we have that lim g'(y + tu,u) = lim 1 + | 0 then 0 for s > 0, and so we get a function i(i) := a\t\ with a > 0, t~p(f(t) is nondecreasing (nonincreasing) on P then the mapping s i-» s~g R, f(x) := ip(\\x\\). Then f*{x*) = >#(||:r*||) for every x* £ X* and f**{x) = <^*#(||a;||) for every x £ X. Proof, (i) Let ip £ A0nN0 and assume that 0 such that 0; then there exists e > 0 such that e. Then ip < ip, and so (p* > ip*- In 0. For 0 < s < e~lip(e) we have that 0] — max {sup{is - e}} < max{se,sup{i(s — e~1ip(e)) | t > e} — se, and so limS4.0 s~lip#(s) = 0. This means that cpy(t) for all* > 0 and c > 1. Let d > 1 and s > 0. Then dq 0} > sup{(ds)(dq/pt) 0} = 0,
A];
S«C/J
that
Formulas for conjugates and
s-subdifferentials
127
(iii) there exists XQ G dom / n dom g such that g is continuous at XQ ; (iv) X is metrizable, f,g have proper conjugates, f is cs-closed, g is cs-complete and 0 G l 6 ( d o m / - domg); (v) X is a Frechet space, f,g are li-convex and 0 G l i > (dom/ — domg); (vi) X is a Frechet space, f,g are Isc and 0 € l 6 ( d o m / — domg); (vii) X is a Frechet space, f,g are Isc and 0 G I C (dom/ — domg); (viii) dimX 0 < oo and 0 G *(dom/ — domg); (ix) g is quasi-continuous and d o m / and domg are united; (x) X is a Frechet space, f,g are li-convex and (0,0) G lb[{(x,x) X} — dom / x dom g).
\x G
Then for x* € X*, x G dom / Pi domg and e > 0 we have: ( / + )*(**) = mi n {/*(** - y*) + g*(y*) | v* G X*} = (/*Og*)(x*), (2.63) de(f + 9)(x) = {J{deif(x)
+ dE2g(x) \ei,e2
> 0, ex + e2 =e),
d(f + g)(x) = df(x) + dg(x). Proof. Taking A = Idx, the conclusion follows from Theorem 2.8.3 under conditions (i) - (iii), (v) - (ix). If (iv) holds, taking Y = X and $(x,y) := f(x) + g(x — y), condition (iv) of Theorem 2.7.1 is verified. If (x) holds consider F : X x X -)• I , F{x,y) := f{x) + g{y) and A : X -> X x X, A(x) :— (a;, a;). Then condition (v) of Theorem 2.8.6 holds; applying it we obtain the conclusion, taking into account that A*{x*,y*) = x* + y*. • The same implications as in Theorem 2.7.1 (mentioned in Proposition 2.7.2) hold. Corollary 2.8.8 Let X be a Frechet space and f, g G A(X). / / / and g are li-convex then for every x G * 6 (dom/ + domg) we have (/Dg)(x) = (/*+$*)•(*) = m&x{(x,x*)
- f*(x*)
- g*(x*) \ x* € X*}. (2.64)
Proof. Let a;o G s 6 (dom/ + domg) and consider h G A(X), h(x) := g(a;o — x). Then dom/i = XQ — domg, and so d o m / - dom/i = d o m / + domg — XQ- Therefore 0 G l 6 ( d o m / — dom/i), and so condition (v) of
Convex analysis in locally convex spaces
128
Theorem 2.8.7 is satisfied. From formula (2.63) applied for x* = 0 we get (fn9)(x0)
= inf. (/Or) + h{x)) = - ( / + h)*(0) = - min ( / • ( * * ) + >»*(-**))• X*£X *
But h*{—x*) = s u p { - (x,x*) — g(xo — x) \ x € X} = g*(x*) — (x0,x*), and so (2.64) holds for x = x0. O Of course, one can obtain the conclusion of the preceding corollary also for other situations corresponding to conditions of Theorem 2.8.7. For example, if there exist A0 £ M, B € "S>x, VQ € Nx0 (x) s u c n that Vo C [/ < A0] n B + [g < A0], where X0 - aff(dom/ + dom^), then Eq. (2.64) holds. Proposition 2.8.9 Let f,g € A(X) and take D = d o m / - domg. If dim(lin£>) < oo then {D = rintZ?, while if X is metrizahle, f,g have proper conjugates, f is cs-closed, g is cs-complete and tbD ^ 0 then lbD = rint D; similarly for the situations corresponding to conditions (v)-(vii) of Theorem 2.8.7. Suppose that X is a Banach space and f, g are li-convex. Then for every x € lbD there exist n, A > 0 such that (x + nUx) n aff D C [f < A] n \UX - [g < A] D \UXProof.
$(x,y)
(2.65)
The first part follows from Proposition 2.7.2 taking Y = X and
=f(x)+g(x-y).
Suppose that X is a Banach space and / , g are li-convex; consider x £ lb D. Replacing / by / , f(u) — f(u + x), we may suppose that x = 0. It follows that condition (v) of Theorem 2.8.7 holds, and, as noted after its proof, condition (i) is verified. Therefore there exist 77 > 0, Ao G B. and B £"Bx such that rjUx n aff D C [/ < Ao] n B - [g < A0]. Taking A' > max{Ao, 0} such that B C X'Ux and A = A' + n we obtain that rjUx n aff D C [/ < A0] 0 B - [g < A0] D (B + nUx)
c [/ < A]n xux - [f < A]n \ux. The proof is complete. Another important result is the following.
•
Formulas for conjugates and
E-subdifferentials
129
Theorem 2.8.10 Let Y be ordered by the convex cone Q, f £ A(X), H : X —> Y' be convex and g £ A(y) be Q-increasing on H(domH) + Q. Then tp :— f + goH is convex. Assume that 0 £ D, where D :— H(domHC\ d o m / ) — domg + Q, and consider Yo := linD. Assume that one of the following conditions holds: (i) for every U £ J^x there exist A > 0 and V € Ky0 such that V C H (XU n [/ < A] n d o m F ) - [g < A] + Q; (ii) there exist XQ 6 R, B £ 1$x and VQ £ Ny0 such that V0 C H (B n [/ < A0] n domH) -[g<
A0] + Q;
(iii) there exists XQ £ d o m / fl i f - 1 (domg) such that g is continuous at H(x0); (iv) X, Y are metrizable, f, g have proper conjugates, either f, g are cs-closed and epiif is cs-complete, or f,g are cs-complete and epiH is cs-closed, and 0 G lbD; (v) X, Y are Frechet spaces, f,g, epiH are li-convex and 0 £ (vi) dim Y0 < oo and
tb
D;
i
0£ D;
(vii) g is quasi-continuous, and domg and H(domH united.
f~l d o m / ) + Q are
Then for x* £ X*, x £ dormp = d o m / n H~1(domg) have:
and e >Q, we
deV{x) = | J {0Er (f + y*° H){x) \y*£Q n
dE2g(H(x)),
(2.66) El
+ e2 = e} . (2.67)
Proof. have
First observe that for all x* £ X*, y* £ Q+ and x £ dom<£ we
V ' O O < (/ + V* o H)*(x*) + g*(y*), de
+
+ y*o H)(x) \y*£Q n
de2g(H(x)),
(2-68) El
+ e2 = e], (2.69)
Convex analysis in locally convex spaces
130
without any supplementary condition on / , g and H. Indeed, let x € dom<^ = d o m / n H-^domg), x* € X* and y* £ Q+. Then f{x) + (y* o H)(x) + (f + y*o H)* (x*) > (x, x*), g(H(x))+g*(y*)>(H(x),y*), whence, adding them side by side, we get g*(y*) + (/ + y* ° H)* (x*) > (a;, a;*) -
jnf
,
=
.1TUix)+g{y)-{x,x''))
inf
(f(x)+g(y)-(x,x*)
+
(z,z*))=:(3.
{x,y+z)EepiH
Using again the fact that g is increasing on H(domH) a =
Jnf
w
xedom H
=
inf iGdom H
j?^
n
+ Q, we have
VW + 9{y) ~ (x,x*))
yeH(x)+Q
(f(x)+g(H(x))
- {x,x*)) = ~
Formulas for conjugates and
s-subdifferentials
131
and P=
inf
inf
(f(x) + g(y) - (x, x*) + (H(x) +q-y,
z")).
iGdomif q€Q, y€Y
It follows that P = - o o if z* $ Q+. If z* £ Q+ then -0=
sup x€dom
sup((x,x*)-f(x)-(H(x),z*)
+
(y,z*)-g(y))
Hy£Y
= (/ + Z*o #)*(*•) + •(**)• Taking into account Eq. (2.68), it is clear that Eq. (2.66) holds. In the case (vii) consider F : I x Y ->• K, F(x,y) := f(x)+g(H(x)+y)(x,x*). Because 0 € D and g is Q-increasing on H(domH) + Q, there exists xo € d o m / n H~1(domg). It follows that F(x0, •) is quasi-continuous and {0} and D — Pry (dom F) are united. Applying Theorem 2.7.1(ix) we have that inf x e x F(x,0) = max y . e y.(—F*(0,y*)). With a similar calculation as above we obtain that Eq. (2.66) holds. Let x 6 donup and x* 6 d£ip(x). Using Eq. (2.66), there exists y* € Q+ such that ip*(x*) = (f + y* o H)*(x*) + g*{y*). It follows that {f+y*oH){x)+{f+y*oHY{x*)-(x,x*)+9{H{x))+g*{y*)-{H{x),y*)
< e.
Taking £ l := (/ + y* o tf)(x) + (/ + y* o #)*(x*) - (x,x*) (> 0 by the Young-Fenchel inequality) and e 2 = e - ei, we have that y* € d£2g(H(x)) and Z* € d£l(f + y* oH)(x). Therefore the inclusion C holds in Eq. (2.67); taking into account Eq. (2.69), we have the desired equality. D We use the preceding theorem to obtain formulas for conjugates and subdifferentials of functions of "max" type. Corollary 2.8.11
Let / i , . . . , / „ E \(X)
y.X^W,
and
¥>(*):=/i(z)V---V/„(z).
Suppose that dom ip = f}™=1 dom /i ^ 0 and e € IR+. For x € dom ip denote by I(x) the set {i G l , n | fi(x) =
Convex analysis in locally convex spaces
132
while for every x £ dom
(Ai, . . . , A„) G A„,
77 e [0,e], Yli=1Xi^x^
- ¥>(x)+ri-£o}
,
dV{x) = U { 5 (E" = 1 A J i ) {x) I (Al!' • •'An) € A"' Vifl{x) Proof.
: Ai = o}.
Let us consider the functions
H:X->(W\Rl),
H(x):=( "•" 3: «"-•«,
(A(*)>••"'/«(*)) [00
if x € fX=i dom/,, otherwise,
g(y):=yiV.---Vyn.
It is clear that condition (iii) of the preceding theorem is verified. Taking into account the expressions of g* and deg(y) given in Corollary 2.4.17 we obtain immediately the relations from our statement. D An application of the previous result is given in the following example. Example 2.8.1
Let / € A(X) and consider /+ := / V 0. Then
f df(x) df+(x) = l U { W ) ( x ) | A e [ 0 , l ] } { d(0f)(x) = didomf(x)
if f(x) > 0, if f(x) = 0, if f(x)<0.
A useful particular case of the result in Corollary 2.8.11, in which we can give explicit formulas for ip* and detp without supplementary conditions, is presented in the following corollary. Corollary 2.8.12
--1J=I
Let f{ 6 A(X,) for i € T~n and Xi-+W,
tp(x1,...,xn):=f1(xi)V---Vfn(xn).
For x = (xi,... ,xn) S domtp = Yl7=i dom/j consider I(x) := {i G l , n | fi(Xi) =
(Xu...,\n)eAn}.
(2.70)
Formulas for conjugates and e-subdifferentials
For x — (xi,..., dMx)
133
xn) E dom
= {j{~[[n.=1dei(^fiKxi)\
Eni=o
( A i , . . . , A „ ) G A n , et > 0, V~^n £i = £
1 Xi Xi
' Z^i=1 ^ ^
- ^
£
- o) >
S ^ ^ - l J l n ^ ^ A i / O C ^ ) ! (A1,...,An)€A„, VigJ(a:) : A; = o} . Proof. We apply the preceding corollary to the functions fa : X ->• R, /i(a;) := fi(xi), then we use Theorem 2.3.1 (viii) for arbitrary n and Corollary 2.4.5. • Other situations when one has explicit formulas for tp* and d
3
+ d(»g)(x) I (A,M) G A 2 , A / ( X ) + ng(x) =
Proof. When / and g verify one of the conditions (i)-(iii), (v), (viii)— (x) of Theorem 2.8.7 and A,/x > 0, then the functions A/ and fig also verify the same condition. If condition (vi) or (vii) holds then, by the relations among the classes of convex functions on page 68, condition (v) holds. So, (Xf + ng)*{x*)=Toia{(Xf)*{u*) + (jjig)*{v*) \u*+v* = x * } a n d d(\f + ng)(x) = d(Xf)(x) + d(fig)(x). Using Corollary 2.8.11 one obtains the conclusion. • Taking into account that for x G dom / , one has df(x) + N(dom f, x) = df(x) (see also Exercise 2.23), d(Xf)(x) = \df(x) for A > 0 and d{0f)(x) = N(domf,x), we get for f,g G A(X) and x G X with f(x) = g(x) G R,
U{W)W+8WWI(A,|f)6A 2 } -co(df(x)Udg{x))
+ N(domf,x)
+N(domg,x).
This formula can be extended easily to a finite number of functions. Using Corollary 2.8.12 we get the following result concerning the maxconvolution.
134
Convex analysis in locally convex spaces
Corollary 2.8.14 have that
Let / i , / 2 € A{X) and e £ 1+ . For every x* £ X* we
(hQf2y(x*)=rnin{(\1f1y(x*)
+ (\2f2y(x*)
| (A1;A2) £ A 2 } .
(2.71)
Suppose that (/i0/ 2 )(a:) = max{/i(2;i),/ 2 (a; 2 )}, where xi £ d o m / i , x~2 £ dom /2 and x = x~i + x2. Then
de{hOh){x) = \J{dEl(^h)(xi)ndE2(\2f2)(x2) e i , e 2 > 0 , £i+e2<e
+ X1f1(x1)
I (Ai,A2) e A2,
+ X2f2(x2)-(f10f2)(x)}.
(2.72)
Furthermore, if fi(x\) = f2(x2) *^en Eq. (2.73) is verified, while if fi(xi) 72(^2) *^en i?g. (2.74) is verified, where d(fi0f2)(x)=[j{d(X1f1)(x1)nd(X2f2)(x2) d(fi0f2)(x) Proof.
I (A1;A2) G A 2 } ,
= dMxx) nN(domf2-:x2).
>
(2.73) (2.74)
Let x* £ X*; then
{fi0f2)*(x*)
= s u p i e X ( ( x , x * ) - i n f f / ^ ) V / 2 ( z 2 ) I *! + x2 = x}) = sup{(ii,a;*) + (a;2,a;*) - fi(xi)
V f2(x2)
\x1} x2 £ X)
= min{(A 1 / 1 )*(x*) + (A 2 / 2 )*(s*) | (Ai,A2) £ A 2 } , the last equality being obtained by using Eq. (2.70). The inclusion "D" of Eq. (2.72) can be verified easily. Consider x* £ de(fi0f2)(x). By Eq. (2.71), there exist (Ai, A2) £ A 2 such that (/i0/2)*(x*) = (Ai/i)*0O + (A 2 / 2 )*(x').
(2.75)
Using the preceding relation we obtain that 0 < [ ( A 1 / 1 ) ( ^ 1 ) + (A 1 / 1 )*( 2; *)-(5; 1 ,a;*)] + [(A 2 / 2 )(i 2 ) + (A 2 / 2 )*(:c')-<S 2 ) a :*)] < (fiOh)(x)
+ ( / i O / 2 ) * 0 0 - (x,x*) < e.
Taking e, := (Aj/i)(zj) + {Xifi)*(x*) - (x~i,x*) > 0, i £ {1,2}, using the preceding relation and Eq. (2.75), we get ( / i 0 / 2 ) ( z ) + £ 1 - Ai/i(3fi) + e 2 - A 2 / 2 (x 2 ) < e, and so x* belongs to the set on the right-hand side of relation (2.72).
Formulas for conjugates and
s-subdifferentials
135
Let us prove now the equalities (2.73) and (2.74). The inclusions "D" follow directly from Eq. (2.72). Let us prove the converse inclusions. Let x* G d{faOfa)(x) = d0(faOfa)(x). By Eq. (2.72), there exist (Ai, A2) G A 2 and £i, £2 > 0 such that x* £1 + e 2 < Xifi(xi)
£dei(Xifi)(x1)ndea(Xif2)(x2), + A 2 / 2 (x 2 ) - (faOfa){x) < 0.
Therefore £1 = £2 = 0 and Ai/i(xi) + A 2 /2(x 2 ) = {fa(>fa){x); hence x* belongs to the set on the right-hand side of Eq. (2.73) when / i ( x i ) = fa{x2)If fi{x~i) > / 2 ( ^ 2 ) , the preceding relation shows that A2 = 0 , Ai = 1, and so x* G 9/i(11),
x* G 0(0 • / 2 )(z 2 ) = <9idom/2(^2) = -/V(dom/ 2 ;x 2 ).
This shows that x* belongs to the set on the right-hand side of relation (2.74). • Note that in Eq. (2.72) we can take E\ + £2 = £ + •••. Moreover, if /1 is continuous at x\ and fa is continuous at a;2, then in Eq. (2.74) we have d(fiOfa)(x) = {0}. Indeed, in this situation, N(domfa;x~2) = {0} (since x~2 G int(dom/ 2 )); since /1O/2 is continuous at x, d(fiQfa)(x) ^ 0. In particular, it follows that X\ is a minimum point of fa. Furthermore, in this situation, /1O/2 is even Gateaux differentiate. Introducing the convention that 0 • df(x) := N(domf;x) for / G A(X) and x G d o m / , formula (2.75) may be written in the form d(fa0fa)(x)
=
dfa(x1)odfa(x2),
where A o B represents the harmonic sum of the sets A and B, which generalizes the inverse sum of A and B. In order to have more explicit formulas in Corollary 2.8.11 it is necessary to impose some supplementary conditions. Let us give such conditions and the corresponding formulas in three situations for n = 2. Corollary 2.8.15 Let fa, fa G A(X), e G 1+ and
Convex analysis in locally convex spaces
136
Then, for all x* € X* and x £ dom/i D dom/2 we have: ¥>*(*•) = min{(Ai/i)*(a;I) + (A 2 / 2 )*(^) | (Ai, A2) e A 2 , x\ + x*2 = x*}, dMx)
= \J{dEl(Xifi)(x)+dE2(X2f2){x) e0+ei
I (Ai,A2) G A 2 , e 0 > ei,e 2 > 0,
+ e2 = e, Xifi(x)
dtp(x) = \J{d(X1h)(x)+d(X2f2)(x)
+ X2f2{x) >
I (Ai,A2) G A2>
Xih(x) + x2f2{x) = tp(x)}. Let defined by
-e0},
a
* £ X* and consider the functions / i , / 2 : X —>• K
/i(a;) : = m a x { ( i , a ; ; ) , . . . , ( j ; , < ) } ,
/ 2 (x) := |{a;,a;*)|.
Applying the preceding corollary, we get the following formulas for every xeX: d
h(x)
= { ]C" = 1 A ^i I (Ai, • • •, A„) € A„, ^ = 0 if (x, x*) < /!(a;)} , r {x*} 0/2 (*) = < {Az* I A € [ - 1 , +1]} 1 {-a;*}
2.9
if if if
(x,x*) > 0 , (x, x*) = 0, (x,x*) < 0 .
Convex Optimization with Constraints
Let us come back to the general problem of convex programming as considered in Section 2.5, (P)
min f(x),
x £ C,
where / € A(X), C C X is a convex set and C (~l d o m / ^ 0; the spaces considered in the present section are separated locally convex spaces if not stated explicitly otherwise. Since x is solution of (P) exactly when it minimizes f + ic, x is solution of (P) if and only if 0 € d(f + i
Convex optimization
Proof.
with
137
constraints
In the conditions of our statement we have that
VxeCndom/
: d(f + ic){x) = df(x) + duc{x) = df(x) +
N{C;x),
whence the conclusion is obvious.
•
Very often the set C from problem (P) is introduced as the set of solutions of a system of equalities and/or inequalities. Let Y be ordered by a closed convex cone Q CY and H : X -» Y' be a Q-convex operator; the set C := {x € X \ H(x)
min f{x), H{x)
An element x € X for which H(x)
min f(x),
H(x)
y.
Let us consider the function $ : I
X
F 4 l ,
/(l) *(*,y):=«{ (x, y) := | v wy ' ' oo
i
f
^ ^ ^ otherwise.
(2.76) v '
The problem (Py) becomes now (Py)
min $(x,y),
i £ l
We obtain, without difficulty, that $ is convex. Moreover $*(z*,-y*) = sup{(x,a;*) + (y, -y*) - *(x,y)
\x€X,yeY}
= sup{(x,x*) - (y,y*> - / ( z ) | x € X,y £ Y,#(aO < Q y} = sup{(a;,a;*) -(H(x)+q,y*) = sup{(z,:r*) - (H(x),y*)
- f(x) \ x € dom//, g € Q} - f{x) \ x £
domH}
+ sup{-(g,y*) | 9 € Q}. Hence $*(x*,-y*)
= sup{(x,x*)
if y* e Q+ and $*{x*,-y*)
- f(x) - (H(x),y*)
= oo if y* g Q+.
\x£domH}
138
Convex analysis in locally convex spaces
The function L:XxQ+->R,
L(x,y*):=
{ ^x ^ oo
) + (H(x),y*)
ifxGdomtf, if x $. dom H,
is called the Lagrange function (or Lagrangian) associated to problem (Po)- The above definition of L shows that L(x,y*) — f(x) + (y* o H){x) with the convention that y*(oo) = oo for y* G Q+, convention which is used in the sequel. By what was shown above, we have that = 8up(-L(x,yt)) = -hdL(x,y*). x x xex ^ Therefore, the dual problem of problem (Po) (see Section 2.6) is V y * e Q + : &(0,-y*)
(£>o)
max ( - $*(0, *)),
y*£Y*,
or equivalently, (Do)
max (mfxeX L(x,y*)),
y* £ Q+•
We have the following result. Theorem 2.9.2 Let f G A(X), x G domf and H : X -> (Y',Q) be a Q-convex operator, where Q
3x0edomf
: -H{x0)
€ intQ.
Then the problem (D0) has optimal solutions andv(Po) = v(Do), i.e. there exists y* G Q+ such that inf{/(x) | H(x)
\ x G X}.
Furthermore, the following statements are equivalent: (i) x is a solution of (Po); (ii) H(x)
+ y*o H)(x)
and
(H(x),y*)
= 0;
(hi) there exists y* G Q+ such that (x, y*) is a saddle point for L, i.e. Vx E X, Vy* G Q+ : L(x,y*) < L{x,y*) < L{x,y*)-
(2.77)
Proof. The Slater condition (S) ensures that (XQ, 0) G dom $ and $(x0, •) is continuous at 0, where $ is the function denned by Eq. (2.76). Applying Theorem 2.7.1, there exists y* G Y* such that v(P0) = - $ * ( 0 , - y * ) . If v(Po) = —oo, then $*(0,2/*) = oo for every y* G Y*; in this case we may
Convex optimization with constraints
139
take y* = 0. If f(Po) > —oo, using the expression of $*, we have that y* e Q+. Therefore v(P0) = inf{/(x) | H(x) < Q 0} = - $ * ( 0 , - | T ) = i n f { i ( x , r ) I a; € X } = v(D0). (i) =S- (ii) Of course, x being a solution for (P 0 ), we have that H(x)
< f(x) < f(x) + (H(x),y*)-
(2.78)
Relation (2.78), without the term from its middle, says that x is a minimum point for f + y* o H, and so 0 € d(f + y* o H)(x); taking x = x in relation (2.78) we obtain that (H{x),y*) - 0. (ii) =>• (hi) Since 0 € d(f + y* o H)(x) we have Vx G X : L ( x , r ) = /(x) + (^(x),y*) < f{x) + (H(x),y*)
= LfoiT).
Furthermore, for y* £ Q+ we have that L(x,y*) = f{x) + (H(x),y*)
< f(x) = f{x) + (H{x),y*)
=
L(x,?).
Therefore Eq. (2.77) holds, i.e. (x,y*) is a saddle point of L. (iii) => (i) Taking successively y* = 0 and y* = 2y* on the left-hand side of Eq. (2.77) we obtain that (Hix),y*) = 0. Using again the left-hand side of Eq. (2.77), we obtain that (i?(x), y*) < 0 for every y* e Q+; thus, using the bipolar theorem (Theorem 1.1.9), we have that —i?(x) € Q++ = Q, i.e. x is an admissible solution of (P 0 ). From the right-hand side of relation (2.77) we get Vx e X, H{x)
< f(x) + (H{x),F)
Therefore x is solution of problem (Po)-
<
fix). D
The element y* € Q+ obtained in Theorem 2.9.2 is called a Lagrange multiplier of problem (P). Note that when H is finite-valued (i.e. d o m # = X) and continuous, V x e d o m / : dif +
y*oH)ix)=8fix)+diy*oH)ix);
Convex analysis in locally convex spaces
140
the fact that Q is closed was used only for the implication (iii) =4- (i) of the above theorem, to prove that x is an admissible solution. An important particular case is when there are a finite number of constraints. Let / , g i , . . . ,gn 6 A(X) and consider the problem (Pi)
min f(x),
gi{x) < 0,1 < i < n.
The dual problem of (Pi) is (Di)
max mix€X(f(x)
+ \igi(x)-]
hA n (/ n (i)), Ai > 0 , . . . , A„ > 0.
Then the following result holds. Theorem 2.9.3 dition holds, i.e.
Let f,gi,...,gn
€ A(X). Suppose that the Slater con-
3xo € d o m / , \/i£l,n
: gi(x0) < 0.
Then: (i) the dual problem (Di) has optimal solutions and u(Pi) = i.e. there exist (Lagrange multipliers) A i , . . . , A„ £ M+ such that
v(D{),
mf{f(x)\gi(x)<0,...,gn(x)<0} -ini
{f(x)+J1g1(x)-\
+ Xn9n(x) \ x 6 X) .
(ii) Let x 6 dom / ; x is a solution of problem (Pi) if and only if gi(x) < 0 for every i 6 l , n and there exist A i , . . . , An € K+ such that Xigi(x) = 0 for i £ l , n and 0 € d (/ + Aiffi + . . . +
\XJ-
*ngn)
If the functions g\,..., alent to
gn are continuous at x, the last condition is equiv-
0 € df(x) + A-!0ffi(2c) + . . . +
\ndgn(x).
Proof. Let us consider H : X -> (Mn*,E!J:), H(x) := (gi(x),.. .,gn(x)) for x G n r = i d o m 5 j , i?(x) = oo otherwise; H is R™-convex. The result stated in the theorem is an immediate consequence of the preceding theorem. When gi are continuous at x one has, by Theorem 2.8.7 (iii), that d(f + Aiffi + . . . + \n9n)(x) and d(\igi)(x)
= Jidg^x).
= df(x) + d(Xigi)(x)
+ ... +
d(\ngn)(x), D
Convex optimization with constraints
141
Corollary 2.9.4 Let f,gi,--.,gn '• X —> R be Gateaux differentiable continuous convex functions. Suppose that there exists x$ £ X such that 9i{xo) < 0 for every i 6 l,n. Then x £ X is solution of problem (Pi) if and only if gi(x) < 0 for every i £ l , n and there exist A i , . . . , An £ R+ such that —\7f(x)=\iVgi(x) Corollary 2.9.5 N([g < l\,x)
+ ... + \nVgn(x)
and \igi(x) = 0 for i £ l,n.
•
Let g 6 A(X), 7 £ ] inf g, 00 [ and S £ [5 < 7]. Tften = \J {d(Xg)(x) | A > 0, X(g(x) - 7) = 0}.
(2.79)
Proof. The inclusion "D" in Eq. (2.79) holds without any condition on g and 7. Indeed, let x* £ d(Xg)(x) for some A > 0 with X(g(x)— 7) = 0. Then for x £ [g < 7] we have that (a: — x, x*) < Xg(x) — Xgix) = X(g(x) — 7) < 0, and so x* £ N([g < ^],x). Let x* £ N([g < j];x). Then x is a solution of the problem (Pi)
min {x, —x*), h(x) := g(x) — 7 < 0,
whence, by Theorem 2.9.3, there exists A > 0 such that Xh(x) = X(g(x) — 7) = 0 and 0 G d(-x* + Xh){x), i.e. x* £ d{Xh){x) = d(Xg)(x). Therefore the inclusion " c " of Eq. (2.79) holds, too. • Note that d(Xg)(x) = Xdg(x) if A > 0 and d(0g)(x) = didomg(x) = N(domg;x). In the case of normed vector spaces, taking A = X in Proposition 3.10.16 from Section 3.10, we have another sufficient condition for the validity of formula (2.79). Note that we can obtain the characterization of optimal solutions in Theorem 2.9.3, when the functions gi are continuous, using Corollary 2.9.5 and the formula for the subdifferential of a sum. Indeed, x £ dom / is a solution of (Pi) if and only if x is minimum point of the function / + icx + htc„, where Cj := {x | gi(x) < 0}. By hypothesis d o m / f l f)" = 1 int d ^ 0, and so, for every x £ d o m / (~l H i l i ^ i w e have d(f + tCl+...
+ icn) (x) = df{x) + N(Cf,x)
+ ... +
N(Cn;x).
Using formula (2.79) we obtain the desired characterization. Theorem 2.9.2 can be extended further to the case when there are also linear constraints. Let us consider the problem (P 2 )
min f(x),
H{x)
Convex analysis in locally convex spaces
142
where T G £ ( X , Z). Consider the Lagrange function associated to problem
L2 : X x (Q+ x Z*) - • 1 ,
L 2 ( a M , V ) := /(a:) + (H(x),y*)
+
(Tx,z*).
The dual problem associated to (P 2 ) is (£>2)
max (mixeXL2(x,y*,z*)),
y* G Q+, z* G Z*.
Then the following result holds. Theorem 2.9.6 Let f G A{X), H : X -> (Y,Q) be a Q-convex operator, where Q C Y is a closed convex cone, T G L(X,Z) be a relatively open operator and x G d o m / . Suppose that there exists XQ € d o m / such that f,H are continuous at xo, —H{XQ) G int<5 and TXQ = 0. Then the problem (D2) has optimal solutions and V{PQ,) = v{D2), i.e. there exists y* G Q+, z* G Z* such that inf{/(x) I H(x)
\ x G X).
Furthermore, the following statements are equivalent: (i) x is solution of problem (P2); (ii) H(x)
and (H(x),y*)
= 0;
(iii) there exists (y*,z*) G Q+ x Z* such that (x,(y*,z*)) point of L2, i.e. L2(x,y*,z*)
< L2{x,y*X)
<
is a saddle
L2{x,y*,T)
for all x G X and all y* G Q+, z* G Z*. Proof.
Let us consider the perturbation function
*:Ix(7xZ)->I, v
;
'
f(x) $(1,y,z) := , y y ^ ' ' ' 00
if H(x)
We intend to prove that condition (i) of Theorem 2.7.1 is verified. Note first that P r y x Z ( d o m $ ) = {{H{x) + q,Tx) | x G d o m / , q £ Q} C Y x ImT.
A minimax
theorem
143
Taking into account that / is continuous at xo G d o m / , there exists Uo G N x ( i o ) such that Vxe%
: f{x)<M:=f(x0)
+ l.
Since —H{XQ) G intQ, there exists Vo G Ny such that —H(XQ) +VQCQ. There exists V € Ny such that V + V CV0. Since if is continuous at x0, there exists {/ € Nx(zo)) [/ C t/o, such that H(x) € H(x0) — V for every x € U. Since T is relatively open, there exists W G NimT such that W C T(LT). Of course, V x W G N y x i m r ( 0 , 0 ) ; let (j/,z) G V x W. There exists x £U such that Tx = z. Since x G f/, we have that i ? ( i ) = H(xo) — y' for some j / ' G V. Then y - H(x) =y + y'-
H(x0) G - # ( x 0 ) + V + V C - # ( x 0 ) + V0 C Q;
hence -ff(x) < Q y. Since x £ U C Uo, we have that $(x,2/,z) < M, i.e. condition (i) of Theorem 2.7.1 is verified. Therefore there exists (y*,z*) G Y* x Z* such that v(P2) = -4>*(0, -y*, -z*). But $*(0, -y*, -z") = s u p ( w ) 6 X x y x Z ( ( i / , -y*) + (z, -z*) - $ ( x , y , z)) = sup{-(y,y*)
- (z,z*) - f(x) | H ( i )
= s u p { - ( # (x) + q, y*) - (Tx, z*) - f(x) = - inf{f(x) + (H(x),y*)
\xeX,qeQ}
+ (Tx, z") \ x G X } + sup{-(q,j/*) | q G Q}.
Thus **cn _„* —?*\ - J - i n f i e x ^ C i . y * , ^ * ) * l U ' y ' z ) ~ \ oo
if if
2/* e Q + , j / * £ Q+.
Therefore y* G Q + if u(P 2 ) G K and we can take j7* = 0 if v(P2) = - o o . The proof of the second part is completely similar to that of Theorem 2.9.2. We note only that, / and H being continuous at xo, we have d(f + y* o H)(x) = df(x) + d(y* o H)(x) for every x G d o m / . •
2.10
A Minimax Theorem
In the preceding section we have obtained some results which are stated using saddle points. In the sequel we establish a quite general result on the
144
Convex analysis in locally convex spaces
existence of saddle points as well as a minimax theorem. Let A and B be two nonempty sets and / : A x B -> JR. It is obvious that sup inf f(x,y) xeAvtB
< inf sup f(x,y). y£BxeA
(2.80)
The results which ensure equality in the preceding inequality are called "minimax theorems," the common value being called saddle value. Note that if / has a saddle point, i.e. there exists (x,y) € A x B such that VxeA,Vy&B
: f(x,y)
< f(x,y)
< f(x,y),
(2.81)
then max inf f(x,y) x£A yeB
= minsup f{x,y), yeB
(2.82)
xeA
where max (min) means, as usual, an attained supremum (infimum). Indeed, let (x,y) € Ax B satisfy Eq. (2.81). It follows that ini: sup f(x,y) v£Bx€A
< sup f(x,y) xeA
< f(x,y)
< infB :(x,y) < sup infB y£ xeAv&
f{x,y).
Using Eq. (2.80) we obtain that all the terms are equal in the preceding relation, and so (2.82) holds (the maximum being attained at x and the minimum at y). Conversely, if Eq. (2.82) holds then / has saddle points. Indeed, let x G A and y e B be such that inf (x,y) = sup ini' f(x,y) y£B
xeAVZB
= irtf swp f(x,y) y€Bx€A
= sup
f(x,y).
x£A
Since inf y 6 s(x,y) < f(x,y) < supx€Af(x,y), from the above relation it follows that all the terms are equal in these inequalities; this shows that (x, y) is a saddle point of / . So we have obtained the following result. Theorem 2.10.1 Let A and B be two nonempty sets and f : A x B ->• R. Then f has saddle points if and only if condition (2.82) holds. • The following result is an enough general minimax theorem (frequently utilized) which gives also a sufficient condition for the existence of saddle points. Theorem 2.10.2 Let X be a locally convex space, Y be a linear space, Ac X be a nonempty convex compact set and B CY be a nonempty convex set. Let also f : A x B -)• M. be a function with the property that f(-,y)
A minimax
theorem
145
is concave and upper semicontinuous for every y G B and f(x, •) is convex for every x E A. Then max inf f(x,y)
— inf ma,xf(x,y).
X&A y£B
y£B
(2.83)
xEA
If moreover Y is a locally convex space, B is compact and f(x, •) is lower semicontinuous for every x € A, then maxmmf(x,y)
= min max f(x,y);
xeA y£B
yeB
(2.84)
x£A
in particular f has saddle points. Proof. Let a e K be such that a > max x£j 4 inf ye B f(x,y). For every x S A there exists yx 6 B such that f(x,yx) < a. Since f(-,yx) is upper semicontinuous at x, there exists an open neighborhood Vx of x such that f(u,yx) < a for every u G Vx. Since A is compact and A C UxeA ^ > there exist x\,..., xp € A such that A C U?=i Vxt • Let 2/j := yXi • Consider the sets d
:=co{{f(x,yi),...,f(x,yp))\xeA}cW,
C2 := {(ui,...,up)
\v,i>a\/i€
l,p} .
It is obvious that C\ and Ci are nonempty convex sets, and C2 has nonempty interior. Moreover C\ fl Ci = 0. In the contrary case there exist q 6 N, ( A i , . . . , A?) 6 Aq and Xi,..., xq € A such that V i e l . p : a < Zf := 2^,
>^jf{xj,yi).
Since f(-,yi) is concave, taking xo = 13?=i ^ j 2 ^ ' Vi G l , p : a < Zj <
we nave
that i o £ i and
f(x0,yi).
There exists io £ l , p such that £0 £ VXi . Therefore f{xo,yi0) < a, a contradiction. Applying Theorem 1.1.3 there exists (/zi,..., fj,p) € W \ {0} such that . ,mf(x,yi) t=l
< V
UiUi.
*—'i=l
Letting u; -> 00 we have that /ij > 0 for every i, and so we can suppose that (/xi,..., Up) € A p . Taking u = ( a , . . . , a) we obtain that ^xeA
: > .
Uif{x,yi)
< a.
Convex analysis in locally convex spaces
146
Let 2/0 := Y%=i ViVi € B. Since f(x, •) is convex, we obtain t h a t f(x, y0) < a for every x € A. Therefore a > i n f j / e B m a x x g ^ / ( a ; , 2 / ) , and so (2.83) holds. In Eq. (2.83) the suprema with respect t o x are attained because the functions f{-,y), y £ B, and i n f y e s f(-,y) are upper semicontinuous and A is compact. W h e n B is compact and the functions f(x,-) are lsc for all x € A we obtain t h a t the infima with respect to y € B are attained in Eq. (2.83), i.e. Eq. (2.84) holds. • Note t h a t the convexity assumptions in the preceding theorem can be weakened. More precisely, we can suppose t h a t A is a nonempty compact subset of a topological space, B is nonempty, and / satisfies t h e following two conditions (similar to concavity and convexity, respectively):
Vxi,...,xp
e A, V ( A i , . . . , A p ) e A p , 3x0
&A,\fyeB
:
f(xo,y) > y \
,Kf(zi,y),
• ' — ' 1 = 1
\/yi,...,yq
GB,
V ( / / ! , . . . , / * , ) € A , , 3y0 eB,Vx£A
: v—yQ
f(x,yo) 2.11
<
2^,.=iHjf{x,yj).
Exercises
Exercise 2.1 Let X be a linear space, / € A.(X) and x,u £ X. Prove that the mapping %p :]0, oo[—• R defined by ip(t) :— t • f(x + t~lu) is convex. E x e r c i s e 2.2 (a) Let I C R be an interval and / : / — > • R. Suppose that / is locally nondecreasing, i.e. for every a € I there exists e > 0 such that the restriction of / to IC\[a — s, a + e] is nondecreasing. Prove that / is nondecreasing. (b) Let g : [a, b] —• R (a, 6 € R, a < b) be a continuous function. 1) Assume that g'+{x) g R exists for every x € [a,b[; show that there exists x' £ [a,b[ such that g(b) — g(a) < g'+(x')(b — a). 2) Assume that g'-(x) € R exists for every x S ]a, 6]; show that there exists x" € ]a, b] such that g(b) — g(a) > gL (x")(b — a). (c) Let X be a locally convex space and A C X be an open convex set. If / : A —> R is locally convex, i.e. for every a G A there exists a neighborhood V of a such that f\vnA is convex, prove that / is convex. Exercise 2.3
Let / : R n -*• R be a quasi-convex function and i ^ f .
For
Exercises x eRn
147
consider the function fx : R n ->• R, fx(t) = f(x + tx). Prove that / is lsc at i t t V i e l "
: fx
is lsc at 0,
/ is use at J o V i e R "
: fx, is use at 0.
If R n is replaced by an infinite dimensional normed space then the above properties do not hold, generally. Exercise 2.4 Let X be a linear space, / : X —> R be a convex function and A > i n f i e x fix)- Prove that {x G X | f(x) < \y
= {x G X | f{x) < A}.
Moreover, if X is a topological vector space and / is continuous, then int{x € X | f(x) < A} = {x G X | / ( z ) < A}. Exercise 2.5 Let X be a topological vector space and / : X —> R a convex function. Prove that: ( a ) [ / < * ] = c l [ / < *] f° r every t G ] inf / , oo[ if and only if / is lsc; (b) [ / < * ] = i n t [ / < *] for every t G ] inf / , cx)[ if and only if / is continuous on d o m / ; ( c ) [/ = *] = b d [ / < t] for every t G ] inf / , oo[ if and only if / is continuous (on X ) . Exercise 2.6 Let / : R" —)• R be a proper convex function for which all the partial derivatives df/dxi exists at a G int(dom/). Prove that / is Gateaux differentiable at a (even Frechet differentiate because / is Lipschitz on a neighborhood of a). Exercise |1 + t\p + is strictly and (f2 is
2.7 Let p G [1, oo[ and
E x e r c i s e 2.8
Take /3 G [1, <x[ and consider the function
/:R2^I,
2
f(x,y):={ (^tan|)^V^T^ [ oo
for x, y > 0, otherwise,
where, by convention, arctan | := ^ for a; > 0. Prove that / is a lsc convex function, but not strictly convex. Exercise 2.9
Consider the function
/: c[o, i] -> R,
fix) := /o1 y r r w o F ^ -
Convex analysis in locally convex spaces
148
Prove that / is convex and Frechet differentiable of order 2 on C[0,1] with
V/(x)(«) = f1 -£L= dt, Jo V l + x
V2f(X)(u, V) = ^
2
,7/^3-2 dt 2
Jo ( l + 2 : ) v l + z 2
for all u,v £ C[0,1]. Moreover, V / is continuous on C[0,1]. E x e r c i s e 2.10
Consider the function /:L1(0,1)->R,
/ ( i ) : = / 0 V l + (i(t))!*,
where L 1 (0,1) is the Banach space of (classes of) Lebesgue integrable functions on the interval [0,1]. Prove that / is a Gateaux differentiable convex function with xu Vx.uSL^O.l) : V/(x)(u) = / :dt, Jo
%/rn
but / is nowhere Frechet differentiable. Exercise 2.11
Using properties of convex functions, prove that
V a , 6 e R + , V p , g e ] l , o o [ , £ + ^ = 1 : ab < ±a p + \b\ the equality being valid if and only if ap = bq, and V n € N, V x i , . . . , x „ 6 R + , V a i , . . . , a „ 6]0,1[, a i + . . . + a „ = 1 : a m + . . . + a n x „ > X™1 • • -xZn, the equality being valid if and only if x\ = • • • = xn. In particular, VneN, Vsi,...,x„ €R+
: £(xi + . . . + xn) >
yX\---xn.
E x e r c i s e 2.12 Let / : X —>• R be a proper function. Prove that / is convex if and only if / + x* is quasi-convex for every x* G X*. Exercise 2.13 Let <x\,... ,an > 0 (n > 1) be such that ai-\ \-an < 1. Prove that the function / : P " —> R, / ( x ) := x"1 • x%2 • • • x"n, is concave; moreover, if a\ + • • • + a„ < 1, then / is strictly concave. E x e r c i s e 2.14 Consider / : R n ->• R, f(xi,...,x„) Prove that / is convex. Exercise 2.15
:= In ( X £ = 1
expxk).
Consider p € R \ {0} and the function xP"->R,
f{t,x):=
t"
nr=i*i'
Prove that / is strictly convex for p € ] — o o , 0 [ U ] n + l , oo[ and convex if p = n + 1.
149
Exercises Let c G R n and A : = { i £ P " \(x\c)> g : A -> R,
0}. Prove that the function
p(x)
(x\c)"
niu^
is strictly convex for p > n + 1 (it is possible to prove that g is strictly convex for p > n). The function g has been used by Karmarkar for establishing his interior point algorithm. Finally prove that the function
h : P n ->• R,
h(x) := (T[n xA
is strictly convex. Exercise 2.16 Let tp : R+ —> R+ be a continuous convex function such that ^i(t) = 0 •«• t = 0. Show that
Jo
i>'- (V>- x «))
=
7o
V-H^-H*))
for every a > 0. Exercise 2.17 function
Let X, Y be normed spaces and T G L(X,Y). f:Y^R,
f(y):=M{\\x\\\Tx
Prove that the
= y},
is a sublinear functional. Moreover, if T is an open operator, then dom f = X and / is continuous. Exercise 2.18 Let / : Rk —> R be a strongly coercive proper lsc function. Prove that co(epi/) is a closed set. E x e r c i s e 2.19 Let X be a locally convex space, / G T(X), g : X —> R be a proper function and a, /3 > 0. Prove that co (co(/ + ag) + jig) = co ( / + ( a 4- /3)g) and ( ( / + ag)** + fig)" = (f + (a + fig)* . Exercise 2.20 Let X be a separated locally convex space and A,B,CcXbe nonempty sets. If C is bounded and A + C C B + C prove that A C coB. Exercise 2.21 Let (X, \\-\\) be a normed space, / G T(X) and L > 0. Prove the equivalence of the following statements: (i) d o m / = X a n d V x , t / G X (ii) 3 a GR, V i € X (iii)
VMGX
: \f(x) - f(y)\ < L \\x - y||;
: / ( x ) < L ||x|| + a;
: /oo(u)
Exercise 2.22 Let X be a locally convex space and / € T(X). foo(u) = sup{/(z + u) — f(x) I a; G d o m / } for every u 6 l .
Prove that
150
Convex analysis in locally convex spaces
E x e r c i s e 2.23 Let X, Y be separated locally convex spaces, / € r ( X ) , F € F(X x Y), x € dom / and A £ R, e £ R+ be such that [/ < A] and d£f(x) are nonempty. Prove that: [/ < A]oo = [foo < 0], ( a e / ( x ) ) 0 0 = N{domf;x), (/*)«, = Sdom/, foe = Sdom/* and {v* £ y* | (F*)oo(0,t;*) < 0} = ( P r y ( d o m F ) ) . In particular, if {v* £ Y* | (F*) o o (0,u*) < 0} is a linear subspace then {0} and Pry (dom F) are united. E x e r c i s e 2.24 Let X be a separated locally convex space and / £ I \ X ) . Consider K := {u £ X | /«,(w) < 0} and X 0 := K D (-.K"). Prove that: (i) if is a closed convex cone, Xo is a closed linear subspace and f(x + u) = f(x) for all a: 6 X and u 6 Xo. (ii)
lin(dom/*)=X6L.
(iii) The function / : X / X 0 -»• I , f(x) := f(x), is well defined, fe r ( X / X 0 ) , /OO(M) = /oo(«) for every u € X, f* = f*\x± and 5 £ / ( £ ) = 9 E /(a;)| X j. for all x € X and £ 6 R+, where x represents the class of x £ X. E x e r c i s e 2.25 Let X be a separated locally convex space and / £ T(X). Assume that there exists u £ X such that /<X>(M) < 0 < fco(-u). Prove that for every e > 0 there exists fe £ T(X) such that f(x) < fc(x) < f(x) + s for every x e X and argmin/ e = 0. E x e r c i s e 2.26 Let X, Y be separated locally convex spaces and F 6 A(X x Y). Assume that F satisfies one of the conditions (ii)-(viii) of Theorem 2.7.1. Prove that the marginal function g : X* —¥ R, g(x") := inf y * e y* F*(x*,y*) is convex, iu*-lsc, the infimum is attained for every x* £ X* and goo(u") = mmv*ey(F*)oo(u* ,v*). Moreover, {v* £ Y* \ (F*)oo(0, «*) < 0} is a linear subspace. E x e r c i s e 2.27 (Toland-Singer duality formula) Let X be a separated locally convex space, / : X —>• R be a proper function and g £ T(X). Then inf (/(*) - ff(i)) = .inf. (g\x')
- f*{x'))
•
E x e r c i s e 2.28 Let X be a separated locally convex space, / £ T(X) be such that 0 € d o m / a n d p £ P. Consider the following assertions: (i) the mapping P 3 t i - > t~pf(tx) (ii) f'(x, —x) +pf(x) (iii) pf(x) (iv)
is nondecreasing for every x £ d o m / ;
< 0 for every x £ d o m / ;
< f'(x,x)
for every x € d o m / ;
{x,x*) >pf(x)
for all (x,x*) € g r d / ;
(v) for every x £ int(dom / ) there exists x* 6 df(x) such that (x, s*) > pf(x). Prove that (i) <=> (ii) <=> (iii) =$- (iv). Moreover, if / is continuous at 0 then all five assertions above are equivalent.
Exercises
151
E x e r c i s e 2.29 Let / : X —> IR be a function such that c o / is proper. Assume that for some x € d o m c o / we have that x = ^2i=i ^»^« a n ( ^ co/(ic) = ^ * = 1 ~\if{x~i) with Xi G d o m / , Ai > 0 for i € ITfc and £)* = 1 ^> = 1- Prove that 8c5/(3f) = n t i df(xi). Exercise 2.30 Let X be a separated locally convex space and / € A(X). Assume that 0 G d o m / and / | x 0 is continuous at 0, where Xo •— aff(dom/). Prove that f = sup{(x,x*) |x* G<9/(0)W < { =
f'(0,x) f'(Q,x) = oo / ' ( 0 , x ) = oo
if if if
xGXo, xGXo\_Xo, xeX\X0,
the supremum being attained for x G Xo. Moreover, Xo is closed if and only if V x G X : / ' ( 0 , x ) = s u p { ( x , x * ) | x* 6 3/(0)} . E x e r c i s e 2.31 Let X be a separated locally convex space and / G A(X) be continuous on i n t ( d o m / ) , supposed to be nonempty. Prove that for all x,y € int(dom/) there exist z £]x,y[and z* G df{z) such that f(y)—f{x) = (y — x, z*). E x e r c i s e 2.32
Let X be a separated locally convex space and / G T(X).
(i) Consider the conditions: a) df is single valued on dom.3/, b) (df)~1(xl)!~) (df)~1(x2) = 0 for all distinct elements Xi,X2 G X*, c) /* is strictly convex on every segment [xi,X2] C Imdf, d) dom df = int(dom/). Prove that a) <=> b) •<=>• c); moreover, if int(dom/) ^ 0 and / is continuous on int(dom/) then a) => d). (ii) If / is continuous on int(dom/) and ( 9 / ) _ 1 is single-valued on show that / is strictly convex on int(dom/).
Imdf,
E x e r c i s e 2.33 Let X, Y be separated locally convex spaces and / G A(X x Y). Prove that if / is continuous at (xo,j/o) £ d o m / , then Prx* (df{x0, j/o)) = df(-,yo)(x0) Exercise 2.34 Consider
and
P r y ( d / ( x 0 , y 0 )) = d / ( x 0 , -){yo)-
Let X be a separated locally convex space and / o , / i 6 A(X).
v := inf{/ 0 (x) | / i ( x ) < 0},
v* := sup inf (/o(x) + A/i(x)), x>o xex
with the convention 0 • oo = oo. Suppose that » £ R . Prove that v* =v&[rfe>Q
: inf{/i(x) | / 0 (x) < y - e} > 0].
E x e r c i s e 2.35 Let {X, \\-\\) be a normed linear space and / : X —> R be a proper convex function. Prove that there exists x* G X* with x* < / if and only if there exists M > 0 such that f(x) + M \\x\\ > 0 for all x G X.
Convex analysis in locally convex spaces
152
Exercise 2.36 Let (X, ||-||) be a normed linear space and / i , / 2 : X -> R be proper convex functions. Prove that there exist x* 6 X* and a € R such that —/i < x" + a < J2 if and only if there exists M > 0 such that fi(xi) + f2(%2) + M\\xi - x21| > 0 for all on, x2 € X. E x e r c i s e 2.37 Let X be a linear space, (Y, ||-||) be a normed linear space, T : X -* Y be a linear operator, yo 6 Y and / : I -> R be a proper convex function. Prove that f(x) + \\Tx + yo\\2 > 0 for all x € X if and only if there exists j / * € y * such that fix) -2(Tx + y0,y*) - ||y*|| 2 > 0 for all x e X. E x e r c i s e 2.38 Let (X, ||||) be a normed vector space, C,D C X be closed convex cones and x G X, x* € X*. Prove that d(x,C)
: = i n f { | | x - x | | | x € C} = m a x { - ( x , x * ) | x* 6 Ux* n C + } ,
d ( x * , C + ) = m i n { | | x * - x * | | | a;* 6 C + } = s u p { - ( x , x * ) | x G C/x n C} , a,ndsupx€Uxncd(x,D)
= supx,eUx,nD+
d(x* ,C+).
E x e r c i s e 2.39 Let / € A(X) and x € d o m / be such that f(x) Consider the sets d
>
inf/.
:= {z* € X* | ( i - x,x*) < f(x) - /(a?) Vx e [/ < /(*)]},
d ^ / ( x ) := {x* a ' l f i - f , x*) < f{x) - / ( J ) Vx € [/ < /(*)]}. Prove that dKf{x) = d-f(x) differential of / at x.
= [1, oo[-d/(x). The set a < / ( x ) is Plastria's sub-
E x e r c i s e 2.40 Let X be a normed space, (an)n>i C X and (A n )„>i C [0,oo[ be such that ^ ^ L j A n = 1. Consider the function
/:X->R,
f{x) = Y°°
A„||x-a„|| 2 .
*•—'n=l
1) Prove that the following statements are equivalent: (a) S^Li-^i|l a "ll 2 < oo (i.e. 0 € d o m / ) , (b) d o m / ^ 0, (c) d o m / = X. 2) Prove that / is finite, convex and continuous when 5Z^Li^ill a "l| 2 < °°3) Suppose that ]C!!!Li^n|lan||2 < °°- Prove that for every x £ X one has 8f{x) = { 2 V 0 0
A n ||x - On|| xmn I x*n e a|| • IKi - a„) Vn € N ) .
E x e r c i s e 2.41 Let X be a normed space and / 6 T(X). following statements are equivalent:
Prove that the
(a) lim||J.||_too f{x) = 00; (b)
[/ < A] is a bounded set for every A > inf^ex / ( x ) ;
(c) there exists Ao > inf^gx / ( x ) such that [/ < Ao] is bounded;
Exercises
153
(d) there exists a, 0 £ R, a > 0, such that f(x) > a||x|| + /? for every x G X; (e) liminf| )x ||_ foo / ( x ) / | | x | | > 0; (f)
0£int(dom/*).
Moreover, if dim X < oo then the conditions above are equivalent to ( c ')
[/ < iaf /] is nonempty and bounded.
Furthermore, if p, q £ ]l, oo[ are such that 1/p + 1/q = 1, the following statements are equivalent: (g)
liminf|NHoo/(x)/||x||p>0;
(h) l i m s u p | | : c , | H o o r(x")/\\x*\\q
< oo;
(i) there exists a, /3 £ R, a > 0, such that f(x) > a||x|| p + P for every i 6 X ; (j) there exists a,/3 £ R, a > 0, such that /*(x*) < cx||a;*||9 + /? for every a;* e X * . E x e r c i s e 2.42 Let f,fn : R m ->• R (n £ N) be convex functions such that (fn{x)) -¥ f{x) for every x £ R m . Assume that / is coercive. Prove that there exist a,P £ R with a > 0, no £ N such that / n ( x ) > a \\x\\ + /3 for all x G R m and n > no. Exercise 2.43 Let (X, ||||) be a n.v.s. and C C X be a nonempty closed convex set. Consider the following assertions: (i) there exists XQ G X* such that (x,x*,) > \\x\\ for every x G C; (ii) there exist xo £ -X", X*, £ X* such that (x — xo, x*>) > ||x — xo|| for every x eC; (iii) int(dom sc) 7^ 0; (iv) a) there exists x*, G X* such that {u, x*>) > 0 for every u £ Coo \ {0} and b) for every sequence (x n ) C C with (||x n ||) —• oo and ( | | x „ | | _ 1 xn) -^ u one has Prove that (i) => (ii) =S> (iii) => (iv). If 0 £ C then (ii) =>• (i). Moreover, if X is a reflexive Banach space then (iv) =>• (ii). Exercise 2.44 Let X be a normed space and / £ A(X) be lower bounded. Consider A > i n f l € x / ( x ) =: inf / and p > 0. We envisage the conditions: (a)
[f<X}CpU;
(b) V x £ X (c) V x ' G
A
: f(x) > inf / + "2'°f/t/x»
A
~lnf 2p
f
• max{0, l|a?l| - p};
: / * ( * ' ) < / * ( 0 ) + p||x'||.
Prove that (a) =>• (b) & (c). Exercise 2.45 equivalences:
Let X be a normed space and / G T(X). Prove the following
Convex analysis in locally convex spaces
154
(a) liminf||.1.||_+00 / ( x ) / | | x | | > 0 - » 0 £ int(dom/*); in this case f(x) liminf 4f-if = sup{/i > 0 | /* is upper bounded on (J.U*}. ||x||-KX>
||X||
(b) liminf || B | H o o f(x)/\\x\\
= 0 « 0 6 Bd(dom/*).
(c) liminfHJII-KX, f(x)/\\x\\
< 0 O 0 ^ cl(dom/*); in this case
f(x)
liminf ~-r{- = - d d o m / . (0). llxH-KX)
E x e r c i s e 2.46
||Z||
Let p 6 ]1, oo[ and f:£p^R,
/ ( ( * „ ) „ > i ) := Y°°
n\xn\n.
Prove that / is finite, convex, continuous and lim||„||_+0O f*(y)/\\y\\
— 1.
E x e r c i s e 2.47 Let X, Y be Hilbert spaces, C C X be a nonempty closed convex set and A G L(X,Y) be a surjective operator. Consider the problem min i||Ac|| 2 ,
(P)
xeC.
A solution x of problem (P) is called a spline function in C associated to A. (a) Prove that (P) has optimal solutions if C + ker ^4 is closed. (b) Prove that x G C is an optimal solution of (P) if and only if y := A* (Ax) satisfies the relation (x\y) = min{(x \y) \ x € C}. E x e r c i s e 2.48 Let X,Y be Hilbert spaces, C := {x 6 X | Vi, 1 < i < k : (x\a,i) < fii}, where ai,...,ajb G -X', Pi,...,Pk G R and A G £ ( X , F ) be a surjective operator. Consider the problem (P)
min
5 | | A E | | 2 , x G C.
Prove that (P) has optimal solutions. Suppose, moreover, that there exists x € X such that (x \ a{) < /3i for every i, 1 < i < k. Prove that l i s a solution of (P) if and only if there exists (\i)i
V Xkdk a n d V i , 1 < i < A: : A;((x| a») — /?;) = 0.
Exercise 2.49 Let X be a Hilbert space and 01,02,03 be three non colinear elements of X (i.e. they are not situated on the same straight-line). Prove that there exists a unique element x G X such that V i £ l
: | | x - O i | | + | | x - a 2 | | + | | i - a 3 | | < ||x - ai|| + ||x - o 2 || + ||x - a 3 ||.
Moreover, prove that x G co{oi,a2,03}. The point x is called the Toricelli point of the triangle of vertices a\, 02,03.
Bibliographical notes
155
E x e r c i s e 2.50 Determine the optimal solutions (when they exist) and the value of the problem (Pf)
min £ (tx{t) + fiy/1 + (u(t)) 2 ) dt, x e Xi,
for every fj, € R + and every i g {1, 2, 3,4}, where Xi:=C[0,l], X 3 := | x e C [ 0 , l ] | / o 1 x ( t ) d t = 0 J , E x e r c i s e 2.51 (P)
X a : = 2/(0,1), X» := { x G 1/(0,1) | /^ x(t)dt = o } .
Consider the (convex) optimization problem
max J,,1 x(t) dt, xeX,
x(0) = x(l) = 0, J,,1 y/l +{x'(t))2dt
< L,
where L > 0 and X = C^O, 1] := {x : [0,1] —• R | x derivable, x continuous on [0,1]} or X = AC[0,1] := {x : [0,1] -¥ R | x absolutely continuous}. Determine the optimal solutions of problem (P), when they exist, and its value (using, eventually, the dual problem).
2.12
Bibliographical N o t e s
Many results of this chapter are well-known and can be found in several books treating convex analysis: [Rockafellar (1970); Hiriart-Urruty and Lemarechal (1993)] (for finite dimensional spaces), [Ekeland and Temam (1974); Ioffe and Tikhomirov (1974); Barbu and Precupanu (1986); Castaing and Valadier (1977); Phelps (1989); Aze (1997)]. In the sequel we point only those results that are not contained in these books but the last one. Sections 2.1-2.5: The assertion (vii) of Theorem 2.1.5 was established in [Zalinescu (1983b)]. The characterization of convex functions using upper Dini directional derivatives in Theorem 2.1.16 as well as Lemma 2.1.15 are established by Luc and Swaminathan (1993). Theorem 2.2.2 was established by Crouzeix (1980) in finite dimensional spaces and Theorem 2.2.11 by Zalinescu (1978). The notion of quasi-continuous function was introduced by Joly and Laurent (1971); Proposition 2.2.15 and Corollary 2.2.16 are established by Moussaoui and Voile (1997). The notions of cs-convex, cs-closed and cs-complete functions are introduced by Simons (1990), while that of lcs-closed function by Amara and CiligotTravain (1999); all the results concerning lcs-closed functions are due to Amara and Ciligot-Travain (1999). Theorem 2.2.22 (for I = N) and Proposition 2.2.24 can be found in [Kosmol and Muller-Wichards (2001)]; when d i m X < oo Theorem 2.2.22, Corollary 2.2.23 and Proposition 2.2.24 are stated in [Marti (1977)] under the weaker hypothesis that the pointwise boundedness or pointwise convergence holds on a dense subset of C. Proposition 2.2.17 is stated in [Zalinescu (1992b)]. Proposition 2.4.3 is established by Rockafellar (1966), Theorem 2.4.11 is proved in the book [Rockafellar (1970)] in R n and stated by Hiriart-Urruty
156
Convex analysis in locally convex spaces
(1982) in the general case; one can find another proof in [Aze (1997)]. The local boundedness of df in Theorem 2.4.13 and Corollary 2.4.10 can be found in many books on convex analysis. The assertions (iv) and (vii) of Theorem 2.4.14 are established in [Zalinescu (1980)]; for (i) see also [Anger and Lembcke (1974)]. Theorem 2.4.18 is from the book [Ioffe and Tikhomirov (1974)]. Theorem 2.5.2 was proved by Polyak (1966) under the more stringent conditions that / is a quasi-convex function which is bounded and Lipschitz on bounded sets and attains its infimum on every closed convex subset of X. Theorem 2.5.5 and Lemma 2.5.3 are established by Borwein and Kortezov (2001), while Lemma 2.5.4 and other results on non-attaining convex functionals are established by Adly et al. (2001a). Sections 2.6-2.9: The systematic use of perturbation functions for calculating conjugates and subdifferentials was done for the first time by Rockafellar (1974). The author of this book continued this approach in [Zalinescu (1983a); Zalinescu (1987); Zalinescu (1989); Zalinescu (1992a); Zalinescu (1992b); Zalinescu (1999)]; this permitted, for example, to give simpler proofs to several results stated in [Kutateladze (1977); Kutateladze (1979a); Kutateladze (1979b)]. Theorem 2.6.2(i) was established by Moussaoui and Seeger (1994), but Theorem 2.6.2(h) and Theorem 2.6.3 are new. Corollaries 2.6.4-2.6.7 can be found in [Hiriart-Urruty and Phelps (1993)] and [Moussaoui and Seeger (1994)]; for other results in this direction see the survey paper [Hiriart-Urruty et al. (1995)]. The sufficient conditions for the fundamental duality formula and for the validity of the formulas for conjugates and e-subdifferentials are, mainly, those from author's survey paper [Zalinescu (1999)], where one can find detailed historical notes; here we mention only the first use (to our knowledge) of them; actually, all these sufficient conditions are mentioned in that paper, excepting those which use li-convex or lcs-closed functions. So, conditions (iii) and (viii) of Theorem 2.7.1 and the corresponding ones in Theorems 2.8.1, 2.8.3, 2.8.7 and 2.8.10 are the classical ones and can be found in all the mentioned books which treat them. Condition (i) of Theorem 2.7.1 was used by Rockafellar (1974) when Yo = Y and by Zalinescu (1998) in the present form (see also [Combari et al. (1999)]), (ii) and (vi) were introduced in [Zalinescu (1983a)] for Yo = Y (and R replaced by a separated lcs ordered by a normal cone) and in [Zalinescu (1999)] in the present form, (iv) was introduced in [Zalinescu (1992b)] with (Hwi) replaced by (Hx), (v) is stated by Amara and Ciligot-Travain (1999) for X, Y Frechet spaces, $ a lcs-closed function and ,b replaced by lc, (vii) was introduced by Rockafellar (1974) for X, Y Banach spaces (X even reflexive) and Yo = Y, and by Zalinescu (1987) in the present form, while (ix) was used by Joly and Laurent (1971) for $ lsc and by Moussaoui and Voile (1997) for arbitrary 3>; note that condition (b) of [Cominetti (1994), Th. 2.2] implies condition (2.54). Theorem 2.7.4(iv) was established in [Zalinescu (1992b)]; the other conditions were introduced in [Zalinescu (1999)] (but in (v) it was assumed that F is lsc and C is closed). Condition (iii) of Theorem 2.8.1 was introduced in [Ekeland and Temam (1974)], (vii) was introduced in [Zalinescu (1984)] (for Y0 = Y), (iv) was intro-
Bibliographical notes
157
duced in [Zalinescu (1992b)], (i) and (viii) in [Zalinescu (1998)] for X,Y normed spaces, while conditions (ii) and (vi) were introduced in [Zalinescu (1999)]. Condition (vii) of Theorem 2.8.3 was introduced by Borwein (1983) for Yo = Y (see also [Zalinescu (1986)]), (x) was introduced by Borwein and Lewis (1992), (ix) was introduced by Moussaoui and Voile (1997), (i) by Zalinescu (1998) for X, Y normed spaces, (ii) was introduced by Combari et al. (1999), (iv) was introduced by Zalinescu (1999) (a slightly stronger form was used in [Simons (1990)]) as well as conditions (v) and (vi). For C G L(X, Y), generally, Theorem 2.8.6 is obtained under the corresponding conditions of Theorem 2.8.3 (taking / = 0). For C a densely defined closed linear operator, Rockafellar (1974) and Hiriart-Urruty Hiriart-Urruty (1982) obtained the results for g lsc, X, Y Banach spaces and Yo = Y (in [Rockafellar (1974)] X is reflexive and e = 0); Aze (1994) obtained the formula for the conjugate under condition (ii) in normed spaces. For general C condition (iv) was used by Zalinescu (1992b); the other conditions (excepting (v)) were used in [Zalinescu (1999)]. Condition (ix) of Theorem 2.8.7 was used by Joly and Laurent (1971), (vii) for X a Banach space was introduced by Attouch and Brezis (1986), (vi) was introduced by Zalinescu (1992b), (i) was introduced by Aze (1994) in an equivalent form in normed spaces, while (ii) was introduced by Combari et al. (1999). Corollary 2.8.8 was obtained by Voile (1994) for X a Banach space, / , g lsc and '* replaced by !C. Formula (2.65) from Proposition 2.8.9 was obtained by Aze (1994) and Simons (1998b) for x = 0 € ( d o m / — domg) 1 when / , g are lsc. The case / = 0 of Theorem 2.8.10 was considered by several authors; condition (iii) was used in [Hiriart-Urruty (1982); Lemaire (1985); Zalinescu (1983a); Zalinescu (1984)]; the algebraic case was considered in [Kutateladze (1979a); Kutateladze (1979b); Zalinescu (1983a)]; Zalinescu (1984) considered a stronger version of (v) (for lsc functions and Yo = Y); note that in these papers g is assumed to be Q-increasing on the entire space. The general case was considered in [Combari et al. (1994)] under (iii) and a condition stronger than (v) (/, g, H lsc and lc instead of lb) and in [Combari et al. (1999)] under condition (i) and a variant of (iv); it was also considered by Moussaoui and Voile (1997) under condition (vii). The formula for d(p(x) in Corollary 2.8.11 can be found in [Combari et al. (1994)]. The formula for d
158
Convex analysis in locally convex spaces
Section 2.10: The minimax theorem is also classical and can be found in the books [Barbu and Precupanu (1986); Simons (1998a)]. Exercises: Exercise 2.2 is from [Penot and Bougeard (1988)], Exercise 2.3 is from [Crouzeix (1981)], Exercise 2.6 is from [Marti (1977)], Exercise 2.15 is from [Crouzeix et al. (1992)] (there in a more complete form), Exercises 2.18 and 2.29 are from [Hiriart-Urruty and Lemarechal (1993)], Exercise 2.19 is from [Lions and Rochet (1986)], Exercise 2.20 is the celebrated cancellation lemma from [Hormander (1955)], Exercise 2.21 is from [Hiriart-Urruty (1998)], Exercise 2.22 is from [Jourani (2000)], the assertions in Exercise 2.23 are well-known (see also [Aze (1997)]), Exercise 2.25 can be found in [Borwein and Kortezov (2001)] (in normed spaces), the formula for goo under condition (vii) of Theorem 2.8.1 in Exercise 2.26 is obtained in [Amara and Ciligot-Travain (1999)] and [Amara (1998)], Exercise 2.27 is the well-known Toland-Singer duality formula (see [Toland (1978)] and [Singer (1979)]), Exercise 2.30 is from [Zalinescu (1999)], the equivalence of a) and c) (for Banach spaces) in Exercise 2.32 can be found in [Bauschke et al. (2001)] (see also [Barbu and Da Prato (1985)]), Exercises 2.33 and 2.34 are from [Eremin and Astafiev (1976)], Exercises 2.35, 2.36 and 2.37 are from [Simons (1998a)], the last formula in Exercise 2.38 is from [Walkup and Wets (1967)], Exercise 2.39 is from [Penot (1998a)], the formula for the subdifferential of the function considered in Exercise 2.40 and its proof are from [Aussel et al. (1995)], the equivalences (e) •£> (f) and (g) & (h) of Exercise 2.41 are from [Zalinescu (1983b)], the equivalence of conditions (ii)-(iv) in Exercise 2.43 are established in [Adly et al. (2001b)] in reflexive Banach spaces, a weaker variant of the implication (a) => (c) of Exercise 2.44 is stated in [Aze and Rahmouni (1996)], Exercise 2.45(c) is from [Borwein and Vanderwerff (1995)], Exercises 2.47, 2.48 axe from [Laurent (1972)].
Chapter 3
Some Results and Applications of Convex Analysis in Normed Spaces
3.1
Further Fundamental Results in Convex Analysis
Throughout this chapter X, Y are normed spaces and X*,Y* are their duals endowed with the dual norms. As an application of Ekeland's variational principle and of some results relative to convex functions, we state the following multi-propose generalization of the Br0ndsted-Rockafellar theorem. Theorem 3.1.1 (Borwein) Let X be a Banach space and f £ r ( X ) . Consider e £ P, XQ £ d o m / , XQ £ def{xa) and /3 € K+. Then there exist xe £ X, y* € Ux- and Ae £ [—1, +1] such that \\xe - Xo\\ + P • \(X£ - X0,X*0)\ < y/E, x% := x*0 + VE(y* + /3\ex*) £ df{xe).
(3.1) (3.2)
Moreover ||* e - soil < Ve,
\te-xl\\
\(xe-x0,x*)\<e x*e£d2ef(x0),
+ P\\xl\\),
+ JilP,
\f(xe)-f(x0)\<e
(3.3) (3.4)
+ ^/fi,
(3.5)
with the convention 1/0 = oo. Proof.
The function
|| • Ho •. x - • K, \\x\\0~\\x\\
+
p-\(x,x*)\,
is, obviously, a norm on X, equivalent to the initial norm. Therefore {X, ||-||0) is a Banach space. Consider the function g := / — XQ. It is obvious 159
160
Some results and applications of convex analysis in normed spaces
that Xo £ dom = d o m / = dom<9£/ and g is lsc and lower bounded: WxGX
: g{x)>g(x0)-e
[& x*0 £
dsf{x0)].
We apply Ekeland's theorem (Theorem 1.4.1) to g, y/e and the metric d given by d(x,y) := \\x — y\\o. So there exists xe £ domg such that g{xE) + y/e- \\x£ -x0\\o \/x£X,
x^xe
< g(x0),
(3.6)
: g(x£) < g(x) + y/e • \\x - xe\\0.
(3.7)
From Eq. (3.6) we obtain that g(xe) + Ve(\\xs - ar0|| +0-\{xe-
x0, XQ)\) < g(x0) < g(xe) + e,
whence Eq. (3.1) follows immediately. Let us consider the function h : X -> E, h(x) := g(x)+y/e-\\x-xE\\0
-
f{x)-{x,xl)+y/E-\\x-xE\\+Py/e-\(x-xe,xl)\;
by Eq. (3.7) xe is a minimum point of h. Therefore 0 £ dh(xe). Since h is the sum of four convex functions, three of them being continuous, and taking into account the expression of the subdifferential of a norm (Corollary 2.4.16) and of the absolute value of a linear functional (at the end of Section 2.8), we have that 0 £ dh(xe) = df(x£)
- x* + y/i • Ux- + PVe • [ - 1 , +1] • x%.
Therefore there exists x* £ df(xs), y* £ Ux* and Xe £ [—1,1] such that Eq. (3.2) is verified. The estimations from Eq. (3.3) follow immediately from Eqs. (3.1) and (3.2). Using again Eqs. (3.1) and (3.2) we obtain that |(a;e -x0,x*
-XQ)\ = y/e-\{xE
- xQ,y* + p\Ex%}\
< v/i(||z e - x0\\ • W\
+ 0\Xe\ -\(xe-
< y/e(\\xs - x0\\ + fi • \{x£
x0,x*0)\)
-x0,x*0)\)
= e.
(3.8)
From Eqs. (3.1) and (3.8) we get |(a;e -x0,x*)\
< \{xe -x0,x*
-x^)\
+ \(x£
-X0,XQ)\
^IP,
Further fundamental
results in convex
analysis
i.e. the estimation in Eq. (3.4) holds, too. Since x$ G dEf(xo) df(xE), we get (X0 ~Xe,X*£
-XQ)
+ (X0 -X£,X*Q)
= (X0 ~Xe,X*) < (X0
< f(x0)
-XE,XQ)
161
and x* G
~f(Xe)
+£.
Using relations (3.1) and (3.8), we obtain that \f(x0) - f(xe)\
< \(x0-xe,x*0)\+s<e
+ VE//3,
i.e. the second relation in (3.5) holds. Using Eq. (3.8) and the fact that XQ G dEf(x0), x* G df(xE), we obtain that (x-X0,X*e)
= (X-X£,X*)
+ (Xe -X0,X*
-XQ)
+ (XE
-X0,XQ)
< f{x) - f(xe) +s + f(x£) - f{x0) + e - f(x) - f(x0) + 2e for every x G X, i.e. x£ G d2ef(x0); holds, too.
hence the first relation in Eq. (3.5) •
The next result is well-known. The first part is an immediate consequence of Borwein's theorem, while the density part, which follows easily from the first one, will be reinforced in Theorem 3.1.4 below. Theorem 3.1.2 (Br0ndsted-Rockafellar) Let X be a Banach space and f € r ( ^ ) - Consider e > 0 and (XO,XQ) G grd6f. Then there exists (x£,x*) G gr<9/ such that \\x£— XQ\\ < ^Jl and \\X*—XQ\\ < y/e. Inparticular domf C cl(dom<9/) and dom/* C cl(Im<9/). D Another consequence of Borwein's theorem is the following result which will be completed in Proposition 3.1.10 below. Proposition 3.1.3
Let X be a Banach space and f G T(X). Then
f(x) = s u p ^ z - z,z*) + f(z) I (z,z*) G grdf} = sup{(x, z*) - /•(**) | z" G lm(df)}
(3.9)
for every x G d o m / . Proof. The second equality in Eq. (3.9) is obvious because f(z)+f*(z*) = {z, z*) for any (z, z*) G grdf. Let x G dom / be fixed. When (z, z*) G grdf we have (a; — z, z*) + f(z) < f(x) for every i £ l ; hence f(x) > sup{(a; z,z*) + f(z) | [z,z*) egrdf}. Because / is lsc, by Theorem 2.4.4 (hi), there
162
Some results and applications of convex analysis in normed spaces
exists x* € de/2f(x). Applying Borwein's theorem we get {xe,x*) £ grdf such that x* 6 def(x). Therefore (xe — x,x*) < f{xe) - f(x) + e, whence f(x) — e < {x — xe,x*) + f(xe). Hence relation (3.9) holds. • As announced before, the density results in Theorem 3.1.2 can be stated in a stronger form. In the next result (xn) —>/ x means that (a;n) —> x and {f(xn)) -* f(x), and similarly for (xn) ->•/• x*. Theorem 3.1.4
Let X be a Banach space and f € r ( X ) . Then
(i) Vx e d o m / , 3((xn,xn))
Cgrdf
: (a;„) -+/ x;
(ii) Vx* E dom/*, 3 ( ( x „ , < ) ) C gvdf
: « ) - • , . x*.
Proof, (i) Consider x € d o m / . Because / is lsc, by Theorem 2.4.4 (hi), for every n S N there exists x* € dn-2f(x). Applying Borwein's 2 theorem for (x,x*), e = n~ and /? = 1, we get (xn,xn) € grdf such that \\xn — x\\ < n _ 1 , \f(xn) — f(x)\ < n~2 + n~x. Hence (xn) -^/ x. (ii) Because / * is lsc, it is sufficient to show that for every e > 0 there exists (y, y*) e df such that \\y* - x*\\ < e a n d / * ( y * ) < f*(x*)+e. Fixx £ d o m / . Let £> 0 and take r := \\x\\ + 2e-1 (e + f(x) + f*(x*) - (x,x*)) > 0. Consider also the function g := fO(x* + irux) € A(X). Then g(x) > (fnx*)(x) = (x,x*) - f*(x*) for every x € X. It follows that g € T(X) and f(x) + (—x,x*) > g~(0) > —f*(x*). By Proposition 3.1.3 there exists (z,z*) € dg such that -g*(z*) = g(z) - (z,z*) > -f*(x*) - e/2. But + (x* + trUx)*(z*) = f*(z*) + r l|ar* - z*\\. Therefore g*(z*) = f*(z*) f*(z*) +
r\\x*-z*\\
Because f*(z*) > (x,z*) - f{x) > - \\x\\ • \\z* - x*\\ + (x,x*) - f(x), from the preceding inequality we obtain that (r — \\x\\) \\z* — x*\\ < e + f(x) + f*(x*) — (x,x*), and so \\z* - x*\\ < e/2. The inequality above shows also that f*(z*) < f*(x*)+e/2. On the other hand, because (z,g~(z)) € cl(epi^), there exists (zn) C dom / and (un) C rUx such that (zn + un) —> z and limsup(/(z n ) + (u„,a;*»
(3.10)
we have that
0 < £„ = f{Zn) + (U„, X*) - (Zn + Un, Z*) + (Un, Z* - X*) +
f*(z*).
Taking into account that (un,z* — x*) < r \\x* — z*\\, from Eq. (3.10) we obtain that limsup£ n < 0. Hence lime„ = 0. Because (un) is bounded, so
Further fundamental results in convex analysis
163
is (zn). Therefore there exists r' > 0 such that (zn) c r'Ux- Take n E N so that S := en < 1 and VS(r' + 2)(1 + \\z*\\) < e/2; set z := zn. Of course, we have that z* G dsf(z). By Borwein's theorem (for /? = 1) there exists (y,y*) G df such that \\z - y\\ < y/S, \\z* - y*\\ < VS{1 + ||z*||) and |/(«) ~ f(v)\ <S + VS. Hence ||j/* - x*\\ < y/S{l + \\z*\\) + e/2 < e and /*(»*) =
+ f*(z*) + f{z) -
f(y)
< VS(V5 + r') (1 + ||z*||) + \f5 \\z*\\ -5 + f(x*) < f*(x*) + e/2 + VS(r' + 2)(1 + ||«*||) < f*(x*)
+s/2 + 6 + V6 + e.
The proof is complete.
•
Using Br0ndsted-Rockafellar's theorem we can add other sufficient conditions for the validity of the conclusions of Theorems 2.8.1 and 2.8.7. Proposition 3.1.5 Let X,Y be Banach spaces, F £ T(X x Y), A G L(X,Y), D = {Ax - y \ (x,y) € d o m F } and E = {Ax - y \ (x,y) € dom&F}. Then ic
E = iiE = ic(coE) = ri(coE) = icD = riD.
Moreover, if one of the above sets is nonempty then icE = E — D; in particular the sets ICE and E are convex. Proof. Consider the linear operator T : X xY ->• Y defined by T(x, y) := Ax — y. Since domdF C dom.F and d o m F is convex, we have that E = T(domdF)
CcoE
= T(co{domdF))
By Theorem 3.1.2 we have that d o m F C domdF. nuity of T, we have that
CD = T ( d o m F ) . Hence, using the conti-
E C D C T ( d o m d F ) C T(domdF) = E. Using the properties of the affine hull (see p. 2), it follows that aff E C aff D c aff E. Therefore, if aff E is closed then aff D — aff E; the above relation shows that %CE c %CD. Let us show the converse inclusion. Let 2/o G %CD and consider the function F 0 6 T(X x Y), F0(x,y) :— F(x,y — yo)Of course, domF 0 = d o m F + (0,j/o) and dom9F 0 = d o m 9 F + {0,y0). Therefore 0 € i c ( P r y ( d o m F 0 ) ) . Taking
164
Some results and applications of convex analysis in normed spaces
such that dF0(x,Ax) = dF(x,Ax — y0) ^ 0. Therefore yo € E. So we obtained that *D = ICD C E C D, which shows that aff E = aff-D and *D C iE. It follows that icE = icD = ic(coE). As observed after the proof of Theorem 2.8.1, in our situation, if %CD is nonempty we have that ic D = rintD. Suppose that %CD 9^ 0 (for example). From what was proved above, we have that rinti? = lD = icD = rintE = lE = icE C E C D C E. Hence E~ = T°E = D:. The conclusion follows.
•
Remark 3.1.1 Taking into account the preceding proposition, for X, Y Banach spaces and F G T(X x 7 ) , we may add to the sufficient conditions in Theorem 2.8.1 the following conditions: 0 G ri{Ax - y \ (x,y) £ ic
0 £ {Ax -y\(x,y)€ ic
0 € {Ax -y\(x,y)e
domdF}, domdF}, co(domdF)},
y 0 = cone{ Ar - y | (x, y) 6 dom OF} is a closed linear space. Indeed, the first three conditions are, evidently (using the preceding proposition), equivalent to 0 £ lc{Ax — y \ (x,y) £ d o m F } . In the fourth case y 0 = aff {Ax — y\(x,y)e co(dom OF)}, and so 0 e lc{Ax — y\(x,y)e domF}. An important particular case of the preceding proposition is when X = Y, A = Idx and F(x,y) = f(x) + g(y) with f,g 6 T(X); in this case D — d o m / — domg and E = domdf — domdg. Using the Borwein theorem one obtains a formula for the subdifferential of a composition g o A of a lsc convex function and a continuous linear operator without qualification conditions. Theorem 3.1.6 Let X,Y be Banach spaces, A e C{X,Y), g £ T(Y), f = g o A and x 6 d o m / . Then x* 6 df(x) if and only if there exists a net ((yi,yt))ieI C grds such that (yt) ->• y := Ax, (g(yi)) ->• g{y), ((yi-y,y*))^Oand(A*y*)^x*. If X is reflexive one can take sequences instead of nets and impose norm convergence instead of w* -convergence.
Further fundamental
Proof.
Let (fa,y*))i€l
results in convex
analysis
165
C gidg be such that fa) -> y, (fa - y,y*)) ->•
0, {gfa)) -> g(y) and (i4'i/r) ^ x*. Then (y - yhy*) i G I and t / S F . It follows that (x' - x,A*y*) + (y- yhy*)
= (Ax' - yuy*)
< gfa-gfa)
for all
< f(x') - gfa)
for alH G / and x' G X. Taking the limit we obtain that (x' — x,x*) < /(x') - f(x) for all x' G X, and so x* G d/(x). Let now x* G df(x) and consider (£„)„6N 4- 0- Using Corollary 2.6.5 we have that x* G w*-clA*(d£ng(y)) for every n G N. Let A/" be a base of w*neighborhoods of x* and consider / = Nx M with (n, V) y (n1, V) iff n > n' and V CV. Then for all i = (n, V) G / there exists z* G dEng(y) such that A*£* G V. Taking /? = 1 and e = e^ in Theorem 3.1.1, there exists (yi,y*) G 3g such that \\yi - T/|| < e„, ||y* - ^ | | < en, \g(yt) - g(y)\ < en(en + l) and \(Vi ~ V,V*)\ < en(en + 1). Because \\A*z* - A*y*\\ < \\A*\\ • \\y* - z*\\ < en \\A*\\ and (A*z*)ieI ^> x*, we obtain that (A*y*)i€l ^> x*. If X is reflexive then w*-cli*(9 fB g(j/)) = cl A* (dSng(y)), and so, for every n G N, we can take z*n G dEng(y) such that ||A*z* — x*|| < e„. The conclusion follows. • Using the preceding result one obtains a similar formula for the subdifferential of the sum of two lsc proper convex functions. Theorem 3.1.7 Let X be a Banach space, / i , / 2 G T(X) and x G dom / i fl dom / 2 . Then x* e S f / i + M (x) if and only if there exist two nets {(xk,i>xt,i))ieI C gr<9/fc, k = l,2, such that ( x M ) i e / ->• x, (fk(xk,i))ieI -> /fc(x), ( ( x M - x , x ^ i ) ) . e / -^0 for k = 1,2 and (x*^ + x*2x*. 7/X is reflexive one can take sequences instead of nets and impose norm convergence instead of w* -convergence. Proof. We apply the preceding result iorY ~ XxX, A: X -> Y defined by Ax := (x,x) and# : Y -> E defined by #(xi,x 2 ) := /i(xi)+/2(x2). Then / := /1+/2 = g°A. The sufficiency is immediate by taking y = (x, x) = Ax, Vi = (xi,i,X2,i) and y* ~ (xl^xlj) for i G i". Let x* G df(x). By Theorem 3.1.6 there exists a net ((yi,y*))ieI C gvdg verifying the conditions of the theorem with y := (x,x). Taking yt = (xi ) j,x 2 ,i) and y* = ( x ' ^ x ^ j ) and taking into account that dg(x\,X2) — dfi(x{) x 0f2(x2), we have that ((xk,i,x*ki))ieI C gvdfk and (xk,i)iei -»• x for k = 1,2. Assume that (fi(xi,i)) A fi(x)- Because /1 is lsc at x, limsup i 6 / /i(xi,i) > /i(x). Thus
166
Some results and applications of convex analysis in normed spaces
there exists S > 0 such that J := {i £ I \ fi(xiti) — fi(x) > 5} is cofinal. It follows that g(yi) - g(y) > 6 + (f2{x2,i) — f2{x)) for all i £ J. Taking the limit inferior, we obtain the contradiction 0 > S. Therefore {fk(%k,i))ieI -^ fk(x) for k — 1,2. The inequalities fi(xi,i)
- fi(x) < (xij -x,x{A)
for i £ I imply that ({xij — x,x{ti})
< (yi -y,y*)
- {h(x2,i)
-
f2(x))
-> 0.
D
Using Br0ndsted-Rockafellar theorem we obtain another famous result. Theorem 3.1.8 (Bishop-Phelps) Let X be a Banach space and C C X, C 7^ X, be a nonempty closed convex set. Then (i)
The set of support points of C is dense in B d C .
(ii) The set of support functionals of C is dense in the set of continuous linear functionals which are bounded above on C. Moreover, ifC is bounded, then the set of support functionals of C is dense in X*. Proof, (i) Let x0 6 Bd C and e £ ]0,1[. Taking into account that C ^ X, there exists xi £ X \C such that \\xi — x 0 || < £2- Applying a separation theorem, there exists XQ £ Sx> such that SVLPX£C{X,XQ) < (XI,XQ). SO, for every x £ C, {x - X0,XQ)
= (x - XI,XQ)
whence XQ £ de2tc(xo)have that
+ (xi - X0,XQ)
< (x - xi,xl)
+ \\x\ - x0\\
<
e2,
Applying the Br0ndsted-Rockafellar theorem we
3xe £ C, 3a;* £ dbc{xe)
: \\x£ - x0\\ < e, \\x* - XQ\\ < e < 1.
Since \\XQ\\ = 1, we have that x*e ^ 0, and so xE is a support point of C with \\x£ — xo\\ < e. The conclusion holds. (ii) Using the second part of Theorem 3.1.2 we have that dom(tc)* C cKJmdic) = cl(Imdt c \ {0}), which shows that the conclusion of the theorem is true.
•
The following theorem will be used for a simple proof of the Rockafellar theorem.
Further fundamental results in convex analysis
167
Theorem 3.1.9 (Simons) Let X be a Banach space, f € T(X) and x € X, n G R be such that inf / < 77 < f(x). Consider L :=
sup -fizz rr, x€X\{x} ||ar — ar||
and dx • X -V R, dj(x) := | | i — x||. Then: (i) 0 < -L < oo and inf (/ + Ldx) > n; (ii) V e e ] 0 , l [ , 3 i / 6 X
: ( / + L«fc)(i/) < i n f ( / + i d * ) + e L | | i - y\\;
(iii) V e € ] 0 , l [ , 3 («,*•) G g r d / and\\z*\\ < (l + s)L;
: {x - z,z*) > (I - e)L\\x - z\\ > 0
(iv) Ve €]0,1[, 3(z,z*) € g r S / : (z - z,z*> + /(*) > 17 and L < \\z*\\<(l + e)L. Proof, (i) Because 77 > inf / , it is clear that L > 0. Since 77 < /(a;) and / is lsc at x, there exists p > 0 such that / ( x ) > 77 for every a; € B(x,p). Furthermore, since / € r ( X ) , there exist x* € X* and a 6 R such that f(x) > (a;!a;*) ~ Q f° r every a; £ X. Therefore 77 — f(x)
V* £ X, \\x - x\\ > p : 5_jlZg) < ||x*|| + _ ] _ < li^H + 2. IF ~ x\\
\\x ~ x\\
P
This relation and the choice of p show that L < 00. From the expression of L we obtain immediately that inf (/ + Ldx) > 77. (ii) Let e e]0,1[. Since (1 - e)L < L, from the definition of L there exists y £ X, y ^ x, such that (77 — f{y))/\\x — y\\ > (1 — e)L, and so (/ + Ldx)(y) < 77 + eLdi(y) < inf(/ + Ldx) + eL\\y - x\\. (iii) Let e €]0,1[ be fixed. The function / + Ldx 1S proper, lsc and bounded from below, while the element y from (ii) is in dom(/ + Ldx) = d o m / . Taking the metric d o n X defined by d(xi,X2) := L\\xi—X2\\, (X,d) is a complete metric space. Using Ekeland's variational principle we get the existence of z € X such that (/ + Ldx)(z) + EL\\Z - 7,11 < (/ + Ldx)(y)
168
Some results and applications of convex analysis in normed spaces
and (f + Ldw)(z) < {f + Ld¥)(x) + eL\\x - z\\
VxeX.
The first relation and (ii) give ||^ — yj| < \\x — y\\, and so z ^ I . The second relation says that z is a minimum point of the function / + Ld^ + sLdz. Taking into account that cfe- and dz are continuous convex functions, it follows that 0 G d(f + Ldw + eLdz)(z) = df(z) + Ldd^{z) +
eLddz(z).
But ddz{z) = d\\ • ||(0) = Ux- and dd^(z) = {x* € Ux* | (z - x,x*) = \\z — x\\}. Hence there exist z* G df(z) and x*, y* £ Ux* such that z* = Lx* + sLy* and (x - z, x*) — \\x - z\\. Therefore ||z*|| < (1 + e)L and (x — z, z*) = {x — z, Lx* + eLy*) = L\\x — z\\ + eL(x — z, y*) > L\\x - z\\ - eL\\x - z\\ = L(l - e)\\x - z\\ > 0. (iv) Let e £]0,1[ be fixed and consider e' := e/3. Let us take M := (1 + 2e')L. Since / + Md% > f + Ld% > r], we can apply Ekeland's theorem for / + Md-x, an element xo of dom / , e' > 0 and the metric defined at (iii). We get so the existence of z e X such that V i e l :
(f + Mtk)(z)<(f
+
Mds)(x)+e'L\\x-z\\.
As in the proof of (iii), there exist z* € df(z), x*, y* € Ux* s u c h that z* = Mx* +e'Ly* and (x-z,x*) = \\x - z\\. Thus ||z*|| < (1 + s)L and (x - z, z*) > (M - e'L)\\x - z\\ = (1 + e'L)\\x - z\\. So, for x = z we have that (x — z, z*) + f(z) = f(x) > rj, while for ~x yi z we have (x-z,z*)
+ f(z) > (l + e')L\\x-z\\
+ f(z) = (f + Ldw)(z)+e'L\\x-z\\
> v.
Therefore (x — z, z*) + f(z) > T). Moreover ||i - z|| • \\z*\\ >(x-x,z*) >V~
= [(x - z, z*) + f(z)] - [(x - z, z*) + f(z)]
f(x)
for every x € X; the last inequality holds because z* G df(z). ||af — x\\ for x 7^ x, we obtain that ||z*|| > L.
Dividing by D
Convexity and monotonicity
of
subdifferentials
169
Using the preceding theorem we reinforce slightly Proposition 3.1.3. Proposition 3.1.10 Let X be a Banach space and f 6 T(X). relation (3.9) holds for every x 6 X.
Then
Proof. Taking into account Proposition 3.1.3 we have to show that oo = sup{(a; — z,z*) + f(z) \ (z,z*) € gr<9/} when x £ d o m / . So, let x £ X \ d o m / and take inf / < 77 < 00. Using Theorem 3.1.9 (iv), there exists {z,z*) E gr<9/ such that (a; - z,z*) + f(z) > rj. The conclusion follows. • The maximal monotone operators are of a great importance in the theory of evolution equations. A significant example of such operators is the subdifferential of a proper lsc convex function on a Banach space. Theorem 3.1.11 (Rockafellar) Let X be a Banach space and f 6 T(X). Then df is a maximal monotone operator. Proof. Let (x,x*) EXxX*\gvdf. Then x* £ df{x), and so 0 i df(x), where / := / — x*. It follows that inf/ < fix). Applying assertion (iii) of Simons' theorem, there exists (z, z*) € g r 9 / such that (x — z,z*} > 0. Consider z* := z* + x*\ we have that (z, z*) £ df and (z -x,z*— x*) < 0, which shows that the set df U {(x,x*)} is not monotone. Therefore df is a maximal monotone operator. • 3.2
Convexity and Monotonicity of Subdifferentials
The aim of this section is to show that the monotonicity of an abstract subdifferential of a lower semicontinuous function ensures its convexity; among these abstract subdifferentials one can mention the Clarke subdifferential we introduce below. Throughout this section (X, ||.||) is a normed vector space. Consider | / M c I and x e clM. In the sequel we shall denote by (x n ) - > M X a sequence (xn) C M with (xn) -» x; more generally, x - > M X will mean x £ M and x —> x. The Clarke's tangent cone of M at x is defined by Tc{M,x):={ueX\V{tn)
10, V(xn)-*Mx,
3(un)^u, V n e N : xn + tnun 6 M).
Recall another cone introduced in Section 2.3: C(M,x) := e l f | J
t-(M-x))
=cone{M-x).
170
Some results and applications of convex analysis in normed spaces
Because clM -x C C(M,x) Note (Exercise!) that
C C(clM,i), we have C{M,x) = C(clM,:r).
0ETc(M,x)cC(M,x).
(3.11)
Several properties of the tangent cone in the sense of Clarke are collected in the following proposition. Proposition 3.2.1 (i) Tc{M,x)
Let 0 ^ M C X and xEc\M.
Then:
is a nonempty closed convex cone;
(ii) TC(M, x) = Tc(cl M, x) = Tc(M D V, x) for all V E M{x); (iii)
if M is a convex set then Tc(M,x)
=
C(M,x).
Proof, (i) It is obvious that Xu E Tc(M,x) for A > 0 and u E Tc(M,x); hence Tc(M,x) is a cone. Let u,v € Tc(M,x) and consider (tn) 4- 0 and (xn) ~>M x. Then there exists (un) -> u such that x'n := xn + tnun G M for every n E N. Of course (xJJ —>• 3;. Hence there exists (vn) —> u such that £n + in(w„ + u„) = x^ + i n w n G M for every n. Therefore u + v E Tc(M,x). Consider now (uk)ke^ C Tc(M,x) with (ufc) -+ u E X. Let us show that u E Tc{M,x). For this take (tn) I 0 and (xn) - > M £• Because ufc E Tc{M,x), there exists (u*) ->• ufc such that xn + tnukn E M for every n G N. Hence xn+tnukn E M for all k, n E N. For every n EN there exists k'n E N such that ||u* — uk\\ < n~l for every k > k'n. Let (fc„) C N be an increasing sequence such that kn > k'n; then ||u* - uk\\ < n _ 1 for all k,n E N with k > A;„. Let u„ := u* n . Of course, £„ + £ n u„ = xn + tnuknn G M for every n. As ||M„ - u|| = \\unn - u|| < ||u*n - ukn || + \\ukn - u\\ < n _ 1 + ||u*n - u||, we have that (un) ->• u. Hence u G Tc(M,x). Therefore Tc(M,x) is a closed convex cone. (ii) Let u E Tc(M,x) and take (tn) I 0 and (xn) - > C I M £• For every n G N there exists xn E M such that \\xn — xn\\ < tn/n, i.e. xn = xn + tnn~1u'n with u'n E Ux- Hence (xn) ->M x. Therefore there exists («„) -> u with xn + tnun = xn + tn(n~lu'n + un) G M C clM for every n E N. As (n _ 1 u^ + u„) —> u, we have that u G I b ( c l M , x ) . Conversely, let u G Tc(c\M,x) and take (tn) I 0 and (z n ) -tM x. There exists (un) -> u with xn + tnun G cl M for every n. Like above, for every n E N there exists u^ G Ux such that a;„ + tnun + tnn~lu'n G M . Because ( u „ + n _ 1 u ^ ) —> ii, we have that w E TQ^M^ X). Hence TQ^M, X) — Tc(dM,x).
Convexity and monotonicity
of
subdifferentials
171
The equality Tc{M,x) = 7 b ( M n V,x) is obvious when V is a neighborhood of x. (hi) Let M be convex. Taking into consideration (ii) we may assume that M is closed. Consider x E M and take (£„) 4- 0 and (xn) —»M X. There exists no € N such that tn < 1/2 for n > no. Take un := x + x - 2xn for n > n0 and un = 0 otherwise. Of course, (un) -¥ x — x. As xn + tnun = (1 — 2tn)xn + tnx + tnx S M for n > no, we have that x-x € Tc{M,x). It follows that C(M,x) C T c ( M , x ) . Using Eq. (3.11) we obtain the conclusion. D From (ii) and Eq. (3.11) we obtain that Tc(M,x)
Cn '
C(MHV,x)
=:
TB{M,x);
'V'gJV(x)
is the well known tangent cone in the sense of Bouligand of M atxed M. Let now / : X —> M. be a proper function and x 6 dom / . It is natural to consider the tangent cone Tc (epi/, (x, f(x))). This cone is related to the Clarke—Rockafellar directional derivative of / at x introduced as follows: TB{M,X)
,*,_ . f'{x,u):=sup
, limsup e>0 tiQ,(x,a)->epi,(x,f(x))
. f := sup ml
. „ mi
fix + tv) — a
ll«-«ll<e
*
sup
e>0 *>0 0
f(x +
ml
f(x)
II"-'"ll<
£
tv)-a
.
*
It follows that , ,_ , . , / ' t( a ; , w ) = s u p inf
sup
f{x + tv) -
mi
lk-"ll<E
£ > 0 <5>0 o
f(x)
,
*
and even /t(x,u)=sup
limsup
inf
e>0 (4.0, x->/5 ll«-«ll<e
/ (
*+
tV
} ~~f{x)
(3.12)
'
if / is lower semicontinuous at x, where x —>/ x~ means x —>• x and f(x) —>• f(x). It is obvious that / t ( x , 0) < 0 and /''"(xjAu) = A/ 1 '(x, u) for A > 0 and u G X. We may have f^(x, 0) = — oo even if / is continuous at x. Take for example / : R -»• E, f(x) := — \/\x\, and x = 0 (Exercise!). The next result shows the connection between the preceding two notions.
172
Some results and applications of convex analysis in normed spaces
Proposition 3.2.2 Let f : X —> R be a proper function and x G d o m / . Then epi ft(x, •) = Tc (epi / , (x, f(x))). In particular ft(x, •) is a Isc convex function; ft(x, •) is a Isc sublinear function if and only if ft(x, 0) = 0. Moreover, if f is convex then epi ft(x, •) = cl (epi f'(x, •)). Proof. Let (u,A) G Tc (epi f, (x, f(x))); assume that ft(x,u) > A, and take ft(x, u) > A' > A. Then there exist EQ > 0, ((xn,an)) -»epi/ (x,f(x)) and (tn) 4- 0 such that inf„ 6 c( U)£o ) t~* (f(xn + tnv) — an) > A'. Because (u, A) 6 Tc (epi f, (x, f(x))), there exists the sequence ((un,Xn)) converging to (u, A) such that (xn,an) + tn(un,\n) G e p i / , i.e. f(xn + tnun) < &n + tnXn for all n G N. Since (un) -» u, there exists no such that un G D(u,eo) for n > no- Hence
v^
-r
f(xn + tnv)-an
f(xn +
tnun)-an
A <
inf — < — < A„ Vn > n 0 . ||w—u||<e0 tn tn Taking the limit we get the contradiction A' < A. Let now f^(x~,u) < A and take ((xn,an)) —>epif (x,f(x)) and (tn) I 0. Let us fix (ek) I 0. Because • infr
5>0
• inft
sup 0
\\x-x\\<S, \a-f(x)\<S
f(x + tv) - a —
\\v-u\\<ek
t
for every k G N, there exists 8k > 0 such that . inf
sup 0«<<5fc,||x-x||<(5 fc ,/(x)
u
f(x + tv) -a — £
ll»- ll< *
. < A.
*
There exists n'k G N such that 0 < t„ < 8k, \\xn — x\\ < 8k, \an — f(x)\ < 8k for all n > n'k. Therefore inf„€£>(U)efc) t" 1 (f(xn + tnv) - an) < A, which shows that for every k and every n>n'k there exists u* 6 D(u, e^) such that f(xn + tnUn) < a „ + Xtn. Consider an increasing sequence (n^) C N such that nk > n'k for every k G N. Taking un := u* for n^
C epi f'(x, •) C cone(epi/ - (x,
f(x))),
Convexity and monotonicity
we have that epip(x,
of
subdifferentials
•) = C (epi/, (x,f(x)))
173
= cl(epi/'(:r, •))•
D
+
Note that when / is not lsc at x, f^(x, •) and / (x, •) may be different, where, as usual, / is the lower semicontinuous envelope of / . Take for example / : R ->• R, f(x) = 0 for x ^ 0, /(0) = 1; / t ( 0 , •) = - c o , but / (0, •) = 0. When / is lsc at x then f*(x, •) and / (x, •) coincide. When / has additional properties, Z1" has a simpler expression. Proposition 3.2.3 Let f : X —> R be a proper function and x 6 d o m / . Then for every u € X we have: (i) limsupj.^. xP(x,u) ,-lv- N / f'(x,u)<sup
< f^(x,u);
• r inf
moreover, if f is lsc atx • inff
sup
f(x + tv)-
£>0 S>0 |j^—^||<<S,0
< inf
then
f(x)
*
sup
.
*>° \\z-x\\<S,0
(3.13)
*
(ii) / / / is continuous at x then
,-iv- x f'{x,u)—
•r
. t f{y + tv) - f(x)
sup inf
sup
mi
e>0 <5>0 \\x-x\\<S,0
.
ll"-ull<e
*
(hi) If f is finite and L-Lipschitz on B(x,r) for some r > 0 and L > 0, then ttr\ • f(x + tu) - / ( : r ) /'(a;,u) = inf( sup s>0
||x-x||<(5,0
t
f(x + tu) — f(x) , ,. .. = limsup^—-—'-— J -±-L < L -\\u\\. (iv) If f is finite and Gateaux differentiate tinuous at x then f^(x, •) = V/(3;).
on B(x,r),
(3.14)
and V / is con-
Proof. Throughout the proof u € X is a fixed element. (i) Let f*(x, u) < X and consider e > 0; there exists 6 > 0 such that sup \\x-x\\<8,0
. inf lk-«ll<
f(x + tv)-f(x) e
^ < A.
*
Take 6' := 6/2. Let x' e £(x,<5') with \f{x') - f(x)\ < 6' and x, t be such that \\x - x'|| <6',0
174
Some results and applications of convex analysis in normed spaces
0 < t < 8 and f(x) < f(x) + 8. It follows that sup {mt^D^r1
(f{x + tv) - f(x))
| t G ]0, 8'}, x e D{x', 8'), f(x)
l
< sup {miv€D(u>e)r
(f(x + tv) - f{x)) \te]0,8],
x £
D(x,8),
f(x) < f(x) + 5} < A for all x' e B(x,8') with \f(x') - f(x)\ < 8'. Therefore f^(x',u) < A for such x', and so lizn supx^. w f^(x,u) < f^(x,u). The first inequality in Eq. (3.13) follows immediately from Eq. (3.12), while the second is obvious. (ii) Taking into account Eq. (3.13), one must show the inequality >. For this take e > 0 and 8 > 0. There exists 8' € ]0,8] such that f(x) < f(x) + 8 for every x € D(x,8'). Then sup 0<£<<5 \\x~x\\
inf
^
H
llu-ull^e
M
>
sup
t
inf u
0
^
e
+
^ ~ ^ , t
whence the conclusion follows. (iii) The inequality < is proved in Eq. (3.13). Assume that / is Lipschitz on B(x,r) with Lipschitz constant L. Take A > lim sup x _,. s ^ 0 Hx+tu)~Hx) and fix e > 0. Consider 80 > 0 such that sup|| ;c _j||< 5o)0
<
f(x + tv) - f(x + tu)
f{x +
tu)-f{x)
+ tU
] ~ fix) < / ( * + tU] ~ ™ + Le.
Hence sup \\x-x\\<5 0
. , inf ll"-«ll<£
fix + tv) - fix) ^ '-—i±-± < t
sup \]x-x\\<6 0
fix + tu) - i fix) — — - ^ - L + Le, t
Convexity and monotonicity
of
subdifferentials
175
and so . mi
sup
inf
*>° ||i-i||<<5,0
<
£
sup
/(a; + fa) - / ( a ; ) — t J fix + tu)J - L f(x) -^——-— -^+Le<\
||a:-x||<<So,0«<<5o
+ Le.
*
Taking the limit for e -> 0, we obtain \ ^> rhm inf -t A
sup
e4-0 d>0 \\x-x\\<6,0
f(x + tv)-f(x) ^—L = ttf—f ] { x , u\) .
-t inf ll"-«ll<e
t
Taking into account (ii), we get Eq. (3.14). Taking 5 = r / ( l + ||w||), for x € B(x,S) and t £]0,S] we have that x + tu,x € B(x,r), and so f{x + tu) — f(x) < tL \\u\\, whence f^(x,u) < L \\u\\. (iv) Let r > 0 be such that / is Gateaux differentiable on B(x, r). Since V / is continuous at x, V / is bounded on a neighborhood of x. So we may assume that ||V/(x)|| < L for x € B(x,r). Using the mean-value theorem we obtain that / is i-Lipschitz on B(x,r), and so Eq. (3.14) holds. Let (tn) i 0 and (xn) -»• x be such that f^(x,u) = \imt~1 (f(xn + tnu) — f(xn)). But xn+tnu, xn £ 5(3;, r) for n > no (for some no £ N); applying the mean value theorem, there exists 6n £]0,i„[ such that f(xn + tnu) — f(xn) — V / ( x n + 6nu)(tnu) for every n > no- Because V / is continuous at x, we obtain that ft(x,u) = Vf(x)(u). O Let us introduce now the Clarke subdifFerential of / : X —> K at xeX with f(x) £ E. This is dcf(x)
= {x* € X* | {u,x*) < f\x,u)
Vu € X } ;
if f(x) £ E we consider that dcf(x) = 0. Taking into account Theorem 2.4.14, dcf(x) ^ 0 if and only if /^(af, 0) = 0. Moreover, dcf(x) is a nonempty w*-compact subset of X* if / is Lipschitz on a neighborhood of x. Other properties of dc are collected in the following result. Theorem 3.2.4 domg.
Let f,g : X —> R be proper functions and x £ d o m / n
(i) If / and g coincide on a neighborhood ofx then dcf(x) (ii) If f is convex then dcf(x) (iii)
=
df(x).
If x is a local minimum of f then 0 €
dcf(x).
— dcg(x).
176
Some results and applications of convex analysis in normed spaces
(iv) \imsupx_>x dcf(x)
C dcf(x),
where kmsupx^
dcf(x)
is the set
{x* G X* | 3 (xn) -+f x, 3 « ) % x*, Vn G N : < G 3 c / ( a ; n ) } • (3.15) (v) If g is finite and Lipschitz on a neighborhood of x then
(f + g)Hx, •) < fHx, •) + g\x, •), dcU + g)(?) c dcf(x) + dcg(x). (vi) / / g is Gateaux differentiate tinuous at x then
on a neighborhood of x and Vg is con-
(f + g)H*, •) = fHx, •) + V5(af),
dc(f
+ g)(x) = dcf(x)
+ Vg(x).
Proof, (i) and (iii) are obvious because p{x, •) = (flv^ix, •) for every V G N(x), while (ii) follows from the second part of Proposition 3.2.2. (iv) Let x* G limsupx^ fX-dcf(y); t n e n there exist (xn) ->/ x and (a£) ^ x* such that x*n G dcf(xn) for all n G N. Let M £ I be fixed. Then (u,a;*) < p{xn,u) for every n G N, whence, by Proposition 3.2.3 (i), (u,x*) < ft(x,u), and so x* G dcf(x). (v) Let r > 0 be such that g is L-Lipschitz on B(x, r); we may assume that L > 1. Let u £ X, e > 0 and 5 > 0 be fixed and take 6' = mm{S/{2L),r/(l + e + \\u\\)} > 0. Let v, x and t be such that \\v-u\\ < e , \\x-x\\ <5',0
+ g)(x) ~
f{x + tv) - f{x) t
g{x + tu) - g{x) t
whence, taking first the supremum with respect to v G D(u,e), we obtain that a < Ai(5) + A2(5) + Le, where a :=
sup ||a!-s||
.
mi H"-«ll<£
(f + g)(x + tv)-(f
+ g)(x)
t
U+9)(x)<(f+gm+s' Ai(S)
:=
sup \\x-x\\<S,0
inf — Il«-«II<£
A2(d) := sup {t'1 (g(x + tu) - g{x)) \x
eD(x,5)}.
—-^-L, t
,
Convexity and monotonicity
of
subdifferentials
177
Therefore /3 < Ai(S) + A2{5) + Le for every 5 > 0, where /3:=inf 5 >0 '
sup ||a!-»||<',0
inf if + 9)(x+ tv) - jf + g)(X) ^ £ Il -"II< * u
(f+9)(x)<(f+g)(x)+S'
Taking the limit of Ai (S) + A2 (5) for S —> 0 (taking into account that A\ and A2 are nondecreasing for 5 > 0), we get inf
sup
inf
(/ + g)(* + * 0 - ( / + g)(*)
5
>°\\x-x\\<5,0
•
f
< inf s>0
t
sup \\x-x\\<6,0
X
f(
f
inf ll"-"ll<
—£
+
tv
)
~
/(*)
-
—L
*
. 9(x + tu) — g(x) + inf sup — '-—^^- + Le. <5 >°||x-x||<(5 t
Taking now the limit for e —> 0, we get the desired conclusion. The relation for the subdifferentials follows from the preceding inequality and the fact that g^(x, •) is continuous. (vi) As mentioned in the proof of Proposition 3.2.3 (iv), g is Lipschitz on a neighborhood of x and so the conclusion of (iv) holds with g^(x, •) = Vg(x). Applying again (iv) for f + g and — g (we may assume that g is finite on X; otherwise take g = 0 outside a neighborhood of x), we obtain that f*(x, •) < (f + g)^(x, •) — Vg(x). The conclusion is now obvious. • In the rest of this section we use an abstract subdifferential. Before introducing this notion let us consider the following class of finite-valued convex functions: C(X) := {g : X -> E | g is convex and Lipschitz}; of course, €(X) is a convex cone in the vector space Rx of all functions from X into E. Let d ( X ) C E x be the cone generated by X*U{d [a , 6] | a, b € X) (C €{X)) and €2{X) C E x be the cone generated by
X*u{dfaM\a,beX}WD0, where
Do := { ^2k>QVkdlk
(Hk) C K+, ^2k>Ql*k
These cones will be used below.
=
*' (u*)*>°
conver
Sent} •
178
Some results and applications of convex analysis in normed spaces
We call an abstract subdifferential on the nonempty class T C M. a multifunction d : X x T =4 X*, which associates to (x,f) a set denoted by df(x), satisfying condition (PI) below: (PI) 0 G limsup y _> x d/(y) + dg(x) if / € T, g € <£(X), f(x) £ 1 and a; is a local minimum of / + g, where limsup _yX df{y) is defined as in Eq. (3.15) and dg(x) is the Fenchel subdifferential of g at x. A stronger form of (PI) is (P2) 0 € ~8f(x) + dg(x) if / 6 T, g e €(X), f(x) 6 1 and a; is a local minimum of / + g. Sometimes one asks (P3) df(x) = df(x) if / e
Tn€(X).
Of course, (P2) holds if (P4) and (P5) below are satisfied: (P4) 0 6 df (x) if / 6 J-, f(x) € E and x is a local minimum of / , (P5) T + €(X) C T and 8{f + g)(x) C df(x) + dg(x) when / G T and 9 e €(X). Note that WX + €(X) = RX,
A(X) + €(X) = A(X),
T(X) + €(X) = T(X).
Remark 3.2.1 From the preceding theorem we observe that Clarke's subdifferential dn and the Fenchel subdifferential d are abstract subdifferentials x on R and A(X), respectively; in fact they satisfy conditions (P1)-(P5). There are many other subdifferentials which satisfy condition (PI). Remark 3.2.2 If the abstract subdifferential d on T satisfies the stronger condition (P2) then for any proper function / 6 T one has df(x) C df(x) for all x € d o m / . Indeed, if x* 6 df(x) then a; is a (local) minimum point of / + (—x*), and so, by (P2), 0 £ df(x) + d(-x*)(x) = df(x) - x*. Hence x* € df(x). The following approximate mean value theorem proves to be useful in nonconvex analysis.
Convexity and monotonicity
of
subdifferentials
179
Theorem 3.2.5
(Zagrodny) Let (X, ||-||) be a Banach space and d be x an abstract subdifferential on T C E . Let f 6 T be Isc, a,b G X with a G d o m / and a ^ b, and r 6 l with r < f(b). Then there exist (xn) —>•/ c € [a, b[ and x*n G df(xn) for every n G N such that (i) r — f(a) < lim inf (6 - a, x*n), (ii) 0 < lim inf (c (iii)
xn,x*n),
f^(r-/(a))
(iv) ||& - a|| (/(c) - /(a)) < ||c - a|| (r - / ( a ) ) . Proof. There exists x* G -X"* such that (6 - a, x*) = r — f(a). Consider h := / — x*; then /i(a) < h(b). Because h is Isc, there exists c G [a,b[ such that h(c) < h(x) for all a; G [a,b]. Therefore c = (1 — /x)a + fib for some /i G [0,1[. It follows that c — a = fi(b — a) and ||c - a|| = fi \\b — a\\. Therefore /(c) - f(a) = h(c) - h(a) + (c — a,x*) < fi(r - / ( a ) ) , whence (iv) follows. Let 7 < h(c); then there exists r > 0 such that 7 < h(x) for every x G [a, b]+rUx- Otherwise, for every n G N there exists xn G [a, b ] + n - 1 { / x such that h(xn) < 7. Hence arn = dn + yn with d„ G [a, 6] and y n G n~1UxAs the segment [a,b] is a compact set, there exists a subsequence (dnk) converging to d G [a,b]. It follows that (xnh) —> d, and so, by the lower semicontinuity of h, we get the contradiction h(d) < 7. Let r > 0 correspond to 7 := h(c) — 1, and take C/ := [a, b] + rUx', of course, U is closed. Like above, for any n G N there exists r„ G ]0, r[ such that /i(z) > h(c) — n~2 for 3; G [a,b] + rnUx', choose tn > n such that 7 + tnrn > h(c) — n~2. Then one has Vz G U : /i(c) < h(x) + tnd[atb](x)
+ n~2.
(3.16)
Indeed, the inequality is obvious for x G [a, b] + rnUx- If x £ U \ ([a, b] + r„[/x) then d[0i6](x) > r„, and so /i(x) + t„d[a)ft](a;) > 7 + i n r n > ft.(c)-n-2. Consider i?„ — h + iu + *nd[0)(,]. Prom Eq. (3.16) we have that Hn(c) < infx Hn + n - 2 ; moreover, Jf„ is Isc and bounded from below. Applying Corollary 1.4.2 for Hn, c, e ~ n~2 and A := n _ 1 , we get un G X such that Hn(un)
< Hn(c),
||c-u„|| < n_1,
(3-17)
1
(3.18)
Hn{un) < Hn(x) + n - ||z - u n ||
Va; G X.
Since (un) —> c G [a, 6[, we may assume that un G intf/ for every n.
180
Some results and applications of convex analysis in normed spaces
Relation (3.18) can be written as ( / + tnd[atb] + n~ldUn - x*) (u„) < ( / + tnd[aib] + n - 1 d „ „ - x*) (x) for every x 6 U, where da denotes d[a,a] = ^{o}- Therefore un is a local minimum of/ + (tnd[a,b] +n~1dUn -x*); the function between the parentheses being convex and Lipschitz, by (PI) we have that 0 6 limsup <9/(2/) + d(tnd[a
n-Hl).
Let yn £ [a, b] be such that d{a^{un) = \\un — yn\\ (such an element exists because [a, b] is compact). Because 0, and so (yn) -> c. Since c ^ b, we may suppose that yn ^ b for every n 6 N. Hence yn = (1-A n )a+A„6 with An € [0,1[. From Proposition 3.8.3(H) on page 239, dd[a,t](u„) = d||-|| (u n -y n )r\N([a,b],y n ). Therefore ((1 - A)o + \b- yn,v*n) = (A - A„) (6 - a , < ) < 0 for every A € [0,1[. Hence (b — a, u*) < 0 for every n S N. Moreover, (c - u n , u*) = (c - y n ,v*) - (u„ - yn,v*) It follows that (b-a,u*n) n_11|& — a\\, whence
> (b-a,x*)
< - ||u n - y„|| < 0.
- ^(fr-a,^)
> r - f(a)
-
liminf (b — a,u*) > r — / ( a ) . Similarly, {c — un,u*n) > (c — u„,x*) — n _ 1 | | c - u n | | , whence liminf (c — un,u*n) > 0. From Eq. (3.17) we get f(un) - (un,x*) < /(c) - {c,x*), and so /(c) < liminf f(un) < l i m s u p / ( u n ) < /(c); hence l i m / ( u n ) = /(c) which shows that (un) —>f c. Because u*k e l i m s u p . ^ ^ df(u), for every k £ N there exist (u„,*) ->•/ Ufc and (u*^) ^> u£ (for n -» oo) such that u*nk G df(un>k)
for all n, A; e N.
Convexity and monotonicity
of
subdifferentials
181
It follows that ((c - uniu,u*nk)) -> ( c - Ufc,u£) for n -> oo. Therefore for every A: € N there exists n'fc € N such that IK,*-Wfc|| < fc_1, \f(un,k)(c-unik,u*nik)-
f(uk)\ < k'1,
\(b-a,u*nk
(c-uk,u*k)\
- u*k)\ < AT1,
< k~l
for n > n'k. Consider (nk) C N an increasing sequence such that nk > n'k for all k e N. Let xn := u„,fc and a;* := u^ fc for nk < n < nk+i- Since for nk
~ Uk\\ + \\uk - C\\ < k~l
+ \\uk - C\\ ,
l / W - Z t o l ^ + l/toWtol, (b - a, < ) = {b-a, u*k) + {b-a, < < f c - u*k) > (b - a, u*k) (c-xn,x*n)
= (c-u n , f c ,u* ) A .) > (c-uk,u*k)
k'1
- fc-1,
we obtain that (xn) —•/ c, lim inf (6 — a, xn) > lim inf (b — a, u£) >r — f(a) n—>oo
k—•oo
and lim inf (c — x n ,a;*) > lim inf (c — Ufc,u^) > 0. n—s-oo
fc-+oo
Taking into account that b — c = (1 — /j,)(b — a) and \\b — c|| = (1 — /i) ||b — o||, from the preceding two relations we get lim inf (b- xn,x*n) = lim inf ((1 - fi) (b- a, a:*) + {c-
xn,x*n))
> (1 — /x) lim inf (b — a,xn) + lim inf (c — a;„,a;*) > K The proof is complete.
(r - / ( a ) ) . •
Remark 3.2.3 The conclusion of Theorem 3.2.5 remains valid if (PI) holds only for functions g in the cone £i(X) or in the cone C2PO defined on page 177. Indeed, for the first part just note that in the proof of Theorem 3.2.5 we used (PI) only for g = i„d[aj6j + n~1dUn — x* € £i(X). For the second part one must notice that one can find tn > 0 such that 7 + tnrn > h(c), and so Eq. (3.16) holds with d[Q>(,] replaced by d?a bi. Applying the Borwein-Preiss variational principle for p = 2, Hn,
182
Some results and applications of convex analysis in normed spaces
c, e := 2n~2 and A := \/2/n, we find un € B(c, y/2/n) and a function 0 2 ,„ generated by the sequences (fJ.n,k)k>o C K+ with X^>0Mn,ib = 1 and (wn,fc)fc>o C B(c, \/2) (see Eq. (1.11) on page 31) such that un is a local minimum of / + (tndfa M + n~1@2,n — x*). Then, in the expression of un we have tn \\un — yn || instead of tn and 6* 6 23/2Ux* instead of &£ 6 C/x* (see also Exercise 2.40). The rest of the proof is the same. Corollary 3.2.6
Let X be a Banach space and d be an abstract subx differential on T C M. . If f G T is a Isc proper function then for every x e d o m / there exists ((xn,xn)) C gr<9/ such that (xn) —»/ x and liminf (x — xn,x^) > 0. In particular d o m / C cl(domd/). Proof. Let x S d o m / . If z is a local minimum of / then 0 belongs to limsupy^ xdf(y) by (P2) (just take g = 0). Therefore there exists (xn) —>f x and (a;*) —> 0 such that a:* 6 df(xn) for every n € N. Of course, lim (x — xn, x^) = 0 in this case, and so the conclusion holds. Assume that x is not a local minimum of / . Then there exists (xn) —> x such that f(xn) < f{x) for every n £ N. Since / is lsc, we get (xn) -»•/ x. Applying the preceding theorem for xn, x and r = f(x), there exists ((xk,n,x%tn))keN C domdf such that (xk,n) ->/ yn G [x„,a;[ for k -» oo and 0
< |jx-lj (/W
_
/(^«)) < Hminf *,_>«, (a; - a;fc>„,a;^„),
/(l/n) < /(*„) + | ^ f (/(*) - /(*»)) < /(*) for every n € N. Since \\yn - x\\ < \\xn - x\\ we have that (yn) -> x, which, together with the preceding inequality and the lower semicontinuity of / , yields (yn) ->•/ x. Then, for every n 6 N there exists k'n € N such that lk*,n - Vn\\ < n _ 1 , |/(a:fc,„) - f(yn)\ < n _ 1 and 0 < (x - Xk,n,x*kn) f o r every k > k'n. Let (fc„) C N be an increasing sequence such that kn > k'n for n € N. Then lkfc„,n - z|| < ||a;*„,n - y n || + \\yn - x\\ < n _ 1 + \\yn - x\\, l/(**»,») - f(x)\
< \f{xkn,n)
- f(yn)\
+ \f(Vn) -
f(x)\
< n - 1 + |/(y„) - /(a:)| , 0 < (z-Z*„,„,4„,n) for all n £ N. Taking xn := Xkn,n and a;* := a;^ n , we have the desired conclusion. •
Convexity and monotonicity
of
subdifferentials
183
When T = A(X) and d is the Fenchel subdifferential, the statement of the preceding corollary is very close to assertion (i) of Theorem 3.1.2. Theorem 3.2.7 X
Let X be a Banach space, d be an abstract subdifferential
on T C K and f £ T be a Isc proper function. If df is a monotone multifunction then f is convex. Proof. We prove the theorem in three steps: 1) a £ d o m / , b £ domdf => [a,b] C d o m / , 2) g r d / C gr<9/, 3) / is convex. 1) Assume that there exists b' £ [a, b] \ dom / ; so b' — Xa + (1 — X)b with A £ ]0,1[. Fix y* £ df(b) and take r £ ffi, r > / ( a ) + A-1(l-A)||6-a||.||2/*||. By Theorem 3.2.5, there exist (xn) - » / c £ [a, b'[ and x* € df(xn) n € N such that liminf (&' — a;n,a;*) > 0,
for every
liminf (b' — a, a;*) > r — f(a).
Since df is monotone we get the contradiction l|6-a||-||y*ll>ll&-c||-||y*||><6-c)i/*> = liminf (b — xn,y*) > liminf (b — xn,x*n) - liminf (A(l - A)" 1 (b' - a,x*n) + (b' - xn, x*n)) > A(l - A) - 1 liminf ( 6 - a , x * ) + liminf (b' >
A ( 1
_
A )
-i
( r
_
/ ( a ) )
-xn,x*n)
.
2) Let x* 6 df{b) with b 6 X. Consider x £ d o m / . Applying again Theorem 3.2.5 for r = f(b), there exist (xn) ->f c € [x, b[ and x* € df(xn) for every n 6 N such that liminf (b-xn,x*n)
> g f f } (/(6) - / ( * ) ) .
Since 9 / is monotone and c = Ax + (1 — A)6 with A € ]0,1], we have that A (6 — x,x*) = (b — c,x*) = liminf (& — x n ,x*) > liminf (b — x n ,a;*) >A(/(6)-/(x)), and so (x — b, x*) < f(x) — f(b) for every x £ d o m / . Therefore x* £ df(x). 3) Let x, y £ dom / and z := (1 — X)x + Xy with A £ ]0,1[. By Corollary 3.2.6, there exists (yn) c dom df such that (yn) ->•/ y. Let a;n := (1-A)x + Ay„ for n £ N; by 1) we have that zn £ d o m / for every n. Using again
184
Some results and applications of convex analysis in normed spaces
Corollary 3.2.6, there exists ((zn,k,zn k)) c STOf s u c r i t n a t (zn,k) —>•/ zn for k —> oo and liminffc_>oo(^n — zn,kiznk) — 0. By 2) we have that z*nk G df(zn>k), and so (x-zntk,z^k)+f(zn,k)
< f(x),
(yn-
zn,k,znk)
+ f{zn>k) < f(yn)
for all n,k G N. Multiplying the first inequality by (1 — A) > 0 and the second one by A > 0 we get (zn - zn,k, z*n>k) + f(zn,k)
< (1 - A)/(x) + Xf(yn)
Vn, k G N.
Taking the limit inferior for k —> oo we get f(zn) < (1 — A)/(x) + Xf(yn) for every n. Passing to the limit inferior for n —> oo and taking into account the lower semicontinuity of / and the fact that (yn) ->/ y we get f(z) < (1 — A)/(a;) + Xf(y), and so / is convex. • We give now another short proof for the Rockafellar theorem on the maximal monotonicity of the subdifferential of a lsc convex function. Theorem 3.2.8 Let X be a Banach space and f € T(X). maximal monotone.
Then df is
Proof. Let x G X and x* ^ df(x). Then x is not a minimum point for g := f — x*, and so there exists z E X such that g{z) < g(x). Let r G E be between these numbers. By Theorem 3.2.5 applied for d on r(X), there exists ((yn,yn)) c Sr9g such that (yn) —>fl y € [z,x[ and lim inf (x - yn,y*n) > | f 5 | [ (r - g{z)) > 0. Therefore (x - yn,y*n) > 0 for n > no, for some n 0 G N. It follows that y* := yno + x* € df(y), with V '•= 2/n0) and (x — y,x* — y*) < 0. Therefore df is maximal monotone. • Theorem 3.2.5 can also be used for proving that two convex functions with the same subdifferential differ by an additive constant. In fact one obtains even more. Theorem 3.2.9
Let C be an open convex subset of the Banach space X jf
and d an abstract subdifferential on T C IK . Let f G T be proper and lower semicontinuous and g G T(X). Assume there exists e G R+ such that df(x) C dg{x) + eUx*
V i G C.
(3.19)
Then C D dom / = C f) dom g and 9{x) - g{y) ~e\\x-
y\\ < f(x) - f(y) < g(x) - g(y) +e\\x~
y\\
(3.20)
Convexity and monotonicity
of
subdifferentials
185
for all x £ C and y £ C fl domg. Proof. Consider 7 > e. By Corollary 3.2.6 we have that d o m / C cl(dom9/), and so C fl domdf ^ 0. Let y £ C fl d o m 9 / and x £ C. Prom our assumption we have that y £ doming, and so y € domg. We are going to prove that /(*) - /(!/) < $(*) - 5(2/) + e II* - 2/11 •
(3-21)
The inequality is obvious if x £ domg or 1 = y, and so assume that x £ domg and x ^ y. Because dg(y) ^ 0, we have that g'(y,u) > - 0 0 for all u £ X. Consider ip : E -»E defined by
g'(y,u),
and this quantity is finite. Therefore there exists to £ ]0, ||:r — y||[ such that g'(y + t0u,u)
9 g'(x0,u) < 9{V + *>">to - 9{V) + | ( 7 - e ) = g
f(x0)-f(y)
(3.22)
Assume that Eq. (3.22) does not hold, i.e.
f(xo) - f(y) > 9{x0) - g(y) + 7 Iko - y\\ • Choose r € E such that r < f(xo) and r - f(y) > g(x0) - g(y) + 7 ||*o - y\\ • By Theorem 3.2.5, there exist (xn) ->f z £ every n £ N such that
\I),XQ\
and x*n £ df(xn)
liminf (a?o -x„,a;*) > — - — — (r - /(j/)).
(3.23) for
186
Some results and applications of convex analysis in normed spaces
Because (xn) —> z we have that (||a;o — x n ||) —> ||xo — z||, and so
]xo-xn\\
I
\\XQ-y\\
Taking into account Eq. (3.23), there exists no G N such that *o-*» - * \ Fo-a;n|| /
>
g(»°)-gy M^cro — 2/11
vn>n0.
+ 7
Since ( x „ ) —>• z G [y, xo[ C C, we m a y a s s u m e t h a t xn G C for n > no- B y E q . (3.19) we have t h a t xn = zn + eu*n with z* G dg(xn) a n d u* G t / x * for n > no- Using also t h e convexity of g, it follows t h a t g(x0)-g(xn) ^ / so^gn ||ar0 — a;„||
\ ||z 0 - z„|l
A
>
9(xo)-9(y)
/
+ 7
_
£
V n
^no_
||ar0 — 3/1
Taking into account the lower semicontinuity of g at z and Eq. (3.19) we get g{xo)-
g(z)
g(x0) - g(y)
,
,
IN-2/H
IFO-^II
Since 2 G [y, io[, z = y + su for some s G [0, io[- Using the convexity of
—ii-
rr- = — :
,
,
— <
(x0,u).
\\x0 - z\\ t0 - s Therefore Eq. (3.22) holds. Consider T~{te
]0, ||rc - y||] I f(y + tu) - f(y) < g(y + tu) - g(y) + jt} .
From Eq. (3.22) we have that t0 G T, and so t := supT G]0, ||a; - y||]. Because
_ , , > * & • « ) = V (<) > V (0) = ff+(»,0),
whence g'(y,u) G 1R. Moreover, y G d o m / D domg. Proceeding as above with y replaced by y (noting that we used only that y G dom / n dom g and g'(y,u) G R), we obtain some s 0 G]0, ||a; - y\\] such that f(y + s0u) - f(y) < g(y + s0u) - g(y) + -ys0.
Convexity and monotonicity
of
subdifferentials
187
As f{y) — f(y) < g(y) — g(y) + 7*, we obtain by addition that f(y +(t + s0)u) - f{y) < g(y + (t + s0)u) - g{y) + 7 ( I + s 0 ), contradicting our choice of t. Hence t = \\x — y\\, and so f(x) — f(y) < g(x) — g(y) + 7 \\x — y\\. Because 7 > e is arbitrary, we obtain that Eq. (3.21) holds. Taking i G C f l dom g and a fixed y G C n dom df, from Eq. (3.21) we obtain that C n dom g c C f l d o m / . Let now j / e C f l d o m / . From Corollary 3.2.6 we get a sequence (yn) C dom df with (yn) —>/ y. Because y G C, we may assume that yn G C for every n € N. Let i G C n dom g be fixed. From Eq. (3.21) we have that
f{x) - f(yn) < g(x) - g(yn) + e\\x-yn\\
Vn € N.
Since g is lsc at y, we get fix) - f(y) < g(x) - g(y) + e\\x-
y\\,
(3.24)
and so y £ dom g. Hence C (~1 dom / = C fl dom g. Let x, y € C fl dom / = C fl dom g; then Eq. (3.24) holds. Interchanging x and 1/ in Eq. (3.24), we obtain also that
f{x) - f{y) > g(x) - g{y) - e \\x - y\\, which proves that Eq. (3.20) holds for all x,y G C f] d o m / . Equation (3.20) being obvious for x & C \ dom / = C\ dom g and y £ Cr\ dom g, the conclusion follows. D A more precise result is the following. Corollary 3.2.10 Let X be a Banach space and d an abstract subdifferential on T C K. . Consider a proper and lower semicontinuous function / € f and g € T(X). If df(x) C dg(x) for every x G X then there exists S;£R such that f = g + k. Proof. Take C — X and e ~ 0 in Theorem 3.2.9. Fixing x0 G d o m / = dom g, we obtain that f(x) = g(x) + f(xo) — g(xo) for every x G X. • Corollary 3.2.11 Let X be a Banach space and f,g€ dg(x) for every x G X then f = g + k for some k €.R.
T(X). Ifdf(x)
C
Proof. Apply the preceding corollary on F(X) and the Fenchel subdifferential. •
188
Some results and applications of convex analysis in normed
spaces
The next result completes Proposition 2.4.3. Corollary 3.2.12 Let X be a Banach space and T : X =4 X* be a maximal cyclically monotone multifunction. Then there exists f G T(X), unique up to an additive constant, such that T = df. Proof. Because T is maximal monotone, g r T ^ 0. Fix (XO,XQ) G g r T and consider fr defined by (2.37) on page 85. Then, by Proposition 2.4.3 we have that fr G T(X) and g r T C grdfr- Since T is maximal monotone we even have g r T = grdfr, and so T = df. The uniqueness follows from Corollary 3.2.11. • 3.3
Some Classes of Functions of a Real Variable and Differentiability of Convex Functions
Consider the following useful classes of functions: A = {
A0
Nk = {ip G A | P 3 11-> t~kip(t) € 1 + is nondecreasing}, Jfc G {0,1,2},
= {<^ G yi | iimmrkip(t) = o}, ke {o, i},
r
= {ip G .A |
To Si
= r n A), r^ := y G r 0 1 iiminft_+0Ot-V(*) > o}, = r n n i , s? ~ {^eEi liimsup^^rVC*) <°o}.
Of course, N2 C Ni C N0, E? C Si C fti C ft0 and Tg C T 0 C T C A^. Moreover, if
(s) := inf{t > 0 | y>(t) > s } ,
(s) := supjr- > 0 |
(3.25)
it is obvious that p e < / , i/ie £ iVo,
Differentiability
of convex
functions
189
Furthermore, one can extend
(i) Ifip£A0r\N0
then tpe,tph £ Cl0; if(p£Cl0r\N0
then
(ii) Let
i> £ r 0 <s> v # e Si and v e rg o v # e sf • (iv) Let p,q £ ]1, oo[ be such that 1/p + 1/q = 1 and
190
Some results and applications of convex analysis in normed spaces
particular v*(s) > sup{st - st/2 \t£ [0,e]} - es/2 > 0. Hence
-
Therefore s i-> s~qip#(s) is nonincreasing on P. The proof for the other statement is similar. (v) For x* £ X* we have that r(x*)=sup{(x,x*)-p(\\x\\)\xeX} = sup{(x,a;*) -
\\x\\ = t}
= sup sup ((x,x*) — (p(x)) =sup(t\\x*\\—(p(x)) t>0 \\x\\=t
= tp* (\\x*\\).
t>0
The equality f**(x) = <^ ## (||a;)|| is obtained similarly.
D
As an illustration of the use of the above classes of functions let us give the following interesting result about the differentiability of a convex function. First recall that a bornology on X is a family B of nonempty bounded subsets of X having the properties: 1) every set B of B is symmetric, 2) B is directed upwards, i.e. for all B±, B2 £ B there exists B3 £ B such that S i U B2 C B3 and 3) 1){B | B £ B} = X. The topology induced by the bornology B on X*, denoted by Tg, is the topology of uniform convergence on every set B of B. The most common homologies on X are the Gateaux, the Hadamard and the Frechet homologies which are obtained taking as B the classes of finite symmetric subsets, the weakly
Differentiability
of convex
functions
191
compact symmetric subsets and the bounded symmetric subsets of X, respectively. To the Gateaux bornology corresponds the weak* topology on X*, while to the Prechet bornology corresponds the norm topology on X*. It is clear that the topology m which corresponds to the bornology B is finer than the weak* topology and coarser than the norm topology. The function g : X —>• E is said to be S-diflFerentiable at x if / is finite on a neighborhood of x and there exists x* £ X* such that lim
/(x
t->0
+
^)-/(x) t
\ '
/
'
v
v
the limit being uniform w.r.t. u 6 B, for every B £ B; x* is denoted by Vf(x) and is called the ^-differential or ^-derivative of / at x. Of course, for the Gateaux, Hadamard and Prechet bornologies one obtains the Gateaux, Hadamard and Prechet derivatives, respectively. Condition 3) implies that / is Gateaux differentiable at x whenever / is B-differentiable at x for the bornology B; in particular the ^-derivative is unique when it exists. T h e o r e m 3.3.2 Let f € A(X) be continuous atx£ bornology on X. Consider the following statements:
doxnf and B be a
(i) / is B-differentiable at x, (ii) / is B-smooth
at x, i.e. l i m ^ o i - 1 ^ ^ ) = 0, where, for t > 0,
o-B(t) := sup{/(x + ty) + f{x - ty) - 2f{x) \ y € B}, (iii) every selection of df is norm to T& continuous at x, (iv) for every x* £ df(x) there exists a selection 7 of df which is norm to TB continuous at x and j(x) = x*, (v)
there exists a selection of df which is norm to TB continuous at x,
(vi) (a;*) -4 x* whenever x* £ df(x), (xn) -)• x,
x* £ df(xn)
for n £ N, and
(vii) (x*n) ^4 x* whenever x* £ df(x), and (en) -» 0, (xn) ->• x,
0 < en, x*n £ denf(xn)
for n £ N,
(viii) (x*) ^4 x* whenever x* € df(x), and (e n ) -> 0.
0 < e„, x* £ denf(x)
for n £ N,
TTien (i) <£> (ii) <=> (iii) «• (iv) <s> (v) o (vi) <£= (vii) o (viii). Moreover, if X is a Banach space and f is Isc then (vi) => (vii).
192
Some results and applications of convex analysis in normed spaces
Proof, Then
(i) => (ii) Assume that / is S-differentiable at x. Let u € X.
lim ^ tio
+ tU) +
^
~ ^
~
2f
^
t
= lim f{* + *4.o = (u,Vf(x))
tu)
t +
~
/(5f)
I lim / ( * ~ t\o (-u,Vf(x))=0.
tu)
t
~
/ (
^
As the limits in the right-hand side of the first equality above is uniform w.r.t. u £ B, for every B £ B, we obtain that / is ^-smooth at x. (ii) => (i) Take x* £ df(x); such an element exists by Theorem 2.4.9. Consider B £ B and take u £ B; one has OBH) > f(x + tu)+f(x-tu)
-2f(x)
>f(x
+ tu) - f(x) -t(u,x*)
> 0,
and so 0
tu)-m
_{UiX.}<*BM
vues.
Hence Eq. (3.26) holds, the limit being uniform w.r.t. u £ B. (i) => (iii) Let 7 : D —• X* be a selection of df, where D := dom df; of course, int(dom/) C D and 7(2;) = V/(af). Let B £ B. Consider aB : K+ - > ! + ,
a B ( t ) := sup{f(x + tu) - f(x) -t(u,j(x))
\u£
B}. (3.27) Because / is S-differentiable at x, lim^o £ _ 1 <7B(£) = 0, i.e. a £ fii. Because / is convex, <7j9 is convex, too. For x £ D, u £ B and £ > 0 we have that (z + tu - x, 7(0:)) < /(& + tu) - /(:r) < / ( £ ) + t (u, j(x)) + aB(t) -
f(x),
whence t (u, j(x) - i{x)) - aB(t) < f(x) - f(x) + (x-x,
7(2)).
Taking first the supremum with respect to u £ B, then with respect to t > 0 on the left-hand side of the above inequality we obtain that (CT B ) # (sup u€B
\{U,J(X)
- j(x))\)
< f(x) - S{x) +
< f(x) - f(x) + \\x - x\\ • | | 7 ( i ) | |
(x-x,j(x)) Vx £ D.
Differentiability
of convex