This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Introduction
xvn
tion, cp = m a x { / i , . . . , / „ } and tp = A 0 / 2  Besides classical conditions one points out very recent ones. For the proof one constructs an adequate perturbation function and uses the fundamental duality theorem. In Section 2.9 we apply the fundamental duality theorem for obtaining necessary and sufficient optimality conditions in convex optimization problems with constraints. These conditions involve the subdifferentials of the functions considered or the corresponding Lagrangian. The results are wellknown. However we mention the formula for the normal cone to a level set stated in Corollary 2.9.5 for not necessarily finitevalued functions which is quite new. The minimax theorem presented in Section 2.10 will be used in the section dedicated to monotone multifunctions. Throughout Chapter 3 the involved spaces are normed spaces. In Section 3.1 besides the classical theorems of Borwein, Br0ndstedRockafellar, BishopPhelps and Rockafellar (on the maximal monotonicity of the subdifferential of a convex function) we present a recent theorem of Simons and use it for a very short proof of Rockafellar's theorem (mentioned before). As a consequence of the Br0ndstedRockafellar theorem we obtain other three conditions for the validity of the formulas for the conjugate and subdifferential of the function F(,A()) (and therefore for the functions
f + go A and f + g). The aim of Section 3.2 is to characterize the convex functions using other types of subdifferentials. In fact we use an abstract subdifferential. An example of such subdifferential is Clarke's one for which we establish several properties. The main tool for such characterizations is the wellknown Zagrodny's approximate mean value theorem; the version we present subsumes several results met in the literature. We present also an integration theorem of Thibault and Zagrodny which yields the fact that two lower semicontinuous convex functions on a Banach space which has the same Fenchel subdifferential coincide up to an additive constant. In Section 3.3 we introduce the class A of functions tp : K+ —> R+ with y>(0) = 0 and several useful subclasses. To any ip € A we associate tp# € A defined by
0}. These classes of functions turn out to be useful in studying wellconditioned convex functions, uniformly convex and uniformly smooth convex functions, as well as in the study of the geometry of normed spaces. As an illustration of the use of these classes of functions we study the differentiability of convex functions with
XV111
Introduction
respect to arbitrary bornologies. Using one of the characterizations and the Br0ndstedRockafellar theorem one obtains the following interesting result of Asplund and Rockafellar: Let X be a Banach space and / : I  > l a proper lower semicontinuous convex function; if / * is Frechet differentiable at x* e int(dom/*) then V/*(x*) € X. In Section 3.4 we introduce the wellconditioned convex functions and give several characterizations of this notion using the conjugate and the subdifferential of the function. An important special case is that of wellconditioning with linear rate. This situation is studied in Section 3.10. In Section 3.5 we study uniformly convex and uniformly smooth convex functions, respectively. To any convex function / one associates the gages pj and 07 of uniform convexity and uniform smoothness, respectively. The gage pf has an important property: the mapping 0 < t i» t~2pf(t) is nondecreasing. Because a/* = ( p / ) * and a/ = (p/*) # f° r a n v proper lower semicontinuous convex function / , the mapping 0 < i 4 t~2crf(t) is nonincreasing for such a function; moreover, one obtains that for such an / , / is uniformly convex if and only if / * is uniformly smooth and / is uniformly smooth if and only if / * is uniformly convex. Then one establishes many characterizations of uniformly convex functions and of uniformly smooth convex functions. In these characterizations appear functions (gages or moduli) belonging to different subclasses of A introduced in Section 3.3. These gages and moduli are sharp enough in order to obtain that / is cstrongly convex if and only if /* is Frechet differentiable on X* and V / * is c _1 Lipschitz. Even if the results are established in general Banach spaces the natural framework for uniformly convex and uniformly smooth convex function is that of reflexive Banach spaces. This is due to the fact that when there exists a proper lower semicontinuous and uniformly convex function on a Banach space whose domain has nonempty interior, the space is necessarily reflexive. Section 3.6 is dedicated to the study of those convex functions which are uniformly convex on bounded sets and uniformly smooth on bounded sets, respectively. Under strong coercivity of the function one shows that these notions are dual. In Section 3.7 we study the function / v : X >• E, f^x) = / „ " tp(t) dt, where
a\\x\\} is a pointed (C 0 C = {0}) closed convex cone with nonempty interior and C+ = R+ • ((p + aU*) = R+ • D((p, a). E x e r c i s e 1.10 Let X, Y be topological vector spaces and 31 : X =t Y be a convex multifunction. Assume that there exists x G X such that int 3^(5:) ^ 0. Prove that for every (xo,j/o) G grft with yo G (Imft) 1 there exists u G X such that j/o G intK(x) for every x &}xo,u]. i},
0. Applying again Theorem 2.1.9 we obtain that ipx,y is convex, and so / is convex. • In the next result we give characterizations for strictly convex functions. Theorem 2.1.12 Let D C (X, \\ • ) be a nonempty, convex and open set, and f : D » E be Gateaux differentiable (on D). Then (i) <$• (ii) •£> (hi) •£= (iv), where (i) / is strictly convex; (ii) Vx,yeD,x?y 0. Proof. By contradiction, assume that Dtp(t, 1) < 0 for every t 6 [a, b[. Fix a > 0; we have that tp(t) <
Introduction
xix
One obtains also characterizations of (local) uniform convexity and (local) uniform smoothness of X with the help of the properties of fv. For example one obtains: X is uniformly convex •£> X* is uniformly smooth •& fv is uniformly convex on bounded sets O (fv)* is uniformly smooth on bounded sets •& {ftp)* is Frechet differentiable and V ( / v ) * is uniformly continuous on bounded sets. Note that a part of the results of this section can be found in the book [Cioranescu (1990)], but the proofs are different; note also that some notions are introduced differently in Cioranescu's book. Another application of convex analysis is emphasized in Section 3.8; here we apply the results on the existence, the uniqueness and the characterizations of optimal solutions of convex programs to the problem of the best approximation with elements of a convex subset of a normed space. In Section 3.9 it is shown that there exists a strong relationship between the wellposedness of the minimization problem min/(:r) s.t. x £ X, and the differentiability at 0 of the conjugate / * of / ; when / is convex these properties are equivalent. Using this result we establish a very interesting characterization of Chebyshev sets in Hilbert spaces and show that the class of weakly closed Chebyshev sets coincides with the class of closed convex sets in Hilbert spaces. Section 3.10 deals with sets of weak sharp minima, wellbehaved convex functions and the study of the existence of global error bounds for convex inequalities. These notions were studied separately for a time, but they are intimately related. As noted above, argmin / is a set of weak sharp minima for / exactly when / is wellconditioned with linear rate. But the inequality f{x) < 0 has a global error bound exactly when argmin[/] + is a set of weak sharp minima for [/] + := max(/, 0). We give several characterizations of the fact that argmin / is a set of weak sharp minima for / , one of them being the fact that up to a constant, the conjugate / * is sublinear on a neighborhood of the origin. Several numbers associated to a convex function are introduced which are related to the conditioning number from numerical analysis. Although the most part of the results from this section are stated in the literature in finite dimensional spaces, we present them in infinite dimensions. The last section of this book, Section 3.11, is dedicated to the study of monotone multifunctions on Banach spaces. We use in the presentation two recent articles of Simons. The proofs are quite technical and use the lower semicontinuous convex function XM associated to the multifunction M :
XX
Introduction
X =} X*, the minimax theorem and a few results of convex analysis. One obtains: two characterizations of maximal monotone multifunctions; the fact that the condition 0 6 int(domTi — domT2) is equivalent to other three conditions involving dom Tj and dom XT{ , and is sufficient for the maximal monotonicity of T\ + T2; dom T and Im T are convex if X is reflexive and T is maximal monotone; domT is convex if int(domT) ^ 0 and T is maximal monotone; T is locally bounded at XQ € (co(domX'))1 if T is a monotone multifunction; Rockafellar's theorem on the local boundedness of maximal monotone multifunctions. The result stating that for a maximal monotone multifunction T on the Banach space X for which dom T is convex the local boundedness of T at x 6 dom T implies that x € int (dom T) seems to be new. When applied to the subdifferential of a proper lower semicontinuous convex function / on the Banach space X, this result gives (for example): / is continuous &• d o m / is open <& df is locally bounded at any x £ d o m / . The exercises are intended to exemplify the topics treated in the book. Many exercises are auxiliary results spread in recent articles, although some of them are extracted from other books. Some exercises are important results which could be parts of textbooks, but which do not fit very well with the aim of the present book. Among them we mention Exercise 3.11 on the Moreau regularization.
Chapter 1
Preliminary Results on Functional Analysis
1.1
Preliminary Notions and Results
In this section we introduce several notions and results on separation of sets as well as some properties of topological vector spaces and locally convex spaces which are frequently used throughout the book, for easy reference. Let X be a real linear (vector) space. Throughout this work we shall use the following notation (x, y being elements of X): [x, y] := {(1 — A)x + Xy \ A e [0,1]}, [x,y[:= {(1  X)x + \y  A € [0,1[}, ]x,y[:= {(1  X)x + Xy  A 6]0,1[}, called closed, semiclosed and open segment, respectively. Note that [x,x] =]x,x[= {x}\ If 0 ^ A, B C X, the Minkowski sum of A and B is A + B := {a + b  a £ A,b £ B). Moreover, if x £ X, X £ R and 0 ^ T C R, then x + A := A + x := A + {x}, A • A = {70  A € A, a £ A} and XA — {A} • A. We shall consider that A + 0 = 0 and A • 0 = 0 • A = 0. A nonempty set A C X is starshaped at a (G A) if [a, x] C A for all 2: £ A; A is convex if [x, y] C A for all x,y £ A; Ais a, cone if 1+ • A C A (in particular 0 € A when A is a cone), R+ is the set of nonnegative reals; A is afflne if Ax + (1  X)y 6 A for all x,y e A and A e R; A is balanced if Ax € ^4 for all x e A and A £ [1,1]; A is symmetric if A =  A . Hence ^4 is balanced if and only if A is symmetric and starshaped at 0. We consider that the empty set is convex and afflne. It is easy to prove that A is afflne o 3 a 6 l , 3XQ linear subspace of X : A = a + X0 <=>Va€^4(3a€^4) : A — a is a linear subspace. When A is affine and a £ A, the linear space XQ := A — a is called the l
2
Preliminary
results on functional
analysis
linear space parallel to A; we consider that the dimension of A is dimXoNote that if (Aj)j € / is a family of affine, convex, balanced subsets or cones of X then f]ieIAi has the same property (Exercise!); we use the usual convention that HieO7^ = ^ Taking into account this remark, we can introduce the notions of affine, convex and conic hull of a set. So, the affine, convex and conic hull of the subset A of X are: aff A := [){V C X \ A C V, V affine}, co A := P){C C X  A C C, C convex}, cone A := f]{C CX\AcC,
C cone},
respectively. Of course, the linear hull of the subset A of X is the linear subspace spanned by A: linA := P  { ^ o C X \ A C X0, X0 linear subspace of X}. It is easy to verify (Exercise!) that aff A = { V "
XiXi n G N, (Ai)i< i < n C R, V "
X, = 1, (Xi)l
coA= {Yl^^i
n
S N, (Aj) C K+, (xt) C A, Yl"^
C A> , = 1
) '
cone A  {Ax  A > 0, x £ A} = 1+ • A, where N is the set of positive integers. Let us mention some properties of the affine and convex hulls. Consider Y another linear space, T : X > Y a linear operator, A,B c X, C C Y nonempty sets. Then: (i) aff(AxC) = aff Ax aff C; (ii) aff T(A) = T(aff A) (iii) aft(A + B) = aff A + aff B; (iv) Vo € A : aff A = a + aff (A  A) (v) aff (A  A) = UA>oA(A  A) if A is convex; (vi) aff A = linA if 0 e A (vii) co(A x C) = co A x coC; (viii) co T(A) = T(coA); (ix) co(A + 5 ) = co A + coB; (x) co(cone A) = cone(co A) (Exercises!). Let M C X be a linear subspace, and let A C X be nonempty; the algebraic interior of A with respect to M is a i n t M ^ : = {a 6 X  Vrr € M, 36 > 0, VA € [0,6] : a + Xxe It is clear that aint^f Ac
A}.
A and that M C aff (A  A) when aintM i ^ D 
Preliminary
notions and results
3
We distinguish two important cases: (i) M = X; in this case we write A instead of aintM A; A1 is called the algebraic interior of A, (ii) M = aS(A — A); in this case aintM A is denoted by M and is called the relative algebraic interior of A. Therefore a £ A1 ii and only if aff A = X and a G 14 (Exercise!). When the set A is convex we have (Exercise!) that: %
V a G i : \m(Aa)
=
cone(AA),
whence aff A = a + cone(A — A) for every a £ A (hence cone(A — A) is the linear subspace parallel to aff A), a e i ' ^ V i e l , 3 A > 0 : a + Xx £ A& cone(A  a) = X, and a G M o V a ; G A, 3 A > 0 : (1 + X)a  Xx G A <$ cone(yl — a) = cone(A — A) <=> cone(A — a) is a linear subspace O M
n(C — a) is a linear subspace;
(1.1)
the dimension of the convex set A is dim A := dim(aff A) = dim (cone(A — A)). Some properties of the algebraic interior are listed below. Let 0 ^ A,B CX, i s l a n d A G E \ {0}; then: (i) *(x + A) =x + iA; (ii) ^XA) = X • % (iii) A + Bi C (A + B)'; (iv) A + Bl = {A + BY if Bi = B; (v) iA + iB c 1{A + B); (vi) *(A. + B) = A + *B if A, B are convex, VI / 0 and *B T^ 0; (vii) M 7^ 0 if A is convex and dim A < 00; (viii) if A is convex then [a, x[ C VI for all a G M. and i £ A In the sequel the results will be established for real topological vector spaces (tvs for short) or real locally convex spaces (lcs for short). When X is a tvs it is wellknown that the class Nx of closed and balanced neighborhoods of 0 G X is a base of neighborhoods of 0; when X is a lcs then the class J^cx of the closed, convex and balanced neighborhoods of 0 G X is also a base of neighborhoods of 0. If X, Y are real linear spaces, we denote by L(X, Y) the real linear space of linear operators from X into Y. The space L(X,R) is denoted by X' and is called the algebraic dual of X; an element of X' is called a linear functional. When X, Y are topological vector spaces, we denote by &{X, Y)
4
Preliminary
results on functional
analysis
the linear space of continuous linear operators from X into Y; the space £ ( X , E) is denoted by X* and is called the topological dual of X. Let now A be an absorbing subset of the linear space X, i.e. 0 £ A1; the Minkowski gauge of A is defined by pA : X > M,
^ ( a : ) := inf{A > 0  a; £
\A).
It is obvious that PA = P[o,i].4 Moreover, if A, B C X and C C Y are absorbing and starshaped sets, where Y is another linear space, one has (Exercise!): {x 6 X  p A (a) < 1} C A C {x € X  P A ( Z ) < 1}, VxGX
: pAns(a;) = max {PA (a;), p B (a;)},
Va; 6 X, 2/ € Y : PAxc(x,y)
=
max{pA(x),pc(y)}
Other useful properties of the Minkowski gauge are mentioned in the next result. Recall that p : X » M is sublinear if p(0) = 0, p(x + y) < p(x) +p(y) [with the convention (+oo) + (—oo) = +oo] and p(Xx) = Xp(x) for all x, y £ X, X € P := ]0, oo[; p is a seminorm if p is a finite, sublinear and even [i.e. p(—x) = p(x) for every x € X] function. Proposition 1.1.1 space X.
Let A be a convex and absorbing subset of the linear
(i) Then PA is finite, sublinear and A1 = {x E X \ PA{X) < 1}; furthermore, if A is symmetric then PA is a seminorm, too. (ii) Assume, moreover, that X is a topological vector space and V is a neighborhood of Q £ X. Then pv is continuous and intV = {xeX\pv{x)
< 1},
clV = {xeX
\pv{x)
< 1}.
The following result will be useful, too. Theorem 1.1.2 X. Then
Let C be a convex subset of the topological vector space
(i) clC is convex; (ii)
if a £ int C and x £ cl C, then [a, x[ C int C;
(iii) intC is convex; (iv)
if int C ^ 0 then cl(int C) = cl C and int(cl C) = int C;
(v)
if int C ^ 0 then Cl = int C.
Preliminary
notions and results
5
Using the Minkowski gauge one obtains the geometrical versions of the HahnBanach theorem, i.e. separation theorems. In the sequel we give several separation theorems for convex subsets of topological vector spaces or locally convex space. Theorem 1.1.3 (Eidelheit) Let A and B be two nonempty convex subsets of the topological vector space X. If int A ^ 0 and B flint A = 0 then there exist x* e X* \ {0} and a £ E such that VxeA,\/yeB
: (x,x*) < a < (y,x*),
(1.2)
or equivalently, supa;*(yl) < inf x*(B). The separation condition (1.2) can be given in a different manner. Let x* e X* \ {0} and a £ t Consider the sets H^a:={x£X\(x,x*)
(x,x*)
Hx*,a := {x e X \ (x,x*) = a } ; similarly one defines H^. and H>.a. All these sets are convex and nonempty. The set Hx*^a is called a closed hyperplane, H<. a and H>* a are called open halfspaces, while H. a and H~.a are called closed halfspaces. Hx, a are open sets; moreover, c\H< a = H% a and {Hf,a)1 = int H~. >a = H< a (Exercises!). Theorem 1.1.3 states the existence of x* G X* \ {0} and a G K such that A c H, a and B C H, a; in this situation we say that Hx*tC[ separates A and B; the separation is proper when AUB <£. Hx*i<x and the separation is strict when A n Hx*^a = 0 or B n Hx*ia = 0. When x0 6 A and ffx*,a separates A and {xo} we say that ifx*,a is a supporting hyperplane of A at xo; XQ is called a support point and x* is called a support functional. Therefore x* € X* \ {0} is a support functional for A if and only if x* attains its supremum on A. Generally, Hx*
6
Preliminary
results on functional
analysis
In the case of locally convex spaces one has the following result for the separation of two sets. T h e o r e m 1.1.5 Let X be a locally convex space and A,B C X be two nonempty convex sets. If A is closed, B is compact and An B = 0, then there exist x* G X* \ {0} and ct\,a.2 G K such that Vx G A, Vy e B : (x,x*) < ax < a2 < (y,x*), or equivalently, sup x* (A) < inf x* (B). The two preceding results can be stated in a more general setting. T h e o r e m 1.1.6 Let A and B be two nonempty convex subsets of the topological vector space X such that int(A — B) ^ 0. Then 0 £ i n t ( A  £ ) < £ • 3x* G X * \ { 0 } : supa;*(yl) < inf x*{B). T h e o r e m 1.1.7 Let A and B be two nonempty convex subsets of the locally convex space X. Then Q$d{AB)&3x*
eX* : supx*{A) < inf x*(B).
The preceding theorem shows the usefulness of having criteria for the closedness of the difference (or sum) of two convex sets. In order to give such a criterion, let A be a nonempty convex subset of the topological vector space X. The recession cone of A is defined by vecA := {u G X  Vo G A : a + uGA}. It is easy to show that rec A is a convex cone and A + rec A = A. When A is a closed convex set we have that vecA = f]t>ot(Aa)
(1.3)
for every a 6 A. In this case it is obvious that rec A is a closed convex cone which is also denoted by Aoo. It is easy to see that when X is a finite dimensional separated topological vector space and A is a closed convex nonempty subset of X, A^ = {0} if, and only if, A is bounded. This is no longer true when dimX ~ oo. Example 1.1.1 Let X := £p with p £ [l,oo] and A := {(x„)„>i G £p  \xn\ < n Vn G N}. It is obvious that A is a closed convex set which is not bounded because nen G A for every n G N, but A^ = {0}. Indeed, if
Preliminary
notions and results
7
u = (un) G Aoo then tu G A for every t > 0; so \tun\ < n for every t > 0, whence u„ = 0. Hence u = 0. The following famous theorem was obtained in [Dieudonne (1966)]. Theorem 1.1.8 (Dieudonne) Let A, B be nonempty closed convex subsets of the locally convex space X. If A or B is locally compact and A^ n J3oo is a linear subspace, then A — B is closed. In convex analysis (as well as in functional analysis) one often uses the following sets associated to a nonempty subset A of the locally convex space X: A°
{x*
A+
{x* ex* \Vxe A {x* ex* IVze A
A^
eX*\Vx€A
(x,x*)>l}, (x,x*)>0}, (x,x*) = 0},
called the polar, the dual cone and the orthogonal space of A, respectively. One verifies easily that A° is a w*closed convex set which contains 0, that A+ is a w*closed convex cone, and, finally, that A1 is a u;*closed linear subspace of X*, where w* = a(X*,X) is the weak* topology on X*. Similarly, for 0 ^ B C X* we define the polar, the dual cone and the orthogonal space; for example, the polar of B is B° := {x G X  Vx* G B : (x,x*) >  1 } . It is obvious that B° is a closed convex set containing 0, B+ is a closed convex cone, and, finally, B1 is a closed linear subspace. One verifies easily that when A,BcX and A G P we have: (i) A° is convex and 0 G A°; (ii) A U {0} C (A°)° =: A°°; (hi) AcB => A° D B°; (iv) (AUB)° = A°nB°; (v) if 0 € AnB then (A + B)+ = (AUB)+ =A+n B+; (vi) (\A)° = {A°; (vii) A° = A+ if A is a cone, and A° = A+ = A1 if A is a linear subspace; (viii) (T(A))° = ( T * ) " 1 ^ 0 ) , if T G &(X,Y), where Y is another locally convex space. A very useful result is the bipolar's theorem. Let X be a topological vector space and A C X; the set coA := cl(coA) is called the closed convex hull of the set A; it is the smallest closed convex set containing A. Similarly, coneA :— cl(coneyl) is called the closed conic hull of A. Theorem 1.1.9
(bipolar) Let A be a nonempty subset of the locally con
Preliminary
8
vex space X.
results on functional
analysis
Then
A00 = co(A U {0}),
A + + = cone(co A),
A±JL
= cl(lin A).
It follows that for the nonempty subset A of the lcs X one has: (a) A°° = A o A is closed, convex and 0 € A; (b) A + + = A <£> A is a closed convex cone; (c) A"1 = A <=> A is a closed linear subspace. Another famous result is the following. Theorem 1.1.10 (AlaogluBourbaki) Let X be a locally convex space and U C X be a neighborhood of the origin. Then U° is w*compact. We finish this preliminary section with some notions and results concerning completeness and metrizability of topological vector spaces. The subset A of the topological vector space X is complete (quasicomplete) if every (bounded) Cauchy net (xi)i^i C A is convergent to an element x £ A. Of course, any complete set is closed and any closed subset of a complete set is complete (Exercise!). Recall that the topological space (X, T) is first countable if every element of X has a (at most) countable base of neighborhoods. Note that a subset A of a first countable tvs X is complete if and only if every Cauchy sequence of A is convergent to an element of A; in particular, a first countable tvs is complete if and only if it is quasicomplete (Exercise!). We shall use several times the hypothesis that a certain topological vector space is first countable. The next result refers to the first countability of locally convex spaces. Proposition 1.1.11
Let (X,T) be a locally convex space. Then
(i) (X, T) is first countable O 3 7 a (at most) countable family of seminorms on X such that r = rg> O r is semimetrizable, i.e. there exists a semimetric d on X such that T = T^; the semimetric d may be chosen to be invariant to translations (i.e. d(x + z,y + z) = d(x,y) for allx,y,z € X). (ii) (X, T) is separated and first countable if and only if T is metrizable, i.e. there exists a metric d on X such that r = T&. Note that when the topology r of a locally convex space X coincides with the topology T<J determined by a semimetric d invariant to translations, the completeness with respect to r and d are equivalent. One says that the locally convex space X is a Frechet space if X is complete and metrizable. It is obvious that every closed linear subspace of a Frechet space is Frechet,
Closedness and inferiority
notions
9
too. One says that the topological vector space X is barreled if every absorbing, convex and closed subset of X is a neighborhood of 0 £ X. As application of the Baire theorem one obtains that every Frechet space is barreled. It is wellknown that in a finite dimensional separated topological vector space any convex and absorbing set is a neighborhood of the origin.
1.2
Closedness and Interiority Notions
Consider X a real topological vector space. We say that the series Yln>i Xn is convergent (resp. Cauchy) if the sequence (£n)n£N is convergent (resp. Cauchy), where Sn := Y^k=i xk f° r every n £ N; of course, any convergent series is Cauchy. Let A C X; by a convex series with elements of A we mean a series of the form Ylm>i ^mXm with (Am) C IR+, {xm) C A and Y,m>i ^™ = ^ if, furthermore, the sequence (xm) is bounded we speak about a bconvex series. We say that A is csclosed if any convergent convex series with elements of A has its sum in A;* A is cscomplete if any Cauchy convex series with elements of A is convergent and its sum is in A. Similarly, the set A is called ideally convex if any convergent bconvex series with elements of A has its sum in A and A is bcscomplete if any Cauchy bconvex series with elements of A is convergent and its sum is in A. It is obvious that any csclosed set is ideally convex, every ideally convex set is convex, every cscomplete set is csclosed and every complete convex set is cscomplete; if X is complete, then A C X is cscomplete (bcscomplete) if and only if A is csclosed (ideally convex). If Xo is a linear subspace of X and A is a csclosed (ideally convex) subset of X, then Xo C\ A is a csclosed (ideally convex) subset of Xo (endowed with the induced topology). Moreover, if A C X and B C Y are nonempty, then A x B is csclosed (cscomplete, ideally convex, bcscomplete) if and only if A and B are csclosed (cscomplete, ideally convex, bcscomplete). If X is first countable, a linear subspace XQ of X is csclosed (cscomplete) if and only if it is closed (complete); moreover, if X is a locally convex space, X0 is closed if and only if Xo is ideally convex. Note also that T(A) is csclosed (cscomplete, ideally convex, bcscomplete) if A C X is csclosed (cscomplete, ideally convex,
'Because X may not be separated, in fact we ask that every limit of (Sn) is in A.
Preliminary
10
results on functional
analysis
bcscomplete) and T : X > Y is an isomorphism of topological vector spaces (Exercise!), Y being another tvs. We consider that the empty set is convex, ideally convex, bcscomplete, cscomplete and csclosed. It is worth to point out that when X is a locally convex space, every bconvex series with elements of X is Cauchy (Exercise!). The class of csclosed sets (and consequently that of ideally convex sets) is larger than the class of closed convex sets, as the next result shows. Proposition 1.2.1
Let A C X be a nonempty convex set.
(i) / / A is closed or open then A is csclosed. (ii) If X is separated and dim A < oo then A is csclosed. Proof, (i) Let £ n >! A n i n be a convergent convex series with elements of A; denote by x its sum. Suppose that A is closed and fix a £ A. Then, for every n G N we have that X)fc=i ^kXk + (l — Yl'kLn+i ^*) a S A. Taking the limit for n > oo, we obtain that x G cl A = A. Suppose now that A is open. Assume that x $. A. By Theorem 1.1.3, there exists x* G X* such that (a  x, x") > 0 for every a G A. In particular (xn — x,x*) > 0 for every n G N. Multiplying by \ n > 0 and adding for n G N we get (since A„ > 0 for some n) the contradiction
0 < ^ n > 1 A„ (xn  x, x*) = (X]„>i XnXn' x*)~ \52n>i
An
) ^' x ")
=
°'
Therefore x G A. So in both cases A is csclosed. (ii) We prove the statement by mathematical induction on n := dim A. If n = 0 A reduces to a point; it is obvious that A is csclosed in this case. Suppose that the statement is true if dim A < n G N U {0} and show it for dim A = n + 1. Without any loss of generality we suppose that 0 G A; then Xo := aff A is a linear subspace with dimX 0 = n + 1. Because on a finite dimensional linear space there exists a unique separated linear topology and in such spaces the interior and the algebraic interior coincide for convex sets, we have that iA — intx 0 A ^ 0. Let J2n>i ^n%n be a convergent convex series with elements of A and sum x. Assume that x fi A. Because A is convex, the set P :— {n G N  A„ > 0} is infinite, and so we may assume that P — N. Applying now Theorem 1.1.3 in XQ, there exists XQ G XQ \ {0} such that (x — x, XQ) > 0 for every x G A. But X)n>i An (xn —X,XQ) = 0. Since {xn —X,XQ) > 0 and Xn > 0 for every n, we obtain that (x n ) C AQ := A(~)Hx*t\, where A :— (X,XQ). Since
Closedness and inferiority
notions
11
dimAo < dimHx*t\ = n, from the induction hypothesis we obtain the contradiction x e A0 c A. Therefore x € A. The proof is complete. • Other properties of csclosed and ideally convex sets are given in the following result. Proposition 1.2.2 (i) If Ai C X is csclosed (resp. ideally convex) for every i £ I then P  i e / A{ is csclosed (resp. ideally convex). (ii) / / Xi is a topological vector space and Ai C Xi is csclosed (resp. ideally convex) for every i 6 I, then Ylizi ^i is csclosed (resp. ideally convex) in Yliei Xi (which is endowed with the product topology). Proof. The proof of (i) is immediate, while for (ii) one must take into account that a sequence (x n ) n € N C X := YlieI Xi converges t o x G X (resp. is bounded) if and only if (xln) converges to xl in Xi (resp. is bounded) for every i £ I. • We say that the subset C of Y is lower csclosed (Icsclosed for short) if there exist a Frechet space X and a csclosed subset B of X x Y such that C = Pry (B). Similarly, the subset C of Y is lower ideally convex (liconvex for short) if there exist a Frechet space X and an ideally convex subset B of X xY such that C = Pry (B). It is obvious that any csclosed (resp. ideally convex) set is Icsclosed (resp. liconvex), any Icsclosed set is liconvex and any liconvex set is convex, but the converse implications are not true, generally. Note also that T(A) is Icsclosed (resp. liconvex) if A C X is Icsclosed (resp. liconvex) and T : X > Y is an isomorphism of topological vector spaces (Exercise!). The classes of Icsclosed and liconvex sets have very good stability properties as the following results show. We give only the proofs for the "liconvex" case, that for the "Icsclosed" case being similar. Proposition 1.2.3 Suppose that Y is a Frechet space and C C F x Z is a liconvex (Icsclosed) set. Then Prz(C) is a liconvex (Icsclosed) set. Proof. By hypothesis, there exists a Frechet space X and an ideally convex subset B C X x (Y x Z) such that C = P r y x z ( B )  Since I x 7 i s a Frechet space and Prz(C) = Prz(B), we have that Prz(C) is a liconvex subset of Z. • Proposition 1.2.4
Let I be an at most countable nonempty set.
(i) If Ci C Y is liconvex (Icsclosed) for every i £ I then (~)ieI Ci is liconvex (Icsclosed).
12
Preliminary
results on functional
analysis
(ii) / / Y{ is a topological vector space and Ci C Yi is liconvex (Icsclosed) for every i 6 I, then Yli&iCi 8S liconvex (Icsclosed) in W^jYi. Proof, (i) For each i £ I there exist X» a Frechet space and an ideally convex set Bt C X» xY such that d = PrY(Bi). The space X := fliei %i is a Frechet space as the product of an at most countable family of Frechet spaces. Let Bi •= {{(xj)jei,y)
e X x Y  (xi,y) £ Bi} .
Then Bi is an ideally convex set by Proposition 1.2.2(h). It follows that B := f]ieI Bi is ideally convex by Proposition 1.2.2(i). Since PrY(B) = Hie/ Ci, ^ f ° u o w s that flie/ Ci is liconvex. (ii) For each i £ I there exist Xj a Frechet space and an ideally convex set Bi cXiX Yi such that d = Pry ; (Bi). By Proposition 1.2.2(h), ]JieI Bt is an ideally convex subset of r i i e / ( ^ j x ^ )  The space X := riig/ ^i 1S a Frechet space; let Y := FJie/ ^i Consider the set B:= {((xi)ieI,(yi)ieI)
£XxY\
(xi,yi)
£ Bi Vt 6 / } .
Since T : I l i e / ( X * x y4)  • X x y , T ( ( x i , y i ) i 6 / ) := ((xi)ieI,(yi)iei) is an isomorphism of topological vector spaces, B — T (Y\ieI Bi) is ideally convex. As C := riig/ C* = ^VY(B), C is liconvex. D Before stating other properties of liconvex and lcsclosed sets, let us define some notions and notations related to multifunctions. Let E,F be two nonempty sets; a mapping ft : E —• 2F is called a multifunction, and it will be denoted by ft : E =X F. The set domft := {x £ E \ 5l(x) £ 0} is called the d o m a i n of the multifunction 51; the image of 51 is Imft := UigB^( a ; )i the g r a p h of 51 is the set grft :— {(x,y) \ y £ 5l(x)} C E x F; the inverse of the multifunction 51 is the multifunction ft"1 : F =} E defined by ft_1(?/) := {x £ E  y £ %(x)}. Therefore domft 1 = Imft, Imft 1 = domft and grft 1 = {(y,x)  (x,y) £ grft}. Frequently we shall identify a multifunction with its graph. For A C E and B C F one defines 51(A) := \JxeA5l(x) and ft"1^) := Uj, e B#  1 (2/); i n particular Imft = ft(.E) and domft = ft_1(F). If 8 : F =4 G is another multifunction, then the composition of the multifunctions S and 51 is the multifunction Soft :E=lG, (Soft) (a) := \Jy€Ci{x) S(y). If ft, S : E =} F and F is a linear space, the sum of ft and S is the multifunction ft + S : E =t F, (ft + S)(x) :=5i(x) + S(x).
Closedness and interiority notions
13
Let now K : X r{ 7 ; we say that 51 is convex (closed, ideally convex, bcscomplete, csconvex, cscomplete, liconvex, lcsclosed) if its graph is a convex (closed, ideally convex, bcscomplete, csconvex, cscomplete, liconvex, lcsclosed) subset of X x Y. Note that 31 is convex if and only if V i , a ' eX,
VAe [0,1] : \3l(x) + (l\)3l(x')c3l(\x
Proposition 1.2.5
+
Let A, B C X, 31, S : X =4 Y and7:Y
{l\)x'). =} Z.
(i) If X is a Frechet space and A, 31 are liconvex (resp. lcsclosed), then 01(A) is liconvex (resp. lcsclosed). (ii) If X is a Frechet space and A, B are liconvex (resp. lcsclosed), then A + B is liconvex (resp. lcsclosed). (hi) If Y is a Frechet space and 31, T are liconvex (resp. lcsclosed), then T o 31 is liconvex (resp. lcsclosed). (iv) IfY is a Frechet space and 31,$ are liconvex (resp. lcsclosed), then 31 + § is liconvex (resp. lcsclosed). Proof,
(i) We have that %{A)
=PrY((AxY)ngrJl).
Using successively Propositions 1.2.4(h), 1.2.4(i) and 1.2.3, it follows that 31(A) is liconvex. (ii) Let T : X xX —> X, T(x,y) := x + y. Since T is a continuous linear operator, grT is a closed linear subspace; in particular T is a liconvex multifunction. Since Ax B is liconvex, by (i) A + B = T(A x B) is a liconvex set. (hi) We have that gr(To3?) = P r X x 4 ( g r : R x Z) n (X x grT)). The conclusion follows from Propositions 1.2.4(h), 1.2.4(i) and 1.2.3. (iv) The sets T~{(x,z,y,y')\x€X,
y,y' eY,
z = y + y'} C (X x Y) x (Y x Y),
A:= {(x,z,y,y') \ (x,y) G g r # , y',z € Y} and B := {(x,z,y,y') \ (x,y') G grS, y,z € Y} are liconvex sets (the first being a closed linear subspace). Therefore gr(# + S) = PrXxY (THAHB) is liconvex. •
14
Preliminary
results on functional
analysis
Let Y be another topological vector space and A C X xY; we introduce the conditions (Ha;) and (Hwa;) below, where x refers to the component xeX: (Ha:) If the sequences ((xn, yn)) C A and (A„)„>! C ffi+ are such that E „ > i K = 1, E n > i ^nVn^as sum y and X)„>i A«a;n is Cauchy, then the series X^n>i ^nxn *s convergent and its sum x G X verifies {x,y)EA. (Hwi) If the sequences ({xn,yn))n>1 C A and (A n ) n >i C K+ are such that ((xn,yn)) is bounded, E n > x An = 1, £ „ > i A„2/n has sum y and ^ r a > 1 A n x n is Cauchy, then the series X^ n>1 ^« x « ^s convergent and its sum x € X verifies (x, y) 6 A. Of course, when X is a locally convex space, deleting "^2n>1 Ana;n is Cauchy" in condition (Hwa;) one obtains an equivalent statement. In the next result we mention the relationships among conditions (Ha;), (Hwi), ideal convexity, csclosedness, cscompleteness and convexity. The proof being very easy we omit it. Proposition 1.2.6 sets.
Let A C X xY
and BcXxYxZbe
nonempty
(i) Assume that Y is complete. Then B satisfies (H.(x,y)) if and only if B satisfies (Ha;); B satisfies (Hwa;) if and only if B satisfies (Hw(x,j/)). (ii) Assume that X is complete. Then A satisfies (Ha;) if and only if A is csclosed; A satisfies (Hwa;) if and only if A is ideally convex. (iii) Assume that Y is complete. Then A satisfies (Ha;) if and only if A is cscomplete; A satisfies (Hwa;) if and only if A is bcscomplete. (iv) If A satisfies (Ha;) then A satisfies (Hwx); if A satisfies (Hwa;) then A is convex. (v) Assume that X is a locally convex space and Prx(A) is bounded. If A satisfies (Ha;) then Pry(^4) is csclosed; if A satisfies (Hwa;) then Pry (A) is ideally convex. We define now several interiority notions. Let 0 / A c X. We denote by rint A the interior of A with respect to aff A, i.e. rint A := intafj A A. Of
Closedness and inferiority
notions
15
course, rint A C %A. Consider also the sets c
. lA 1
.
riA:=  i6
.
{
if aff A is a closed set, otherwise,
rint A 0
 lA
if aff A is a closed set, otherwise,
if XQ is a barreled linear subspace, otherwise,
where X0 = \in(A — a) for some (every) a G A; XQ is the linear subspace, parallel to aff A. In the sequel, in this section, A C X is a nonempty convex set. Taking into account the characterization (1.1) of *A, we obtain that x € tcA <£> cone(A — x) is a closed linear subspace of X & M
X(A — x) is a closed linear subspace of X,
and x £
A £> cone(^4 — a;) is a barreled linear subspace of X <£> M
n(A — x) is a barreled linear subspace of X.
If X is a Frechet space and aff A is closed then %CA = lbA, but it is possible to have ibA ^ 0 and icA = 0 (if aff A is not closed). The quasi relative interior of A is the set qri A := {x € A  cone(yl — z) is a linear subspace of X}. Taking into account that in a finite dimensional separated topological vector space the closure of a convex cone C is a linear subspace if and only if C is a linear subspace (Exercise!), it follows that in this case qri A = %A = ic A = ibA Below we collect several properties of the quasi relative interior. Proposition 1.2.7 L{X,Y). Then:
Let A C X be a nonempty convex set and T £
a £ qri A •& a G A and cone(A — a) = cone(A  A) O a E i and a  A C cone(yi  a), i
A C qriA = AnqriA,
V a G q r i A , Vx £ A : [a,z[CqriA,
(1.4) (1.5)
Preliminary
16
results on functional
analysis
and T(qriA) C qriT(A). In particular qriA is a convex set. Assume that qriA ^ 0; then qriA = A and *(T(A)) C T(qriA) C qriT(A) C T(A) c T(qriA).
(1.6)
Moreover, if Y is separated and finite dimensional then T(qiiA)=i(T(A)).
(1.7)
Proof. The first equivalence in Eq. (1.4) is immediate from the definition of qriA. Of course, cone (A — a) = cone (A — A) implies that a — A C cone(A  a). Conversely, if a — A C cone(A  a) then cone(A  a) = cone(a — A) C cone (A — a), which shows that cone (A — a) is a linear subspace. Therefore Eq. (1.4) holds. If a £ lA, from Eq. (1.1) we have that a £ A and cone(A — a) is a linear subspace, and so cone(A — a) is a linear subspace, too. Hence a € qriA. The equality qriA = A n qriA follows immediately from the relation coneC = cone C, valid for every nonempty subset C of X. Let a € qri A, x e A, A € [0,1[ and a\ :— (1 — X)a + Xx. Then AADAa
A
= ( l  A)(A  a) + A(A  z) D (1  A)(A  a),
and so, taking into account Eq. (1.4), we have cone(A  A) D cone(A  a\) D cone ((1  A)(A  a)) = cone (A — a) = cone (A — A). Therefore cone(A — A) = cone(A — a^) Since a\ € A, from Eq. (1.4) we obtain that a\ G qriA. The proof of Eq. (1.5) is complete. Let a £ qriA; then, by Eq. (1.4), a  A C cone(A — a), and so Ta  T{A) = T(aA)cT = cone (T(A) 
(cone(A  a)) C T (cone(A  a)) Ta),
which shows that Ta € qriT(A). Assume now that qriA ^ 0 and fix a £ qriA. It is sufficient to show Eq. (1.6); then the equality qriA = A follows immediately from the last inclusion in Eq. (1.6) for T = Idx and from Eq. (1.5). Let y e i (T'(A)); using Eq. (1.1), there exists A > 0 such that (1 + X)y XT a € T(A). So, (1 + X)y  XTa = Tx for some x £ A. It follows that y = Txx, where xx ~ (1 + A) _1 (Aa + x). But, using Eq. (1.4), x\ 6 qriA, and so y 6 T(qriA).
Closedness and inferiority
notions
17
The second inclusion in Eq. (1.6) was already proved, while the third is obvious. So, let y G T(A); there exists x G A such that y = Tx. By Eq. (1.5), (1X)a + Xx G qriAfor A G]0,1[, and so (lX)Ta + Xy G T(qriA) for A G ]0,1[. Taking the limit for A —> 1, we obtain that y G T(qri A). When Y is separated and finite dimensional we have (as already observed) that i(T(A)) = qriT(A); then Eq. (1.7) follows immediately from Eq. (1.6). • The notion of quasi relative interior is related to that of united sets. Let X be a locally convex space and A,B C X he nonempty convex sets; we say that A and B are united if they cannot be properly separated, i.e. if every closed hyperplane which separates A and B contains both of them. The next result is related to the above notions. Proposition 1.2.8 Let X be a locally convex space, A,BcX empty convex sets and x G X. (i) A and B are united O cone(AB) is a linear subspace.
be non
is a linear subspace •& (A — B)~
(ii) Assume that cone (A — x) is a linear subspace. Then x G c\A. Moreover, i/aff A is closed and rint A ^ 0 then ~x G rint A. Proof, (i) Assume that A and B are united but C := cone(A — B) is not a linear subspace. Then there exists XQ G (—C) \ C. By Theorem 1.1.5 there exists x* G X* such that {x0, x*) < 0 < (z, x*) for every z G C. Then 0 < (x  y,x*) for all x G A, y G B, and so (x,x*) < X < (y,x*) for all x G A, y G B, for some A G M. Therefore Hx* t\ separates A and B. It follows that (x,x*) = X = (y,x*) for all x G A, y G B, and so 0 < (z,x*) for every z G C. Thus we have the contradiction (x0,x*) = 0. Therefore cone (A — B) is a linear subspace. Let C := cone (A — B) be a linear subspace and (x,x*) < X < (y, x") for all x G A, y G B, for some x* G X* and A e i . Then 0 < (z,x*) for every z G A — B, and so 0 < (z, x*) for z € C. Then, since C is a linear subspace, 0 = (z,x*) for every z G C which implies immediately that Hx*,\ contains A and B. Therefore A and B are united. The other equivalence is an immediate consequence of the bipolar theorem (Theorem 1.1.9). (ii) By (i) we have that {x} and A are united. Assuming that x $ cl A, using again Theorem 1.1.5, we obtain that {x} and A can be properly separated. This contradiction proves that x E. cl A.
18
Preliminary
results on functional
analysis
Suppose now that aSA is closed and rint A / 0. Without loss of generality we suppose that x = 0. By what was shown above we have that x = 0 € clA C cl(affA) = aff A. Thus X0 := aS A is a linear space. Assuming now that 0 £ intx 0 A ^ 0, we obtain that {x} and A can be properly separated (in XQ, and therefore in X) using Theorem 1.1.3. This contradiction proves that x € rint A. • From the preceding proposition we obtain that when X is a locally convex space and A C X is a nonempty convex set, the quasi relative interior of A is given by the formula
qri A = A (~1 {x € X \ {x} and A are united}. The next result shows that the class of convex sets with nonempty quasi relative interior is large enough. Proposition 1.2.9 Let X be a first countable separable locally convex space and A C X be a nonempty cscomplete set. Then qri A ^ 0. Proof. Since X is first countable, by Proposition 1.1.11, the topology of X is determined by a countable family 7 = {pn \ n 6 N} of seminorms. Without loss of generality we suppose that pn < pn+i for every n € N. Since Ty is semimetrizable, the set A is separable, too. Let A0 = {xn  n € N} C A be such that A C cl A0. Consider j3n € ]0,2~n] such that /3npn(xn) < 2 _ n . The series ^ n > 1 Pnxn is Cauchy (since for m > n and p e N w e have that PniT^Z^Xk)
< ET=mPkPn(xk)
< ET=Z^Pk(xk)
< 2"™+!). Taking
^n '•= (Z)n>i Pn) 1Pn, ]C n >i ^ n 1 " *s a Cauchy convex series with elements of A. Because A is cscomplete, the series ^ n > 1 Xnxn is convergent and its sum x € A. Suppose that x ^ qri A. Then there exists XQ € (—C)\C, where C := cb~m(A — x). Using Theorem 1.1.5, there exists x* € X* such that (xo,x*) < 0 < {z,x*) for all z € C. In particular ( *) > 0 for — = every x £ A. But E n > i ^» ( ^ ^^o) 0 Since {xn — X,XQ) > 0 and A„ > 0 for every n, we obtain that (x  x, x*) = 0 for every x € AQ. Since AQ is dense in A and x* is continuous, we have that {x — x, x*) = 0 for every x € A, and so (z,a;*) = 0 for every z E C. Thus we get the contradiction (x0,x*) = 0. Therefore qri A ^ 0. D
Open mapping
1.3
theorems
19
Open Mapping Theorems
Throughout this section the spaces X, Y are topological vector spaces if not stated otherwise. We begin with some auxiliary results. Lemma 1.3.1 Let X,Y be first countable topological vector spaces, 31 : X =$ Y be a multifunction and XQ G X. Suppose that grIR satisfies condition (Hwa;). Then
where Nxl^o) ** the class of all neighborhoods of XQ. Proof. Let y0 G C\u£7fx(x0)int (clft(t/)). Replacing grft by gv% {x0,y0) if necessary, we may assume that (xo,yo) = (0,0). Let U G NxSince X is first countable, there exists a base (Un)n>i C N x of neighborhoods of 0 such that Un + Un C Uni for every n > 1, where [7o := ?/• Because 0 G f " ) ^ ^ int (cl#([/)), there exists (V n ) n >i C Ny such that V„ C int (cl3i([7n)) for every n > 1. Since Y is first countable, we may suppose that (Vn)n>i is a base of neighborhoods of 0 G Y and, moreover, y n + i + Vn+i C KJ for every n > 1. Consider j / ' G int (clCR.(C/i)); there exists fi G]0,1[ such that y := (1 — lJ)~1y' £ cl3?([/i). There exists (£1,2/1) e g r $ such that x\ G C/i and 2/  2/i G M^2 It follows that /x 1 (y  j/i) G V2 C cl!R(t/2) There exists (^2,2/2) G grR such that xi G L/2 and /x_1(2/ _ 2/i) — 2/2 G M^3 It follows that /U~2y — fJ~2yi — H~lyi 6 V3 C c\"R{Uz). Continuing in this way we find ({xm,ym))m>1 C grR such that xm e Um and n~m+xy  n~m+1yi m+2 M 2/2  •••  ym G fiVm+i for every m > 1. Therefore u m := y 2/1/^2/2 At",_12/m G ^ m K i + i C V m + i for every m > 1. Moreover, m_1 M 2/m = vmi vm e \im~xVm  fimVm+1, and so y m £ Vro  fiVm+1 C Vm + Vm C V m _i for m > 2. It follows that (x m ) m >i > 0, (y m ) m >i > 0 and (u m ) m >i > 0, whence J2m>i nm~lym has sum y. Taking Am := ( l  / i ) ^ m _ 1 , we have that (A m ) m >i C R)., E m > i A™ = *> E m > i Am2/m h a s sum (1  ju)y = y', the sequence ((x TO ,y m )) is bounded (being convergent) and, because £mtf„ + i Ama;TO € C/„+i + Un+2 + ••• + Un+P C f7„ +1 + Un+1 C f/n for every n > 1, the series 5Z m > 1 A m x m is Cauchy. By hypothesis there exists x' £ X, sum of the series ]Cm>i Am#m> such that (x',y') £ g r X Let x := (1  n)~lx' We have that YZi=i Hm~lxm G t/i + U2 + • • • + Un C t/i I Ui C t/. Since U is closed we have that x E.U, and so a;' G (1 — n)U C
20
Preliminary
results on functional
analysis
U. Thus y' £ "R{U). Therefore int (cl3l(l7i)) C R(U). 0 € int%(U). The proof is complete.
In particular •
Note that we didn't use the fact that X or Y is separated. Observe also that it is no need that x0 £ dom"R when y0 £ f\ue^x(xo) * nt ( c ^ ( ^ 0 ) ' but, necessarily, xo £ cl(domD?) and yo £ int(ImlR) in our conditions. Note also that condition (Hwa;) may be weakened by asking that the sequence ((xm,ym)) C A be convergent instead of being bounded. In the case of normed spaces one has the following variant of the result in Lemma 1.3.1. Having the normed vector space (nvs) (X, ), we denote by Ux the closed unit ball {x £ X \ \\x\\ < 1}, by Bx the open unit ball {x € X  a; < 1} and by Sx the unit sphere {x 6 X  a; = 1}. Lemma 1.3.2 Let (X,  • ), (Y,  • ) be two normed linear spaces and let 31: X =4 Y be a multifunction. Suppose that condition (Hwa;) holds and (x0,y0) &X xY. If yo + nUy C cl (^(XQ +
pUx)),
where rj,p > 0, then yo + nBy C%(x0 + pUx). Proof. We may take (a;o,2/o) = (0,0). One follows the same argument as in the proof of the preceding lemma, but with Un := pUx and Vn := nBy for n > 1. Consider y' 6 nBy and take p, £]0,1[ such that y :— (1 — p) y' £ c\"R(pUx) We find the sequence {{xn,yn))n>l C grft such that (xn) C pUx and v„ := y  2/1  py2 ^ " _ 1 2 / n £ pnr)BY for n > 1. _1 Hence (u n ) > 0 and /x" 2/„ = u„_i  vn, whence \\yn\\ < 77(1 + p) for n > 1. Taking An := (1 — p)pn^1 > 0 for n > 1, X) n >i ^" = 1> * n e s e r i e s S « > i ^n%n is Cauchy and the series X3n>i ^n2/n is convergent with sum y'. Since 51 satisfies the condition (Hwi), we obtain that the series J2n>i ^nXn is convergent with sum x' and (x',y') £ gr!R. Of course, x' £ pUx Hence
nBy C npUx)
•
Lemma 1.3.3 Let X,Y be topological vector spaces, 3? : X =4 Y be a closed convex multifunction and XQ € X. Suppose that X is complete and first countable. Then condition (1.8) holds.
Open mapping
theorems
21
Proof. Let y0 G f)u£Nx(x0)int ( cl %( U )) Replacing gr51 by g r R  { x 0 , y0) if necessary, we may assume that (xo,yo) = (0,0). Let us first show that V i £ ] 0 , l [ , VU,U' £Kx
• tc\3l(U)c
31 {t{U + U')).
(1.9)
Fix t G ]0,1[ and take tn := t2 for n > 1. Of course, lim£„ • • • t\ = t. Consider U' G Kx There exists a base of neighborhoods (t/„) n eN C K x such that U\ +U\ C U' and Un+i + Un+i C Un for every n € N. Let Un := (1  i „ )  1 i „ • tyUn For every n G N there exists Vj( 6 Ky such that V^ C cl#([/'„). Consider V„ := ^ ( l  t„)V n . Let £/ G K x and j/ G cl %{U); we intend to show that ty G 31 (*(£/ + £/'))• We construct a sequence x = XQ G U, X\ G t/i, . . . , xn G J7 n ,... with the property: V n G N : t n .. .hy G clK (i„ .. .tx (x + ••• + xni
+Un)).
(1.10)
Since (t/  Vi) n 3?([7) ^ 0, there exist yx G Vi and x G J7 such that y — yi G 3£(a:); hence tij/ = *i(y  yi) + (i  *0 ((i  hyhm) C ti3i(a;) + (1  t i ) c l K ( ^ ) C c\5t{h(x
G t&ix)
+ (l 
W
+ Ui)).
Suppose that we already have x £ U, xi £ Ui,... ,xni desired property. Since
G £/ni with the
(i„ . . . tiy  Vn+i) n 31 (i„ . . . h(x + • • • + x„_i + Un)) + 0, there exist yn+i G Vn+i
an
d xn G C/n such that
tnhyyn+i
G3i(tn...ti(H
F x „  i + xn)).
Therefore tn+l •••hy
= tn+l(tn
hy
— 2/n+l) + ( 1  < n + l ) ( ( 1
£ tn+i$.(tn...ti(x{
ha;„_! +xn))
_
tn+1)~~
+ (1
tn+1yn+i)
tn+1)V„+1
C i„+i^(*n .. . t i ( i + • • • + a; n i + a;„)) + (1  t„+i) d 3 l ( t / ; + 1 ) C cl3t(t n +i ...h(x\
hx„_i +xn +
Un+i)).
So the desired sequence is obtained. Since xn+\ H h x „ + p G Un+i + • • • + Un+P C Un for all n,p € N, the series 2 n > i x™ ' s convergent. Denote by x1 its sum. Since i i +  + i » E i i + f/i and Ui is closed we have that x1 Gxi+Ux C U'.
22
Preliminary
results on functional
analysis
Let now U" E Nx and V" E Ny be arbitrary. There exists n E N such that tn ...h(x\ \xni+Un) C t(x+x')+U" andtn .. .hy G ty+V". By Eq. (1.10) we obtain that tn ... txy E cl!R(t(x + x') + U"). It follows that (ty + V")nJi(t{x + x') + U") ^ Hi, and so there exist x" E U" and y" G V" such that ty + y" E $.(t(x + x')+x"), i.e. (t(x + x'),ty) + (x",y") € g r # . It follows that (t{x + x'), ty) E cl (gr 3V) = grft. Therefore ty E ft (t(U + U')). To complete the proof, let U E Nx(0). Then there exists U' E Nx such that [/' + [/' C 1U. From Eq. (1.9) we obtain that  c l # ( E / ' ) C 3? (  ( [ / ' + [/')) C #(*/). Since 0 E C\ue^x{xo) int (cl3i(U)), we have that 0G int£([/). ° D Note that Lemma 1.3.3 is a particular case of Lemma 1.3.1 when Y is first countable; otherwise these results are independent. Corollary 1.3.4 Let Y be a first countable topological vector space and C CY be an ideally convex set. Then intC = int(clC). Proof. Consider X an arbitrary Frechet space (for example X = R) and ft : X =t Y denned by IR(0) = C, R(x) = 0 for x ^ 0. Of course condition (Hwir) is satisfied. Taking y0 E int(clC) and xo = 0 we have that 2/o G f)ue^x(xo)int ( c l 3 i ( C 7 ))' a n d s o 2/o € int C. O Theorem 1.3.5 (Simons) Let X and Y be first countable. Assume that X is a locally convex space, "R : X =$ Y satisfies condition (Hwa;), yo E lb (ImJi) and x0 E 3l_1(j/o) Then yQ E int af f(i m s) R(U) for every U E N x ( z 0 ) . In particular i6(ImIR) = rint(Imft) if i6 (ImK) ^ 0. Proof. Once again we may consider that (xo,yo) = (0,0); so Y0 := aff(ImCR) — lin(Im3i). Replacing, if necessary, Y by YQ, we may suppose that Y is barreled and 0 G (ImIR)\ Let U E Ncx; since grft is a convex set and R(U) = Pry (gr ft n U x Y), %(U) is convex, too. "R{U) is also absorbing. Indeed, let y E Y. Because Imft is absorbing, there exists A > 0 such that Xy E Imft. Therefore there exists x E X such that (x,Xy) E grft. Since U is absorbing, there exists fi G]0,1[ such that fix E U. As grft is convex we have that (fj,x,fj,Xy) = fx(x,Xy) + (1 — /x)(0,0) G grft, whence /xAj/ G R(U). Therefore $.(U) is also absorbing. It follows that cl (£([/)) is an absorbing, closed and convex subset of the barreled space Y. Therefore 0 G int (clft(C/)). Using Lemma 1.3.1 we obtain that 0 G intft(Z7) for every neighborhood U of 0 G X. The last part is an immediate consequence of what was obtained above. •
Open mapping
theorems
23
Corollary 1.3.6 Let X be a Frechet space, Y be first countable and 51 : X =$ Y be liconvex. Assume that yo € t6(Im!R) and XQ 6 5i~1(yo)Then y0 £ intaff(im3j) 5l(U) for all U E Nx(zo) In particular *b(Im5l) = rint(Im^) provided i6 (ImK) ^ 0. Proof. There exist a Frechet space Z and an ideally convex multifunction S : Z x X =4 Y such that grIR = P r x x y ( g r S )  Then § verifies condition (Hw(z,i)) by Proposition 1.2.6 (ii). Of course, there exists ZQ 6 Z such that yo £ S(zo,zo) Since ImS = Imft, by the preceding theorem, j / 0 € intaff(imX) #(E0 = intaff(imK) HZ x ^ ) for every U G Kx(^o)• Theorem 1.3.7 (Ursescu) Let X be a complete semimetrizable locally convex space and 51 : X =£ Y be a closed convex multifunction. Assume that yo £ t6(ImCR) and xo € 5l~1(yo) Then yo € int a ff( Im ^) 5l(U) for every U £ Xj(xo). In particular i 6 (Im^) = rint(ImK) if ib{Im 51) ^ 0. Proof. If Y is first countable it is obvious that the conclusion follows from Simons' theorem. Otherwise the proof is exactly the same as that of Simons' theorem but using Lemma 1.3.3 instead of Lemma 1.3.1. • An immediate consequence of Theorem 1.3.5 is the following corollary. Corollary 1.3.8 Let Y be a first countable barreled space and C CY be a lower ideally convex set. Then Cl = intC. Proof. Let yo G Cl. There exist a Frechet space X and an ideally convex multifunction 51: X =3 Y such that C = Im 51. The conclusion follows from Theorem 1.3.5 taking again U — X. O In fact the conclusion of the above corollary holds if C is the projection on Y of a subset A of X x Y with (a') X is a first countable locally convex space and A satisfies condition (Hwx) or (b') X is a semimetrizable complete locally convex space and A is closed and convex. Putting together Corollaries 1.3.4 and 1.3.8 we get the next result. Corollary 1.3.9 Let Y be a first countable barreled space and C C Y be an ideally convex set. Then Cl = intC = int(clC) = ( c l C ) \ • In normed spaces the following inversion mapping theorem holds. T h e o r e m 1.3.10 (Robinson) Let (X, ) and (Y, ) be normed vector spaces, 51 : X =S Y be a convex multifunction and (xo,yo) € g r ^  Assume
Preliminary
24
results on functional
analysis
that y0 + rjUy C 3\{XQ + pUx) for some n,p > 0. Then d f c . K  1 ^ ) ) < P+\\xx0\\ v\\yyo\\
,d(
^ ( a .))
VarGX, Vy£y0
+ r,BY.
Proof. Replacing g r # by g r # — (xo,yo), we may assume that (xo, yo) = (0,0). Let x E X and y G nBy. The conclusion is obvious if x £ domIR or y € &(z), so suppose that neither is true. Choose 6 > 0 and find ye € 3£(:c) such that 0 < \\ye  y\\ < d[y, R(x)) + 6; define a := n  \\y\\ > 0 and take e G ]0, a[. Consider ye :=y + {ae)\\yye\\~1
(y  ye);
thus y £  < \\y\\ + (a — s) =n — e, and so ye G nBy Therefore there exists xe G pUx with y£ G R(x€). Define A := \\y yg\\(ae + \\y  y ^ l )  1 G ]0,1[. Then 2/ = (1  X)ye + Xys G (1  A)tt(a:) + A#(x £ ) C E((l  A)z + Aa;£). Thus (1  A)z + Xx£ G 3£_1(j/), whence d ( x , ^  1 ^ ) ) < A   x  x e   . As ll^ — ^ell < \\x\\ + ll^ell < P + INI a n d A < (a — e ) _ 1 y — yg\\, we obtain that d (x^iy))
<(P+
\\x\\) (a  e ) ' 1 (d(y, X(x)) + 9).
Letting 6, e —> 0, we obtain the conclusion.
•
Combining the preceding result and Lemma 1.3.2 we obtain the following important result in normed spaces. The implication (i) =$> (ii) is met in the literature as the RobinsonUrsescu theorem. Theorem 1.3.11 Let (X, ), (Y, ) be two normed vector spaces and 31 : X z t 7 be a multifunction. Suppose that Y is a barreled space, g r # verifies condition (Hwx) and (xo,yo) G gr3i. Then the following conditions are equivalent: (i)
yoeQmX)';
(ii) 2/0 e int:R(xo + Ux)\ (iii) 3n > 0, VA G [0,1] : 2/0 + XnUy C IR(a;o + At/ X ); (iv) 37,77 > 0, Vz G z 0 + r?t/ X; Vy G y0 + nUy : d ( ^ S T ^ y ) ) < 7d(y,K(a;)); (v)
3 r ? > 0 , V y G y o + r ? [/x, V x G X
d(y,3i(i)).
: d(x,R\y))
< ^Efff •
Open mapping
theorems
25
Proof. The implications (hi) =>• (ii) =>• (i) are obvious. The implication (i) =$> (ii) follows immediately from Simons' theorem, while the implication (ii) =$> (v) is given by the preceding theorem. (iv) => (iii) Let 7' > 7; we may assume that 777' < 1, because, in the contrary case, we replace 77 by 77' := I / 7 ' < 77. Let y £ yo + A?7?7y, y 7^ j/o, w i t h A e ] 0 , l ] . Thend(a;o,^ 1 (y)) < 7 • d{y,3L(x0)) < 7' y  yQ\\. Hence there exists x € 5l~1(y) such that x —io < 7'l2/~2/o < 7'Ar/ < 77. Therefore y0 + A^C/y C R(x0 + \UX)(v) => (iv) Taking a; € XQ + §t/x, 2/ G j/o + f^y' w e obtain that d (x, ft"1 (y))
VVeNHTio) :
R(V)=T1(V)£Nx(x0),
which means that T is continuous at XQ.
•
Corollary 1.3.13 (BanachSteinhaus) Let X, Y be Frechet spaces and T : X —> Y be a bijective linear operator. Then T is continuous if and only if T _ 1 is continuous; in particular, if T is continuous then T is an isomorphism of Frechet spaces. Proof.
Apply the closed graph theorem for T and T  1 , respectively. D
Theorem 1.3.14 (open mapping theorem) Let X, Y be Frechet spaces and T G L(X,Y) be onto. Then T is open. Proof. Of course, T is a closed convex relation and Im T = Y. Let D C X be open and take 7/0 G T{D)\ there exists XQ G D such that 7/0 = Tx0. Applying the Ursescu theorem for this point, since D is a neighborhood of XQ, we have that T(D) is a neighborhood of T/0. Therefore T(D) is open.D
Preliminary
26
results on functional
analysis
An interesting and useful result is the following. Corollary 1.3.15 Let X, Y be Frechet spaces, A C X and T : X >• Y be a continuous linear operator. Suppose that I m T is closed. Then T(A) is closed if and only if A + ker T is closed. Proof. Replacing, if necessary, F by I m T and T by T" : X > I m T , T'{x) := T(x) for x G X, we may suppose that T is onto. Consider T : Xj kerT —> Y, T(x) := T(x), where x is the class of x. It is easy to verify that T is well defined, linear and bijective. Let q : X —> M. be a continuous seminorm; since T is continuous, p := qoT is a continuous seminorm, too. For all x € X and u € ker T we have that ( X/keiT, ir(x) := x, we obtain that T(A) is closed & n(A)  A is closed <$• IT'1 (A) = A + ker T is closed, and so the conclusion holds.
•
As an application of the Simons and Ursescu theorems we give the following two interesting results, useful in studying optimal control problems. We recall that a process is a multifunction C : X => Y whose graph is a cone; when the graph of the process C is convex or closed one says that e is a convex process or closed process, respectively. The adjoint of the process 6 is the w*closed convex process 6* : Y* =3 X* whose graph is the set {(y*,x*) £ Y* x X*  (x*,y*) € (gre)+). Theorem 1.3.16 Let X, Y, Z be Banach spaces, 6 : X =$Y be an ideally convex process and T G £(•£, Y). Consider the following statements:
(i) ImT d m 6 ; (ii) 3 P l > 0 , V(»*,*•) GgrC* : \\T*y*\\ <
(hi) 3P2>o
: T{UZ) c
Pi\\x*\\;
P2e(uxy,
(iv) 3p3>0 : T(Uz)Cp3c\(e(Ux)). Then (i) «• (hi), (ii) •£• (iv) with p\ = p3 and (hi) => (iv) with p3 = p2. Moreover, ifT is onto then (iv) => (hi) with (any) p2 > p3. Proof. The implications (hi) =$• (i) and (iii) => (iv) (with p3 = p2) are obvious.
Open mapping theorems
27
(i) => (hi) Let 3? := T~1oQ; one obtains immediately that 3J is an ideally convex process with ImIR = Z. Applying the Simons theorem (Theorem 1.3.5) for (a;o,2/o) = (0)0); there exists p > 0 such that pUz C 3£(t/x), which means that pT(Uz) C G(Ux) Taking P2 := p"1, (hi) holds. (iv) => (ii) Let (y*,x*) G grC*, i.e. (z*,y*) 6 (grC)+. Then   T V I I = sup ( z ,  T V > = sup ( T z ,  y * ) < sup fo.y*) z
sup (y, y*) < p3 sup{(a;, x*) \ (x,y) G grC, a; < 1} yee(ux)
because (y, —y*) < (x, —x*) for (x,y) G grC. So (ii) holds with p\ = p3. (ii) =*• (iv) Suppose that (iv) does not hold and take z G Uz such that Tz $ /93cl(C([/x)) Since C(f/x) is convex, using Theorem 1.1.5, there exist y* G Y* and A G E such that %T*T)
= (Tz,y*) < A < (p3y,y*)
Vy G G(UX).
It follows that A < 0, and so we can take A = p3. Hence —p3 > (z, T*y*) > r*y*ll and  1 < (y,^) = (x,0) + (y,y*) for (x,y) G grC with x G Ux, whence T*y* > p3 and (0, y*) G (gr 6 n Ux x 7 ) ° = «;*  cl ((gr 6)+ + tf*. x {0}) = (gre)+ + ?7x' x { 0 } because Ux* x {0} is W*compact (we have used the AlaogluBourbaki theorem). Therefore there exists x* G Ux* such that (y*,x*) G gr(2* (<S> (  r ,j/*) G (grC)*). Since \\T*y*\\ > p3 > p3 \\x*\\, (ii) does not hold for pi= p3. (iv) =^ (iii) when T is onto. Since T is onto and Z, Y are Banach spaces, T is open (see Theorem 1.3.14). It follows that int(T(Uz)) = T{BZ). But G(Ux) is csclosed. Indeed, by Proposition 1.2.2, A := grCnUx x Y is csclosed, and so, X being complete, A satisfies condition (Hz) (see Proposition 1.2.6(h)); by Proposition 1.2.6(v) we have that Pry(A) = G(Ux) is csclosed. Using Corollary 1.3.4 we obtain that intC([/x) = iat(c\G(Ux))So, from (iv) we obtain that T(BZ) C p3Q(Ux), and so T{UZ) C p p3. D A similar result holds in dual spaces.
Preliminary results on functional analysis
28
T h e o r e m 1.3.17 Let X,Y,Z be normed spaces, T e £>{X,Y) and 6 : X =$ Z be a convex process. Consider the following statements: (i)
ImT'Clme*;
(ii) 3 p > 0 , V(x,z) (iii) 3 p > 0 :
S g r C : Ta; < p;z; T*(UY*)CPe*(Uz>).
Then (i) O (iii) and (ii) O (iii) with the same p. Proof. The equivalence of (i) and (iii) follows from the equivalence of (i) and (iii) of the preceding theorem; just apply it for the Banach spaces X*, Y*, Z*, the continuous linear operator T* and the closed convex process
e*. (iii) =>• (ii) The proof follows the same lines as the proof of (iv) =£• (ii) in the preceding theorem, so we omit it. (ii) => (iii) Even if the proof is similar to (ii) =>• (iv) in the preceding theorem, we give the details. So, suppose that (iii) does not hold and take y* e Uy* such that T*y* £ pG*(Uz>). We may assume that y* £ Sx(otherwise replace y* by 2/* _1 y*). Using the fact that Uz* is w*compact and gre* is u>*closed it follows easily that G*(Uz*) is uAclosed; being also convex, we can apply Theorem 1.1.5 in (X*,w*). Therefore there exist x £ X and A G E such that (Tx,y*) = {x,T*y*) > A > (x,px*)
Vx* € e*{Uz>).
Hence A > 0, and we can take X = p. Hence Tx > p and 1 > (x, x*) for every x* € e*(Uz>), i.e.  1 < (x,x*) + (0,z*) for (x*,z*) 6 (grC)+ with z* £ Uz* • It follows that (af,0) € ( ( g r Q + f l l x Uz.)° = cl ((gre)++ + {0} x UZ)
=
cl(gve+{0}xUz).
Hence there exist ((xn,zn)) C grC and (z'n) C Uz such that (xn) > x and (wn) := (zn + z'n) >• 0. It follows that (Ta;„) >• \\Tx\\ and \\zn\\ = \\wn — z'n\\ < 1 + iwn for every n, whence plimsup \\zn\\ < p < \\Tx\\ < liminf Ta;„. Therefore there exists n 0 such that Ta; n  > pz„ for n >n0, which proves that (ii) does not hold for the same p. O
Variational
1.4
principles
29
Variational Principles
A very important result in nonlinear analysis is the "Ekeland variational principle." Theorem 1.4.1 (Ekeland) Let {X,d) be a complete metric space and f : X —» M. be a proper lower semicontinuous and lower bounded function. Then for every xo G dom / and e > 0 there exists xe G X such that f(xE) < f(x0)
ed(xQ,xE)
and f{xe)
Vx6l\{x£}.
Proof. Let XQ G dom / and e > 0 be fixed. For every x G X consider the set F(x) := {y G X \ f{y) + ed(x,y) < f(x)}. Note that the conclusion is equivalent to the existence of an xe G F(xo) such that F(xs) = {xE}. Since the function X 3 y *> f(y) + ed(x, y) G E is lower semicontinuous (lsc for short), F(x) is closed. Note that x G F(x) C d o m / for every x G d o m / and F(x) = X for x G X \ d o m / . Also note that F(y) C F(x) for every y G F(x). The inclusion is obvious for x ^ d o m / . So, let x G d o m / , y G F(x) and z G F(y). Then f{z)+sd(y,z)
f{y)+sd(x,y)
< f{x),
d(x,z) < d(x,y) + d{y,z).
Multiplying the last relation by e, then adding all three relations we obtain that f(z) + ed(x,z) < f(x), i.e. z G F(x). Since / is bounded from below, s0 := inf{/(x)  x G F(xo)} G E; take X\ G F(xo) such that / ( ^ i ) < SQ + 2~1. Then consider si := inf{/(z)  x G F(xi)} G E and take x2 G F(a;i) such that ffa) < s\ + 2~2. Continuing in this way we obtain the sequences (sn)n>o C M. and (xn)n>o C X such that sn = inf{/(a;)  x G F(xn)},
xn+1 G F(xn),
for all n > 0. Because i^(a; n+ i) C F(xn), as xn+i G F ( i „ ) , ed(a; ri+ i,a; n ) < f(xn)
 f(xn+1)
f{xn+1)
< sn + 2  "  1
s„+i > s„ for n > 0. Moreover,
< f{xn)  sn < f(xn)
 s„_i < 2~ n
for n > 1, whence d(x„ +p ,a;„) < e~121~n for n , p > 1. This shows that (xn) is a Cauchy sequence, and so (xn) converges to some xs G X. Since
30
Preliminary
results on functional
analysis
xm G F{xn) for m > n, we have that xE € clF(a; n ) = F(xn) for every n > 0. In particular xE £ F{XQ). Let a; 6 F(ar £ ); then x € F(xn) for every n > 0, and so, as above, ed(x,xn) < f(x„) — f(x) < f(x„)  sn < 2~n, which shows that (xn) —>• x. As the limit is unique, we get x = xe, which shows that F{xe) = {xe}. The proof is complete. • In applications one often uses the following variant of the Ekeland variational principle. In the sequel by infx / , or simply inf/, we mean mi{f(x)  x € X}, where / : X > I . Corollary 1.4.2 Let (X,d) be a complete metric space and f : X » R be a bounded below lower semicontinuous proper function. Let also e > 0 and xo S d o m / be such that f(xo) < mix f + £• Then for every A > 0 there exists x\ S X such that f(x\)
< f(x0),
d(xx,x0)
< A
and f(xx)
< f{x) + eXidfaxx)
Va; £ X \
{xx}.
Proof. Applying the preceding theorem for xo and eA _ 1 , we get x\ € X satisfying the second relation of the conclusion and f(x\)+e\~1d(xo,x\) < f(xo). Hence f{x\) < f{xo). Moreover, because f(x0) < inf^ / + £ < f(x\) + e, we get also that d{x\, XQ) < A. D A good compromise is obtained by taking A = Jz in the preceding result. An interesting application of the Ekeland theorem is the following result. Corollary 1.4.3 Let (X, ) be a Banach space and f : X > R be a lower semicontinuous function. Assume that f is Gateaux differ•entiable and bounded from below. If (xn)n^ is a minimizing sequence for f, i.e. (f(xn)) > infjt / , then there exists a minimizing sequence (xn) for f such that (\\xn xn\\) > 0 and (V/(x„)) » 0. Proof. Consider en := f(xn) — infx f £ K+ If f{xn) = infx f we take xn := xn; otherwise, applying the preceding corollary for xn, en and A = y/e„~ we get xn e X such that f(x„) < f(xn), \\x  xn\\ < ^fe^ and f(xn)
MxeX.
Note that these relations hold also in the case e„ = 0 (with our choice). Taking x := xn + tu with t > 0 and u £ X, we get y/s^ \\u\\ < t~l(f(xn +
Variational
principles
31
tu) — f{xn)). Taking the limit for t —¥ 0 we get — y/e^\\u\\ < (u,S7f(xn)) for all u £ X. It follows that V/(z„) < y/e^. The conclusion follows. • The next result is met in the literature as the smooth variational principle. This name is due to the fact that, at least in Hilbert spaces, the perturbation function is smooth (for p > 1). The result will be stated in a Banach space (X, ) even if it holds in any complete metric space (with the same proof). Before stating it let us observe that, when (un)n>o C X is bounded, (nn)n>o C M+ is such that Yln>o ^n — 1 a n ^ P £ [1J °°[> the function
Qp : X + E, Qp(x) := ^2n>0^n ^ ~ Un^ '
(L11)
is well defined and Lipschitz on bounded sets; in particular 0 P is continuous. Theorem 1.4.4 (BorweinPreiss) Let (X, ) be a Banach space, f : X —> M. be a proper lower semicontinuous and bounded from below function and p 6 [1, oo[. If xo £ X and e > 0 are such that f(xo) < mix f + £ then for every A > 0 there exists a sequence (wn)n>o C B(xo,X) converging to some u € X such that f(u) < mix f + e,
\\xo  u\\ < A
and f(u) + e\pQp{u)
< f(x) + e\pQp{x)
VxeX,
(1.12)
where Qp is defined by Eq. (1.11). Proof.
Fix A > 0 and consider 7, S, n, /i, 6 > 0 such that
6 < M ( l  (7 ? 7~ 1 ) 1 / T> (1.13) p and 6 := (1  fi)e\~ . Let uo := #o and / 0 := / . Taking fi(x) := fo(x) + 5 \\x  u0\\p for a; £ I , we have that fi(u0) = /o(«o) > mix h Hence there exists u\ 6 X such that f\(u\) < 6f0(u0) + (1 — 6) mix fi Taking f2(x) := fi(x) + 5/i\\x — ui\\p for 1 £ I , we have that /2(ui) = fi(ui) > mix fiHence there exists u2 £ X such that f2(u2) < 6f\(u\) + (1 — 0)infx / 2 . Continuing in this way we obtain a sequence (u„) n >o C X and a sequence of functions (fn)n>o such that f(x0)  mixf
< V < 7 < e,
fn+i(x)=fn(x)+6fin\\xun\\p
H
Vx£l,Vn>0,
(1.14)
32
Preliminary
results on functional
analysis
and fn+l{Un+l)
Vn>0.
(1.15)
It is clear that / „ < fn+i; taking sn := infj*:/ n , (s„)„>o C E is a nondecreasing sequence. Let an := / „ ( u n ) ; because fn+i{un) = fn(un) > infx/n+i, from Eq. (1.15) we obtain that an+\ < an for every n > 0. Using again Eq. (1.15), we get sn < s„+i < an+i < 0an + (1  9)sn+i
< an,
and so an+i  sn+i < Qan + (1  6)sn+i  sn+i = 6(an  sn+i) < 0(an  s„), whence ansn<9n(a0s0)
Vn > 0.
(1.16)
It follows that lims„ = lima„ € E. Taking x := u n +i in Eq. (1.15), we get fln > a«+i = fn{un+i) + Sn" \\un+1  un\f >sn + 6p,n u„+i  un\\p , which, together with Eqs. (1.13) and (1.16), yields 6n(a0  s0) < 6nr)
Sfi"   « n + 1  u „ f
(77/<5) 1 / J , (6'/^)"/P,
\\un+m  u „   < (v/8)1/p(0/l*)n,p{l
Vn > 0.
whence
~ Wltf1')'1
Vn,m > 0.
Since 0 < #//z < 1, the sequence (u n )„>o is Cauchy, and so it converges to some u 6 X. Moreover, the inequality above and (1.13) imply that I K  unll < (V/S)1/P(vhr1/P
= (l/S)1/p
 fi)1^ = A (1.17) for all n,m > 0. In particular (un) C B(xo,X) Letting m —> co in the above inequality we get \\u — un\\ < A for n > 0, and so \\u — xo\\ < A. Consider n„ := fin(l  fx) > 0; hence J2n>of1n = 1 Let 0 P be defined by Eq. (1.11). From Eq. (1.14) we obtain that fn+i{x)=f{x)+e\pY.nk=0^\\xuk\\p
< (e/Sy/p(l
Vn>0, V x e l ,
33
Variational principles and so f(x) + eX pQp(x) — lim/„(a;) > lims„. Using Eq. (1.17) we get f(un) + e\~pQp(un)
= fn(un)
+ e\~p Y]
^k \\un  uk\\p fikX" = an + e}
ak
*— / k=n+l
K=n+1
for every n > 1. Taking into account the continuity of 0 P and the lower semicontinuity of / , the preceding inequality yields f(u) + eX~pQp(u) < lima„ = lims„. Therefore (1.12) holds. From Eq. (1.17) we obtain that Qp{x0) < Y " ^Un I K  Un\\P < J~] ^—'n>l
HkjS"1 = fJ.jS1 < fJ,XP,
^—'n>l
and so, by Eq. (1.12), f{u) < f(x0) + eX~pep{x0)
< f(x0) +sfi<
infx / + 7 + £/* < infx f + £
The proof is complete.
D
Note that ©i(:r) Qi(y)\ < \\x — y\\, and so Eq. (1.12) becomes f(u) < /(a;)+£A1a;—w for every 2; € Xwhenp= 1. So, under the slight stronger condition f(x0) < mix / + £ one recovers the conclusion of Corollary 1.4.2, but the condition f(x\) < f(xo). Although not directly related to what follows, we give the next two dual interesting results which have not been published by their author, C. Ursescu. Theorem 1.4.5 Let (X,d) be a complete metric space and (F„) n eN be a sequence of closed subsets of X. Then
cl
(U„ e N i n t F ")= d i n t (U„ e N ^)
d18)
Theorem 1.4.6 Let X be a complete metric space and (Dn)ne^ sequence of open subsets of X. Then
int
(n„ 6 N c i ^) = i n t c i (n„ e N ^)
be a
(LI9)
Note that Theorem 1.4.5 can be obtained immediately from Theorem 1.4.6 by taking Dn = X\Fn (and Theorem 1.4.6 is obtained from Theorem 1.4.5 by taking Fn = X \ Dn). We give only the proof of Theorem 1.4.6. Proof. Since the inclusion "D" in Eq. (1.19) is obvious, we show the converse one. So let x G int (" n e N clD n ) and r > 0; take xi := x.
34
Preliminary
results on functional
analysis
Then there exists ri G]0,r[ such that D{xi,r\) C f]n£Nc\Dn; therefore D(xi,r\) C clD„ for every n £ N. It follows that B(x\,ri) n D\ is an open nonempty set. There exist X2 € X and r2 6]0,ri/2] such that D(x2,r2) C B(xi,n) D £>i. It follows that D f o , ^ ) C D(a;i,ri) C cl£>2) and so B{x2,r2) fl £>2 is an open nonempty set. Continuing in this way we find the sequences (a?n)neN C X and (rn)n£N C F such that (r„) n£ m >• 0 and D(xn+\,rn+i) C B(x„,r„) n £>„ for every n. In particular D(a; n +i,r n +i) C D(xn,r„) for every n. Since X is a complete metric space, using Cantor's theorem, (~}neND(xn,rn) = {x'} for some x' € X. It follows that x' £ £>„ for every n. Since x' 6 D(xi,r{) C B(x,r), we have that B(x,r)nf\n€JiDn. As r > 0 is arbitrary, x £ cl (finer*^M Therefore int ( f  n e N c l D „ ) C cl ( f  n € N A i ) , whence the inclusion "C" in Eq. (1.19) holds, too. • From Theorem 1.4.5 we obtain immediately the famous Baire's theorems: Theorem 1.4.7 (Baire) Let (X,d) be a complete metric space. Then any countable intersection of dense open subsets of X is dense. Proof. Let A = C\n€N Dn with Dn open and dense for every n £ N. From Eq. (1.19) it follows that X = int(clA), and soc\A = X. • Remind that an intersection of a countable family of open subsets of the topological space (X, r ) is called a Gs set. Theorem 1.4.8 (Baire) If (Fn)ngN is a sequence of closed subsets of the complete metric space (X,d) such that X = \Jne^Fn, then at least one of Fn 's has nonempty interior. Proof.
1.5
The conclusion is immediate from Eq. (1.18).
•
Exercises
Exercise 1.1 (Caratheodory) Let X be a linear space of dimension n 6 N and let A C X be nonempty. Prove that co A
= { Y^ill^
 M i e W T C R+, (xi) ieT ^+T C A, Y^ill^
= !} •
Exercise 1.2 Let X be a finite dimensional normed space and A C X be a nonempty convex set. Prove that 'A =£ 0, "(cl A) — 'A, cl(*A) = cl A and for every a € 'A and z g cl A we have }a, z[ C lA.
Exercises
35
E x e r c i s e 1.3 Let X be a separated locally convex space, H C X be a closed hyperplane and A C X be a nonempty convex set. If int# (A O H) ^ 0 and A <£ H prove that int ^4 ^ 0. Moreover, if M C X is a closed affine set with finite codimension and intM{A f~l M) ^ 0, prove that rint A ^ 0. E x e r c i s e 1.4 Let X be a separated locally convex space and A, C C X be nonempty convex sets with int C ^ 0. Prove that int(A + C) = A + int C. Moreover, if A n int C ^ 0, then cl(A DC) = cl yl l~l cl C. In particular, if C is a convex cone with nonempty interior, then cl C + int C — int C. E x e r c i s e 1.5 Let X be a separated locally convex space, Xo C X be a linear subspace, a i , . . . , ap G X, ipi,..., tpk G X" and a i , . . . , a* G R, where p, A; G N. Prove that: (a) if Xo is closed and C = { E L i  ^ I Vi G L7p : A, > 0}, then X 0 + (7 is a closed convex cone. In particular C is a closed convex cone; (b) Xo + {x £ X  Vi 6 1, k : (x, ipi) < at} is a convex closed set. E x e r c i s e 1.6 Prove that
Let (X, (•  •)) be a Hilbert space, xo £ X \ {0} and a G [0,7r/2]. P ( a ) := {x £ X  Z(a;, x0) < a}
is a closed convex cone, where Z(x,y) := Arccos
" . . Moreover, P(a)°
=
P(7r/2a). E x e r c i s e 1.7 Let n £ N \ {1}, p > 0 and a i , . . . , a „ £]0,1[ be such that a i + . . . + an = 1. Consider ATp := { ( x i , . . . , x „ , x n + i ) G R n + 1  xi,...,x„
> 0, x n +i < px"1 •• a;""} •
Prove that (Kp)+ = .Ry, where p' := (pa" 1 • • • a " " )  1 . In particular, Kp is a closed convex cone. Exercise 1.8
Prove that P := {(x,t/,z) £ R 3  x,z > 0 , 2 x z > t/ 2 }
is a closed convex cone and P+ = P. (P is called the "ice cream" cone.) Exercise 1.9 Let X be a normed space, ip G X* \ {0} and 0 < a < \\(p\\. Prove that the set C := {x G X 
Preliminary results on functional analysis
36
E x e r c i s e 1.11 Let X, Y be two separated locally convex spaces and 6 : X =4 Y be a multifunction whose graph is a closed linear subspace of X x Y. Prove that domC is dense if and only if C*(j/*) is a singleton for every y* G dome*. In particular, dom (2 is dense and C(x) is a singleton for every x G dom G if and only if domC* is w*dense and C*(y*) is a singleton for every y* € dom6*. Exercise 1.12 Let (X, d) be a metric space. Prove that (X, d) is complete if for any Lipschitz function / : X —• R+ there exists x € X such that f(x) < f(x) + d(x,x) for every x £ X. This shows that the completeness assumption in Ekeland's variational principle is essential. Exercise 1.13 Let (X, d) be a complete metric space and / : X —)• R be a proper lsc and lower bounded function. Suppose that for every x G X such that f(x) > inf / , there exists x G X\{x} such that f(x) + d(x,x) < f(x). Prove that argmin / ^ 0 and d(x, argmin / ) < f(x) — inf / for every x E X. Exercise 1.14 Let (X, d) be a complete metric space, / : X —• R be a proper lsc lower bounded function and 3£ : X =t X. Assume that for every x G X there exists y G 3l(x) such that d(x,y) + f(y) < f(x). Prove that 31 has fixed points, i.e. there exists xo S X such that xo € 3£(xo)E x e r c i s e 1.15
Let (X, ) be a normed space and / : X > R. Prove that
(i) limn^n^oo f(x) = oo if and only if [/ < A] is bounded for every A G R. (ii) Assume that X = Rfc, / is lsc and limnsn^oo f(x) = oo (i.e. f is coercive). Prove that there exists x G R* such that f(x) < f(x) for every x G Rfc.
1.6
Bibliographical Notes
Section 1.1: For the notions and results on topology and topological vector spaces not recalled in this section one can consult many classical books (see [Kelley (1955); Willard (1971); Bourbaki (1964); Holmes (1975)]). The proofs of the results mentioned in this section can be found in [Holmes (1975)] or [Bourbaki (1964)]. Section 1.2: Ideally convex sets were introduced by Lifsic (1970), csclosed sets by Jameson (1972), cscomplete sets by Simons (1990), lower csclosed sets by Amara and CiligotTravain (1999), while condition (Hx) was used by Zalinescu (1992b). The properties stated in Proposition 1.2.1 are mentioned in [Kusraev and Kutateladze (1995)]. All the results concerning lcsclosed sets, as well as Proposition 1.2.2 (for csclosed sets), are from [Amara and CiligotTravain (1999)]. The set *CA, introduced by Zalinescu (1987), is also introduced by Jeyakumar and Wolkowicz (1992) under the name of strong quasi relative interior of A. The notation lbA was introduced in [ZalinescuN(1992b)] but the condition 0 G lbA was intensively used by Simons (1990) and Zalinescu (1992b). The quasi relative interior was introduced by Borwein and Lewis (1992); they stated the properties mentioned in Proposition 1.2.7 in locally convex spaces. Proposition 1.2.9 is
Bibliographical notes
37
stated in [Borwein and Lewis (1992)] for A a csclosed set and X a Frechet space. Joly and Laurent (1971) introduced the notion of united sets, but Proposition 1.2.8 is mainly established by Moussaoui and Voile (1997). Section 1.3: Lemma 1.3.1 is proved by Amara and CiligotTravain (1999) for X a metrizable lcs, Y a Frechet space and ft csclosed, while the statement and proof of Lemma 1.3.3 are those of Ursescu (1975); Corollary 1.3.4 (for Banach spaces) is due to Lifsic (1970). The statement of Theorem 1.3.5 is very close to that of [Kusraev and Kutateladze (1995), Th. 3.1.18]; when X, Y are Frechet spaces our statement is slightly more general. For X, Y metrizable locally convex spaces and CR. satisfying (Ha;) Theorem 1.3.5 is equivalent to the open mapping theorem of Simons (1990). Theorem 1.3.7 is obtained by Ursescu (1975) for Y0 := aff(Imft) = Y. It is established in [Robinson (1976)] for X, Y Banach spaces and Yo = Y; in this case the above result is met in the literature under the name of RobinsonUrsescu theorem. Corollary 1.3.8 was obtained by Amara and CiligotTravain (1999) for lcsclosed sets. Theorem 1.3.10 is due to Robinson (1976), while Theorem 1.3.11 can be found in [Li and Singer (1998)]. Theorems 1.3.16 and 1.3.17 are due to Carja (1989) (with different proofs). Section 1.4: The Ekeland variational principle [Ekeland (1974)] is met in the literature, generally, in the form given in Corollary 1.4.2; the statement and proof of Theorem 1.4.1 are from [Aubin and Frankowska (1990)]. The statement and proof of the BorweinPreiss variational principle are from [Borwein and Preiss (1987)]. Theorems 1.4.5 and 1.4.6 can be found in the recent book [Carja (1998)]. Exercises: The exercises are generally known results or auxiliary results from various papers. So Exercise 1.1 is the famous Caratheodory theorem, Exercise 1.3 is from [Joly and Laurent (1971)], Exercise 1.5 is from [Laurent (1972)], Exercise 1.7 (for n = 2 and p = p) is from [Bauschke et al. (2000)], Exercise 1.10 is stated in [Li and Singer (1998)] in Banach spaces in a slightly weaker form, Exercise 1.12 is the main result of [Sullivan (1981)], Exercise 1.13 is from [Hamel (1994)], Exercise 1.14 is the known Caristi's fixed point theorem (see [Caristi (1976)]).
This page is intentionally left blank
Chapter 2
Convex Analysis in Locally Convex Spaces
2.1
Convex Functions
We begin this section by introducing some notions and notations. For a function / : X —¥ E and A g l consider: dom/
= {x e x  f(x) < +00},
e p i / = {(x,t) eX epi s / =
x E I f(x) < t},
{(x,t)\f(x)
[/ — 00 for every x £ X. It is evident that d o m / = Pry (epi/). Let X be a real linear space and / : X > E; we say that / is convex if Var.l/GX, V A g [ 0 , l ] : /(Arc + (1  \)y) < \f(x)
+ (1  X)f(y),
(2.1)
with the conventions: (+oo) + (—00) = +00, 0(+oo) = +00, 0(—00) = 0. Note that when x = y, or A £ {0,1}, or {a;, y} (£. dom / , the inequality (2.1) is automatically satisfied. Therefore the function / is convex if V x , j / e d o m / , x ^ , V A G ] 0 , l [ : /(Aar + ( l  \ ) y ) < \f(x) + (1  \ ) f ( y ) . (2.2) 39
Convex analysis in locally convex spaces
40
If relation (2.2) holds with " < " instead of " < " we say that / is strictly convex. Similarly, we say that / is (strictly) concave if —/ is (strictly) convex. Since every property of convex functions can be transposed easily to concave functions, in the sequel we consider, practically, only convex functions. To avoid multiplication with 0, taking into account the above remark, we shall limit ourselves to A G ]0,1[ in the sequel. In the following theorem we establish some characterizations of convex functions. Theorem 2.1.1
Let f : X —>• E. The following statements are equivalent:
(i) / is (strictly) convex; (ii) the functions tpxy : E —)• R, (px,y(t) := f((l — t)x+ty), convex for all x, y G X (x ^ y);
are (strictly)
(hi) dom f is a convex set and Vx,yedomf,
V A e ] 0 , l [ : f(\x + (1  X)y) < Xf(x) + (1 
X)f(y)
(the inequality being strict for x ^ y); (iv) V n € N, \/xi,...,xn f(XlXl
E X, VAi,...,A„ G]0,1[, A i +    + A„ = 1 :
+ • • • + Xnxn) < X1f{x1) + ••• + Xnf(xn)
(the inequality being strict when Xi,...,
(2.3)
xn are not all equal);
(v) epi f is a convex subset of X x E; (vi) epi s / is a convex subset of X x E. Proof. The implications (i) => (ii), (ii)=^(i), (i) =>• (hi), (iii) =4> (i) and (iv) => (i) are obvious. (i) =£• (v) Let {xi,ti),(x2,t2) 6 e p i / and A e]0,1[. Then x\,x2 G d o m / , f(xi) < ti and f(x2) < t2. Since / is convex, from Eq. (2.1) we obtain that f{Xxx + (1  X)x2) < Xf{Xl)
+ (1  X)f(x2)
< Xtx + (1  X)t2,
whence X{xi,t\) + (1  X)(x2,t2) € e p i / . Therefore e p i / is convex. (v) =^ (iv) Let k 6 N, A!,...,A fc <E]0,1[, AI + • •  + Xk = 1, xu ... ,xk e X. If there exists i such that f{x{) = oo, then Eq. (2.3) is obviously true. If f{xi) G E for every i, 1 < i < k, then (a;;, f{xi)) € epi / for every i; since e p i / is convex, J2i=i^i(xiif(xi)) € e Pi/> which proves that the inequality
Convex
functions
41
(2.3) is verified. Finally, suppose that /(x,) < oo for every i and that there exists io with f(xi0) = — oo; consider i; G E such that (x;,£i) 6 e p i / for every i ^ io', since ( i j 0 , —n) € e p i / for every n 6 N, we have /(AiXiH
hAfcXfc) < AitiH
hAj 0 _i£j 0 _i+Ai 0 (n)+Ai 0 + ii i o + i+..
.+\ktk
for n G N. Letting n »• oo in the above inequality we get f (J2i=i^ixi) = —oo; hence Eq. (2.3) is verified. The proofs for (i) => (vi) and (vi) => (iv) are similar to those of (i) =>• (v) and (v) => (iv), respectively. The proof for the "strictly convex" case is similar. • Note that in (v) and (vi) there are no counterparts corresponding to / strictly convex. Proposition 2.1.2 If f : X —> E is sublinear then f is convex. Moreover, f is sublinear if and only if e p i / is a convex cone with (0, —1) £ e p i / . Proof. Let / be sublinear. It is obvious that / is convex, and so e p i / is a convex set which contains (0,0). If (x,t) G e p i / and A > 0 then f(Xx) = \f(x) < Xt, and so X(x,t) G e p i / ; hence e p i / is a convex cone. Since /(0) = 0 >  1 , we have that (0,  1 ) g e p i / . Assume now that e p i / is a convex cone with ( 0 ,  1 ) ^ e p i / . It is immediate that / is convex (epi/ being convex) and f(Xx) = Xf(x) for A > 0 and x G X. So, for x,y E X we have that /(a; + y) = 2 / (  x +  y ) < fix) + fiy) I f /(0) < 0 then (0, t) € e p i / for some t > 0, whence the contradiction (0, —1) 6 e p i / . Therefore /(0) = 0, and so / is sublinear. • The indicator function of the subset A of X is
iA:X>R,
iA(x) := j
0 +oo
if if
x G A, x e X\A.
Note that dom iA = A and epi t,A = A x R+. From the preceding theorem we obtain that LA is convex if and only if A is convex. If / is convex the sets [/ < A] and [/ < A] are convex for every A G E. The converse is generally false. A function / with the property that " [ / < A] is convex for every A G E" is said to be quasiconvex. So, / : X » E is quasiconvex if and only if V z , j / G X , VAG[0,1] : / ( ( l  A)* + Ay) < max{/(a;),/(i/)}.
42
Convex analysis in locally convex spaces
The notion of convex function is extended naturally to mappings with values in ordered linear spaces. Let Y be a real linear space and Q C Y be a convex cone; Q generates an order relation on Y: y\ < 2/2 (or 2/1 • Y'; the operator H is Qconvex if Va;,2/eX, V A e ] 0 , l [ : H{Xx + (1  X)y) < XH{x) + (1 
X)H{y).
The operator H : X >• F U {00} is Qconcave if  i J : X » F* is Qconvex. If A £ L(X, Y) then ^4 is Qconvex for every convex cone Q C Y. As in the case Y = E, the domain and the epigraph are defined by: domi? := {x £ X  #(2;) < 00} a n d e p i i ? := {(x, y) eX xY \ H(x) < y}. The characterizations of convex functions given in Theorem 2.1.1 are valid for a Qconvex operator. A function / : (Y, Q) —> E is Qincreasing if 2/1 f(,Vi) < /(2/2)• For such a function we consider that /(oo) = +00. It is obvious that every function is {0}increasing, and that a linear functional ip : Y —> E is Qincreasing if and only if (p(y) > 0 for every y £ Q. One defines similarly the Qdecreasing functions. In the next result we mention some methods for deriving new convex functions from known ones. For a, (3 £ E we set a V /? := max{a,/?} and a A P := min{a, /3}. Theorem 2.1.3
Let X,Y
be linear spaces and Q CY
be a convex cone.
(i) / / fi : X —> E is convex for every i € / (/ ^ 0) then sup i g j fi is convex. Moreover, epi(sup i € / /,) = P  i 6 / e p i / j . (ii) / / / 1 , /2 : X > E are convex and X £ E+, tften /1 + fi and A/i are convex, where 0 • /1 := tdom/i • Moreover, dom(/i 1/2) = dom fi n dom fi,
dom(A/i) = dom / 1 .
(hi) If fn : X —t M. is convex for every n £ N and f : X —> E is such that f(x) = limsup n _ >0O fn(x) for every x £ X, then f is convex.
Convex
43
functions
(iv) If A C X xR is a convex set then the function ipA is convex, where ipA:X^R,
(v) / / F : X x Y —> E is convex, then the marginal ciated to F is convex, where h:Y^%
(2.4)
function
h asso
h(y):=mix€XF(x,y).
(2.5)
(vi) Let g : Y —> E be a convex function. If H : X —> Y' is Qconvex and g is Qincreasing, then g o H is convex; this conclusion holds also if H : X —> Y U {—oo} is Qconcave and g is Qdecreasing. In particular, if A 6 L(X, Y) then go A is convex. (vii)
Let f : X —>• E, g :Y —> K 6e proper convex functions, and let
$,V:XxY>R,
{x,y):=f(x) + g(y),
¥(x,y)
:= f(x)V
g(y).
Then $ and \& are convex and proper. Moreover, d o m $ = d o m * = dom / x dom g, inf $ = inf / + inf g and inf $ = inf / V inf g. (viii) If f : X —> R is convex and A G L(X, Y) then the function Af:Y^R,
(Af)(y):=mi{f(x)\Ax
= y},
is convex. Moreover dom(Af) = A(domf) and inf Af = inf/. (ix) If / i , f2 : X —> R are convex and proper, their convolution maxconvolution, defined by
and
f1Df2 : X > K,
( / i D / 2 ) ( i ) := inf {/i(a;i) + / 2 (z 2 )  an + x2 = x} ,
/1O/2 : X > E,
(f10f2){x)
:= inf { / i ( n ) V / 2 (z 2 ) I Zi + *2 = z} ,
are convex. Moreover dom(/ 1 D/ 2 ) = dom^O./^) = dom/x + d o m / 2 , inf fiDfi = inf /1 + inf / 2 , inf f10f2 = inf /1 V inf f2, epi 4 (/iD/2) = epi s /1 + epi5 / 2
(2.6)
VA 6 E : [AO/2 < A] = [A < A] + [f2 < A].
(2.7)
and
Proof, (i) It is clear that epi(sup iejr fi) = f)ieIepifiSince fi is convex for every i, using Theorem 2.1.1, we have that epi/i is convex, and so epi(sup i € / fi) is convex. The conclusion follows using again Theorem 2.1.1. (ii) is immediate.
Convex analysis in locally convex spaces
44
(iii) Let A s]0,1[ and x,y € d o m / . Since limsup/„(x), limsup/„(y) < oo, there exists n0 £ N such that fn(x), fn(y) < oo for every n > no Since / „ is convex, we have fn(Xx + (1  X)y) < \fn(x)
+ (1 
X)fn(y).
Taking the limit superior we obtain f(Xx + (1  X)y) < Xf(x) + (1 
X)f(y).
Therefore / is convex. (iv) Let (xi,ti), (x2,t2) £ episipA and A £]0,1[. Then there exist si,S2 £ ffi such that (xi,si),(x2,s2) € ^4 and si < £i, S2 < t2 Since A is convex, A(:ci,si) + (1  A)(z2,S2) = {Xx\ + (1  A)x2,Asi + (1  A)s2) £ A. Therefore, (PA(XXI + (1 — A)x2) < Xsi + (1 — X)s2 < Aii + (1 — A)i2, and so X(xi,ti) + (1 — A)(a;2,t2) £ epi s y>A Hence ip^ is convex. (v) Note that epish = P r y x R ( e p i s F ) ,
dom h = Pry (dom F).
(2.8)
The conclusion follows from Theorem 2.1.1(vi). (vi) We observe that (/ o H)(x) = inf y 6 y F(x,y), where F(x,y) := g(y) + t,ep\H(x,y), because / is (Jincreasing. Since obviously F is convex, by (v) we obtain that (/ o H) is convex, too. The other case is proved similarly. The second part is immediate taking into account that A is {0}convex and g is {0}increasing. (vii) One obtains immediately that dom $ = dom $ = dom / x dom g and that $ and \? are convex. The formulas for inf $ and inf * are wellknown. (viii) We have that (Af){y) = mixZX F{x,y), where F(x,y) := f(x) + >gTA(x,y) It is obvious that F is convex. From (v) we obtain that Af is convex. As d o m F = {(x,Ax) \ x £ d o m / } , we have that dom(Af) — A(domf). The formula for inf Af is obvious. (ix) Let us consider the functions $, \P : X x X y R defined by *(xux2)
:= fiixx) + f2(x2),
*(zi,x2)
:= h{xi)y
f2{x2),
and A : X x X > X, A{x1,x2) := zi + x2. Then ( ^ D ^ ) ^ ) = (A$)(x) and (fiQf2)(x) = (A^!)(x). By (v) we have that $ and $ are convex; using (viii) we obtain that / i D / 2 and /1O/2 are convex, too. The relations on the
Convex
45
functions
domains, epigraph and level sets follow by easy verification. The formulas for inf / i D / 2 and inf /1O/2 follow immediately from (vii) and (viii). • The formulas (2.6) and (2.7) motivate the use of "episum convolution" and "levelsum convolution" for the convolution and maxconvolution, respectively. The convolution and maxconvolution are extended in an obvious way to a finite number of functions; from Eqs. (2.6) and (2.7) one obtains that these operations are associative. The convex functions which take the value —00 are rather special. Proposition 2.1.4 Let f : X —»• E be a convex function. If there exists XQ € X such that f(xo) = —00 then f{x) = —00 for every x € ' ( d o m / ) . In particular, if f is sublinear and 0 S ' ( d o m / ) , then f is proper. Proof. Let x £ ' ( d o m / ) ; since xo £ d o m / , there exists /J, > 0 such that y := (1 + fi)x — \ix§ € d o m / . Taking A := (p, + 1 ) _ 1 e]0,1[, x = (1 — \)XQ + Ay, and so / O r ) < ( l  A ) / ( x 0 ) + A/(2,) =  o o . The case of / sublinear is immediate from the first part.
•
In the sequel we denote by A(X) the class of proper convex functions defined on X. Let now  / C c I and / : C > M; we say that / is convex if C is convex and Vx,y€C,
V A € ] 0 , 1 [ : /(Ax + (1  X)y) < Xf(x) + (1 
\)f(y).
It is easy to see (Exercise!) that the function / above is convex if and only if
?X^W
?M(
f{x)
if
X e C
>
is convex in the sense of the definition given at the beginning of this section. Of course, we can proceed in the opposite direction; so a proper function / : X —>ffiis convex if and only if /dom/ is convex in the above sense. The consideration of (convex) functions with values in M. has certain advantages, as we shall often see in the sequel. Taking into account the equivalence (i) <£> (ii) of Theorem 2.1.1, it is useful to know properties of convex functions of (only) one variable. The following result collects some properties of such convex functions.
46
Convex analysis in locally convex spaces
Theorem 2.1.5
Let f € A(E) be such that int(dom/) ^ 0.
(i) Let t\,ti € d o m / , t\ < £2. Suppose that there exists XQ €]0,1[ such that / ( ( l  A0)ii + X0t2) = (1  A 0 )/(*i) + A 0 /(£ 2 ) Then VAe[0,l] : / ( ( l  A ) t 1 + A t 2 ) = (lA)/(*i)+A/(*2), ht s . t  t V t € [ t i , t 2 ] : /(*) = r—rf(ti) + i—±m). *t 2 2—  tt\ i
(2.9) (2.10)
12 — T\
(ii) Let to 6 d o m / . The function Vt0
: dom/ \ {i0} »• K, <*„(*) == /( *1 f (*o), t — to
is nondecreasing; if f is strictly convex then tpt0 is increasing. (iii) Let to e d o m / . Tfte following limits exist: t4.to
I — to
*>to
I — to
r.(tt):ta!StJM.mpmzmet, *t*o
t — to
t
(2.12)
t — to
and
/K*o)
(2.13)
moreover /l(£o),/+(*o) € K whenever t0 € int(dom/). Therefore f is left and right derivable at any point o/int(dom/). Moreover
r e [ /  W , /  ( t o ) ] n l « v t e i : T(tt0) < f(t)  f{t0).
(2.14)
/ / / is strictly convex, in relation (2.14) the inequality is strict for t ^ to. (iv) Lei fi,i 2 £ d o m / , £1 < t 2 . Then f\.(h) < f'_{ti); if f is strictly convex then f\_{t\) < /!_(£ 2 ). Therefore the functions f'_ and f'+ are nondecreasing on d o m / . Furthermore, f is strictly convex if and only if f'_ [resp. f'+] is increasing on int(dom/). (v) The function f is Lipschitz on every compact interval included in int(dom/), and so f is continuous on int(dom/); moreover, for every to E int(dom/) we have: l i m / l ( i ) = l i m / ; ( i ) = f_(t0),
l i m / ; ( t ) = l i m / l ( 0 = f'+ifo).
tjlO
tllQ
tjtQ
t^to
47
Convex functions
(vi) The function / is monotone on int(dom/), or there exists to & int(dom/) such that f is nonincreasing on ] — oo,£o] H d o m / and nondecreasing on [io,oo[n d o m / . (vii) Let to € d o m / and A 6]0,1[. Then the mapping tp : d o m / —> R, if>(t) := \f(t) — f(to + \(t — to)), is nonincreasing on Ii :=] — oo,to\Hdom/ and nondecreasing on Ir := [io,oo[ndom/. / / / is strictly convex then ip is decreasing on I\ and increasing on Ir. Proof. To begin with, let ti,t2,t3 € d o m / be such that t± < t2 < t3. Then t2 = {1  X)tx + \t3 with A = (t3  t2)/(t3  h) € ]0,1[; therefore
m)<^m) + ^ m ) ,
(2.15)
the inequality being strict if / is strictly convex. Subtracting successively f(h), f(h), f(t3) from both members of Eq. (2.15), multiplying then by and t£u> it,tl)(tlt2) A > respectively, we obtain
m)m) t2 — t\
~
m)w)
< /(f 3 )/(* 2 ) )
t3 — tl t\ —t2 ~ f(tl)  /(* 3 ) < / ( * 2 )  / ( * 3 ) > *i — *3
~
t3 —12 (216)
*2 — i 3
these inequalities being strict if / is strictly convex. (i) Let ti,t2 € d o m / and Ao G]0, l [ b e such that / ( ( l  A0)ii + A0i2) = (1 — A 0 )/(*i) + X0f(t2). Suppose that there exists A e]0,1[ such that / ( ( l  \)ti + Xt2) < (1  X)f(h) + Xt2; assume that A < Ao (one proceeds similarly for A > A0). Taking 6 := (1  Ao)/(l  A) e]0,1[, we have that A0 = dX + (1  B) • 1 and (1  A0)*i + A0*2 = 0((1  A)*i + Xt2) + (1  6)t2; so we get the contradiction / ( ( l  X0)h + Xt2) < Of((l  X)h + Xt2) + (1 <6[(lX)f(t1)
+ Xf(t2)} +
= (lX0)f(t1)
+ Xof(t2).
9)f(t2) (l6)f(t2)
Therefore Eq. (2.9) holds. Taking t€ [h,t2] and A = ( t  * i ) / ( i 2  * i ) £ [0,1] in Eq. (2.9), we obtain Eq. (2.10). (ii) Let t 0 £ d o m / and tx,t2 £ d o m / \ {t0}, h < t2. Considering successively the cases t\ < t2 < to, h < t0 < t2, to < t\ < t2, from Eq. (2.16) we obtain that (ft0 is nondecreasing; if / is strictly convex, ipt0 is even increasing.
48
Convex analysis in locally convex spaces
(iii) To begin with, let to € int(dom/). Taking into account that ipt0 is nondecreasing on everyone of the nonempty sets d o m / n ] i 0 , oo[ and d o m / n ] — oo,t 0 [, the following limits exist: ,.hm(p (t),,x := hm ,. /(*)  /(to) • f /(«) ~ /(to) — = inf — ^< oo, to
*4*o
t\.to
,.
m
t — to
fit) ~ /(to)
r
t>to
1™ V«o(*) : = i m — : — r — =
SU
tjto
t
tfto
t — to
t — to
fjt)  f(t0) ^ P —:—:
> °°>
t — to
and so /+(to) and / i ( t o ) exist. Since ipto is nondecreasing on dom / \ {to}, the inequality (2.13) holds. What was proved above shows that fL{to), f'+ito) G K. If to = max(dom/) then f(t) = oo for t > t0, whence /+(to) = oo; of course f'_ito) < oo (the existence of / i ( t 0 ) follows from the monotonicity of
M
< f (to) < r < /;(to) <
im^lM.
tt0  • /  1  u ^   J +1 " ' t'to Note that the first inequality, in the above relation, is strict if the first quantity is finite and the function / is strictly convex; similarly for the last inequality and the last quantity. From these inequalities we have immediately Eq. (2.14), with strict inequality if / is strictly convex. Conversely, assume that r 6 l and r(t —10) < fit) — /(to) for every t € K; dividing by t — to in each of the cases t > to and t < to and taking the limit for t —> to, we obtain that r 6 [/L(to),/+(to)](iv) Let ti,t2 £ d o m / , t\ < t2 and consider t €]ti,t 2 [; using Eqs. (2.11), (2.12) and (2.16) we obtain that
r+{h)
< /(')/(«*> < m)m) = /(«o/(«a) < , ( }
the inequalities being strict if / is strictly convex. Using Eq. (2.13) we get that f'_ and f'+ are nondecreasing, even increasing if / is strictly convex. Suppose that / is not strictly convex; then there exists ti,t2 G d o m / , ti < t2, and A0 6]0,1[ such that / ( ( l  A0)ti + A0t2) = (1  A 0 )/(ti) + A 0 /(t 2 ) Therefore Eq. (2.10) holds, whence
vte]tut2[ f'(t) = fW =
fit
?{itl).
t2 — ti
Convex
functions
49
Because ]ii,i2[C int(dom/), we obtain that f'_ and f'+ are not increasing on int(dom/). (v) Let ti,t2 £ int(dom/), ti < t2 and t,t' e]ti,t2[, t < t'. From Eqs. (2.11), (2.12) and (2.16) we get
< /;(tl) J+K ; 
IMzM < m^M < t/('»)/(«) < ^^' r (ta)> tit f  t t j
2
whence \f(t')  f(t)\ < M\t'  t \ , where M := max{/^(*i), \fL(t2)\} G R Therefore / is Lipschitz on ]ti,<2[ Since every compact subinterval of int(dom/) is contained in an interval ] i i , ^ [ with [ti,t2] C int(dom/), we get the desired conclusion. Let now to G d o m / be such that / is leftcontinuous at to (therefore to > inf(dom/)); for example to € int(dom/). We already know that V t e d o m / n ]  o o , t 0 [ : /!(*) < f+(t) < /l(*o) € ]  oo.oo]. Let A 6 E, A < / i ( i 0 ) ; from the definition of the least upper bound and relation (2.12), there exists *i 6 dom/, ii < to, such that A < (f(h) — f{to))l{t\  t0). By the leftcontinuity of / at t0, there exists t2 £]ti,t0[ such that A < (/(ti)  f{h))/{h  t2). Let £ €]*2,*i[ Using again Eq. (2.16) we get A
fit)  f(t2) tt2
=
f(t2)  f{t) t2t
J~{>
Therefore \im^t0 f'it) — u m tt
50
Convex analysis in locally convex spaces
(vii) Let t i , t 2 G Ir be such that ti < t 2 . Then to + A(*i  t0) < h < t 2
and
t 0 + A(tx  t 0 ) < t 0 + A(t2  i 0 ) < t 2 .
So, from the convexity of / , we have
m) < ^ffii^m
+ \(tl  to)) + t3%xl§;Xm),
f(to + X(h  to)) < ^ f f S T ^ / f a + A(ti  h)) +
I^^L_f(t2),
the second inequality being strict if / is strictly convex. Multiplying the first inequality by A, then adding them, we get A/(*i) + /(to + A(t2  t 0 )) < / ( t 0 + A(ti  t 0 )) + A/(t 2 ), i.e. V'(ti) < ipfo), the inequality being strict if / is strictly convex. The proof is similar on /(. • The preceding theorem shows that /dom/ is continuous on int(dom/) when / : K ¥ K is a proper lower semicontinuous convex function. We have even more. Proposition 2.1.6 Let / G A(R). Then / c i(dom/) JS upper semicontinuous (on cl(dom/)j. Moreover, if f is lower semicontinuous then / c i(dom/) is continuous. Proof. As mentioned above, / is continuous at any to € int(dom/). Let to G d o m / . If d o m / = {to} it is nothing to prove. In the contrary case we can take to = inf(dom/) and some t € d o m / with t > to. Let (t„) C d o m / \ {to} converge to to. We may assume that tn
VfGE\dom/,
t
VtGK\dom/, t>sup(dom/).
Convex
functions
51
Theorem 2.1.7 Let ip : E —> E be a nondecreasing function and a G E fee SMC/I t/m£ V'C0) € E. Then /:E^E,
/(*):= J > ( s ) d s ,
is a proper lower semicontinuous convex function with I:— {s G R 
= f+,
(217)
where the integral is taken in Lebesgue sense and
ip+(t) := 'mi{
for t G E. Moreover, if g : E —>R is a proper lower semicontinuous function such that g'_ < ip < g'+ then g — f + a for some a G E.
convex
Proof. As usual, Ja ip(s) ds := — Jb ip(s) ds if a > b. The statement is obvious if / is a singleton. So we assume that i n t i ^ 0. If t G E \ c l / it is obvious that f(t) = oo. Hence I C d o m / C c l / . If
 /(*o) = / £ ¥>(*) ^ < (tA  t0)v(*A) = A(ti 
*O)V(*A),
f(ti)  f(tx) = Jtl
toMtx).
Multiplying the first relation by (1  A) and the second one by A, then adding them, we obtain that f(t\) < (1 — A)/(to) + A/(*i). Since f\cu is continuous we obtain this inequality also for to,ti G d o m / and A G]0,1[; hence / is convex. Let to G d o m / n c l / and t 6 E, t > to; because ip(s) > ip+(to) for s > to, we have that V(t) >
f{t) f{to)

t — t0
=  J  /%(s)cfe >
t o
and so / ' ( t 0 ) = ip+(t0). Similarly, fL(to) = ip{to). Since these relations are obvious for t0 $ d o m / n c l / , Eq. (2.17) holds. Let now g : E —> E be a proper lsc convex function such that g'_ < ip < g'+. It follows that int(domg) = i n t / . Taking into account Theorem 2.1.5(v) we obtain that g'_(t) =
Convex analysis in locally convex spaces
52
and so g'_(t) = fL{t) and g'+(t) — /+(£) for every t € int I. Taking h : int I —> K, h(t) := f(t) — g(t), we have that h is derivable and h! = 0. It follows that ft is a constant function. Therefore there exists a G E such that g(t) = f(t)+a for all t € int I. Using the preceding proposition it follows that g = f + a. • The following consequence will be used several times. Corollary 2.1.8
Let f : E —• E be a proper convex function.
Then
\/t,t' e int(dom/) : /(*')  f(t) = / / ' /;(*) ds = / / ' / i ( s ) da. Moreover, if f is lower semicontinuous, then the preceding equalities hold for all t,t' € d o m / . Proof. Assume that int(dom/) ^ 0. When t := sup(dom/) e l o r t := inf(dom/) € E, we replace, if necessary, f(t) by limt.j.j/(£) and f(t) by lini£4.(/(£); then / is lower semicontinuous. Applying the preceding theorem for ip = f'+ and cp — f'_ the conclusion follows. D The following result is used frequently to show that a function of one variable is convex. T h e o r e m 2.1.9 Let I C E be a nonempty open interval and f : I —> E be a derivable function. The following statements are equivalent: (i) / is convex; (ii) V t , a e / : / ' ( * ) • ( *  * ) < / ( * )  / ( * ) ; (iii) Vt,s € J : (f'{t) — f'(s)) (iv)
• (t — s) > 0, i.e. / ' is nondecreasing;
('i// is twice derivable on I) 'it £ I : /"(£) > 0.
Proof, (i) =$> (ii) Let / be convex. Since / is derivable we have that f'(s) = / i ( s ) = /+(s) for every s e I. The conclusion follows from Eq. (2.14) taking r = f'(s). (ii) =$• (iii) Let s,t £ I. By hypothesis we have that / ' ( * )
•
(
*

*
)
<
/ ( * )

/ ( * ) ,
/ ' ( * )
•
(
«

*
)
<
/ ( * )

/ ( * ) •
Adding these two inequalities side by side we obtain the conclusion.
Convex functions
(hi) =4> (i) Let tuh }tut2[. Then
53
€ I, h < t2, A €]0,1[ and tx := (1  A)fi + Xt2 €
(1  X)f(t1) + Xf{t2) = (l\)[f(t1)f(tx)]
f(tx) +
X(f(t2)f(tx))
= (1  AJ/'faXt!  t A ) + A/'(r 2 )(i 2  tx) = A(l  A)/'(n)(ti  i 2 ) + A(l  A)/'(r 2 )(i 2  tO = A(lA)(i1i2)(/'(r1)/'(r2))>0, with Ti &]ti,t\[, r 2 €.]tx,t2[ (obtained by using the Lagrange theorem); of course, we have used the fact that / ' is nondecreasing. Suppose now that / is derivable of order 2 on J. In this case (iii) •& (iv) by a known consequence of the Lagrange theorem. • Theorem 2.1.10 Let I C R be an open interval and f : I —> R be a derivable function. The following statements are equivalent: (i) / is strictly convex; (ii) Vt,8el,t?8
: /'(*) • ( *  « ) < f(t) 
(iii) Vt,s £ I, t ^ s : (/'(*) — f'(s))
f(s);
• (t — s) > 0, i.e. f is increasing;
(iv) (if f is twice derivable on I) Vt E I : /"(*) > 0 and {t e I \ f"(t) — 0} does not contain any nontrivial interval. Proof. The proof is completely similar to that of the preceding theorem; just replace, where necessary, the inequalities by strict inequalities and, of course, use the properties of increasing functions. • Similar characterizations to those of the above theorems can be immediately formulated for concave and strictly concave functions. As immediate applications of the above two theorems we obtain that: (i) / i : R » R, f\(t) := \t\p, where p £]l,oo[, is strictly convex; (ii) / 2 : R+ —> R, / 2 (£) := tp, where p e]0,1[, is strictly concave and increasing; (iii) / 3 : P > R, f3(t) := tp, where p e R!_, is strictly convex and decreasing; (iv) fi : R > R, fi(t) : tp if t > 0, / 4 (t) := 0 if t < 0, where p e]l, oo[, is convex and nondecreasing; (v) /s : R > R, /s(£) := exp(i), is strictly convex and increasing; (vi) / 6 : P > R, / 6 (i) := lnt, is strictly concave and increasing; (vii) f7 : R+ > R, f7(t) := tint if t > 0, /r(0) := 0, is strictly convex; (viii) f% : R >• R, f%(t) := V l + 1 2 , is strictly convex. In the following two theorems we characterize the convexity of Gateaux differentiable functions defined on linear normed spaces.
Convex analysis in locally convex spaces
54
Theorem 2.1.11 Let D C (X, .) be a nonempty, convex and open set, and / : £ )  > E be a Gateaux differentiable function (on D). The following statements are equivalent: (i) / is convex; (ii) \/x,yED
:
(yx,Vf(x))
(hi) Vx,y£D
: (y  x, Vf(y)  V/(x)> > 0;
(iv) Va; G £>, Vu G X : differentiable on D).
V2/(:E)(M,U)
> 0 (when f is twice Gateaux
Proof. For x,y G D consider Ix
+ ty).
(2.18)
It is wellknown that ¥>*,»(*) = V/((lt)a:+*y)(j/a:),
<
W = V2
f{{lt)x+ty)(yx,yx) (2.19) for every t G / x ^ ; of course, the second formula is valid when / is twice Gateaux differentiable on D. (i) => (ii) Let x,y € D. By Theorem 2.1.1 we have that
0<(ts)(
s)x  sy, V / ( ( l  t)x + ty)  V / ( ( l  s)x + sy)).
Using Theorem 2.1.9 we obtain that ipX:y is convex, whence, by Theorem 2.1.1, / is convex. Suppose now that / is twice Gateaux differentiable on D. (i) =*> (iv) Let x € D and u € X. Taking into account that D is open, D  x is absorbing; hence there exists a > 0 such that y := x + au G D. From the hypothesis we have that ipx>y is convex, whence by Theorem 2.1.9, < Px,y(t) > 0 for every t G Ix>y. In particular < „ ( 0 ) = V 2 /(*)(t/ x,yx)=a2
V2f(x)(u,u)
> 0.
Convex
55
functions
Therefore the conclusion holds. (iv) => (i) Let x,y € D. For every t G Ix,y, using Eq. (2.19), we have
: (y  x, V/(z)) < f{y) 
(iii) Vx,y£D,x^y
f(x);
: (y  x, V/(y)  V/(x)) > 0;
(iv) \/x E D, V w £ l \ {0} : S72f(x)(u,u) Gateaux differentiable on D).
> 0 (when f is twice
Proof. Of course, using Theorem 2.1.1, we know that / is strictly convex if and only if the function
Let f G A(X) and x G d o m / . Then for every u e X
f(x, u) := lim ^ J v ' no
+
fa)
^}
t
= inf & t>o
+ tu)
' ™ t
G I,
(2.20) '
K
and VueX
: f'(x,u)
+ u)f(x),
(2.21)
the inequality being strict if f is strictly convex, u ^ 0 and f'(x,u) < oo. Moreover fix, •) is sublinear and dom f'(x, •) = cone(dom/ — x). If x G 4 (dom/) then f'(x,) is proper, while if x £ (dom/) 1 then f'(x,u) G K for every u € X. Proof. Let u £ X and rp : R > R, tp(t) := f(x+tu). Taking into account that VJ — ifiXtX+u (
Convex analysis in locally convex spaces
56
2.1.5 we obtain the existence of
^(Q)=lim^f^inf^f), i.e. Eq. (2.20) holds. Using again Theorem 2.1.5 we obtain Eq. (2.21), with the corresponding variant for / strictly convex and u ^ 0. It is obvious that f'{x,0) — 0 and that f'(x,Xu) = Xf'(x,u) for every u G X and A > 0. Let now u,v £ X. We have f(x + t(u + v)) = f (  ( x + 2tu) + \{x + 2tv)) < \f(x + 2tu) + \f{x + 2tv), and so f(x + t(u + v))  f(x) f(x + 2tu)f(x) t ~ 2t
f(x +
2tv)f(x) 2t '
Letting t ]. 0 we obtain V«,«£l
: f'(x,u
+ v) < f'(x,u)
+
f'(x,v).
Therefore f'(x, •) is sublinear. Note that dom/'(x, •) = E+ • ( d o m /  x) = cone(dom/ — a;). If a; € *(dom/) then dom f'(x,) = lin(dom/ — x), whence 0 € l (domf'(x, •)). From Proposition 2.1.4 we have that f'(x, •) is proper. If a; € (dom /)* then dom f'(x, •) = X and f'(x, •) is proper, which proves that f'(x,u) G M for every u £ X. • The number f'(x, u) G 11 is called the directional derivative of / at x in the direction u. It is possible that f'(x,) take the value — oo. Consider the function / : R > E, /(*) :=  ^ 1  i 2 if i < 1, / ( i ) := oo if i > 1; we have /'(—l,i) =  o o for every t > 0 (Exercise!). The preceding result can be extended in a significant way. Theorem 2.1.14 function
Let / G A(X), x G d o m / anrf e G M+. Then the
fe{x,) : *  > ! ,
^,«):=jnf / ( a ; +
to) /(3;)+e , t
is sublinear, dom /^(x, •) = cone(dom /  x),
(2.22)
Convex functions
57
and ft(x,u)
+ u)f(x)+£
= l i m fe{x,u)
V«6l,
= inf fe{x,u)
Furthermore, if x £ *(dom/) £/ien f'e{x,) i/ien f'e{x,u) € E /or every u £ l .
(2.23)
Vw £ X.
is proper, while if x £ ( d o m / ) 1
Proof. From the definition of f'e{x, •) it is clear that f'E(x,u) < oo if and only if there exists t > 0 such that x + tu 6 dom f, i.e. u £ cone(dom f — x). Hence Eq. (2.22) holds. It is obvious that f^(x,0) = 0 and for A > 0 fs{x,Xu)
= inf
—
A=
Xfe(x,u).
Let now u,v e X and s,£ > 0. Then / (x + ^ ( « + « ) ) = / ( ^ ( s + *u) + ^ ( z + tvj) <^rtf(x
+ su) + ^rtf(x
+ tv).
It follows that (f(x+£i(u
+
v))f(x)+s)/£ +t f(x + su) f{x)+e f(x + < ~ s
tv)f(x)+e t
Therefore, for all s, t > 0 we have /(a! + g u )  / ( g ) + g , / ( a +
fa)/(aQ+e
Taking the infimum in the righthand side, successively with respect to s and t, we obtain that £ ( x , u + i ; ) < / ; ( z , u ) + £(£,
< f'ei(x,u)
<
f'S2(x,u)
Convex analysis in locally convex spaces
58
hold, we obtain x ilimf. tu i . u \ = inf • f f'(x,u) tit \ = •mif inf + f f(
= inf inf / ( * + * " )  / ( * ) + £ t>0£>0 t = f'(x,u) for every u £ X. theorem.
tu
x £ )f(  )+
f(x + t u ) ~ t>0 t
= inf
f(x)
The other conclusions are proved like in the preceding •
The number f'e{x,u) € K is called the £directional derivative of / at x in the direction u. We end this section with a characterization of convex functions in arbitrary topological vector spaces using a generalized directional derivative. We shall give other characterizations in Section 3.2 using generalized subdifferentials. So, let X be a topological vector space and / : X > M a proper function. We define the upper Dini directional derivative of / at x 6 X in the direction « £ l b y Df(x
u\
_ / limsupt4.0t_1(/(a;ltu)/(x)) \ — oo
if
x € dom/, otherwise.
When / is convex and x G d o m / from Theorem 2.1.13 we have that Df(x,u) = f'(x,u) for all x € d o m / and u € X. For the proof of the announced characterization we need the following auxiliary result which is interesting in itself, too. Lemma 2.1.15 Let a, b 6 M, a < b, and (p : [a,b] —>• E be a lower semicontinuous proper function with
\
Because ip is lsc, it follows that ip(t) < ip(a) + a(t — a). Assume that t < b. Then, as above, there exists t" E]t,b] such that
Convex functions
59
ip(a) + a(i — a) + a(t — i)= ip(a) + a(t — o) for t £ [M"[, contradicting the choice of t. Hence t = b, and so desired inequality is proved. As the inequality tp(b) < tp(a) + a(b — a) holds for all a > 0, we obtain the contradiction ip(b) < f(a). The conclusion holds. • Theorem 2.1.16 Let X be a topological vector space and f : X >• R be a proper lower semicontinuous function. Then f is convex if and only if Df{x,yx)+Df(y,xy)<0 for all x, y £ X for which the sum makes sense. Proof. Assume first that / is convex. If x £ d o m / then Df(x,y — x) = —oo, and so Df(x, y — x)\ Df(y, x — y) equals — oo or does not make sense. Assume that x,y £ d o m / . Then Df(x,y
x) = f'(x,y
Df(y, xy)
x) < f(y) 
f(x),
= f(y, x  y) < f(x) 
f(y);
summing the inequalities side by side we get the conclusion. We prove the sufficiency by contradiction. Assume that / is not convex. Then there exists x,y € d o m / , x ^ y, and A g]0,1[ such that / ( ( l  X)x + \y) > (1  A)/(z) + Xf(y). Let
I be defined by
(*) := / (tx + (1  t)y)  tf(x)  (1 
tf(y), t)f(y).
Of course,
(1  A) = ifi(X), and D
 t)x + ty,yx)
+ f(x)  f{y)
^V(*. 1) = Df(tx + (1  t)y, x  y) + /(y)  f(x)
V t e [0, A[, Vt 6 [0,1  A[.
Prom the preceding lemma we get t\ € [0, A[ and ti £ [0,1 — A[ such that ~Dtp(ti,l) > 0 and 'Dip{t2,1) > 0. Taking u := (1  ii)z + ^ y and v := ^2^+(1 — ^2)2/, we have that u — v = s(x — y) with s := 1 —ti—*2 S]0,1]. Moreover, Df(u, v — u) + Df(v, u  v) = s1 (p
= 8 (D
+ Dil>(t2,l)) > 0 ,
f(y))
Convex analysis in locally convex spaces
60
a contradiction. The proof is complete.
2.2
•
SemiContinuity of Convex Functions
In this section X is a separated locally convex space if not stated explicitly otherwise. We begin with some characterizations of lower semicontinuous convex functions. Theorem 2.2.1
Let f : X —> R. The following conditions are equivalent:
(i) / is convex and lower semicontinuous; (ii) / is convex and wlower
semicontinuous;
(iii) epi / is convex and closed; (iv) epi / is convex and wclosed. Proof. It is wellknown that: / is lsc •£> e p i / is closed in I x I « [/ < M is closed VA £ 1. The equivalence of conditions (i)(iv) follows immediately applying Theorem 2.1.1. D In the sequel we shall denote by T(X) the class of proper lower semicontinuous convex functions on X. The following criterion for convexity is sometimes useful. Theorem 2.2.2 Let f : X —> K be a proper function satisfying the following conditions: (i) /(0) = 0, (ii) V i £ l , VA 6 P : /(Ax) = Xf(x), (iii) / is quasiconvex. Suppose that either (a) or (b) holds, where (a) V i £ X : f(x) > 0, (b) d o m / C cl{a; e X  f(x) < 0} and f is lsc at every point x G d o m / . Then f is sublinear. Proof. Note first that f(x + y) < f{x) + f(y) if x,y E d o m / and either f(x) • f{y) > 0 or 0 = f{x) < f(y). Indeed, if f{x) • f{y) > 0 then
whence f(x + y) < f(x) + f(y). If 0 = f(x) < f(y), for n £ N we have that J£T/(*
+ V) = f(nTTnx
+ n^iV) < max{f(nx),f(y)}
= f(x) + f(y).
Taking the limit for n —• oo we obtain again that f{x + y)
Semicontinuity
of convex
functions
61
Suppose that (b) holds. Let A := {x G X \ f(x) < 0} C d o m / . If x € d o m / \ ^ 4 , by hypothesis, there exists a net (a;i)iej C A, with (xi) —>• a;. Moreover, 0 < f(x) < \iminfieif(xi)
< limswpieIf(xi)
< 0,
whence (/(a;,)) —> /(a;) = 0. Therefore f(x) < 0 for every x € d o m / . Let x,y E d o m / . Since the roles of x,y are symmetric, we have to consider the following three situations: a) x,y £ A, ft) x,y G d o m / \ A and 7) x G d o m / \ A and 2/ G A. In the case a) we have that f(x)f(y) > 0, and so f(x + y) < f(x) + f(y). In the case /3) we have that f(x) = /(y) = 0 and a; + y G d o m / , whence f(x + y) < 0 = / ( x ) + / ( y ) . In the case 7) there exists a net (xi)iei C A such that (xi) —> a;; then (/(a:;)) —>• 0. Since a; + y G d o m / and /(a^ + y) < f(xi) + f(y) for every i e J [by a)], taking the limit inferior we obtain that f(x + y) < f(x) + f(y). • Corollary 2.2.3 Let f : X —> R be a quasiconvex, proper, positively homogeneous and Isc function, / + := / V 0 and g := f + LC\A, where A := {a; G X I f{x) < 0} U {0}. Then f+ and g are sublinear and f = / + A g. Proof. It is obvious that / + and g are quasiconvex, lsc and positively homogeneous. The subhnearity of /+ and g follows applying the preceding theorem. Because f(x) < 0 for x G clA, we have also that / — /+ A g. • A result of the same type is given by the following corollary. Corollary 2.2.4 Let f : X —> E be a quasiconvex and positively homogeneous function. If f is upper bounded on a neighborhood of the origin or dimX < 00, then f+ is sublinear. Proof. Suppose that there exists Ao > 0 such that [/ < Ao] is a neighborhood of 0. Since [/ < A] = ^ [ / < A0] for every A > 0, we have 0 G int[/ < A] for every A > 0. Let 0 < A < fi and x G cl[/ < A]. Since 0 G int[/ < A], from Theorem 1.1.2 we have that ^x G int[/ < A]. Hence x G f int[/ < A] = int (f [/ < A]) = int[/ < p] C [/ < »]. Therefore VA > 0 : cl[/ < A] C f 
\f
Hence [/ < A] is a closed set for every A > 0. Since [/ < 0] = HM>o[^ — lA> the set [/ < 0] is closed, too. Since [/+ < A] = [/ < A] for A > 0 and
Convex analysis in locally convex spaces
62
[f+ < A] — 0 for every A < 0, it follows that / + is lsc. Taking into account that / + is quasiconvex, /+ is sublinear (applying Theorem 2.2.2). Suppose now that dimX < oo. By hypothesis, [/ < 1] is convex and absorbing. Since dimX < oo, as observed at the end of Section 1.1, we obtain that 0 G int[/ < 1]. The conclusion follows as above. • Note that, under the conditions of the preceding theorem, /+ is continuous (see Theorems 2.2.9 and 2.2.21). A result similar to that of Proposition 2.1.4 is the following. Proposition 2.2.5 Let f : X —» E be a lsc convex function. If there exists XQ G X such that f(xo) — — oo, then f(x) = —oo for every x G dom / . In particular, if f is sublinear then f is proper. Proof. Suppose that there exists x G d o m / such that f(x) =: t G E. Then (x, t) G epi / and (xo,t — n) G epi / for every n G N. It follows that V n G N : ±(x0,t  n) + (1  ±){x,t) = (±x0 + *^x,t
 l) G e p i / ;
therefore (x,t — 1) G cl(epi/) = e p i / , whence the contradiction f(x) < f{x)  1. • Propositions 2.1.4 and 2.2.5 motivate the consideration, in the sequel, of (almost only) proper convex functions. It is useful to consider the lower semicontinuous envelope or lower semicontinuous regularization / := ci(ePi/) of the function / : X —> E (see Eq. (2.4)). Because cl(epi/) is closed we have that e p i / = cl(epi/). Theorem 2.2.6
Let f : X —> K be a convex function.
Then
(i) / is convex; (ii)
if g : X —> E is convex, lsc and g < f then g < f;
(iii) the function f does not take the value —oo if and only if f is bounded from below by a continuous affine function; (iv) if there exists xo G X such that f(xo) = — oo (in particular if f(xo) = — ooj then f(x) = —oo for every x G d o m / D d o m / . Proof, (i) Since e p i / = cl(epi/) and e p i / is convex, we have that e p i / is convex; hence / is convex. (ii) Let g be lsc, convex and g < / ; then e p i / C epig. Taking the closure of the sets, the conclusion follows.
Semicontinuity
of convex
functions
63
(iii) Suppose that / does not take the value — oo. If f(x) = oo for every x € X then f(x) > (x, 0) + 0 for every x € X. Consider dom / ^ 0 and let x € dom / . Then (x, t) fi epi / , where t := f(x) — 1. Since epi / is a convex, closed and nonempty set, using Theorem 1.1.5 we obtain (x*,a) € X* x E such that V (a;, t) € epi / : {x,x*) + at < (x, x*) + at. Taking x = x and t = f(x) +n, n £ N, we obtain that a < 0. Dividing, by —a > 0, we can suppose that a = — 1. Thus Vz e d o m / D d o m / :
f{x)>f(x)>(x,x*)+j,
where •y :—t— {x,x*}. Therefore / is bounded from below by a continuous affine function. Conversely, if f(x) > (x,x*) + 7 =: g(x), where x* G X* and 7 £ R, then g is convex and lsc; by (ii) we have that g < f. Therefore / does not take the value —00. (iv) If f(xo) =  0 0 , using the preceding theorem, f(x) — —00 for every x 6 d o m / . It is obvious that d o m / D d o m / . • Proposition 2.2.7 Assume that f : X —> ffi is sublinear. sublinear o / is lsc at 0 •£> / is proper.
Then f is
Proof. Because e p i / = cl(epi/) is a convex cone, the first equivalence follows by Proposition 2.1.2. If / is sublinear, by Proposition 2.2.5 we have that / is proper. Conversely, if / is proper then 0 > /(0) > —00 which implies that 7(0) = 0. • To an arbitrary function / : X —> E one associates, naturally, a lower semicontinuous and convex function; this function is denoted by co/ and has the property that epi(co/) = cl(co(epi /)) and is called the lsc convex hull. It is obvious that co/ < / < / . An application of the lsc convex hull of a function is the property in the next example used in the study of HamiltonJacobi equations. Example 2.2.1 Let X be a locally convex space, / € r ( X ) , g : X > E be a proper function and a, 0 > 0. Then co(co(f+ag)+/3g) =co(f+(a+/3)g). (See the solution of Exercise 2.19 for details.) Related to the upper semicontinuity of a convex function, we remark that when / : X > E is upper semicontinuous (use for short) at x0 € dom / ,
64
Convex analysis in locally convex spaces
f is upper bounded (by a real constant) on a neighborhood of xo; therefore xo G int(dom/) 7^ 0. In the sequel we prove that the converse is true for every convex function. Let us first establish the following auxiliary result. Lemma 2.2.8 Let f G A(X) and x0 G d o m / . Suppose there exist U G J4CX and m Gffi__such that VxGx0
+ U : f(x) < f(x0) + m.
(2.24)
Then Vxexo + U : \f(x)f(x0)\<mpu(xx0),
(2.25)
where pi; is the Minkowski gauge ofU. In particular f is continuous atXQ. Proof. Replacing / by g : X >• R, g(x) = f(x0 + x) — f{x0), we may suppose that xo = 0 and f(xo) = 0. Therefore f(x) < m for every x € U. Prom Proposition 1.1.1 we have that pu is a continuous seminorm and U = {x G X I pu{x) < 1}. Let x 6 U. Suppose first that t :— pu{x) > 0. Then y :— t~1x € U. Since / is convex and t < 1, we obtain that f(x) < tf{y) + (1  t)f(0) < tm = mpu{x). Suppose now that pu{x) = 0. Then nx G U for every n € N. It follows that f(x) < ±f(nx) + ^ / ( O ) < \m for every n 6 N. Therefore, once again, f(x) < mpu(x). Since 0 = / (\x' + \{x')) < \f{x') + y(x'), we obtain that f(x') < f(x') for every x' e X. Using this inequality we obtain that \f(x)\ < mpu(x) for every x eU. Therefore Eq. (2.25) holds. • Using the preceding lemma we get the following important result. Theorem 2.2.9 / / the convex function f : X —> K is bounded above on a neighborhood of a point of its domain then f is continuous on the interior of its domain. Moreover, if f is not proper then f is identically  c o on int(dom/). Proof. Suppose that there exist Xo £ d o m / , V G N(xo) and m G M such that f(x) < m for every x G V. Then V C d o m / , and so xo G int(dom / ) 7^ 0. If / takes the value  0 0 then, by Proposition 2.1.4, f(x) = —00 for every x G int(dom/). Therefore the conclusion holds in this case. Let / be proper and x G int(dom/). Then there exists /J, > 0 such that xi := (1 + fi)x  nx0 G dom / . Taking A := (1 + p,)~l G ]0,1[, we have that / (An + (1  X)x) < A/(xi) + (1  \)f(x)
< \f(Xl)
+ (1  A)m =
: m i
6R
Semicontinuity
of convex
functions
65
for every x € V. But V\ := Xxi + (1 — A)V £ N(x). Applying the preceding lemma we obtain that / is continuous at x. • Corollary 2.2.10 Let f : X —>• E be a convex function. Then f is continuous on int(dom/) if and only i/int(epi/) is nonempty in X x E. Proof. Suppose that / is continuous at some XQ £ int(dom/). Then, for m &]f(xo),oo[, there exists U £ N x such that f(x) < m for every x e xo + U. So (xo, m + 1) G int(epi/) because (xo + U) x [m, oo[ C e p i / . Conversely, assume that (xo,M £ hit(epi/); then there exist U £ N x and e > 0 such that (xo + U) x [t0 — e, t0 + e[ C epi / , and so f(x) < m := t0 + e for every x 6 xo + U. By Theorem 2.2.9 we have that / is continuous on int(dom/) ( ^ 0). • Under the conditions of Lemma 2.2.8 we have a stronger conclusion. Theorem 2.2.11 Let f € A(X) and XQ £ d o m / . Suppose that there exist U € J^cx and m 6 E+ suc/i t/iat condition (2.24) is satisfied. Then for every p e]0,1[, Vx,y £x0+pU
: /(cc)  f(y)\ < mpu(xy). 1p Proof. As in the proof of Lemma 2.2.8, we may assume that xo = 0 and /(a;o) = 0. Let p e]0,1[ and consider x,y 6 pU. We consider two cases: a) pu{x — y) ^ 0 and b) pu{x — y) = 0. In the proof we shall use the fact that pu is a continuous seminorm, U = {x \ pu{x) < 1} and intf/ = {x  pu{x) < 1} (see Proposition 1.1.1). a) Let
tpu{x — y) — pu(—y), and so limt_,.oo ¥>(*) = oo. As (p(l) = pu(x) < P < 1, there exists t > 1 such that ¥>(£) = 1. Then 2 := te+(lt)j/ £ U. It follows that x = (li~1)y + i1z, and so f(x) < (1 — i~1)f(y) + i~1f(z). From Lemma 2.2.8 we have that \f(y) I < mpu(y) < wp. Therefore /(*)  f(v) < ^
(m
 f(y)) < * _ 1 (m + mp).
But i{xy)  zy, whence ipu(xy)  pu(zy) >Pu(z)pu(y) > 1p. The preceding inequalities imply that fix)  f(y) < m\^Pu{x 1p
 y).
(2.26)
66
Convex analysis in locally convex spaces
b) In this case we have, using again Lemma 2.2.8, that f(t(x — y)) = 0 for every t £ i So, WG]0,1[
:
f(tx)=f(ty+(lt)J^(xy))
Since x G int U C int(dom / ) , by Theorem 2.2.9, / is continuous at x. From the above inequality we obtain that f(x) < f(y) taking the limit for t —> 1. Hence (2.26) also holds in this case. The conclusion follows from Eq. (2.26) changing x and y. • Corollary 2.2.12 Let (X, ) be a normed space and f G A(X). Suppose that XQ G dom / and for some p > 0 and m > 0, Vz G D(x0,p)
: f(x) < f(x0) + m.
Then Vp'e]0,p[,Vx,yeD(xQ,p')
: \f(x)  f(y)\ <  • P^ • \\x  y\\. p pp'
Proof. Consider U := D(0;p) — pUx Then pu{x) = p _ 1 x for any x 6 X. The conclusion is immediate from the preceding theorem. • The conclusion of Theorem 2.2.11 says, in fact, that / is Lipschitz on a neighborhood of xo • We say that the function / : X —> K is Lipschitz on a set A C X if / is finite on A and there exists a continuous seminorm p on X such that \f(x) — f(y)\ < p(x — y) for all x,y E A; we say that / is locally Lipschitz on A if for every x G A there exists a neighborhood V of a: such that / is Lipschitz on V. Corollary 2.2.13 If f £ A(X) is bounded above on a neighborhood of a point of its domain then f is locally Lipschitz on the interior of its domain. Proof. Let x G int(dom/). Applying Theorem 2.2.9 we obtain that / is continuous at x, and so / is bounded above on a neighborhood of a;. Applying now Theorem 2.2.11 we obtain that / is Lipschitz on a neighborhood of a;. • Recall that in Theorem 2.1.5 we have already proved that a function / G A(E) is locally Lipschitz on int (dom/), while in Proposition 2.1.6 we proved that /dom/ is continuous if / is, moreover, lsc. Another consequence of Theorem 2.2.9 is the following result.
Semicontinuity
of convex
functions
67
Corollary 2.2.14 Let ft 6 A{X) forl
( dom / „
and either f (resp. g) is identically —oo on int(dom/) (resp. int(dom<7),); or f (resp. g) is proper and continuous on int(dom/) (resp. int(domg)^.D Recall that ftOfa and /1O/2 are defined in Theorem 2.1.3. Prom Theorem 2.2.9 (or Corollary 2.2.10) we obtain that g is continuous on int(domp) if f,g are convex, g < f and / is continuous on int(dom) (supposed to be nonempty). A similar result is true for a larger class of convex functions. The convex function / : X —• E is said to be quasicontinuous if aff(dom/) is closed and has finite codimension (i.e. its parallel linear subspace has finite codimension), rint(dom/) 7^ 0 and / a ff(dom/) ls continuous on rint(dom/). The set A C X is quasicontinuous if LA is quasicontinuous; it follows that A is quasicontinuous exactly when aff A is a closed affine set of finite codimension and rint^4 7^ 0. The following result holds. Proposition 2.2.15 Let f,g : X —>• E be convex functions such that g < f. If f is quasicontinuous, then g is quasicontinuous, too. Proof. Without loss of generality we assume that 0 S dom / and /(0) < 0. Then aff (dom / ) is a linear subspace and Y0 := aff (epi / ) = aff (dom / ) x E. Of course Y0 has finite codimension and is closed. Moreover, by Corollary 2.2.10, we have that inty 0 (epi/) 7^ 0. Since e p i / C epig, we have that inty 0 (y 0 nepip) ^ 0. It follows (see Exercise 1.3) that rint(epi5) 7^ 0. Since aff (epi g) = aff (dom g) x E, using again Corollary 2.2.10, we obtain that g aflf(domp) i s continuous, and so g is quasicontinuous. • Applying the preceding result to indicator functions one obtains Corollary 2.2.16 B.
Let A C B C X. If A is quasicontinuous then so is D
There are several classes of convex functions, larger than the class of lower semicontinuous convex functions, which will reveal themselves to be useful in the sequel. We introduce them now. Let / : X —> E. We say that / is csconvex if
E,
n
.* , ™
Afc/(xfc) fc=l
68
Convex analysis in locally convex spaces
whenever J2n>i ^nxn is a convex series with elements of X and sum x € X. Of course, if / is csconvex then / is convex, while if / is lsc and convex, / is csconvex (Exercise!). Also, we say that / is ideally convex, bcscomplete, csclosed, cscomplete, liconvex or lcsclosed if epi / is ideally convex, bcscomplete, csclosed, cscomplete, liconvex or lcsclosed, respectively. Of course, taking into account the relationships among these notions for sets and Proposition 2.2.17 (i) below, we have: / lsc, convex
=>•
/ scconvex
af cscomplete =>• / csclosed =>• f lcsclosed ^ ^ ^ / bcscomplete =>• f ideally convex => / liconvex
=> f convex,
the reversed implications being not true, in general. Taking into account Proposition 1.2.3 and that for / : X » R we have [/ < A] = Fix (epi/ n (Xx }  oo, A]), [ / < A] = P r x ( e p i / n ( X x ]  co, A[), the sets [/ < A] and [/ < A] are liconvex (lcsclosed) for every A € R if / is liconvex (lcsclosed). Let A C X; since epi LA = A X R+ and ffi is a Frechet space, we have that A is ideally convex (bcscomplete, csclosed, cscomplete, liconvex, lcsclosed) if and only if i& is so. Remark 2.2.1 When
R is a continuous affine functional (that is
R is csconvex (ideally convex, csclosed) if and only if / + is so (Exercise!). Proposition 2.2.17
Let f, g : X —>• R have nonempty
domains.
(i) / / / is csconvex then f is csclosed. Conversely, if f is csclosed and is minorized by a continuous affine functional then f is csconvex. (ii) / / / and g are ideally convex (resp. csclosed) and are minorized by continuous affine functionals then f + g is ideally convex (resp. csclosed). Moreover, if g is bcscomplete (resp. cscomplete) then f + g is bcscomplete (resp. cscomplete). Proof. Taking into account the Remark 2.2.1 above, when / or/and g is bounded from below by a continuous affine function we may assume that even / > 0 or/and g > 0.
Semicontinuity
of convex
functions
69
(i) The proof of the first part is immediate. Suppose that / is csclosed and / > 0. Let X) n >i ^nXn be a convergent convex series with elements of X and sum x G X. Since / is convex (epi/ being convex), we may suppose that An > 0 for every n £ N; moreover, we may assume that xn G dom / for every n. Because f(xn) > 0, there exists r := l i m n  ^ X)*!=i ^kfi^k) G EU{oo}. If T < oo, because (xn,f(xn)) € e p i / and J2n>i ^n{xn,f{xn)) = (X,T), and / is csclosed, we have that (x,r) £ e p i / , whence f(x) < r. If T = oo, the preceding inequality is obvious. Hence / is csconvex. (ii) We prove only the second part of the "ideally convex" case, the rest of the proof being similar. So, let / be ideally convex, g be bcscomplete and / , g > 0. Let J2n>i ^n(xn,rn) be a Cauchy bconvex series with elements of epi(/ + g). Then rn = sn + tn with (xn,sn) G e p i / n and (xn,tn) G epig for every n. Because 0 < sn,tn < rn, we have that (sn), (tn) are bounded, s := ^2n>i ^nSn € 1 + , t := 5 2 n > 1 Xntn G M+ and r = s + t. Because g is bcscomplete we obtain that the bconvex series ^2n>1Xn(xn,tn) with elements of epi g is convergent with sum (x,t) £ epig for some x G X. Hence the bconvex series Yln>i ^n{xn,sn) with elements of e p i / is convergent with sum (x, s) G e p i / . Therefore (x,r) G epi(/ + g). Thus f + g is bcscomplete. • The classes of liconvex and lcsclosed functions have good stability properties. Let us begin with the following characterizations. Proposition 2.2.18 Let / : X » K have nonempty domain. Then the following statements are equivalent: (i) / is liconvex (resp. lcsclosed); (ii) epi s / is liconvex (resp. lcsclosed); (iii) there exist a Frechet space Y and an ideally convex (resp. csclosed) function F : X x Y » E such that f(x) = inf yC y F(x,y) for every x G X (i.e. f is the marginal function associated to an ideally convex (resp. csclosed) function). Proof. We prove the "liconvex" case, the proof for the other case being similar. (i) => (ii) Taking % § : X =} E such that gr 31 = epi / and gr S = X x P, we have that "R and S are liconvex multifunctions. Since E is a Frechet space, using Proposition 1.2.5 (iv), epi s / — e p i / + {0} x P = gr(IR + §) is liconvex.
70
Convex analysis in locally convex spaces
(ii) =$• (i) Since epi,, / is liconvex, as above, the set An := epi s / + {0}x ] — i,oo[ is liconvex for every n 6 N. Since e p i / = f]n&NAn, from Proposition 1.2.4 (i) we obtain that e p i / is liconvex. (i) => (iii) Since / is liconvex, there exist a Frechet space Y and an ideally convex set A C Y x X x E such that e p i / = PIXXR(A). Consider the function F: XxYxR^R defined by F(x,y,r) := r + iA(y,x,r). Then f(x) = inf ( a . i r ) e e p i / r = i n f ^ ^ ^ r = inf ( j, ) r ) e y x R F(a;,j/,r) for every x £ X. Since A is ideally convex, so is LA] using Proposition 2.2.17 (ii) and Remark 2.2.1 we obtain that F is ideally convex. The conclusion follows because Y x E is a Frechet space. (iii) =>• (ii) Let f{x) = inf y e y F(x, y) for every a; € X, where Y is a Frechet space and F is an ideally convex function. From (i)=^(ii) it follows that epi s F is liconvex. Then, by Eq. (2.8), epi s / is liconvex. • Other useful properties of liconvex and lcsclosed functions are collected in the following result. Proposition 2.2.19 (i) / / / „ : X > E is liconvex (resp. lcsclosed) for every n E N , then s u p n e N / „ is liconvex (resp. lcsclosed). (ii) If fi,h '• X » E are liconvex (resp. lcsclosed) functions and A € E+, then / i + / 2 and A/i are liconvex (resp. lcsclosed). (iii) If F : X xY ^ R is liconvex (resp. lcsclosed) and X is a Frechet space, then h : Y —»• R, h(y) := inixex F(x,y), is liconvex (resp. lcsclosed). (iv) Let Y be a Frechet space and g : Y —> R. If Q C Y is a convex cone, H : X —>• (Y',Q) has liconvex (resp. lcsclosed) epigraph and g is liconvex (resp. lcsclosed) and Qincreasing, then g o H is liconvex (resp. lcsclosed). In particular, if A : X —> Y is a linear operator with liconvex (resp. lcsclosed) graph and g is liconvex (resp. lcsclosed), then g o A is liconvex (resp. lcsclosed). (v) / / X is a Frechet space, A : X —> Y is a linear operator with liconvex (resp. lcsclosed) graph and f : X —>• E is liconvex (resp. lcsclosed), then Af is liconvex (resp. lcsclosed). (vi) If X is a Frechet space and / i , / 2 : X > E are liconvex (resp. lcsclosed) functions then / i D / 2 and /1O/2 are liconvex (resp. lcsclosed).
Semicontinuity
of convex
functions
71
Proof. Again, we treat only the "liconvex" case. (i) Because epi (sup n € N /„) = ri n eN e Pi/™' * n e conclusion follows using Proposition 1.2.4 (i). (ii) Taking % : X =t R, g r # i := epi/j (i = 1,2), we have that epi(/i + / 2 ) = gr(3?i +3?2) The conclusion follows from Proposition 1.2.5 (iv). Also, epi(A/i) = T(epi/i) for A > 0, where T : X x E  > X x R i s the isomorphism of topological vector spaces given by T(x, t) = (x, Xt); hence A/i is liconvex in this case. If A = 0, A/i = tdom/i But d o m / i = Prjt(epi/i), and so d o m / i is liconvex by Proposition 1.2.3, whence 0/i is liconvex. (hi) We have, by Eq. (2.8), that epi s / = Pry X R(epi s .F). Since episF is liconvex and X is a Frechet space, we get from Proposition 1.2.3 that epi s / is liconvex. (iv)(vi) Using the same constructions as in the proofs of Theorem 2.1.3 (vi), (viii) and (ix), the conclusions follows from (iii). • If X is a barreled space, the lower semicontinuity of a convex function (even weaker conditions) ensures its continuity on the interior of its domain. Theorem 2.2.20 Let X be a barreled space and f : X —> E be convex. Suppose that either (a) X is first countable and f is liconvex or (b) / is lower semicontinuous. Then ( d o m / ) ' = int(dom/) and f is continuous on int(dom/). Proof, (a) By Proposition 2.2.18, there exist a Frechet space Y and a liconvex function F : X xY > R such that f(x) = inf y € y F(x, y) for every x £ X. Consider the multifunction X : 7 x l r j I with gr3? := {(y,t,x) \ (x,y,t) G epiF}. Of course, Ji is ideally convex and ImR = d o m / . Let xo € (dom f)1 (if this set is nonempty) and {ya,tQ) € 3l _1 (a;o). Applying Simons' theorem (Theorem 1.3.5), we have that U := R(Yx ]  00, t0 + 1[) is a neighborhood of x0. So, for every x € U there exist y 6 Y and t < to + 1 =: m such that f(x) < F(x,y)
72
Convex analysis in locally convex spaces
Proof. By Proposition 1.2.1 we have that e p i / is csclosed, and so / is lcsclosed. The conclusion follows from the assertion (a) of the preceding theorem. • The application of Corollary 2.2.12 and Theorem 2.2.20 yields the following uniform boundedness principle for convex functions. Theorem 2.2.22 Let X be a Banach space and C C X be an open convex set. Consider {fi)i^i o, nonempty family of continuous convex functions from C into E. If (fi{x)).j is bounded for every x £ C then for every x £ C there exist rx,Lx > 0 such that Ux := x + rxllx C C and \fi(y) — fi(z)\ < Aclly  z\\ for all y,z £ Ux and all i £ I (i.e. (fi) is locally equiLipschitz on C). Proof. By hypothesis, for every x £ C there exists Mx £ E+ such that \fi(x)\ < Mx for all i £ J. Let r'x > 0 be such that U'x := x + r'xUx C C and consider Fi : X > E be denned by Fi{y) := fi(y) for y £ U'x and Fi(y) := oo otherwise. Then Ft £ T(X). Consider F := supieI Ft E T(X). The hypothesis shows that d o m F = Ux. By Theorem 2.2.20 we obtain that F is continuous on int(dom.F) — x + rxBx Therefore there exist r 'x £]°><[ and M > 0 such that F(y) < M for all y G x + r'J,Ux. Then for y G x + r'J.Ux and i G / we obtain that ft(y)  fi(x) < M — (Mx) = M+Mx =: L'x. Using now Corollary 2.2.12 we have that fi is Lipschitz with constant Lx :~ ZL'x/r'^ o i i x + rxUx, where rx := r'^/2. The conclusion follows. • Taking X a Banach space, Y a normed space, and {Xi  i G 1} C £ ( X , Y) (I 7^ 0) with {TiX \ i £ 1} bounded for every x £ X, the conditions of the preceding theorem are satisfied by the family of functions (fi)iei, where U{x) := Ti(a:), and C := X. Hence \\Ti(x)\\ = \h(x)  / 4 (0) < Lx for x £ rllx and i £ I, for some r,L > 0; hence \\Ti\\ < L for every i £ I. So we obtained the classic uniform boundedness principle in Functional Analysis. From the preceding result we obtain that pointwise convergence implies uniform convergence on compact sets. Corollary 2.2.23 Let X be a Banach space and C C X be an open convex set. Consider (/„) a sequence of continuous convex functions defined on C with values in E such that (fn(x)) > f(x) for every x £ C with f : C —> E. Then (/„) converges uniformly to f on the compact subsets ofC
Semicontinuity
(or, equivalently, (fn{xn)) to x E C).
of convex
73
functions
> f(x) for every sequence (xn) C C converging
Proof. First of all observe that / is convex (by Theorem 2.1.3) and locally Lipschitz (by the preceding theorem). Let x,xn E C (n E N) be such that (x„) > x. By Theorem 2.2.22 there exist r, L > 0 such that Ux := x + rUx C C and \fn{y)  fn(z)\
 fn(x)\ +
+ \fn(x)  f(x)\ \fn(x)f(x)\
for n > nx, whence (fn(xn))n *• f(x). Assume now that there exists some compact subset K of U such that (/„) does not converge uniformly to / on K. Then there exist e > 0, P C N an infinite set and a sequence (xn)nep C K such that \fn(xn) — f(xn)\ > £• Since K is compact, we may assume that (xn) > x 6 K...C C. Then, as shown above, {fn{xn))p > f{x). Since / is continuous, we have also that (f(xn)) • p —>• f(x), which yields the contradiction 0 > e. The next result corresponds to a known theorem for continuous linear operators. Proposition 2.2.24 Let X be a Banach space and C C X be an open convex set. Assume that / , / „ : C —> M. (n € N) are continuous convex functions. Then (fn(x)) —> f(x) for every x 6 C if and only if (a) (fn{x)) is bounded for every x G C and (b) (fn(x)) > f{x) for every x € D for some dense subset D of C. Proof. The necessity is obvious. Assume that conditions (a) and (b) above are satisfied but (fn{x)) does not converge to f(x) for some x E C. Hence there exist e > 0 and P C N an infinite subset such that \fn(x)  f(x)\ > e for every n E P. By Theorem 2.2.22 we find r,L > 0 such that Ux := x + rUx C U and / , / „ (n E N) are LLipschitz on Ux. Since D is dense, there exists (xk) C D converging to x; we assume that Xk E Ux for every fcgl Then for n E P and k E N we have S < \fn(x)
 f(x)\
< \fn(x)
< 2L \\xk  x\\ + \fn{xk)
 fn(Xk)\

f{xk)\.
+ \fn(xk)
~ f(xk)\
+ \f(xk)

f(x)\
Convex analysis in locally convex spaces
74
Fixing k G N such that \\xk  x\\ < e/(4L), we get /„(x fc )  f(xk)\ for n G P, contradicting that (fn(xk)  n G N) converges to f(xk).
> e/2 •
Let / G T(X); we call the recession function of / the function /oo : X —>• IK whose epigraph is ( e p i / ) ^ . Let XQ G d o m / ; taking into account formula (1.3), we have that (u, A) G epi/oo < » V i > 0 : (x 0 ,/(x 0 )) + £(u, A) G e p i /
/(x 0 + fa)  /(x 0 ) ^
<
Jim
/(X 0 + fa)  / ( x 0 )
tSOO
£
= gup
/(Xp + fa) 
(>0
/(XQ) < A
t
~
Therefore VtiSl
: /oo(u) = lim t—>oo
7
;
(2.27)
r
thus /oo(u) >  o o for every u G X and /oo(0) = 0. Because epi/oo is a closed convex cone we have that f^ is a lsc sublinear functional. The preceding relations show that f(x + u) < /(x) + /oo(u)
V x G d o m / , Vu G X,
[/ < A]^ = {u e X  /«,(«) < 0} V A G E w i t h [ / < A ] # 0 ,
(2.28) (2.29)
when / € T(X). Taking into account the discussion on page 6, relation (2.29) shows that the function / € T(X) has bounded level sets when dimX < oo. This result is no longer true when dimX = oo; take f.i. / := PA, the Minkowski functional associated to the set in Example 1.1.1 (/oo = / in this case). Remark 2.2.2 Note that, for / G F(X) and x G X, the mapping t >>• / ( x + fa) is nonincreasing from E into E when /oo(u) < 0; in particular d o m / + E+u = d o m / . Moreover, if /<»(«) < 0 and /oo(~w) < 0 then f(x + tu) = f{x) for all x G X and t G E. Indeed, let ti < t2. If / ( x + t\u) < oo then / ( x + t2u)  f(x + tiu) _ f(x + tin + (t2  h)u)  f(x + tiu) t2 —t\ t2 — *i
Conjugate
75
functions
and so f(x + t2u) < f(x + t\u); the inequality is obvious if f(x + tiu) = oo. It follows that d o m / + R+u = d o m / . When /«>(«) < 0 and /oo(w) < 0 we can apply the previous result for u and — u and obtain that the mapping t H> f(x + tu) is constant.
2.3
Conjugate Functions
In this section X and Y are separated locally convex spaces if it is not stated explicitly otherwise. Let / : X —> M; the function f*:Xm+%
f*{x*):=suv{(x,x*)f(x)\xeX},
(2.30)
is called the conjugate or Fenchel conjugate of / . Note that if there exists XQ 6 X such that /(xo) = ~°o then f*(x*) — oo for every x* € X*, while if f(x) = oo for every x then f*{x*) =  o o for every a;* £ X*. When / is proper (but also for / not proper, using the convention inf 0 = oo), we have f*(x*)
= sup{(x,a;*)  f(x)  x € d o m / } .
The conjugate of a function h : X* > K. is defined similarly: /i* : X  > K ,
/i*(a;) := sup{(a;,a;*)  h{x*) \ x* e X*}.
In fact, considering the natural duality between X and X*, it is the same definition; this distinction is useful in the case of normed spaces. The above remark concerning /* is also valid for h*. In the following theorem we establish several simple properties of conjugate functions. Theorem 2.3.1
Let f,g
: X > I , h : Y >• 1 , k : X* )• I and
A€l(X,Y). (i) /* is convex and w*lsc, k* is convex and Isc; (ii)
the YoungFenchel
inequality
V x e l . V i ' e r
below holds:
: f{x) +
f*(x*)>(x,x*);
(iii) f<9^9*
76
Convex analysis in locally convex spaces
(v) if a > 0 then (af)*(x*) = af'ia^x*) for every x* G X*; if/3 ^ 0 tfien (f(l3))*(x*) = f'ifi^x*) for every x* G X*; (vi) if g(x) = f(x + x0) for x G X, then g*(x*) = f*(x*) every x* G X*;
 {x0,x*)
for
(vii) if x*0 G X* then (/ + x*0)*{x*) = f*(x*  x*0) for every x* G X*; (viii) if f, h are proper and $ : I x 7 > 1, $(x, y) := f(x) + h(y), then $*(x*,y*) = f*(x*) + h*{y*) for all (i*,y*) E X* x Y*; (ix)
{AfY
=f*oA*
and (fDg)* = f* + g*.
Proof, (i) If / is not proper, we have already seen that / * is constant, and so / * is convex and w*continuous. If / is proper we have that / * = su Pzedom/ *Px, where ipx : X* *• R, ipx(x*) := {x,x*)  f(x). It is obvious that for every x G d o m / , (px is affine (hence convex!) and u>*continuous (hence w*lsc!). Therefore / * is convex and u;*lsc. For the statement about h we use the same arguments. (ii) By Eq. (2.30) we have VxeX,Vx*
eX*
: /'(a;*)>
which gives immediately the YoungFenchel inequality. (iii) is an immediate consequence of the definition and of the relation f<9_ _ (iv) We already remarked that co/ < f < f, whence, using (iii), we get / * < / * < (co/)* Let x* G X* and a G R be such that f*(x*) < a. Then (x,x*) — f(x) < a for every x G X, whence ip(x) := (x,x*) — a < f(x) for every x. Since tp G T(X), and epi(p D e p i / , we have that epiy> D epi(co/) = co(epi/). Therefore (f(x) < (cof)(x) for every x G X, and so (x,x*) — cof(x) < a for every x; hence (co/)*(a;*) < a. Thus / * = / = (co/)*(v), (vi), (vii) and (viii) are immediate. (ix) We have (Afy(y*)
= sup{(y,y*)  (Af)(y)] = sup ((j,,**) inf f(x)) y£Y y£Y V {x\Ax=y} J = sup{(y,y*)  f{x) \(x,y)£X =
x Y, Ax = y}
sup{(x,A*y*)f(x)\x£X}
= f*(A*y*) =
(roA*)(y*).
Similarly one proves the corresponding relation for (fDg)*.
D
Conjugate
77
functions
I the sequel we denote by T*(X*) the class of those functions in A(X*) which are w*lower semicontinuous. The discussion at the beginning of this section and the preceding theorem yields the next result. Corollary 2.3.2
Let f : X > E and h : X*  • E. Then
(i) /* € T*(X*) <S> d o m / ^ 0 and 3x* G X*, a G K, Vrr G X : f(x) > (x,x*) + a ; (ii) h* G r ( X ) <S> dom/i / 8 and 3 i 6 I , a e E, Va:* £ X* : /i(:r*) > (x,a;*) + a. • The following result is fundamental in duality theory. Theorem 2.3.3 and f **:=(/*)*
Proof.
(of the biconjugate) Let f e T(X).
Then f* € T*{X*)
= f.
Applying Theorem 2.2.6 we get x$ G X* and a Gffisuch that V i e l
: / ( i ) > (x,x*0)+a.
(2.31)
Thus, by the preceding corollary, we have that /* G T*(X*). Moreover, Theorem 2.3.1 shows that /** < / . Let x € X be fixed and consider t G E such that t < f(x); therefore (x, t) ^ e p i / . Applying Theorem 1.1.5 for {(x,t)} and e p i / , there exist (x*,a) G X* x E and A G E such that V ( z , i ) G e p i / : (x,x*) + ta < A < (x,x*) + ta.
(2.32)
Taking (x, t) = (x, f(x) + n), n G N, with x G d o m / , we obtain that V n G N : (x,x*) + af(x) + na < A < (x,x*) + ta. Letting n > oo we obtain that a < 0. Take first a < 0. Dividing eventually by —a > 0, in Eq. (2.32) we can suppose that a = —1. Thus (x,x*)  f(x) < A V i £ d o m / . Hence /*(#*) < A < (x,af)  <, and so
t<(x,x*)r(r)
78
Convex analysis in locally convex spaces
This together with Eq. (2.31) yields f(x) > (x,
XQ)
+ a > (x,
XQ)
+ a + t(x, x*) + tc — t(x, x*)
for all x € dom / and all t > 0; this implies successively: MxeX,
V i > 0 : tc + t(x,x*)  a> (x,XQ+tx*) V £ > 0 : tc + t{x,x*) a
V i > 0 : f**(x)

f(x),
> f*(x*Q + tx*),
> (x,x*0+tx*)f*(x*0+tx*)
> a+
tc+(x,x*0).
But there exists t > 0 such that a + tc + (x, a;*,) > t; hence f**(x) > t in this case, too. Thus we obtained f**(x) > f{x). Therefore /** = / . • The preceding theorem shows that for any function / : I > 1 we have ((/*)*) = /* (for the duality (X,X')) which shows that there is no interest to consider conjugates of order greater than 2. It also shows that the conjugation is an isomorphism between T(X) and T*(X*). More precise information on the biconjugate of an arbitrary function is furnished by the following result. Theorem 2.3.4 (i) —oo.
Let f : X —>•ffihave nonempty domain.
/ / co/ is proper, then /** = c o / ; if co/ is not proper, then /** =
(ii) Suppose that f is convex. If f is Isc atxG d o m / , then f(x) /**(x); moreover, if f(x) € R, then /** = / and f is proper.
=
Proof, (i) The function co/ is convex and lsc. If co/ is proper, using the preceding theorem and Theorem 2.3.1(iv), we have that
co/= (co/r = (/•)* = /**• If co/ is not proper, since dom(co/) D dom / / 0, co/ takes the value  c o ; hence /* = (co/)* = oo, and so /** = —oo. (ii) Taking into account that / is convex, co/ = / . Since / is lsc at x, we have that f(x) = f(x). It is obvious that f**(x) = f(x) if f(x) =  o o . Let f(x) £ M; then f(x) € E, and so / is proper. From the first part we have that /** = J, whence f**(x) = J(x) = f(x). • Corollary 2.3.5 Let f,g £ A(X). If fUg is proper, then (/* + g*)* = fOg = fOg~, while in the contrary case (/* +
: (fDg)(x)
=
(f*+g*)*(x).
The sub differential of a convex
Proof.
function
79
From Theorem 2.3.1 we have that
uogr = r+g* = T+r = (f^9)*\ since /•
sA(x*) :=sup{(x,x*)\x
&A};
(2.33)
for 0 ^ B C X* the support function SB : X —> E is defined similarly. It is obvious that SA is w*lsc and sublinear while SB is lsc and sublinear. Furthermore, if C C X is another nonempty set then SA+C — $A + $c and SAUC — SA V sc Note that (IA)*(X*)
= sup(a;,a;*) = sA(x*) = sup (x,x*) = x£A
(LCOA)*(X*)
= ScoA
X€COJ4
Moreover we have that icoA — cotyi and dom SA = dom(tJ4)* = {x* € X*  x* is upper bounded on A}; therefore doms^ = X* if and only if A is u;bounded. In the next section we shall see that LA is useful in determining the normal cone of C. 2.4
The Subdifferential of a Convex Function
We have already seen in Section 2.1 that if the proper convex function / : (X,  • ) > M. is Gateaux differentiable a t i G int(dom / ) then
V z e d o m / ( V a ; e X ) : (x x, Vf(x)) < f(x)  f(x). Taking into account this relation, it is quite natural to consider the elements x* E X* which satisfy the inequality VxEX
: (xx,x*)
even if / is not Gateaux differentiable at x.
(2.34)
80
Convex analysis in locally convex spaces
In this section the spaces under consideration are separated locally convex spaces if not stated otherwise. Let / : X ¥ E and x £ X be such that f(x) G E. An element x* G X* is called a subgradient of the function / at x if relation (2.34) is satisfied; the set of all the subgradients of the function / at x is denoted by df(x) and is called the subdifferential or Fenchel sub differential of / at x. We consider that df (x) = 0 if f(x~) £ E; of course we can have df(x) = 0 even if f(x) G E. Thus we obtain a multifunction df : X =4 X*. By the preceding considerations we have that domdf C d o m / . We say that / is subdifferentiable at x G X if df(x) ^ 0. Note that if x* G df(x), the afflne function tp : X —• R, (p(x) := (x,x*) — (x,x*) + f(x) minorizes / and coincides with / at x; this proves that V ( x , i ) € e p i / : (x,x*) — t < a := (x,x") — f(x), which shows that the hyperplane {(x,t) € X x R \ (x, x*) — t • 1 = a] is a, non vertical (since the coefficient of t is ^ 0) supporting hyperplane (see p. 5 for the definition). Recall that in Theorem 2.1.5 we have already determined the subdifferential of a function / 6 A(M) at to £ dom / :
0/(*o) = [/:(*o),/;(to)]nR. In the sequel we shall establish properties of the subdifferential and methods for computing it also for X ^ E. In the next result we collect several easy properties of the subdifferential. Theorem 2.4.1
Let f : X > 1 and x 6 X be such that f(x) 6 E. Then:
(i) df(x) C X* is a convex and w*closed (eventually empty) set. (ii)
Ifdf(x)^
and
d(cof)(x)
= dj(x) =
df(x);
in particular /** = co/ and f is proper and Isc at ~x. (iii) / / / is proper, dom f is a convex set and f is subdifferentiable at every x £ dom / , then f is convex. Proof,
(i) Let x\, x*2 e df{x) and A G ]0,1[. Then
V z G X : (xx,x*1)
(xx,x*2)
The sub differential of a convex function
81
Multiplying the first relation with A > 0, the second with 1  A > 0, then adding them, we obtain
Vx£l :
(xx,Xx*1+(lX)x*2)
whence Ax* + (1 — \)x% G df(x). Let x* G X* \ df(x). Then there exists XQ G X such that (xo — x,x*) > /(xo)  fix). Let a G E be such that {x0 — x,x*) > a > f(x0) — f{x). Then V := {x* \ (x0 x,x*) > a} is a neighborhood of x* for the topology w* = a(X*,X). It is obvious that V n df(x)  0. So df(x) is w*closed. (ii) We already know that co/ < f < f Let x* G df(x) and y>:K>K,
ip(x):=(xx,x*)
+ f(x);
(fi is convex, continuous and
: (xx,x*)
< (cof){x)  (co/)(z) < f{x) 
f(x),
whence d(cof)(x) C df(x). Therefore d(cof)(x) = df(x) = df(x). (iii) By (ii) we have that f(x) = cof(x) for every x G d o m / . Since co/ is a convex function and dom / is convex, it is obvious that / is convex. • The property (ii) of Theorem 2.4.1 justifies why we consider only proper convex functions when discussing subdifferentiability (in the sense of convex analysis!). In a similar way we introduce the subdifferential of a function h : X* —> l a t a point x* G X* with h{x*) G E: dh(x") = {x € X  Vx* G X* : {x,x*  x*} < h(x*) 
h(x*)}.
(As for conjugation, this distinction is important only when working with normed spaces; in the case of locally convex spaces we always consider the natural duality between X and X*, when the dual of X* is X.) Similar results to those of Theorem 2.4.1 hold; more precisely dh(x*) is a closed convex (eventually empty) set and if dh(x*) ^ 0 then h is proper and w*\sc at x*; the other statements are also valid if one takes the closures with respect to the weak* topology.
82
Convex analysis in locally convex spaces
In practice (for example for solving numerically problems using computers) it is possible to determine the subgradients only approximately. In this sense the following notion of subgradient reveals itself to be useful. Let / : X >• 1 , x G X with f(x) G M. and e G ! + ; the element x* G X* is called an esubgradient, of the function / at x~ if Vxel
: (xx,x*)
(2.35)
the set of esubgradients of / at x is denoted by def(x) and is called the esubdifferential of / at ~x. As for the subdifferential, if f(x) ^ M we consider that def(x) = 0; we obtain a multifunction dEf : X =£ X* with dom(<9E/) C d o m / . Note that / is proper if d£f{x) ^ 0 for some e > 0; if 0 < £i < £2 < oo then Of(x) = d0f(x)
C 3 ei /(af) C 5 e a / ( i ) .
Moreover VeGl^
: 9e/(i) = fl
a,/(i).
The £subdifferential of a function /t: I * > K at ? € I * with /i(x*) G E is introduced similarly. In the following theorem we collect several properties of the subdifferential and of the £subdifferential. Before stating this theorem, let us introduce some notions: we say that the set M C X x X* is monotone if V(x,x*),(y,y*)€M
: (a:  y,x*  y*) > 0;
M is strictly monotone if V{x,x*),
(y,y*)eM,
x^y
: (x  y,x*  y*) > 0;
M is maximal monotone if a) M is monotone and b) M' C X x X* monotone and M C M' imply M = M', i.e. if M is a maximal element of the class of monotone subsets ordered by inclusion. We say that the multifunction T : X =3 X* is monotone, strictly monotone or maximal monotone if gr T is monotone, strictly monotone or maximal monotone, respectively. Of course, when X = E and T is singlevalued (i.e. T(a;) is a singleton for every x G domT), T is (strictly) monotone if and only if the function Td om T is nondecreasing (increasing). In the next result we do not assume that the functions are convex.
The subdifferential
of a convex
83
function
Theorem 2.4.2 Let f,g : X >• R, h : Y > R be proper functions, A 6 L(X,Y), x G domf n domg, y G dom/i and e E M+. T/»en: (i) dsf(x)
is a convex w* closed set.
(ii) x* G 3 e / ( z ) «• /(x) + f*(x*)
<(x,x*)+e^x£
def*(x*).
(iii) a;* G df(af) «• /(x) + /*(a;*) < (x,x*) & f(x) + f*(x*) xedf*(x*). (iv) dom(<9£/) C d o m / , lm(dsf)
= (x,x*) =>
C dom/* and df is monotone.
(v) 3/(3;) ± 0 «• /(3f) = m a x a . e x . ((x,x*) 
f*{x*)).
(vi) de(f + x*)(x) = x* + def(x) for x* E X*; 0e(A/)(3:) = \ds/xf(x) and d(Xf)(x) — Xdf(x) for X > 0; if g(x) = f(x + XQ) for x G X then dEg(x) = dEf(x + x0). (vii) Assume that y = Ax; then A* (deh(y)) C de(h o A)(x). ( vii i)
U^6[0,e] (dvf(*) + 9SVg(x))
C 3 £ ( / + g)(x).
(ix)
Suppose X is a normed space, f is Isc, £j > 0 and (xi,x*) G
grdEif
for every t £ 7. If (ei)ig/ —> e < oo and either (a) (xi)iei —• a;,
(x*)iei ^> a;* and (x*)ie/ is normbounded or (b) (x»)i e / ^> £, (a;j)j e j is normbounded and (x*) —t x*, then (x,x*) G grdsf. In particular grdef is closed in X x X* (for the norm topology). Proof, (i) The fact that dEf(x) is convex and u>*closed is shown similarly to the first part of Theorem 2.4.1. (ii) We have that x* ed£f(x)
oVxGX
: (x  x,x*) < f{x)  f(x)
» V i £ l
: {x,x*)f(x)<(x,x*)f(x)
+e +e
&f*(x*)<(x,x*)f{x)+e &f{x)
+
Assume now that x* G def(x). f"(x)
f*(x*)<(x,x*)+e. Then, by what precedes,
+ /*(**) < f(x) + fix*)
<
(x,x*)+e,
and so x € d£f*(x*). (iii) We already know that (37, x*) < f(x) + f*(x*) (the YoungFenchel inequality!); the equivalences follow now from (ii) taking e = 0. (iv) It is obvious that dom(<9£/) C d o m / , while the inclusion Imdef C d o m / * follows from (ii).
84
Convex analysis in locally convex spaces
Let (x,x*), (y,y*) £ gr<9/. Then x,y £ d o m / and {y  x, x*) < f(y)  f(x),
(x  y,y*) < f(x)  f(y).
Adding these relations we get (y — x,x* — y*) < 0, and so df is monotone. (v) If x* € df(x), taking into account (iii) and the YoungFenchel inequality, we have Vx'el*
:
(x,x*)r(x*)=f(x)>(x,x*)r(x*),
and so the implication "=^" is true. The converse implication is an immediate consequence of the equivalences of (iii). (vi)(viii) follow easily from the definition of the esubdifferential. (ix) Suppose that X is a normed space. Let (e'j)ie/ > e < oo and ((xi,x*))i€l have the properties from (a) or (b). Taking into account that \X{, 3Jj j
\Xy X / — \X{
Xj X^ J ~\ \X) X* X*) = (Xi,X* X*) +
(XiX,X*)
for every i, we get ((y — Xi,x*)) —t (y — x,x*) for every y 6 X in both cases. But V i e / , Vy£X
: f(xi) +
(yxi,x*)
taking the limit inferior, we obtain x* G def(x).
O
In the preceding theorem we obtained that df is a monotone multifunction. In fact df is cyclically monotone. One says that T : X =} X* is cyclically monotone if for all n S N and ((xi,x*))._. C g r T one has Y"
(xi+1Xi,x*)<0,
(2.36)
*—'i=0
where xn+i := XQ. Taking n = 1 in Eq. (2.36) we obtain that {x\  x$, x0') + (x0 — xi,x\) < 0, i.e. (xi —xQ,xl — XQ) > 0, for all (X0,XQ), (XI,XI) £ grT. Hence every cyclically monotone multifunction is monotone. When / : X > R is a proper function, df is a cyclically monotone multifunction. Indeed, let n e N and ((xi,a;*))" =0 C gr<9/; then (xi)™=0 C d o m / and (xi+i xt,x*)
< f(xi+i)
 f(xi) Vi, 0 < i < n,
where, as above, xn+\ := XQ. Adding these relations side by side for i = 0 , 1 , . . . , n, we get Eq. (2.36). Note that to a nonempty cyclically monotone multifunction one can associate a function / 6 T(X).
The subdifferential
of a convex
85
function
Proposition 2.4.3 Let T : X =4 X* be a cyclically monotone function and (icc^o) £ g r ^  Consider fr : X —>• M defined by h{x)
:= sup f (a;  z n , < ) + X ) .=Q (a:*+1 ~ ^ ^ ) ) >
where the supremum is taken for all families ((x,,x* ))" = 0 C g r T N. T/ien / T e T(X), /T(XO) = 0 and T(x) C dfT(x) for every x
multi
( 2  37 ) withne eX.
Proof. Because fr is a supremum of continuous afHne functions, fa is lsc, convex and nowhere  o o . Note that domT C d o m / y . Indeed, let (x,x*) G grT. Taking k G N, ((a:i,xj))* =0 C grT, n := fc + 1 and ( x „ , < ) := ( i , ? ) , from Eq. (2.36) we obtain that
E
iti
_
. {Xi+i 1=0
 Xi, X*) + (X
Xk,X*k)
+ (X0  X, X*) < 0,
whence fr{x) < (x — x0,x*). Hence x G d o m / y . In particular the preceding inequality shows that / T ( ^ O ) < 0. Taking n = 1 and (xi,x*) = (XO,XQ) in the definition of fr(xo), we get fr(xo) > 0. Hence / T ( ^ O ) = 0. Therefore
h € T(X). Let now (x,x*) G g r T and x G X . Consider an arbitrary it < fr{x) + (x — x,x*). Then there exist k G N and ((XJ,X*)J C g r T such that _
r—\fci
/ x  ( x  x , x * ) < {xxk,x*k) + 2 ^ . = 0 fci+i a;»,a;*>Taking n = k + 1 and (x n ,a;*) := (x,x*), the preceding inequality yields n—1
Ei=o (
Xi+1
~XuX^
 ^W
Letting (i —¥ fr (x) + (x — x, x*), we obtain that fr (x) + (x — x, x*) < fa (x), and so x* G 9/r(S). D In the next theorem we use the convexity of the function / . Theorem 2.4.4
Let f G A(X), x G d o m / and e Gffi+. T/»en:
(i) def(x) = df'e(x, )(0); moreover, ifx£ differentiate at x then df(x) = {V/(x)}.
( d o m / ) 1 and / is Gateaux
(ii) 7/ / is strictly convex then df is strictly monotone. (hi) / is lsc atx if and only ifd£f(x) ^ 0 for every e > 0; d£f*(x*) ^ 0 if f (x*) eRande>0. (iv) Suppose that f is lsc at x. Then x* G dEf(x) <£> x G def*(x*).
Convex analysis in locally convex spaces
86
Proof. (i) Ii x* £ dsf(x), replacing x by x + tu, with t > 0, in Eq. (2.35), then dividing by t we obtain V«€l
: (u, x ) < inf —
^^
—
fe{x,u),
and so a;* £ df^(x, )(0). The converse inclusion follows from the inequality f'e(x, u) < f(x + u) — f(x) + e, valid for every u £ X. If / is Gateaux differentiable at x then f'+{x, •) — V/(aT); the conclusion is obvious. (ii) Let x,y £ d o m / , x / y, and x* € df(x), y* £ df(y). From (i) we have that (yx,x*)
< f+(x,yx)
<
f(y)f(x),
&  y, y*) < f+(y, xy)<
f(x) 
f(y),
because / is strictly convex. Adding the above two relations we obtain that (y — x, x* — y*) < 0. This shows that df is strictly monotone. (iii) Suppose that / is lsc at x and let us take e > 0; using Theorem 2.3.4 we have f(x) = r(x)
= sup{(z,z*)  f*(x*)
I x* G X*} >
f(x)e.
Therefore there exists a;* £ X* such that {x, x*)—f*(x*) > f(x)—s, whence x* £ def(x). Conversely, suppose that dEf(x) ^ % for every e > 0. Then V e > 0 , 3x*£X*
: f(x)  e < {x,x*}  f*(x*)
<
f(x),
whence f(x) < f**(x). Since / * * < / < / , it follows that J{x) = f(x), i.e. f is lsc at x. Let x* £ X* be such that f*(x*) £ R and e > 0. By the definition of /*, there exists x £ X such that f*{x*) < (x,x*) — f(x) + e, whence (x,x*x*)
< (x,x*)  f(x)  f*(x*)+e<
i.e.x£def*(x*). (iv) Since / is lsc at x, we have that f**(x) immediate using Theorem 2.4.2 (ii).
f(x*)

f*(x*)+e,
= f{x); the conclusion is •
Remark 2.41 Note that for / 6 A(X), x £ d o m / and U £ Mx{x) we have that df(x) = d(f + iu)(x), but def{x) and d£(f + iu)(x) may be
The subdifferential
of a convex
87
function
different for e > 0, i.e. df is a local notion while dsf global one for convex functions.
(with e > 0) is a
Indeed, since f'(x, u) — (f + t[/)' (x, u) for every u £ X, the first remark follows from assertion (i) of the preceding theorem. For the second remark let us consider / € A(R) given by f(x) = x2 and U = [—1,1]. Then ds{f + iu)(0) = def{0) = [2y/i,2y/e\ for e € [0,1] and de{f + t a )(0) = [1 £,l + e]£ [2y/i,2^/i\ = 0 e /(O) for e > 1. Generally, the formula d(\f)(x) = Xdf(x) is not true for A = 0; this formula, however, is true if x £ (dom/) 1 . In assertions (vii) and (viii) of Theorem 2.4.2 we have equality only under supplementary conditions called generally "constraint qualification conditions;" in Section 2.8 we shall give such conditions. Let A C X be a nonempty convex set and a e A; then diA{a) = {x* eX*\Vx£A
:
(xa,x*)<0}
= {x*  V x G A : (x,x*) < (a,x*)} = {x*  Vx 6 cone(A  a) :
(x,x*)<0}
+
= —(cone(A — a ) ) = —cone(A — a)+. The set <9M(O) is denoted by N(A;a) and is called the normal cone of A at a S A, while the set cone (A — a) is denoted by C(A;a) (even if A is not convex); evidently, the set C(A;a) is a closed (convex if A is so) cone. Taking into account the relation established above and the bipolar theorem (Theorem 1.1.9), we have that N{A;a) = (C(A;a))+
and C(A;a) = (N(A;a))
+
.
It is obvious that N(A; a) = {0} if a £ A*. We observe that x* € N(A; a) \ {0} if and only if Hx.^a^x*) is a supporting hyperplane to A at a and Ac//,, ,,. The set df(x) may be empty even if / is lsc at x. For example, the function / : R > E, f(x) := Vl  x2 for \x\ < 1, f(x) := oo for a; > 1 (already considered in Section 2.1) is lsc, finite at 1, but df(l) = 0. Moreover 9 ( 0  / ) (1) = R_. In the preceding section we have met some situations in which it was easy to compute the conjugate of a function. We shall point such situations for the esubdifferential, too.
88
Convex analysis in locally convex spaces
Corollary 2.4.5 Let fi : Xi —> R for i E \,n be proper functions and x = (xi,... ,xn) E Yli=i dom/j. Let us consider the function
n
n
.
—
XitR,
•
^n
:= > .
fi(xi),
ande E K+• Then de
9
^ ^ )
£i
 °>
£l H
+ e« = e j •
/n particular
n
n
. dfi(xi). Proof. We have already seen in the preceding section (for n = 2, but the extension is immediate) that y*{x\,... ,xn) = Y12=ifi(xi) Therefore, by Theorem 2.4.2 (ii), x* E de
O 3 e\,...,
1 ( z „ , < ) + e
en > 0, ei + . . . + en = e, Vi G T~n : /i(xi) + /*(z*)  (xi,x*) < et,
O 3 £i, . . . , 6n > 0, £! + . . . + £„ = £,
Vi G l , n : x* €
dEifi(xi),
whence the conclusion.
•
Corollary 2.4.6 Let f : X » R be a proper function and A E &(X, Y). If x E d o m / and y E Y are such that y = Ax and (Af)(y) = f(x), then for every e E R+ we have de(Af)(y)=A'1(def(x)). Proof.
Using Theorem 2.4.2 (ii) we have that
y* E d£(Af)(y)
o (Af)(y) + (A/)*(j,*) < (y,y*) +e & f(x) + f*(A*y*) *> A*y* G def(x)
< (Ax,y*) + e = (x,A*y*) + e &y*E
A*'1
(def(x)).
We used the relation (Af)* = f* o A*, proved in Theorem 2.3.1 (ix).
•
The subdifferential
of a convex
89
function
Corollary 2.4.7 Let / i , . . . , / „ : X > R (n € N) be proper functions. Suppose that for every i € l , n there exists x~i £ Aora.fi suc/t i/iaf (/lD • • • • / „ ) ( ! ! + . . . + ! „ ) = / i ( l i ) + • " • + /„(x„).
(2.38)
Then for every e € E+ a e ( / i D . . . D / „ ) ( i i + ••• + ! „ ) = [ J { ^ e i / l ( ^ l ) n    n 9 e „ / „ ( l „ )  £ l , . . . , e „ > 0, £X + . . . + £ „ = £ } .
In particular
0(/iD... n/n)(xi + • • • + xn) = a/i(ii) n • • • n 5/„(in). Conversely, if dfi(x~i) Pi • • • (~1 dfn(xn)
^ 0, i/ien relation (2.38) is vafo'd.
Proof. We apply Corollary 2.4.5 to the functions fi, • • ,fn and Corollary 2.4.6 to / : X n >• E defined by / ( x i , . . . ,£„) := / i ( x i ) + • • • + fn(xn) and to the operator A : X " —> X defined by A(xi,..., xn) := X\ + • • • + xn. If x* £ 5/i(«i) (~1 • • • n dfn{xn), then Vi £ l , n , V i j GX" :
(xiXi,x*)
hence / I ( X J ) H h fn{xn) < / i ( x i ) ^ 1 fn{xn) for all x i , . . . , x n e X such that xi + • • • + xn = x~i + • • • + xn, and so Eq. (2.38) holds. • The next result shows that the convolution has a regularizing effect. Corollary 2.4.8 Let / i , / 2 e A(X), xt e dom/i for i e {1,2} and x — xi +X2 Assume that (fiDf2)(x) = /i(x"i) + f2(x2) and fiDf2 is subdifferentiable at x. If f\ is Gateaux differentiable at x\ then f\Of2 is Gateaux differentiable atx and V(fiDf2){x) = V / ^ x j ) . Moreover, if X is a normed vector space and / i is Frechet differentiable at xi then /1D/2 is Frechet differentiable at x. Proof. By the preceding corollary we have that d(f\Of2) (x) = df\ (x~i) n 0/ 2 (z 2 ) Let x* £ d(f1nf2)(x). Then (u,x*) < (haf2)(x < fl(xi
+ u)  (AD/aXaf)
+u) + / 2 ( x 2 )  fi{xi)
 /2(x2)
(239)
for every u € X. If fi is Gateaux differentiable at x\ then, from the preceding corollary and Theorem 2.4.4 (i), we have that x* = V / i ( x i ) .
Convex analysis in locally convex spaces
90
Using (2.39) we obtain that (fiOf2)'(x,u) = (w, V/i(a; 1 )) for every u € X, which shows that /iD/2 is Gateaux differentiable at x and V(/iD/2)(x) = V/!(li).
Assume now that X is a normed space and f\ is Frechet differentiable at x\. From the preceding situation we have that /1IH/2 is Gateaux differentiable at x and V(/iD/ 2 )(5;) — V/i(afi). Using again Eq. (2.39), we have that 0 < (/id/ 2 )(af + u)  (fiDf2)(x)
 (u,V/i(ii)>
< fi{xi + u)  / i ( i i )  (u, V/i(ii)>
Vu G X;
hence /1CH/2 is Frechet differentiable at 2; and V(/iD/2)(af) = V/i(aFi). D In Section 2.6 we shall extend the results of Corollaries 2.4.6 and 2.4.7 to the case where the infimum is not attained in y and x, respectively. The following result establishes a sufficient condition for the subdifferentiability of a convex function; this result is of exceptional importance. Theorem 2.4.9 Let f G A(X). If f is continuous at a; G d o m / , then def(x) is nonempty and w* compact for each e G M+. Furthermore, for every e > 0, f'e(x, •) is continuous and V«6l
: f'e{x,u)
= max{(«,x*)  x* G d£f(x)}.
(2.40)
Proof. Suppose that / is continuous at x G d o m / (from Theorem 2.4.4 we already know that def(x) ^ 0 for e > 0!). Let rj > 0. Since / is continuous at x, there exists V G K x such that Vxex
+ V : f{x)
(2.41)
Therefore (aT + V) x [f(x) + n, oo[C e p i / , whence int(epi/) ^ 0. The set e p i / being convex and (x, f(x)) £ int(epi/) (because (x,f(x) — 5) $. e p i / for every S > 0), we can apply a separation theorem; so, there exists (a;*,a) G X* x E \ {(0,0)} such that V ( z , f ) € e p i / : (x,x*) + at < (x,x*) + af(x).
(2.42)
Taking x — x and t = /(aT) + n, n G N, we get a < 0. If a = 0, using Eq. (2.42), we obtain that {x — x,x*) < 0 for every x G d o m / , and so, by Eq. (2.41), we have that (a;,a;*) < 0 for every x GV; since V is a neighborhood of the origin, this implies that x* = 0. Thus we obtain the contradiction:
The subdifferential
of a convex
function
91
(a;*,a) = (0,0). Therefore a < 0; so we can consider a =  1 in Eq. (2.42). This relation becomes V x e d o m / : (x,x*)  f(x) < (x,x*) — f(x), i.e. x* e df(x). Let now e > 0. For every x* G dsf(x) we have V i i g y : (u,x*)
= df£(x, )(0), using Eq. (2.41),
+ u)f(x)+£
+ e,
(2.43)
whence {u,x*) >  1 for every u £ (r] + e)~1V (recall that V is symmetric) which proves that def(x) C (fa + e )  1 V)° = (r, + e)Va.
(2.44)
By the AlaogluBourbaki theorem (Theorem 1.1.10) we have that V° is w*compact; since def(x) is w*closed, it follows that d£f(x) is w*compact. Taking into account that x £ int(dom/), from Theorem 2.1.14, we have that domf^x, •) = X; hence, using Theorem 2.2.13 and Eq. (2.43), the function fg(x,) is continuous. Furthermore, using again Eq. (2.43), we have that f'e(x,u)
> sup{(u,a;*)  x* £
def(x)}.
We intend to prove that in the above inequality we have equality and the supremum is attained. For this end let u € X \ {0}. Consider XQ := Eu and ip : Xo > E, p(tu) := tf'e(x,u); it is obvious that ip(u') < f'E(x,u') for every u' 6 Xo, and so, applying the HahnBanach theorem, there exists x* 6 X' such that (u,x*) = tp(u) = f^{x,u) and (u',x*) < f'e(x;u') for every u' € X. Since f^(x,) is continuous, x* is continuous, too; hence x" e df'£{x, )(0) = dEf(x). The proof is complete. • Using the preceding result we obtain the following criterion for the Gateaux differentiability of a continuous convex function. Corollary 2.4.10 Let f € A(X) be continuous atxe domf. Then f is Gateaux differentiate at x~ if and only if df(x) is a singleton. Proof. The necessity was already observed in Theorem 2.4.4(i). Assume that df(x) = {x*}. Using Theorem 2.4.9 we obtain that f'(x,u) = (u,x*) for every u € X. Because lim^o*  1 (f(x + tu) — f{x)) = —f'(x,—u) =
Convex analysis in locally convex spaces
92
— (—u,x*) = (u,x*), we have that / is Gateaux differentiable and Vf(x) = x*. D Related to Frechet differentiability of convex functions see Theorem 3.3.2 in the next chapter. When / is not continuous at x £ d o m / , relation (2.40) may be false; see Corollary 2.4.15 and Exercise 2.30. However the following result holds. Theorem 2.4.11
Let f £ T(X), x £ dom / , and e £ ]0, oo[. Then
Vu£X
: f'e(x,u)=sup{{u,x*)\x*
£d£f(x)}.
(2.45)
Therefore f'e(x, •) is a Isc sublinear function. Furthermore, for every u £ X, f'(x,u)
= lim (swp{{u,x*)  x* £
def(x)})
= inf (sup{(u,a:*)  x* £ def(x)}). Proof. whence
(2.46)
In Theorem 2.4.4(i) we have seen that dEf(x)
V / 6 3E/(f),Vii6l
: {u,x*) <
—
df^.(x,)(0),
ft(x,u).
Therefore the inequality " > " holds in Eq. (2.45). For proving the converse inequality, let u £ X and A £ R with A < f'£(x,u). From the definition of f'e (x, u), we have that Vt>0:X
+ t u )
;
m + £
(2.47)
and A
<
lim
f^
t—>oo
+ tu)f(x)+e t
=
^ (—>oo
f(x
+
tu)m
= /oo(u),
(2.48)
t
Let us consider A := e p i / and B := {(x + su, f(x) + Xs — e) \ s > 0}. It is obvious that A and B are nonempty closed convex sets and, from Eq. (2.47), A n B = 0, i.e. (0,0) ^ A — B. Moreover B is locally compact (as subset of a finite dimensional separated locally convex space). Since Aoo = epi /oo and B00=R+(u, A), from Eq. (2.48) we obtain that 4oonBco = {(0,0)}. Then, by Corollary 1.1.8, A — B is a closed set. Applying Theorem 1.1.7, there exists (x*,a) 6 X* x E such that V(x,t) £ epi/, V s > 0 : 0 > (x  x
su,x*) + a(t  f(x)  sX + e).
The sub differential of a convex function
93
Taking x = x and letting t > oo we obtain that a < 0; if a = 0, for s = 0 we get the contradiction 0 > 0. Therefore a < 0 and we can suppose that a = — 1. The above relation becomes V x G d o m / , V s > 0 : 0 > (a; x,x*)
 (/(«)  /(a?) + e ) +s(A  ( u , i * » .
For s = 0 we obtain that x* G dsf(x), while for s > oo we obtain (u,a7*) > A. Therefore A < sup{(u,a;*)  a;* £ 5 e /(aT)}. Since A < f^(x,u) is arbitrary, the relation (2.45) is true. The equality (2.46) follows from relation (2.45) and Theorem 2.1.14. • The subdifferentiability criterion of Theorem 2.4.9 can be extended. Theorem 2.4.12 Let f G A(X) and X0 := aff(dom/). / / f\Xo is continuous atxG domf, then df(x) ^ 0. In particular, if dim X < oo, then df(x) ^ 0 for every x G *(dom/). Proof. Without any loss of generality, we can suppose that x = 0; then X0 = lin(dom/). The function g := f\x0 is convex, proper and continuous at 0. By Theorem 2.4.9 we have that dg{0) # 0. Let v? G dg{0); hence ip € XQ and V x g d o m j : tp[x) — (p(0) < g(x) — g(0). Using the HahnBanach theorem we get i* G X* such that x*\x0 = tp. The above inequality shows that Vx G d o m / = dome? : {x  0,x*) = tp{x) < g(x)  g(0) = f(x)  / ( 0 ) , whence x* G 9/(0). If dimX < oo, then dimXo < oo. By Theorem 2.2.21, g is continuous on int(dom/) = l ( d o m / ) . The conclusion follows from the first part. • Note that under the conditions of Theorem 2.4.12 df(x) is, generally, not bounded, and so it is not «;*compact. But, under the conditions of Theorem 2.4.9, in the case of normed spaces, df has supplementary properties (mentioned in the next theorem). The local boundedness of monotone operators was proved by R.T. Rockafellar (see Theorem 3.11.14); the converse is true in Banach spaces for lower semicontinuous functions as shown by Corollary 3.11.16. Theorem 2.4.13 Let X be a normed space, f G A(X) be continuous on int(dom/) and e > 0. Then dsf is locally bounded on int(domc?/) =
94
Convex analysis in locally convex spaces
int(dom/). Moreover, if f is bounded on bounded sets then f is Lipschitz on bounded sets and dsf is bounded on bounded sets. Proof. By Theorem 2.4.9 we have that int(dom/) C dome?/ C d o m / ; hence int(dom<9/) = int(dom/). Let xo £ int(dom/); since / is continuous at x0, from Corollary 2.2.12 we get the existence of m, p > 0 such that Vx,y€D(x0,2p)
: \f(x)  f(y)\ < m\\x  y\\.
Let us fix a; € D(xo,p). Then for y £ x + pUx we have that f(y) < f(x) + mp, i.e. Eq. (2.41) holds with V := pUx and r] :— mp. Using Eq. (2.44), we obtain that def(x) C (r? + e)V° = (m + e/p)Ux* for every x £ D(x0,p). Assume now that / is bounded on bounded sets. Then d o m / = X. Taking p > 0, / is bounded above on 2pUx, and so Eq. (2.41) holds with V := 2pUx and some r\ > 0. By Corollary 2.2.12 / is Lipschitz on pUx, and, by Eq. (2.44), def(x) C (?? + e)p~1Ux* for x e D{x0,p). The proof is complete. • For other relationships between the continuity of / and the local boundedness of df see Corollary 3.11.16. In Theorem 2.4.4 we have seen that finding the esubdifferential of a convex function at a point reduces to compute the subdifferential of a sublinear function at the origin. Other subdifferentials (the subdifferentials of Clarke, of MichelPenot, etc.), for nonconvex functions are introduced through certain sublinear functions. So, we consider it is worth giving some properties of sublinear functions. T h e o r e m 2.4.14 Let f,g:X>M.,h:Y>Rbe sublinear functions, T £ &(X, Y) and B,C C X* be nonempty sets. Then: (i)
/ * = ^9/(0);
(ii) 5/(0) ^ 0 o / is Isc at 0; (hi) for every x G dom / and for every e £ R+ we have:
dm
= {x*edf(p)\(x,x')
= f(x)},
d£f{x) = {x* e 3/(0) I (x,x°) > f(x)  £},
def(0) = 0/(0);
(iv) if f is Isc at 0 then Vxel
: J(x)=sup{{x,x*)\x*
edf{0)}.
The subdifferential
of a convex
function
95
(v) Suppose that f and g are Isc. Then f < g <3> 9/(0) C dg(0). (vi)
The support function SB of B is sublinear, Isc, SB = (<B)* and — coB, the closure being taken with respect to the weak* topology w* on X*; moreover, SB < sc if and only if B C coC. 0SB(O)
(vii) / / h is Isc then d(h o A)(0) = w*c\ (A*{dh{0))); (viii)
if f and g are Isc, then d(f + g)(Q) = w*c\ (3/(0) + dg(0)).
Proof,
(i) For every x* € X* we have
f*(x*)
= s u p ^ O  f{x) I x € d o m / } > (0,a;*)  /(0) = 0.
If a;* £ df(0) then (x,x*) < f(x) for every x £ d o m / , whence f*(x*) = 0. If x* £ <9/(0), there exists x e X such that (x, x*) > f(x). In this situation we have f*{x*) > sup{(tx,x*)
 /(tx)  t > 0} = sup{i(x,a;*)  /(x)  t > 0} = oo.
(ii) If 9/(0) ^ 0 then, from Theorem 2.4.1(h), we have that / is Isc at 0. Suppose that / is Isc at 0. Then /(0) = /(0) = 0, and so, by Theorem 2.3.4, /(0) = /**(0) = 0. Assuming that <9/(0) = 0 we obtain that /* = £9/(0) = +00, and so the contradiction /** = — 00. Therefore 9/(0) ? 0. (iii) Let x £ d o m / and e > 0. If x* £ 9/(0) and (a;,a;*) > /(a;) — e then WxeX
: {xx,x*)
= {x,x*)(x,x*}
i.e. x* £ d£f(x). Conversely, let x* £ def(x); then the above inequality holds. Taking x = 0 we get (af, a;*) > f(x) — e; taking now x :=x + ty, t > 0, y £ X, we obtain that *(l/,**) < / ( i + ty)  f{x) +e<
fix) + tfiy)  fix) + e = tf(y) + s.
Dividing by t > 0 and letting then t > 00, we obtain that (y,x*) > fiy) for every y € X, i.e. x* £ 9/(0). For e = 0 we have 0/(3:) = {x* € 0/(0) I (a;,a:*) > fix)} = {x* £ 0/(0)  (x,x*) = while for x = 0 we obtain that 9 £ /(0) = K
G 9/(0) I (0,ar*) > /(0)  e }  9/(0).
fix)},
96
Convex analysis in locally convex spaces
(iv) Suppose that / is lsc at 0. Then /(0) = /(0) = 0. Using Theorem 2.3.4, we have that /** = J. Therefore J(x) = sup{(a;,a;*)  f*(x*)
 x* 6 X*} = sup{(x,x*)
\ x* € 0/(0)}.
(v) It is obvious that 0/(0) C dg(0) if / < g. The converse implication follows immediately from (iv). (vi) At the end of Section 2.3 we noted that SB is lsc, sublinear and sB = {LB)*'• From the obvious inclusion B C 8SB(0) we get coB c <9SB(0) (because <9SB(0) is convex and w*closed). Let x* fi coB. Using Theorem 1.1.5 in the space (X*,w*) and taking into account that (X*,w*)* — X, there exist x E X and A s K such that Vx* G coB D B : (x,x*) > A >
(x,x*),
whence (x,x*) > ssi'x), ie. x* £ 8SB(0). The conclusion follows. (vii) By Theorem 2.4.2 we have that A*(dh(0) C d(h o A)(0). Let x* i w*c\ (A*(dh(0))). Using Theorem 1.1.5, there exist x € Y and A e R such that (x,x*) > A > (x,Ay*) — {Ax,y*) for every y* e dh(Q). From (iv) we get A > h(Ax), and so x* $. d(h o A)(0). (viii) Let H : X x X > 1 , H(x,y) := f(x)+g(y), andT:I>XxX, Tx := (x,x). It is obvious that H is sublinear and lsc, and T is continuous and linear; moreover, dH(0,0) = 5/(0) x dg{0) and T*(x*,y*) = x* + y*. By (vii) we obtain that d(f + g)(0) = d(H o T)(0) = «>*cl (T*(dH(0,0)))
= w* cl(5/(0) + dg(0)).
The proof is complete.
•
Using the preceding result we have Corollary 2.4.15 Let f G \(X) and x G d o m / . Then df(x) ^ 0 if and only if f'(x, •) is lsc at 0. In this case f'{x,)(u)
= sup{(u,x*)  Z* G df(x)}
Vuel.
Proof. The conclusion follows from the formula df(x) — df'(x, )(0) (see Theorem 2.4.4) and assertions (i) and (iv) of the preceding theorem. • Note that, even for X = E 2 and / G T(X) subdifferentiable at x, f'(x,) may not be lsc; take for example / := i\j and x = (0,1), where U:={(x,y)em2 \x2+y2
The sub differential of a convex
function
97
Corollary 2.4.16 Let (X,  • ) be a normed space, f : X > R, f(x) := \\x\\, andx G X. Then, denoting Ux* by U*, we have: r Proof.
= iu;
df(0) = ir,
df(x) = {x*eU*\(x,x*)
= \\x\\}.
The above formulas follow immediately from Theorem 2.4.14. •
Another example of sublinear function is given in the following result. Corollary 2.4.17 Let / : E n >• E, f(x) := xi V • • • V xn, and x e W1, where n G N. Then 5/(0) = A n , / * = IA„, def(x)
= {y e An\xiyi
+ ...+xnyn>
f(x)e},
where A » : = { ( A i , . . . , A n ) 6 " B I A i > 0 , Ai + . . . + A„ = 1}. Proof.
It is obvious that / is sublinear and continuous. Moreover y G <9/(0) » V x G C
: xlVl
+ ••• + xnyn
< n V • • • V xn.
Taking Xj = 0 for j ^ i and Xi = —1, we obtain that — j/» < 0, i.e. yi>0 for every i. Taking then X{ — t G K for every i, we obtain that £(j/H \yn) < t for every i, whence y±\ \yn = l. Therefore 9/(0) C A n . The converse inclusion is immediate. The other relations follow directly from Theorem 2.4.14. • The following theorem furnishes a formula for the subdifferential of a supremum of convex functions. Other formulas will be given in Section 2.8. Theorem 2.4.18 (loffeTikhomirov) Let (A,T) be a separated compact topological space and fa : X —> E be a convex function for every a G A. Consider the function f := supaGAfa andF(x) := {a G A  fa{x) = f(x)}. Assume that the mapping A 3 a 4 / „ ( i ) G E is upper semicontinuous and XQ G dom / is such that fa is continuous at XQ for every a £ A. Then df(x0)=co(\J
dfa(x0)).
(2.49)
Proof. Since A is compact and a H> fa{x) is upper semicontinuous we have that F{x) is a nonempty compact subset of A for every x G X. The inclusion D being true without any condition on A and / (and easy to prove), let us show the converse inclusion. If f(xo) — —oo, then fa(xo) =
Convex analysis in locally convex spaces
98
—oo for all a £ A = F(x0), and so df(x0) = dfa(x0) — 0; the conclusion holds. We assume now that f(x0) £ R. Let us note first that in our conditions x0 £ (dom / ) ' . Indeed, let x £ X. Consider a fixed 70 > f(xo) (> fa(xo))Because fa is continuous at xo, for every a £ A there exists ta > 0 such that fa(xo + tax) < 70. Since /? t> /^(^o + tax) is upper semicontinuous, the set Aa := {/3 £ A \ fp(x0 + £Q:r) < 70} is an open set containing a. As A = UaeA^<*> there exist a i , . . . , an £ A such that A = \J™=1 Aai. Let t := m i n { £ a i , . . . , i Qn } > 0. It follows that fa(x0 + tx) < 70 for every a £ A, and so f(xo + tx) < 70. Hence x0 £ ( d o m / ) \ Since fa is continuous at xo, dfa(xo) 7^ 0 for every a £ F(xo), and so Q ^ 0, where Q is the set on the righthand side of Eq. (2.49). Suppose that there exists x* £ df(x0) \ Q Using Theorem 1.1.5 in the space (X*,w*), there exist x £ X and e > 0 such that VaeF(io),Vi'e9/a(i0)
: (x,x*)  e > (x,x*),
(2.50)
or equivalently, by Theorem 2.4.9, V a e F ( i „ ) : (x,x*)e>{fa)'{x0;x).
(2.51)
Because x0 £ (dom/) 1 , we may suppose that x0+x £ d o m / . Let t £]0,1[; then xo + tx £ dom / . There exists at £ A such that fat (XQ + tx) — f(x0 + tx). Because (1  t)fa,(x0) + tfat(x0 + x) > fat(x0 + tx) and x* £ df(x0), we obtain that (1  t)fat (Xo) > f(x0
+ tx)  tfat (x0 + X) > f(x0
+ tx)  tf(x0
+ x)
> f(xo)  t (x,x*)  tf(x0 + x). It follows that limt^o fat(xo) = f(xo). The space (A,T) being compact, there exists a convergent subnet (a^^j^j of (a t ) t e ] 0 ,i[ converging to ao £ A; hence fao(x0) = f(x0), i.e. a0 £ F(x0) Let s e]0,1[ be fixed (for the moment) and take t £ ]0, s]. Using Eq. (2.50) we obtain that fat(xo
+SX)  fat(x0)
s
fqt(x0
>
+ tx)  fgt{x0)
t
f(x0

+ tx)  f (x0)
t
(x,x*).
We have that tp(j) < s for j y j s for some j s £ J. Taking t = ip(j) with
The general problem of convex
programming
99
j y j s and using the upper semicontinuity hypothesis, we obtain that Va€]0>i[:
Ui^ +
^Ui^) s
and so (jao)'{xo;x) > (x,x*), contradicting Eq. (2.51). D Of course, for conjugates and for the esubdifferentials it is desirable to dispose of numerous formulas. There exist effectively a large set of such formulas we shall establish in Section 2.8. 2.5
The General Problem of Convex Programming
By problem of convex programming we mean the problem of minimizing a convex function / : X —> R, called objective function (or cost function) on a convex set C C X called the set of admissible solutions, or set of constraints. We shall denote such a problem by (P)
min f(x),
xeC.
Of course, to consider this problem, X must be a linear space, however most of the results will be obtained in the framework of separated locally convex spaces or even normed spaces. To problem (P) we can associate a problem (apparently) without constraints: (P)
min f(x),
x G X,
where / := / + vein order for the problem (P) to be nontrivial, it is natural to assume that C n dom / ^ 0 (<£>• dom / ^ 0) and that / does not take the value — oo on C (i.e. f does not take the value — oo). We call value of problem (P) the extended real v(P) := v(f,C)
:= m£{f(x)
 x G C} £ I ;
we call (optimal) solution of problem (P) an element x G C with the property that f(x) = v(P); this means that x is a (global) minimum point for the function / . We denote by S(P) or S(f, C) the set of optimal solutions of problem (P). Therefore S(P) = {x G C  Vx G C : f(x) < f(x)} = {zGXVxGX
: f(x) < f(x)} = S(P)
Convex analysis in locally convex spaces
100
if C fl dom / ^ 0. The set S(f, X) is denoted also by argmin / . Of course, an important problem is that of the existence of solutions for (P), resp. (P). The most important result which assures the existence of solutions for (P) is the famous Weierstrass' theorem. Because the underlying spaces are not compact we have to use some coercivity conditions. We say that / : X —> E is coercive if limn^n^oo f(x) — oo. It is obvious that / is coercive if and only if all the level sets [/ < A] are bounded (see also Exercise 1.15); when / is convex then / is coercive if and only if the level set [/ < A] is bounded for some A > inf / (see Exercise 2.41). We have the following result. Theorem 2.5.1 (i)
Let f £ T(X).
If there exists A > v(f,X)
such that [f < A] is wcompact,
then
s{f,x)^d>. (ii) If X is a reflexive Banach space and f is coercive then S(f,X)
^ ill.
Proof, (i) Of course, v(f,X) = v(f, [f < A]). Since / is lsc and convex, / is tulsc. The conclusion follows using the Weierstrass theorem applied to the function /  [ / < A ] 
(ii) Because / is coercive (see Exercise 1.15), [/ < A] is bounded for every A e E. Since [/ < A] is wclosed and X is reflexive, we have that [/ < M i s wcompact for every A G E. The conclusion follows from (i). • Of course, in the preceding theorem, the condition that / is convex can be replaced by the fact that / is quasiconvex. The next result shows that the reflexivity of the space X is almost necessary in Theorem 2.5.1. Theorem 2.5.2 Let (X, ) be a Banach space. Assume that there exists a proper function f : X —> M. satisfying the following conditions: (i) (ii)
[/ < f(x)] l'5 closed, convex and bounded for every x € dom / ; / attains its infimum on every nonempty closed convex subset of
X; (iii) there exist xo,xi £ d o m / and r > 0 such that f is Lipschitz on [f < /On)] and [f < f(x0)] + rUx C [/ < / f a ) ] , where Ux := {x € X 
IMI < 1}Then X is reflexive. Proof. Of course, f(x0) < f(xi); if f(x0) = f{xi) then the relation [/ < f(xo)] + rUx C [/ < f(xi)] contradicts the boundedness of [/ <
The general problem of convex programming
101
f(xi)]. Hence f(x0) < f(xi). Since / is Lipschitz on [/ < f(x\)], there exists L > 0 such that \f(x)  f(x')\ < L\\xx'\\ for all x,x' £ [f < f(x\)]. Since /[/ 0 such that allx C A c /SC/x It follows that a  1 1     = paux > PA > Ppux = P~x , where p ^ is the Minkowski gauge associated to A. By Theorem 1.1.1 PA is a seminorm, and so it is a norm equivalent to ; moreover, by Proposition 1.1.1, the unit closed ball with respect to PA is cl A To obtain that X is reflexive, by the famous James' theorem (see [Diestel (1975), Th. 1.6]), it is sufficient to show that every x* € X* attains its infimum on A. Let x* £ X* \ {0} and a* := inf{(x,x*) \ x £ S}. Because S is bounded, a* € E. Let H := {x £ X \ (x,x*) = a*}. It is obvious that (S+eUx)nH # 0 for every e > 0. Taking e 6 ]0, S0] and xe £ (S+eUx)r\H, we obtain that f(xe) < f{x2) + eL, and so mixen f(x) < f(x2). By (ii) there exists xi £ H such that f(x~i) = inf x € ij f(x), and so x\ £ Sf)H. Therefore (xi,x*) < (x,x*) for every x £ S. Similarly, there exists x2 £ S such that (x2,x*) > (x,x*) for every x £ S, whence there exists x := Xi —x2 £ A such that (x,x*) < (x,x*) for every x £ A. • Also the coercivity condition in Theorem 2.5.1(h) is essential as the next theorem will show. In order to establish it we introduce some preliminary notations and results. Let / 6 T(X) and denote by 11/ the set {g £ T(X)  domg = d o m / , supx€domf
\f(x)  g(x)\ < oo, inf / = inf g}.
Since for g £ Hf we have that / — 7 < / < / + 7for some 7 £ M+, we have that /oo = g^ and / , g are simultaneously coercive or not coercive. Consider d : Ilf x Ilf ^ M+, Lemma 2.5.3 metric space.
d{gug2)
:= s u p x 6 d o m / \f{x)  g{x)\ .
The mapping d is a metric on 11/ and (11/, d) is a complete
Proof. It is obvious that d is a metric. Let (gn)n>i C 11/ be a Cauchy sequence. Then (gn{x))n>1 C E is a Cauchy sequence, and so (gn(x)) >
102
Convex analysis in locally convex spaces
g(x) E R for every x G d o m / . Moreover, ( s u p x e d o m / \gn(x)  g(x)\)n^,1 > 0. We extend g to X setting g(x) := oo for x G X \ dom / ; hence dom g — dom / . Because (gn(x)) —> g(x) for every a; G X, from Theorem 2.1.3(h) we have that g is convex. Let e > 0; then there exists nE such that <7„(a;) — s < g(x) < 9n(x) + £ for all n > nE and a; G d o m / . It follows that g is proper (being minorized by gnc — e even on X) and inf / — e = inf gn — e < inf 0 which shows that inf g = inf / (even if inf/ =  c o ) . We must only show that g is lsc. Take first x G domg = d o m / and A < g{x). There exists e > 0 such that A < g(x) — 2e. As above, there exists n = ne such that g(y) — e < gn(y) < g(y) + £ for every y G d o m / . In particular g(x) — e < gn(x). Because gn is lsc at x, there exists U G Mx{x) such that 0 andy E X. Then the se£co(epi/U{(y,inf / — e)}) is i/te epigraph of a function fyi£ E T(X) with inf fViS = inf / — e and argmin / y>£ = y + K. Moreover, if f{y) < inf / + e then f — 2e < fy<£ < f, and so g := / y i £ + e G 11/ and d(f,g) < e. Proof. Consider a := inf / — e < inf / . Denote A := co (epi / U {(y, a)}) and let (x, t) E A and s > t. We want to show that t > a and (a;, s) E A. If these happen then epiipA = ^4 [VA being defined in Theorem 2.1.3(iv)], fy,e := VA £ r ( X ) , / y , £ (y) = a = inf /j, ) £ and fy>e < / . Moreover, if f(y) < inf / + e, then f(y) — 2e < a, which shows that (y, a) E epi(/ — 2e). As / — 2e < f, we obtain that A C epi(/  2e), and so /  2e < / y ) £ . Indeed, there exist the nets (Ai)igj C [0,1] and ((xi,ti))i€l C e p i / such that (a;,*) = lim, e / (\i(xi,U) + (1  Aj)(2/,a)), and so x  lim i 6 / (XiXi + (1 — Aj)y) and t = limjg/ (A;£; + (1 — Aj)a). Since [0,1] is a compact set, we may suppose that (A;) i e / > A G [0,1]. Of course, A;£; + (1  Xi)a > Ajinf / + (1  \i)a > a for every i E I, and so i > a. If (x,i) = {y,a) then limn^oo (^(x 0 , to +ns na) + n^{y, a)) = (x, s), and so (x, s) E A,
The general problem of convex
programming
103
where (xo,to) is a fixed element of e p i / . In the contrary case Aj > 0 for i >z_ j 0 for some i 0 £ I. Then limi^ io (Xi(xi,ti+X^1(st))+ (1Xi)(y,a)) = (x, s), and so (a;, s) £ A. Let u £ K; then (u,0) £ (epi/)oo Fixing (xo,io) £ e p i / , we have that (x0 + nu,io) £ e p i / for every n € N, and so ^(zo + nu, to) + ^^{y, a) £ A for every n £ N. Taking the limit we obtain that (y + u, a) £ A, whence fy<s(y + u) < a. Hence y + uG argmin/ y>£ . Conversely, let z € argmin/ y , £ . Assume that u := z  y ^ 0. It follows that (z,a) € A, and so there exist the nets (A;)j € / C [0,1] and ((xi,ti))ieI C e p i / such that (2,a) = lim i 6 / (Aj(x;,£;) + (1  Xi)(y,a)). Because z ^ y, A» > 0 for i y i0 (for some io € / ) • As above, we may assume that (A;)j 6 / >• A € [0,1]. If A ^ 0 then ((xj,ii)) i g / >• (A1.z + (1  X~1)y,a), a contradiction because ti > inf/ > a. Hence A = 0. It follows that limjg/ A; {xi,t{) = (u,0). Let (a;,i) £ e p i / . Then Aj (xi,U) + (1  A,)(a;,i) € e p i / , and so, taking the limit, we obtain that (x, t) + (u, 0) € epi / . This shows that (u, 0) £ rec(epi/) = (epi/)oo Therefore u £ K. Hence argmin/y ]E = y + K. • Theorem 2.5.5 Let X be a reflexive Banach space and f £ T(X) be such that K n K  {0}, where K := {u £ X  /<»(«) < 0}. Then the following statements are equivalent: (i) / is not coercive, (ii)
there exists g £Uf
such that argming = 0,
(iii)
{g &Uf \ argming = 0} is a dense G$ set (see page 34)
Proof. It is obvious that (iii) => (ii). (ii) =>• (i) Let g SUfbe such that argming = 0. From Theorem 2.5.1(h) we have that g is not coercive. Since g £ Uf, there exists 7 > 0 such that 9 — 7 < / < 5 + 7> which implies that / is not coercive, too. (i) =* (iii) First of all note that for any g £ Uf, g is not coercive and /oo = Soo (because /  7 < P < / + 7for some 7 € M+). If inf/ =  0 0 then Q := {g £ Uf \ argming = 0} = Uf, and so the conclusion holds. Let inf / £ R. Without loss of generality we assume that inf / = 0. For every n £ N consider the set Gn •= {g e Uf I min x e n C / x g(x) > inf g = 0}. The use of min instead of inf is possible because g is wlsc and nllx is wcompact. Also note that Qn is open in (Uf,d); just take g € Qn and
104
Convex analysis in locally convex spaces
0 < 5 < mixenUx g(x); then the ball B(g,8) C Qn. As Q = f]n€^Gn, it is sufficient to show that Qn is dense for any n > 1. For this fix n > 1, e > 0 and /i G 11/. We may assume that /i(0) < e (otherwise replace /i by h(xo + •)> where xo is taken such that h(xo) < e). There exists y G X such that h(y) < e and (y + if) n nUx = 0 Indeed, when K = {0} take y € X \ nUx such that h(y) < e; this is possible because the set [h < e] is not bounded (see Exercise 2.41). Assume now that K ^ {0}. Then there exist z G K and r > 0 such that (z + if) n rUx = 0, or equivalently z ^ r[/x — K. Otherwise K C fl r >o (rUx — K) C —K, a contradiction. The last inclusion is obtained as follows: consider z e Hr>o (rUx  K)\ then z = ^un — kn with un e Ux and kn € if, for every n € N, and so (fc„) >• —z € if. Consider g := /iy]£ +e. By Lemma 2.5.4 we have that g G 11/, d(/i, g) <e and argmin/i = y + if. Since (2/ + if) D nUx = 0, we have that g G QnTherefore Qn is dense in 11/. • Note that the reflexivity of the space was used in the proof of the preceding theorem only to ensure that the infimum of g on nUx is attained; so the preceding result remain valid when working on the dual of a normed space, the considered convex functions being w*lsc. Also note that the condition Xo := if fl—if = {0} in the preceding theorem is not essential; if Xo 7^ {0} one obtains a similar result taking into account the constructions in Exercise 2.24. For a similar result when X is not a normed space see Exercise 2.25. Another important problem in optimization theory is the uniqueness of the solution when it exists. The following result gives an answer to this problem. Proposition 2.5.6 Let f G A(X). Then S(f,X) is a convex set. Furthermore, if f is strictly convex then S(f, X) has at most one element. Proof. Let x G S(f,X); then S(f,X) = [f < f(x)], whence S(f,X) is convex. Let now / be strictly convex, and suppose that S(f,X) contains (at least) two distinct elements xi and xi\ since / is proper, S(f,X) C d o m / . Then we obtain the contradiction
v(f,X) < f (\Xl + x 2 ) < i/On) + \f[x2) = v(f,X). Therefore S(f, X) has at most one element.
•
The general problem of convex
programming
105
As we already know, the practical method for determining extremum points of a function is to determine the points which verify the necessary conditions then to retain those which verify the sufficient conditions. In convex programming we have a very simple necessary and sufficient condition for optimal solutions. Theorem 2.5.7 If f £ A(X), then x E dom / is a minimum point for f if and only if 0 E df(x). Proof. Indeed, f(x) < f(x) for every x E X if and only if x £ dom / and 0 < f(x) — f(x) for every x £ X, which means that 0 € df(x). O Therefore, in convex programming, the minimum necessary condition is also a sufficient condition. In optimization theory local optimal solutions also play an important role; if g : X —>• E, we say that x £ X is a local minimum (resp. local maximum) point if there exists V € Nx{x) such that f(x) < f(x) (resp. f(x) > f(x)) for every x £ V. The convex programming problems present a particularity. Proposition 2.5.8
Let f : X —• E be a convex function.
(i) Ifx(z dom f is a local minimum point for f, then x is also a global minimum point; (ii) if x € dom / is a local maximum point for f, then x is a global minimum point for f. Proof, for every therefore y := (1 
(i) By hypothesis there exists V £ Nx(x) such that f(x) < f(x) x € V. Suppose that there exists x £ X such that f(x) < f(x); f(x) £ E. Since V £ J^x(x), there exists A £]0,1[ such that X)x + Xx £ V. Therefore /(5?) < f(v) = / ( ( I  A)3F + Ax) < (1  A)/(3F) + Xf(x),
whence the contradiction f(x) < f(x). Therefore x is a global minimum point for / . (ii) By hypothesis there exists U £ J^cx such that f(x + x) < f(x) for every x £ U. If f(x) = — oo, it is clear that x is a global minimum point of / . Suppose that f(x) £ E (hence / is proper because x € int(dom/)). Then
Vx€U
: f{x) = f{\{x + x) + \{xxj)
<\f{x + x) +
\f{xx)
106
Convex analysis in locally convex spaces
Therefore f(x) = f(x) > f(x) for every x € x + U € N j ( x ) . Hence a; is a global minimum point of / . D This result explains why in a convex programming problem we look only for global minimum points. In practical problems, solved numerically on computers, frequently it is not possible to determine the exact optimal solutions (because one works with approximate values). A simple example in this sense is the problem of minimizing the function / : E —» R, f{t) := (t — 7r)2. Taking into account this fact, the notion of approximate solution is proved to be useful. More precisely, if e e R+, we say that x £ C is an e (optimal) solution of problem (P) if f(x) < f(x) + e for every x 6 C; we denote by SS(P) or Ss(f,C) the set of esolutions of problem (P). It is obvious that when C fl dom / / 0 we have that Se(f, C) = Se(f, X); moreover, if / is proper and S£{f,C) ^ 0, then v(f,C) G R and Se(f,C) = {x £ C  f(x) < v(f, C) + e}. Related to esolutions we have the following result. Proposition 2.5.9 Let f £ A(X), x e d o m / and £ € P. Then Se(f,X) is convex; the set Se(f,X) is nonempty if f is bounded from below. Furthermore, x € Sc(f,X) if and only if 0 6 d£f(x). •
2.6
Perturbed Problems
We shall see (especially) in the following two sections that it is very useful to embed a minimization problem (P)
min f(x),
x 6 X,
in a family of minimization problems. In this section X, Y are separated locally convex spaces if not stated explicitly otherwise and / : X —> R. Let us consider a function $ : I x F 4 l having the property that f(x) — $>(x, 0) for every x € X; $ is called a perturbation function. For every y £ Y consider the problem (Py)
min $(x,y),
x 6 X.
It is obvious that problem (P) coincides with problem (Po). To remain in the convex framework, we assume that $ is a convex function. Of course, when / : X —> R is convex we can find a function $ having the demanded property: $(x,y) := f(x) for all (x,y) E X xY. For
Perturbed
problems
107
obtaining useful results one chooses adequate perturbation functions, as we shall see in the sequel. Let $ : I x 7 4 l b e a convex function and h : Y » E, h{y) = v(Py), its associated marginal function (see page 43); h is also called the value or performance function associated to problems (Py). As noted in Theorem 2.1.3(v), h is convex, while from Eq. (2.8) we have that dom/i = Pry(dom$). The problem (P)
min $(z,0), x £ X,
is called the primal problem; we associate to it, in a natural way, the following dual problem {D)
max ($*(0,2/*)), y* € Y*.
It is obvious that (£>) is equivalent to the convex programming problem (!?')
min $*(0,2/*), y*
eY*.
The equivalence has to be understood in the sense that the problems (D) and (£>') have the same (e)solutions; moreover v(D') = —v(D) (of course, for a maximization problem the notions of (e)solution, local solution and value are defined dually to those for minimization problems). It is nice to observe that (£>') and (P) are of the same type. In the following results we establish some properties which connect the problems (P), (D) and the function h. Theorem 2.6.1 Let $ : X x Y > E and h : Y >• E be the marginal function associated to $ . Then: (i) h* (y*) = $* (0, y*) for every y*
eY*.
(ii) Let (x,y) € X x Y be such that ${x,y) 6 R. Then (0,j/*) € d$(x,y)
O h(y) = $(x,y)
and y* £ dh{y).
(hi) v(P) = h(0) and v(D) = /i**(0). Therefore v(P) > v(D); in this case we say that one has weak duality. (iv) Suppose that $ is proper, x € X and y* € Y*. Then (0,y*) € d$(x~, 0) «/ and only if x is a solution of problem (P), y* is a solution of (D) and v(P) = v{D) G E. Assume, moreover, that $ is convex. (v) /i(0) £ E and /i is Isc at 0 <£> u(P) = u(D) £ E; in this case one says that we have strong duality.
108
Convex analysis in locally convex spaces
(vi) h(0) G E and dh(0) / 0 & v(P) = v(D) G E and (D) has optimal solutions. In this situation S(D) = dh(0). (vii) Suppose that $ is proper. Then [ h is proper] •£> [ h* is proper] •£> [ h is minorized by an affine continuous functional] •& 3y* 6 7 ' , 3 a GE, V(x,y)eXxY Proof,
: $(x,y)
> (y,y*) + a.
(2.52)
(i) We have
h*(y*) = sup({y,y*)  h{y)) = sup ((y,y*)  inf y€Y
y€Y \
= sup sup((y,y*)
$(x,y))
=
y€YxeX
$(x,y))
^ ^
sup
/
((x,0) + (y,y*) 
${x,y))
(x,y)€XxY
= **(<), y*). (ii) Assume that $(x,y) Theorem 2.4.2 (iii), h(y) < *(x,y)
G E. Let (0,j/*) G d$(x,y).
Then by (i) and
= (x,0) + (y,y*)  $*(0,i/*) = (y,y*)  h*(y*) < h(y),
and so h(y) = $(x,y) hold then
and y* G dh(y). Conversely, if these two conditions
*{x,y) = h(y) = {y,y*)  h*{y*) = (x,0) + (y,y*)  **(0,y'), whence, again by Theorem 2.4.2 (iii), (0,2/*) G d$(x,y). (iii) It is obvious that v(P) — /i(0); moreover, v(D) = sup ($(0,r/*)) = sup ((0,2/*) h*(y*)) y*EY*
= h**{0).
y*&Y*
Therefore v(P) >v(D). (iv) If (0,7*) G<9$(x,0) then v(P) = h(Q) < $(x,0) = $*(0,2/*) < v(D) = x**(0) < h(0), whence x is a solution of (P), y* is a solution of (£>) and v(P) = v(D) G E. Conversely, if these last assertions are true, we have $*(0,j7*) = h**(0) = h(Q) = S(z,0) G E, whence ( z , 0 ) G d o m $ and §(x,0)+
**(0,y*) = (x,0) + (0,y*),
Perturbed problems
109
which shows that (0,y*) G d$(x,0). Assume now that $ is convex. (v) If h is lsc at 0 and h(0) G R we have that h(0) G E, whence h** = ~h. Therefore v(D) = h**(0) = ft(0) = v{P) G E. Conversely, if this last relation is true, then h(0) = h(0), i.e. h is lsc at 0. The conclusion holds. (vi) Suppose that /i(0) G E and dh(0) ^ 0; then /i is lsc at 0, whence, by (iv), we have that v(P) = v(D) G E. Let y* G dh(0). Then h(0) + h*(y*) = 0, whence Vy* G Y* : v(D) = v(P) = h(Q) = h*(y*) = **(<), IT) >
*'{0,y*).
Therefore 0 ^ dh(0) C S(D). Conversely, suppose that v(P) = v(D) G R and that (D) has solutions. Let y* G S(D). Then ft(0) = h**(0) = h*(y*) G E, whence y* G 9ft(0). Thus we have that 0 # S(D) C 0ft(O). (vii) The mentioned equivalences are obvious since h is convex, and dom/i^0. • We say that the problem (P) is normal if v(P) = v(D) G E; (P) is stable if w(P) = v(D) G R and (D) has optimal solutions. Theorem 2.6.1 above gives characterizations of these notions. In the following result we establish formulas for deh(y) and deh*(y*) when h is proper. Theorem 2.6.2 e G E+ . Then:
Let $ G A(X x Y) satisfy condition Eq. (2.52) and
(i) /or every y G dom h
deh(y) = f l , > 0 LUjcfo* e y * I (°'^) G ^+^0^)} 7)>0
*(x,y)
(ii) /or every y* G dom/i* ; cU*(y*) = p 
cl{yGY3xGX
: (Q,y<) €
de+v*(x,y)}.
110
Convex analysis in locally convex spaces
Proof,
(i) It is obvious that
n
{»*i(o,!/ , )6fl t+ ,^»)}
n
V>0
$(x,y)
c
n.„U ( = >*l(o,i/*)ea e + ) J $( a :,i/)}
1
l
ri>0
^^x€X
C a e /i(y). Let y* G deh(y), i.e. h(y) + h*{y*) < (y,y*) +e, and let rj > 0 and a; € X be such that $(x,y) < h(y) + rj. Then $(x, 2/) + $*(0, y*) < ft(y) + r, + h*{y*) < (y, y") + e + r,, i.e. (0,2/*) € d£+n$(x,y).
Therefore
dsh(y) C f
f
{y*\ (0,y*) G &+„*(*, y)},
V>0
and so the desired equalities hold. (ii) Let y € d£h*(y*) and r\ > 0. Then />**(
= % ) + h*(y*) < (y,y*) + e.
It follows that for every V G N(j/) there exists yv G V such that Ml/v)+ &*(!/*) <
+ h*(y*) = $(xv,yv)
i.e. (0,y*) G de+n
+ $*(0,y*) < (yv,y*)
+e + rj,
Therefore
j / G c l { t / G y  3 a ; G X : (0,y*) G 3 £ + 7 ? $(z,y)}. Taking into account that 77 > 0 is arbitrary, we obtain that d£h*(y*)cf)
d{y\3xGX
: (0,y') G
de+r,*(x,y)}.
Let y be in the set from the righthand side and 77 > 0. Then, using the continuity of y*, there exists Vo G N(y) such that (z,y*) < (y,y*) + r)/2 for every z G VQ. On the other hand, for every V G N(y) there exist
Perturbed problems
(xv,Vv) hence
111
€ X x Y such that yv GVTlVo and (0,y*) G
Kw) + h*{y*) < ${xv,yv)
+ $*(0,y*) < (yv,y*)+e
de+r,/2$(xv,yv);
+ i]/2 < (y,v*)+e+T].
Therefore V V e N f e ) : inf % ) + A*(y*)< (J/,!/*>+£ + »/, yev i.e. h(y) + h*(y*) < (y,y*) +E + TJ. Since TJ > 0 is arbitrary, we obtain that h"(y) + h*(y*) = h(y) + h*(y*) < (y,y*) + e, i.e.ytd£h*(y*). • The preceding result is useful for deriving formulas for esubdifferentials for other functions. Theorem 2.6.3 Let $ E T(X x Y) and cp : X » E, ip(x) := Then for every e E E+ and every x E dom ip we have dMx)=C\ = f
F(x,0).
w*c\{x*eX*\3y*eY*
: (x*,y*) E
d£+v$(x,0)}
w*d{x*eX*\3y*eY*
: (x,0) E
de+v^(x*,y*)}.
Proof. Let us consider the spaces X*, Y* endowed with their weak* topologies and the function k:X*^W,
k(x*):=
inf
y*€Y*
$*(x*,y*).
Then k* : X >• I , A;*(a;) = $**(x,0) = $(x,0) = ip(x). Using Theorem 2.6.2, we have dMx)
= dEk*(x) = f l = f 1
The proof is complete.
w*c\{x*  3j/* : (a;,0) E &+,**(**>!/*)} >J1>0 '77 X )
W*C1{Z*3T/*
: (x'.y'Jeft+^O)}. D
Let us apply the results of Theorems 2.6.2 and 2.6.3 in some particular situations. Stronger conclusions, but under more restrictive conditions, will be established in Section 2.8.
112
Convex analysis in locally convex spaces
Corollary 2.6.4 for which
Let A G H{X,Y)
3y* £Y*,
and f : X —> R be a convex function
3a G R, Vz e X : f(x)>
(x,A*y*)+a.
If (Af)(y) G 1 (O y G A(dom/) = d o m A / ) , j / * G dom(/* o A*) and e > 0, then
de{Af){y) = fl n>0
[J
A\de+rif{x))
Ax=y
=n
n
»7>0
A*\d£+rif(x)),
Ax=y,f(x)<(Af)(y)+r,
and ds(f*oA*)(y*) Proof.
= f]d{yeY\3xeX
: Ax = y, A*y* G
de+nf(x)}.
Let us consider K *,y) ,in
•=
'  oo
if
Axjty.
Then $*(x*,t/*) = /*(x* + A V ) and (O.y*) G 0„$(a;,y) <S> Ax = y, A*j/* G fy/fr). The result follows immediately from Theorem 2.6.2. Corollary 2.6.5
Let A G L(X, Y) and f G T(Y).
d£(foA)(x)
=
D Then
O^lv'dA^de+rffiAx))
for every x G A _ 1 ( d o m / ) = dom(/ o A) and every e > 0. Proof. Let us consider $ : I x y ^ l , $(x,y) := f(Ax+y), and ip(x) :— $(x,0) = f(Ax). Then$*(x*,y*) = f(y*) ii A*y* =x*, $*(x*,y*) = oo otherwise. Moreover (x*,y*)edv*(x,0)&A*y*=x'
and f(Ax) + f*(y*)
& A*y* = x* and y* G The conclusion follows from Theorem 2.6.3.
< (Ax,y*) + rj
dvf(Ax). •
The fundamental
Corollary 2.6.6 3x* eX*, If (hnf2)(x)
duality
formula
113
Let / i , / 2 € A(X) for which
3a e l , \/xeX,
V i e {1,2} : fi{x) > {x,x*) + a.
e E and e > 0, then:
de(fiDf2){x)
= f) n>0
1J
(0 e i /i(a; 
y)ndeJ2(y))
yeX,Ei>0,e+n=ei+£2
=n n ri>0yeSrl(x)
u (^x/i^^n^/ad/)),
£i>0,e+r)=£i+£2
d{huf2){x) = r\v>0Uy€X (Wx
 y)n ^/a(i/)).
where Sv(x) := {y e X \ Mx  y) + f2{y) < {fiDf2)(x)
+ r?}.
Proof. Let us consider / : X x X )• E, f{x\,x2) := fi(xi) + f2(x2) and A e £ ( X x X , X ) , A(xi,x2) := £i + £2 The conclusion follows from Corollary 2.6.4. • Corollary 2.6.7 then:
Let / i , / 2 € T(X). / / a; e d o m / i n d o m / 2 and e > 0
ds{h+mi)=n ™*d ( u (^Aw+aeaMx)) 7)>0
5(/i + f2)(x)
= fl
y£i>0,£+77=ei+e 2
t i T  d ^ / i C * ) + d„f2(x)).
Proof. Let us consider / : I x I  > I , / ( ^ i i x2) := / i ( x i ) + 72(^2) and A e £(X, X x X), Ax := (x,x). The conclusion follows from Corollary 2.6.5. •
2.7
The Fundamental Duality Formula
The following theorem is very useful for obtaining important results in convex programming; this is the reason for calling formula (2.53) the fundamental duality formula of convex analysis. Theorem 2.7.1 Let $ e A(X x Y) be such that 0 6 P r y ( d o m $ ) . Consider Yo := lin (Pry (dom $ ) ) . Suppose that one of the following conditions is satisfied:
114
Convex analysis in locally convex spaces
(i)
there exists Ao £ E such that VQ := {y £ Y \ 3x £ X, $(x,y)
<
AO}S:NVO(O);
(ii)
there exist Ao G E and x0 £ X such that VUeNx
: {y£Y\3x£x0
+ U, $(x,y)
< Ao} e Ky o (0);
(iii) there exists XQ £ X such that (a;o;0) £ d o m $ and $(xo,) tinuous at 0;
is con
(iv) X and Y are metrizable, epi $ satisfies condition (Hwz) on page 14 and 0£ i 6 ( P r y ( d o m $ ) ) ; 0e
(v) X is a Frechet space, Y is metrizable, $ is a liconvex function and i6 (Pry(dom$));
(vi) X is a Frechet space, $ is Isc and 0 £
j6
(Pry(dom$));
(vii) X, Y are Frechet spaces, $ is Isc and 0 €
lc
(Pry(dom$));
(viii) dimlo < 00 and 0 £ *(Pry(dom$)); (ix) there exists Xo £ X such that sets {0}, P r y ( d o m $ ) are united.
$(XQ,)
is quasicontinuous and the
Then either h(0) = —00 or h(0) £ E and h\y0 is continuous at 0. In both cases we have inf $ ( x , 0 ) = max (  **(0,i/*)). x€X
'
y*€Y'
v
"
(2.53) '
Furthermore, x £ X is a minimum point for $(,0) if and only if there exists y* £ Y* such that (0,y*) £ d$(x,0). Proof. Since 0 £ P r y ( d o m $ ) , h(0) < 00. If h(0) = —00 we have h*(y*) — °° f° r every y* 6 Y*, whence —$*(0, j/*) =  0 0 = h(0) for every y* £ Y*. Therefore the conclusion is true in this case. Let us consider now the case h(0) £ K. Suppose that condition (i) is verified. Then h\y0 is bounded above by Ao on VQ. Since h is convex, h\y0 is continuous at 0. By Theorem 2.4.12 we have that dh(0) f 0. Then relation (2.53) follows Theorem 2.6.1 (vi). (ii) =£ (i) This implication is obvious (just take U — X). (iii) => (ii) Suppose that (iii) holds. Since $(xo,) is continuous at 0, the set V0 := {y  $(xo,y) < $(x0,0) + 1} is a neighborhood of 0 (in particular YQ = Y). Taking Ao := $(a;o,0) + 1, it is obvious that V0 C {y \3x £ x0 + U : $(x,y) < A0} for every U £ Nx Therefore (ii) holds.
The fundamental
duality
formula
115
(iv) => (ii) Suppose that (iv) holds. Let us consider the relation 31 : X xR=$Y whose graph is given by
grtt := {(x,t,y) \ (x,y,t) € epi$}. From the hypothesis 51 satisfies condition (Hwi), whence, by Proposition 1.2.6(i), % satisfies condition (Hw(z,i)), too. Moreover 0 € i6 (Im3?). Let (xo,io) e X x R be such that 0 6 Jl(xo,to) Applying Theorem 1.3.5 we obtain that "R{{x0 + U)x ]  oo,t0 + 1]) £ ^y 0 (0) for every U eNX This shows that (ii) holds with Ao := t0 + 1. (v) => (ii) By Proposition 2.2.18, there exist a Frechet space Z and a csclosed function F : Z xX xY ^R such that $(x, y) = inf z e z F(z, x, y) for all (x, y) € X xY. The conclusion follows like in the preceding case by replacing X by Z x X, x by (z,x) and U by Z x U. (vi) => (ii) The proof is the same as for (iv) =$• (ii) with the exception that one uses Ursescu's theorem (Theorem 1.3.7) instead of Simons' theorem. It is obvious that (vii) implies conditions (iv), (v) and (vi). If (viii) is verified, the conclusion follows from Theorem 2.4.12. (ix) => (i) It is obvious that $(a; 0 , •) > h. By Proposition 2.2.15 we have that h is quasicontinuous. It follows that rint(dom/i) ^ 0. Using Proposition 1.2.8 we obtain that 0 £ rint(dom/i). Therefore h\y0 is continuous at 0, and so (i) holds. Of course, if x is a minimum point for $(,0) then x is a solution of (P) (from p. 107); it follows that $(x,0) = v(P) = v{D) G R. Let y* be a solution of (D) (in our conditions a solution exists). Then, from Theorem 2.6.1(iv), (0,2/*) £ d$(x,0). The converse implication follows from the same result. • When the properness condition in the preceding theorem is violated relation (2.53) is automatically verified. Indeed, in this case there exists (xi,yi) £ X x Y such that $(xi,j/i) =  o o , and so h(yi) = —oo. Because 0 € '(dornh), by Proposition 2.2.5 we have that h(0) —  o o . In applications it is important to have conditions on $ which also ensure that the functions $, $(a;, y) := <&(x,y) — {x,x*) with a;* 6 X*, satisfy them. Such conditions are conditions (ii)(ix) of Theorem 2.7.1, but this is not always true for (i).
116
Convex analysis in locally convex spaces
Other conditions of this type are: 3A 0 G R, 3B G %x : {y G Y  3x G B, $(x,y) V?7 G Kx, 3A > 0 : {yeY\3x£XU,
$(x,y)
< A0} G >JVo(0), (2.54) < A} G Ny o (0), (2.55)
where "Bx is the class of bounded subsets of X and, as in Theorem 2.7.1, Y0 = l i n ( P r y ( d o m $ ) ) . Proposition 2.7.2 Let $ G A(X x F ) . Conditions (i) — (viii) foez'ng those from Theorem 2.7.1, we have: (iii) =>• (2.54) =S> (2.55) «• (ii) => (i), (2.55) => (2.54) «/ X is a normed vector space, (vii) =>• (iv) A (v) A (vi), (iv) V (v) V (vi) =* (ii) and (viii) =* (2.54). Moreover, taking D = P r y ( d o m $ ) , one has: if dim(linZ)) < oo t/ien *D = rint£>; i/ X, Y are metrizable, e p i $ satisfies H(x) and lbD ^ 0 then thD = rintZ?; similarly for the situations corresponding to conditions (v)(vii). Proof. The implications (vii) =>• (iv) A (v) A (vi), (iv) V (v) V (vi) =>• (ii) =*• (i) were already observed (or proved) during the proof of the preceding theorem. The implication (iii) =$• (2.54) is obvious; just take B — {XQ}. (2.54) =>• (2.55) Let U G K x ; there exists /x > 0 such that B C fill. Taking A = max{Ao,/i} we have that {yeY\3x£B,
$(x,y)
< A0} C {y G Y \ 3x G ^?7, $(a;,j/) < A0} C f e e Y  3 x G XU, 9(x,y)
< A}.
The conclusion follows. (ii) =>• (2.55) Consider A0 G M. and zo G X given by (ii). Let U G N * . There exists fi > 0 such that a;0 € /if/. Let V = {y€Y\3x£x0 + U, $(x,y) < Ao} G Ny 0 . Taking A = max{Ao,/« + 1} and y € V, there exists x G x0 + U such that $(x,y) < A0. As x G x0 + U C /J,U + U = (fi + 1)U C At/, the conclusion follows. (2.55) =>• (ii) It is obvious that there exists x0 G X such that $(xo, 0) < oo. Consider Ao = max{$(a;o,0),0} + 1 and let U G Nx There exists Uo G Nx such that UQ + Uo C U. There exists also Ai > 0 such that XQ G Aif/0. By hypothesis, there exists A > Ao + Ai such that Vb = {y G y  3a; G AE/0, #(a;,i/) < A} G Ny o (0). Let V = X^VQ and take yeV. As Xy G Vo> there exists x' G Af/o such that $(x',Xy) < X. It follows that $ ((1  A^Xzo.O) + A " V , A v ) ) < (lA 1 )$(a;o,0)+A 1 $(a;',A 2 / ) < A0,
The fundamental
duality
formula
117
whence $ (x,y) < Ao, where x := xo + X~1(xl — Xo) G Xo + Uo + Uo C Xo + U. (2.55) => (2.54) when X is a normed vector space. Take U = Ux = {x G X  x < 1}. There exists A0 > 0 such that {y G Y \ 3x G X0U, ${x,y) < Ao} € Ny o (0). As XoU is bounded, the conclusion follows. (viii) =$• (2.54) Suppose that dim Yb < oo and 0 £ i ( P r y ( d o m $ ) ) . It follows that there exist j / i , . . . ,ym G Pry(dom$) such that Vo = co{y\,..., 2/m} £ !Nyo(0). For every i € l , m there exists Zj G X such that (xi,yi) G d o m $ . Let Ao = max{$(a;j,y;)  1 < i < m} and B — co{xi,..., xm}. It is obvious that B is bounded and for y G Vo there exist A j , . . . , Am > 0 with Y!T=i ^i = 1 s u ch that j / = YlT=i ^'Vi Then a; = YllLi ^ixi € ^ anc ^ *(z,2/) < A0. The fact that lD — v'mtD if dim(lin£>) < oo is obvious. Let X, Y be metrizable, e p i $ satisfy (Hx), and consider y0 G lbD. Taking <J?0 defined by
for every x* G X*. In particular ip G T*(X*). Proof. It is obvious that $(,0) G r ( X ) in our conditions. Let x* G X* and consider $ : I x F  > I denned by $(x,y) := $(x,y) — (x,x*). As observed above, the function $ satisfies the same condition as $ among those mentioned in the statement of the corollary. Applying Theorem 2.7.1 (and eventually the preceding proposition), we have that inix&x $(x,Q) = m a x y . e y . (  $ * ( 0 , ?/*)). But $*(0,2/*) = $*(x*,y*), and so the conclusion follows. • We state another duality formula which will be useful in the sequel. Theorem 2.7.4 Let F G A(X x Y), 6 : X =t Y be a convex multifunction, and D = U{G(x) — y \ (x,y) G d o m F } . Assume that 0 G D and let Y0 = linD. If one of the following conditions holds: (i) for every U G Nx there exist A > 0 and V G Ny0 such that {0} x V C g r C n (XU xY)[F<
A];
Convex analysis in locally convex spaces
118
(ii) there exist Ao G K, B G T>x and VQ G Ny0 such that {0} x V0 C grC n (B x Y)  [F < A0]; (iii) there exists (xo,yo) G grC (~l d o m F such that F(xo, •) is continuous at y0; (iv) X, Y are metrizable, 0 6 lbD and either F is cscomplete and C is csclosed, or F is csclosed and C is cscomplete; (v) X, Y are Frechet spaces, F and C are liconvex, and 0 G lbD; (vi) dim YQ < oo and 0 € *£>, £/ien £/iere exists z* G Y* such that inf{F(x,y) Proof.
\ (x, y) G gr 6} = inf{F(x,y) + (z, z*) \ (x,y + z) e gr e } .
Let Z := Y and
* : ( I x Z ) x 7 4 l ,
$(x,z;y) := F(x,z) + t g r e ( x , y + z).
It follows easily that $ is convex and Pry(dom$) = D. It is obvious that the conclusion of the theorem is equivalent to inf( X ] 2 ) e xxZ^( a ; , 2 : iO) = max^.gy. — $*(0,0;2/*). So we have to show that if one of the conditions of the theorem is verified then a condition of Theorem 2.7.1 holds. If (i) holds it is immediate that $ verifies condition (i) of Theorem 2.7.1. The implication (ii) => (i) is obvious. We also have that (iii) => (ii); just take B := {x0}, A0 := F(x0,yo) + 1 and V0 = {y G Y \ F(x0,y0+y) < A0} (in this case Y0 — Y). Similar to the proof in Proposition 2.7.2 we have that (vi) =>• (ii). (iv) => (i) Let Y^n>i ^n(xn, zn, yn, tn) be a convex series with elements of epi $ such that J2n>i ^nXn and J2„>i ^nzn are Cauchy, J2n>i Kyn = y G Y and J2n>i ^tn = t G K Then (xn,zn,tn) G e p i F and (xn,zn + yn) G grC for every n > 1. If F is cscomplete it follows that J2n>1 Xnxn and S n > i ^nZn are convergent with sums x G X and z G Z, respectively; moreover, (x,z,t) G epi.F, whence (x,z + y) G grC since grC is csclosed. The same conclusion holds in the other case. Therefore $ verifies condition (H(x,z)) in our hypotheses. Hence condition (iv) of Theorem 2.7.1 is verified. So, by Proposition 2.7.2, relation (2.55) holds. Therefore for U x Y G Nxxz there exists A > 0 such that {y G Y  3 (x, z) G XU x Y, $(x, z; y) < A} = {y&Y\3xG\U,
3zGZ
: (x,y + z) G e, F{x,z)
< A} G Ny o (0),
The fundamental
duality
formula
119
and so (i) holds. (v) => (i) Consider the set A:={(x,z,y)
eX
x Z xY
\y + z£
= {{x,y' y,y)\{x,y')egre, = grC x {0} + {0} x {(y,y)
G(x)} yeY}
\y
eY}.
Using Propositions 1.2.4 (ii) and 1.2.5 (ii) we obtain that A is liconvex (as sum of two liconvex subsets of a Frechet space). Since $(x,z;y) — F(x, z) + LA{X, Z, y) and the functions F and LA a r e liconvex, $ is liconvex, too. Thus $ satisfies condition (v) of Theorem 2.7.1; the other conditions being obviously satisfied, as in (iv) =4 (i), we get that (i) holds, too. • Note that every condition of the preceding theorem is verified by F, F(x,y) — F(x,y) — (x,x*), where x* € X*, when the same condition is verified by F. Taking gr 6 = X x {0}, the conclusion of the preceding theorem is just the conclusion of Theorem 2.7.1. Conditions (i), (ii), (iii) and (vi) become conditions (2.55), (2.54), (iii) and (viii) of Theorem 2.7.1, respectively; to conditions (iv) and (v) correspond slightly stronger forms of conditions (iv) and (v) of Theorem 2.7.1, respectively. Remark 2.7.1 If F(x,y) = f{x) + g(y) with / € A(X), g € A(Y), for condition (i) of Theorem 2.7.4 it is sufficient (and necessary if f,g have proper conjugates) to have VUelSx,
3 A > 0 , 3V € X y 0 : V C [g < A]  e(\U n [/ < A]),
while for condition (ii) of Theorem 2.7.4 it is sufficient (and necessary if / , g have proper conjugates) to have 3 A0 S 1, Be Sjr, V0 £ NYo
: V0 C [g < A0]  G{B n [/ < A0]).
Of course, these two conditions are equivalent if X is a normed space. Corollary 2.7.5 Let f € A(X) and A 6 &(X,Y). the following conditions is verified:
Suppose that one of
(i) / is continuous on int(dom/), assumed to be nonempty, and A is relatively open, i.e. A(D) is open in ImA for every open subset D C X; (ii) X and Y are metrizable, either (a) / is cscomplete or (b) / is csclosed and gr A satisfies condition (Hx), and * 6 A(dom/) ^ 0;
120
Convex analysis in locally convex spaces
(iii) X is a Frechet space, Y is metrizable, f is a liconvex function and A(domf)jt
ib
(iv) X is a Frechet space, f is lower semicontinuous and %bA(dom f) ^ 0; (v) dim (lin A(dom/)) < oo. Then, either Af is —oo on *(^4(dom/)) or Af is proper and {Af)\y0 continuous on ' (A(dom f)). Moreover Vj/G^dom/)) Proof.
: (Af)(y) =max{{y,y*)
 r(A*y*)\y*
is
eY*}.
Let us consider y0 6 l (A(dom/)) and $ : X x Y *• I ,
$(ar,y) := f(x) + tgTA(x, 2/o + y).
We have that Pry (dom $) = yl(dom/)  y0 If condition (ii), (iii), (iv) or (v) is verified, then $ satisfies condition (iv), (v), (vi) or (viii) of Theorem 2.7.1, respectively. Suppose that condition (i) is satisfied. (Obviously, it is impossible that condition (iii) of Theorem 2.7.1 be verified in this case: F(xo, •) = i{Ax0}) Since A is relatively open we have that *(A(dom/)) = intim/i (A(dom/)) = A(int(dom/)) (Exercise!), whence j/o = AXQ for some Xo € int(dom/). It follows that / is bounded above on a neighborhood Vo of Xo, and so h (the marginal function associated to $) is bounded above by the same constant on J4(VO) — yo, which is a neighborhood of 0. Therefore condition (i) of Theorem 2.7.1 is verified. So, under each of the five conditions, we have that (Af)(y0)
= inf $(a:,0) = max ($*(0,y*)) = max « y 0 , y * >  / * ( > l V ) ) , xex y*eY* yer*
doing similar calculations to those from Theorem 2.3.1(ix).
•
Corollary 2.7.6 Let f1}..., fn G A(X) and take f : / i D • • • • / „ . Suppose that one of the following conditions is verified: (i) / i is continuous at some point in dom / i , (ii) i6
X is a Frechet space, the functions / i , . . . , / n are liconvex and
(dom/)^0, (iii) dimX < oo.
Then either f is —oo on ! ( d o m / ) or /aff(dom/) is finite and continuous on (domf), in which case f is sub differentiable on this set. Furthermore, for every x € ' ( d o m / ) we have that
t
f(x) = max ((*, x*)  / • ( * • )
/ • ( * ' ) ) = ( / ; + ••• +
rn)*(x).
Formulas for conjugates and esub differentials
121
Proof. Recall that dom / = dom / i \ + dom / „ . In the cases (ii) and (iii) the result is an immediate consequence of Corollary 2.7.5 taking $ and A as in Corollary 2.4.7, while for (i) one does the proof by induction. •
2.8
Formulas for Conjugates and eSubdifferentials, Duality Relations and Optimality Conditions
In the preceding sections we have considered only situations in which it was simple to compute conjugates and esubdifferentials: sum of functions with separated variables, convolution of convex functions, and functions of type Af. For the other types of functions, generally, it is more difficult to compute the conjugate functions or the esubdifferentials. In this section, we intend to establish sufficient conditions, as general as possible, in order to ensure the validity of such formulas. In this section the spaces X,Xi,..., Xn and Y are separated locally convex spaces if not stated explicitly otherwise. We begin with the following result. T h e o r e m 2.8.1 Let F £ A(X x Y), A £ H{X,Y) and
E,
x such that {0} x Vo C {{x,Ax)
\xeB}[F<
Ao];
(ii) for every U £ Nx there exist A > 0 and V £ Ny0 such that {0} x V C {(x, Ax)\xe
At/} ~[F<
A];
(iii) there exists xo £ X such that (:ro,j4a;o) £ d o m F and F(xo,) continuous at AXQ;
is
(iv) X and Y are metrizable, 0 E tbD and either F is csclosed and gr A is cscomplete or F is cscomplete; (v) X is a Frechet space, Y is metrizable, F is liconvex and 0 £ lhD; (vi) X is a Frechet space, F is Isc and 0 £ lbD; (vii) X and Y are Frechet spaces, F is Isc and 0 £ %CD; (viii) dimy 0 < oo and 0 £ 'D,
Convex analysis in locally convex spaces
122
(ix) there exists XQ £ X such that F(xo, •) is quasicontinuous and {0} and D are unitedThen for x* £ X*, x £ domip and e > 0 we have: tp*(x*) = mm{F*(x*  A*y\y*) dM*) Proof.
= {A*y*+x*
\ y* £ Y*},
(2.56)
 (x*,y*) € dEF(x, Ax)}.
(2.57)
It is easy to verify that Vx* £ X* :
 A*y*,y*) \ y* £ Y*}
for every function F and every operator A £ &(X, Y). Consider $ : X x Y » E, $(x,y) = F(x,Ax — y). Then $ satisfies one of the conditions (2.54) or (ii)  (ix) of Theorem 2.7.1 when F satisfies one of the conditions (i) — (ix), respectively. It follows that the function $, defined by
 (x,x*) \ x £ X} = inf {$(z,0)  x £
X).
Applying Theorem 2.7.1, we obtain that  V ( a ; * ) = i n f { S ( x , 0 )  X e X} = max {  * ' ( 0 , y") \y* £Y*}.
(2.58)
But **(0, V*) = sup{(ar, a;*) + (y, y*)  F(x, Ax  y) \ (x,y) £ X x Y} = sup{(a;,a;*) + (z  Ax,y*)  F(x,z)
\ (x,z) £ X x Y}
= sup{(a;, x*  A*y*) + (z, y*)  F(x, z) \(x,z)£Xx =
Y}
F*(x*A*y*,y*).
Prom Eq. (2.58) we get immediately Eq. (2.56). Note that the inclusion "D" in Eq. (2.57) is true for every function F and every operator A £ L(X,Y). Let x £ domip and x* £ detp(x). Then (see Theorem 2.4.2) cp(x) + ip*(x*) <
(x,x*)+e.
By Eq. (2.56) there exists y* £ Y* such that
+ F*(x*  A*y\y*)
A*y*,y*),
< (x,x*  A*y*) + (Ax,y*) + e,
Formulas for conjugates and
ssubdifferentials
123
i.e. (x*  A*y*,y*) =: (x*,y*) £ deF(x,Ax). Therefore x* = x* + A*y*, which proves that the inclusion "C" in Eq. (2.57) holds, too. • Note that (viii) V (hi) => (i) => (ii), (iv) V (v) V (vi) =>• (ii), (vii) => (iv) A (v) A (vi), and (ii) =$• (i) if X is a normed space. Note also that similar properties to those stated in the second part of Proposition 2.7.2 can be given for the situations of Theorem 2.8.1. Corollary 2.8.2
Under the conditions of Theorem 2.8.1 we have that
inf F(x,Ax) xeX
= max (  F*(A*y*,y*)).
(2.59)
y*€Y*
Furthermore, x~ is minimum point for
there exist Xo £ R, B £ "Bx and VQ £ Ny0 such that V0CA{[f<\0\nB)[g<\0\;
(ii) for every U £ J^x there exist A > 0 and V £ Ny0 such that VcA([f<X]nXU)[g<X\; (hi) there exists xo £ d o m / n A~l(domg) AxQ;
such that g is continuous at
(iv) X,Y are metrizable, 0 £ lb(A(dom f) — domg), f and g have proper conjugates, either f, g are csclosed and gr A is cscomplete, or f, g are cscomplete;
124
Convex analysis in locally convex spaces
(v) X is a Frechet space, Y is metrizable, f, g are liconvex functions and Oeib (A(dom / )  dom g); (vi) X is a Frechet space, f,g are Isc and 0 £ lb(A(dom f) — domg); (vii) X, Y are Frechet spaces, f,g are Isc and 0 £ tc (^4(dom/) — domg); (viii) dimlo < oo and 0 € *(A(domf) — domg); (ix) g is quasicontinuous and A(dom/) and domg are united; (x) Y = Wl, qri(dom/) ^ 0 and 4 ( q r i ( d o m / ) ) l~l Momg ^ 0. Then for every x* 6 X*, x 6 domy? and e > 0 we have: p ' O O = min{/*(**  A*y*) + g*(y*)  y* € F * } , de
+ d£2g(Ax) \ si,e2 > 0, ei + e 2 = e},
^ ( x ) = fl/(a:) + A*(dg(Ax)).
(2.60)
P r o o / . We apply Theorem 2.8.1 to A and F : X x Y > 1 defined by F(x,y) := f(x) + g(y). It is easy to see that if one of the conditions (i)(ix) holds, then the corresponding condition of Theorem 2.8.1 is verified. If (x) holds, using the properties of the algebraic relative interior in finite dimensional spaces (p. 3) and Proposition 1.2.7, we have *(i4(dom/)domff) = i ( A ( d o m / ) )  i ( d o m g ) = ^ ( q r i ( d o m / ) )  i ( d o m g ) , and so (viii) holds, too. The conclusion follows then applying Theorems 2.8.1, 2.3.1 (viii) and Corollary 2.4.5. • Note that, as in Theorem 2.8.1, we have that (viii) V (iii) => (i) => (ii), (iv) V (v) V (vi) =» (ii), (vii) =>• (iv) A (v) A (vi), (ii) => (i) if X is a normed space, and of course, as mentioned in the proof, (ix) =>• (viii). Applying the preceding result we obtain a formula for normal cones. Corollary 2.8.4 Let A 6 Z(X,Y) and L C X, M C Y be convex sets. Suppose that one of the following conditions is verified: (i) there exists XQ S L such that Ax0 6 int M, (ii) X,Y M),
are of Frechet spaces, L,M
are liconvex and 0 € lb(A(L) —
(iii) d i m F < o o and 0 £ 1{A(L)  M). Then for every x € LC\ A~1(M) N(Lr\Al(M);x)
= N{L;x) + A*{N(M; Ax)).
(2.61)
Formulas for conjugates and
esubdifferentials
125
Proof. Using the preceding theorem for f := IL, 9 •= IM and A, formula (2.61) follows from formula (2.60). • Corollary 2.8.5 Under the conditions of Theorem 2.8.3 we have the following relation, called the FenchelRockafellar duality formula, inf. {f(x)+g(Ax)) igA
= ma* (  / ' (  A Y ) y
g*(y*)).
€'
Furthermore, x is a minimum point for f + go A if and only if there exists y* G Y* such that A*y* £ df(x) andy* £ dg(Ax). Proof.
We proceed as in Corollary 2.8.2 (or apply this corollary).
•
Two particular cases of Theorem 2.8.3 are important in applications: / = 0 and A = Idx The next theorem is stated even for A replaced by a convex process C. T h e o r e m 2.8.6 Let g £ A(F) and C : X =4 Y be a convex process. Assume that 0 G D, where D := ImC — domg. Consider YQ := linD and the function ip : X —> R,
0 and V € Ky0 such that V C [g < A]  e(XU); (ii) there exist XQ € R, B € 3x X]  G(B);
and V0 € Ny0 such that V0 C [g <
(iii) there exists j/o € dorngfllmC such that g is continuous at yo; (iv) X, Y are metrizable, g has proper conjugate, either g is csclosed and C is cscomplete, or g is cscomplete and C verifies (Hz), and 0 £ lbD; (v) X, Y are Frechet spaces, g, C are liconvex and 0 6 lbD, (vi) dim Y0 < oo and 0 6 ' D . Then Vi'el*
: ip*{x*)=mm{g*{y*)
\ x* G C*(
Moreover, ifxE domtp = C  1 (domg) is such that
0 then de(p(x) C C* {deg(y)) (with equality i/grC is a linear sub space). Proof. Then
Let x* £ X*. Consider F : X x Y )• I , F(x,y)

= mi{F{x,y)
:= g(y) 
(x,x*).
\ (x,y) G grC}.
Convex analysis in locally convex spaces
126
If one of the conditions (i)(vi) holds then the corresponding condition of Theorem 2.7.4 holds. (In fact, in case (iv), if g is cscomplete and C satisfies (Ha;) one verifies directly that the function $ from the proof of Theorem 2.7.4 satisfies (E(x,z)).) So, by Theorem 2.7.4, there exists y* £ Y* such that
0. Let x* € dE(p(x). Since • (ii) =» (i) and (iv) V (v) => (i). Important situations when gr C is a linear subspace are: C = A"1 with A £ &(X, Y) and 6 = A with A a densely defined closed operator {i.e. A : D(A) > Y, D(A) being a dense linear subspace of X, A being a linear operator and gr A a closed subset of X x Y; see also Exercise 1.11). T h e o r e m 2.8.7 Let / , 0 and V £ N.*,, VC[f<X]n\U[g<
0, £ = £x + e 2 . Then f{x) + (y* o H){x) + (f + y*o H)* {x*) < (x, x*) + ex, g(H(x))+g*(y*)<(H(x),y*)+e2. Adding them side by side we get ip(x) + g*(y*) + (/ + y* ° H)* (x*) < {x,x*) + e. Using Eq. (2.68) we obtain that x* € deip(x). Therefore Eq. (2.69) holds. Let F(x, y) := f(x) + g(y) and G : X =t Y with gr C := epi H; then F £ A(X x Y) and 6 is convex. Since g is increasing on H(dom H) + Q, it follows that (p(x) = int{F(x,y) + iep\H{x,y) \ y 6 Y] for every x € X. Hence ip is the marginal function associated to the convex function F + iepi H ', hence ip is convex. If one of the conditions (i) — (vi) is verified, then F and 6 satisfy the corresponding condition of Theorem 2.7.4. (When / and g are cscomplete and have proper conjugates, similarly to the proof of Proposition 2.2.17, we get that F is cscomplete.) As noticed after the proof of that theorem, also the perturbed function F, F(x,y) = F(x,y) — {x,x*), satisfies the same condition. Therefore there exists z* £ Y* such that a : =
p*{x*) = min{(Ai/i + • • • + A n / n )*(x*)  ( A i , . . . , An) e A„},
+ (z,y*)  (x,y + z) £ grC}
= sup{(z, a;*)  g(y)  (y1  y, y*)  (x, y') £ gr C, y £ Y) = sup{(y,y*) g{y)
\ y £ Y}
+ sup{(a;,a;*) + (z,y*)
 igre(x,z)
\ x € X,z € Y}
= 9*(v*) + t(gr e)+ (x*, y") = g*{y") + tgr e* (y*,x*). Since V / e T
: V*(x*)
+ Lgve.(z*,x*),
(2.62)
the conclusion follows. Let x £ domy = A~1(domg) be such that
A];
S«C/J
that
Formulas for conjugates and
ssubdifferentials
127
(iii) there exists XQ G dom / n dom g such that g is continuous at XQ ; (iv) X is metrizable, f,g have proper conjugates, f is csclosed, g is cscomplete and 0 G l 6 ( d o m /  domg); (v) X is a Frechet space, f,g are liconvex and 0 G l i > (dom/ — domg); (vi) X is a Frechet space, f,g are Isc and 0 € l 6 ( d o m / — domg); (vii) X is a Frechet space, f,g are Isc and 0 G I C (dom/ — domg); (viii) dimX 0 < oo and 0 G *(dom/ — domg); (ix) g is quasicontinuous and d o m / and domg are united; (x) X is a Frechet space, f,g are liconvex and (0,0) G lb[{(x,x) X} — dom / x dom g).
\x G
Then for x* € X*, x G dom / Pi domg and e > 0 we have: ( / +
+ dE2g(x) \ei,e2
> 0, ex + e2 =e),
d(f + g)(x) = df(x) + dg(x). Proof. Taking A = Idx, the conclusion follows from Theorem 2.8.3 under conditions (i)  (iii), (v)  (ix). If (iv) holds, taking Y = X and $(x,y) := f(x) + g(x — y), condition (iv) of Theorem 2.7.1 is verified. If (x) holds consider F : X x X )• I , F{x,y) := f{x) + g{y) and A : X > X x X, A(x) :— (a;, a;). Then condition (v) of Theorem 2.8.6 holds; applying it we obtain the conclusion, taking into account that A*{x*,y*) = x* + y*. • The same implications as in Theorem 2.7.1 (mentioned in Proposition 2.7.2) hold. Corollary 2.8.8 Let X be a Frechet space and f, g G A(X). / / / and g are liconvex then for every x G * 6 (dom/ + domg) we have (/Dg)(x) = (/*+$*)•(*) = m&x{(x,x*)
 f*(x*)
 g*(x*) \ x* € X*}. (2.64)
Proof. Let a;o G s 6 (dom/ + domg) and consider h G A(X), h(x) := g(a;o — x). Then dom/i = XQ — domg, and so d o m /  dom/i = d o m / + domg — XQ Therefore 0 G l 6 ( d o m / — dom/i), and so condition (v) of
Convex analysis in locally convex spaces
128
Theorem 2.8.7 is satisfied. From formula (2.63) applied for x* = 0 we get (fn9)(x0)
= inf. (/Or) + h{x)) =  ( / + h)*(0) =  min ( / • ( * * ) + >»*(**))• X*£X *
But h*{—x*) = s u p {  (x,x*) — g(xo — x) \ x € X} = g*(x*) — (x0,x*), and so (2.64) holds for x = x0. O Of course, one can obtain the conclusion of the preceding corollary also for other situations corresponding to conditions of Theorem 2.8.7. For example, if there exist A0 £ M, B € "S>x, VQ € Nx0 (x) s u c n that Vo C [/ < A0] n B + [g < A0], where X0  aff(dom/ + dom^), then Eq. (2.64) holds. Proposition 2.8.9 Let f,g € A(X) and take D = d o m /  domg. If dim(lin£>) < oo then {D = rintZ?, while if X is metrizahle, f,g have proper conjugates, f is csclosed, g is cscomplete and tbD ^ 0 then lbD = rint D; similarly for the situations corresponding to conditions (v)(vii) of Theorem 2.8.7. Suppose that X is a Banach space and f, g are liconvex. Then for every x € lbD there exist n, A > 0 such that (x + nUx) n aff D C [f < A] n \UX  [g < A] D \UXProof.
$(x,y)
(2.65)
The first part follows from Proposition 2.7.2 taking Y = X and
=f(x)+g(xy).
Suppose that X is a Banach space and / , g are liconvex; consider x £ lb D. Replacing / by / , f(u) — f(u + x), we may suppose that x = 0. It follows that condition (v) of Theorem 2.8.7 holds, and, as noted after its proof, condition (i) is verified. Therefore there exist 77 > 0, Ao G B. and B £"Bx such that rjUx n aff D C [/ < Ao] n B  [g < A0]. Taking A' > max{Ao, 0} such that B C X'Ux and A = A' + n we obtain that rjUx n aff D C [/ < A0] 0 B  [g < A0] D (B + nUx)
c [/ < A]n xux  [f < A]n \ux. The proof is complete. Another important result is the following.
•
Formulas for conjugates and
Esubdifferentials
129
Theorem 2.8.10 Let Y be ordered by the convex cone Q, f £ A(X), H : X —> Y' be convex and g £ A(y) be Qincreasing on H(domH) + Q. Then tp :— f + goH is convex. Assume that 0 £ D, where D :— H(domHC\ d o m / ) — domg + Q, and consider Yo := linD. Assume that one of the following conditions holds: (i) for every U £ J^x there exist A > 0 and V € Ky0 such that V C H (XU n [/ < A] n d o m F )  [g < A] + Q; (ii) there exist XQ 6 R, B £ 1$x and VQ £ Ny0 such that V0 C H (B n [/ < A0] n domH) [g<
A0] + Q;
(iii) there exists XQ £ d o m / fl i f  1 (domg) such that g is continuous at H(x0); (iv) X, Y are metrizable, f, g have proper conjugates, either f, g are csclosed and epiif is cscomplete, or f,g are cscomplete and epiH is csclosed, and 0 G lbD; (v) X, Y are Frechet spaces, f,g, epiH are liconvex and 0 £ (vi) dim Y0 < oo and
tb
D;
i
0£ D;
(vii) g is quasicontinuous, and domg and H(domH united.
f~l d o m / ) + Q are
Then for x* £ X*, x £ dormp = d o m / n H~1(domg) have:
and e >Q, we
deV{x) =  J {0Er (f + y*° H){x) \y*£Q n
dE2g(H(x)),
(2.66) El
+ e2 = e} . (2.67)
Proof. have
First observe that for all x* £ X*, y* £ Q+ and x £ dom<£ we
V ' O O < (/ + V* o H)*(x*) + g*(y*), de
+
+ y*o H)(x) \y*£Q n
de2g(H(x)),
(268) El
+ e2 = e], (2.69)
Convex analysis in locally convex spaces
130
without any supplementary condition on / , g and H. Indeed, let x € dom<^ = d o m / n H^domg), x* € X* and y* £ Q+. Then f{x) + (y* o H)(x) + (f + y*o H)* (x*) > (x, x*), g(H(x))+g*(y*)>(H(x),y*), whence, adding them side by side, we get g*(y*) + (/ + y* ° H)* (x*) > (a;, a;*) 
jnf
,
=
.1TUix)+g{y){x,x''))
inf
(f(x)+g(y)(x,x*)
+
(z,z*))=:(3.
{x,y+z)EepiH
Using again the fact that g is increasing on H(domH) a =
Jnf
w
xedom H
=
inf iGdom H
j?^
n
+ Q, we have
VW + 9{y) ~ (x,x*))
yeH(x)+Q
(f(x)+g(H(x))
 {x,x*)) = ~
Formulas for conjugates and
ssubdifferentials
131
and P=
inf
inf
(f(x) + g(y)  (x, x*) + (H(x) +qy,
z")).
iGdomif q€Q, y€Y
It follows that P =  o o if z* $ Q+. If z* £ Q+ then 0=
sup x€dom
sup((x,x*)f(x)(H(x),z*)
+
(y,z*)g(y))
Hy£Y
= (/ + Z*o #)*(*•) + • K, F(x,y) := f(x)+g(H(x)+y)(x,x*). Because 0 € D and g is Qincreasing on H(domH) + Q, there exists xo € d o m / n H~1(domg). It follows that F(x0, •) is quasicontinuous and {0} and D — Pry (dom F) are united. Applying Theorem 2.7.1(ix) we have that inf x e x F(x,0) = max y . e y.(—F*(0,y*)). With a similar calculation as above we obtain that Eq. (2.66) holds. Let x 6 donup and x* 6 d£ip(x). Using Eq. (2.66), there exists y* € Q+ such that ip*(x*) = (f + y* o H)*(x*) + g*{y*). It follows that {f+y*oH){x)+{f+y*oHY{x*)(x,x*)+9{H{x))+g*{y*){H{x),y*)
< e.
Taking £ l := (/ + y* o tf)(x) + (/ + y* o #)*(x*)  (x,x*) (> 0 by the YoungFenchel inequality) and e 2 = e  ei, we have that y* € d£2g(H(x)) and Z* € d£l(f + y* oH)(x). Therefore the inclusion C holds in Eq. (2.67); taking into account Eq. (2.69), we have the desired equality. D We use the preceding theorem to obtain formulas for conjugates and subdifferentials of functions of "max" type. Corollary 2.8.11
Let / i , . . . , / „ E \(X)
y.X^W,
and
¥>(*):=/i(z)VV/„(z).
Suppose that dom ip = f}™=1 dom /i ^ 0 and e € IR+. For x € dom ip denote by I(x) the set {i G l , n  fi(x) =
Convex analysis in locally convex spaces
132
while for every x £ dom
(Ai, . . . , A„) G A„,
77 e [0,e], Yli=1Xi^x^
 ¥>(x)+ri£o}
,
dV{x) = U { 5 (E" = 1 A J i ) {x) I (Al!' • •'An) € A"' Vifl{x) Proof.
: Ai = o}.
Let us consider the functions
H:X>(W\Rl),
H(x):=( "•" 3: «"•«,
(A(*)>••"'/«(*)) [00
if x € fX=i dom/,, otherwise,
g(y):=yiV.Vyn.
It is clear that condition (iii) of the preceding theorem is verified. Taking into account the expressions of g* and deg(y) given in Corollary 2.4.17 we obtain immediately the relations from our statement. D An application of the previous result is given in the following example. Example 2.8.1
Let / € A(X) and consider /+ := / V 0. Then
f df(x) df+(x) = l U { W ) ( x )  A e [ 0 , l ] } { d(0f)(x) = didomf(x)
if f(x) > 0, if f(x) = 0, if f(x)<0.
A useful particular case of the result in Corollary 2.8.11, in which we can give explicit formulas for ip* and detp without supplementary conditions, is presented in the following corollary. Corollary 2.8.12
1J=I
Let f{ 6 A(X,) for i € T~n and Xi+W,
tp(x1,...,xn):=f1(xi)VVfn(xn).
For x = (xi,... ,xn) S domtp = Yl7=i dom/j consider I(x) := {i G l , n  fi(Xi) =
(Xu...,\n)eAn}.
(2.70)
Formulas for conjugates and esubdifferentials
For x — (xi,..., dMx)
133
xn) E dom
= {j{~[[n.=1dei(^fiKxi)\
Eni=o
( A i , . . . , A „ ) G A n , et > 0, V~^n £i = £
1 Xi Xi
' Z^i=1 ^ ^
 ^
£
 o) >
S ^ ^  l J l n ^ ^ A i / O C ^ ) ! (A1,...,An)€A„, VigJ(a:) : A; = o} . Proof. We apply the preceding corollary to the functions fa : X >• R, /i(a;) := fi(xi), then we use Theorem 2.3.1 (viii) for arbitrary n and Corollary 2.4.5. • Other situations when one has explicit formulas for tp* and d
3
+ d(»g)(x) I (A,M) G A 2 , A / ( X ) + ng(x) =
Proof. When / and g verify one of the conditions (i)(iii), (v), (viii)— (x) of Theorem 2.8.7 and A,/x > 0, then the functions A/ and fig also verify the same condition. If condition (vi) or (vii) holds then, by the relations among the classes of convex functions on page 68, condition (v) holds. So, (Xf + ng)*{x*)=Toia{(Xf)*{u*) + (jjig)*{v*) \u*+v* = x * } a n d d(\f + ng)(x) = d(Xf)(x) + d(fig)(x). Using Corollary 2.8.11 one obtains the conclusion. • Taking into account that for x G dom / , one has df(x) + N(dom f, x) = df(x) (see also Exercise 2.23), d(Xf)(x) = \df(x) for A > 0 and d{0f)(x) = N(domf,x), we get for f,g G A(X) and x G X with f(x) = g(x) G R,
U{W)W+8WWI(A,f)6A 2 } co(df(x)Udg{x))
+ N(domf,x)
+N(domg,x).
This formula can be extended easily to a finite number of functions. Using Corollary 2.8.12 we get the following result concerning the maxconvolution.
134
Convex analysis in locally convex spaces
Corollary 2.8.14 have that
Let / i , / 2 € A{X) and e £ 1+ . For every x* £ X* we
(hQf2y(x*)=rnin{(\1f1y(x*)
+ (\2f2y(x*)
 (A1;A2) £ A 2 } .
(2.71)
Suppose that (/i0/ 2 )(a:) = max{/i(2;i),/ 2 (a; 2 )}, where xi £ d o m / i , x~2 £ dom /2 and x = x~i + x2. Then
de{hOh){x) = \J{dEl(^h)(xi)ndE2(\2f2)(x2) e i , e 2 > 0 , £i+e2<e
+ X1f1(x1)
I (Ai,A2) e A2,
+ X2f2(x2)(f10f2)(x)}.
(2.72)
Furthermore, if fi(x\) = f2(x2) *^en Eq. (2.73) is verified, while if fi(xi) 72(^2) *^en i?g. (2.74) is verified, where d(fi0f2)(x)=[j{d(X1f1)(x1)nd(X2f2)(x2) d(fi0f2)(x) Proof.
I (A1;A2) G A 2 } ,
= dMxx) nN(domf2:x2).
>
(2.73) (2.74)
Let x* £ X*; then
{fi0f2)*(x*)
= s u p i e X ( ( x , x * )  i n f f / ^ ) V / 2 ( z 2 ) I *! + x2 = x}) = sup{(ii,a;*) + (a;2,a;*)  fi(xi)
V f2(x2)
\x1} x2 £ X)
= min{(A 1 / 1 )*(x*) + (A 2 / 2 )*(s*)  (Ai,A2) £ A 2 } , the last equality being obtained by using Eq. (2.70). The inclusion "D" of Eq. (2.72) can be verified easily. Consider x* £ de(fi0f2)(x). By Eq. (2.71), there exist (Ai, A2) £ A 2 such that (/i0/2)*(x*) = (Ai/i)*0O + (A 2 / 2 )*(x').
(2.75)
Using the preceding relation we obtain that 0 < [ ( A 1 / 1 ) ( ^ 1 ) + (A 1 / 1 )*( 2; *)(5; 1 ,a;*)] + [(A 2 / 2 )(i 2 ) + (A 2 / 2 )*(:c')<S 2 ) a :*)] < (fiOh)(x)
+ ( / i O / 2 ) * 0 0  (x,x*) < e.
Taking e, := (Aj/i)(zj) + {Xifi)*(x*)  (x~i,x*) > 0, i £ {1,2}, using the preceding relation and Eq. (2.75), we get ( / i 0 / 2 ) ( z ) + £ 1  Ai/i(3fi) + e 2  A 2 / 2 (x 2 ) < e, and so x* belongs to the set on the righthand side of relation (2.72).
Formulas for conjugates and
ssubdifferentials
135
Let us prove now the equalities (2.73) and (2.74). The inclusions "D" follow directly from Eq. (2.72). Let us prove the converse inclusions. Let x* G d{faOfa)(x) = d0(faOfa)(x). By Eq. (2.72), there exist (Ai, A2) G A 2 and £i, £2 > 0 such that x* £1 + e 2 < Xifi(xi)
£dei(Xifi)(x1)ndea(Xif2)(x2), + A 2 / 2 (x 2 )  (faOfa){x) < 0.
Therefore £1 = £2 = 0 and Ai/i(xi) + A 2 /2(x 2 ) = {fa(>fa){x); hence x* belongs to the set on the righthand side of Eq. (2.73) when / i ( x i ) = fa{x2)If fi{x~i) > / 2 ( ^ 2 ) , the preceding relation shows that A2 = 0 , Ai = 1, and so x* G 9/i(11),
x* G 0(0 • / 2 )(z 2 ) = <9idom/2(^2) = /V(dom/ 2 ;x 2 ).
This shows that x* belongs to the set on the righthand side of relation (2.74). • Note that in Eq. (2.72) we can take E\ + £2 = £ + •••. Moreover, if /1 is continuous at x\ and fa is continuous at a;2, then in Eq. (2.74) we have d(fiOfa)(x) = {0}. Indeed, in this situation, N(domfa;x~2) = {0} (since x~2 G int(dom/ 2 )); since /1O/2 is continuous at x, d(fiQfa)(x) ^ 0. In particular, it follows that X\ is a minimum point of fa. Furthermore, in this situation, /1O/2 is even Gateaux differentiate. Introducing the convention that 0 • df(x) := N(domf;x) for / G A(X) and x G d o m / , formula (2.75) may be written in the form d(fa0fa)(x)
=
dfa(x1)odfa(x2),
where A o B represents the harmonic sum of the sets A and B, which generalizes the inverse sum of A and B. In order to have more explicit formulas in Corollary 2.8.11 it is necessary to impose some supplementary conditions. Let us give such conditions and the corresponding formulas in three situations for n = 2. Corollary 2.8.15 Let fa, fa G A(X), e G 1+ and
E, >p := m a x { / i , / 2 } . Suppose that one of following conditions is verified: (i) there exists XQ G dorafa n d o m / 2 such that fa is continuous at XQ; (ii) X is a Frechet space, fa, fa are liconvex and 0 G l 6 (dom/i — dom/2); (iii) dim X < 00 and ^dom fa) n *(dom fa) ^ 0.
Convex analysis in locally convex spaces
136
Then, for all x* € X* and x £ dom/i D dom/2 we have: ¥>*(*•) = min{(Ai/i)*(a;I) + (A 2 / 2 )*(^)  (Ai, A2) e A 2 , x\ + x*2 = x*}, dMx)
= \J{dEl(Xifi)(x)+dE2(X2f2){x) e0+ei
I (Ai,A2) G A 2 , e 0 > ei,e 2 > 0,
+ e2 = e, Xifi(x)
dtp(x) = \J{d(X1h)(x)+d(X2f2)(x)
+ X2f2{x) >
£ T(E) and is continuous on domy D [0, a; — j/]. It is obvious that '(t) = g'(y + tu,u) for every £ £ domyj. By Theorem 2.1.5(v) we have that lim g'(y + tu,u) = lim
I (Ai,A2) G A2>
Xih(x) + x2f2{x) = tp(x)}. Let defined by
e0},
a
* £ X* and consider the functions / i , / 2 : X —>• K
/i(a;) : = m a x { ( i , a ; ; ) , . . . , ( j ; , < ) } ,
/ 2 (x) := {a;,a;*).
Applying the preceding corollary, we get the following formulas for every xeX: d
h(x)
= { ]C" = 1 A ^i I (Ai, • • •, A„) € A„, ^ = 0 if (x, x*) < /!(a;)} , r {x*} 0/2 (*) = < {Az* I A € [  1 , +1]} 1 {a;*}
2.9
if if if
(x,x*) > 0 , (x, x*) = 0, (x,x*) < 0 .
Convex Optimization with Constraints
Let us come back to the general problem of convex programming as considered in Section 2.5, (P)
min f(x),
x £ C,
where / € A(X), C C X is a convex set and C (~l d o m / ^ 0; the spaces considered in the present section are separated locally convex spaces if not stated explicitly otherwise. Since x is solution of (P) exactly when it minimizes f + ic, x is solution of (P) if and only if 0 € d(f + i
Convex optimization
Proof.
with
137
constraints
In the conditions of our statement we have that
VxeCndom/
: d(f + ic){x) = df(x) + duc{x) = df(x) +
N{C;x),
whence the conclusion is obvious.
•
Very often the set C from problem (P) is introduced as the set of solutions of a system of equalities and/or inequalities. Let Y be ordered by a closed convex cone Q CY and H : X » Y' be a Qconvex operator; the set C := {x € X \ H(x)
min f{x), H{x)
An element x € X for which H(x)
min f(x),
H(x)
y.
Let us consider the function $ : I
X
F 4 l ,
/(l) *(*,y):=«{ (x, y) :=  v wy ' ' oo
i
f
^ ^ ^ otherwise.
(2.76) v '
The problem (Py) becomes now (Py)
min $(x,y),
i £ l
We obtain, without difficulty, that $ is convex. Moreover $*(z*,y*) = sup{(x,a;*) + (y, y*)  *(x,y)
\x€X,yeY}
= sup{(x,x*)  (y,y*>  / ( z )  x € X,y £ Y,#(aO < Q y} = sup{(a;,a;*) (H(x)+q,y*) = sup{(z,:r*)  (H(x),y*)
 f(x) \ x € dom//, g € Q}  f{x) \ x £
domH}
+ sup{(g,y*)  9 € Q}. Hence $*(x*,y*)
= sup{(x,x*)
if y* e Q+ and $*{x*,y*)
 f(x)  (H(x),y*)
= oo if y* g Q+.
\x£domH}
138
Convex analysis in locally convex spaces
The function L:XxQ+>R,
L(x,y*):=
{ ^x ^ oo
) + (H(x),y*)
ifxGdomtf, if x $. dom H,
is called the Lagrange function (or Lagrangian) associated to problem (Po) The above definition of L shows that L(x,y*) — f(x) + (y* o H){x) with the convention that y*(oo) = oo for y* G Q+, convention which is used in the sequel. By what was shown above, we have that = 8up(L(x,yt)) = hdL(x,y*). x x xex ^ Therefore, the dual problem of problem (Po) (see Section 2.6) is V y * e Q + : &(0,y*)
(£>o)
max (  $*(0,
y*£Y*,
or equivalently, (Do)
max (mfxeX L(x,y*)),
y* £ Q+•
We have the following result. Theorem 2.9.2 Let f G A(X), x G domf and H : X > (Y',Q) be a Qconvex operator, where Q
3x0edomf
: H{x0)
€ intQ.
Then the problem (D0) has optimal solutions andv(Po) = v(Do), i.e. there exists y* G Q+ such that inf{/(x)  H(x)
\ x G X}.
Furthermore, the following statements are equivalent: (i) x is a solution of (Po); (ii) H(x)
+ y*o H)(x)
and
(H(x),y*)
= 0;
(hi) there exists y* G Q+ such that (x, y*) is a saddle point for L, i.e. Vx E X, Vy* G Q+ : L(x,y*) < L{x,y*) < L{x,y*)
(2.77)
Proof. The Slater condition (S) ensures that (XQ, 0) G dom $ and $(x0, •) is continuous at 0, where $ is the function denned by Eq. (2.76). Applying Theorem 2.7.1, there exists y* G Y* such that v(P0) =  $ * ( 0 ,  y * ) . If v(Po) = —oo, then $*(0,2/*) = oo for every y* G Y*; in this case we may
Convex optimization with constraints
139
take y* = 0. If f(Po) > —oo, using the expression of $*, we have that y* e Q+. Therefore v(P0) = inf{/(x)  H(x) < Q 0} =  $ * ( 0 ,   T ) = i n f { i ( x , r ) I a; € X } = v(D0). (i) =S (ii) Of course, x being a solution for (P 0 ), we have that H(x)
< f(x) < f(x) + (H(x),y*)
(2.78)
Relation (2.78), without the term from its middle, says that x is a minimum point for f + y* o H, and so 0 € d(f + y* o H)(x); taking x = x in relation (2.78) we obtain that (H{x),y*)  0. (ii) =>• (hi) Since 0 € d(f + y* o H)(x) we have Vx G X : L ( x , r ) = /(x) + (^(x),y*) < f{x) + (H(x),y*)
= LfoiT).
Furthermore, for y* £ Q+ we have that L(x,y*) = f{x) + (H(x),y*)
< f(x) = f{x) + (H{x),y*)
=
L(x,?).
Therefore Eq. (2.77) holds, i.e. (x,y*) is a saddle point of L. (iii) => (i) Taking successively y* = 0 and y* = 2y* on the lefthand side of Eq. (2.77) we obtain that (Hix),y*) = 0. Using again the lefthand side of Eq. (2.77), we obtain that (i?(x), y*) < 0 for every y* e Q+; thus, using the bipolar theorem (Theorem 1.1.9), we have that —i?(x) € Q++ = Q, i.e. x is an admissible solution of (P 0 ). From the righthand side of relation (2.77) we get Vx e X, H{x)
< f(x) + (H{x),F)
Therefore x is solution of problem (Po)
<
fix). D
The element y* € Q+ obtained in Theorem 2.9.2 is called a Lagrange multiplier of problem (P). Note that when H is finitevalued (i.e. d o m # = X) and continuous, V x e d o m / : dif +
y*oH)ix)=8fix)+diy*oH)ix);
Convex analysis in locally convex spaces
140
the fact that Q is closed was used only for the implication (iii) =4 (i) of the above theorem, to prove that x is an admissible solution. An important particular case is when there are a finite number of constraints. Let / , g i , . . . ,gn 6 A(X) and consider the problem (Pi)
min f(x),
gi{x) < 0,1 < i < n.
The dual problem of (Pi) is (Di)
max mix€X(f(x)
+ \igi(x)]
hA n (/ n (i)), Ai > 0 , . . . , A„ > 0.
Then the following result holds. Theorem 2.9.3 dition holds, i.e.
Let f,gi,...,gn
€ A(X). Suppose that the Slater con
3xo € d o m / , \/i£l,n
: gi(x0) < 0.
Then: (i) the dual problem (Di) has optimal solutions and u(Pi) = i.e. there exist (Lagrange multipliers) A i , . . . , A„ £ M+ such that
v(D{),
mf{f(x)\gi(x)<0,...,gn(x)<0} ini
{f(x)+J1g1(x)\
+ Xn9n(x) \ x 6 X) .
(ii) Let x 6 dom / ; x is a solution of problem (Pi) if and only if gi(x) < 0 for every i 6 l , n and there exist A i , . . . , An € K+ such that Xigi(x) = 0 for i £ l , n and 0 € d (/ + Aiffi + . . . +
\XJ
*ngn)
If the functions g\,..., alent to
gn are continuous at x, the last condition is equiv
0 € df(x) + A!0ffi(2c) + . . . +
\ndgn(x).
Proof. Let us consider H : X > (Mn*,E!J:), H(x) := (gi(x),.. .,gn(x)) for x G n r = i d o m 5 j , i?(x) = oo otherwise; H is R™convex. The result stated in the theorem is an immediate consequence of the preceding theorem. When gi are continuous at x one has, by Theorem 2.8.7 (iii), that d(f + Aiffi + . . . + \n9n)(x) and d(\igi)(x)
= Jidg^x).
= df(x) + d(Xigi)(x)
+ ... +
d(\ngn)(x), D
Convex optimization with constraints
141
Corollary 2.9.4 Let f,gi,.,gn '• X —> R be Gateaux differentiable continuous convex functions. Suppose that there exists x$ £ X such that 9i{xo) < 0 for every i 6 l,n. Then x £ X is solution of problem (Pi) if and only if gi(x) < 0 for every i £ l , n and there exist A i , . . . , An £ R+ such that —\7f(x)=\iVgi(x) Corollary 2.9.5 N([g < l\,x)
+ ... + \nVgn(x)
and \igi(x) = 0 for i £ l,n.
•
Let g 6 A(X), 7 £ ] inf g, 00 [ and S £ [5 < 7]. Tften = \J {d(Xg)(x)  A > 0, X(g(x)  7) = 0}.
(2.79)
Proof. The inclusion "D" in Eq. (2.79) holds without any condition on g and 7. Indeed, let x* £ d(Xg)(x) for some A > 0 with X(g(x)— 7) = 0. Then for x £ [g < 7] we have that (a: — x, x*) < Xg(x) — Xgix) = X(g(x) — 7) < 0, and so x* £ N([g < ^],x). Let x* £ N([g < j];x). Then x is a solution of the problem (Pi)
min {x, —x*), h(x) := g(x) — 7 < 0,
whence, by Theorem 2.9.3, there exists A > 0 such that Xh(x) = X(g(x) — 7) = 0 and 0 G d(x* + Xh){x), i.e. x* £ d{Xh){x) = d(Xg)(x). Therefore the inclusion " c " of Eq. (2.79) holds, too. • Note that d(Xg)(x) = Xdg(x) if A > 0 and d(0g)(x) = didomg(x) = N(domg;x). In the case of normed vector spaces, taking A = X in Proposition 3.10.16 from Section 3.10, we have another sufficient condition for the validity of formula (2.79). Note that we can obtain the characterization of optimal solutions in Theorem 2.9.3, when the functions gi are continuous, using Corollary 2.9.5 and the formula for the subdifferential of a sum. Indeed, x £ dom / is a solution of (Pi) if and only if x is minimum point of the function / + icx + htc„, where Cj := {x  gi(x) < 0}. By hypothesis d o m / f l f)" = 1 int d ^ 0, and so, for every x £ d o m / (~l H i l i ^ i w e have d(f + tCl+...
+ icn) (x) = df{x) + N(Cf,x)
+ ... +
N(Cn;x).
Using formula (2.79) we obtain the desired characterization. Theorem 2.9.2 can be extended further to the case when there are also linear constraints. Let us consider the problem (P 2 )
min f(x),
H{x)
Convex analysis in locally convex spaces
142
where T G £ ( X , Z). Consider the Lagrange function associated to problem
L2 : X x (Q+ x Z*)  • 1 ,
L 2 ( a M , V ) := /(a:) + (H(x),y*)
+
(Tx,z*).
The dual problem associated to (P 2 ) is (£>2)
max (mixeXL2(x,y*,z*)),
y* G Q+, z* G Z*.
Then the following result holds. Theorem 2.9.6 Let f G A{X), H : X > (Y,Q) be a Qconvex operator, where Q C Y is a closed convex cone, T G L(X,Z) be a relatively open operator and x G d o m / . Suppose that there exists XQ € d o m / such that f,H are continuous at xo, —H{XQ) G int<5 and TXQ = 0. Then the problem (D2) has optimal solutions and V{PQ,) = v{D2), i.e. there exists y* G Q+, z* G Z* such that inf{/(x) I H(x)
\ x G X).
Furthermore, the following statements are equivalent: (i) x is solution of problem (P2); (ii) H(x)
and (H(x),y*)
= 0;
(iii) there exists (y*,z*) G Q+ x Z* such that (x,(y*,z*)) point of L2, i.e. L2(x,y*,z*)
< L2{x,y*X)
<
is a saddle
L2{x,y*,T)
for all x G X and all y* G Q+, z* G Z*. Proof.
Let us consider the perturbation function
*:Ix(7xZ)>I, v
;
'
f(x) $(1,y,z) := , y y ^ ' ' ' 00
if H(x)
We intend to prove that condition (i) of Theorem 2.7.1 is verified. Note first that P r y x Z ( d o m $ ) = {{H{x) + q,Tx)  x G d o m / , q £ Q} C Y x ImT.
A minimax
theorem
143
Taking into account that / is continuous at xo G d o m / , there exists Uo G N x ( i o ) such that Vxe%
: f{x)<M:=f(x0)
+ l.
Since —H{XQ) G intQ, there exists Vo G Ny such that —H(XQ) +VQCQ. There exists V € Ny such that V + V CV0. Since if is continuous at x0, there exists {/ € Nx(zo)) [/ C t/o, such that H(x) € H(x0) — V for every x € U. Since T is relatively open, there exists W G NimT such that W C T(LT). Of course, V x W G N y x i m r ( 0 , 0 ) ; let (j/,z) G V x W. There exists x £U such that Tx = z. Since x G f/, we have that i ? ( i ) = H(xo) — y' for some j / ' G V. Then y  H(x) =y + y'
H(x0) G  # ( x 0 ) + V + V C  # ( x 0 ) + V0 C Q;
hence ff(x) < Q y. Since x £ U C Uo, we have that $(x,2/,z) < M, i.e. condition (i) of Theorem 2.7.1 is verified. Therefore there exists (y*,z*) G Y* x Z* such that v(P2) = 4>*(0, y*, z*). But $*(0, y*, z") = s u p ( w ) 6 X x y x Z ( ( i / , y*) + (z, z*)  $ ( x , y , z)) = sup{(y,y*)
 (z,z*)  f(x)  H ( i )
= s u p {  ( # (x) + q, y*)  (Tx, z*)  f(x) =  inf{f(x) + (H(x),y*)
\xeX,qeQ}
+ (Tx, z") \ x G X } + sup{(q,j/*)  q G Q}.
Thus **cn _„* —?*\  J  i n f i e x ^ C i . y * , ^ * ) * l U ' y ' z ) ~ \ oo
if if
2/* e Q + , j / * £ Q+.
Therefore y* G Q + if u(P 2 ) G K and we can take j7* = 0 if v(P2) =  o o . The proof of the second part is completely similar to that of Theorem 2.9.2. We note only that, / and H being continuous at xo, we have d(f + y* o H)(x) = df(x) + d(y* o H)(x) for every x G d o m / . •
2.10
A Minimax Theorem
In the preceding section we have obtained some results which are stated using saddle points. In the sequel we establish a quite general result on the
144
Convex analysis in locally convex spaces
existence of saddle points as well as a minimax theorem. Let A and B be two nonempty sets and / : A x B > JR. It is obvious that sup inf f(x,y) xeAvtB
< inf sup f(x,y). y£BxeA
(2.80)
The results which ensure equality in the preceding inequality are called "minimax theorems," the common value being called saddle value. Note that if / has a saddle point, i.e. there exists (x,y) € A x B such that VxeA,Vy&B
: f(x,y)
< f(x,y)
< f(x,y),
(2.81)
then max inf f(x,y) x£A yeB
= minsup f{x,y), yeB
(2.82)
xeA
where max (min) means, as usual, an attained supremum (infimum). Indeed, let (x,y) € Ax B satisfy Eq. (2.81). It follows that ini: sup f(x,y) v£Bx€A
< sup f(x,y) xeA
< f(x,y)
< infB :(x,y) < sup infB y£ xeAv&
f{x,y).
Using Eq. (2.80) we obtain that all the terms are equal in the preceding relation, and so (2.82) holds (the maximum being attained at x and the minimum at y). Conversely, if Eq. (2.82) holds then / has saddle points. Indeed, let x G A and y e B be such that inf (x,y) = sup ini' f(x,y) y£B
xeAVZB
= irtf swp f(x,y) y€Bx€A
= sup
f(x,y).
x£A
Since inf y 6 s(x,y) < f(x,y) < supx€Af(x,y), from the above relation it follows that all the terms are equal in these inequalities; this shows that (x, y) is a saddle point of / . So we have obtained the following result. Theorem 2.10.1 Let A and B be two nonempty sets and f : A x B >• R. Then f has saddle points if and only if condition (2.82) holds. • The following result is an enough general minimax theorem (frequently utilized) which gives also a sufficient condition for the existence of saddle points. Theorem 2.10.2 Let X be a locally convex space, Y be a linear space, Ac X be a nonempty convex compact set and B CY be a nonempty convex set. Let also f : A x B )• M. be a function with the property that f(,y)
A minimax
theorem
145
is concave and upper semicontinuous for every y G B and f(x, •) is convex for every x E A. Then max inf f(x,y)
— inf ma,xf(x,y).
X&A y£B
y£B
(2.83)
xEA
If moreover Y is a locally convex space, B is compact and f(x, •) is lower semicontinuous for every x € A, then maxmmf(x,y)
= min max f(x,y);
xeA y£B
yeB
(2.84)
x£A
in particular f has saddle points. Proof. Let a e K be such that a > max x£j 4 inf ye B f(x,y). For every x S A there exists yx 6 B such that f(x,yx) < a. Since f(,yx) is upper semicontinuous at x, there exists an open neighborhood Vx of x such that f(u,yx) < a for every u G Vx. Since A is compact and A C UxeA ^ > there exist x\,..., xp € A such that A C U?=i Vxt • Let 2/j := yXi • Consider the sets d
:=co{{f(x,yi),...,f(x,yp))\xeA}cW,
C2 := {(ui,...,up)
\v,i>a\/i€
l,p} .
It is obvious that C\ and Ci are nonempty convex sets, and C2 has nonempty interior. Moreover C\ fl Ci = 0. In the contrary case there exist q 6 N, ( A i , . . . , A?) 6 Aq and Xi,..., xq € A such that V i e l . p : a < Zf := 2^,
>^jf{xj,yi).
Since f(,yi) is concave, taking xo = 13?=i ^ j 2 ^ ' Vi G l , p : a < Zj <
we nave
that i o £ i and
f(x0,yi).
There exists io £ l , p such that £0 £ VXi . Therefore f{xo,yi0) < a, a contradiction. Applying Theorem 1.1.3 there exists (/zi,..., fj,p) € W \ {0} such that . ,mf(x,yi) t=l
< V
UiUi.
*—'i=l
Letting u; > 00 we have that /ij > 0 for every i, and so we can suppose that (/xi,..., Up) € A p . Taking u = ( a , . . . , a) we obtain that ^xeA
: > .
Uif{x,yi)
< a.
Convex analysis in locally convex spaces
146
Let 2/0 := Y%=i ViVi € B. Since f(x, •) is convex, we obtain t h a t f(x, y0) < a for every x € A. Therefore a > i n f j / e B m a x x g ^ / ( a ; , 2 / ) , and so (2.83) holds. In Eq. (2.83) the suprema with respect t o x are attained because the functions f{,y), y £ B, and i n f y e s f(,y) are upper semicontinuous and A is compact. W h e n B is compact and the functions f(x,) are lsc for all x € A we obtain t h a t the infima with respect to y € B are attained in Eq. (2.83), i.e. Eq. (2.84) holds. • Note t h a t the convexity assumptions in the preceding theorem can be weakened. More precisely, we can suppose t h a t A is a nonempty compact subset of a topological space, B is nonempty, and / satisfies t h e following two conditions (similar to concavity and convexity, respectively):
Vxi,...,xp
e A, V ( A i , . . . , A p ) e A p , 3x0
&A,\fyeB
:
f(xo,y) > y \
,Kf(zi,y),
• ' — ' 1 = 1
\/yi,...,yq
GB,
V ( / / ! , . . . , / * , ) € A , , 3y0 eB,Vx£A
: v—yQ
f(x,yo) 2.11
<
2^,.=iHjf{x,yj).
Exercises
Exercise 2.1 Let X be a linear space, / € A.(X) and x,u £ X. Prove that the mapping %p :]0, oo[—• R defined by ip(t) :— t • f(x + t~lu) is convex. E x e r c i s e 2.2 (a) Let I C R be an interval and / : / — > • R. Suppose that / is locally nondecreasing, i.e. for every a € I there exists e > 0 such that the restriction of / to IC\[a — s, a + e] is nondecreasing. Prove that / is nondecreasing. (b) Let g : [a, b] —• R (a, 6 € R, a < b) be a continuous function. 1) Assume that g'+{x) g R exists for every x € [a,b[; show that there exists x' £ [a,b[ such that g(b) — g(a) < g'+(x')(b — a). 2) Assume that g'(x) € R exists for every x S ]a, 6]; show that there exists x" € ]a, b] such that g(b) — g(a) > gL (x")(b — a). (c) Let X be a locally convex space and A C X be an open convex set. If / : A —> R is locally convex, i.e. for every a G A there exists a neighborhood V of a such that f\vnA is convex, prove that / is convex. Exercise 2.3
Let / : R n *• R be a quasiconvex function and i ^ f .
For
Exercises x eRn
147
consider the function fx : R n >• R, fx(t) = f(x + tx). Prove that / is lsc at i t t V i e l "
: fx
is lsc at 0,
/ is use at J o V i e R "
: fx, is use at 0.
If R n is replaced by an infinite dimensional normed space then the above properties do not hold, generally. Exercise 2.4 Let X be a linear space, / : X —> R be a convex function and A > i n f i e x fix) Prove that {x G X  f(x) < \y
= {x G X  f{x) < A}.
Moreover, if X is a topological vector space and / is continuous, then int{x € X  f(x) < A} = {x G X  / ( z ) < A}. Exercise 2.5 Let X be a topological vector space and / : X —> R a convex function. Prove that: ( a ) [ / < * ] = c l [ / < *] f° r every t G ] inf / , oo[ if and only if / is lsc; (b) [ / < * ] = i n t [ / < *] for every t G ] inf / , cx)[ if and only if / is continuous on d o m / ; ( c ) [/ = *] = b d [ / < t] for every t G ] inf / , oo[ if and only if / is continuous (on X ) . Exercise 2.6 Let / : R" —)• R be a proper convex function for which all the partial derivatives df/dxi exists at a G int(dom/). Prove that / is Gateaux differentiable at a (even Frechet differentiate because / is Lipschitz on a neighborhood of a). Exercise 1 + t\p + is strictly and (f2 is
2.7 Let p G [1, oo[ and
E x e r c i s e 2.8
Take /3 G [1, <x[ and consider the function
/:R2^I,
2
f(x,y):={ (^tan)^V^T^ [ oo
for x, y > 0, otherwise,
where, by convention, arctan  := ^ for a; > 0. Prove that / is a lsc convex function, but not strictly convex. Exercise 2.9
Consider the function
/: c[o, i] > R,
fix) := /o1 y r r w o F ^ 
Convex analysis in locally convex spaces
148
Prove that / is convex and Frechet differentiable of order 2 on C[0,1] with
V/(x)(«) = f1 £L= dt, Jo V l + x
V2f(X)(u, V) = ^
2
,7/^32 dt 2
Jo ( l + 2 : ) v l + z 2
for all u,v £ C[0,1]. Moreover, V / is continuous on C[0,1]. E x e r c i s e 2.10
Consider the function /:L1(0,1)>R,
/ ( i ) : = / 0 V l + (i(t))!*,
where L 1 (0,1) is the Banach space of (classes of) Lebesgue integrable functions on the interval [0,1]. Prove that / is a Gateaux differentiable convex function with xu Vx.uSL^O.l) : V/(x)(u) = / :dt, Jo
%/rn
but / is nowhere Frechet differentiable. Exercise 2.11
Using properties of convex functions, prove that
V a , 6 e R + , V p , g e ] l , o o [ , £ + ^ = 1 : ab < ±a p + \b\ the equality being valid if and only if ap = bq, and V n € N, V x i , . . . , x „ 6 R + , V a i , . . . , a „ 6]0,1[, a i + . . . + a „ = 1 : a m + . . . + a n x „ > X™1 • • xZn, the equality being valid if and only if x\ = • • • = xn. In particular, VneN, Vsi,...,x„ €R+
: £(xi + . . . + xn) >
yX\xn.
E x e r c i s e 2.12 Let / : X —>• R be a proper function. Prove that / is convex if and only if / + x* is quasiconvex for every x* G X*. Exercise 2.13 Let <x\,... ,an > 0 (n > 1) be such that ai\ \an < 1. Prove that the function / : P " —> R, / ( x ) := x"1 • x%2 • • • x"n, is concave; moreover, if a\ + • • • + a„ < 1, then / is strictly concave. E x e r c i s e 2.14 Consider / : R n >• R, f(xi,...,x„) Prove that / is convex. Exercise 2.15
:= In ( X £ = 1
expxk).
Consider p € R \ {0} and the function xP">R,
f{t,x):=
t"
nr=i*i'
Prove that / is strictly convex for p € ] — o o , 0 [ U ] n + l , oo[ and convex if p = n + 1.
149
Exercises Let c G R n and A : = { i £ P " \(x\c)> g : A > R,
0}. Prove that the function
p(x)
(x\c)"
niu^
is strictly convex for p > n + 1 (it is possible to prove that g is strictly convex for p > n). The function g has been used by Karmarkar for establishing his interior point algorithm. Finally prove that the function
h : P n >• R,
h(x) := (T[n xA
is strictly convex. Exercise 2.16 Let tp : R+ —> R+ be a continuous convex function such that ^i(t) = 0 •«• t = 0. Show that
Jo
i>' (V> x «))
=
7o
VH^H*))
for every a > 0. Exercise 2.17 function
Let X, Y be normed spaces and T G L(X,Y). f:Y^R,
f(y):=M{\\x\\\Tx
Prove that the
= y},
is a sublinear functional. Moreover, if T is an open operator, then dom f = X and / is continuous. Exercise 2.18 Let / : Rk —> R be a strongly coercive proper lsc function. Prove that co(epi/) is a closed set. E x e r c i s e 2.19 Let X be a locally convex space, / G T(X), g : X —> R be a proper function and a, /3 > 0. Prove that co (co(/ + ag) + jig) = co ( / + ( a 4 /3)g) and ( ( / + ag)** + fig)" = (f + (a + fig)* . Exercise 2.20 Let X be a separated locally convex space and A,B,CcXbe nonempty sets. If C is bounded and A + C C B + C prove that A C coB. Exercise 2.21 Let (X, \\\\) be a normed space, / G T(X) and L > 0. Prove the equivalence of the following statements: (i) d o m / = X a n d V x , t / G X (ii) 3 a GR, V i € X (iii)
VMGX
: \f(x)  f(y)\ < L \\x  y;
: / ( x ) < L x + a;
: /oo(u)
Exercise 2.22 Let X be a locally convex space and / € T(X). foo(u) = sup{/(z + u) — f(x) I a; G d o m / } for every u 6 l .
Prove that
150
Convex analysis in locally convex spaces
E x e r c i s e 2.23 Let X, Y be separated locally convex spaces, / € r ( X ) , F € F(X x Y), x € dom / and A £ R, e £ R+ be such that [/ < A] and d£f(x) are nonempty. Prove that: [/ < A]oo = [foo < 0], ( a e / ( x ) ) 0 0 = N{domf;x), (/*)«, = Sdom/, foe = Sdom/* and {v* £ y*  (F*)oo(0,t;*) < 0} = ( P r y ( d o m F ) ) . In particular, if {v* £ Y*  (F*) o o (0,u*) < 0} is a linear subspace then {0} and Pry (dom F) are united. E x e r c i s e 2.24 Let X be a separated locally convex space and / £ I \ X ) . Consider K := {u £ X  /«,(w) < 0} and X 0 := K D (.K"). Prove that: (i) if is a closed convex cone, Xo is a closed linear subspace and f(x + u) = f(x) for all a: 6 X and u 6 Xo. (ii)
lin(dom/*)=X6L.
(iii) The function / : X / X 0 »• I , f(x) := f(x), is well defined, fe r ( X / X 0 ) , /OO(M) = /oo(«) for every u € X, f* = f*\x± and 5 £ / ( £ ) = 9 E /(a;) X j. for all x € X and £ 6 R+, where x represents the class of x £ X. E x e r c i s e 2.25 Let X be a separated locally convex space and / £ T(X). Assume that there exists u £ X such that /<X>(M) < 0 < fco(u). Prove that for every e > 0 there exists fe £ T(X) such that f(x) < fc(x) < f(x) + s for every x e X and argmin/ e = 0. E x e r c i s e 2.26 Let X, Y be separated locally convex spaces and F 6 A(X x Y). Assume that F satisfies one of the conditions (ii)(viii) of Theorem 2.7.1. Prove that the marginal function g : X* —¥ R, g(x") := inf y * e y* F*(x*,y*) is convex, iu*lsc, the infimum is attained for every x* £ X* and goo(u") = mmv*ey(F*)oo(u* ,v*). Moreover, {v* £ Y* \ (F*)oo(0, «*) < 0} is a linear subspace. E x e r c i s e 2.27 (TolandSinger duality formula) Let X be a separated locally convex space, / : X —>• R be a proper function and g £ T(X). Then inf (/(*)  ff(i)) = .inf. (g\x')
 f*{x'))
•
E x e r c i s e 2.28 Let X be a separated locally convex space, / £ T(X) be such that 0 € d o m / a n d p £ P. Consider the following assertions: (i) the mapping P 3 t i  > t~pf(tx) (ii) f'(x, —x) +pf(x) (iii) pf(x) (iv)
is nondecreasing for every x £ d o m / ;
< 0 for every x £ d o m / ;
< f'(x,x)
for every x € d o m / ;
{x,x*) >pf(x)
for all (x,x*) € g r d / ;
(v) for every x £ int(dom / ) there exists x* 6 df(x) such that (x, s*) > pf(x). Prove that (i) <=> (ii) <=> (iii) =$ (iv). Moreover, if / is continuous at 0 then all five assertions above are equivalent.
Exercises
151
E x e r c i s e 2.29 Let / : X —> IR be a function such that c o / is proper. Assume that for some x € d o m c o / we have that x = ^2i=i ^»^« a n ( ^ co/(ic) = ^ * = 1 ~\if{x~i) with Xi G d o m / , Ai > 0 for i € ITfc and £)* = 1 ^> = 1 Prove that 8c5/(3f) = n t i df(xi). Exercise 2.30 Let X be a separated locally convex space and / € A(X). Assume that 0 G d o m / and /  x 0 is continuous at 0, where Xo •— aff(dom/). Prove that f = sup{(x,x*) x* G<9/(0)W < { =
f'(0,x) f'(Q,x) = oo / ' ( 0 , x ) = oo
if if if
xGXo, xGXo\_Xo, xeX\X0,
the supremum being attained for x G Xo. Moreover, Xo is closed if and only if V x G X : / ' ( 0 , x ) = s u p { ( x , x * )  x* 6 3/(0)} . E x e r c i s e 2.31 Let X be a separated locally convex space and / G A(X) be continuous on i n t ( d o m / ) , supposed to be nonempty. Prove that for all x,y € int(dom/) there exist z £]x,y[and z* G df{z) such that f(y)—f{x) = (y — x, z*). E x e r c i s e 2.32
Let X be a separated locally convex space and / G T(X).
(i) Consider the conditions: a) df is single valued on dom.3/, b) (df)~1(xl)!~) (df)~1(x2) = 0 for all distinct elements Xi,X2 G X*, c) /* is strictly convex on every segment [xi,X2] C Imdf, d) dom df = int(dom/). Prove that a) <=> b) •<=>• c); moreover, if int(dom/) ^ 0 and / is continuous on int(dom/) then a) => d). (ii) If / is continuous on int(dom/) and ( 9 / ) _ 1 is singlevalued on show that / is strictly convex on int(dom/).
Imdf,
E x e r c i s e 2.33 Let X, Y be separated locally convex spaces and / G A(X x Y). Prove that if / is continuous at (xo,j/o) £ d o m / , then Prx* (df{x0, j/o)) = df(,yo)(x0) Exercise 2.34 Consider
and
P r y ( d / ( x 0 , y 0 )) = d / ( x 0 , ){yo)
Let X be a separated locally convex space and / o , / i 6 A(X).
v := inf{/ 0 (x)  / i ( x ) < 0},
v* := sup inf (/o(x) + A/i(x)), x>o xex
with the convention 0 • oo = oo. Suppose that » £ R . Prove that v* =v&[rfe>Q
: inf{/i(x)  / 0 (x) < y  e} > 0].
E x e r c i s e 2.35 Let {X, \\\\) be a normed linear space and / : X —> R be a proper convex function. Prove that there exists x* G X* with x* < / if and only if there exists M > 0 such that f(x) + M \\x\\ > 0 for all x G X.
Convex analysis in locally convex spaces
152
Exercise 2.36 Let (X, ) be a normed linear space and / i , / 2 : X > R be proper convex functions. Prove that there exist x* 6 X* and a € R such that —/i < x" + a < J2 if and only if there exists M > 0 such that fi(xi) + f2(%2) + M\\xi  x21 > 0 for all on, x2 € X. E x e r c i s e 2.37 Let X be a linear space, (Y, ) be a normed linear space, T : X * Y be a linear operator, yo 6 Y and / : I > R be a proper convex function. Prove that f(x) + \\Tx + yo\\2 > 0 for all x € X if and only if there exists j / * € y * such that fix) 2(Tx + y0,y*)  y* 2 > 0 for all x e X. E x e r c i s e 2.38 Let (X, ) be a normed vector space, C,D C X be closed convex cones and x G X, x* € X*. Prove that d(x,C)
: = i n f {   x  x    x € C} = m a x {  ( x , x * )  x* 6 Ux* n C + } ,
d ( x * , C + ) = m i n {   x *  x *    a;* 6 C + } = s u p {  ( x , x * )  x G C/x n C} , a,ndsupx€Uxncd(x,D)
= supx,eUx,nD+
d(x* ,C+).
E x e r c i s e 2.39 Let / € A(X) and x € d o m / be such that f(x) Consider the sets d
>
inf/.
:= {z* € X*  ( i  x,x*) < f(x)  /(a?) Vx e [/ < /(*)]},
d ^ / ( x ) := {x* a ' l f i  f , x*) < f{x)  / ( J ) Vx € [/ < /(*)]}. Prove that dKf{x) = df(x) differential of / at x.
= [1, oo[d/(x). The set a < / ( x ) is Plastria's sub
E x e r c i s e 2.40 Let X be a normed space, (an)n>i C X and (A n )„>i C [0,oo[ be such that ^ ^ L j A n = 1. Consider the function
/:X>R,
f{x) = Y°°
A„xa„ 2 .
*•—'n=l
1) Prove that the following statements are equivalent: (a) S^Li^il a "ll 2 < oo (i.e. 0 € d o m / ) , (b) d o m / ^ 0, (c) d o m / = X. 2) Prove that / is finite, convex and continuous when 5Z^Li^ill a "l 2 < °°3) Suppose that ]C!!!Li^nlan2 < °° Prove that for every x £ X one has 8f{x) = { 2 V 0 0
A n x  On xmn I x*n e a • IKi  a„) Vn € N ) .
E x e r c i s e 2.41 Let X be a normed space and / 6 T(X). following statements are equivalent:
Prove that the
(a) limJ._too f{x) = 00; (b)
[/ < A] is a bounded set for every A > inf^ex / ( x ) ;
(c) there exists Ao > inf^gx / ( x ) such that [/ < Ao] is bounded;
Exercises
153
(d) there exists a, 0 £ R, a > 0, such that f(x) > ax + /? for every x G X; (e) liminf )x _ foo / ( x ) /   x   > 0; (f)
0£int(dom/*).
Moreover, if dim X < oo then the conditions above are equivalent to ( c ')
[/ < iaf /] is nonempty and bounded.
Furthermore, if p, q £ ]l, oo[ are such that 1/p + 1/q = 1, the following statements are equivalent: (g)
liminfNHoo/(x)/xp>0;
(h) l i m s u p   : c ,  H o o r(x")/\\x*\\q
< oo;
(i) there exists a, /3 £ R, a > 0, such that f(x) > ax p + P for every i 6 X ; (j) there exists a,/3 £ R, a > 0, such that /*(x*) < cxa;*9 + /? for every a;* e X * . E x e r c i s e 2.42 Let f,fn : R m >• R (n £ N) be convex functions such that (fn{x)) ¥ f{x) for every x £ R m . Assume that / is coercive. Prove that there exist a,P £ R with a > 0, no £ N such that / n ( x ) > a \\x\\ + /3 for all x G R m and n > no. Exercise 2.43 Let (X, ) be a n.v.s. and C C X be a nonempty closed convex set. Consider the following assertions: (i) there exists XQ G X* such that (x,x*,) > \\x\\ for every x G C; (ii) there exist xo £ X", X*, £ X* such that (x — xo, x*>) > x — xo for every x eC; (iii) int(dom sc) 7^ 0; (iv) a) there exists x*, G X* such that {u, x*>) > 0 for every u £ Coo \ {0} and b) for every sequence (x n ) C C with (x n ) —• oo and (   x „   _ 1 xn) ^ u one has Prove that (i) => (ii) =S> (iii) => (iv). If 0 £ C then (ii) =>• (i). Moreover, if X is a reflexive Banach space then (iv) =>• (ii). Exercise 2.44 Let X be a normed space and / £ A(X) be lower bounded. Consider A > i n f l € x / ( x ) =: inf / and p > 0. We envisage the conditions: (a)
[f<X}CpU;
(b) V x £ X (c) V x ' G
A
: f(x) > inf / + "2'°f/t/x»
A
~lnf 2p
f
• max{0, la?l  p};
: / * ( * ' ) < / * ( 0 ) + px'.
Prove that (a) =>• (b) & (c). Exercise 2.45 equivalences:
Let X be a normed space and / G T(X). Prove the following
Convex analysis in locally convex spaces
154
(a) liminf.1._+00 / ( x ) /   x   > 0  » 0 £ int(dom/*); in this case f(x) liminf 4fif = sup{/i > 0  /* is upper bounded on (J.U*}. xKX>
X
(b) liminf  B  H o o f(x)/\\x\\
= 0 « 0 6 Bd(dom/*).
(c) liminfHJIIKX, f(x)/\\x\\
< 0 O 0 ^ cl(dom/*); in this case
f(x)
liminf ~r{ =  d d o m / . (0). llxHKX)
E x e r c i s e 2.46
Z
Let p 6 ]1, oo[ and f:£p^R,
/ ( ( * „ ) „ > i ) := Y°°
n\xn\n.
Prove that / is finite, convex, continuous and lim„_+0O f*(y)/\\y\\
— 1.
E x e r c i s e 2.47 Let X, Y be Hilbert spaces, C C X be a nonempty closed convex set and A G L(X,Y) be a surjective operator. Consider the problem min iAc 2 ,
(P)
xeC.
A solution x of problem (P) is called a spline function in C associated to A. (a) Prove that (P) has optimal solutions if C + ker ^4 is closed. (b) Prove that x G C is an optimal solution of (P) if and only if y := A* (Ax) satisfies the relation (x\y) = min{(x \y) \ x € C}. E x e r c i s e 2.48 Let X,Y be Hilbert spaces, C := {x 6 X  Vi, 1 < i < k : (x\a,i) < fii}, where ai,...,ajb G X', Pi,...,Pk G R and A G £ ( X , F ) be a surjective operator. Consider the problem (P)
min
5   A E   2 , x G C.
Prove that (P) has optimal solutions. Suppose, moreover, that there exists x € X such that (x \ a{) < /3i for every i, 1 < i < k. Prove that l i s a solution of (P) if and only if there exists (\i)i
V Xkdk a n d V i , 1 < i < A: : A;((x a») — /?;) = 0.
Exercise 2.49 Let X be a Hilbert space and 01,02,03 be three non colinear elements of X (i.e. they are not situated on the same straightline). Prove that there exists a unique element x G X such that V i £ l
:   x  O i   +   x  a 2   +   i  a 3   < x  ai + x  o 2  + x  a 3 .
Moreover, prove that x G co{oi,a2,03}. The point x is called the Toricelli point of the triangle of vertices a\, 02,03.
Bibliographical notes
155
E x e r c i s e 2.50 Determine the optimal solutions (when they exist) and the value of the problem (Pf)
min £ (tx{t) + fiy/1 + (u(t)) 2 ) dt, x e Xi,
for every fj, € R + and every i g {1, 2, 3,4}, where Xi:=C[0,l], X 3 :=  x e C [ 0 , l ]  / o 1 x ( t ) d t = 0 J , E x e r c i s e 2.51 (P)
X a : = 2/(0,1), X» := { x G 1/(0,1)  /^ x(t)dt = o } .
Consider the (convex) optimization problem
max J,,1 x(t) dt, xeX,
x(0) = x(l) = 0, J,,1 y/l +{x'(t))2dt
< L,
where L > 0 and X = C^O, 1] := {x : [0,1] —• R  x derivable, x continuous on [0,1]} or X = AC[0,1] := {x : [0,1] ¥ R  x absolutely continuous}. Determine the optimal solutions of problem (P), when they exist, and its value (using, eventually, the dual problem).
2.12
Bibliographical N o t e s
Many results of this chapter are wellknown and can be found in several books treating convex analysis: [Rockafellar (1970); HiriartUrruty and Lemarechal (1993)] (for finite dimensional spaces), [Ekeland and Temam (1974); Ioffe and Tikhomirov (1974); Barbu and Precupanu (1986); Castaing and Valadier (1977); Phelps (1989); Aze (1997)]. In the sequel we point only those results that are not contained in these books but the last one. Sections 2.12.5: The assertion (vii) of Theorem 2.1.5 was established in [Zalinescu (1983b)]. The characterization of convex functions using upper Dini directional derivatives in Theorem 2.1.16 as well as Lemma 2.1.15 are established by Luc and Swaminathan (1993). Theorem 2.2.2 was established by Crouzeix (1980) in finite dimensional spaces and Theorem 2.2.11 by Zalinescu (1978). The notion of quasicontinuous function was introduced by Joly and Laurent (1971); Proposition 2.2.15 and Corollary 2.2.16 are established by Moussaoui and Voile (1997). The notions of csconvex, csclosed and cscomplete functions are introduced by Simons (1990), while that of lcsclosed function by Amara and CiligotTravain (1999); all the results concerning lcsclosed functions are due to Amara and CiligotTravain (1999). Theorem 2.2.22 (for I = N) and Proposition 2.2.24 can be found in [Kosmol and MullerWichards (2001)]; when d i m X < oo Theorem 2.2.22, Corollary 2.2.23 and Proposition 2.2.24 are stated in [Marti (1977)] under the weaker hypothesis that the pointwise boundedness or pointwise convergence holds on a dense subset of C. Proposition 2.2.17 is stated in [Zalinescu (1992b)]. Proposition 2.4.3 is established by Rockafellar (1966), Theorem 2.4.11 is proved in the book [Rockafellar (1970)] in R n and stated by HiriartUrruty
156
Convex analysis in locally convex spaces
(1982) in the general case; one can find another proof in [Aze (1997)]. The local boundedness of df in Theorem 2.4.13 and Corollary 2.4.10 can be found in many books on convex analysis. The assertions (iv) and (vii) of Theorem 2.4.14 are established in [Zalinescu (1980)]; for (i) see also [Anger and Lembcke (1974)]. Theorem 2.4.18 is from the book [Ioffe and Tikhomirov (1974)]. Theorem 2.5.2 was proved by Polyak (1966) under the more stringent conditions that / is a quasiconvex function which is bounded and Lipschitz on bounded sets and attains its infimum on every closed convex subset of X. Theorem 2.5.5 and Lemma 2.5.3 are established by Borwein and Kortezov (2001), while Lemma 2.5.4 and other results on nonattaining convex functionals are established by Adly et al. (2001a). Sections 2.62.9: The systematic use of perturbation functions for calculating conjugates and subdifferentials was done for the first time by Rockafellar (1974). The author of this book continued this approach in [Zalinescu (1983a); Zalinescu (1987); Zalinescu (1989); Zalinescu (1992a); Zalinescu (1992b); Zalinescu (1999)]; this permitted, for example, to give simpler proofs to several results stated in [Kutateladze (1977); Kutateladze (1979a); Kutateladze (1979b)]. Theorem 2.6.2(i) was established by Moussaoui and Seeger (1994), but Theorem 2.6.2(h) and Theorem 2.6.3 are new. Corollaries 2.6.42.6.7 can be found in [HiriartUrruty and Phelps (1993)] and [Moussaoui and Seeger (1994)]; for other results in this direction see the survey paper [HiriartUrruty et al. (1995)]. The sufficient conditions for the fundamental duality formula and for the validity of the formulas for conjugates and esubdifferentials are, mainly, those from author's survey paper [Zalinescu (1999)], where one can find detailed historical notes; here we mention only the first use (to our knowledge) of them; actually, all these sufficient conditions are mentioned in that paper, excepting those which use liconvex or lcsclosed functions. So, conditions (iii) and (viii) of Theorem 2.7.1 and the corresponding ones in Theorems 2.8.1, 2.8.3, 2.8.7 and 2.8.10 are the classical ones and can be found in all the mentioned books which treat them. Condition (i) of Theorem 2.7.1 was used by Rockafellar (1974) when Yo = Y and by Zalinescu (1998) in the present form (see also [Combari et al. (1999)]), (ii) and (vi) were introduced in [Zalinescu (1983a)] for Yo = Y (and R replaced by a separated lcs ordered by a normal cone) and in [Zalinescu (1999)] in the present form, (iv) was introduced in [Zalinescu (1992b)] with (Hwi) replaced by (Hx), (v) is stated by Amara and CiligotTravain (1999) for X, Y Frechet spaces, $ a lcsclosed function and ,b replaced by lc, (vii) was introduced by Rockafellar (1974) for X, Y Banach spaces (X even reflexive) and Yo = Y, and by Zalinescu (1987) in the present form, while (ix) was used by Joly and Laurent (1971) for $ lsc and by Moussaoui and Voile (1997) for arbitrary 3>; note that condition (b) of [Cominetti (1994), Th. 2.2] implies condition (2.54). Theorem 2.7.4(iv) was established in [Zalinescu (1992b)]; the other conditions were introduced in [Zalinescu (1999)] (but in (v) it was assumed that F is lsc and C is closed). Condition (iii) of Theorem 2.8.1 was introduced in [Ekeland and Temam (1974)], (vii) was introduced in [Zalinescu (1984)] (for Y0 = Y), (iv) was intro
Bibliographical notes
157
duced in [Zalinescu (1992b)], (i) and (viii) in [Zalinescu (1998)] for X,Y normed spaces, while conditions (ii) and (vi) were introduced in [Zalinescu (1999)]. Condition (vii) of Theorem 2.8.3 was introduced by Borwein (1983) for Yo = Y (see also [Zalinescu (1986)]), (x) was introduced by Borwein and Lewis (1992), (ix) was introduced by Moussaoui and Voile (1997), (i) by Zalinescu (1998) for X, Y normed spaces, (ii) was introduced by Combari et al. (1999), (iv) was introduced by Zalinescu (1999) (a slightly stronger form was used in [Simons (1990)]) as well as conditions (v) and (vi). For C G L(X, Y), generally, Theorem 2.8.6 is obtained under the corresponding conditions of Theorem 2.8.3 (taking / = 0). For C a densely defined closed linear operator, Rockafellar (1974) and HiriartUrruty HiriartUrruty (1982) obtained the results for g lsc, X, Y Banach spaces and Yo = Y (in [Rockafellar (1974)] X is reflexive and e = 0); Aze (1994) obtained the formula for the conjugate under condition (ii) in normed spaces. For general C condition (iv) was used by Zalinescu (1992b); the other conditions (excepting (v)) were used in [Zalinescu (1999)]. Condition (ix) of Theorem 2.8.7 was used by Joly and Laurent (1971), (vii) for X a Banach space was introduced by Attouch and Brezis (1986), (vi) was introduced by Zalinescu (1992b), (i) was introduced by Aze (1994) in an equivalent form in normed spaces, while (ii) was introduced by Combari et al. (1999). Corollary 2.8.8 was obtained by Voile (1994) for X a Banach space, / , g lsc and '* replaced by !C. Formula (2.65) from Proposition 2.8.9 was obtained by Aze (1994) and Simons (1998b) for x = 0 € ( d o m / — domg) 1 when / , g are lsc. The case / = 0 of Theorem 2.8.10 was considered by several authors; condition (iii) was used in [HiriartUrruty (1982); Lemaire (1985); Zalinescu (1983a); Zalinescu (1984)]; the algebraic case was considered in [Kutateladze (1979a); Kutateladze (1979b); Zalinescu (1983a)]; Zalinescu (1984) considered a stronger version of (v) (for lsc functions and Yo = Y); note that in these papers g is assumed to be Qincreasing on the entire space. The general case was considered in [Combari et al. (1994)] under (iii) and a condition stronger than (v) (/, g, H lsc and lc instead of lb) and in [Combari et al. (1999)] under condition (i) and a variant of (iv); it was also considered by Moussaoui and Voile (1997) under condition (vii). The formula for d(p(x) in Corollary 2.8.11 can be found in [Combari et al. (1994)]. The formula for d
158
Convex analysis in locally convex spaces
Section 2.10: The minimax theorem is also classical and can be found in the books [Barbu and Precupanu (1986); Simons (1998a)]. Exercises: Exercise 2.2 is from [Penot and Bougeard (1988)], Exercise 2.3 is from [Crouzeix (1981)], Exercise 2.6 is from [Marti (1977)], Exercise 2.15 is from [Crouzeix et al. (1992)] (there in a more complete form), Exercises 2.18 and 2.29 are from [HiriartUrruty and Lemarechal (1993)], Exercise 2.19 is from [Lions and Rochet (1986)], Exercise 2.20 is the celebrated cancellation lemma from [Hormander (1955)], Exercise 2.21 is from [HiriartUrruty (1998)], Exercise 2.22 is from [Jourani (2000)], the assertions in Exercise 2.23 are wellknown (see also [Aze (1997)]), Exercise 2.25 can be found in [Borwein and Kortezov (2001)] (in normed spaces), the formula for goo under condition (vii) of Theorem 2.8.1 in Exercise 2.26 is obtained in [Amara and CiligotTravain (1999)] and [Amara (1998)], Exercise 2.27 is the wellknown TolandSinger duality formula (see [Toland (1978)] and [Singer (1979)]), Exercise 2.30 is from [Zalinescu (1999)], the equivalence of a) and c) (for Banach spaces) in Exercise 2.32 can be found in [Bauschke et al. (2001)] (see also [Barbu and Da Prato (1985)]), Exercises 2.33 and 2.34 are from [Eremin and Astafiev (1976)], Exercises 2.35, 2.36 and 2.37 are from [Simons (1998a)], the last formula in Exercise 2.38 is from [Walkup and Wets (1967)], Exercise 2.39 is from [Penot (1998a)], the formula for the subdifferential of the function considered in Exercise 2.40 and its proof are from [Aussel et al. (1995)], the equivalences (e) •£> (f) and (g) & (h) of Exercise 2.41 are from [Zalinescu (1983b)], the equivalence of conditions (ii)(iv) in Exercise 2.43 are established in [Adly et al. (2001b)] in reflexive Banach spaces, a weaker variant of the implication (a) => (c) of Exercise 2.44 is stated in [Aze and Rahmouni (1996)], Exercise 2.45(c) is from [Borwein and Vanderwerff (1995)], Exercises 2.47, 2.48 axe from [Laurent (1972)].
Chapter 3
Some Results and Applications of Convex Analysis in Normed Spaces
3.1
Further Fundamental Results in Convex Analysis
Throughout this chapter X, Y are normed spaces and X*,Y* are their duals endowed with the dual norms. As an application of Ekeland's variational principle and of some results relative to convex functions, we state the following multipropose generalization of the Br0ndstedRockafellar theorem. Theorem 3.1.1 (Borwein) Let X be a Banach space and f £ r ( X ) . Consider e £ P, XQ £ d o m / , XQ £ def{xa) and /3 € K+. Then there exist xe £ X, y* € Ux and Ae £ [—1, +1] such that \\xe  Xo\\ + P • \(X£  X0,X*0)\ < y/E, x% := x*0 + VE(y* + /3\ex*) £ df{xe).
(3.1) (3.2)
Moreover * e  soil < Ve,
\texl\\
\(xex0,x*)\<e x*e£d2ef(x0),
+ P\\xl\\),
+ JilP,
\f(xe)f(x0)\<e
(3.3) (3.4)
+ ^/fi,
(3.5)
with the convention 1/0 = oo. Proof.
The function
 • Ho •. x  • K, \\x\\0~\\x\\
+
p\(x,x*)\,
is, obviously, a norm on X, equivalent to the initial norm. Therefore {X, 0) is a Banach space. Consider the function g := / — XQ. It is obvious 159
160
Some results and applications of convex analysis in normed spaces
that Xo £ dom
: g{x)>g(x0)e
[& x*0 £
dsf{x0)].
We apply Ekeland's theorem (Theorem 1.4.1) to g, y/e and the metric d given by d(x,y) := \\x — y\\o. So there exists xe £ domg such that g{xE) + y/e \\x£ x0\\o \/x£X,
x^xe
< g(x0),
(3.6)
: g(x£) < g(x) + y/e • \\x  xe\\0.
(3.7)
From Eq. (3.6) we obtain that g(xe) + Ve(\\xs  ar0 +0\{xe
x0, XQ)\) < g(x0) < g(xe) + e,
whence Eq. (3.1) follows immediately. Let us consider the function h : X > E, h(x) := g(x)+y/e\\xxE\\0

f{x){x,xl)+y/E\\xxE\\+Py/e\(xxe,xl)\;
by Eq. (3.7) xe is a minimum point of h. Therefore 0 £ dh(xe). Since h is the sum of four convex functions, three of them being continuous, and taking into account the expression of the subdifferential of a norm (Corollary 2.4.16) and of the absolute value of a linear functional (at the end of Section 2.8), we have that 0 £ dh(xe) = df(x£)
 x* + y/i • Ux + PVe • [  1 , +1] • x%.
Therefore there exists x* £ df(xs), y* £ Ux* and Xe £ [—1,1] such that Eq. (3.2) is verified. The estimations from Eq. (3.3) follow immediately from Eqs. (3.1) and (3.2). Using again Eqs. (3.1) and (3.2) we obtain that (a;e x0,x*
XQ)\ = y/e\{xE
 xQ,y* + p\Ex%}\
< v/i(z e  x0\\ • W\
+ 0\Xe\ \(xe
< y/e(\\xs  x0\\ + fi • \{x£
x0,x*0)\)
x0,x*0)\)
= e.
(3.8)
From Eqs. (3.1) and (3.8) we get (a;e x0,x*)\
< \{xe x0,x*
x^)\
+ \(x£
X0,XQ)\
^IP,
Further fundamental
results in convex
analysis
i.e. the estimation in Eq. (3.4) holds, too. Since x$ G dEf(xo) df(xE), we get (X0 ~Xe,X*£
XQ)
+ (X0 X£,X*Q)
= (X0 ~Xe,X*) < (X0
< f(x0)
XE,XQ)
161
and x* G
~f(Xe)
+£.
Using relations (3.1) and (3.8), we obtain that \f(x0)  f(xe)\
< \(x0xe,x*0)\+s<e
+ VE//3,
i.e. the second relation in (3.5) holds. Using Eq. (3.8) and the fact that XQ G dEf(x0), x* G df(xE), we obtain that (xX0,X*e)
= (XX£,X*)
+ (Xe X0,X*
XQ)
+ (XE
X0,XQ)
< f{x)  f(xe) +s + f(x£)  f{x0) + e  f(x)  f(x0) + 2e for every x G X, i.e. x£ G d2ef(x0); holds, too.
hence the first relation in Eq. (3.5) •
The next result is wellknown. The first part is an immediate consequence of Borwein's theorem, while the density part, which follows easily from the first one, will be reinforced in Theorem 3.1.4 below. Theorem 3.1.2 (Br0ndstedRockafellar) Let X be a Banach space and f € r ( ^ )  Consider e > 0 and (XO,XQ) G grd6f. Then there exists (x£,x*) G gr<9/ such that \\x£— XQ\\ < ^Jl and \\X*—XQ\\ < y/e. Inparticular domf C cl(dom<9/) and dom/* C cl(Im<9/). D Another consequence of Borwein's theorem is the following result which will be completed in Proposition 3.1.10 below. Proposition 3.1.3
Let X be a Banach space and f G T(X). Then
f(x) = s u p ^ z  z,z*) + f(z) I (z,z*) G grdf} = sup{(x, z*)  /•(**)  z" G lm(df)}
(3.9)
for every x G d o m / . Proof. The second equality in Eq. (3.9) is obvious because f(z)+f*(z*) = {z, z*) for any (z, z*) G grdf. Let x G dom / be fixed. When (z, z*) G grdf we have (a; — z, z*) + f(z) < f(x) for every i £ l ; hence f(x) > sup{(a; z,z*) + f(z)  [z,z*) egrdf}. Because / is lsc, by Theorem 2.4.4 (hi), there
162
Some results and applications of convex analysis in normed spaces
exists x* € de/2f(x). Applying Borwein's theorem we get {xe,x*) £ grdf such that x* 6 def(x). Therefore (xe — x,x*) < f{xe)  f(x) + e, whence f(x) — e < {x — xe,x*) + f(xe). Hence relation (3.9) holds. • As announced before, the density results in Theorem 3.1.2 can be stated in a stronger form. In the next result (xn) —>/ x means that (a;n) —> x and {f(xn)) * f(x), and similarly for (xn) >•/• x*. Theorem 3.1.4
Let X be a Banach space and f € r ( X ) . Then
(i) Vx e d o m / , 3((xn,xn))
Cgrdf
: (a;„) +/ x;
(ii) Vx* E dom/*, 3 ( ( x „ , < ) ) C gvdf
: « )  • , . x*.
Proof, (i) Consider x € d o m / . Because / is lsc, by Theorem 2.4.4 (hi), for every n S N there exists x* € dn2f(x). Applying Borwein's 2 theorem for (x,x*), e = n~ and /? = 1, we get (xn,xn) € grdf such that \\xn — x\\ < n _ 1 , \f(xn) — f(x)\ < n~2 + n~x. Hence (xn) ^/ x. (ii) Because / * is lsc, it is sufficient to show that for every e > 0 there exists (y, y*) e df such that \\y*  x*\\ < e a n d / * ( y * ) < f*(x*)+e. Fixx £ d o m / . Let £> 0 and take r := \\x\\ + 2e1 (e + f(x) + f*(x*)  (x,x*)) > 0. Consider also the function g := fO(x* + irux) € A(X). Then g(x) > (fnx*)(x) = (x,x*)  f*(x*) for every x € X. It follows that g € T(X) and f(x) + (—x,x*) > g~(0) > —f*(x*). By Proposition 3.1.3 there exists (z,z*) € dg such that g*(z*) = g(z)  (z,z*) > f*(x*)  e/2. But + (x* + trUx)*(z*) = f*(z*) + r lar*  z*\\. Therefore g*(z*) = f*(z*) f*(z*) +
r\\x*z*\\
Because f*(z*) > (x,z*)  f{x) >  \\x\\ • \\z*  x*\\ + (x,x*)  f(x), from the preceding inequality we obtain that (r — \\x\\) \\z* — x*\\ < e + f(x) + f*(x*) — (x,x*), and so \\z*  x*\\ < e/2. The inequality above shows also that f*(z*) < f*(x*)+e/2. On the other hand, because (z,g~(z)) € cl(epi^), there exists (zn) C dom / and (un) C rUx such that (zn + un) —> z and limsup(/(z n ) + (u„,a;*»
(3.10)
we have that
0 < £„ = f{Zn) + (U„, X*)  (Zn + Un, Z*) + (Un, Z*  X*) +
f*(z*).
Taking into account that (un,z* — x*) < r \\x* — z*\\, from Eq. (3.10) we obtain that limsup£ n < 0. Hence lime„ = 0. Because (un) is bounded, so
Further fundamental results in convex analysis
163
is (zn). Therefore there exists r' > 0 such that (zn) c r'Ux Take n E N so that S := en < 1 and VS(r' + 2)(1 + \\z*\\) < e/2; set z := zn. Of course, we have that z* G dsf(z). By Borwein's theorem (for /? = 1) there exists (y,y*) G df such that \\z  y\\ < y/S, \\z*  y*\\ < VS{1 + z*) and /(«) ~ f(v)\ <S + VS. Hence j/*  x*\\ < y/S{l + \\z*\\) + e/2 < e and /*(»*) =
+ f*(z*) + f{z) 
f(y)
< VS(V5 + r') (1 + z*) + \f5 \\z*\\ 5 + f(x*) < f*(x*) + e/2 + VS(r' + 2)(1 + «*) < f*(x*)
+s/2 + 6 + V6 + e.
The proof is complete.
•
Using Br0ndstedRockafellar's theorem we can add other sufficient conditions for the validity of the conclusions of Theorems 2.8.1 and 2.8.7. Proposition 3.1.5 Let X,Y be Banach spaces, F £ T(X x Y), A G L(X,Y), D = {Ax  y \ (x,y) € d o m F } and E = {Ax  y \ (x,y) € dom&F}. Then ic
E = iiE = ic(coE) = ri(coE) = icD = riD.
Moreover, if one of the above sets is nonempty then icE = E — D; in particular the sets ICE and E are convex. Proof. Consider the linear operator T : X xY >• Y defined by T(x, y) := Ax — y. Since domdF C dom.F and d o m F is convex, we have that E = T(domdF)
CcoE
= T(co{domdF))
By Theorem 3.1.2 we have that d o m F C domdF. nuity of T, we have that
CD = T ( d o m F ) . Hence, using the conti
E C D C T ( d o m d F ) C T(domdF) = E. Using the properties of the affine hull (see p. 2), it follows that aff E C aff D c aff E. Therefore, if aff E is closed then aff D — aff E; the above relation shows that %CE c %CD. Let us show the converse inclusion. Let 2/o G %CD and consider the function F 0 6 T(X x Y), F0(x,y) :— F(x,y — yo)Of course, domF 0 = d o m F + (0,j/o) and dom9F 0 = d o m 9 F + {0,y0). Therefore 0 € i c ( P r y ( d o m F 0 ) ) . Taking
164
Some results and applications of convex analysis in normed spaces
such that dF0(x,Ax) = dF(x,Ax — y0) ^ 0. Therefore yo € E. So we obtained that *D = ICD C E C D, which shows that aff E = affD and *D C iE. It follows that icE = icD = ic(coE). As observed after the proof of Theorem 2.8.1, in our situation, if %CD is nonempty we have that ic D = rintD. Suppose that %CD 9^ 0 (for example). From what was proved above, we have that rinti? = lD = icD = rintE = lE = icE C E C D C E. Hence E~ = T°E = D:. The conclusion follows.
•
Remark 3.1.1 Taking into account the preceding proposition, for X, Y Banach spaces and F G T(X x 7 ) , we may add to the sufficient conditions in Theorem 2.8.1 the following conditions: 0 G ri{Ax  y \ (x,y) £ ic
0 £ {Ax y\(x,y)€ ic
0 € {Ax y\(x,y)e
domdF}, domdF}, co(domdF)},
y 0 = cone{ Ar  y  (x, y) 6 dom OF} is a closed linear space. Indeed, the first three conditions are, evidently (using the preceding proposition), equivalent to 0 £ lc{Ax — y \ (x,y) £ d o m F } . In the fourth case y 0 = aff {Ax — y\(x,y)e co(dom OF)}, and so 0 e lc{Ax — y\(x,y)e domF}. An important particular case of the preceding proposition is when X = Y, A = Idx and F(x,y) = f(x) + g(y) with f,g 6 T(X); in this case D — d o m / — domg and E = domdf — domdg. Using the Borwein theorem one obtains a formula for the subdifferential of a composition g o A of a lsc convex function and a continuous linear operator without qualification conditions. Theorem 3.1.6 Let X,Y be Banach spaces, A e C{X,Y), g £ T(Y), f = g o A and x 6 d o m / . Then x* 6 df(x) if and only if there exists a net ((yi,yt))ieI C grds such that (yt) >• y := Ax, (g(yi)) >• g{y), ((yiy,y*))^Oand(A*y*)^x*. If X is reflexive one can take sequences instead of nets and impose norm convergence instead of w* convergence.
Further fundamental
Proof.
Let (fa,y*))i€l
results in convex
analysis
165
C gidg be such that fa) > y, (fa  y,y*)) >•
0, {gfa)) > g(y) and (i4'i/r) ^ x*. Then (y  yhy*) i G I and t / S F . It follows that (x'  x,A*y*) + (y yhy*)
= (Ax'  yuy*)
< gfagfa)
for all
< f(x')  gfa)
for alH G / and x' G X. Taking the limit we obtain that (x' — x,x*) < /(x')  f(x) for all x' G X, and so x* G d/(x). Let now x* G df(x) and consider (£„)„6N 4 0 Using Corollary 2.6.5 we have that x* G w*clA*(d£ng(y)) for every n G N. Let A/" be a base of w*neighborhoods of x* and consider / = Nx M with (n, V) y (n1, V) iff n > n' and V CV. Then for all i = (n, V) G / there exists z* G dEng(y) such that A*£* G V. Taking /? = 1 and e = e^ in Theorem 3.1.1, there exists (yi,y*) G 3g such that \\yi  T/ < e„, y*  ^   < en, \g(yt)  g(y)\ < en(en + l) and \(Vi ~ V,V*)\ < en(en + 1). Because \\A*z*  A*y*\\ < \\A*\\ • \\y*  z*\\ < en \\A*\\ and (A*z*)ieI ^> x*, we obtain that (A*y*)i€l ^> x*. If X is reflexive then w*cli*(9 fB g(j/)) = cl A* (dSng(y)), and so, for every n G N, we can take z*n G dEng(y) such that A*z* — x* < e„. The conclusion follows. • Using the preceding result one obtains a similar formula for the subdifferential of the sum of two lsc proper convex functions. Theorem 3.1.7 Let X be a Banach space, / i , / 2 G T(X) and x G dom / i fl dom / 2 . Then x* e S f / i + M (x) if and only if there exist two nets {(xk,i>xt,i))ieI C gr<9/fc, k = l,2, such that ( x M ) i e / >• x, (fk(xk,i))ieI > /fc(x), ( ( x M  x , x ^ i ) ) . e / ^0 for k = 1,2 and (x*^ + x*2x*. 7/X is reflexive one can take sequences instead of nets and impose norm convergence instead of w* convergence. Proof. We apply the preceding result iorY ~ XxX, A: X > Y defined by Ax := (x,x) and# : Y > E defined by #(xi,x 2 ) := /i(xi)+/2(x2). Then / := /1+/2 = g°A. The sufficiency is immediate by taking y = (x, x) = Ax, Vi = (xi,i,X2,i) and y* ~ (xl^xlj) for i G i". Let x* G df(x). By Theorem 3.1.6 there exists a net ((yi,y*))ieI C gvdg verifying the conditions of the theorem with y := (x,x). Taking yt = (xi ) j,x 2 ,i) and y* = ( x ' ^ x ^ j ) and taking into account that dg(x\,X2) — dfi(x{) x 0f2(x2), we have that ((xk,i,x*ki))ieI C gvdfk and (xk,i)iei »• x for k = 1,2. Assume that (fi(xi,i)) A fi(x) Because /1 is lsc at x, limsup i 6 / /i(xi,i) > /i(x). Thus
166
Some results and applications of convex analysis in normed spaces
there exists S > 0 such that J := {i £ I \ fi(xiti) — fi(x) > 5} is cofinal. It follows that g(yi)  g(y) > 6 + (f2{x2,i) — f2{x)) for all i £ J. Taking the limit inferior, we obtain the contradiction 0 > S. Therefore {fk(%k,i))ieI ^ fk(x) for k — 1,2. The inequalities fi(xi,i)
 fi(x) < (xij x,x{A)
for i £ I imply that ({xij — x,x{ti})
< (yi y,y*)
 {h(x2,i)

f2(x))
> 0.
D
Using Br0ndstedRockafellar theorem we obtain another famous result. Theorem 3.1.8 (BishopPhelps) Let X be a Banach space and C C X, C 7^ X, be a nonempty closed convex set. Then (i)
The set of support points of C is dense in B d C .
(ii) The set of support functionals of C is dense in the set of continuous linear functionals which are bounded above on C. Moreover, ifC is bounded, then the set of support functionals of C is dense in X*. Proof, (i) Let x0 6 Bd C and e £ ]0,1[. Taking into account that C ^ X, there exists xi £ X \C such that \\xi — x 0  < £2 Applying a separation theorem, there exists XQ £ Sx> such that SVLPX£C{X,XQ) < (XI,XQ). SO, for every x £ C, {x  X0,XQ)
= (x  XI,XQ)
whence XQ £ de2tc(xo)have that
+ (xi  X0,XQ)
< (x  xi,xl)
+ \\x\  x0\\
<
e2,
Applying the Br0ndstedRockafellar theorem we
3xe £ C, 3a;* £ dbc{xe)
: \\x£  x0\\ < e, \\x*  XQ\\ < e < 1.
Since \\XQ\\ = 1, we have that x*e ^ 0, and so xE is a support point of C with \\x£ — xo\\ < e. The conclusion holds. (ii) Using the second part of Theorem 3.1.2 we have that dom(tc)* C cKJmdic) = cl(Imdt c \ {0}), which shows that the conclusion of the theorem is true.
•
The following theorem will be used for a simple proof of the Rockafellar theorem.
Further fundamental results in convex analysis
167
Theorem 3.1.9 (Simons) Let X be a Banach space, f € T(X) and x € X, n G R be such that inf / < 77 < f(x). Consider L :=
sup fizz rr, x€X\{x} ar — ar
and dx • X V R, dj(x) :=   i — x. Then: (i) 0 < L < oo and inf (/ + Ldx) > n; (ii) V e e ] 0 , l [ , 3 i / 6 X
: ( / + L«fc)(i/) < i n f ( / + i d * ) + e L   i  y\\;
(iii) V e € ] 0 , l [ , 3 («,*•) G g r d / and\\z*\\ < (l + s)L;
: {x  z,z*) > (I  e)L\\x  z\\ > 0
(iv) Ve €]0,1[, 3(z,z*) € g r S / : (z  z,z*> + /(*) > 17 and L < \\z*\\<(l + e)L. Proof, (i) Because 77 > inf / , it is clear that L > 0. Since 77 < /(a;) and / is lsc at x, there exists p > 0 such that / ( x ) > 77 for every a; € B(x,p). Furthermore, since / € r ( X ) , there exist x* € X* and a 6 R such that f(x) > (a;!a;*) ~ Q f° r every a; £ X. Therefore 77 — f(x)
V* £ X, \\x  x\\ > p : 5_jlZg) < x* + _ ] _ < li^H + 2. IF ~ x\\
\\x ~ x\\
P
This relation and the choice of p show that L < 00. From the expression of L we obtain immediately that inf (/ + Ldx) > 77. (ii) Let e e]0,1[. Since (1  e)L < L, from the definition of L there exists y £ X, y ^ x, such that (77 — f{y))/\\x — y\\ > (1 — e)L, and so (/ + Ldx)(y) < 77 + eLdi(y) < inf(/ + Ldx) + eL\\y  x\\. (iii) Let e €]0,1[ be fixed. The function / + Ldx 1S proper, lsc and bounded from below, while the element y from (ii) is in dom(/ + Ldx) = d o m / . Taking the metric d o n X defined by d(xi,X2) := L\\xi—X2\\, (X,d) is a complete metric space. Using Ekeland's variational principle we get the existence of z € X such that (/ + Ldx)(z) + EL\\Z  7,11 < (/ + Ldx)(y)
168
Some results and applications of convex analysis in normed spaces
and (f + Ldw)(z) < {f + Ld¥)(x) + eL\\x  z\\
VxeX.
The first relation and (ii) give ^ — yj < \\x — y\\, and so z ^ I . The second relation says that z is a minimum point of the function / + Ld^ + sLdz. Taking into account that cfe and dz are continuous convex functions, it follows that 0 G d(f + Ldw + eLdz)(z) = df(z) + Ldd^{z) +
eLddz(z).
But ddz{z) = d\\ • (0) = Ux and dd^(z) = {x* € Ux*  (z  x,x*) = \\z — x\\}. Hence there exist z* G df(z) and x*, y* £ Ux* such that z* = Lx* + sLy* and (x  z, x*) — \\x  z\\. Therefore z* < (1 + e)L and (x — z, z*) = {x — z, Lx* + eLy*) = L\\x — z\\ + eL(x — z, y*) > L\\x  z\\  eL\\x  z\\ = L(l  e)\\x  z\\ > 0. (iv) Let e £]0,1[ be fixed and consider e' := e/3. Let us take M := (1 + 2e')L. Since / + Md% > f + Ld% > r], we can apply Ekeland's theorem for / + Mdx, an element xo of dom / , e' > 0 and the metric defined at (iii). We get so the existence of z e X such that V i e l :
(f + Mtk)(z)<(f
+
Mds)(x)+e'L\\xz\\.
As in the proof of (iii), there exist z* € df(z), x*, y* € Ux* s u c h that z* = Mx* +e'Ly* and (xz,x*) = \\x  z\\. Thus z* < (1 + s)L and (x  z, z*) > (M  e'L)\\x  z\\ = (1 + e'L)\\x  z\\. So, for x = z we have that (x — z, z*) + f(z) = f(x) > rj, while for ~x yi z we have (xz,z*)
+ f(z) > (l + e')L\\xz\\
+ f(z) = (f + Ldw)(z)+e'L\\xz\\
> v.
Therefore (x — z, z*) + f(z) > T). Moreover i  z • \\z*\\ >(xx,z*) >V~
= [(x  z, z*) + f(z)]  [(x  z, z*) + f(z)]
f(x)
for every x € X; the last inequality holds because z* G df(z). af — x\\ for x 7^ x, we obtain that z* > L.
Dividing by D
Convexity and monotonicity
of
subdifferentials
169
Using the preceding theorem we reinforce slightly Proposition 3.1.3. Proposition 3.1.10 Let X be a Banach space and f 6 T(X). relation (3.9) holds for every x 6 X.
Then
Proof. Taking into account Proposition 3.1.3 we have to show that oo = sup{(a; — z,z*) + f(z) \ (z,z*) € gr<9/} when x £ d o m / . So, let x £ X \ d o m / and take inf / < 77 < 00. Using Theorem 3.1.9 (iv), there exists {z,z*) E gr<9/ such that (a;  z,z*) + f(z) > rj. The conclusion follows. • The maximal monotone operators are of a great importance in the theory of evolution equations. A significant example of such operators is the subdifferential of a proper lsc convex function on a Banach space. Theorem 3.1.11 (Rockafellar) Let X be a Banach space and f 6 T(X). Then df is a maximal monotone operator. Proof. Let (x,x*) EXxX*\gvdf. Then x* £ df{x), and so 0 i df(x), where / := / — x*. It follows that inf/ < fix). Applying assertion (iii) of Simons' theorem, there exists (z, z*) € g r 9 / such that (x — z,z*} > 0. Consider z* := z* + x*\ we have that (z, z*) £ df and (z x,z*— x*) < 0, which shows that the set df U {(x,x*)} is not monotone. Therefore df is a maximal monotone operator. • 3.2
Convexity and Monotonicity of Subdifferentials
The aim of this section is to show that the monotonicity of an abstract subdifferential of a lower semicontinuous function ensures its convexity; among these abstract subdifferentials one can mention the Clarke subdifferential we introduce below. Throughout this section (X, .) is a normed vector space. Consider  / M c I and x e clM. In the sequel we shall denote by (x n )  > M X a sequence (xn) C M with (xn) » x; more generally, x  > M X will mean x £ M and x —> x. The Clarke's tangent cone of M at x is defined by Tc{M,x):={ueX\V{tn)
10, V(xn)*Mx,
3(un)^u, V n e N : xn + tnun 6 M).
Recall another cone introduced in Section 2.3: C(M,x) := e l f  J
t(Mx))
=cone{Mx).
170
Some results and applications of convex analysis in normed spaces
Because clM x C C(M,x) Note (Exercise!) that
C C(clM,i), we have C{M,x) = C(clM,:r).
0ETc(M,x)cC(M,x).
(3.11)
Several properties of the tangent cone in the sense of Clarke are collected in the following proposition. Proposition 3.2.1 (i) Tc{M,x)
Let 0 ^ M C X and xEc\M.
Then:
is a nonempty closed convex cone;
(ii) TC(M, x) = Tc(cl M, x) = Tc(M D V, x) for all V E M{x); (iii)
if M is a convex set then Tc(M,x)
=
C(M,x).
Proof, (i) It is obvious that Xu E Tc(M,x) for A > 0 and u E Tc(M,x); hence Tc(M,x) is a cone. Let u,v € Tc(M,x) and consider (tn) 4 0 and (xn) ~>M x. Then there exists (un) > u such that x'n := xn + tnun G M for every n E N. Of course (xJJ —>• 3;. Hence there exists (vn) —> u such that £n + in(w„ + u„) = x^ + i n w n G M for every n. Therefore u + v E Tc(M,x). Consider now (uk)ke^ C Tc(M,x) with (ufc) + u E X. Let us show that u E Tc{M,x). For this take (tn) I 0 and (xn)  > M £• Because ufc E Tc{M,x), there exists (u*) >• ufc such that xn + tnukn E M for every n G N. Hence xn+tnukn E M for all k, n E N. For every n EN there exists k'n E N such that u* — uk\\ < n~l for every k > k'n. Let (fc„) C N be an increasing sequence such that kn > k'n; then u*  uk\\ < n _ 1 for all k,n E N with k > A;„. Let u„ := u* n . Of course, £„ + £ n u„ = xn + tnuknn G M for every n. As M„  u = \\unn  u < u*n  ukn  + \\ukn  u\\ < n _ 1 + u*n  u, we have that (un) >• u. Hence u G Tc(M,x). Therefore Tc(M,x) is a closed convex cone. (ii) Let u E Tc(M,x) and take (tn) I 0 and (xn)  > C I M £• For every n G N there exists xn E M such that \\xn — xn\\ < tn/n, i.e. xn = xn + tnn~1u'n with u'n E Ux Hence (xn) >M x. Therefore there exists («„) > u with xn + tnun = xn + tn(n~lu'n + un) G M C clM for every n E N. As (n _ 1 u^ + u„) —> u, we have that u G I b ( c l M , x ) . Conversely, let u G Tc(c\M,x) and take (tn) I 0 and (z n ) tM x. There exists (un) > u with xn + tnun G cl M for every n. Like above, for every n E N there exists u^ G Ux such that a;„ + tnun + tnn~lu'n G M . Because ( u „ + n _ 1 u ^ ) —> ii, we have that w E TQ^M^ X). Hence TQ^M, X) — Tc(dM,x).
Convexity and monotonicity
of
subdifferentials
171
The equality Tc{M,x) = 7 b ( M n V,x) is obvious when V is a neighborhood of x. (hi) Let M be convex. Taking into consideration (ii) we may assume that M is closed. Consider x E M and take (£„) 4 0 and (xn) —»M X. There exists no € N such that tn < 1/2 for n > no. Take un := x + x  2xn for n > n0 and un = 0 otherwise. Of course, (un) ¥ x — x. As xn + tnun = (1 — 2tn)xn + tnx + tnx S M for n > no, we have that xx € Tc{M,x). It follows that C(M,x) C T c ( M , x ) . Using Eq. (3.11) we obtain the conclusion. D From (ii) and Eq. (3.11) we obtain that Tc(M,x)
Cn '
C(MHV,x)
=:
TB{M,x);
'V'gJV(x)
is the well known tangent cone in the sense of Bouligand of M atxed M. Let now / : X —> M. be a proper function and x 6 dom / . It is natural to consider the tangent cone Tc (epi/, (x, f(x))). This cone is related to the Clarke—Rockafellar directional derivative of / at x introduced as follows: TB{M,X)
,*,_ . f'{x,u):=sup
, limsup e>0 tiQ,(x,a)>epi,(x,f(x))
. f := sup ml
. „ mi
fix + tv) — a
ll««ll<e
*
sup
e>0 *>0 0
f(x +
ml
f(x)
II"'"ll<
£
tv)a
.
*
It follows that , ,_ , . , / ' t( a ; , w ) = s u p inf
sup
f{x + tv) 
mi
lk"ll<E
£ > 0 <5>0 o
f(x)
,
*
and even /t(x,u)=sup
limsup
inf
e>0 (4.0, x>/5 ll««ll<e
/ (
*+
tV
} ~~f{x)
(3.12)
'
if / is lower semicontinuous at x, where x —>/ x~ means x —>• x and f(x) —>• f(x). It is obvious that / t ( x , 0) < 0 and /''"(xjAu) = A/ 1 '(x, u) for A > 0 and u G X. We may have f^(x, 0) = — oo even if / is continuous at x. Take for example / : R »• E, f(x) := — \/\x\, and x = 0 (Exercise!). The next result shows the connection between the preceding two notions.
172
Some results and applications of convex analysis in normed spaces
Proposition 3.2.2 Let f : X —> R be a proper function and x G d o m / . Then epi ft(x, •) = Tc (epi / , (x, f(x))). In particular ft(x, •) is a Isc convex function; ft(x, •) is a Isc sublinear function if and only if ft(x, 0) = 0. Moreover, if f is convex then epi ft(x, •) = cl (epi f'(x, •)). Proof. Let (u,A) G Tc (epi f, (x, f(x))); assume that ft(x,u) > A, and take ft(x, u) > A' > A. Then there exist EQ > 0, ((xn,an)) »epi/ (x,f(x)) and (tn) 4 0 such that inf„ 6 c( U)£o ) t~* (f(xn + tnv) — an) > A'. Because (u, A) 6 Tc (epi f, (x, f(x))), there exists the sequence ((un,Xn)) converging to (u, A) such that (xn,an) + tn(un,\n) G e p i / , i.e. f(xn + tnun) < &n + tnXn for all n G N. Since (un) » u, there exists no such that un G D(u,eo) for n > no Hence
v^
r
f(xn + tnv)an
f(xn +
tnun)an
A <
inf — < — < A„ Vn > n 0 . w—u<e0 tn tn Taking the limit we get the contradiction A' < A. Let now f^(x~,u) < A and take ((xn,an)) —>epif (x,f(x)) and (tn) I 0. Let us fix (ek) I 0. Because • infr
5>0
• inft
sup 0
\\xx\\<S, \af(x)\<S
f(x + tv)  a —
\\vu\\<ek
t
for every k G N, there exists 8k > 0 such that . inf
sup 0«<<5fc,xx<(5 fc ,/(x)
u
f(x + tv) a — £
ll» ll< *
. < A.
*
There exists n'k G N such that 0 < t„ < 8k, \\xn — x\\ < 8k, \an — f(x)\ < 8k for all n > n'k. Therefore inf„€£>(U)efc) t" 1 (f(xn + tnv)  an) < A, which shows that for every k and every n>n'k there exists u* 6 D(u, e^) such that f(xn + tnUn) < a „ + Xtn. Consider an increasing sequence (n^) C N such that nk > n'k for every k G N. Taking un := u* for n^
C epi f'(x, •) C cone(epi/  (x,
f(x))),
Convexity and monotonicity
we have that epip(x,
of
subdifferentials
•) = C (epi/, (x,f(x)))
173
= cl(epi/'(:r, •))•
D
+
Note that when / is not lsc at x, f^(x, •) and / (x, •) may be different, where, as usual, / is the lower semicontinuous envelope of / . Take for example / : R >• R, f(x) = 0 for x ^ 0, /(0) = 1; / t ( 0 , •) =  c o , but / (0, •) = 0. When / is lsc at x then f*(x, •) and / (x, •) coincide. When / has additional properties, Z1" has a simpler expression. Proposition 3.2.3 Let f : X —> R be a proper function and x 6 d o m / . Then for every u € X we have: (i) limsupj.^. xP(x,u) ,lv N / f'(x,u)<sup
< f^(x,u);
• r inf
moreover, if f is lsc atx • inff
sup
f(x + tv)
£>0 S>0 j^—^<<S,0
< inf
then
f(x)
*
sup
.
*>° \\zx\\<S,0
(3.13)
*
(ii) / / / is continuous at x then
,iv x f'{x,u)—
•r
. t f{y + tv)  f(x)
sup inf
sup
mi
e>0 <5>0 \\xx\\<S,0
.
ll"ull<e
*
(hi) If f is finite and LLipschitz on B(x,r) for some r > 0 and L > 0, then ttr\ • f(x + tu)  / ( : r ) /'(a;,u) = inf( sup s>0
xx<(5,0
t
f(x + tu) — f(x) , ,. .. = limsup^——'— J ±L < L \\u\\. (iv) If f is finite and Gateaux differentiate tinuous at x then f^(x, •) = V/(3;).
on B(x,r),
(3.14)
and V / is con
Proof. Throughout the proof u € X is a fixed element. (i) Let f*(x, u) < X and consider e > 0; there exists 6 > 0 such that sup \\xx\\<8,0
. inf lk«ll<
f(x + tv)f(x) e
^ < A.
*
Take 6' := 6/2. Let x' e £(x,<5') with \f{x')  f(x)\ < 6' and x, t be such that \\x  x' <6',0
174
Some results and applications of convex analysis in normed spaces
0 < t < 8 and f(x) < f(x) + 8. It follows that sup {mt^D^r1
(f{x + tv)  f(x))
 t G ]0, 8'}, x e D{x', 8'), f(x)
l
< sup {miv€D(u>e)r
(f(x + tv)  f{x)) \te]0,8],
x £
D(x,8),
f(x) < f(x) + 5} < A for all x' e B(x,8') with \f(x')  f(x)\ < 8'. Therefore f^(x',u) < A for such x', and so lizn supx^. w f^(x,u) < f^(x,u). The first inequality in Eq. (3.13) follows immediately from Eq. (3.12), while the second is obvious. (ii) Taking into account Eq. (3.13), one must show the inequality >. For this take e > 0 and 8 > 0. There exists 8' € ]0,8] such that f(x) < f(x) + 8 for every x € D(x,8'). Then sup 0<£<<5 \\x~x\\
inf
^
H
lluull^e
M
>
sup
t
inf u
0
^
e
+
^ ~ ^ , t
whence the conclusion follows. (iii) The inequality < is proved in Eq. (3.13). Assume that / is Lipschitz on B(x,r) with Lipschitz constant L. Take A > lim sup x _,. s ^ 0 Hx+tu)~Hx) and fix e > 0. Consider 80 > 0 such that sup ;c _j< 5o)0
<
f(x + tv)  f(x + tu)
f{x +
tu)f{x)
+ tU
] ~ fix) < / ( * + tU] ~ ™ + Le.
Hence sup \\xx\\<5 0
. , inf ll"«ll<£
fix + tv)  fix) ^ '—i±± < t
sup \]xx\\<6 0
fix + tu)  i fix) — —  ^  L + Le, t
Convexity and monotonicity
of
subdifferentials
175
and so . mi
sup
inf
*>° ii<<5,0
<
£
sup
/(a; + fa)  / ( a ; ) — t J fix + tu)J  L f(x) ^——— ^+Le<\
a:x<<So,0«<<5o
+ Le.
*
Taking the limit for e > 0, we obtain \ ^> rhm inf t A
sup
e40 d>0 \\xx\\<6,0
f(x + tv)f(x) ^—L = ttf—f ] { x , u\) .
t inf ll"«ll<e
t
Taking into account (ii), we get Eq. (3.14). Taking 5 = r / ( l + w), for x € B(x,S) and t £]0,S] we have that x + tu,x € B(x,r), and so f{x + tu) — f(x) < tL \\u\\, whence f^(x,u) < L \\u\\. (iv) Let r > 0 be such that / is Gateaux differentiable on B(x, r). Since V / is continuous at x, V / is bounded on a neighborhood of x. So we may assume that V/(x) < L for x € B(x,r). Using the meanvalue theorem we obtain that / is iLipschitz on B(x,r), and so Eq. (3.14) holds. Let (tn) i 0 and (xn) »• x be such that f^(x,u) = \imt~1 (f(xn + tnu) — f(xn)). But xn+tnu, xn £ 5(3;, r) for n > no (for some no £ N); applying the mean value theorem, there exists 6n £]0,i„[ such that f(xn + tnu) — f(xn) — V / ( x n + 6nu)(tnu) for every n > no Because V / is continuous at x, we obtain that ft(x,u) = Vf(x)(u). O Let us introduce now the Clarke subdifFerential of / : X —> K at xeX with f(x) £ E. This is dcf(x)
= {x* € X*  {u,x*) < f\x,u)
Vu € X } ;
if f(x) £ E we consider that dcf(x) = 0. Taking into account Theorem 2.4.14, dcf(x) ^ 0 if and only if /^(af, 0) = 0. Moreover, dcf(x) is a nonempty w*compact subset of X* if / is Lipschitz on a neighborhood of x. Other properties of dc are collected in the following result. Theorem 3.2.4 domg.
Let f,g : X —> R be proper functions and x £ d o m / n
(i) If / and g coincide on a neighborhood ofx then dcf(x) (ii) If f is convex then dcf(x) (iii)
=
df(x).
If x is a local minimum of f then 0 €
dcf(x).
— dcg(x).
176
Some results and applications of convex analysis in normed spaces
(iv) \imsupx_>x dcf(x)
C dcf(x),
where kmsupx^
dcf(x)
is the set
{x* G X*  3 (xn) +f x, 3 « ) % x*, Vn G N : < G 3 c / ( a ; n ) } • (3.15) (v) If g is finite and Lipschitz on a neighborhood of x then
(f + g)Hx, •) < fHx, •) + g\x, •), dcU + g)(?) c dcf(x) + dcg(x). (vi) / / g is Gateaux differentiate tinuous at x then
on a neighborhood of x and Vg is con
(f + g)H*, •) = fHx, •) + V5(af),
dc(f
+ g)(x) = dcf(x)
+ Vg(x).
Proof, (i) and (iii) are obvious because p{x, •) = (flv^ix, •) for every V G N(x), while (ii) follows from the second part of Proposition 3.2.2. (iv) Let x* G limsupx^ fXdcf(y); t n e n there exist (xn) >/ x and (a£) ^ x* such that x*n G dcf(xn) for all n G N. Let M £ I be fixed. Then (u,a;*) < p{xn,u) for every n G N, whence, by Proposition 3.2.3 (i), (u,x*) < ft(x,u), and so x* G dcf(x). (v) Let r > 0 be such that g is LLipschitz on B(x, r); we may assume that L > 1. Let u £ X, e > 0 and 5 > 0 be fixed and take 6' = mm{S/{2L),r/(l + e + \\u\\)} > 0. Let v, x and t be such that \\vu\\ < e , \\xx\\ <5',0
+ g)(x) ~
f{x + tv)  f{x) t
g{x + tu)  g{x) t
whence, taking first the supremum with respect to v G D(u,e), we obtain that a < Ai(5) + A2(5) + Le, where a :=
sup a!s
.
mi H"«ll<£
(f + g)(x + tv)(f
+ g)(x)
t
U+9)(x)<(f+gm+s' Ai(S)
:=
sup \\xx\\<S,0
inf — Il««II<£
A2(d) := sup {t'1 (g(x + tu)  g{x)) \x
eD(x,5)}.
—^L, t
,
Convexity and monotonicity
of
subdifferentials
177
Therefore /3 < Ai(S) + A2{5) + Le for every 5 > 0, where /3:=inf 5 >0 '
sup a!»<
inf if + 9)(x+ tv)  jf + g)(X) ^ £ Il "II< * u
(f+9)(x)<(f+g)(x)+S'
Taking the limit of Ai (S) + A2 (5) for S —> 0 (taking into account that A\ and A2 are nondecreasing for 5 > 0), we get inf
sup
inf
(/ + g)(* + * 0  ( / + g)(*)
5
>°\\xx\\<5,0
•
f
< inf s>0
t
sup \\xx\\<6,0
X
f(
f
inf ll""ll<
—£
+
tv
)
~
/(*)

—L
*
. 9(x + tu) — g(x) + inf sup — '—^^ + Le. <5 >°xx<(5 t
Taking now the limit for e —> 0, we get the desired conclusion. The relation for the subdifferentials follows from the preceding inequality and the fact that g^(x, •) is continuous. (vi) As mentioned in the proof of Proposition 3.2.3 (iv), g is Lipschitz on a neighborhood of x and so the conclusion of (iv) holds with g^(x, •) = Vg(x). Applying again (iv) for f + g and — g (we may assume that g is finite on X; otherwise take g = 0 outside a neighborhood of x), we obtain that f*(x, •) < (f + g)^(x, •) — Vg(x). The conclusion is now obvious. • In the rest of this section we use an abstract subdifferential. Before introducing this notion let us consider the following class of finitevalued convex functions: C(X) := {g : X > E  g is convex and Lipschitz}; of course, €(X) is a convex cone in the vector space Rx of all functions from X into E. Let d ( X ) C E x be the cone generated by X*U{d [a , 6]  a, b € X) (C €{X)) and €2{X) C E x be the cone generated by
X*u{dfaM\a,beX}WD0, where
Do := { ^2k>QVkdlk
(Hk) C K+, ^2k>Ql*k
These cones will be used below.
=
*' (u*)*>°
conver
Sent} •
178
Some results and applications of convex analysis in normed spaces
We call an abstract subdifferential on the nonempty class T C M. a multifunction d : X x T =4 X*, which associates to (x,f) a set denoted by df(x), satisfying condition (PI) below: (PI) 0 G limsup y _> x d/(y) + dg(x) if / € T, g € <£(X), f(x) £ 1 and a; is a local minimum of / + g, where limsup _yX df{y) is defined as in Eq. (3.15) and dg(x) is the Fenchel subdifferential of g at x. A stronger form of (PI) is (P2) 0 € ~8f(x) + dg(x) if / 6 T, g e €(X), f(x) 6 1 and a; is a local minimum of / + g. Sometimes one asks (P3) df(x) = df(x) if / e
Tn€(X).
Of course, (P2) holds if (P4) and (P5) below are satisfied: (P4) 0 6 df (x) if / 6 J, f(x) € E and x is a local minimum of / , (P5) T + €(X) C T and 8{f + g)(x) C df(x) + dg(x) when / G T and 9 e €(X). Note that WX + €(X) = RX,
A(X) + €(X) = A(X),
T(X) + €(X) = T(X).
Remark 3.2.1 From the preceding theorem we observe that Clarke's subdifferential dn and the Fenchel subdifferential d are abstract subdifferentials x on R and A(X), respectively; in fact they satisfy conditions (P1)(P5). There are many other subdifferentials which satisfy condition (PI). Remark 3.2.2 If the abstract subdifferential d on T satisfies the stronger condition (P2) then for any proper function / 6 T one has df(x) C df(x) for all x € d o m / . Indeed, if x* 6 df(x) then a; is a (local) minimum point of / + (—x*), and so, by (P2), 0 £ df(x) + d(x*)(x) = df(x)  x*. Hence x* € df(x). The following approximate mean value theorem proves to be useful in nonconvex analysis.
Convexity and monotonicity
of
subdifferentials
179
Theorem 3.2.5
(Zagrodny) Let (X, ) be a Banach space and d be x an abstract subdifferential on T C E . Let f 6 T be Isc, a,b G X with a G d o m / and a ^ b, and r 6 l with r < f(b). Then there exist (xn) —>•/ c € [a, b[ and x*n G df(xn) for every n G N such that (i) r — f(a) < lim inf (6  a, x*n), (ii) 0 < lim inf (c (iii)
xn,x*n),
f^(r/(a))
(iv) &  a (/(c)  /(a)) < c  a (r  / ( a ) ) . Proof. There exists x* G X"* such that (6  a, x*) = r — f(a). Consider h := / — x*; then /i(a) < h(b). Because h is Isc, there exists c G [a,b[ such that h(c) < h(x) for all a; G [a,b]. Therefore c = (1 — /x)a + fib for some /i G [0,1[. It follows that c — a = fi(b — a) and c  a = fi \\b — a\\. Therefore /(c)  f(a) = h(c)  h(a) + (c — a,x*) < fi(r  / ( a ) ) , whence (iv) follows. Let 7 < h(c); then there exists r > 0 such that 7 < h(x) for every x G [a, b]+rUx Otherwise, for every n G N there exists xn G [a, b ] + n  1 { / x such that h(xn) < 7. Hence arn = dn + yn with d„ G [a, 6] and y n G n~1UxAs the segment [a,b] is a compact set, there exists a subsequence (dnk) converging to d G [a,b]. It follows that (xnh) —> d, and so, by the lower semicontinuity of h, we get the contradiction h(d) < 7. Let r > 0 correspond to 7 := h(c) — 1, and take C/ := [a, b] + rUx', of course, U is closed. Like above, for any n G N there exists r„ G ]0, r[ such that /i(z) > h(c) — n~2 for 3; G [a,b] + rnUx', choose tn > n such that 7 + tnrn > h(c) — n~2. Then one has Vz G U : /i(c) < h(x) + tnd[atb](x)
+ n~2.
(3.16)
Indeed, the inequality is obvious for x G [a, b] + rnUx If x £ U \ ([a, b] + r„[/x) then d[0i6](x) > r„, and so /i(x) + t„d[a)ft](a;) > 7 + i n r n > ft.(c)n2. Consider i?„ — h + iu + *nd[0)(,]. Prom Eq. (3.16) we have that Hn(c) < infx Hn + n  2 ; moreover, Jf„ is Isc and bounded from below. Applying Corollary 1.4.2 for Hn, c, e ~ n~2 and A := n _ 1 , we get un G X such that Hn(un)
< Hn(c),
cu„ < n_1,
(317)
1
(3.18)
Hn{un) < Hn(x) + n  z  u n 
Va; G X.
Since (un) —> c G [a, 6[, we may assume that un G intf/ for every n.
180
Some results and applications of convex analysis in normed spaces
Relation (3.18) can be written as ( / + tnd[atb] + n~ldUn  x*) (u„) < ( / + tnd[aib] + n  1 d „ „  x*) (x) for every x 6 U, where da denotes d[a,a] = ^{o} Therefore un is a local minimum of/ + (tnd[a,b] +n~1dUn x*); the function between the parentheses being convex and Lipschitz, by (PI) we have that 0 6 limsup <9/(2/) + d(tnd[a
nHl).
Let yn £ [a, b] be such that d{a^{un) = \\un — yn\\ (such an element exists because [a, b] is compact). Because 0, and so (yn) > c. Since c ^ b, we may suppose that yn ^ b for every n 6 N. Hence yn = (1A n )a+A„6 with An € [0,1[. From Proposition 3.8.3(H) on page 239, dd[a,t](u„) = d (u n y n )r\N([a,b],y n ). Therefore ((1  A)o + \b yn,v*n) = (A  A„) (6  a , < ) < 0 for every A € [0,1[. Hence (b — a, u*) < 0 for every n S N. Moreover, (c  u n , u*) = (c  y n ,v*)  (u„  yn,v*) It follows that (ba,u*n) n_11& — a\\, whence
> (ba,x*)
<  u n  y„ < 0.
 ^(fra,^)
> r  f(a)

liminf (b — a,u*) > r — / ( a ) . Similarly, {c — un,u*n) > (c — u„,x*) — n _ 1   c  u n   , whence liminf (c — un,u*n) > 0. From Eq. (3.17) we get f(un)  (un,x*) < /(c)  {c,x*), and so /(c) < liminf f(un) < l i m s u p / ( u n ) < /(c); hence l i m / ( u n ) = /(c) which shows that (un) —>f c. Because u*k e l i m s u p . ^ ^ df(u), for every k £ N there exist (u„,*) >•/ Ufc and (u*^) ^> u£ (for n » oo) such that u*nk G df(un>k)
for all n, A; e N.
Convexity and monotonicity
of
subdifferentials
181
It follows that ((c  uniu,u*nk)) > ( c  Ufc,u£) for n > oo. Therefore for every A: € N there exists n'fc € N such that IK,*Wfc < fc_1, \f(un,k)(cunik,u*nik)
f(uk)\ < k'1,
\(ba,u*nk
(cuk,u*k)\
 u*k)\ < AT1,
< k~l
for n > n'k. Consider (nk) C N an increasing sequence such that nk > n'k for all k e N. Let xn := u„,fc and a;* := u^ fc for nk < n < nk+i Since for nk
~ Uk\\ + \\uk  C\\ < k~l
+ \\uk  C\\ ,
l / W  Z t o l ^ + l/toWtol, (b  a, < ) = {ba, u*k) + {ba, < < f c  u*k) > (b  a, u*k) (cxn,x*n)
= (cu n , f c ,u* ) A .) > (cuk,u*k)
k'1
 fc1,
we obtain that (xn) —•/ c, lim inf (6 — a, xn) > lim inf (b — a, u£) >r — f(a) n—>oo
k—•oo
and lim inf (c — x n ,a;*) > lim inf (c — Ufc,u^) > 0. n—soo
fc+oo
Taking into account that b — c = (1 — /j,)(b — a) and \\b — c = (1 — /i) b — o, from the preceding two relations we get lim inf (b xn,x*n) = lim inf ((1  fi) (b a, a:*) + {c
xn,x*n))
> (1 — /x) lim inf (b — a,xn) + lim inf (c — a;„,a;*) > K The proof is complete.
(r  / ( a ) ) . •
Remark 3.2.3 The conclusion of Theorem 3.2.5 remains valid if (PI) holds only for functions g in the cone £i(X) or in the cone C2PO defined on page 177. Indeed, for the first part just note that in the proof of Theorem 3.2.5 we used (PI) only for g = i„d[aj6j + n~1dUn — x* € £i(X). For the second part one must notice that one can find tn > 0 such that 7 + tnrn > h(c), and so Eq. (3.16) holds with d[Q>(,] replaced by d?a bi. Applying the BorweinPreiss variational principle for p = 2, Hn,
182
Some results and applications of convex analysis in normed spaces
c, e := 2n~2 and A := \/2/n, we find un € B(c, y/2/n) and a function 0 2 ,„ generated by the sequences (fJ.n,k)k>o C K+ with X^>0Mn,ib = 1 and (wn,fc)fc>o C B(c, \/2) (see Eq. (1.11) on page 31) such that un is a local minimum of / + (tndfa M + n~1@2,n — x*). Then, in the expression of un we have tn \\un — yn  instead of tn and 6* 6 23/2Ux* instead of &£ 6 C/x* (see also Exercise 2.40). The rest of the proof is the same. Corollary 3.2.6
Let X be a Banach space and d be an abstract subx differential on T C M. . If f G T is a Isc proper function then for every x e d o m / there exists ((xn,xn)) C gr<9/ such that (xn) —»/ x and liminf (x — xn,x^) > 0. In particular d o m / C cl(domd/). Proof. Let x S d o m / . If z is a local minimum of / then 0 belongs to limsupy^ xdf(y) by (P2) (just take g = 0). Therefore there exists (xn) —>f x and (a;*) —> 0 such that a:* 6 df(xn) for every n € N. Of course, lim (x — xn, x^) = 0 in this case, and so the conclusion holds. Assume that x is not a local minimum of / . Then there exists (xn) —> x such that f(xn) < f{x) for every n £ N. Since / is lsc, we get (xn) »•/ x. Applying the preceding theorem for xn, x and r = f(x), there exists ((xk,n,x%tn))keN C domdf such that (xk,n) >/ yn G [x„,a;[ for k » oo and 0
< jxlj (/W
_
/(^«)) < Hminf *,_>«, (a;  a;fc>„,a;^„),
/(l/n) < /(*„) +  ^ f (/(*)  /(*»)) < /(*) for every n € N. Since \\yn  x\\ < \\xn  x\\ we have that (yn) > x, which, together with the preceding inequality and the lower semicontinuity of / , yields (yn) >•/ x. Then, for every n 6 N there exists k'n € N such that lk*,n  Vn\\ < n _ 1 , /(a:fc,„)  f(yn)\ < n _ 1 and 0 < (x  Xk,n,x*kn) f o r every k > k'n. Let (fc„) C N be an increasing sequence such that kn > k'n for n € N. Then lkfc„,n  z < a;*„,n  y n  + \\yn  x\\ < n _ 1 + \\yn  x\\, l/(**»,»)  f(x)\
< \f{xkn,n)
 f(yn)\
+ \f(Vn) 
f(x)\
< n  1 + /(y„)  /(a:) , 0 < (zZ*„,„,4„,n) for all n £ N. Taking xn := Xkn,n and a;* := a;^ n , we have the desired conclusion. •
Convexity and monotonicity
of
subdifferentials
183
When T = A(X) and d is the Fenchel subdifferential, the statement of the preceding corollary is very close to assertion (i) of Theorem 3.1.2. Theorem 3.2.7 X
Let X be a Banach space, d be an abstract subdifferential
on T C K and f £ T be a Isc proper function. If df is a monotone multifunction then f is convex. Proof. We prove the theorem in three steps: 1) a £ d o m / , b £ domdf => [a,b] C d o m / , 2) g r d / C gr<9/, 3) / is convex. 1) Assume that there exists b' £ [a, b] \ dom / ; so b' — Xa + (1 — X)b with A £ ]0,1[. Fix y* £ df(b) and take r £ ffi, r > / ( a ) + A1(lA)6a.2/*. By Theorem 3.2.5, there exist (xn)  » / c £ [a, b'[ and x* € df(xn) n € N such that liminf (&' — a;n,a;*) > 0,
for every
liminf (b' — a, a;*) > r — f(a).
Since df is monotone we get the contradiction l6ay*ll>ll&cy*><6c)i/*> = liminf (b — xn,y*) > liminf (b — xn,x*n)  liminf (A(l  A)" 1 (b'  a,x*n) + (b'  xn, x*n)) > A(l  A)  1 liminf ( 6  a , x * ) + liminf (b' >
A ( 1
_
A )
i
( r
_
/ ( a ) )
xn,x*n)
.
2) Let x* 6 df{b) with b 6 X. Consider x £ d o m / . Applying again Theorem 3.2.5 for r = f(b), there exist (xn) >f c € [x, b[ and x* € df(xn) for every n 6 N such that liminf (bxn,x*n)
> g f f } (/(6)  / ( * ) ) .
Since 9 / is monotone and c = Ax + (1 — A)6 with A € ]0,1], we have that A (6 — x,x*) = (b — c,x*) = liminf (& — x n ,x*) > liminf (b — x n ,a;*) >A(/(6)/(x)), and so (x — b, x*) < f(x) — f(b) for every x £ d o m / . Therefore x* £ df(x). 3) Let x, y £ dom / and z := (1 — X)x + Xy with A £ ]0,1[. By Corollary 3.2.6, there exists (yn) c dom df such that (yn) >•/ y. Let a;n := (1A)x + Ay„ for n £ N; by 1) we have that zn £ d o m / for every n. Using again
184
Some results and applications of convex analysis in normed spaces
Corollary 3.2.6, there exists ((zn,k,zn k)) c STOf s u c r i t n a t (zn,k) —>•/ zn for k —> oo and liminffc_>oo(^n — zn,kiznk) — 0. By 2) we have that z*nk G df(zn>k), and so (xzntk,z^k)+f(zn,k)
< f(x),
(yn
zn,k,znk)
+ f{zn>k) < f(yn)
for all n,k G N. Multiplying the first inequality by (1 — A) > 0 and the second one by A > 0 we get (zn  zn,k, z*n>k) + f(zn,k)
< (1  A)/(x) + Xf(yn)
Vn, k G N.
Taking the limit inferior for k —> oo we get f(zn) < (1 — A)/(x) + Xf(yn) for every n. Passing to the limit inferior for n —> oo and taking into account the lower semicontinuity of / and the fact that (yn) >/ y we get f(z) < (1 — A)/(a;) + Xf(y), and so / is convex. • We give now another short proof for the Rockafellar theorem on the maximal monotonicity of the subdifferential of a lsc convex function. Theorem 3.2.8 Let X be a Banach space and f € T(X). maximal monotone.
Then df is
Proof. Let x G X and x* ^ df(x). Then x is not a minimum point for g := f — x*, and so there exists z E X such that g{z) < g(x). Let r G E be between these numbers. By Theorem 3.2.5 applied for d on r(X), there exists ((yn,yn)) c Sr9g such that (yn) —>fl y € [z,x[ and lim inf (x  yn,y*n) >  f 5  [ (r  g{z)) > 0. Therefore (x  yn,y*n) > 0 for n > no, for some n 0 G N. It follows that y* := yno + x* € df(y), with V '•= 2/n0) and (x — y,x* — y*) < 0. Therefore df is maximal monotone. • Theorem 3.2.5 can also be used for proving that two convex functions with the same subdifferential differ by an additive constant. In fact one obtains even more. Theorem 3.2.9
Let C be an open convex subset of the Banach space X jf
and d an abstract subdifferential on T C IK . Let f G T be proper and lower semicontinuous and g G T(X). Assume there exists e G R+ such that df(x) C dg{x) + eUx*
V i G C.
(3.19)
Then C D dom / = C f) dom g and 9{x)  g{y) ~e\\x
y\\ < f(x)  f(y) < g(x)  g(y) +e\\x~
y\\
(3.20)
Convexity and monotonicity
of
subdifferentials
185
for all x £ C and y £ C fl domg. Proof. Consider 7 > e. By Corollary 3.2.6 we have that d o m / C cl(dom9/), and so C fl domdf ^ 0. Let y £ C fl d o m 9 / and x £ C. Prom our assumption we have that y £ doming, and so y € domg. We are going to prove that /(*)  /(!/) < $(*)  5(2/) + e II*  2/11 •
(321)
The inequality is obvious if x £ domg or 1 = y, and so assume that x £ domg and x ^ y. Because dg(y) ^ 0, we have that g'(y,u) >  0 0 for all u £ X. Consider ip : E »E defined by
g'(y,u),
and this quantity is finite. Therefore there exists to £ ]0, :r — y[ such that g'(y + t0u,u)
9 g'(x0,u) < 9{V + *>">to  9{V) +  ( 7  e ) = g
f(x0)f(y)
(3.22)
Assume that Eq. (3.22) does not hold, i.e.
f(xo)  f(y) > 9{x0)  g(y) + 7 Iko  y\\ • Choose r € E such that r < f(xo) and r  f(y) > g(x0)  g(y) + 7 *o  y\\ • By Theorem 3.2.5, there exist (xn) >f z £ every n £ N such that
\I),XQ\
and x*n £ df(xn)
liminf (a?o x„,a;*) > —  — — (r  /(j/)).
(3.23) for
186
Some results and applications of convex analysis in normed spaces
Because (xn) —> z we have that (a;o — x n ) —> xo — z, and so
]xoxn\\
I
\\XQy\\
Taking into account Eq. (3.23), there exists no G N such that *o*»  * \ Foa;n /
>
g(»°)gy M^cro — 2/11
vn>n0.
+ 7
Since ( x „ ) —>• z G [y, xo[ C C, we m a y a s s u m e t h a t xn G C for n > no B y E q . (3.19) we have t h a t xn = zn + eu*n with z* G dg(xn) a n d u* G t / x * for n > no Using also t h e convexity of g, it follows t h a t g(x0)g(xn) ^ / so^gn ar0 — a;„
\ z 0  z„l
A
>
9(xo)9(y)
/
+ 7
_
£
V n
^no_
ar0 — 3/1
Taking into account the lower semicontinuity of g at z and Eq. (3.19) we get g{xo)
g(z)
g(x0)  g(y)
,
,
IN2/H
IFO^II
Since 2 G [y, io[, z = y + su for some s G [0, io[ Using the convexity of
—ii
rr = — :
,
,
— <
(x0,u).
\\x0  z\\ t0  s Therefore Eq. (3.22) holds. Consider T~{te
]0, rc  y] I f(y + tu)  f(y) < g(y + tu)  g(y) + jt} .
From Eq. (3.22) we have that t0 G T, and so t := supT G]0, a;  y]. Because
_ , , > * & • « ) = V (<) > V (0) = ff+(»,0),
whence g'(y,u) G 1R. Moreover, y G d o m / D domg. Proceeding as above with y replaced by y (noting that we used only that y G dom / n dom g and g'(y,u) G R), we obtain some s 0 G]0, a;  y\\] such that f(y + s0u)  f(y) < g(y + s0u)  g(y) + ys0.
Convexity and monotonicity
of
subdifferentials
187
As f{y) — f(y) < g(y) — g(y) + 7*, we obtain by addition that f(y +(t + s0)u)  f{y) < g(y + (t + s0)u)  g{y) + 7 ( I + s 0 ), contradicting our choice of t. Hence t = \\x — y\\, and so f(x) — f(y) < g(x) — g(y) + 7 \\x — y\\. Because 7 > e is arbitrary, we obtain that Eq. (3.21) holds. Taking i G C f l dom g and a fixed y G C n dom df, from Eq. (3.21) we obtain that C n dom g c C f l d o m / . Let now j / e C f l d o m / . From Corollary 3.2.6 we get a sequence (yn) C dom df with (yn) —>/ y. Because y G C, we may assume that yn G C for every n € N. Let i G C n dom g be fixed. From Eq. (3.21) we have that
f{x)  f(yn) < g(x)  g(yn) + e\\xyn\\
Vn € N.
Since g is lsc at y, we get fix)  f(y) < g(x)  g(y) + e\\x
y\\,
(3.24)
and so y £ dom g. Hence C (~1 dom / = C fl dom g. Let x, y € C fl dom / = C fl dom g; then Eq. (3.24) holds. Interchanging x and 1/ in Eq. (3.24), we obtain also that
f{x)  f{y) > g(x)  g{y)  e \\x  y\\, which proves that Eq. (3.20) holds for all x,y G C f] d o m / . Equation (3.20) being obvious for x & C \ dom / = C\ dom g and y £ Cr\ dom g, the conclusion follows. D A more precise result is the following. Corollary 3.2.10 Let X be a Banach space and d an abstract subdifferential on T C K. . Consider a proper and lower semicontinuous function / € f and g € T(X). If df(x) C dg(x) for every x G X then there exists S;£R such that f = g + k. Proof. Take C — X and e ~ 0 in Theorem 3.2.9. Fixing x0 G d o m / = dom g, we obtain that f(x) = g(x) + f(xo) — g(xo) for every x G X. • Corollary 3.2.11 Let X be a Banach space and f,g€ dg(x) for every x G X then f = g + k for some k €.R.
T(X). Ifdf(x)
C
Proof. Apply the preceding corollary on F(X) and the Fenchel subdifferential. •
188
Some results and applications of convex analysis in normed
spaces
The next result completes Proposition 2.4.3. Corollary 3.2.12 Let X be a Banach space and T : X =4 X* be a maximal cyclically monotone multifunction. Then there exists f G T(X), unique up to an additive constant, such that T = df. Proof. Because T is maximal monotone, g r T ^ 0. Fix (XO,XQ) G g r T and consider fr defined by (2.37) on page 85. Then, by Proposition 2.4.3 we have that fr G T(X) and g r T C grdfr Since T is maximal monotone we even have g r T = grdfr, and so T = df. The uniqueness follows from Corollary 3.2.11. • 3.3
Some Classes of Functions of a Real Variable and Differentiability of Convex Functions
Consider the following useful classes of functions: A = {
1 + 
A0
Nk = {ip G A  P 3 11> t~kip(t) € 1 + is nondecreasing}, Jfc G {0,1,2},
= {<^ G yi  iimmrkip(t) = o}, ke {o, i},
r
= {ip G .A 
To Si
= r n A), r^ := y G r 0 1 iiminft_+0OtV(*) > o}, = r n n i , s? ~ {^eEi liimsup^^rVC*) <°o}.
Of course, N2 C Ni C N0, E? C Si C fti C ft0 and Tg C T 0 C T C A^. Moreover, if
0 then
(s) := inf{t > 0  y>(t) > s } ,
0 
0 for s > 0, and so we get a function
(3.25)
it is obvious that p e < / , i/ie £ iVo,
Differentiability
of convex
functions
189
Furthermore, one can extend
i(i) := a\t\ with a > 0,
(i) Ifip£A0r\N0
then tpe,tph £ Cl0; if(p£Cl0r\N0
then
(ii) Let
i> £ r 0 <s> v # e Si and v e rg o v # e sf • (iv) Let p,q £ ]1, oo[ be such that 1/p + 1/q = 1 and
t~p(f(t) is nondecreasing (nonincreasing) on P then the mapping s i» s~g
R, f(x) := ip(\\x\\). Then f*{x*) = #(:r*) for every x* £ X* and f**{x) = <^*#(a;) for every x £ X. Proof, (i) Let ip £ A0nN0 and assume that 0 such that
0; then there exists e > 0 such that
e. Then ip < ip, and so (p* > ip* In
190
Some results and applications of convex analysis in normed spaces
particular v*(s) > sup{st  st/2 \t£ [0,e]}  es/2 > 0. Hence
0. For 0 < s < e~lip(e) we have that
0] — max {sup{is 
e}} < max{se,sup{i(s — e~1ip(e))  t > e} — se, and so limS4.0 s~lip#(s) = 0. This means that
cpy(t) for all* > 0 and c > 1. Let d > 1 and s > 0. Then dq
0} > sup{(ds)(dq/pt)

0} =
Therefore s i> s~qip#(s) is nonincreasing on P. The proof for the other statement is similar. (v) For x* £ X* we have that r(x*)=sup{(x,x*)p(\\x\\)\xeX} = sup{(x,a;*) 
0, 0. Moreover,
0. (vi) => (vii) Let 8 := (ph. By Lemma 3.3.l(i) we have that 6 £ N0 n fi0Furthermore, from the definition of 9, we have that #(a;*) > ds{x) for every (x,x*) £ Of. (vii) =$• (vi) Let
\\x\\ = t}
= sup sup ((x,x*) — (p(x)) =sup(t\\x*\\—(p(x)) t>0 \\x\\=t
= tp* (\\x*\\).
t>0
The equality f**(x) = <^ ## (a;) is obtained similarly.
D
As an illustration of the use of the above classes of functions let us give the following interesting result about the differentiability of a convex function. First recall that a bornology on X is a family B of nonempty bounded subsets of X having the properties: 1) every set B of B is symmetric, 2) B is directed upwards, i.e. for all B±, B2 £ B there exists B3 £ B such that S i U B2 C B3 and 3) 1){B  B £ B} = X. The topology induced by the bornology B on X*, denoted by Tg, is the topology of uniform convergence on every set B of B. The most common homologies on X are the Gateaux, the Hadamard and the Frechet homologies which are obtained taking as B the classes of finite symmetric subsets, the weakly
Differentiability
of convex
functions
191
compact symmetric subsets and the bounded symmetric subsets of X, respectively. To the Gateaux bornology corresponds the weak* topology on X*, while to the Prechet bornology corresponds the norm topology on X*. It is clear that the topology m which corresponds to the bornology B is finer than the weak* topology and coarser than the norm topology. The function g : X —>• E is said to be SdiflFerentiable at x if / is finite on a neighborhood of x and there exists x* £ X* such that lim
/(x
t>0
+
^)/(x) t
\ '
/
'
v
v
the limit being uniform w.r.t. u 6 B, for every B £ B; x* is denoted by Vf(x) and is called the ^differential or ^derivative of / at x. Of course, for the Gateaux, Hadamard and Prechet bornologies one obtains the Gateaux, Hadamard and Prechet derivatives, respectively. Condition 3) implies that / is Gateaux differentiable at x whenever / is Bdifferentiable at x for the bornology B; in particular the ^derivative is unique when it exists. T h e o r e m 3.3.2 Let f € A(X) be continuous atx£ bornology on X. Consider the following statements:
doxnf and B be a
(i) / is Bdifferentiable at x, (ii) / is Bsmooth
at x, i.e. l i m ^ o i  1 ^ ^ ) = 0, where, for t > 0,
oB(t) := sup{/(x + ty) + f{x  ty)  2f{x) \ y € B}, (iii) every selection of df is norm to T& continuous at x, (iv) for every x* £ df(x) there exists a selection 7 of df which is norm to TB continuous at x and j(x) = x*, (v)
there exists a selection of df which is norm to TB continuous at x,
(vi) (a;*) 4 x* whenever x* £ df(x), (xn) )• x,
x* £ df(xn)
for n £ N, and
(vii) (x*n) ^4 x* whenever x* £ df(x), and (en) » 0, (xn) >• x,
0 < en, x*n £ denf(xn)
for n £ N,
(viii) (x*) ^4 x* whenever x* € df(x), and (e n ) > 0.
0 < e„, x* £ denf(x)
for n £ N,
TTien (i) <£> (ii) <=> (iii) «• (iv) <s> (v) o (vi) <£= (vii) o (viii). Moreover, if X is a Banach space and f is Isc then (vi) => (vii).
192
Some results and applications of convex analysis in normed spaces
Proof, Then
(i) => (ii) Assume that / is Sdifferentiable at x. Let u € X.
lim ^ tio
+ tU) +
^
~ ^
~
2f
^
t
= lim f{* + *4.o = (u,Vf(x))
tu)
t +
~
/(5f)
I lim / ( * ~ t\o (u,Vf(x))=0.
tu)
t
~
/ (
^
As the limits in the righthand side of the first equality above is uniform w.r.t. u £ B, for every B £ B, we obtain that / is ^smooth at x. (ii) => (i) Take x* £ df(x); such an element exists by Theorem 2.4.9. Consider B £ B and take u £ B; one has OBH) > f(x + tu)+f(xtu)
2f(x)
>f(x
+ tu)  f(x) t(u,x*)
> 0,
and so 0
tu)m
_{UiX.}<*BM
vues.
Hence Eq. (3.26) holds, the limit being uniform w.r.t. u £ B. (i) => (iii) Let 7 : D —• X* be a selection of df, where D := dom df; of course, int(dom/) C D and 7(2;) = V/(af). Let B £ B. Consider aB : K+  > ! + ,
a B ( t ) := sup{f(x + tu)  f(x) t(u,j(x))
\u£
B}. (3.27) Because / is Sdifferentiable at x, lim^o £ _ 1 <7B(£) = 0, i.e. a £ fii. Because / is convex, <7j9 is convex, too. For x £ D, u £ B and £ > 0 we have that (z + tu  x, 7(0:)) < /(& + tu)  /(:r) < / ( £ ) + t (u, j(x)) + aB(t) 
f(x),
whence t (u, j(x)  i{x))  aB(t) < f(x)  f(x) + (xx,
7(2)).
Taking first the supremum with respect to u £ B, then with respect to t > 0 on the lefthand side of the above inequality we obtain that (CT B ) # (sup u€B
\{U,J(X)
 j(x))\)
< f(x)  S{x) +
< f(x)  f(x) + \\x  x\\ •   7 ( i )  
(xx,j(x)) Vx £ D.
Differentiability
of convex
functions
193
Since / is continuous at x, by Theorem 2.4.13, df is bounded on a neighborhood of x, and so 7 is bounded on a neighborhood of x. Prom the above inequality we obtain that l i m ^ ^ ^ ^ s ) * (sup u £ B \(u,7(x) — 7(x))) = 0. Because, by Lemma 3.3.1, er# G To, we get lim^yj s u p u e B \(u,f(x) — 7(x)) = 0. Therefore 7 is norm to Tg continuous at x. (iii) =>• (iv) and (iv) => (v) are obvious. (v) =£ (i) Let 7 be a selection of df which is norm to T& continuous at x. Consider B 6 B and e > 0. Then there exists 5 > 0 such that D(x, 6) C D and s u p u € B \(u,j(x)  7(x)) < e for all x G D(x,6). Because B is bounded, B c /3Ux for some ft > 0. Fix u G 5 and take <£ : K > 1 ,
f(x+tu)f(x)t(u,j(x))
+ tu,u)
 (u,7(x)),
< ip'+(t) for every t G int(domy>).
=
+ su)j(x))
ds
for all t G [0,S']. Hence limt4.oi1o\B(£) = 0, where WB is denned by Eq. (3.27). It follows that / is Sdifferentiable at x and V/(x) = 7(1). (iii) => (vi) Let a;* G df(x) and ((x n ,x*)) C gr<9/ be such that (xn) >• x. Assume that x* is not the limit of the sequence (x*) for the topology TB It follows that there exist e 0 > 0, B G B and an infinite subset P of N such that s u p u 6 B \{u,x*n — x*)\ > e 0 for every n G P. From (iii) => (i) we have that df(x) = {x*}, and so xn ^ x for n E P. Since (x n ) —> x, we may assume that xn ^ xm for distinct n,m E P Let 7 be a selection of 9 / such that 7(x n ) := x* for n E P, j(x) := x* and 7(x) an (arbitrary) element of df(x) otherwise. From (iii) we obtain that 7 is norm to TQ continuous at x, and so (x*) n e p TA x*, contradicting our choice of P. (vi) =>• (iii) Let 7 be a selection of df, and assume that 7 is not norm to rg continuous at x. Then (since x has a countable base of neighborhoods) there exists a sequence (x n ) C D converging to x such that (7(x n )) does not converge to 7(2:) w.r.t. TB; this contradicts (vi). (vii) =>• (vi) is obvious; just take en = 0 for n G N. (vii) => (viii) is obvious; just take for n G N. (viii) =>• (vii) Let x* E df(x), 0 < en and x* G dSnf(xn) be such that
194
Some results and applications of convex analysis in normed spaces
(en) >• 0 and (xn) > x. Consider e := sup{e„  n 6 N}. Since / is continuous at x, by Corollary 2.2.12, / is Lipschitz on a neighborhood of x, while from Theorem 2.4.13 we have that b\f is bounded on a neighborhood of x. Hence there exist r,m > 0 such that \f{x) — f(y)\ < m\\x — y\\ for x,y e D(x,r) and def(D(x,r)) C mUx* There exists n0 € N such that xn £ D(x,r) for n > n 0 . Let x* € dSnf{x). Then {y  x, x*) = (y  xn, x*) + (xn  x, x*) < f(y)  f(xn)
+en + \\xn  x\\ • \\x"\\
< f(y)  f(x) + m \\x  xn\\ +en + m \\xn  x\\ for all y € X, and so d
enf(xn)
C den+2m\\xnx\\f(x)
Vn > n0.
Because (e„ + 2m \\xn — x\\) —> 0, from our hypothesis we obtain that
(OneP^X*Assume now that X is a Banach space. (vi) =^ (vii) Let x* e df(x), 0 < e„, a;* e dSnf(xn) be such that (e n ) ^ 0 and (x„) —> x. Because xn 6 int(dom/) for large n and / and / coincide on int(dom/), we may suppose that / is lsc. By the Borwein theorem, for every n € N there exists (x'n,x'*) e gr<9/ such that \\xn  x'n\\ < ^/E^ and 11a;*  x'*\\ < y/e^. Therefore (a;^) >• x. By (vi) we obtain that (x'*)n£N A x*. As (\\xn — x'*\\) ¥ 0 and TB is coarser than the norm topology, we obtain that (a;*)neN TA x*. • Because any selection of df coincides with the gradient of / at the points x where df{x) is a singleton, from the preceding theorem we get the following. Corollary 3.3.3 If f & A(X) is continuous and Gateaux differentiable on the open set D c d o m / then f is Bdifferentiable on D if and only if V / is norm to TB continuous on D. • As a consequence of Theorems 3.3.2 and 3.1.2 we get the following interesting result. Corollary 3.3.4 Let X be a Banach space and f £ T(X). differentiable atx* 6 int(dom/*) then V/*(aT*) € X.
If f* is Frechet
Proof. By the Br0ndstedRockafellar theorem there exists ((a;„,a;*)) C grdf such that « ) >• x*. Then xn 6 df*(x*n) for the duality (X*,X**).
Well conditioned functions
195
Taking the Prechet homology, from the implication (i) => (vi) of the preceding theorem we get (xn) » V/*(z*). Because X is closed in X** we obtain that Vf*(x*) G X. • A similar result holds for Gateaux differentiability of /* under supplementary conditions on X. Corollary 3.3.5 Let X be a weakly sequentially complete Banach space and f G T(X). / / / * is Gateaux differentiable at x* G int(dom/*) then V / * 0 T ) G X. Proof. As in the proof of the preceding corollary, there exists a sequence ((xn,x*n)) in grdf such that (a;*) + x*. We have again that xn G 9/*(x*) for the duality (X*,X**). Taking now the Gateaux homology, from the implication (i) =>• (vi) of the preceding theorem we get (xn) » Vf*(x*) for the weak* topology of X**, i.e. ((x*,xn)) » (x*, V/*(z*)> for all x* G X*. In particular (xn) is a weakly Cauchy sequence, and so, by hypothesis, (xn) converges weakly to some x G X. It follows that V/*(i*) =x € X. • Note that every reflexive Banach space is weakly sequentially complete. The next result shows that for convex functions the usual sufficient condition for Prechet differentiability is also necessary. Corollary 3.3.6 Let f G T(X) be continuous and Gateaux differentiable on a neighborhood of x G d o m / . Then f is Prechet differentiable atx if and only if V / is continuous at x. Proof. We may assume that / is continuous and Gateaux differentiable on the ball B(x,r) C d o m / with r > 0. Because V / can be prolonged to a selection of df, the conclusion follows from Theorem 3.3.2. • Note that for convex functions defined on finite dimensional normed vector spaces Gateaux and Frechet differentiability coincide (use for example Theorem 3.3.2 and take into account the fact that the norm and weak topologies coincide). This is not true for infinite dimensional normed spaces as the functions in Exercises 2.10 and 3.1 show. 3.4
Well Conditioned Functions
Let / G A{X) be such that argmin/ := {x G X \ f{x) = i n f / } ^ 0, •0 G A and 5 C X be a nonempty closed convex set. We say that / is
196
Some results and applications of convex analysis in normed
spaces
^conditioned with respect to S if MxeX
: f{x)>mff
+ i(>(ds{x))
(see Eq. (3.62) on page 237 for the definition of ds). If there exists rj > 0 such that the above inequality holds for every x € X with ds(x) < T), we say that / is locally ipconditioned with respect to S. Let us note that if / is V'conditioned with respect to the subset S and tp £ A0, then argmin/ C S. Taking into account this fact, we say that / is ^conditioned if / is ^conditioned with respect to argmin/. Assume that S := argmin/ ^ 0 and consider the conditioning gage of / defined by
V>/:t+ *•!+,
ipf(t)~mi{f(x)Mf\xeX,ds(x)=t}.
(3.28)
It is clear that tpf 6 A and / is ^/conditioned. The following result holds. Proposition 3.4.1 ipf £ Nx. Proof. Let t > exists a sequence ipf(ct). Let (en) I ct+en. Let An :=
Let / £ A(X) be such that S := argmin/ ^ 0. Then
0 and c > 1; we must show that ipj{ct) > cipf(t). There (xn) C X such that ds{xn) = ct and (f(xn) — inf/) —> 0. For every n there exists an £ S such that \\xn — an\\ < (a;„ — an\\ — ct + t) / \\xn — an\\ 6 [C1,1[. We have that
ds ((1  Xn)an + A„a;n) > ds(xn)
 (1  A„) \\xn  an\\ = ctct
+ t = t.
Because A i> ds ((1 — A)an + Xxn) is a continuous function, there exists fin £ ]0, An] such that ds ((1 — £«„)an + nnxn) = t We obtain so that V'/W < / ( ( I  fJn)an + HnXn)  inf / < (1 fin)f(an)
+ Hnf(xn)
inf/
= Mn (/(^n)  inf / ) < An (f(x„)  inf / ) . Because (An) * c _ 1 , we obtain that ipf(t) < c~l • ip/(ct).
O
Let us remark we used only that / ( ( l  X)a + Xx) < (1 — X)f(a) + Xf(x) for every a £ S, x £ X and A £ [0,1], in other words the fact that / is starshaped at every a 6 5; of course, this condition ensures the convexity of 5. Corollary 3.4.2 Let f and S be as in Proposition 3.41 If ip '• [0,a] > M.+ , where a > 0, is such that Vx£X,
ds(x)
: f(x) > inf / + V
(ds(x)),
Well conditioned functions
197
then f is ipaconditioned, where ,
™
™
, , ,
[ V>(*) I x^t
for for
te \0,a\, t €ja,oo[.
Proof. From the definition of ipf we have that ipf > ip on [0,a]. For ds(x) = 7 > a we have that / ( i ) > inf / + V/(7) > inf / + V>/(a) >
inf
/ + ~^(a)
= inf / + ^ d s ( : r ) = inf / + « (d 5 (a:)), a whence the conclusion.
•
Proposition 3.4.1 shows that ipf(t) > 0 for every t > t0 if ipf (to) > 0. It is important the case in which ipf(t) > 0 for every t > 0, i.e. ipf £ Ao', in such a case we say that / is wellconditioned. The following result holds. Theorem 3.4.3 Let X be a Banach space and f S T(X) be such that S := argmin/ ^ 0. T/je following statements are equivalent: (i) £/iere exist ip £ A and to > 0 s«c/t i/rni VW > 0 for every t G ]0, to[ and f is ipconditioned; (ii) for every sequence (xn) C X with (f(xn)) —> inf/ we have that (ds(xn)) > 0; (iii) tftere ezisis ip £To such that f is ipconditioned; (iv) B t r e E i , V x * 6 l * : /*(x*) < /*(0) + tj(i*) + <7(x*); (v) 3 V € r 0 , V i e S , V ( z , a : * ) e g r 0 / : ( i  i , a ; * ) > (vi)
B ^ e y i o n A f o , V(x,x*)egrdf
:
ip(ds(x));
(vii) 3flGn 0 n7Vo, V(a;,a:*)egra/ : ds(x) < 9(\\x*\\); (viii) V(z n ) C X : (d 8 / ( T n ) (0)) > 0 => (d s (x„)) > 0. Furthermore, in the equivalence (iii) <£> (iv) we can take a = ip&, ip =
198
Some results and applications of convex analysis in normed spaces
tpf £ A0 fl iVi), we have that ip := cotpf £ T 0 (see Lemma 3.3.1(iii)). Of course, / is ^conditioned. (iii) => (ii) is obvious, since for ip £ T 0 we have that (tp(tn)) > 0 =>• (*n) >• 0. (ii) =>• (i) Suppose that there exists £o > 0 such that V/(*o) = 0 Then there exists (x„) C X such that ds(xn) = t0 and (f(xn)  inf / ) > 0, which evidently contradicts the hypothesis. As / is ^/conditioned, the conclusion holds. (iii) =>• (iv) Using Theorems 2.8.10, 2.3.1(ix) and Corollaries 2.4.7, 2.4.16 (see also Exercise 3.7) we have that (V o d s ) * = t £ + ^ # °      ,
{t*s + ^o\\.\\)*=^ods.
(3.29)
Using Eq. (3.29), for x* € X* we have that f{x*)
= sup((x,x*)f(x)) x€X
< sup {(x,x*)  i n f / 
ip(ds(x)))
xeX
= no) +tyo dsy (x*) = no) + L*S(X*)+^*(\\x*\\). The conclusion follows with a := ip#; a € Si by Lemma 3.3.1 (iii). (iv) =>• (iii) is obtained similarly, using Eq. (3.29), with ip := a#. (iii) => (v) Let x £ S and (x,x*) € grdf. Then f(x)>f{x)+ip(ds{x))
and f{x) > f(x) + (x 
x,x*).
Adding these two relations we obtain the conclusion with the same ip. (v) =>• (vi) From (v) we obtain that VxeS,
V(x,x*)
Egrdf
: \\x  x\\ • \\x*\\ > ip(ds(x)),
whence x* • ds(x) > tp(ds(x)) for every (x,x*) £ grdf. The conclusion follows taking ip € A0 defined by
(iii) Let 7 £]0,1[ and V : K+ ^ W+,
V(*) = ( l  7 ) *  v ( 7 * ) 
Well conditioned
functions
199
Suppose that (iii) does not hold for this ip. Then there exists x E X such that f(x) < ini f+ip(ds(x)). It follows that x £ S and there exists c E ]0,1[ such that f(x)<mif We take e = c • ip(^j(ds{x))) there exists u E X such that f(u) + e • \\u — x\\ < f(x)
+
ciP(ds(x)).
> 0. Using Ekeland's variational principle,
and f(u) < f(x) + e • \\u — x\\ Vx E X.
The last relation being equivalent to df(u)C\eUx* i1 0, there exists u* E X* such that (u,u*) E gr<9/ and u* < e. FVom the first relation, by the choice of x, the definitions of x[> and e, and the hypothesis, it follows that \\u — x\\ < (1 — y)ds{x)
and
(p(j(ds(x))).
But ds(u) > ds(x)  \\u  x\\ > ds(x)  (1  j)ds(x)
= 7 • ds(x),
whence the contradiction
<^(7 • ds(x)) > c • v?(7 • ds(x)). Hence (iii) holds for this tj). Because t • ^(jt) > JQ
(t) := (1  7) JQ tp(^s) ds. (vii) => (viii) is obvious. (viii) =>• (vii) Consider the function 6 : 1 + > ! + ,
6(t) = sup{d s (x) I {x,x*) 6 grdf, a;* < t}.
It is obvious that 9 is nondecreasing, 6(0) = 0 (i.e. 8 E NQ) and ds(x) < 6(\\x*\\) for every (a:,a;*) E grdf. If limto^(*) > 0, there exists £0 > 0 such that 0(t) > £0 for every t > 0. Prom the definition of 6, for every n E N there exists (xn,x^) E df such that a;* < 1/n and ds(xn) > EQ. Of course this contradicts the hypothesis. D We remark that we used the convexity of %[> only for the implication (iv) => (iii); for the other implications it is sufficient that ij) E N\. The function ipf verifies this condition. Also, we used that X is a Banach space only in the implication (vi) =>• (iii). We also note that in the implications (iii) <$ (iv), (v) =>• (vi) <S> (vii) •£> (viii), (vi) =*> (iii) =>• (ii) =*> (i) we can
200
Some results and applications of convex analysis in normed spaces
take S C X a nonempty closed convex set, and in the implication (iii) => (v) we used only the fact that S C argmin/. Corollary 3.4.4 Let X be a Banach space, f € T(X) and (x, x*) € grdf. The following statements are equivalent: (i) 3iPeT0,
Vx€X
: f(x)>f(x)
(ii) 3 f f € S i , V i * 6 P
: f*(x*)
(iii) 3 ^ € r 0 , \/(x,x*)Ggrdf (iv) 3ip£A0nN0,
(xx,x*)+rP(\\xx\\);
< f(x*)+{x,x*
 x*)+a(\\x*  x*\\);
: (x  aT,x*  z*) > V(a;  z);
\/(x,x*)egrdf
(v) 3den0nN0,v(x,x*)egTdf (vi) V(xn)cX
+
: y>(x  z   ) < x*  x*\\;
•.
\\xx\\ <eq\x* x*\\y,
: (da/(:Cn)(x*))>0=^(:En)>z;
(vii) x* 6 int(dom/*) and / * is Frechet differentiable at x*. Proof. Taking the auxiliary function g : X —> R denned by g(x) = f(x) — (x, x*), we may assume that x* = 0. The equivalence of conditions (i)(vi) follows from Theorem 3.4.3 by taking S := {x} (taking into account, eventually, the remarks done after the proof of that theorem). (ii) =>• (vii) Taking into account that (x, x*) £ gr<9/, we have that Vi'eT
: 0 < f*(x*)
 f*(x*)
 (x,x* x*)
whence x* £ int(dom/*) and lim sup
r(x*)r(x*)(x,x*x*)
< l i
m s u p
t0
t
^ = l i m ^ = 0 . *4° t
This shows that / * is Frechet differentiable at a;* and V/*(a;*) = x. (vii) => (ii) Because (x*,x) € g r ( 9 / ) _ 1 C gr<9/* (the last one for the duality (X*,X**)), we obtain that x e df*{x*). Since / * is Frechet differentiable at x*, we have that V/*(aT") = x. For t € K+ consider a(t) := suv{r{x*)
 f*{x*)  (x,x* x*)
= sup{r(x*+tu*)r(x*)t(x,u*)
\ \\x* x*\\ = t}   K   = 1}.
Because the function 11> f*(x* + tu*)  f*(x*) — t {x,u*) (6 M.+) is convex and lsc, it follows that a G T. Furthermore, from the Frechet differentiability of the function / * it follows that lim^o £1
Well conditioned
functions
201
A sufficient condition for (i) from the preceding corollary is that / ( ( l  X)x + Xx) + A(l  X)rp(\\x  i) < (1  X)f(x) + Xf(x)
(3.30)
for all x £ X and A £ ]0,1[. Indeed, if Eq. (3.30) holds and x € dom / then {f{x + X(x  x))  /0c))/A < f{x)  f{x)  (1  X)^{\\x  x\\) for all x £ X and all A 6 ]0,1[, whence Vxel
: f{x)>f{x)
+ f'{x,xx)+ip{\\xx\\),
(3.31)
and so VieX,Vi*ea/(i)
: f{x)>f{x)
+ (xx,x*)+iP{\\xx\\).
(3.32)
A function / e A(X) for which there exists ip &Ao satisfying condition (3.30) is said to be uniformly convex at x (6 d o m / ) . Let / 6 A(X) and x € d o m / . The gage of uniform convexity of / at x is related to relation (3.30), and is defined by Pf,x{t) '•= i n f \ —
A)/(z) + Xfjx)  / ( ( l  X)x + Ax) A£]0,1[, A(l  A) \\x — x\\ = t
= inf { <* ' X»W
+
Y(f_+^ ~ ^
+ A
«»>  u e SX, A 6 ]0,1[} (3.33)
(with the convention oo  oo = oo). Using Theorem 2.1.5(vii) one obtains easily that pf^ € No (Exercise!). We may introduce also a gage related to relation (3.31). More exactly consider tif,x(t) ••= inf {f(x)  f(x)  f'(x, x  x)  x E dom / , a;  x\\ = t} = inf {f(x + tu) — f(x) — tf'(x, u)  u e Sx H cone(dom /  x)} . Because the mapping M+ 3 11> f(x+tu) — f(x)—tf'(x, u) 6 M+ belongs to T C N\, we obtain that j ? ^ € Nx. As in the proof of (3.30) =* (3.31) we have that i9/,j > pf^. Thus, when / is uniformly convex at x € d o m / (i.e. P/,x 6 ^o) then $/,j € .Ao One may ask if the converse implication holds. The answer is affirmative when / is Frechet differentiable at x € dom / .
202
Some results and applications of convex analysis in normed spaces
Proposition 3.4.5 Let f 6 F(X) be Frechet differentiable at x 6 d o m / . If flf,x € AQ then f is uniformly convex at T. Proof.
Let t > 0 be such that D(x, t) C int(dom/). By hypothesis
f{x)>f{x)
+ f'{x,xx)+6
VxeX,
\\xx\\ = t/2,
where 6 = i9/ iS (i/2) > 0. On the other hand, because / is Prechet differentiable at x, a selection 7 : dom df > X* of df exists which is continuous at x (by Theorem 3.3.2). Hence there exists n > 0 such that Il7(*)  V/(3c) < S/t for z  x\\ < j]. Let a e]0,1[ be such that 1  a < 2t}/t. Consider x 6 X arbitrary satisfying a; — x\\ — t and take z := ax + (la)^^= x\±=^{xx) € int(dom/). Hence \\z x\\ = ^^t < r], and so \\j(z)  Vf(x)\\ < 5/t. Of course, f(x) > f(z) + f'(z,xAs (37 + x) = ^hx + ^tiz,
z) = f(z) + / ' (z, we
l
^(x
x)) .
(3.34)
have that
STT/(z) + S T l / W > / ( § ( * + *)) •
(335)
Multiplying Eq. (3.35) with a+1 and adding the result to (3.34) we obtain f(x) + af(x) > (1 + a ) / (I(3f + x)) + f (z, ±=2(3f  x)) , whence f{x) + f{x)  2 / {\{x + x))>^
(/ (1(3 + a:))  f(x) + \f< (z,x  x)) 6
>^(
+ y^xx)
+ I / ' (z,3f a;))
>^(<5+i
Uniformly convex and uniformly
3.5
smooth convex
203
functions
Uniformly Convex and Uniformly Smooth Convex Functions
We "globalize" the notion of uniformly convex function at a point as follows. We say that the proper function / : X —> K is pconvex, where p £ A, if / ( ( l  X)x + Xy) + A(l  X)p(\\x  i/H) < (1  X)f(x) + Xf(y). for all x, y £ X and all A £ ]0,1 [. Of course, if / is pconvex then / is convex; we say that / is uniformly convex if there exists p £ AQ such that / is pconvex. We say that / is strongly convex (with rate c or c strongly convex) if / is pcconvex for some c > 0, where pc £ AQ, pc{t) •= \ct2. It is clear that when / is uniformly convex then / is uniformly convex at every point of its domain and when / is uniformly convex at every point of its domain then / is strictly convex. When / £ A(X) and A C X is a convex set we say that / is uniformly convex on A if / + LA is uniformly convex. It is natural to introduce the gage of uniform convexity of the convex function / . Namely, it is the function p/ : K+ »• R+ defined by p/(i)=inf
(lX)f(x)+Xf(y)~f((lX)x
+ Xy)
A(l  A)
AG]0,1[,
x,y £ d o m / , \\x — y\\ = t > . Of course, Pf > p when / is pconvex. Hence / is uniformly convex if, and only if, p/ £ Ao. The gage of uniform convexity of the convex function / has a remarkable property. Proposition 3.5.1
Let f £ A(X).
Vt>0, Vc>l i.e. the function 11» t~2pf(t)
Then :
Pf(ct)>c2pf(t),
is nondecreasing on P.
Proof. Let t > 0 and c £ ]1,2[ be such that p/(ct) < oo. Let us fix e > 0. There exist x,y £ d o m / and A G]0,  ] such that y — x\\ = ct and n , ^ Pf(ct)
. . (lX)f(x) + pe >
+ Xf(y)f((lX)x ^ — 
+ Xy)
.
204
Some results and applications of convex analysis in normed spaces
Consider x\ := (1  X)x + Xy and xc := (1  c l)x + c ly. We have that \\xc — x\\ — t and x\ = (1 — c\)x + cXxc (of course cX 6 ]0,1[). Hence f(xc)
< (1  c1)^)
f(xx)
< (1  cX)f(x) + cXf{xc)  cA(l 
f(xx)
> (1  X)f(x) + Xf(y)  A(l  X)pf(ct)  eX(l  A).
+ c'fiy)
 c\l

c^pfict), cX)pf(t),
From these relations (multiplying the first with cX > 0) we obtain that c2Pf(t) < Pf(ct) + ec• ±Eh < PM)
+££~C
Letting e > 0, we obtain that c2/9/(£) < pf(ct) for all c € ]1, 2[ and t > 0. By mathematical induction one obtains immediately that c2npf(t) < Pf(cnt) for c e ] l , 2[, n £ N and t > 0, and so c2pf(t) < pf(ct) for every c > 1 and t > 0. The conclusion holds. D A dual notion (as we shall see) for uniform convexity is that of uniform smoothness. Let / € A(X), a £ A and O ^ i c d o m / . We say that / is ersmooth on A if for all x,y £ X and all A 6]0,1[ such that (1 — A)a; + Xy £ A we have / ( ( l  X)x + Xy) + A(l  X)a(\\x  y\\) > (1  A)/(i) + A/(y),
(3.36)
or equivalently
f(x) + A(l  AMIMI) > (1  X)f(x  Xy) + Xf(x + (1  X)y) for all z £ A, y £ X and A £]0,1[; we say that / is crsmooth if / is crsmooth on dom / . Having in view this definition, it is natural to consider the following gage of uniform s m o o t h n e s s o n A of the function / : af,A{t)
••= s u p ^
(1  A)/(:r) + Xf(y)  / ( ( l  X)x + Xy) A€]0,1[, A(l  A) x,y£X,
(lX)x
+ Xy£A,
f (1  X)f(x  Xty) + Xf(x + (1  X)ty)  f(x) sup
\\xy\\=t\ Ae]Q>1[j
A(l  A)
x £ A, y £ Sx
}•
(3.37)
Uniformly convex and uniformly
smooth convex functions
205
c/.dom / is denoted by CTJ and is called the gage of uniform smoothness of / . It is obvious that 07,4 < a if / is crsmooth on A. Sometimes it is useful to consider the following midpoint gage of smoothness: a°fA(t) := sup {f(x) + f{y)  2f{\x + \y) \ \x + \y £ A, \\x  y\\ = 2t} = sup {f{x  ty) + f{x + ty)  2f{x) \ x £ A, y £ SX}
(3.38)
It is easy to obtain that 2cr°fA{t/2) < af,A(t) < o°fA{t) for t > 0. As above, c^dom/ w m ^ e denoted simply
Let f £ A(X), h £ A{X*) have proper conjugates
(i) J / / and h are pconvex then f* and h* are p# smooth, respectively. (ii) / / / and h are crsmooth then f* and h* are o# convex, respectively. Proof, (i) Suppose that / is pconvex. Let t > 0 be fixed and consider x* eX*,y* £ Sx* a n d A £ ] 0 , l [ ; t a k e ^ := x*\ty*, x\ := x* + (lX)ty*. Consider also 70 < f*{xo) and 7J < f*{x{). From the definition of / * there exist Xo, xi £ X such that 70 < {X0,XQ)  f(x0),
71 < (xi,xl)

f{x{).
Multiplying the first relation by 1 — A and the second one by A, then adding them to both sides of the (YoungFenchel) inequality 0 < f(xx) +f*{x*)
~{xx,x*),
206
Some results and applications of convex analysis in normed spaces
where x\ :— (1 — X)xo + Xx\, we obtain that (1  A)7o + A71 < f(xx) + /*(**)  (1  A)/(ar0)  A / f o ) + A(l  A)* (x, < f*(x*)
+ A(l  A) (t \\x0  i i    p(\\x0  nil))
+
x0,y*)
\(l\)P*(t).
Letting 7; —>• f*(x*) for i = 1,2, we obtain that /* is p # smooth. The same calculation shows that h* : X —> E is /9*smooth if /i is p convex. (ii) Suppose now that / is crsmooth. Consider x^,x\ 6 X* and A 6 ]0,1[. For x, y € X, denoting x*x := (1  X)XQ + Az*, we have that
>(lX)((xXy,x*0)f(xXy)) +
X({x + (lX)y,x*1)f(x
> (1  A) (x  Xy,X*0) + X(x+(1
+
(lX)y))
X)y,x\)  f(x)  A(l  A)a(y)
= {x, x*x)  f{x) + A(l  A) ((y, x\  x*0) 
a(\\y\\)).
Since x, y € X are arbitrary, we have that
(ix)r(x*0) + xr(x*1) > snpxeX
((x, x*x)  f{x)) + A(l  A)supJ/€js: ((y, x{  XQ) 
= rix*x) +
XilX)o#i\\x*x*1\\),
which means that / * is cr#convex. The proof for the fact that h* is cr#convex when h is crsmooth is similar. • Corollary 3.5.4 Let f 6 T(X). Then o{* = ipf)* and 07 = ( p / . ) # Furthermore t !• t~2afit) is nonincreasing on P. In particular, if there exists to > 0 such that 07 (£0) < 00 then a fit) < 00 for every t > 0. Proof. From the preceding proposition we have that /* is (p/) # smooth # and (<7/) convex, while / = /** is (p/.) # smooth and (07.)* convex. Therefore we have that 07 < (/?/•)* a n d Pf > (07)*. Passing to the conjugates in the second inequality we obtain that 07 < (p/*) # < (07)**. Therefore erf = (p/») # = (07)** (in particular we have obtained again
Uniformly convex and uniformly
smooth convex functions
207
that aj is convex and lower semicontinuous). In a similar way we obtain that 07. = (/9/)*. Taking into account Proposition 3.5.1, Lemma 3.3.1(iv) and the relation Of = (p/*)*, we have that t >> t~2a;{t) is nonincreasing on P. The remainder of the conclusion is immediate. • We say that / G A(X) is uniformly smooth (on the nonempty subset A of dom / ) if there exists a G fii such that / is crsmooth (on A). When A is a singleton {x} C dom / , / is uniformly smooth on {x} if and only if / is smooth at x in the sense of Theorem 3.3.2. Taking into account Proposition 3.5.2, if / is uniformly smooth o n ^ C l then there exists £o > 0 such that A + SQBX C d o m / , while if / is uniformly smooth then d o m / = X. Summarizing Propositions 3.5.13.5.3 (and using Lemma 3.3.1(iii)) we obtain the following important result. Theorem 3.5.5
Let f £T(X).
Then
(i) / is uniformly convex 44> / * is uniformly smooth; (ii) / is uniformly smooth •$=>• f* is uniformly convex. It is quite natural to say that the proper function / : X —> E is uniformly Frechet differentiable on 0 ^ A C X if there exists £o > 0 such that A + SQBX C d o m / , / is Frechet differentiable at any point of A and nm /Or+
tH)
*»)/(*) t
V M
uniformly with respect to x € A and u € Sx Of course, when A = {a} this means that / is Frechet differentiable at a. Similarly to the proof of Theorem 3.3.2, one obtains easily that / G A(X) is uniformly Frechet differentiable on A C dom / if and only if / is continuous at some point of its domain and is uniformly smooth on A. The following result gives other characterizations of uniformly smooth convex functions. Theorem 3.5.6
Let f G A(X). Consider the following
statements:
(i) / is uniformly smooth; (ii)
( d o m / ) ! / 0 and there exists a<x G fii such that
V x G ( d o m / r , V y G X : f(y) < f{x) + f{x,yx)
+ a2{\\y  x\\); (3.39)
(iii) d o m / = X, f'(x, •) is linear for every x G X and there exists &$ €
•
208
Some results and applications of convex analysis in normed spaces
Oi such that \/x,y£X
+ f,(y,yx)
: f'(x,xy)
(iv) grdf ^ 0 and there existsCT4G fti
SMCA
(3.40)
i/ioi
y(x,x*)egrdf,
V j / e X : /(i/)+a4(ya:); (3.41) (v) dom 9 / = X and there exists as G fii such that V(x,x'),(v,v')£&df
• (yx,y*x*)
(3.42)
(vi) dom df = X and there exists ipi G To such that
v{x,x*),(v,v')e&df
: m>f(x)
+ (yx,x*) + M\\y*x'\\); (3.43)
(vii) dom df = X and there exists V>2 € T 0 suc/i i/iai V (*,**), (y,y*)egidf
: (i/i,y*a;*) > ^(y*i');
(3.44)
(viii) dom df — X and there exists y> £ Ao(l No such that \/(x,x*),(y,y*)
£grdf
: \\y  x\\ > V{\\y*  x*\\);
(3.45)
(ix) dom df = X and there exists 6 £ flo f) No such that V(z,z*), ( \\y*  x*\\; (x) d o m / = X, f is Frechet differentiate $7I such that Vx,yeX
: (xy,
Vf(x)
on X and there exists &§ G
 V/(j/)) < a6(\\y  x\\);
(xi) d o m / = X, f is Frechet differentiate uous.
(3.46)
(3.47)
and V / is uniformly contin
Then conditions (i)  (iii) and (iv)  (xi) are equivalent, respectively, and (iv) =£• (ii). Furthermore, if f is continuous at some point of dom / (or X is Banach space and f is lower semicontinuous) then the eleven statements are equivalent; in such a case if one of conditions (i)  (xi) holds then dom / = X and f is continuous. Proof, (i) =>• (ii) Let a G fii satisfy Eq. (3.36); we already observed above that d o m / = X. From Eq. (3.36) we have that A" 1 (f{x + \{y  x))  f(x)) + (1  AMHi,  x\\) > f(y) 
f(x)
Uniformly convex and uniformly
smooth convex functions
209
for all x,y G X and A G]0,1[; taking the limit for A ¥ 0 we obtain Eq. (3.39) with cr2 : = a(ii) => (iii) Let x0 G ( d o m / ) \ From Theorem 2.1.13 we have that f'(xo,u) G R for every u G X. Let t0 > 0 be such that <72(£0) < oo. From Eq. (3.39), it follows that {y G X \ \\y  x0\\ = t0} C d o m / , and so B(x0,t0) C d o m / (because d o m / is a convex set). Therefore (dom/) 1 + toBx C dom / , whence, as in the proof of Proposition 3.5.2 we obtain that d o m / = X. Changing x and y in Eq. (3.39), then adding side by side the corresponding relations, we obtain
V i j e l
: 0
+ f'(y,xy)+2a2(\\yx\\).
(3.48)
Let x G X be fixed and u £ X \ {0}. Consider the function
• R, tp(t) = f(x + tu);
>p'_{t) =f'{x+
tu,u).
Taking into account Theorem 2.1.5, for — t < 0 < t we obtain that
'(0) < <^+(0) < ¥>'_(*), and so f'(x,u)
+ f'{x, u) = v ; ( 0 )  ¥>'_(0) < ¥»'_(*) = f'(x
+ tu,u)
ip'+(t)
 f{x + tu,u)
1
< (2i) 2 CT2 (2iH) = (2it i ) 1 2 CT2 (2i U ) • u for every t > 0. Letting t —>• 0, we obtain that f'(x,u) + f'(x,—u) < 0. Since the converse inequality holds (f'(x, •) being sublinear), we have that f'(x, u) = f'(x,u), and so f'(x, •) is linear. In this situation Eq. (3.40) coincides with Eq. (3.48) (and so 03 = 2o2). (iii) => (ii) Let ^3 : 1+ > R + be defined by CT3(t) = sup{cr3(s)  s G [0, t]}. It is obvious that W3 e fli C\ No and ^3 > a. Consider the function ip defined in the proof of the preceding implication. In our situation
0. (vi) =>• (vii) and (vii) =*> (vi) are immediate with 8 := iph and ip := 8e, respectively. (vi) =>• (iii) as in the proof of Theorem 3.4.3, with tp2(t) '•— (1 7) J0 (fijs) ds, where 7 G ]0,1[ is fixed. (vii) •&• (viii) is obvious; for <= take 0 the gage of uniform continuity of 0, respectively. (vii) O (viii) is obvious with 6 := (ph and tp := 6e, respectively. (viii) =>• (i) Let (x,x*), {y,y*) £ g r d / ; from the hypothesis we have that (y x,y*
210
Some results and applications of convex analysis in normed spaces
It follows t h a t
f(x + u)  f{x) = p(l)  tp(0) = £ if/it) dt
'(0) + /J1"11 t~la3{t) dt = f'{x, u) + a2(\\u\\), where ^ ( i ) = / 0 s~1a:3(s) ds; it is obvious that o (i) From (ii) =>• (iii) we have that d o m / = X and f'(x, •) is linear for every x £ X. Let 07 : K+ —> R+ be defined by af(t)  sup{f(y)
 f(x)  f'(x, y  x) \ x,y G X, \\y  x\\ = t}
= sup{/(a; + tu)  f(x)  tf'(x,u)\x,ueX,
\\U\\ = 1}.
(3.49)
Since t H> f[x + tu) — f(x) — tf'(x,u) is convex and continuous for all x,u € X, it follows that
f(x) < f(xx)+f'(xx,xxx) + af(\\xxx\\), f(y) < f(xx) + f'(xx,y  xx) + 37(y  xx\\). Multiplying the first relation with 1 — A and the second one with A, then adding them side by side, we obtain (1  \)f(x)
+ Xf(y)
< f(xx) + A(l  X)[f'(xx,x
y) + f'(xx,y
+ (1  X)af(\ \\y  x\\) + \af((l
x)]
 A) \\y  x\\)
(3.50)
• (i) Let (X0,XQ) 6 grdf and take t0 > 0 such that c 4 (i) < 1 for t G [0,i 0 ] From Eq. (3.41) we obtain that f(x) < f(x0) + t0 \\XQ\\ + 1 for x G B(xo,to), and so B(xo,to) C d o m / and / is continuous at every point of B(xo,to). Hence domdf + toBx C domdf C d o m / , which implies that dom df = dom / = X and / is continuous on X.
Uniformly convex and uniformly
smooth convex functions
211
Let x,y G X and A G]0,1[. With the notation from the proof of the implication (ii) =* (i), there exists x*x € df(x\). Replacing f'(x,) by x* G df(x) and f'{x\, •) by x*x in the proof of that implication, we obtain that / is 2
+ (yx,y*)
and
0 < (y  x,x*  y*) + a5{\\y  x\\),
Eq. (3.41) follows by adding side by side these relations, with 04 := 05. (iv) =» (vi) From (iv) =£• (i) we have that dom / = X and / is continuous. Let (a;, a;*), (y, y*) G gr<9/ be fixed and take an arbitrary 2 G X. From Eq. (3.41) we have that f(y + z) < f(y) + (z,y*) + cr4(H^), and so 0
f(x) (y +
< f(y)  f{x) (yx,
zx,x*)
x*)  (z, x*  y*) + a 4 (z).
Therefore VzeX
:
(z,x*y*)ai(\\z\\)
Taking the supremum with respect to z £ X and using Lemma 3.3.1(v) we obtain that Eq. (3.43) holds with ipi := af. By Lemma 3.3.1(iii) we have that V"i G r 0 . (vi) => (vii) Changing x and y in Eq. (3.43) and summing the obtained relations side by side, Eq. (3.44) holds with ^2 := 2V>i(vii) =$• (viii) From the obvious inequality (y — x,y* — x*) < \\y — x\\ • \\y* — x*\\ we obtain immediately that Eq. (3.45) holds with ip G A0nN0, ip(t) := t~li})2{t) for t> 0. (viii) => (ix) and (ix) =^ (viii) The first implication is obvious by taking 6 := iph and the second one by taking ip := 6e using Lemma 3.3.l(i). (ix) => (v) The implication is obvious by taking cr5 G fii, (T5(t) :— t6(t) for t > 0. From what was shown above we have that conditions (i)(iii) and (iv)(ix) are equivalent, respectively, and (iv) =£ (i). Moreover d o m / = X and / is continuous if (iv) holds.
212
Some results and applications of convex analysis in normed spaces
(ix) => (xi) From what was said above, / is continuous; from Eq. (3.46) we have that df is single valued; by Corollary 2.4.10 we have that / is Gateaux differentiable. Using again Eq. (3.46), V / is uniformly continuous. Using Corollary 3.3.3 we obtain that / is Frechet differentiable. (xi) =$> (ix) This implication is obvious; just take 6 the gage of uniform continuity of V / . (v) =>• (x) In our hypothesis, as shown above, / is Frechet differentiable. Hence Eq. (3.47) holds with a& := 05. (x) =£• (v) is obvious with 05 := erg. (ii) => (iv) when / is continuous at some point of its domain. From (ii) => (hi) we have that d o m / = X, and so / is continuous on X, and f'(x, •) = V/(x) for every x € X. Therefore Eq. (3.41) holds with <74 := a2. (ii) => (iv) when / is lsc and X is a Banach space. From Theorem 2.2.20 we have that / is continuous on (dom/) 1 ^ 0. The conclusion follows from the previous case. • The proof of the preceding theorem shows that 07 < 07 < 2 5 / . Moreover, when / is continuous, one can take 02 = 04 and 03 = 0 5 = CFQ. Remark 3.5.1 In Theorem 3.5.6 the implication (ix) =>• (iv) holds with ai(t):=J*e(s)d8. Indeed, if (ix) holds, by the preceding theorem d o m / = X and / is Frechet differentiable. Let x,y E X. Taking V
:E^K,
+
ty)f(x)t(yx,Vf(x)),
we have that
tp is derivable
V / ((1  t)x + ty)  V/(a:)) < \\y  x\\ • 8 (t \\y  x\\)
for every ( 6 1. It follows that /(!/)  f(x) ~{yx,
Vf(x))
= J,,1
= fSy~xh(s)ds
(t \\y  x\\) dt
= ai(\\yx\\).
Corollary 3.5.7 Let f : X —> E be a continuous convex function and p, q £ E be such that 1 < p < 2 < q and p _ 1 + q"1 = 1. The following statements are equivalent: (i) there exists L\ > 0 such that f is auniformly smooth, where o~(t) := ^t" for t > 0;
Uniformly convex and uniformly
(ii) 3 L 2 > 0 , Vx,yeX, f(y)
smooth convex
Vx* e 8f(x) + (yx,x*)
functions
213
: +
^\\xy\\p;
(iii) 3 L 3 > 0 > V ( a : , a : * ) , ( i / , y * ) € g r a / : f(y) > fix) + (y x,x*) + ^ (iv) 3 i 4 > 0 , V(x,x'),
(y,y*)£grdf
ar*  y*\\q ;
:
(yX,y*X*)<^\\Xyf(v) 3 L 5 > 0 , V(x,x'),
(y,y*)£grdf
:
Vx,yeX.
Proof. The conditions (i)(vi) of the corollary correspond to conditions (i), (iv) (or (ii)), (vi), (v) (or (iii)), (vii) and (ix) of the preceding theorem, respectively. So, we have that (i) => (ii) with L 2 ~ L\ and (ii) => (i) with Li := 22~PL2 (using directly Eq. (3.50)). Also, (ii) =* (iii) with L3 := L\~l, (iii) =» (v) with L5:= L3, (v) => (vi) with L 6 := (L5q/2)P~\ (vi) => (ii) with Z/2 := ^6 (by the preceding remark), (ii) =*• (iv) with L\ := L2 and (iv) =>• (ii) with L 2 := 2L±/p (for this one uses the proof of the implication (iii) => (ii) of the theorem). • Remark 3.5.2 For p = 2we may take the same Lfc = L > 0 for all k £ 1,6 in the preceding corollary, as the proof shows. The next result shows that every uniformly convex function on a Banach space is coercive and its subdifferential is onto. Proposition 3.5.8 convex. Then
Let X be a Banach space and f £ T(X) be uniformly
fix) limiiif i  H > 0 IMI"» HxH2
and
badf
= X*.
Moreover, if (xn) C X is a minimizing sequence for f then (xn) converges to the unique minimum point of f.
214
Some results and applications of convex analysis in normed spaces
Proof. By the Br0ndstedRockafellar theorem there exists grdf, and so Vz G X : f{x) > f(x0) + (x
x0,z5).
(XO,XQ)
G
(3.51)
It follows that f(x0) + A (x  x0,
XQ)
< f ((1  X)x0 + Xx) < (1  X)f(x0) + Xf(x)  A(l  X)Pf(\\x  aroll)
for all x G X and A G]0,1[, and so f(x) > f(x0) + (x  XQ,XQ) + (1 A)/?/(z  £o) for x G X and A G ]0,1[. Letting A > 0 we obtain that f(x0) + {x x0,x*0) + Pf(\\x  a:0) < f{x)
(3.52)
for all x e X . Dividing the last inequality by a; — zo , taking into account that (x — XQ, XQ) > — \\x — x0\\ • \\XQ\\ and Proposition 3.5.1, we obtain that liminf :c _ >00 /(a;)/a; > lim'mtt^co Pf(t)/t2 > 0. From Eq. (3.51) we obtain that / is bounded below on bounded sets, and so, / being coercive, / is bounded from below. Let (xn) C d o m / be a minimizing sequence of / , i.e. (f{xn)) >• inf / 6 E. Because / is /^/convex we have
\pf{\\xn  xm\\) < \f{xn) + \f{xm)
<\f{Xn) +
 f{\xn + \xm)
\f{xm)Mf
for all n,m e N. Because (tn) > 0 when (tn) C [0,oo[ and (Pf(tn)) > 0, we obtain that (x„) is a Cauchy sequence. As X is a Banach space, there exists x e X such that (xn) »• x. The function / being lsc, inf/ < f(x) < lim/(a; n ) = inf/. Therefore f(x) = inf/, and so 0 € df(x). Replacing (X0,XQ) by (x, 0) in Eq. (3.52) we obtain that VxeX
: P/(z:r)
From this relation it follows immediately that every minimizing sequence of / converges to ~x. Let now x* G X*. Since f — x* is uniformly convex, by what precedes, there exists x € X such that f(x) — (x, x*} < f(x) — {x, x*) for every x E X, i.e. x* G 0/(x). Therefore I m 9 / = X*. D The subdifferential of / G T(X) is onto even in slightly weaker conditions. We call / : X —> E strongly coercive if rima._>00 /(a;)/ a; = oo.
Uniformly convex and uniformly
smooth convex
functions
215
Corollary 3.5.9 Let X be a Banach space and f G T(X). Assume that f is strongly coercive and uniformly convex on every convex and bounded subset of X. Then Im df = X*. Proof. Let us show first that / attains its infimum on X. Indeed, let A > inf/. Since / is coercive, there exists r > 0 such that [/ < A] C rUxSince / is uniformly convex on C := rUx, we have that g := f + LC is uniformly convex and lower semicontinuous. From the preceding result there exists x G X such that g(x) = inf g. Since inf
: f{y) > f{x)+f'{x,yx)+M\\y
~ x\\);
G gidf, Vy € X :
f(y)>Hx) + (yx,x*) + M\\yx\\); (iv) 3^3 G To, V(x,x*),(y,y*)£gvdf Hy)>f{x)
+
(yx,x*)+M\\vx\\);
(v) 3 V > 4 e r 0 , V(x,x*),(y,y*)£grdf {yx,y*
:
:
x*) > ip4(\\y  x\\);
(vi) 3
V (*,*•), (y,y*) Cgrdf
: \\y*  x*\\ >
(vii) 30eN0nn0,
V(x,x*), (y,y*)egrdf
•. \\yx\\ < e(\\y* x*\\);
(viii) (df)1 is singlevalued and uniformly continuous on Imdf; (ix) / * is uniformly smooth; (x) 3 ( n G S i , V ( x , x * ) G g r 9 / , \/y* £ X* : /*(!/*) < r ( x * ) + ( x , y *  x * ) + C T l ( !  y *  x *   ) ;
(3.53)
216
Some results and applications
of convex analysis in normed
(xi) d o m / * = X * and 3
£ grdf
spaces
:
(yx,y*x*)
(df)1(i) O (ix) follows from Theorem 3.5.5. (iii) •£> (x) as in the proof Theorem 3.4.3, with a\ := ipf and il>2 •"= of(x) => (ix) First of all let us note that dom/* = X*. Indeed, in our conditions there exists to > 0 such that of(to) < 00. From Eq. (3.53) we have that Im df+toBx C dom /*. On the other hand, from the Br0ndstedRockafellar theorem we have that dom/* C Imdf (the closure being taken in the norm topology of X*). Therefore dom/* = X*. Since /* is w*lsc, /* is lsc, and so /* is continuous. Let x*, y* e X* and A £ ]0,1[ consider a;^ := (1  A)a;* + Ay*. Suppose thatCTjdly* a;*) < 00. Since Im<9/ = X*, there exists ((un,un)) C grdf such that (u*) —> x\. From the hypothesis we have that /*(**) < f*«)
+ <«n,x*  < ) + ^ ( I K 
/•(1/*) < / * « ) + («n,y*  < ) W I K
x'\\), ~y*\\)
Multiplying the first relation with 1  A and the second one with A, then adding them, we obtain that ( l  A ) / * ( 0 + A/*(y*)
< / * « ) + (un,x*x  < ) + (1 
AMIK
 x*\\) + AdidK  y*\\)
Uniformly convex and uniformly smooth convex functions
217
Since / * is continuous, df* : X* =4 X** is locally bounded (see Theorem 2.4.13); as g r ^ / ) " 1 C gr<9/*, (df)1 is locally bounded, too. It follows that the sequence (un) is bounded. Furthermore, we have that u*  x* \\ ¥ A   j / *  i l l < \\y*x*\\and\\u'ny*\\^(l\)\\y*x*\\ < \\y*  x*\\. Passing to the limit in the above relation and taking into account the convexity of u\, we obtain that (1  A)/*(a;*) + A/*(y*) < / * K ) + 2A(1  A ^ d l i , * 
x*\\).
Therefore /* is 2
f*(yn) < f*(x*)+(yn,yn
 **>,
o < (yn x,x*
y*n)+o2(\\yn  x*\\).
Adding these two inequalities side by side we obtain that f*(y:)
+ (x,y*nx*)+cr2(\\yny*\\)
Vn G N.
Because o2 is continuous on [0, oo[\{to}, where to := sup(dom
there exists cx > 0 such that f is pconvex with p(t) := ^tq
(ii) 3c 2 >0,Vx,2/Gdom/ : f(y)>f{x)
+ f'(x;yx)
+
for
218
Some results and applications of convex analysis in normed spaces
(hi)
3c3 > 0 , V(x,x*)egrdf, f(y)>f(x)
VyGX
+ {yx,x')
(iv) 3c4>0,V(x,x*),(y,y*)egrdf f(y)>f(x)
(vi) 3c6>0^(x,x*),(y,y*)egvdf
+
f\\xy\\9;
+
f\\xy\\q;
:
+ (yx,x*)
(v) 3c5>0,V(x,x*),(y,y*)€grdf
(vii)
:
: (y  x,y*  x") > *f \\x  y\\q ; : \\x*  y*\\>
2
f\\x 
yW^1; c
ij>
there exists cj > 0 suc/i t/tai / * is asmooth, where a(t) := 1—tp;
(viii) 3c 8 > 0, W(x,x*) e g r 3 / , V i / * e r
:
/*(y*) < /*(**) + (x,y*  **) + ^
\\x*  y*\\p ;
(ix) d o m / * = X * and 3 c 9 > 0 , V(x,x*), (y,y*) e grdf
:
(yX,y*X*)<2^\\X*y*\f; (x) dom/* = X*, f* is Frechet differentiable on X* and 3c10>0,Vx*,y*eX*
: V/*(**)  V/*(i/*) < c\oP \)x*  2 / T " 1 .
Proof. The conditions (i)(vi), (vii)(x) of the corollary correspond to conditions (i)(vi) and (ix)(xii) of Theorem 3.5.10. One must only put in evidence the relations among the different constants. These follow from the relations among the gauges in (the proof of) Theorem 3.5.10 and among the constants in Corollary 3.5.7. So, the implications (i) =>• (ii) => (iii) =>• (iv) =>• (v) => (vi) =>• (x) and (viii) =3> (ix) hold, with c2 := c\, c3 := c 2 , d := C3, C5 := a, c% :— C5, cio := 2ce/<7 and eg := eg, respectively. For the equivalence (vii) <S> (i) apply Corollary 3.5.4; one obtains the implications with cj := Ci and c\ := cj. Applying again Theorem 3.5.10, everyone of the conditions (vii)(x) implies that /* is finite, Frechet differentiable on X* and grdf* = (grdf)"1 (the former set being considered for the duality (X*,X**)). So, the equivalence of conditions (vii)(x) with the corresponding relations among the constants C7C10, is obtained applying Corollary 3.5.7. D Remark 3.5.3 For p = 2 we may take the same c^ = c > 0 for all A; 6 1,10 in the preceding corollary, as the proof shows.
Uniformly convex and uniformly
smooth convex
functions
219
In the next result we add some characterizations of uniformly smooth convex functions on Banach spaces which differ slightly from those of Theorem 3.5.6. Theorem 3.5.12 Let X be a Banach space and f £ T(X). statements are equivalent:
The following
(i) / is uniformly smooth; (ii) /* is uniformly convex; (iii) 3Vi € To, Vy* € X*, V(x,x*)
£ grd/ :
/*(y*) > A * * ) + <*>if  **> + iMIIv*  **ll); (iv) 3ax£
Ei, Vy 6 X, V f o z ' ) £ g r d / : f(y) < fix) + (yx,x*)
(v) 3^2 £ To, V(x,x*),
+
(y,y*)egrdf
:
(yx,y*x*)>ip2(\\y*x*\\Y, (vi) d o m / = X and 3
x*)
< cr2{\\y  x\\);
(vii) 3 ^ £ . A o n i V 0 , V(a:,a:*), (y,y*)egrdf
: 111/i > ¥»(y*i*);
(viii) 3 ^ € i V 0 n « o , V(x,x*), ( 2 / , y * ) € g r a / : Hy*  s* < fl(j/  x\\). In (iii) and (v) one can take V'I and tp2 in TQ instead of T0, while in (iv) and (vi) one can take a\ and a2 in S^ instead ofEi, respectively. Proof, (i) O (ii) is obtained in Theorem 3.5.5. Moreover /* is a*convex. Because X is a Banach space and / is lsc conditions (iv) and (vi) coincide with conditions (iv) and (v) of Theorem 3.5.6; so the equivalence of conditions (i), (iv) and (vi) follows using this theorem. (ii) =>• (iii) =$• (v) => (vii) are obvious, with tpi := af, fa := 2fa and
x*) < \\y 
Z 
• Hi,* x*\\<
\\y  x\\ 6(\\y  x\\) = a6(\\y 
x\\),
220
Some results and applications of convex analysis in normed spaces
where a6(t) := t6(t), and so Eq. (3.42) holds. It is obvious that
Uniformly convex and uniformly
smooth convex functions
on bounded sets
221
uniformly convex. Applying the preceding theorem we obtain that X* is reflexive, and so X is reflexive, too. • 3.6
Uniformly Convex and Uniformly Smooth Convex Functions on Bounded Sets
In the sequel we study the uniform convexity or uniform smoothness on bounded subsets. Having / £ A(X) and r > 0, we denote by pf>r, af>r and a°fr the gages of uniform convexity, uniform smoothness and midpoint uniform smoothness of / on rl/x, respectively. We say that / is uniformly convex on bounded sets if / is uniformly convex on rUx for all r > 0. Similarly, we say that / is uniformly smooth on bounded sets if dom f = X and / is uniformly smooth on rUx for all r > 0. We begin with the following auxiliary results. Lemma 3.6.1 lent:
Let f € T(X).
Then the following statements are equiva
(i) / is strongly coercive, (ii) V r > 0 , 3 a 6 i , V i £ X : f{x)>r\\x\\
+ a,
(iii) dom / * = X* and f* is bounded on bounded sets. The statements obtained from (i)(iii) interchanging f, f* and X, X*, respectively are also equivalent. Proof. Since / and /* are proper, convex and lower semicontinuous, they are bounded from below by amne continuous functions, and so they are bounded below on bounded sets. (i) => (ii) Take r > 0; since / is strongly coercive, there exists p > 0 such that f(x) > r \\x\\ for x > p. Since / is bounded from below by some a < 0 on pUx, we have that f(x) > r \\x\\ + a for all x £ X. (ii) =£. (i) Assume that f(x) > r x+a for every x € X; then, obviously, liminfu^n^oo f{x)/ \\x\\ > r. The conclusion is obvious. (ii) <s> (iii) Let g := r +a. We have that f> g & f* < g* — iTux. a <«• f*(x*) < —a for every x* £ rUx* The conclusion follows. The proof of the other statements are similar. • Proposition 3.6.2
Let f
eT{X).
(i) Assume that f is strongly coercive. If f is uniformly convex on bounded sets then f* is uniformly smooth on bounded sets, while if f is
222
Some results and applications of convex analysis in normed spaces
uniformly smooth on bounded sets then f* is uniformly convex on bounded sets. (ii) Assume that f* is strongly coercive. remain valid interchanging f and /*.
Then the assertions in (i)
Proof. We prove only (i), the proof of (ii) being similar. Let / be strongly coercive. Let r > 0 be fixed. Of course, there exists a e l such that f*(x*) > a + 1 for every x* € (r + l)Ux*. Because / is strongly coercive, there exists r' > 0 such that f(x) > (r + 1) a; — a for every x 6 X, \\x\\ > r'. Since for x* 6 (r + l)Ux> and a; > r' we have that (x, x*)  f(x) < (r + 1) x  (r + 1) x + a < a + 1, it follows that Vz* € (r + l)t/ x « : f*{x*)=sup{{x,x*)f{x)\xer'Ux}.
(3.54)
Assume first that / is uniformly convex on bounded sets and take r > 0. The above argument shows that there exists r' > 0 satisfying (3.54). Take t, x*, y*, X, XQ, x*, 70, 71, x0 and xi like in the proof of Proposition 3.5.3(i), but with t e]0,1] and x* € rllx* Then XQ,X* £ (r + l)Ux*, and so, by Eq. (3.54), we may take xo, x\ 6 r'Ux Continuing in the same way like in the proof of Proposition 3.5.3(i), but with pjy instead of p, we obtain that &f*,r(t) < (Pf,r')*(t) for every t e]0,1]. It follows that / * is uniformly smooth on rUx* • Assume now that / is uniformly smooth on bounded sets and take r > 0. As above, there exists r' > 0 satisfying Eq. (3.54). Consider XQ,XI € rllx*, x e r'Ux, y £ X, A e]0,1[ and denote x*x := (1  X)XQ + Xx*. Proceeding like in the proof of Proposition 3.5.3(ii), but with ajy instead of a, and taking into account Eq. (3.54), one obtains that / * is (afy)*convex on rUx*. The conclusion follows from Lemma 3.3.1. • In the next two results we point out the relationships among uniform convexity on bounded sets, uniform smoothness on bounded sets and uniform continuity on bounded sets of the gradient. Proposition 3.6.3 statements:
Let f G T(X) be convex.
Consider the following
(i) / is bounded and uniformly smooth on bounded sets, (ii) / is Frechet differentiable on X = d o m / and V / is uniformly continuous on bounded sets, (hi) / * is strongly coercive and uniformly convex on bounded sets.
Uniformly convex and uniformly
smooth convex functions
on bounded sets
223
Then (i) O (ii) <= (iii). Moreover, if f is strongly coercive then (i) =$• (hi); in this case X* is reflexive (also X is reflexive if X is a Banach space). Proof, (i) =£• (ii) Since / is bounded on bounded sets, by Theorem 2.4.13, / is Lipschitz on bounded sets; since / is uniformly smooth on bounded sets, / is smooth at any x £ X, and so / is Frechet differentiable on X (see Theorem 3.3.2). In order to show that V / is uniformly continuous on bounded sets, let r > 0 and L > 0 be a constant Lipschitz for / on (r + l)Ux Consider x,x' 6 rUx, t G]0,1] and y € Sx Taking into account the inequalities f{x + ty) + f{x  ty)  2 fix) > fix + ty)  fix)  t (y, V/(x)> > 0, and the similar one for x', we have that V/(i)(ty)V/(a;')(ty) < /0r)  fix')\ + \fix + ty)  fix' + ty)\ +  / ( * + ty)  fix)  Vfix)ity)\ <2a°firit)
+
+ \fix' + ty)  fix')  V/(x # )(«y)
2L\\xx'\\.
Hence V/(aO  V/(x') = s u P y 6 S x t  1 \Vfix)ity)
 V/(z')('2/)l
<2t1o°f
+ 1)UX, \\x  x'\\ < t}.
224
Some results and applications of convex analysis in normed
spaces
Then for t € ]0,1], x G rUx and y € Sx we have that
f{x + ty) + f(x  ty)  2f(x) = (f(x + ty)  f{x)) + (f{x  ty) 
f(x))
= V / ( x + 6y)ity)  Vf(x  ffy){ty) = * (V/(x + Oy)  Vf(x < t \\Vfix + Oy)  Vfix
 O'y)) (y)
 9'y)\\ < tu{2t),
where 6,6' € [0, t] are obtained by using the mean value theorem. It follows that a0,rit) < iw(2£) for t 6 [0,1], and so, because V / is uniformly continuous on rUx, f is uniformly smooth on rUx(iii) =J» (i) Because /* is strongly coercive, by Lemma 3.6.1 / is bounded on bounded sets; using Proposition 3.6.2(ii) we obtain also that / is uniformly smooth on bounded sets. (i) =• (iii) (When / is strongly coercive.) Again, by Proposition 3.6.2(i) / * is uniformly smooth on bounded sets, while from Lemma 3.6.1 we obtain that / * is strongly coercive. In this case / * + t r c/ x . is uniformly convex and continuous at 0, and so, by Theorem 3.5.13, X* is reflexive. Of course, if X is a Banach space, X is reflexive, too. • P r o p o s i t i o n 3.6.4
Let f £ T(X). Consider the following
statements:
(i) / is strongly coercive and uniformly convex on bounded sets, (ii) / * is bounded and uniformly smooth on bounded sets, (iii) / * is Frechet differentiable on X* = dom /* and Vf* is uniformly continuous on bounded sets. Then (i) => (ii) o (iii). Moreover, if f is bounded on bounded sets then (ii) =*• (i); in this case X* is reflexive (also X is reflexive if X is a Banach space). Proof, (ii) •& (iii) follows from the preceding proposition applied to /*. (i) => (ii) follows immediately applying Lemma 3.6.1 and Proposition 3.6.2. (ii) => (i) (When / is bounded on bounded sets.) By Lemma 3.6.1 / * is strongly coercive; being also uniformly smooth on bounded sets, using Proposition 3.6.2, we have that / is uniformly convex on bounded sets. In this case, since X* is a Banach space, we obtain that X* is reflexive applying the preceding proposition; so, if X is a Banach space then X is reflexive. •
Uniformly convex and uniformly
smooth convex functions
on bounded sets
225
In the next result we characterize uniform convexity on bounded sets without asking coercivity of the function. P r o p o s i t i o n 3.6.5 Let X be a Banach space and f € T(X). Then f is uniformly convex on bounded sets if and only « / $ / , B ( £ ) > 0 for every t > 0 and every bounded set B C X with B n dom / ^ %, where $},B{t) := inf{/(y)  f(x)  f'{x,y
x)\xeBndomf, 2 / € d o m / , \\yx\\
= *}.
Proof. Assume that / is uniformly convex on bounded sets and consider B C X a bounded set with B D d o m / ^ 0. Let r > 1 be such that B C (r — l)Ux By hypothesis g := f + LTUX is uniformly convex. Hence, from Theorem 3.5.10, there exists ip € To such that g(y) > g(x) + g'(x,y — x ) + i>(\\y ~ x\\) f° r a u %>y £ domp, whence /(y) > /(a;) + g'(x,y — x) + ip(\\y — x\\) for all x 6 BH dom / and y £ rf/x H dom / . But for such x and y we have that g'(x,y — x) = f'{x,y  x). It follows that /(y) > f(x) + f'(x,y — x) + if>(t) for all x £ B 0 d o m / , y € d o m / with y — x\\ = t and £ € (0,1]. Since t9/,s = infx£.Bndom/ $/,x (see page 201 for the definition of tf/.z) and i9/]X € TVi, # / , B S N\, too; in particular $/,# is nondecreasing. But •df>B(t) > i/j(t) for t € [0,1], and so i?/,s(*) > 0 for every t > 0. Conversely, assume that $ftB if) > 0 for every t > 0 and every bounded set B C X with B n d o m / ^ 0. Consider r > 0 such that rt/x n d o m / ^ 0. T h e n / 0 / ) > /(a;) + /'(a;, 2/  x) +tif,rUx (\\y  x\\) for all x <= r a n d o m / and y € d o m / . Taking # = f + huXi w e obtain that g(y) > g(x)+g'(x,y — x) + flf,rUx{\\y ~ x\\) f° r a u ^iJ/ € domg. As seen above, i?/ r s € N\. By our hypothesis we have that flf^B € .Ao; using Exercise 3.12 we obtain that •tp = COI?/,B S TQ. It follows that condition (ii) of Theorem 3.5.10 holds, and so g is uniformly convex. Therefore / is uniformly convex on bounded
sets.
•
We end this section with characterizations of uniform convexity and uniform smoothness on bounded sets in the case X = W1. P r o p o s i t i o n 3.6.6
Let f € r ( R n ) .
(i) Assume that either n — 1 or dom / is closed and /dom/ is continuous. Then f is uniformly convex on bounded sets if and only if f is strictly convex. (ii) Assume that dom / = E". Then f is uniformly smooth on bounded sets if and only if f is differ•entiable (Gateaux or Frechet).
226
Some results and applications of convex analysis in normed
spaces
Proof. (i) Suppose first that / is uniformly convex on bounded sets. Taking into account that pftX is nondecreasing for x G d o m / , it follows that / is uniformly convex at any x G d o m / . Then, as observed at the beginning of Section 3.5, / is strictly convex. Assume now that d o m / is closed and /dom/ is continuous but / is not uniformly convex on bounded sets. Then there exists r > 0 such that rUx H dom / ^ 0 and / + trux is not uniformly convex. Taking into account Exercise 3.3 we have that for some e > 0 and any n € N, there exist xn,yn G rUx H dom / such that \xn — yn\ = 2e and §/(*») + §/(l/») < / {\(Xn + Vnj) + £•
(3.55)
Since (xn) and (yn) are bounded we may suppose that (xn) > x G cl(dom / ) and (yn) —> y G cl(dom/); of course, a; — y\\ = 2e. Because d o m / is closed and /ci(dom/) is continuous, x,y,\{x + y) G d o m / and (f(xn)) >• f(x), (/(l/n)) »• f(y), {f{\{xn + Vn))) »• / (  ( x + y)). Taking the limit in Eq. (3.55) we get \f(x) + \f{y) < f(^(x + y)), and so / is not strictly convex. In the case n = 1, without any supplementary condition, by Proposition 2.1.6 we have that / c i(dom/) is continuous. As above, we get (xn), (yn) C [—r, rjndom / and e > 0 for which Eq. (3.55) holds. Taking x, y € cl(dom / ) the limits of (xn) and (yn), we have that \f{x) + \f{y) < f(\(x+y)). Since  ( x + y) £ int(dom/), the preceding inequality implies that x,y € d o m / , and so / is not strictly convex in this case, too. (ii) Since d o m / = E n , by Corollary 2.2.21, / is continuous, and so / is bounded on bounded sets. On the other hand, because the Gateaux and Frechet bornologies coincide on Rn, Theorem 3.3.2 shows that Gateaux and Frechet differentiability of / coincide in our case. Moreover, by Corollary 3.3.3, V / is continuous when / is differentiable, and so V / is uniformly continuous on bounded sets in such a case. The conclusion follows applying Proposition 3.6.3. D
3.7
Applications to the Geometry of Normed Spaces
The subdifferential of the norm has an interesting geometric interpretation. We have already seen that a hyperplane is a set Hx*ta := {x € X \ (x, x*) = a}, where x* G X* \ {0} and a G R; if dimX > 1 then Hx. 0. One can establish easily (Exercise!) that for
Applications
to the geometry of normed spaces
227
x*,y* G Sx*, a,/3 > 0 we have Hx* 0; then Hx*>a is a supporting hyperplane of Ux atx£ Sx if and only if a = 1 and x* G d\\ • \\(x). Therefore the set of supporting hyperplanes of Ux at x G Sx is {ff x .,i  x* e a • (x)}
(3.56)
(Exercise!). If Ux has a unique supporting hyperplane at every point of its boundary Sx we say that X is smooth; note that this is a geometric property (of the norm), but not a topological property. The dual property (as we shall see later on) is that of strictly convex space. We say that X is strictly convex if
Vx,y€X,x*y,
1^1 = 115,11 = 1
:
I( I + y ) < 1.
Of course, the above condition is equivalent to \{x + y) G Bx for all x,y G Ux, x^y. Throughout this section ip : R+ —>• K+ is a weight function, i.e.
0 => (xn) —> 0 for every sequence (xn) C X. Let x* € X* be fixed. Suppose that /*+(0;:r*) > 0, i.e. there exists fi > 0 such that f*(tx*) > tfj, for every t > 0. It follows that f*(^x*) > \[i for every n € N. Therefore there exists (xn) C X such that • 1, p(\) := /(Ax + (1  X)y) is a lsc convex function and [0,1] C dom<^. By Proposition 2.1.6 (p is continuous on [0,1]. As (p(0) < t < ip(l), there exists A G]0,1[ such that y>(A) = t. Taking z := Xx + (1  X)y the conclusion follows. • 0 such that ip(x) > 0, 2) $(x) > 0 for every x > 0, 3) liminf^xx, y>(x)/x > 0. 1), (ii) was established in [Vladimirov et al. (1978)] and [Aze and Penot (1995)]. Section 3.8: The results on best approximation are classical. Most of them can be found in the books [Holmes (1972); Precupanu (1992)]; the other results are folklore. However we mention that the formula (3.66) for the directional derivative of the distance function was obtained by Lewis and Pang (1998)) in Euclidean spaces. Section 3.9: The implication (a) =>• (c) of Theorem 3.9.1 is the main result of [Soloviov (1993)] (see also [Soloviov (1997)]); the convex case is established (even in a more general framework) in [Asplund and Rockafellar (1969)] (see also a\\  x\\ = a\\x\\,
V n e N : H^ll  Hx*il > 1 n
n
{XniX*)
> 1 (a:,,,^) ^ > f(Xn) > 0. (3.67) n
n
It follows that (x n ,x*) > fi for every n 6 N, which shows that (xn) has no subsequences weakly converging to 0. Suppose that (xn) contains a bounded subsequence {xnk). From relation (3.67) we have that 0 < f{xnk) < \\xnh\\ • IN*/"* > 0, whence, from our hypothesis we get the contradiction xnk —> 0. Therefore an := a;n —> oo. Suppose that limsup n _ +00 an/n < oo. Let tn := y/a^, (—> oo) and yn := ^xn; of course, (llVnll) »• oo. Then 0 < f(yn) < £ / ( x „ ) < %h\\x*\\ >" 0. Therefore (fiVn)) > 0, and so (yn) ^> 0; this contradicts the fact that (2/n) > oo. Therefore lim s u p j j ^ ^ an/n — oo. Let (nk) C N be an increasing sequence such that (ank/nk) > oo. Let us take tk := ankl^/nk (> oo) and t/fc := ^x nfc ; of course (\\yk\\) > oo. Then
o < /(»*) < ^/(* n j < ^  i m = Uz* " °tk
tkUk
JUk
Therefore (f(yk)) »• 0, whence (yk) —>• 0, contradicting the fact that (yk) is not bounded. What we showed above proves that f*'+(0;x*) < 0 for every x* £ X*. Therefore, 0 G (dom/*) 8 . Since /*+(0, •) is sublinear (and finite in this case), we have f*'+(0;x*) = 0 for every x* £ X*. It follows that /* is Gateaux differentiable at 0 and V/*(0) = 0 . D Theorem 3.9.2 Assume that X is a Banach space and f : X —>• R is a proper function with dom df* nonempty and open, the sub differential
246
Some results and applications of convex analysis in normed spaces
being taken for the duality (X*, X**). Then f is convex provided one of the following conditions holds: (i) / is Isc and f* is Frechet differ•entiable at any x* € dom df*, (ii) X is weakly sequentially complete, f is weakly Isc and /* is Gateaux differentiable at any x* £ dom df*. Proof. Let g := /** (for the duality (X, X*)). Because g* = f* is proper we have that g € T(X) and g < f. Using Corollaries 3.3.4 and 3.3.5 for g, in both cases we have that Vf*(x*) 6 X for any x* € dom df*. Let D := {V/*(a;*)  x* G dom df*} C X. Using the preceding theorem, in both cases we have that D C d o m / and f(x) = g(x) for every x £ D. Let x 6 domg. Using the Br0ndstedRockafellar theorem, there exists (xn) C dom<9<7 such that (xn) —> x and (g(xn)) > g(x). Since Imdg C dome?/*, it follows that (xn) C D. Thus f{x) < liminf f(xn) = liminfg(x n ) = g(x), which, together with g < f, implies that f(x) — g(x). As for x ^ domg we have that f(x) > g(x) = oo, we obtain that f(x) = g(x) for all x 6 X, and so / is convex. • Remark 3.9.1 (a) The condition "dom df* is nonempty and open" in the preceding theorem can be replaced by "dom df* = (domdf*)1 ^ 0"; (b) under the conditions of the preceding theorem / is strictly convex on int(dom/). Indeed, when dom df* = (domdf*)1 ^ 0, we have 0 / (dom 3/*)* C (dom/*) 1 = int(dom/*) and /* is continuous on int(dom/*) (see Theorem 2.2.20). Hence, by Theorem 2.4.9, int(dom/*) C dom df* (for the duality (X*,X**)). It follows that dom df* is open. The second statement follows from Exercise 2.32. In the next result the convexity of a closed subset of a Hilbert space is characterized by the differentiability of the square of the distance function. Theorem 3.9.3 Let (X, (•  •)) be a Hilbert space and 0 ^ C C X be a closed set. Then C is convex if, and only if, d^~, is Frechet differentiable. Proof. We identify X* with X by the Riesz theorem. Consider / = i 2 + ic Then f*(y) = sup {(x  y)  \ \\xf \xZC}
= W\y\?\dUy)
= \ \\y\\2  \ inf { \\y  x\\2
\x£C)
Characterizations
of convexity in terms of
smoothness
247
for every y € X. Therefore
4 = llH 2 2/',
(368)
and so dom f*=X and /* is continuous on X. In particular dome*/* = X is open. From Eq. (3.68) we obtain that /* is Frechet differentiable if d2c is so. Assume that d^ is Frechet differentiable; then / * is also Frechet differentiable. Because / is lower semicontinuous (the set C being closed), assertion (i) of the preceding theorem shows that / is convex, and so C = dom / is a convex set. Assume now that C is a convex set. By Theorems 2.8.7 (iii) and 3.7.2 (i) we have that f* = \ \\\\ OIQ and the convolution is exact. Since \  is Frechet differentiable and dom df* — X, by Corollary 2.4.8 we have that / * is Frechet differentiable. D Remark 3.9.2 Formula (3.68) shows that d?c is the difference of two (finite and continuous) convex functions for every nonempty subset C of a Hilbert space; in particular dc (and also dc) has directional derivatives at any point. The next result refers to Chebyshev subsets of Hilbert spaces. Theorem 3.9.4 Let (X, (•  •)) be a Hilbert space and 0 ^ C C X be a weakly closed set. Then the following four conditions are equivalent: (i) C is convex; (ii) C is a Chebyshev set; (iii) dc is Gateaux differentiable; (iv) dc is Frechet differentiable. Proof. The implication (i) => (ii) follows from Theorem 3.8.1, the equivalence of (i) and (iv) follows from the preceding theorem, while (iv) =s> (iii) is immediate. As in the proof of the preceding theorem, consider / = 11 + ic and identify X* with X. Because C is weakly lower semicontinuous, / is weakly lsc, too. Assuming that (iii) holds, from Eq. (3.68) we have that / * is Gateaux differentiable on X* = X. Taking into account that X is weakly sequentially complete, by Theorem 3.9.2 (ii) we obtain that C is convex. Hence (iii) =* (i).
248
Some results and applications of convex analysis in normed spaces
(ii) =• (iii) Assume that C is a Chebyshev set and denote by Pc(x) the best approximation of x G X by elements of C. We shall show that VxeX
: df*(x) = {Pc(x)},
(3.69)
and so, by the continuity and convexity of /*, we obtain that / * is Gateaux differentiable, or, equivalently, d2, is Gateaux differentiable. Indeed, let x € X. Then, by Eq. (3.68), / * (*) = \ \\x\\2 \\\x=
Pc(x)\\2
= (x  Pc(x))
 \
\\Pc{x)f
(x\Pc(x))f(Pc(x)),
and so, by Theorem 2.4.2 (iii), Pc{x) € df*(x) for every x € X. Let now x e Xbe fixed and y 6 df*(x). Take xn := x+n"1 (y — Pc(x)); of course, (xn) —> x. As observed in Remark 3.8.2, Pc is normweak continuous, and so (Pc(xn)) —> Pc(x). Because df* is monotone, we have that 0 < (xn  x  Pc{xn)
 y) = n'1 (y  Pc(x)  Pc(xn)
 y),
whence (y  Pc(x)  Pc(xn) — y) > 0 for every n 6 N. Taking the limit for n » oo, we get  y  P c (a;)  > 0, i.e. y = Pc(x), and so Eq. (3.69) holds for every x E X. O
3.10
Weak Sharp Minima, Wellbehaved Functions and Global Error Bounds for Convex Inequalities
As in the other sections of this chapter, (X,  • ) is a normed vector space. Let / 6 A(X) and 5 C X be a nonempty set. One says that 5 is a set of weak sharp minima if there exists a > 0 such that f(x)>f(x)+ads(x),
VxeX,
Vie5;
(3.70)
this notion generalizes that of sharp minimum point, obtained when S — {x}. It is clear that S C argmin / C cl S if S is a set of weak sharp minima for / . Hence S = argmin/ when S is closed. This is a reason for taking in the sequel S = argmin/. In this case Eq. (3.70) becomes f(x)>udf
+ ads(x)
VxeX,
Weak sharp minima, wellbehaved functions and global error bounds
249
i.e. f is ^conditioned, with ij)a{t) = at for t > 0. Applying Theorem 3.4.3 we obtain a part of the next result, where, as usual, d(x, 0) := oo. Theorem 3.10.1 Let f G T(X) be such that S := argmin/ ^ 0 and a > 0. Consider the following statements:
a:
V x G X : f(x)>mif
+
(ii
Vx*eaUx.
= r(0)
(iii (iv
V x G S , V(x,z*) G g r d f :
: f*(x*)
d(0,df{X\S))
(v Vx* 6 aBx* (X,X*)]; (vi) Vx £X, (vii) VxeS,
ads(x); + L*s(x*); (xx,x*)>ads(x);
>a; '• df*(x*) C 5 [df*(x*) being taken for the duality
Vx ePs(x)
••
f'(x,xx)>ads(x);
1
VM€$^ (A^(5,X))
: / ' ( z , u ) > a u;
(viii) V I £ 5, Vu 6 $ y ( ^ ( S , ! ) ) n c o n e ( d o m /  x ) : f'(x,u) (ix) V x G S , V u e l
> a \\u\\;
: /'(z,u)>ad(u,C(S,z));
(x) Vx e 5, aUx* n iV(S,x) C 0/(x). TTien (i) <£> (ii) =» (iii) =» (iv) O (v), (i) =4> (vi) =• (vii) <3> (viii) and (vi) => (ix) =>• (x). / / X is a Banach space then (v) =£• (i). Moreover, if X is a reflexive Banach space then (vii) =>• (i) and (x) =>• (i); hence all ten conditions are equivalent in this case. Proof. The implications (i) =£ (iii) => (iv) follow immediately from the implications (i) =>• (v) =>• (vi) of Theorem 3.4.3 (taking V1 := V"a defined above), while (iv) £> (v) is obvious. The equivalence (vii) •£> (viii) is also obvious because dom/'(5;, •) = cone(dom/ — x). (i) => (ii) Taking if) = ipa, condition (iii) of Theorem 3.4.3 is satisfied; hence /* (0) e K and A * ' ) < r(0)+i.s(*')+1>*{\\*'\\)
= r(0)+ts{xl+iaUx.
(**) Vz* G X*.
Because for a G S we have that (a,a;*) < f(a) + f*{x*) =  / * ( 0 ) + f(x*), we obtain that /*(0) + t,*s(x*) < f(x*) for every x* G X*. Therefore the conclusion holds. (ii) =$• (i) From the hypothesis, taking again ip = ipa, we have that Wx*eX*
: f*(x*)
+ L*s(x*)+rP#(\\x*\\).
Using the implication (iv) => (iii) of Theorem 3.4.3, the conclusion holds.
250
Some results and applications of convex analysis in normed spaces
(i) => (vi) Take x G X and x G Ps(x). Then for every t G ]0,1[ we have that x G f s ( ( l  t)x + tx) (see Corollary 3.8.5), and so f (x + t(x — x))  f{x) > ads ((1  t)x + tx) = at \\x  x\\ =
atds{x).
Dividing by t and letting M O w e get the conclusion. (vi) =* (vii) Let x G S and u G $ ^ ( ^ ( 5 , 1 ) ) . By Corollary 3.8.5 we have that x G Ps(aT + u). From (vi) we get the conclusion. (vi) => (ix) Let x e S and u € X. From (vi) and Proposition 3.8.3(iii) we have that f'(x,u)
> a • ds(x + u) >a{ds(x)
+ (ds)'(x,u))
>ad(u,C(S,x)).
(ix) =^ (x) Let x £ S. From (ix) and Proposition 3.8.3 (iii) we have that f'(x,u) > a(ds)'(x,u) for every u e X, and so atlx* n N(S,x)
= a • dds(x) = a • {ds)'(x, )(0) C f'(x, )(0) =
df(x).
(v) => (i) when X is a Banach space. Suppose that (i) does not hold. Then there exists x 6 X such that f(x) < inf / + a • ds(x). It follows that x $. S. Therefore there exists a' € ]0, a[ such that f(x) < inf / + a' • ds(x). We fix e s ] a ' , a [ . Using Ekeland's variational principle, we obtain the existence of u € X such that f(u) + e\\u
x\\ < f(x) and /(it) < f(x) +e\\u
x\\ Vx e X. (3.71)
The last relation is equivalent to df{u) D eUx* ^ 0 Hence there exists u* £ df(u) such that u* < e < a. From our hypothesis it follows that df*(u*) C 5, and so u 6 5. By the choice of a' and the first relation of (3.71) we have that s • ds(x) < e • \\u — x\\ < a' • ds(x). As ds(x) > 0, we obtain the contradiction e < a'. Assume now that X is a reflexive Banach space. (vii) =$> (i) Let x G dom / and take x € Ps(x) (such an element exists by Theorem 3.8.1). By Theorem 3.8.4(iv) $x(x x) D N(S,x) ^ 0. It follows that x — x £ &X1 (N(S,x)), and so f'(x,x — x) > a \\x — x\\ = ads{x). The conclusion follows from the inequality f'(x,x — x)< f(x) — f(x). (x) =» (i) Let x G X \ S and take z G Ps(x). By Theorem 3.8.4 (iv), there exists x* G $x(a; — x)n N(S,x). Then u* := a\\x — aT_1(a; — x) G
Weak sharp minima, wellbehaved functions and global error bounds
aUx' n N(S,x),
whence u* G df(x).
251
It follows that
f(x) = f(x) — f(x) > (x — x,u*) = a\\x — x\\ = a • ds{x). The proof is complete.
•
Using the preceding theorem we get the following estimate for the best coefficient in Eq. (3.70). Corollary 3.10.2 Let X be a Banach space and f 6 T(X) such that S := argmin/ ^ 0. Then
Moreover, if X is reflexive, then
inf xex\s
m_ZMi=M(fr d(x,S)
I V
x—x
= inf {f'(x,u)\xeS, — inf
x E dom f\S,
x 6 Ps(x)
x — x\\ '
f'(x,u) d(u,C(S,x))
ueSxn
Qx1 (N(S,
x))}
xes, uex\C{S,x)
Proof. It is obvious that anyone of the numbers appearing in the statement of the corollary is nonnegative. By the equivalences (i) •£> (vi) <4> (vii) o (ix) 4> (iv) of the preceding theorem, if one of them is positive then all of them are equal. The conclusion follows. • Remark 3.10.1
For every nonempty closed convex set A C X we have
d(u,C(A,x))>\\u\\,
yxtA^ue^iNiA^)).
(3.72)
Indeed, we have that A — argmindA and dA(x) > inf dA + 1 • dA(x) for every x € A. From the implication (i) => (vii) of the preceding theorem and Proposition 3.8.3(iii) one gets Eq. (3.72). As we shall see later the notion of weak sharp minima is closely related to that of global error bound for (convex) inequality systems. We introduce now another notion which is related to wellconditioning and weak sharp minima. As seen in Theorem 3.4.3, when argmin / is nonempty, / e T(X) is wellconditioned if and only if (d(0,df(xn))) —> 0 =>•
252
Some results and applications of convex analysis in normed spaces
(d(xn,S)) > 0. So, it is natural to consider a class of functions with a similar property: T := {/ G T(X)  (d (0, df(xn)))
+ 0 => ( / ( i n ) ) »• inf / } .
(3.73)
In the next result we give a characterization of the elements of T. Proposition 3.10.3
Let / G r ( X ) . Then / G T «• F/(t) > 0, Vi > inf/,
where, for t £ i , f 7 (t) := inf {d(O,0/(a;))  * G X, / ( x ) > t} = d{0,df(X\
Proof.
[/ < t))). (3.74)
It is obvious that
/ ^ » 3 ( i
n
) c l
: ( d ( 0 , 5 / ( x „ ) ) )  > 0 , (/(!„)) ^
«*• 31 > inf/, 3(x n ) cX\[f
inf/
: (d(0,a/(x„))) > 0
^ 3 f > i n f / : f/(t) = 0 . Hence the conclusion holds.
•
It is obvious that ff is nondecreasing. Moreover, for t < inf / we have that r~f(t) = d(Q,Imdf); in particular, by Br0ndstedRockafellar's theorem, F/(inf/) = d(0,dom/*) when X is a Banach space. For a more detailed discussion of the position of 0 and dom/* see Exercise 2,45. In order to give other expressions for r/ (t) we introduce several related quantities. First, we mention a simple property which will be used repetitively in this section. Lemma 3.10.4 Let f G T(X), t G R, and x, y G dom / such that f(y) < t < f(x). Then there exists z e]x,y[ such that f(z) = t. In particular \\y  z\\ < \\y  x\\ and \\x  z\\ < \\y  x\\. Proof. Of course, the mapping
Weak sharp minima,
and global error bounds
253
Proposition 3.10.5 Let f 6 T(X) and x £ d o m / with f(x) Consider the following numbers:
> inf/.
/(*)  fiv) 2 / € [ / < / ( * ) ] } , \\x 2/11
7i(x) : = s u p
<^
wellbehaved functions
f / ( » ) t
>,},
— a;
73(a;) : = s u p   / ' (x, yp— 74(2;)
= sup{/'(a;,u) \ue
75(2:)
= d(o,a/(x)),
76 (2;)
= sup {f'{x,
2/ € [/ < / ( * Sx},
u)\ueSxn
N{[f < f{x)],x)}
.
Then 71(2;) = 72(2;) = 73(2;) = 74(2;) = 75(2;) s]0,00]. Moreover, if f is continuous at x and X is a Hilbert space, identified with X* by Riesz theorem, then 71 (2;) = 7 6 (a;). Proof. Let us denote, during the proof, the set [/ < f(x)] by S. It is obvious that 71(2;) > 0, 72(2;) > 0 and 75(2;) > 0 (because 5 ^ 0 ) . Since f'(x,y — x) < f(y)  f(x) < 0 for y G S, we also have that 73(2;) > 0, and so 74(2;) > 0 as, obviously, 74(2;) > 73(2:). Let first t < f(x) be such that [/ < t] ^ 0, and show that sup
fix)  fiv)
Si?) 1
\\xy\\
d(x, [ / < * ] ) "
(3.75)
Consider (yn) C [/ < t] such that (a;  yn\\) > d(x, [f < t\). Then s u p fi*) ~ fiv) > /(*)  /(yn) ;. fix) 1 i/e[/<*] \\x~y\\ ~ \\xyn\\ ~la;2/nll
)
f{x) 1 d(x, [/<*])'
which proves the inequality > in Eq. (3.75). Conversely, let y € [/ < i\. Using Lemma 3.10.4 we get A € [0,1[ such that f(y') — t, where y' :— A2; + (1  X)y. Of course, \\x  y'\\ = (1  A) \\x  y  and t < Xf(x) + (1  A)/(y), whence (1  A) (f(x)  f{y)) < f(x)  t. It follows that / ( * ) _  fiv) \\xy\\
_ (1  A) ifix)  f(y)) (lX)\\xy\\
fjx)  t \\xy'\\
<
<
f{x)  t dix,[f
which proves the inequality < in Eq. (3.75). Hence Eq. (3.75) holds.
254
Some results and applications of convex analysis in normed spaces
Now, by Eq. (3.75), we have 71 (a;) = sup
sup —
t
\\xy\\
r^ = sup t
\
= 72 0*0
(Note that the values of t with [/ < t] = 0 influence neither 71 (z) nor 72 (z)) Let y & S and take u := (y — x)/ \\y — x\\. Since /+(a:, y — x) < f(y) — f(x), we have that (f(x) — f(y)) / x  y\\ < —f'(x,u). It follows that 7i(x) < 73(0;). On the other hand, we have that x + tu E S for t E ]0,   2 /  aril 1 ], and so x f f(x) ~ f(x + tu) 1 u) = sup < ,, ; , rrr i e l O . l l f f  a r i r ] } <Yi(ar), I x  ( a : + tu) which implies that 73 (x) < 71 (x). Hence 71 (x) = 73(2:). It was observed above that 0 < 73(2;) < 74(1). Let u E Sx Suppose first that there exists t > 0 such that y := x + tu E S; then —f'(x,u) = ~f'ixi iyI^[) < 73(2;). In the contrary case f(x + tu) > f(x) for every t > 0, and so f'(x,u) > 0, whence —f'(x,u) < 0 < 73(2;). Hence 73(2;) = 74(a)Assume that 7 £ 1 is such that 74(2;) < 7 (thus 7 > 0). It follows that —f'(x,u) < 7u for all u E X, and so 0 < 7 u + f'(x,u) for u E X. Hence 0 is a minimum point for 7 + f'(x,), and so 0 £ 5 ( 7 INI + /'(*,)) (0). But ,/, fix,
d (7 INI + fix,
•)) (0) = yUx.
+ df'(x, )(0) = 7 tf x  + 8f(x).
(3.76)
Therefore *yUx D df(x) ^ 0, which implies that d(0,df(x)) < 7. Conversely, if d(0,df(x)) < 7 (hence 7 > 0 because 0 ^ df(x)), then 7C/X* n df(x) 7^ 0 (since the norm of X* is u;*lsc and df(x) is iy*closed), and so 0 E "fUx* + df(x). By Eq. (3.76) we obtain that 0 is a minimum point of 7 ll'll + f'(x> •)> which shows that 74(2;) < 7. Hence 74(2;) = 75(2;). Assume now that X is a Hilbert space (identified with its topological dual) and / is continuous at x; then, by Theorem 2.4.9 f'(x,u)=max{(u\x*)\x*
Edf{x)}
Vu£l,
(3.77)
and, by Corollary 2.9.5, K := N([f < f(x)],x) = M+df(x). It is clear that 7e(2;) < 75(2:). Let y E S and take u := (x — y)/ \\x  y\\; it was observed above that 7 :— —f'(x,u) > 0. Then for x* E df(x) we have that
Weak sharp minima,
wellbehaved functions
and global error bounds
255
(u\x*) = (u\x*) > f'(x,u) > 7. Take u = PK(u). By Theorem 3.8.4(vi) we have that (u  u\v  u) < 0 for every v G K. Replacing v by tv with t > 0 we get (u — u\v) < 0 = (u — u\u) for all v G K. In particular 7 < (u\x*) < (u\x*) for all x* G df(x), and so 5 ^ 0. Since P K is Lipschitz with Lipschitz constant 1 [see Proposition 3.8.6(iii)] and 0 = PK{0), we have that u < 1. Taking Mo := u/ \\u\\, we have that u0 G Sx r\N{[f < f(x)],x) and, by (3.77), f'(x,u) <  7 . Hence 7e(») > 7> whence je(x) > 73(a;). • With the aid of the numbers 71 (x)76 (x) we introduce the functions r) :] inf/, oo[> [0,oo],
r)(t) = inf {ji(x) \ x G [/ = t}} ,
i G 176.
Of course, if [/ = t] = 0 then r\ (t) = +00. From the preceding result we obtain immediately Corollary 3.10.6 Let f € T(X). Then r)(t) = r){t) = r^(i) = r)(t) = rhf(i) for every t > inf/. Moreover, if X is a Hilbert space identified with its dual and / is continuous at every point of [f = t] ^ 0 then r\ (t) = r5 (t). In the sequel we shall denote by 77 any of the functions r^r^ defined above. From the preceding proposition we obtain that r/(t) = inf { r / ( « )  s > t } ,
Vt>inf/,
(3.78)
where ff is defined by Eq. (3.74). Example 3.10.1 1) Any coercive function / G T(X) with X a reflexive Banach space is in T; see the solution of Exercise 3.15 for details. 2) Let / : E2 > E be defined by f{x, y) := y/x2 + y2  x. It is obvious that / is a sublinear functional with inf / = 0 and argmin/ = K+ x {0}. One has ff(t) = r/(t) = 0 for t > 0 (see the solution of Exercise 3.18 for details). Consider now the function // : [inf / , oo[> [0,oo] defined by
As usual, we take inf 0 = +00. Hence //(inf/) = 0 if inf/ is not attained; if inf / is attained then argmin / is a set of weak sharp minima if and only if //(inf/) > 0. Even for t > inf / the number // is related to weak sharp minima. Indeed, for t > inf/, taking h : X >• E with h(x) := m a x { / ( i ) 
256
Some results and applications of convex analysis in normed spaces
t, 0} we have that inf h = 0 and argmin/i = [ / < * ] • Then [/ < t] is a set of weak sharp minima for h if and only if lf(t) > 0. This remark suggests having formulas for lf{t) similar to those in Corollary 3.10.2. Proposition 3.10.7 Let X be a Banach space and f G T(X). Consider t € [inf/, oo[ with [/ < t] ^ 0. Then lf(t)=d{0,df(X\[f
(3.80)
Assume now that X is reflexive. Then lf(t) = inf  / ' (y, y ^ y )
xedomf\[f
P[f
(3.81)
= inf {/'(i/.u)  y G [/ = t], u G Sx D S* 1 (JV([/ < f],j/))}
(3.82)
= inf
.
d{u,C([f
ye [/ = *], « a \ c ( [ / < y
(3.83) Moreover, if either (i) i > inf / or (ii) iV([/ < £],&) = B+df(x) x £ [f = t], then It (t) = kf(t)
:= inf  / ' (y, ^ j
for every
y G [/ = t], u G * ^ (3/(2/)) J . (3.84)
Proof. Ii t — inf/ the conclusion is given by Corollary 3.10.2. Assume that t > inf / and denote by h(t), l2(t), h(t), and h(t) the quantities on the righthand side of Eqs. (3.80)(3.83), respectively. Consider h : X > R with h(x) := max{/(i)  t, 0}; we have that inf h = 0 and argmin/i = [/ < £]. The equality of lf{t) and li(t) follows directly from Corollary 3.10.2 because U := X \ [f < t] is open and f\u = h\u +1, and so df(x) = dh(x) for every x e U. Since, obviously, //(£) = /^(O), we apply again Corollary 3.10.2 for h. Hence, in order to obtain Eqs. (3.81)(3.83), it is sufficient to observe the following: a) Let x G d o m / \ [/ < t] and y G P[f
Weak sharp minima,
wellbehaved functions
and global error bounds
257
c) Let y £ [/ = t] and u G X \C([f < t],y). Because u $. C([f < t],y), we have that y+su £ [f < t] for every s > 0. Hence f(y+su) = h(y+su)+t for all s > 0. If y + su £ d o m / for s > 0 then f'(y,u) — h'(y,u) = oo. Assume that y + sou € dom / for some so > 0; as above we obtain that f(y) = t and so, once again, f'{y,xy) = h'(y,xy). Let us prove now relation (3.84). The case (ii) is an immediate consequence of Eq. (3.82). So, let t > inf/. Because df{y) C N([f < t],y) for every y G [/ = t], we have that kf(t) > l3(t) — lf(t). Let now x G d o m / \ [/ < t] and take y € P[f
iz — x\\2 subject to f(z) — t < 0,
and the Slater condition holds, we have that 0 € 3 (    • — x\\2 + \(f — t)) for some A > 0, i.e. y is a minimum point of — x 2 + A(/ — t). Hence $x(x — y) f)d(Xf)(y) ^ 0. Assuming that A = 0, we obtain that \\y — x\\ < \\z — x\\ for every z G dom / . Taking z := x we obtain the contradiction \\y — x < 0, i.e. y = x. Therefore A > 0. Then u :— \~1(x — y) e $ ^ {dfiv)) ^ follows that I fit) = l2(t) > kf(t). The proof is complete. • Note that the equality N([f < t],x)  R+df(x) for x G [/ = t] is not guaranteed for arbitrary / G T(X) when t > inf / (compare with Corollary 2.9.5). However it holds if / is continuous at x £ [f = t]. Relation (3.80) shows that // is nondecreasing on [inf/, oof when X is a Banach space. In fact If has this property in any normed vector space. Proposition 3.10.8
Let f € T(X) and inf/ < tx < t2 < oo. Then
J/(*0 < J/(*2)
and
lf{tx)
In particular If is nondecreasing on [inf / , oo[. Moreover, if X is a reflexive Banach space then rf{t) < lf(t) for every t €]inf/, oo[, and so rf is nondecreasing on ] inf/,oo[; hence rf = ff in this case. Proof. then
Let first x G d o m / and take t\,t2
€ K such that ti < t2 < f(x);
d(x, [f < h})  d(x, [/ < t2])'
(3.85)
258
Some results and applications of convex analysis in normed
spaces
It is obvious that Eq. (3.85) holds if [/ < *i] = 0. Assume that [/ < t{\ f 0 and take x G [/ < *i]. Let A := (/(x)  t2)/{f(x)  h) G ]0,1[. Then / (Ax + (1  A)x) < A/(x) + (1  A)/(x) < A^ + (1  A)/(x) = t2. Hence Ax + (1  A)x G [/ < t2], and so d(x, [f < t2]) < A x  x. Taking the infimum for x G [/ < ii] we get Eq. (3.85). The inequality Eq. (3.85) is obvious if x ^ dom / and [/ < t\] =^ 0. Using Eq. (3.85) we obtain that inf
. / y  \
x6dom/\[/<*i] d(x, [f < ii])
< <
inf
*&*
x6dom/\[/
. . ml
fjx) ~ t2 —.—rr,
x€dom/\[/
and inf , / ( ?  * < inf ^  4 l iedom/\[/
Weak sharp minima, wellbehaved functions and global error bounds Since z G [/ < t] is arbitrary, we get rf(t) = rf(t) < lf(t).
259 •
Corollary 3.10.9 Let X be a reflexive Banach space, f € r ( X ) and t > inf/. / / / is Gateaux differentiable on [f = t] C int(dom/) then If{t) = rf(t) = inf{V/(x) \xe[f
= t}}.
(3.86)
Proof. If [/ = t] — 0 then, obviously, r/(t) = //(£) = oo, and so Eq. (3.86) holds. Let [/ = t] be nonempty and take x0 G [/ < t] and a; G [/ = £]. Since x G int(dom / ) we obtain that }xo, x] C int(dom / ) by using Theorem 1.1.2(ii). As / is continuous on int(dom/), it follows that ]a;o,a;[c int[/ < t]. Because x e[f = t], x $ int[/ < t]\ hence, by Theorem 1.1.3 applied for x and [/ < t], we get u* G Sx* n N([f < t],x). Let u € ^ ^ ( u * ) . Taking into account relation (3.82) we get V/(a;) > (u,Vf(x)) = f'(x,u). It follows that r / ( t ) = r 5 ( t ) = i n f {   V / ( a : )    are [/ = *]}
> inf {f(x,u) \xe[f = t],ue Sxn^x1 (N([f < tlx))} = '/(*)• Using Proposition 3.10.8 we obtain that Eq. (3.86) holds.
•
One says that / £ T(X) is well behaved (asymptotically) or has good asymptotic behavior if lf(t) > 0 for every t > inf/. The next result shows that the class T of functions defined in Eq. (3.73) coincides with the class of functions with good asymptotic behavior when the space is reflexive. Theorem 3.10.10 Then f e?
Let X be a reflexive Banach space and f G T(X).
& rf{t) > 0 V i G ] i n f / , o o [ O lf(t) > 0 Vi G]inf / , oo[.
Proof. The conclusion is an immediate consequence of Proposition 3.10.8 and relation (3.78). • As mentioned before, the set argmin/ is a set of weak sharp minima of / (i.e. / is wellconditioned with linear rate) exactly when inf/ is attained and //(inf/) > 0. One can ask if there are other relations between wellconditioning and wellbehaving of convex functions. Proposition 3.10.11 Let f G F(X). If f is wellconditioned then f has good asymptotic behavior. Moreover, limtj.inf f(t — inf / )  1 i / ( £ ) = oo.
260
Some results and applications of convex analysis in normed spaces
Proof. Let S := argmin/ = [/ < inf/] ^ 0. Of course, if d o m / = S it is nothing to prove (lf{t) = oo for all t > i n f / ) . We may suppose that inf/ = 0 and d o m / ^ S. Since / is well conditioned, by Proposition 3.4.1, the function ip := ipf € AQ f~l Ni; moreover, because d o m / ^ S, tp is finite at some to > 0, and so ip € CIQ. Therefore, by Lemma 3.3.1, ipe € flo n Ao The choice of V> shows that f(x) > rp (d(x,S)) for every x e X. Let t > 0 = inf / and a; 6 d o m / \ [/ < *]. Taking a > 0, we have either d(x,5) > a, and so ^ g g ) > ^ g ^ ' g ^ > ^ (because i> e NJ, or d(x,S) < a, and so ,v x L > K Hence d(x,S)
f
d(x, S)
— a'
. ft 1>(a
> sup < mm —, I \a a
inf {a > 0  ip(a) > t}
a > 0} > sup
a > 0, V(a) > t
t ipe(t)
Taking si = 0 and s 2 — t in Eq. (3.85), we have that f(x)t ^ f{x) > d{x,[f
. * > ~ VeW
It follows that lf(t) > t/rpe(t) > 0 for every t > 0, and so / is wellbehaved. Furthermore, t~xlf{t) > l/ipe(t), and so limt4.o^12/(£) = oo because ipe £ QQ. D It is obvious that any function / € T(R) has good asymptotic behavior, but, for example, / := exp is not well conditioned because argmin/ = 0. The next result furnishes a sufficient condition on the function If in order that / be well conditioned. Theorem 3.10.12 Let X be a Banach space and f 6 T{X) be such that inf/ >  o o . Let If be defined by Eq. (3.79). If'//(<) > 0 for every t > inf/, i.e. f has good asymptotic behavior, and there exists a > 0 such that 6(a) < oo, where 8{s):
_ rs+mt' '" /inf/
dt lf(t)
Vs>0,
then f is wellconditioned and ipf > 6e. Proof. We may assume that inf / = 0. If / is constant on dom / there is nothing to prove. Because // is nondecreasing and positive on ]0,oo[ (eventually taking the value oo) and 6(a) < oo, we have that 6(s) > 0
Weak sharp minima,
wellbehaved functions
and global error bounds
261
for every s > 0; we set 0(0) := 0. It follows that 6 is continuous and nondecreasing on K+; 6 is even increasing on / ( d o m / ) . It follows that 6 G A) f~l iVo fl fio; and so 8e G Ao (by Lemma 3.3.1). Let xo G d o m / with f(xo) > 0 and p € ]1, oo[. Consider the sequence (s„) C P, sn :— p~nf(x0) for every n G NU{0}. Then the series J2n>i *1~(g~)n ls convergent. Indeed, If being nondecreasing (see Proposition 3.10.8), //(s n ) < lf{t) < Z/(s n _i), and so l/Z/(s„_i) < l/lf{t) < l///(s„) for s n < £ < s„_i and n G N. Hence Sn—1 ~ $n ^ T—; r <
I /
at Sn—1 T ^ T < ~T?
Sn $n $n+l N— = P  T  ? \—
w RT VnGN.
//(Snl) 7 Sn //(*) lf(sn) lf(sn) 15 Therefore the series ^ n > 1 ^"r ^ is convergent and its sum is smaller than r«o Jo
dt i,(t)
The relations above show that ^\
2^n>l
S n _ i — Sn _
lf(sn)
/v^
S n  1 — S n _ Sp —
~P\^n>l
lf(sni)
Si\
If (S0) J
dt _ so — si ) < oo. _^1_ ^L_1L
lj{t)~
(3.87)
lf(sQ)
Let s > 0 be fixed. Since so = /(^o) > si > inf / , from the definition of //, there exists x[ G [f < si] such that x 0 — a^H < (s0 — si)/Z/(si) + 2 _ 1 e. By Lemma 3.10.4 there exists xi G [zo^i] with /(a;i) = Si; of course, ll^o — xi\\ < (so — s i ) / ' / ( s i ) + 2 _ 1 e. Continuing in this way we find a sequence (a;n)n>o C d o m / such that f(xn)
= Sn,
\\xn  Xn+1\\
< y^  r  + —r l s f\ n+l) % +
V 71 > 0.
(3.88)
Prom Eq. (3.87) we obtain that the series ^2n>0 \\xn — xn+i\\ is convergent, and so (xn) is a Cauchy sequence. As X is a Banach space, the sequence (xn) converges to some x G X. It follows that 0 = inf / < f(x) < liminf f(xn) < lims„ = 0. Hence x G 5 := argmin/, and so S ^ 0. Using Eq. (3.88) we obtain that Si—\
S{
11*0  xn\\ < ^i=i Wx^  Xi\\ < £ . = 1 [ijr^1 ^ H S i  i  S i l l S ^ ^
whence, by Eq. (3.87),
.
( r° dt
soSl\
c
+ 2i
r° dt dt + e.
262
Some results and applications of convex analysis in normed spaces
As e > 0 and p > 1 are arbitrary, we obtain that d(xo,S) < \\XQ — x\\ < 6(s0) = 0 (f(x0)). It follows that f(x0) > 6e {d{x0,S)). As x0 G d o m / \ S was arbitrary and 6e G AQ, f is well conditioned. • Consider now the system of inequalities 9i(x)<0, where gi,...,gm
fori = l,...m,
(3.89)
£ A(X). We assume that the solution set C := {x G X  gt(x) < 0, Vi € T7m}
of the system (3.89) is nonempty. We are interested in the following global error bound for (the solutions set) C: dc{x) < a • max [gi(x)]+
Vx G X,
(3.90)
l
for some a > 0 (which, of course, depends on the functions gi), where 7+ := 7 V 0 for 7 € E. Of course, g := maxj^j  i € l , m } € A(X), the system (3.89) is equivalent to g{x) < 0,
x e X,
(3.91)
and C = [g < 0]; Eq. (3.90) becomes dc(z) < a • [g(x)]+ V x G X .
(3.92)
Taking /i := g+ := g V 0, we have that inf h — 0 (under our assumption that C / 0) and C = argmin/i. Hence the global error bound for C holds if and only if C is a set of weak sharp minima for h. On the other hand Eq. (3.92) is equivalent to lg(0) > a  1 , and so the global error bound for C holds if and only if [g < 0] ^ 0 and / 5 (0) > 0. In the next result we collect several sufficient conditions for the existence of a > 0 verifying Eq. (3.92). T h e o r e m 3.10.13 Let g G T(X) be such that C := [g < 0] ^ 0. Consider the following conditions: (i)
there exists u G X such that goo(u) < 0;
(ii)
there exists n > 0 such that sup x€ r
(hi)
there exists n > 0 such that sup xe r fl=0 i d(x, [g < —rj\) < 00;
<0 i
d(x, [g < —n]) < 00;
Weak sharp minima,
wellbehaved functions
and global error bounds
263
(iv) inf
sup
inf
">°x&[g=o]y£[9<n]
J t M = inf g(y)
(v) 0 > infff andrg(0) (vi)
sup ^ l i ^ ^ l
I>O I € [J=O]
< oo;
(3.93)
V
> 0;
Z 9 (0)>0.
Then (i) =4> (ii) =» (iii) <$• (iv) =^ (v) and (ii) => (vi). If g is not constant on domg and [g < 0] C int(domg) i/ien (iii) => (ii). If X is a reflexive Banach space, then (v) =4> (vi); moreover, if g is not constant, [g = 0] C int(domg) and g is Gateaux differentiate on [g = 0] then (vi) =>• (v) andlg(0) = rg(0). Proof. First note that the equality in Eq. (3.93) follows from Eq. (3.75) when [g = 0] ^ 0. When [g = 0] = 0, both sides of the equality are —oo. The implications (ii) =s> (iii) & (iv) are obvious. (i) => (ii) Let us take u € Sx such that 0. Consider a: € [p < 0]. Taking 7 := —r)/goo(u) > 0, we have that g{x + 7U) < g(x) + 75oo(«) < 75oo(u) = ~V, and so d(x, [5 < —n}) < \\x + 7U — a; = 7 < 00. (iii) =£• (ii) Assume that g is not constant on domg and [g < 0] C int(dom<7). We shall show that sup{d(x, [g < rj\) I x € [g = 0]} = sup{d(x, [3 <  I J ] )  a; € [3 < 0]} > 0 (3.94) for any n > 0. If [p < 0] = 0 then [p < 0] = [g = 0] and Eq. (3.94) is obvious. Assume that g(xo) < 0 and take x £ domg with g(x) ^ g(xo); we may assume that g{x0) < g{x). If g(x) > 0, by Lemma 3.10.4, there exists y e]x0,x] such that g(y) = 0. Assume that g(x) < 0. Since 0 < g(x) — g(xo) < t_1 (g{xo + t(x—xo))—g(xo)) for t > 1, there exists t > 1 such that p(xo+t(a;—xo)) > 0. The set A := {* > 1  g(xo + t(x — XQ)) < 0} is a closed bounded interval; take t := max A. Since XQ +t(x — xo) G [g < 0] C int(domg), there exists t > t such that xo + t(x — x0) € dom g. It follows that g(x0 + t(x — x0)) > 0, and so necessarily xQ + t(x — xo) € [3 = 0]. Hence there exists y E [g = 0] such that x Z.\x§,y\. Let us prove Eq. (3.94) for n > 0. This relation is obvious if [g < —rj\ = 0. Assume that [g < —rj\ •£ 0 and take x £ [g < 0]\[g < —rj\. Consider
264
Some results and applications of convex analysis in normed
spaces
x
o £ [9 < — rj] As above, there exist y £ [g — 0] and A e]0,1[ such that x — Xy + (1  A)xo. Take now z 6 [g < rj]; then Az + (1  A)xo £ [g < rj], and so d(x, [g < rj]) < \\x  Az  (1  X)x0\\ = A \\y  z\\ < \\y  z\\, whence d(x, [g < rj]) < d(y, [g < r)]). It follows that sup x€ [ 9=0 ] d(x, [g < —rj]) > sup x£ [ 5<0 ] d(x, [g < —rj]). As the converse inequality is obvious, Eq. (3.94) holds. Hence (hi) =$> (ii) in our case. (iv) ^ (v) If 0 = inf g then C = [g = 0] ^ 0 and [g < 77] = 0 for any 77 > 0. Hence d(x, [g < —rj]) = 00 for x £ [g = 0] and 77 > 0, whence the contradiction inf,,>o sup a . € [ 9=0 i rj~1d(x, [g < —77]) = 00. Hence 0 > inf g. The conclusion then follows from the relation r s (0) = r(0) in Corollary 3.10.6 and the obvious inequality • , d(x, [g < 77]) . d(x, [g < rj]) 1 uy u^y 1111 inf usup — ^_> u sup inf — 2n / ~.\ * i>°xe{g=o] V xe[g=o]i>° V r g{0) " "
t
(ii) => (vi) Because sup a . 6 [ a<0 ] d(x, [g < —rj]) < 00, we have that [g < —77] 7^ 0. Consider 0 < /x < 77. Since [g < —/j] C [g < —rj], we obtain that su Pxg[0<ji] d(x, [g < —rj]) < 00. From the implication (ii) =>• (v) with 0 replaced by — fi we obtain that rg(—fj,) > 0. Using Proposition 3.10.8 we obtain that / 9 (0) > rg(fj.) > 0. Assume now that X is a reflexive Banach space. Then the implication (v) => (vi) is true because, by Proposition 3.10.8, lg(0) > rg(0). (vi) => (v) Assume that g is not constant, / s (0) > 0 and g is Gateaux differentiable on [g — 0] C int(dom #). It is sufficient to show that 0 > inf g; then the conclusion follows from Corollary 3.10.9. If [g = 0] = 0 then 0 > inf g (because C ^ 0). Assume that [g = 0] ± 0; if [g < 0] = 0 then every x € C is a minimum point of g, and so 0 S dg(x) — {Vg(x)}. Assuming that g is 0 on dom<7 then [g = 0] = doing = X (domg being closed and open), a contradiction. It follows that domg \ [g < 0] ^ 0, and so, by relation (3.81) in Proposition 3.10.7, / 5 (0) = 0. Hence [g < 0] ^ 0.D Coming back to the system (3.89), several conditions in the preceding theorem can be formulated in terms of the functions gi, i G l,m, when g := max i € j^g { . Let us denote the set {j e l,m  gj(x) = g(x)} by I(x) for x € domg. Assume that gi G A.(X) is continuous on int(dom<7;) and 0 ¥• [9 < 0] C int(domgi) for every i e l,m. Then, by Corollary 2.8.13, we
Weak sharp minima,
wellbehaved functions
and global error bounds
265
have that
dg(x) = co (U i 6 / ( B ) d9i^)
Va: e
b < 0],
and so, by Theorem 2.4.9, g'{x,u) = max max{(u,x*) \ x* 6 dgi(x)}
\/x G [g < 0], Vu G X.
iei(x)
Moreover, since epi^ = f)iei^epigi ¥" 0, the relation 500 = max.ieY^(9i)oo holds. For example, condition (i) of the preceding theorem reduces to the existence of u € X such that (gi)oo(u) < 0 for every i £ 1, m. The condition [g < 0] 7^ 0 in (iv) says that the Slater condition holds for the system of inequalities (3.74), while the condition r s (0) > 0 may be described in several ways, taking into account that rg = r3g for 1 < j < 5. One of them (corresponding to rjj) is
0 d
^ (u, e[ , = o ] «>(U 6/{ , ) ^(*))).
another one (corresponding to rg) being inf
sup min
xe[g=0] u€Sx
«£/(x)
min
(u,x*) > 0.
x'edgi(x)
We can consider (3.91) as being an unconstrained inequality, in contrast with g(x) < 0,
xeA,
(3.95)
where g € T(X) and A C X is & nonempty closed convex set. This inequality is equivalent with the system g(x) < 0,
dA(x) < 0,
x£X.
(3.96)
Of course, the Slater condition does not hold for the system (3.96). We assume that the solution set C := {a; € A  g(x) < 0} of the system (3.95) is nonempty. We say that the global error bound holds for (3.95) if there exists a > 0 such that dc{x)
Vx£l.
(3.97)
Theorem 3.10.14 Let X be a reflexive Banach space, g € T(X) and A C X be a closed convex set such that C := {x £ A \ g(x) < 0} is
266
Some results and applications of convex analysis in normed spaces
nonempty. Assume that one of the conditions below holds: PA{x) n domg ^ 0 \/x£[g<
0],
(3.98)
[g < 0] n A C int(doms).
(3.99)
Then the following statements are equivalent: (i) there exists a > 0 such that Eq. (3.97) holds, (ii) 3 7 > 0 , V i e AD[g = 0], Vu 6 $^(iV(C,2;)) : max{<;'(a;,u),d(u,C(j4,a;))} > 7u • Proof.
Consider /i := max{<7,oU}. By Proposition 3.10.7 we have that
lh(0)=mt{ti(x,u)
u£Sxn^1(N(C,x))}.
\x£C,
Let x £ C; we wish to evaluate h'(x, u) for u € X. Because domh'(x, •) = doing'(x, •) = cone(dom<7 — x), we take u £ cone(dom p — x). Let first x £ [g = 0] n A. Taking into account Proposition 3.8.3(iii), we have that 9ix h\x,u)=m^jlim v ' [no
+ tu) 9{x
 \lirndA{x t t\a
= m&x{g'(x,u),d(u,C(A,x))}
+
tu) dA{x)
t

\ J
.
Let now x £ [g < 0] D A. Since u £ cone(domg — x), there exists t' > 0 such that x' := x + t'u £ doing. As 0 such that x + tu £ [g < 0] for t £ [0,1]. It follows that ,,. max{g(x+ h (x, u) = lim = d(u,C(i4,a;)).
tu),dA(x+
tu)\
dA(x = hm —
+tu) (3.100)
Let x £ [g < 0}nA and u £ S^fl*^ 1 (_/V(C,a;))ncone(dom0a;). Let us show in our conditions that h'(x,u) > 1. As seen above, there exists t > 0 such that x + tu £ [g < 0] for t £ [0,1]. By Corollary 3.8.5 we have that x + tu £ C and x £ Pc(x + tu) for t > 0. It follows that z + iu ^ A. Assume that x £ PA(x + tu). Suppose first that Eq. (3.98) holds. Then there exists x £ PA(x + tu) n domp. Of course, \\x + iu — x\\ < \\x + tu — x\\ — t. Since g\[xjX] ls continuous and g(x) < 0, there exists A G]0,1[ such that
Weak sharp minima, wellbehaved functions
and global error bounds
267
g(Xx + (1  X)x) < 0. Of course, x' := Xx + (1 — X)x 6 A, and so x' G C. The contradiction follows: t  d(x + tu,C) < \\x + tux'\\
< X\\x + tux\\
+ (lX)\\x
+ tux\\
< t.
Suppose now that Eq. (3.98) holds. Because x fi PA(X + tu), there exists x E A such that a; + tu  x\\ < \\x + tu x\\ = t. From Eq. (3.98) we have that x G int(dom #), and so there exists A G ]0,1[ such that Xx + (1 — X)x G dom g. Decreasing A if necessary, we may assume that g{Xx + (1  X)x) < 0. Proceeding as above we get the same contradiction. Hence x G PA(X + tu); using Eqs. (3.100) and (3.72) we obtain that h'+(x,u) > 1. Concluding the above discussion we obtain that /x > /^(O) > min{l,/u}, where H := inf { maac(g'(x,u),d(u,C(a,x)))
\ x G [g = 0] fl A, u£Sxn$x1(N(C>x))}
The conclusion follows.
•
In the next result we present two sufficient conditions in order that the global error bound holds for the constrained system (3.95). Proposition 3.10.15 Let X be a reflexive Banach space, g G T(X) and A be a closed convex subset of X such that C := [g < 0] fl A ^ 0. Assume that either C C int(domg), or (3.98) and A fl [g = 0] C int(dom<7) holds. Consider the following statements: (i) (ii)
there exists a > 0 such that Eq. (3.97) holds, 3 K > 0 , V Z G An[g
= Q], Vi* G N(A,x),
Vy* G dg(x)
11**11 + 1 < K   Z * + ^ U .
:
(3.101)
(iii) the set dg {A PI [g = 0]) is bounded and
0 1
^ (U^ n[fl=0] (^()+^(^)))
Then (iii) =^ (ii) =^ (i). Proof. It is obvious that Eq. (3.98) or Eq. (3.99) holds, and so conditions (i) and (ii) of the preceding theorem are equivalent. It follows that everyone of the statements (i), (ii) and (iii) above is true if A n [g = 0] — 0, and so the conclusion holds in this case. Assume that AD[g = 0] ^ 0; in our conditions AC\ [g = 0] C int(domg).
268
Some results and applications of convex analysis in normed spaces
(hi) => (ii) By hypothesis, there exist 7,77 > 0 such that x* + y*\\ > 7 and \\y*\\
= N(A,x)+N([g
+ M+dg(x)
(3.102)
for every x £ A fl [g = 0]. Let x £ A n [g  0] and u £ Sx n * ^ ( J V ( C , i ) ) . As observed in the proof of the preceding theorem, h'(x,u) — max.{g'(x, u),d(u,C(A,~x))}, where h := m a x f p , ^ } . Since 5 and d^ are continuous at x, h is so. It follows that h'(x,u) = max{(u,u*)  u* £ dh(x)}. From Corollary 2.8.13 and Proposition 3.8.3(h) we have that dh(x) = co {dg(x) U dA(x)) = co (dg(x) U (Ux* n N ( A , z ) ) ) . Take u* £ $ x ( u ) n N(C,x). From Eq. (3.102), there exist x* £ N(A,x), y* £ dg(x) and A > 0 such that u* = x* + Xy*. Since u* ^ 0, fi := a;* + A > 0. It follows that / i ~ V £ dh(x). On the other hand, by Eq. (3.101) we have that 1 = u* = \\x* + Xy*\\ > K _ V Hence max.{g'(x,u),d(u,C(A,x))} The proof is complete.
= h'(x,u) > ( u , ^ _ 1 u * ) = / J _ 1 > K _ 1 . •
During the proof of the preceding result we obtained formula (3.102) under the hypothesis that A D int[
Monotone multifunctions
269
Proposition 3.10.16 Let g £ A(X) and C := {x £ A \ g(x) < 0}. Assume that the system (3.95) is metrically regular at x £ An [g = 0], i.e. there exist 5,7 > 0 such that jdc{y)<max{g(y),dA(y)}
Vy£B(x,8).
(3.103)
Ifx £ (domgY, then Eq. (3.102) holds. Proof. The inclusions D are easy (and true always). Conversely, let x* £ N(C,x) C\ Sx> By Proposition 3.8.3(H) we have that x* £ ddc{x), and so 72;* £ d(h + (•B(:C,5))(X) = dh(x), where h = max.{g,dA} (see Remark 2.4.1). By Corollary 2.8.11, dh
(*) = U Ae[0ll] K1  A )^( a ; ) + A W ) ;
the equality d(A#)(a;) = \dg(x) holds also for A = 0 because a; £ (domg)\ Using again Proposition 3.8.3(h) we obtain that x* £ N(A,x) + M+dg(x). Hence (3.102) holds. • A particular case of Proposition 3.10.16 is when g — ds with B c X a nonempty closed convex set. Then Eq. (3.103) implies that N{A DB,x)
= N(A,x)
+
N(B,x).
(Compare with Corollary 2.8.4 for X = Y and A =
3.11
ldx.)
Monotone Multifunctions
Consider the subset SE '•= \ fJ '• E —> [0, oo[ supp/x is finite, V^
fi(x) = 1 >
of E B , where £? is a nonempty set and supp/j := {x £ E \ fi(x) ^ 0}; if A C E is a nonempty set, then
270
Some results and applications
of convex analysis in normed
spaces
In the sequel we take E = X x X*. Consider also the functions P : tox*. »• X,
p(/i) := £ ( ^ .
. ^ , * * ) " x,
) 6 M
q : ttxX >• X*, qM := Y,(x,x.)eXxXAx>x*) r : ftxx  • E,
r(/x) := £ ( B i B . ) 6 J f
XJC .M*,**)
' x* > " <*'**>'
Note that p, q and r are affine functions, i.e. p (Xpi + (1 — A ) ^ ) = <\p(/Ui) + (1  A)p(/x2) for every /ii,/i 2 € ?xxJf, A e [0,1], and similarly for q,r. The following characterization of monotone subsets of X x X* holds. Lemma 3.11.1 and only if
Let M C X x X* be a nonempty set; M is monotone if
V>e
r(n)>{p(n),q(n)).
Proof. Suppose that M is monotone and let ji € <;M', then supp/x = {(xi,xl),..., (x n ,a;*)} for some n G N. Take a* := n(xi,x*) for 1 < i < n. Then:
Hi*) ~ (P(M).?(/*)) = ^2i=1ai(xi,xi)
Conversely, taking (x±,x\), CM, we have that
~ \J2i=1aiXi'J2
{x2,xl)
a x =1
J j)
E M and /J = \&(Xi,x\) + ^ ( x 2 , ^ ) €
KA»)  (P(M),«0*)) = I ( ^ i ^ i ) + I (Z2,Z2>  ( l ^ i + \xi, \x\ + \x*2) = \{xi x2,x\ x2) > 0, and so M is monotone.
•
To M C X x X* we associate the function ™ XM:X^R,
, s XM(X)
(x,q(u)) — r(u) :=
sup
P6CM
v
'
y v w /
v w
l + lb(M)ll
'
Monotone
multifunctions
271
It is obvious that XM is a lsc convex function as supremum of a family of continuous affine functions. In the sequel domM denotes P r x ( M ) . Lemma 3.11.2 Let I / M C X x X* ie o monotone set and M = M  (w,w*), where (w,w*) G X x X* is fixed. Then: (i) d o m M C co(domM) C domxM(ii) domM = domM — w and domx^f = domxM — w. (hi) Let Xo C X be a closed linear subspace such that dom M C Xo and M+{Q}xX£ = M. Consider MQ := {{x,x*\Xo)  (x,x*) G M } C XQxX^. Then XM0 =
XM\X0,
domxMo = domxM C X0
and M is maximal monotone O Mo is maximal monotone (in XQ X XQ ). (iv) Suppose that M is maximal monotone and domM C x0+Xo, where Xo C X is a closed linear subspace and x0 G X. Then M + {0} x XQ = M and domxM C aff(domM). Proof, (i) Let (x,x*) € M and fj, G TM be fixed. For (y,y*) € M, from the monotonicity of M, we have that (y  x, y*  x*) = (j/, j/*)  (j/, a;*)  (a;, y*) + (x,
x*)>0.
Multiplying with (J,(y,y*) for (y,y*) G supp/x, then adding the corresponding relations, we obtain that r
(v) ~
(P(M),Z*>
 (x,q(lJ)) + (x,x*) > 0,
and so \(x,x*)\ + \\p(ji)\\ • \\x*\\ > (x,x*)  (p(fi),x*) > (x,q{n)) ~ r(fi). It follows that XM(X) < max{(a;,a;*),x*} < oo. Therefore domM C domxM! since domxM is a convex set, we have co(domM) C domxM, too.
272
Some results and applications of convex analysis in normed spaces
(ii) It is obvious that dom M = dom Mw. We note that if ju € cjp taking n(x,x*) := Jl({x,x*)(w,w*)) [<£> Jl(x,x*) = fJ,{(x,x*) + (w,w*))], then /J, £ <;M SO, for ]2 e ^ and the corresponding /x € ?M, we have that p(u) = Y^
u(x,x*) x = 5 ^
^
^{x,x*) • (x  w)
= P(M)  w, and similarly ?(£) = gr(/i)w*,
r(Ji) = r ( »  (w,q(n))  {p(iJ,),w*) + (w,w*).
Let a; € X and Jl G Cjg • From the above relations we have that (x  w, q(ji))  r(p,) = (xw,
q{n)  w*)  r(n) + (w, q(n)) +
= {x,q{^)) r(n) <XM(x)(l where /3 :— max.{\(x,w*)\,
+ {p(fi),w*}
+ \\p(jj)\\)+P(l
+ \\p(ji)\\),
\\w*\\}, and so
(xw,q{Ji))r{Jl)
^ r ^
~{x,w*)
(p(n),w*){w,w*)
1 + lbOOH .
n
,
n
/
/ x , flx
< (XM(X) + « 1 +  b 0 i )  < (i + IMIMx*(«) + fl •
Therefore domxM — w C domxjgr • Since M = M + (w, w*), it follows also the converse inclusion. (iii) It is obvious that M0 is a monotone subset of XQ X XQ. (It is wellknown that XQ is identified with {z*x 0 I x* 6 X*} endowed with the norm u* := inf{a:'  x*\Xo = u*}.) Let a; 6 X \ XQ and fix (2:0,^0) £ ^ Since a; — xo ^ ^0, there exists u* 6 X^1 such that (x — x0,u*) > 0. Therefore, taking fj, = <5(Xo,x*+tu*); (a:,^ + £u*)  (X0,XQ + tu*) XM\X)
>
——r.
fj
(x  x0,x%) + t (x  x0, u*) =
——r
n
I + IFOII I + IWI for every t > 0, and so XM(X) = 00. It follows that dom%M C XQ. Let (i € CM; consider Ho : XQ x XX 4 E,
/z0(a:,u*) := V "
fi(x,x*).
It is clear that /io £ CM0 • Furthermore, we have that POo) = p(/x).
90o) = g(/i)Uo and r(/i 0 ) = r(fi).
Monotone multifunctions
273
It follows that for x G X0, (x,q(fj)) r(fi)
1 + IP(/0II
_ (x,q(fj,0))  r(/j,0)
1 + IIP(W))
'
Conversely, let /io G TM0 with supp^o = {(a;i,uj),...,(a; n ,u*)}, n G N. For every i G l , n we take (only) one a;* G X"* such that x*\x0 = u*, and define /i G CM by fi{x,x*) = fi0(xi,u*) if (a;,a;*) = (a:*, a:?), /u(a;,a;*) = 0 otherwise. We note that to this fi corresponds, by the procedure described above, /i 0  Therefore, XM0(X) = XM{X) for every x G X0, i.e. XM0 = XM\X0
Suppose that M is maximal monotone and let (x,u*) G XQ X XQ be such that (y  x,v* —u*) > 0 for every (y,v*) G M0. Then there exists x* G X* such that x*\x0 = u*. It follows that (y  x,y* — x*) = {y — x,y*\x0 — u*) > 0 for every (y,y*) G M, and so (x,x*) G M; hence (x,u*) G M 0 . Suppose now that MQ is maximal monotone and let (x,x*) G X x X* be such that (y — x,y* — x*) > 0 for every (y,y*) G M. Fixing (2/o,2/o) € M > t h e n (yo,Vo + z*) G M for every z* G X ^ . Therefore (7/0 — x,yl + z* — x*) > 0 for every z* G XQ 1 , which implies that x G (XQ 1 ) = X0. We obtain so that (y  x,y*\x0  x*\x0) > 0 for every (2/,2/*) € M, and so (a;,a;*xo) G M). From the definition of Mo there exists z* G X* such that (a;,**) G M and 2*x0 = x*\x0 (& z*  z* € XQ 1 ). From our hypothesis it follows that (a;, a;*) G M, and so M is maximal monotone. (iv) Suppose now that M is maximal monotone and d o m M C i o + ^ O i where XQ C X is a closed linear subspace and xo G X. Consider (a;, a;*) G M and u* G X ^ . Then V(J/,J/*)GM
: (2/  a:,y*  (x* + u')> = (y  x,y*
 a:*) > 0,
because y — x G X0, and so (a;,a;*w*) G M. Therefore M + {0} XXQ = M. Fix xi G domM; consider M\\= M (a;i,0) and Xi := lin(domMi) = aff(domM) — x\. Since M\ is maximal monotone, from what precedes it follows that Mi + {0} x X^ = Mi, while from (iii) it follows that dom XMX C X\. Using (ii) we obtain that domxM C x\ + Xi — aff(domM). D Consider the function h : ttxx x X* > K,
/i(/«,^):='(/*)(p(Ai),a:*>
(3104)
274
Some results and applications of convex analysis in normed spaces
The function h is affine with respect to fi and x*, respectively, and w*continuous with respect to x*. Lemma 3.11.3 Let X be a Banach space and Mi, M2 be nonempty monotone subsets of X x X*. Suppose that 0 G (domxMi  domxM 2 ) J •
(3.105)
Then there exists 7 > 0 such that V(/ii,/i 2 )
GCMJ X < T M 2 , 3 A G
[0,27], (xl,x*2) e l ' x r
: a:I,   ^   < 7,
2
+ h(n2,x*2) + A \\q(fn) + q(ji2)\\ > (A + a:J + x*2\\2 ). (3.106)
h(to,xi) Proof. that
By Proposition 2.8.9, there exists n G (0,1] and v G [l,oo) such
r]U C {x G X I XMi(x) < v, Ikll < ^}  {a; £ I I XM2(x) < v, \\x\\ < v). (3.107) Let 7 := 5v2/r] and fix (/ii,/j 2 ) G ?Mi X ?M 2  Consider first the case in which max{g(//i) ,   Q ( ^ 2 )   } < 7 In this situation we take {x\,x2) — (9(^1)11(^2)) and A = <7(A*I) + g(/i2) From Lemma 3.11.1 we have that h(fii,x*) = r(fii)  (p(fJi),q{Hi)) >0iori — 1,2, and so Eq. (3.106) holds. Consider now the case max{g(/ii), g(/i 2 )} > 7 We may assume that 7 < g(/ii). There exists v £ X such that IMI=7?
and
(v,q(ni))
> 77? = 5v2.
(3.108)
From Eq. (3.107) there exist x,y G X such that XMl{x) < v, There exist also
XM2(y) < v,
2/ < v
and
xy
= v.
0 G X* such that
lk*ll = */
and
(p{m),x*) = v \\p(fii)\\, i = 1,2.
Since 7 > 1/ we have that a;* < 7 for i = 1,2; since XMi(x) < v and a; = w + y, we have that Kfii,xl)
=r(m)
 {p(fii),x{) = ri/ix) + v Wpifi^W > (x,q(ni))  1 /
= (v + y,q(m))
v
= {v,q(fJi)) + (y,q(fii))
v.
From Eq. (3.108) we obtain that h(m,xl)
> 5v2 + (y,q(m))
v>Av2
+ (y,q(fn)).
(3.109)
Monotone multifunctions
Since
XM2{V)
275
< v,
h(H2,x*2) =r(n2){p{H2),X2)
=r{ti2)+v\\p(ii2)\\
> (y,q(fJ^))v
(3.110)
Prom Eqs. (3.109) and (3.110) we obtain h{m,xl)+h(n2,x;)
> 4v2 + (y,q{m) + q{n2))v
> 3i/ 2 i/g(/ii) + g(/i 2 ),
and so 2h(fi1,x*1) + 2h(n2,x*2) + 2v • <jf(/n) + q(jn)\\  v2  \\x*x + x*2\\2 > &v2  v2  \\x\ + x*2f > 5u2  4z/2 > 0. Hence Eq. (3.106) holds with A = v (< 27) in this case.
•
Theorem 3.11.4 Let X be a reflexive Banach space and M\, M2 be nonempty monotone subsets of X x X*. Suppose that condition (3.105) is satisfied. (i) Then there exist (a:*, £2) £ X* x X* and z £ X such that for every (2/1,2/*) € Mi, (2/2,2/2) € M 2 we have 2 (Vl  z,y{  x\)+2 (2/2  2,2/2*  x*2) > *2 + a:I + x*2\f+2 {z,x\ + x*2). (3.111) (ii) If moreover Mi and M2 are maximal monotone then there exist z £ X and Xt 1 X 2 € X* such that (z,x*) £ Mi, (z,x2) £ M 2 and INI2 + I K + Z2II2 + 2 (z, x\ + x*2) = 0.
(3.112)
In particular dom Mi n dom M2 / 0. Proof, (i) Consider 7 > 0 given by the preceding lemma, A := <;MX X ?M 2 , B := {(x\,xl,z) £ X* x X* x X  a;*, i* < 7, ,z < 27} and the function k : A x 5 » E defined by M0*i>M2), (^,0:2,3)) := 2(/i(/x1,a;J) + / i ^ . i a ) ~ 11 I2
11 *
,
(Z,Q(»I)
+ 9(^2)))
* M2
11*11 Ita + ^ll • It is obvious that the sets A, B are convex, and k is convex (even affine) with respect to (^i,/z 2 ) € A and concave with respect to (x{,x2,z) £ B. Furthermore, endowing X* xX* x X with the product topology w* xw* xw,
276
Some results and applications of convex analysis in normed spaces
B is compact and k is upper semicontinuous with respect to Using the minimax theorem (Theorem 2.10.2) we have that
(x\,x\,z).
inf max k = max inf k. A
B
B
A
Let (ni,fi2) 6 A. Using the preceding lemma, there exist A € [0,27] and x\,x2 € X* with :r*, \\x2\\ < 7 such that 2h(n1,x*1) + 2h(n2,x*2) + 2\ • \\q(m) + q(fi2)\\  A2  \\xl + x*2\\2 > 0. Since X is reflexive, there exists z 6 X such that
INI = A, (z,q{ni) + q(fi2)) = A • \\q(ni) + q(^)\\ • It follows that k((fii,fi2),(xl,x2,z)) > 0. Therefore inf^maxsA; > 0. Hence maxsinf^fc > 0, i.e. there exists (x\,x2,z) 6 B such that 2(h(jn,xl) + h(fi2,x*2)  (z,q(l*i) + 9(/*2)» > N l 2 + ll*i + for all (MI,A»2) obtain that
e
^f
4 Taking ^ = £(„<,„•) with (yi,y*) € Mj, i = 1,2, we
2 ((2/1,2/!*)  <2/i,**) + (2/2,2/2*)   (^,2/r +2/2)) >
\\z\\2+\\xl+x*2\\2
i.e. Eq. (3.111) holds. (ii) For (xl,x2,z) found at (i), from Eq. (3.111) we obtain that (i/i,»i)eMi
(v2,yJ)eM 2
> \ (ikll2 + ll*i + *2l2 + 2 (*,** + z5)) . Since for a; £ X and a;* € X* the inequality x 2 +x* 2 +2(x,x*) > x 2 + x* 2 2:rHz* = ( N     z * l l ) 2 (3.113) holds, we obtain that /
^f „ ^
~ Z ' "l
_ X
l) + ,
ln f
,
(2/2  2, 2/2  *3> > °
Because Mi is monotone, it is obvious that ,
^
„ (2/i^2/i*i) = 0
Monotone
277
multifunctions
for {z,x\) G Mi, and, since Mi is maximal monotone, (yi,yt)e.M1 for (z,x{) $ Mi. The same argument being true also for M 2 , we obtain that (z,xl) G Mi and (z,x%) £ M 2 . Furthermore Eq. (3.112) holds. • It is obvious that X x {0} C X x X* is a maximal monotone set whose domain is X. Taking in the preceding theorem one of the monotone sets to be X x {0} we obtain the following result. Theorem 3.11.5 Let X be a reflexive Banach space and M C X x X* be a nonempty monotone set. Then (i)
there exists (x,x*) £ X x X* such that V ( y , y ' ) G M : 2 (y  x,y*  x*) > \\xf + \\x*\\2 + 2 (x,x*);
(ii) M is maximal monotone if and only if for every (w,w*) 6 X x X* t/iere exists (a;, a;*) € M suc/i £/m£ XU)2 +   X *  U ; *   2 + 2 ( X  K ; , X *  W * )
=0.
(3.114)
Proof, (i) Taking Mi := M in the preceding theorem and M 2 := X x {0}, condition (3.105) is satisfied, and so there exists (a;*,x 2 ,z) £ X* x X* x X verifying Eq. (3.111). Fixing (yi,y*) £ Mi = M we obtain that WyGX
: (yiz,ylxl)
+
(z,X2)>(y,xl);
therefore x% = 0. Taking (x,x*) = (z,x\), the conclusion is immediate. (ii) Suppose that M is maximal monotone and consider (w,w*) G X x X*. It is obvious that Mi := M — (iu,w*) and M 2 : = I x {0} are maximal monotone. Using Theorem 3.11.4(ii) we obtain the existence of (u, u*) G Mi such that u + u* + 2(u,u*) = 0, and so there exists (x,x*) G M satisfying Eq. (3.114). We prove now the converse implication. Let (w,w*) G X x X* be such that (y w,y* —w*) > 0 for every (y,y*) € M. For this (w,w*) G XxX*, using our hypothesis, there exists (x,x*) € M satisfying Eq. (3.114), and so   x  i u   2 + a;* u>* 2 < 0, i.e. (w,w*) = (x,x*) e M. • Taking into account the expression of the duality mapping 4>x given in Eq. (3.59), from Eq. (3.113) we obtain that IWI2 + z*H2 + 2 (x, x*) = 0 O x* G *x(x)
<* x* €
$x(x).
278
Some results and applications of convex analysis in normed spaces
This remark gives us the possibility to obtain other characterizations of maximal monotone subsets (multifunctions). Theorem 3.11.6 Let X be a reflexive Banach space and M be a nonempty monotone subset of X x X*. Then M is maximal monotone if and only if M + gr{$x) = X x X*. Proof. Assertion (ii) of the preceding theorem shows that the maximal monotonicity of M is equivalent to each of the following relations: V{w,w*) E X x X*, 3(x,x*) \/(w,w*) eX
xX*
£ M : w*  x* € : (w,w*)6M
+
$x(wx), gr($x).
Therefore the conclusion of the theorem holds.
•
Theorem 3.11.7 Let X be a reflexive Banach space and T : X =t X* be a monotone multifunction. IfT is maximal monotone then I m ( $ x +T) — X*. Conversely, if X is smooth and strictly convex, and!m($x+T) = X*, then T is maximal monotone. Proof. To begin with, suppose that T is maximal monotone. Let x* £ X*; because (0,a;*) £ X x X*, from the preceding theorem we get the existence of the elements {y,y*) £ g r T and (u,u*) £ g r $ x such that (y>U*) + (~u,u*) = (0,a;*). Therefore u = y and x* = y* + u*, i.e. x* £T(y) + $x{y). Suppose now that X is strictly convex and smooth. Prom Theorem 3.7.2 we have that $ x is singlevalued and strictly monotone. Let (x, x*) e X x X* be such that (y — x,y* — x*) > 0 for every (y,y*) € g r T . Since x*+$x{x) € X* = I m ( $ x + T ) , there exists u £ X such that x* + $x(x) G $ x ( u ) + T(u). It follows that 0 < (u  x,x* + $x{x)  $ x ( " )  x*) =  {x  u,$x{x)
 $x{u))
< 0,
and so (x — u, $x(x) — $x(w)) = 0. Since $ x is strictly monotone, we have that x = u, and therefore (x,x*) € g r T . Hence T is maximal monotone.• Another version of the preceding result is the following. Theorem 3.11.8 Let X be a reflexive Banach space and T : X z3 X* be a monotone multifunction. If T is maximal monotone then Vy<EX,3x£X
: 0 £ T(x) + $x(x
 y).
(3.115)
Monotone
279
multifunctions
Conversely, if X is smooth and strictly convex, and Eq. (3.115) holds, then T is maximal monotone. Proof. It is sufficient to note (identifying X** with X) that $ x * = ^x1' T is maximal monotone if and only if T _ 1 : X* =4 X is maximal monotone and Im{$x,
+T'1)
= X &Vy £ X, 3x* £ X* : j , £ $x.(x*) + T ^ x * ) &Vy£X,
3x£X
: *x(j/i) nT(i) ^ 0
o V y € X , 3 x € X : 0 £ T(x) + $x(x The proof is complete.
 y). D
Remark 3.11.1 Since T C X x X* is maximal monotone if and only if AT is maximal monotone for some/any A > 0, it follows that, when X is strictly convex and smooth, we have that T is maximal monotone <=> I m ( A $ x + T) = X* for some/any A > 0 <£• Vy € X, 3x £ X : 0 6 XT(x) + $x{x ~ y) for some/any A > 0. Another very important result is the following one. T h e o r e m 3.11.9 Let X be a reflexive Banach space and2\, T2 :Xz=s, X* be maximal monotone multifunctions. Suppose that 0 £ (domxTx  domxT 2 ) 1 • Then T\ + T2 is maximal monotone. In particular domTi D domT 2 ^ 0. Proof. Let Mx = gr 7\, M 2 = gr T2 and M = gr(Ti +T2). The hypothesis shows that Mi and M 2 are maximal monotone and condition (3.105) is satisfied. From Theorem 3.11.4 it follows that dom MiOdom M2 = domTin domT 2 T^ 0, and so M is a nonempty monotone set. To prove the maximal monotonicity of M we use Theorem 3.11.5. Let (w,w*) £ X x X* and T\,f 2 : X =} X*, fi{x) := Tj(x + w)  ±w* (i = 1,2). It is obvious that Ti (i = 1,2) is maximal monotone, while from Lemma 3.11.2 we get that grTi, grT 2 satisfy condition (3.105). We can apply Theorem 3.11.4; so there exists (z^z^z) £ X* x X* x X such that (z,z?) £ grf; (i = 1,2) and
280
Some results and applications of convex analysis in normed
spaces
Denoting z + w by x and z* + \w* by x*, we obtain that (x,x*) E grTi (i = 1,2) and ii
M2
, II
*
,
*
*I2
, o /
*
i
*
*\
r\
\\x — w\\ + \\x1 + x2 — w  + 2{x — w,x1 + x2 — w } = 0. Taking x* = x\ \ x2, we obtain that (x,x*) € M and II
Il2 ,
II
*
*2
. o /
„*
*\
rv
a;u; + a; — «;  + 2 (a; — iy,a; — w ) = 0. Using Theorem 3.11.5 we have that M is maximal monotone, and so T \ + I 2 is maximal monotone. • The sufficient condition from the preceding theorem is formulated using the auxiliary function x, which is not easy to calculate. In the following theorem we establish other sufficient conditions (in fact equivalent with the mentioned one) for the maximal monotonicity of the sum. Theorem 3.11.10 Let X be a reflexive Banach space and T\,T2 : X =3 X* be maximal monotone multifunction. Then int (domTi — domT^) — int (domxxi
—
domxr 2 ) = (domxrj — domXT2Y • (3.116) Therefore int (domTi — domT2) is a convex set; moreover, the following statements are equivalent: 0 6 {domxT, domxrj,
(3.117) i
0 € (co(domT1)co(domT2)) , i
(3.118)
0 6 (domTi  domT 2 ) ,
(3.119)
0 G int (domTj  domT 2 ),
(3.120)
each of these conditions ensuring that T\ + T2 is maximal monotone. Furthermore, if int (domTi  d o m ? ^ ) 7^ 0 (equivalently, i/(domxTi — domxr 2 ) 1 is nonempty), then int (domTi  domT 2 ) = domTi  domT 2 = domxTj  domxT2> (3.121) and so dom Ti — dom T2 is a convex set.
Monotone
Proof.
multifunctions
281
Taking into account Lemma 3.11.2, we have that int (domTi  domT 2 ) C (domTi 
domT2y
C (co(domTi)  co(domT 2 )) 1 C (domxTx  d o m x T 2 ) \ which shows that the inclusions C from Eq. (3.116) hold and (3.120) =>• (3.119) => (3.118) => (3.117). Furthermore, by Proposition 2.8.9 we have that int (domxTi — domxT 2 ) = (domxTi  domxT 2 )* •
(3.122)
Let us show that (domxTi  domxT 2 ) 1 C domTi  domT2, which together with Eq. (3.122) will prove Eq. (3.116). Let w be an element of (domxTi — domxT 2 ) 1  Consider the multifunction Ti defined by T\{x) = T(x + w). We have that domTi = domTi —w and domxf = domxTi —w. Therefore Ti and T2 verify condition (3.105). From Theorem 3.11.4 it follows that domTi n domTx 7^ 0, »e. w € domTi  domT 2 . Therefore Eq. (3.116) holds. From this relation we obtain that (3.117) =J> (3.120). From what was proved above and Theorem 3.11.9 we have that each of the conditions (3.117)(3.120) ensures that T\ + T 2 is a maximal monotone multifunction. Furthermore, if int (domTi — domT 2 ) ^ 0, from Eq. (3.116) and int (domTi

domT 2 ) C domTi  domT 2 C domxTi — domxr 2
we get Eq. (3.121). The convexity of the set int (domTi  d o m T 2 ) and, when this is nonempty, of the set domTi — domT 2 follows from Eqs. (3.116) and (3.121). • Conditions (3.117)(3.120) from preceding theorem can be "relativized." Theorem 3.11.11 Let X be a reflexive Banach space and Ti,T 2 : X =4 X* be maximal monotone multifunctions. Then ic
(domTi  domT 2 ) = "(domxTj  domxr 2 ) •
(3.123)
282
Some results and applications of convex analysis in normed spaces
Therefore l c (domTi — domT2) is a convex set and the following are equivalent: 0 £ ^domxTx  dom X T 2 ), 0 € i c (co(domTi)co(domT 2 )), 0 £
ic
statements
(3.124) (3.125)
(domTi  d o m r 2 ) ,
(3.126)
domTi — domT2 is neighborhood of the origin in lin (domTi — domT2), (3.127) M
A (domTi — domT2) is a closed linear subspace, (3.128)
each of these conditions ensuring that 7\ + T2 is maximal monotone. Furthermore, if sc (domTi  d o m T 2 ) ^ 0 (equivalently, «/8C(domxTi — domxT 2 ) is nonempty), then ic
(domTi
 domT 2 ) = domTi  domT 2 = domxTx  domxr 2 ,
(3.129)
and so dom T\ — dom T2 is a convex set. Proof. To begin with, suppose that condition (3.124) holds; we wish to prove that Ti + T2 is maximal monotone; if we succeed we shall have that domTi n domT 2 ^ 0, i.e. 0 £ domTi  domT 2 . In our hypothesis XQ := aff(domxT! — domxr 2 ) is a closed linear subspace. Replacing eventually Tj by T; where Ti{x) = Ti(x+w) (i = 1,2) with w £ dom XTi H dom \T2 > w e m a Y assume that 0 £ dom \TX l~) dom xr 2 , and so domTi UdomT2 C XQ, Let i £ {1,2}. Since Tj is maximal monotone, it follows that grTj + {0} x X0X = grTj. Consider T° : X0 =t X0X,
T?(x) = {x*\Xo  x* £ Tt(x)}.
From Lemma 3.11.2 (iii) it follows that Tf is maximal monotone and dom XT; = dom \T° • X0 being a reflexive Banach space and 0 £ (dom XT° — domxT°)% from the preceding theorem we have that T® + T° is maximal monotone and (domxro — domxT 0 ) 4
=
int(domT° — domT°)
(with respect to XQ),
i.e. tc
(domxTi  domxT 2 ) =
%c
{domTi  domT 2 ) = intx 0 (domTi  domT 2 ). (3.130)
Monotone
multifunctions
283
It is obvious that (2\ + T 2 )° = T? + T2°. From Lemma 3.11.2 (iii) we have that T\ + T2 is maximal monotone. Let us prove now the other statements. Taking into account Lemma 3.11.2(iii), (iv), we have that domTi — domT 2 C domxTi — domxr 2 C aff(domTi) — aff(domT 2 ) Caff (dom Ti  d o m T 2 ) , which imply that aff(domTi  domT 2 ) C aff(domxTi  d o m x r j C aff (dom T\  domT 2 ). Therefore aff (dom T\ —dom T2) is closed if and only if aff (dom \Ti —dom XT 2 ) is closed; in this situation these sets are equal. Taking into account this remark it follows that the inclusion C holds in Eq. (3.123). Let us prove the converse inclusion. Let w G tc (domxTi — domxT 2 ); this means that X0 := aff(domxTi  domxr 2 ) is a closed set and w G l (domxTi — domxr 2 ) Taking 7\ as in the proof of the preceding theorem, we have that 0 G i e (domxf — domxr 2 ) From the first part of the proof it follows that domTi H domT 2 ^ 0, whence we have that w G domTi domT 2 . We have obtained so that l
(domxTi
—
domxr 2 ) C domTi — domT 2 (C domxTi — domxr 2 C X0),
and so the inclusion D holds in Eq. (3.123), too. Note that relation Eq. (3.130), proved above when Eq. (3.124) holds, shows that (3.124) => (3.127). Condition (3.127) ensures that 0 G ^ d o m T i domT 2 ) and aff (domTi — domT 2 ) = lin(domTi — domT 2 ) is closed. Hence (3.127) =*• (3.126). The implications (3.126) =• (3.125) =$• (3.124) are obvious. The implication (3.127) => (3.128) is obvious, too. Suppose that (3.128) holds. Denoting by X0 the closed linear space (J A > 0 (domTi — domT 2 ), we obtain that Xo = UA>O (co(domTi) — co(domT 2 )), i.e. condition (3.125) holds. Therefore conditions (3.124)(3.128) are equivalent. If zc(domTi  d o m T 2 ) ^ 0 (or equivalently lc (domxTi — domxr 2 ) 7^ 0), then relation (3.129) follows similarly as in the proof of the preceding theorem (or one reduces to this theorem). • When X is reflexive, if T : X =$ X* is maximal monotone, taking also
284
Some results and applications of convex analysis in normed
spaces
the maximal monotone set {0} x X*, we obtain that (domxTT
= int(domx:r) = int(co(dom T)) = int(domT) = (domT)*, (3.131) and so int(domT) is a convex set; if this set is nonempty, then (domT)* = domT = co(domT) = dom^T,
(3.132)
and so domT is a convex set, too. In the above relations we may replace * with ic. The convexity of the set domT also holds without interiority conditions. Theorem 3.11.12 Let X be a reflexive Banach space and T : X r3 X* be a maximal monotone multifunction. Then dom T and Im T are convex sets. Proof. It is obvious that for every I E I and A > 0 the multifunction TXi\ defined by TXi\(u) = XT(u + x) is maximal monotone. Using Theorem 3.11.7 we have that 0 6 Im($x + TC,A), »e. there exists y € X such that 0 6 $x(y  x) + XT(y). We have obtained so that \/x G X, A > 0, 3(xx,x*x)
G XxX*
: A  1 : ^ G T(xx),
x*x G
*x(xxx). (3.133) Let x G X be fixed; for every A > 0 consider (x\,x*x) given by Eq. (3.133). Consider also u G domT and u* G T(u) (fixed for the moment). From (x\ — u, A_1a;A  u*) > 0 we obtain that
:EA — x\\ = (x\ — x, —x*x) < (x — u,x*x) + X(x — xx,u*) + X(u  x,u*), (3.134) and so, for AG (0,1] we have that \\xx  x\\2 < \\x  u\\ • \\xx\\ + X\\xx  x\\ • \\u*\\ +X\\x
u\\ • U* (3.135)
where a := a; — u + Au* and /? := A a;  u • u* (we have taken into account that \\x^\\ — \\xx — x ). Prom Eq. (3.135) it follows that there exists M > 0 such that x A  = \\xx  x\\ < M for every A G (0,1]. Let 7 := limsupA_j.0+ a;,\  x\\ < oo and (An) C (0,1] be such that (A„) ¥ 0 and (\\xXn — x\\) >• 7. Since (x\n) is bounded and X is reflexive, we may suppose that x*Xn ^> x* G X*. Considering A = Xn in Eq. (3.134) and then taking the limit, we obtain that j 2 < (x — u,x*) for
Monotone
multifunctions
285
every u € domT; from this we get immediately that j 2 < (xu,x*) for every u £ co(domT). Therefore, l i m ^ o ^ A = x (with respect to the norm) for every x € co(domT). Since (x\) C domT, we obtain that co(domT) c domT. Hence domT is a convex set. Since I m T = d o m T  1 and T _ 1 is maximal monotone, we have that I m T is a convex set, too. • The hypothesis that X be reflexive for having Eqs. (3.131) and (3.132) is not necessary (under interiority conditions). T h e o r e m 3.11.13 Let X be a Banach space and T : X =} X* be a maximal monotone multifunction. Then condition (3.131) holds. In particular int(domT) is a convex set. Furthermore, i/int(domT) ^ 0 then condition (3.132) holds, and so domT is a convex set. Proof. To prove condition (3.131) it is sufficient to show that domT D (domxr) 1  Taking into account Lemma 3.11.2(h) it is even sufficient to prove that 0 G domT if 0 e ( d o m x r ) ^ Let 0 e (dom^r)' From Theorem 2.2.20 we have that XT is continuous at 0, and so there exists n, v > 0, 7] < 1 < v, such that XT(X) < v for x < n. Therefore V / i G s r , Vz, Hxll
(3.136)
Let K := u/n > 1. Let us prove that V/LiSCT : r(p) + KMn)\\>0.
(3.137)
Let fi €
max
inf h(n,x*) = inf
max
(r(/z)  (p(/x),z*))
= inf (r(^) + «   p ( / i )   ) > 0 .
286
Some results and applications of convex analysis in normed spaces
Hence there exists u* € KUX* such that h(fi,u*) > 0 for every fi £ STTaking /j = S^XtX^ with (x,x*) £ g r T , we obtain that (x,x*) — (x,u*) = (x — 0,x*— u*) > 0 for every (x,x*) £ grT. Since T is maximal monotone, it follows that (0,u*) 6 grT, andsoO € domT. Therefore Eq. (3.131) holds. Suppose now that int(domT) ^ 0 (equivalently, (domxr) 1 7^ $)• Since int(domT) c domT C d o m ^ r , from Eq. (3.131) and Theorem 1.1.2 we obtain that Eq. (3.132) holds. • The next result refers to the local boundedness of monotone multifunctions. The multifunction T : X ^Z X* is called locally bounded at x £ X if there exists 77 > 0 such that T (B(x, r))) is a bounded set; of course, T is locally bounded if T is locally bounded at any x € X. Theorem 3.11.14 Let X be a Banach space and T : X =t X* be a monotone multifunction. Suppose that XQ G (co(domT))\ Then T is locally bounded at x$. Proof. From Lemma 3.11.2 we have that co(domT) C domx^. Therefore x0 G (domxr)*) and so XT is continuous at XQ. Then there exists 77,7 > 0 such that XT(X) < 7 for every x G D(xo,2n). In particular, V(x,x')e&T,\\xx0\\
: (* + ".**><*.**> < 7> I + IHI
and so x* < 7 ( 1 + 77 + x)/r? =: 1/.
•
The following partial converse of the preceding theorem holds. Theorem 3.11.15 Let X be a Banach space and T : X =$ X* be a maximal monotone multifunction for which dom T is a convex set. If T is locally bounded atxG domT, then x £ int(domT). Proof. Let C := domT; therefore C is a nonempty closed convex set. From our hypothesis there exists 77 > 0 such that T (B(x,r])) is a bounded set. Since x £ domT, there exists (xn) C domT such that (xn) —> x; we can suppose that (xn) C B(x,r)). Let (x*) C X* be such that xn € T(xn) for every n € N. Of course (x*) is bounded, and so it has a subnet (x*)iei which u;*converges to x* £ X*. Let (XJ)J £ / be the corresponding subnet of (xn). We have that (XJ — y,x* — y*) > 0 for every (y,y*) £ g r T and i £ / . Passing to the limit we obtain that (x~ — y, x* — y*) > 0 for every (y>y*) 6 &T. Since T is maximal monotone we have that (x,x*) £ g r T ,
Monotone
287
multifunctions
and so x G domT. Because T (B(x,r]/2)) C T(B(x,T})) for every x £ domTfl B(x,r]/2), with similar arguments as above we obtain that C n B{x,n/2)
= domT n B{x,n/2)
CdomT.
(3.138)
Suppose that x $ int C, and so x G Bd C. From BishopPhelps theorem, it follows that there exists x € Bd C D B(x, rj/2) and x* € X* \ {0} such that (xx,x*)>0
VxGCDdomT.
From Eq. (3.138) we have that x G domT; let XQ G T ( X ) . Taking into account the preceding relation, for t > 0 we have that (x y,x*Q + tx*  y*} = (x  y, x*Q  y*) + t (x  x, x*) > 0 for all {y,y*) G grT, and so {x^ +tx* \ t > 0} C T(x); this contradicts the boundedness of T(B(x,r))). Therefore x € intC; from Eq. (3.138) it follows immediately that x G int (domT). D The following result shows that the converse of Theorem 2.4.13 holds in Banach spaces for Isc convex functions. Corollary 3.11.16
Let X be a Banach space and f G T(X).
(i) Let x G d o m / . x G int(dom/).
Then df is locally bounded at x if and only if
(ii) The following assertions are equivalent: (a) df is locally bounded at any x G d o m / , (b) d o m / is open, (c) / is continuous. (iii) df is locally bounded at any x G X if and only if dom f = X and f is continuous. Proof, (i) The sufficiency is proved in Theorem 2.4.13. Assume that df is locally bounded at x. By the Br0ndstedRockafellar theorem we have that x G dom df, while from the Rockafellar theorem we have that df is maximal monotone. Using the preceding theorem we obtain that x € int(dom<9/) c int(dom/). (ii) (a) <£> (b) is obvious from (i); (c) => (b) because d o m / = {x G X \
f{x) < oo}. (b) =r> (c) Note first that / is continuous at any x G X \ dom / because / is Isc. The space X being complete and / Isc, by Theorem 2.2.20 we have that / is continuous at any x G (dom/) 1 = int (dom/). Therefore / is continuous. D
288
Some results and applications of convex analysis in normed spaces We close this section with the following important theorem.
T h e o r e m 3.11.17 (Rockafellar) Let X be a Banach space and T : X =3 X* be a maximal monotone multifunction. Suppose that either X is reflexive and T is locally bounded at some point of dom T or int (co(dom T)) ^ 0. Then i n t ( d o m T ) ^ 0 and its closure is equal to d o m T . Furthermore T is locally bounded at every point o / i n t ( d o m T ) and is not locally bounded at any point of B d ( d o m T ) . Proof. If X is Banach space and int ( c o ( d o m T ) ) ^ 0, from Theorem 3.11.13 we obtain t h a t i n t ( d o m T ) ^ 0 and d o m T is a convex set. T h e rest of the conclusion follows from Theorems 3.11.14 and 3.11.15. If X is a reflexive Banach space, using Theorem 3.11.12, we have t h a t d o m T is convex, and so, from Theorem 3.11.15, we have t h a t i n t ( d o m T ) ^ 0. T h e rest of the conclusion follows from the first part. •
3.12
Exercises
Exercise 3.1
Consider the function
f.e'^R,
/(*)£L,M1+1A, *•—'k>l
where x = {xk)k>\ Prove that / is welldefined, continuous and strictly convex. Moreover show that / is Gateaux differentiate on I1 with V / ( x ) = ( ( 1 + AT1) \xk\1/k ^
sgn(x fc ))
V i = (*»)*>! 6 I1,
' k>i
where sgn(7) := 1 for 7 > 0 and sgn(7) :=  1 for 7 < 0, but / is nowhere Prechet differentiate. Exercise 3.2 Let / : (°° >• 1 be defined by f{x) := limsupa;„. Prove that d o m / = £°°, / i s convex and continuous but nowhere Gateaux differentiable. E x e r c i s e 3.3 Let (X, \\ • ) be a normed space and / € A(X). Prove that the statements (i), (ii) and (iii) below are equivalent: (i) / is uniformly convex; (ii) Ve > 0, 3d > 0, Vx,y £ d o m / , \\x  y\\ > (>,=)6, f(Xx + (1  X)y) < A/(x) + (1  X)f(y)  A(l  A)<5; (iii)
VA e [0,1]
:
Ve > 0, 3d > 0, Vz,j/ g d o m / , \\x  y\\ > ( > , = ) * : / (  x + \y) <
yw + kmo. (iv) Prove that the statements obtained from (ii) and (iii) by fixing x G dom / are equivalent to the fact that / is uniformly convex at x.
Exercises
289
Exercise 3.4 Let / : t1 ¥ R be defined by f(x) := E n > i kA=1+1/A: By Exercise 3.1 / is strictly convex, continuous and Gateaux differentiable but nowhere Prechet differentiable. Prove that p/,x(t) = 0 for every t > 0, flf,x(t) > 0 for all t > 0 and x G A := {x G £l  h m s u p ^ ^ \xk\1/k < 1}, and df,x(t) = 0 for all t > 0 and x € ix \ A. Exercise 3.5
Let (X, ) be a normed space.
(i) Let A := {x"n  n 6 N} C Sx* and consider the function
f.X^R,
f(x):=J2
, 2 n (x,x;) 2 .
*—'n>l
Prove that / is convex and uniformly Frechet differentiable, and  t~l/pipe(t) and P 3 ( H t"1/piph{t) are nonincreasing (nondecreasing) if the mapping P 3 t i> t~p(p(t) is nondecreasing (nonincreasing). Exercise 3.7 Let C C (X, ) be a nonempty closed convex set, g := dc, and ip 6 T (see page 195). Prove that 9*=LUX.
+i*c,
(1>°dc)'=i.c+il>*°\\\\,
( i c + V> # °IHl)"=V>°«fc.
Moreover, if x 0 6 X \ C and s € Pc(zo) then dg(x0) = {x* G Sx* DN(C,x) \ (xo x,x*) — \\x0 x\\}Exercise 3.8 Assume that X is a reflexive Banach space and / G T(X) is such that S := a r g m i n / is nonempty and bounded. Prove that df(S) is a neighborhood of 0 (for the norm topology) if S is a set of weak sharp minima of F. Exercise 3.9 Let X be a normed vector space, g G A(X) be such that S := {x G X  g(x) < 0} is nonempty and h := max( 0 such that ds{x) < ah(x) for all x G X with d§(x) < S; when S — {z}, one says that g is metrically regular at z. Consider the following conditions: (i) S is a set of weak sharp minima for ft; (ii) g is metrically regular at any nonempty set S C S; (iii) g is metrically regular at S; (iv) g is metrically regular at any z G S.
290
Some results and applications of convex analysis in normed spaces
Prove the following implications: (i) <=> (ii) •!=> (iii) => (iv). Moreover, if S is compact prove that (iv) => (i). Exercise 3.10 Let X be a normed space, / 6 I \ X ) and fi > 0. Let us consider the Baire regularization of / defined by / B (,/!) : X > R,
fB(x,n)
:= inf {/(y) + M x  «/
\y€X).
1) Prove that for every fi > 0 the function /s(,fi) is either identical —oo, or finite, convex and Lipschitz with Lipschitz constant /J,. Moreover, / B (  , ^ I ) < / B (  , M 2 ) < / for 0 < (ii < H2 and limM_>oo / B ( X , ^ ) = f(x)
for every x G X .
2) Let so G X be fixed. Prove the equivalence of the following statements: (a) df(xo)Clfj,UX' / 0, (b) f(x0) = fB(x0,(J,), (c) / B (xo,/i) G R and a / s ( x o , / i ) =
a/(x 0 ) n nUx> • Exercise 3.11 Let X be a normed space, / e r ( X ) and fi > 0. Let us consider the Moreau regularization of / defined by / M (  , / I ) : X > R,
/ M ( X , P ) := inf { f(y)
+ %\\x  y\\2 \ y e X}
.
1) Prove that for every fi > 0 the function / M (  , P ) is finite, convex and continuous, / M (  , M 0 < /M(,A*2) < / if 0 < /zi < /Z2 and limM_>oo fM(x,fi) = / ( x ) for every x G X . Moreover, if x* 6 df(x) then / ( x ) — ^x* 2 < / M ( X , / ^ ) . In the sequel we assume that X is a reflexive Banach space. 2) Prove that there exists J^ : X —• X (unique if X is also strictly convex) such that V x € X : /M(x,^) = /(JM(x))+fxJM(x)2; furthermore, limM_,oo J^(x) = x and lim^x^ / (J^(x)) = / ( x ) for every x € dom/. 3) ^
Assume that df(x) ^ 0. Prove that x* < dafM(0)
£ 5/M(X,^).
for all fj, > 0 and
Moreover, if x^ e 5 / M ( X , ^ ) for all ^J > 0 and x* ;
—>
x*
for some net {(ii)iei —> oo then x* S Pd/(x)(0); conclude that lim^too xjl = <*d/(*)(0). 4) Assume that X is smooth. Prove that /M(,/*) is Gateaux differentiable on X and V / M ( X , P ) = /z$x(x — J^(x)) for every x 6 X , and w* — lim^,00 V / M ( X , ^ ) = Xo if 5 / ( x ) ^ 0, where xS is the unique element of minimal norm of df(x). Moreover, if X* has the KadecKlee property (see p. 233) then XQ = lim^Kx, V/M(x,/i). 5) Prove that /M(JA') is Frechet differentiate on X and V / M (  , M ) tinuous when X is locally uniformly smooth.
IS c o n
~
Exercise 3.12 Let
Exercises
291
Exercise 3.13 Let (X, (•  •)) be a Hilbert space and K C X be a closed convex cone. For every i 6 l w e denote by PK(X) the projection of a; on K; PK(X) exists and is unique. Let us consider K~ := —K+ = {y € X  Vx € K : (x \ y) < 0}. Prove that Vx6X
: x = PK(x) + PK(x)
and (PK(x)\PK(x))
Conversely, prove that x\ = PK(X) and X2 = PK{x) x\ € K, X2 € K~ and [x\ \ X2) = 0.
= 0.
if a; = x\ + X2 with
Exercise 3.14 Let X, Y be Hilbert spaces, A € £ ( X , Y) be a surjective operator, C C Y be a nonempty closed convex set and yo € Y be fixed. (a) Prove that the operator T := A o 4 * is bijective (X* and Y* being identified with X and Y, respectively). (b) Prove that the problem (P) min a;2, Ax + yo G C, has a unique solution. Moreover, x £ X is the solution of (P) if and only if x = (A* a T _ 1 ) ( y — yo), where y is the solution of the equation y = Pc(y — pT_1y + pT~lyo) for some (any) p > 0. Exercise 3.15 Let X be a reflexive Banach space and g G T(X) be a coercive function, i.e. lim,!I._K50 g(x) = 00. Prove that g € T (see Eq. (3.73) on page 252). Exercise 3.16 Let X be a reflexive Banach space and g e T(X). Prove that g 6 T whenever (a) 0 € (doing*) 1 , or more generally, (b) 0 € '(domg*) a n c i lin(dom
Vx € A.
Let 31,92,33 : K2 ¥ R be defined by:
gi(x,y) := yjx2 + y2  x  1,
g2{x,y) := y2  2x  1,
g 3 := max{gi,g 2 }
The functions 31,32,33 are convex, 31 and 32 being differentiable on 1R2 \ {(0,0)} and R 2 , respectively. Prove that »»i(t) = fe9i(*) = 0 Vt >  1 = inf Si, r M ( t ) = * OT (t) = 2 Vt >  0 0 = inf 32, r 9 3 (0) = r 9 1 (0) = 0, fc93(0) = k92(0) = 2, where rg and fc9 are defined on page 255.
292
3.13
Some results and applications of convex analysis in normed spaces
Bibliographical Notes
Section 3.1: We used the book [Phelps (1989)] in the presentation of the theorems of Borwein, Br0ndstedRocka£ellar and BishopPhelps, as well as Proposition 3.1.3; in the statement of Borwein's theorem we added an estimation given in [Penot (1996b), Prop. l]. The statement (i) of Theorem 3.1.4 can be found in [Phelps (1993)], while (ii) is established by Attouch and Beer (1993). For Simons' theorem and the proof of the Rockafellar theorem we used [Simons (1994b); Simons (1994a)]. Note that in many books one prefers to give the proof of Rockafellar's theorem in reflexive Banach spaces using Theorem 3.11.7. Proposition 3.1.5 is stated by Zalinescu (1999); for X = Y, A  ldx and F(x, y) f(x)+g(y), Attouch and Thera (1996) (for X reflexive) and Simons (1998b) (in an equivalent formulation) obtained the weaker formula i n t ( d o m / — domp) = int(domc)/ — domdg). Theorems 3.1.6 and 3.1.7 are obtained by Penot (1996b). A result similar to Theorems 3.1.7 is obtained by Thibault (1995a). Section 3.2: The results on the Clarke tangent cone, the ClarkeRockafellar directional derivative and the Clarke subdifferential can be found in the books [Clarke (1983); Rockafellar (1981)]. Theorem 3.2.5 (with the conclusions (i), (iii) and (iv)) was originally stated by Zagrodny (1988) for the ClarkeRockafellar subdifferential and r = f(b). The present statement subsumes those in [Thibault (1995b)] and [Aussel et al. (1995), Th. 4.2]; the proof follows the lines of the corresponding result in [Thibault (1995b)]. Theorem 3.2.7 was stated for the first time by Poliquin (1990) in R" for do; the present statement subsumes the corresponding results in [Luc (1993), Th. 3.1], [Correa et al. (1994)] and [Aussel et al. (1995), Th. 5.5] (see also Remark 3.2.3). The proof of Theorem 3.2.8 belongs to Luc (1993). The statements and proofs of Theorem 3.2.9 and Corollaries 3.2.10, 3.2.11 are from [Thibault and Zagrodny (1995)]; there it is assumed that the subdifferential d satisfies several conditions (among them conditions (P4) and (P5)) which imply our condition (PI). Note that Corollary 3.2.11 is a famous result of Rockafellar (1966). Section 3.3: The lowest and greatest quasiinverses were introduced by Penot and Voile (1990). The equivalence of (i) and (ii) of Theorem 3.3.2 for the Frechet homology can be found in the book [Phelps (1989)], while that for the Gateaux homology in [Giles (1982)]; the equivalence of conditions (i), (iii), and (v) for the Gateaux and Frechet homologies can be found in [Giles (1982)]; the equivalence of (i), (vi), (vii) and (viii) are obtained by Borwein and Vanderwerff (2000). Corollaries 3.3.4 and 3.3.5 are stated by Asplund and Rockafellar (1969). Section 3.4: In writing this section we followed [Cornejo et al. (1997)]. Note that Proposition 3.4.1 was obtained by Penot (1996a) under more general conditions. Corollary 3.4.4 is established, mainly, by Zalinescu (1983b). Theorem 3.4.3 generalizes Corollary 3.4.4 to the case when argmin/ is not a singleton; the equivalence of conditions (i), (ii), (v) and (viii) is established by Lemaire (1992). For a detailed study of the problems considered in this section for the nonconvex
Bibliographical notes
293
case see [Penot (1998b)]. Proposition 3.4.5 is established by Butnariu and Iusem (2000) for Frechet differentiable functions. Section 3.5: The results of this section are mainly from [Zalinescu (1983b)] and [Aze and Penot (1995)]. The strongly convex functions were introduced by Polyak (1966), the uniformly convex functions by Levitin and Polyak (1966) and the uniformly smooth convex functions by Aze and Penot (1995); the notion of uniformly smooth function introduced by Shioji (1995) coincides, for convex functions, with that of Aze and Penot (1995). Proposition 3.5.1 was established by Vladimirov et al. (1978), while Propositions 3.5.2, 3.5.3, Corollary 3.5.4 and Theorem 3.5.5 were established by Aze and Penot (1995). Theorem 3.5.6 is new for X a general normed space. The equivalences of the conditions of Theorem 3.5.12 to which one adds the conditions (iv) and (v) of Theorem 3.5.6 are established by Aze and Penot (1995), too. Theorem 3.5.10 and the implication (i) => (iii) of Proposition 3.6.4 were established, in reflexive Banach spaces, by Zalinescu (1983b). Theorem 3.5.13 and its corollary shows that the existence of uniformly convex or uniformly smooth convex functions on a Banach space is very close to the refiexivity of the space; Theorem 3.5.13 was established by Vladimirov et al. (1978) under the supplementary hypothesis that the function / is finite valued and bounded on bounded sets. The equivalence of conditions (i) and (vi) in Corollary 3.5.7 is mentioned in [Borwein and Preiss (1987)], while the equivalence of conditions (iii), (v) and (vi) (see also Remark 3.5.2) can be found in [HiriartUrruty (1998)] for X  R", p = 2 and / Frechet differentiable. The equivalence of conditions (i), (iii) and (v) in Corollary 3.5.11 was established by Rockafellar (1976) in Hilbert spaces for p = 2 with the same constant c (see also Remark 3.5.3). Applications to the study of uniformly convex and locally uniformly convex Banach spaces can be found in [Vladimirov et al. (1978); Zalinescu (1983b); Aze and Penot (1995)]. Sections 3.6, 3.7: The results of Section 3.6 are new. The characterizations of the geometrical properties of Banach spaces presented in Section 3.7 are classical for
294
Some results and applications of convex analysis in normed spaces
[Dontchev and Zolezzi (1993)]). The other results in this section can be found in the survey paper [Soloviov (1997)]. Apparently Theorem 3.9.3 was stated by Vlasov (1972), while Theorem 3.9.4 was stated by Efimov and Stechkin (1959). Section 3.10: The presentation of this section follows, mainly, [Zalinescu (2001)]. The notion of set of weak sharp minima, in the present form, was introduced by Burke and Ferris (1993), generalizing the notion of sharp minimum point introduced by Polyak (1980); in the same paper Polyak introduces the notion of set of weak sharp minima in a slightly different form. The equivalence of conditions (i), (vi), (ix) and (x) is stated by Burke and Ferris (1993) for d i m X < oo; (i) <^ (x) is mentioned in [Jourani (2000)] for X a Hilbert space for / € A(X) with S closed (an inspection of the proof shows that this is true also in our case). The equivalence of (i) and (ii) of Theorem 3.10.1 is established by Lemaire (1989). The proof for the equivalence of (i), (ii) and (v) is taken from [Cornejo et al. (1997)]. The class T of functions defined by Eq. (3.73) was introduced and studied by Auslender and Crouzeix (1989) in finite dimensional spaces. The numbers //(£), kj(t) and rf(t) were introduced in [Auslender and Crouzeix (1989)] (as in Eqs. (3.79) and (3.82)); in this paper Propositions 3.10.3, 3.10.8, Eq. (3.84) of Proposition 3.10.7 and Theorem 3.10.10 were stated, too. Formula (3.84) Auslender et al. (1993) simplify several proofs from [Auslender and Crouzeix (1989)] and mention that those results hold in reflexive Banach spaces; one can find the reflexive case of Proposition 3.10.8 and Theorem 3.10.10 in [Cominetti (1994)]. Proposition 3.10.11 and Theorem 3.10.12 are from [Dolecki and Angleraud (1996)]. The problem of global error bounds for convex inequalities is studied in many articles. The results stated in this section are related to those in the articles [Auslender and Crouzeix (1988); Deng (1997); Deng (1998); Lewis and Pang (1998); Klatte and Li (1999)]. Excepting [Deng (1997); Deng (1998)], the results in the above mentioned papers are stated in finite dimensional spaces. Klatte and Li (1999) introduced the numbers r^(0) (1 < i < 6) for / finite valued with 0 > inf / and established Corollary 3.10.6. The implications (i) => (vi) and (ii) => (vi) of Theorem 3.10.13 can be found in [Deng (1997)] and [Deng (1998)], respectively, while the other ones can be found in [Klatte and Li (1999)] (in the mentioned framework of finite valued functions on R"). Theorem 3.10.14 and Propositions 3.10.15, 3.10.16 were established by Lewis and Pang (1998). Section 3.11: Coodey and Simons (1996) associated an adequate convex function to a multifunction. Simons (1998b) gave an interesting approach of monotone multifunctions theory by using these associated convex functions. Excepting Theorem 3.11.15, which is an amelioration of a result in [Aze and Penot (1995)], its corollary, and Theorem 3.11.17, which is established in [Rockafellar (1969)] (with a different proof), all the results can be found in the book [Simons (1998a)] and partially in [Coodey and Simons (1996); Simons (1998b)]. Exercises: Exercises 3.1 and 3.4 are from [Butnariu and Iusem (2000)], Exercise 3.2 is from [Phelps (1989)], Exercises 3.3 and 3.12 are from [Zalinescu (1983b)], for more general renorming theorems than those in Exercise 3.5 see
Bibliographical notes
295
[Deville et al. (1993)], the second formula from Exercise 3.7 is proved in [Cornejo et al. (1997)], Exercise 3.8 is from [Burke and Ferris (1993)] (there X = R"), Exercise 3.9 is from [Deng (1998)] (with g finite and continuous), Exercise 3.10 is from [Borwein and Vanderwerff (1995)], Exercise 3.11 (without (v)) is from [Barbu and Precupanu (1986)], Exercise 3.13 is a classical result from [Moreau (1965)], Exercise 3.14 is from [Kassara (2000)], Exercise 3.15 (for X = Rn) is from [Auslender and Crouzeix (1989)], Exercise 3.17 for X a Banach space can be found in [Deng (1997)] for g finite and continuous and in [Jourani (2000)] for g satisfying the condition d(g+iA)(x) — dg(x) + N(A,x) for all x € ( A n d o m g ) \ C , the functions considered in Exercise 3.18 are from [Klatte and Li (1999)].
This page is intentionally left blank
Exercises — Solutions
E x e r c i s e 1.1 Let x £ coA and consider P := {m £ N  3 A i , . . . , A m 6 P, xi, •.. ,xm € A : 5Zr=i ^k = 1> S r = i ^fc2^ = x} Of course, taking into account the formula for coA on page 2, P ^ 0. Let p := m i n P € N. Assume that p > n + 1. Since p E P, there exist A i , . . . , Ap 6 P and xi, • • ., xp 6 A such that Y7k=i ^fc = 1 a n < i £/b=i ^A^A! = x. Since xi — xp,..., x p _i — x p are linearly dependent, there exists (/ii,.. • ,(ipi) € R p _ 1 \{0} such that Sfc=i P * ( ^ — X P ) = 0, or equivalently, there exists ( ^ i , . . . , /ip) £ R p \ {0} such that 53*=i Hk = 0 and S^iMfc^fc = 0 It follows that x = J2fc=i(^* + ty**)35* for every i £ R. Taking £ = t := min{—Afc//jfc  jtj, < 0} = — Aj//^ > 0, we obtain that x = ^jt^jKAfc +tfik)xk, and so m i n P < p — 1, a contradiction. E x e r c i s e 1.2 Without loss of generality we suppose that 0 £ A. Let Xo •= lin A. If XQ = {0}, then A = {0} and the properties of the statement are obvious. Suppose that dimXo > 1. Taking into account that dimXo < d i m X < oo, Xo is a closed linear subspace, and so we may replace X by Xo, which is the same as saying that \inA = X; in this situation *A = A1. Since X is finite dimensional and A is convex, to prove the relations of the statement it is sufficient, taking into account Theorem 1.1.2, to verify that A1 ^ 0. Since lin A = X, there exists a base { e i , . . . , ej,}, k > 1, of X with elements of A. The element x := ^ ( e H he*) £ A\ Indeed, let x = AieH VXkek S X and 5 := min{ 2k\ , , . . . , 2k\ . }; for every t € ] — 5,8[ we have that x + tx € A. Therefore x £ A*. E x e r c i s e 1.3 Doing a translation, we may suppose that 0 € H , and so H = kerx* for some x* £ X* \ {0}. Let x 0 € intn(A n H) and xi £ A \ H; without loss of generality we suppose that (xi,x*) = 1. Prom the hypothesis, there exists U eWx such that (x0 + U)r\H  x0 + UHH C A. Let U0:={x&U\ \{x,x*)) < 1/4} 6 Kx Consider fi > 0 such that /x(xo — xi) € i/o and take 0 < 7 < 2u+\) • 297
Exercises  Solutions
298 Let u G Uo be fixed. Then
\x0 +  x i + 7 U = {\  7 («,£*)) (xo + u1) + (  + 7 ( u , x * ) ) x i ,
(*)
where « := 1/2 _ 7 7
1/2  7 V ) + Ml72T<^>) ' 1 ^ ^ + '<"'''>' + <"»''» * f' the last inequality being valid for our choice of 7. Since 0, w and /x(xo — £1) are in the convex set Uo, we have that u' G [/, and so xo + u' G A. Prom (*) we obtain that  x o + \x\ +~fu G A by the convexity of A Therefore \xo + 5X1 +7C/0 C A, which proves that int A ^ 0. Let now M C X be as in the statement of the exercise. If A C M it is nothing to prove. Otherwise let xi £ A\ M and consider Mi := aff(M U {xi}). Doing a translation, we assume that x\ — 0. Therefore Mi = lin M — (M — xo) + Rxo for some xo € M. By the Dieudonne theorem (Theorem 1.1.8) Mi is a closed linear subspace of X. Since M is a closed hyperplane of Mi and (AnMi)tlM = AHM, by what precedes we have that ixitMi{A C\ Mi) ^ 0. If Mi = aff A the proof is complete. Otherwise, since Mi is a closed afiine subset of X in a finite number of steps (< codimM) we obtain that rint A = mtagA A 7^ 0. Exercise 1.4 Since A + i n t C = U a eA( a + hitC), the set A + i n t C is open. Hence A + int C C int (A + C). Let x ^ A + int C. Since the set A + int C is a convex set with nonempty interior, there exists x* G X* \ {0} such that Vo € A, Vc 6 int C : (x, x*) < (a, x") + (c, x*). Since C C cl(int C) we have that (x, x*) < (x, x*) for every x £ A + C. Therefore x £ int(A + C), which proves that int(A + C) = A + int C. Suppose now that A n int C 7^ 0. The inclusion cl(A n C) C cl A l~l clC is obvious (and true without any hypothesis on A and C); so it is sufficient to prove the converse inclusion. Without loss of generality, we suppose that 0 € A 0 int C (doing, if needed, a translation). Let pc be the Minkowski gauge associated to C. By Proposition 1.1.1 pc is continuous and i n t C = { x € X  p c ( x ) < 1},
clC = {xG X \pc(x)
< 1}.
Let x € c\A PI clC; there exists ( i „ ) C A such that (xn) —• x. Since p c is continuous, we have p c ( x n ) —• pc(x) < 1. If the set P := {n G N  pc(x„) < 1} is infinite, then xn G A n C for everyraG P, whence x = lim n € p xn G cl(A n C). If P is finite, there exists no G N such that pc(xn) > 1 for every n > no. Since x G clC, we have that pc(xn) + 1. For every n > no, consider xn := „„ ," U l a?n Since A is convex and 0,x n G A, we have that x n G A. Since
Exercises  Solutions
299
pc{xn) = npc{xn)/(npc{xn) + 1) < 1, we have that xn e A n C for every n > no But (x n ) » x, and so x G cl(A fl C) in this case, too. Therefore cl(AnC) = c l A n d C . Suppose now that C is a convex cone with nonempty interior; we have int C = int(clC). Using what we proved above, taking A = c l C we have that cl C + int C = cl C + int(cl C) = int(cl C + cl C) = int(cl C) = int C. E x e r c i s e 1.5 (a) It is obvious that Xo + C is a convex cone. Let us show by mathematical induction on p that Xo + C is closed. Let p = 1. If ai G Xo we have that Xo + C = Xo, and so Xo + C is closed. Suppose that a\ £ Xo Let us consider (xn) C Xo + C, (x n ) —> x G X. Then there exist (y„) C Xo, (tn) C R+ such that xn = yn + tnai for every n G N. If (tn) —> oo then ( —^j/n ) —> a i , whence we get the contradiction oi G clXo = Xo. Therefore (tn) contains a bounded subsequence, and so it contains a subsequence (t„k)keN convergent to t £ R+. Therefore (ynk) —> x — ta\ =: y G Xo, which proves that x = y + ta\ G Xo + C. Therefore the statement is proved for p = 1. Suppose now that the statement is true for p > 1; we are going to prove it for p + 1. As for p = 1, consider two situations. To begin with, suppose that — o i , . . . , — a p +i G Xo + C; then, as one can easily verify, X 0 + C — Xo + l i n { o i , . . . , a p + i } . Since the dimension of l i n { a i , . . . , a p + i } is less than or equal to p + 1 < oo, using Theorem 1.1.8, we obtain that Xo + C is a closed linear subspace. Suppose now that there exists % such that —a* ^ Xo + C; without loss of generality, we assume that i = p + 1. Let
C •= {Y^^i^l
Vi, 1 < i < p : Ai >0J.
By the hypothesis of mathematical induction we have that Xo + C closed. Let (xn) C Xo + C, (xn) >• x e X. There exist (j/„) C X 0 + C, (tn) C JR+ such that xn = yn + t„ap+i for every n G N. If (t n ) —• oo then Xo + C 9 T"Un —> —fflp+i, whence — a p +i G Xo + C C Xo + C, which is a contradiction. Therefore (tn) contains a subsequence (tnk)keN convergent to t G R+. Therefore (ynk) —> x — ta,p+i =: y G Xo + C which proves that x = y + £a p +i G Xo + C. Therefore the statement is true for p + 1. Thus we obtained that Xo + C is a closed set for every p > 1. (c) Let K := {x G X  V i G l,fc : (x,ifii) < ai}. If the set K is empty, the conclusion is obvious. Suppose that K ^ 0. Consider the functions T,A:
X>R*,
r(s)~((x,
A ( i ) := a  T ( i ) ,
where a := ( a i , . . . , ajt) e l * . It is obvious that T is a continuous linear operator. Therefore A is continuous and T(Xo) is a linear subspace of Rfc. It follows that T(X 0 ) is closed. Let us consider P : {y G Rfc  Vi G TTfc : 2/i > 0}; then
300
Exercises  Solutions
K = A~l{P) and A(K) C P. Let (xn) C X0 + K, (xn) *• x G X. There exist (2/n) C Xo and (z„) C K such that x„ = yn + 2n for every n G N. Therefore T(X 0 ) + P 9 A(z„)  T(yn) = A(xn) > A(x). By (b) we have that T(X0) + P is closed, whence A(x) G T'(X'o) + P. Therefore there exists x0 G X0 such that A(x + xo) € P, i.e. x £ Xo + K. Hence Xo + K is a closed set. E x e r c i s e 1.6 Without loss of generality, we assume that xo = 1. It is clear that \x G P(a) if A G R and x G P(a). Let x, y G P(a); we have that {x I xo) > z • cos a,
(y  x0) > \\y\\ • cos a,
and so (x + y I xo) > (x + II2/H) • cos a > \\x + y\\ • cos a. Therefore x + y € P ( Q ) . It is evident that P ( a ) is closed. Note that P(a) = {x = Xxo + y G X  A = (x \ xo), y = x — Axo, (x  xo) > x • cos a } = JAxo + 2/  A > 0 , {y\xo) =^0, A > yj\2 + \\y\\2 • cos a } = {Ax0 +2/ I A > 0, (yxo) = 0, A sin a > j/ • cos a } . Therefore P(0)
= {Ax0
I A > 0},
P (TT/2) = {Axo + y/X
> 0, (y  x 0 ) = 0}.
Since P(a) is a cone, we have that P(a)° = P(a)+. To begin with, let us show that P ( 0 ) + = P(TT/2). Let x = Axo and y = fixo + v, with A,p > 0, (u  xo) = 0. Then (x \ y) = Xy. > 0, an so P(TT/2) C P ( 0 ) + . Let now y e P ( 0 ) + . There exists fi € R and » 6 l such that (v \xo) = 0, y = /IXQ+V. Then (xo  ^ x o + u ) = ft > 0, which proves that y G P(TT/2). Therefore P(0)+ = P(TT/2). Let us consider now a €]0, f [ . Let x G P ( a ) and y G P ( f / 2 — Q). By what was proved above, there exist X,/i > 0, u , u e X such that x — Xxo + u,
(u I xo) = 0, A sin a > \\u\\ • cos a,
y = fj,xo + v,
(v\xo) = 0, fj, cos a > \\v\\ • sin a.
Therefore X/J. > \\u\\ • \\v\\. We obtain that (x\y)
= Xfi+(u\v)
> XfM\\u\\ • ]\v\\ > 0.
It follows that y G P(a)+, whence P(?r/2  a) C P(a)+. Let now y G P ( a ) + ; there exists /i G R and u G X such that (v \ xo) — 0, y = fj,xo +v. If v — 0, since xo G P ( a ) , we have that (xo  y) = (xo  fixo) = ^ > 0,
Exercises  Solutions
301
sin oi whence y G P(n/2 — a). Suppose that v ^ 0 and consider x := XQ — TTT, \\v\\ • cos a Then x G P{a), and so sinQ
i \ \
(X \y) = Lb — rr—r. K
'yj
p
t> cosa
^ i x
sin a
' '
H

.. .,
• (V \v) = Lb y
v
• \\v\\ > 0 .
cosa
" " 
We get so that Lisin(n/2 — a) > \\v\\cos(n/2 — a), i.e. y 6 P(TT/2 — a). Therefore P(a)+ = P{ir/2 — a) in this case, too. Furthermore, we remark that P(a) is pointed if and only if a G [0, 7r/2[, and i n t P ( a ) = {x G X \ Z(x,x 0 ) < a}. Exercise 1.7 Let (u\,... ,un,un+i) € Kp< and ( x i , . . . j Xn)#n+l ) G Kp. by the generalized means inequality (see Exercise 2.11), xwi + ...+ xnun
= on (a]~ xiwi) + ... + a„ (a~
Then,
xnu„)
^ /  l \ai i l \a» > [ax XlUl) • • • [an XnUn)
> (a? 1 • • • a n " ) _ 1 p  V _ 1 I z n + i l K + i l = x„+lU n +i > —Xn+\Un+\. Therefore x\u\ + . . . + xnu„ + xn+\un+i > 0, and so Kp< C (Kp)+. Let now ( i n , . . . , u n , u n + i ) € {Kp)+. Then xiin + . . . +xnun +x„+iun+i for every ( x i , . . . , xn, xn+i) G Kp, and so XlUl + . .. +X„U„ > px"1
• " X " " «n+l
>0
V x i , . . . , X n > 0.
Taking xt = 1 and Xj = 0 for j ^ j , we obtain that Ui > 0 for every i G l,ra. Moreover, if m = 0 then, taking X2 = . . . = xn — 1, we obtain ui + ... + un > p~lxai  u n + i  for every x > 0. Leting x —>• oo we get u„+i = 0. The same conclusion is true if U{ = 0 for some i G 2, n. Assume that u, > 0 for every i G l,n and take XJ := ai/ui. The above inequality implies that 1 > pa™1 • •  a " " (u" 1 • •  u " " )  1 « n +i, whence  u n + i  < p'u™1 • • u%n. Hence (ui,... ,u„,un+i) G Kpi. It follows that Kpi — (Kp)+. Hence Kp is a closed convex cone for every p > 0. Exercise 1.8 Taking xo :— (1,0,1) in Exercise 1.6, we get easily P = Also, taking n — 2 and a\ = a? = 1/2, it is obvious that P = K y j 
P(ir/4).
Exercise 1.9 Let / : X —* R, / ( x ) = QX — f{x). It is obvious that / is a sublinear continuous functional. Therefore C is a closed convex cone. If x, —x G C then ¥>(x) > ax, (f(x) 
Exercises  Solutions
302
and so, adding these relations, we have that 0 > 2az, i.e. x = 0. Therefore C is pointed. Since a < \\(p\\, there exists x G X such that aa; < f(x), and so 0 > inf/. Therefore, using Exercise 2.4, we have that int C = {x G X  f(x) < 0 = /(0)} # 0. Furthermore, using Corollary 2.9.5, we have that N(C; 0) = R+ • 5/(0) = R+ • (aUx> 
= R+ • (tp + aUx*)
Exercise 1.10 Let y £ intCR(aT) and V G Jix be such that y+V C 3?(x). Because 2/o G (Im^R)*, there exists A > 0 such that y\ := (1 + A)yo — Ay £ ImK. It follows that »/o =_(1A)y + Ayi, where A := (1 + A )  1 £]0,1[. Let xi G 5l~1(yi) and take u := (1 — X)x+Xxi. Consider /j, G]0,1] and (a;M,2/M) := (1—fj,)(xo,yo)+fJ(xi,yi) 6 g r X Then y0 = l^y + (1  7/i)s/M. for every v £ V we have (ZM>S/O + 7M«) = 7 M ( 2 T . 2 / +
W
where
_ zM := J^x
_
,,
+ (1 
M +x_*x
e
^0' ^
) + (17M)(:C/'.2/M) ^ S r K
where ;
7^ : =
N 7M)XM
that
VpG]0,l],
_
A — LtX
= —=
Jt follows
U
=a:o +  — =
fl + A — fl\
=u.
(J, + A — (J,A
Hence yo + J^V C R(a:'M), and so yo £ int9l(a;^). Since every element of the •£_ ^ with fi £]0,1], we obtain that yo € interval ]0,1] can be written writtej as int3?(a;) for every x G]:EO,M]. Exercise 1.11 Suppose that domC is dense and x\,X2 G C*(y*); this means (see page 26) that {x,x{) = (y,y*) — (x,x%) for every (x,y) £ grC (because grC is a linear subspace), and so (x,x\ — x^) = 0 for every x G domC. As domC is dense, it follows that x\ = x%. Suppose now that domC is not dense and take x G X \ cl(domC). By Theorem 1.1.5, there exists x* € X* such that (x, x*} < 0 = {x,x*} for every x € domC (because domC is a linear subspace). So, (x*,0) G (grC) + , whence 0,x* G C*(0). Because x* ^ 0, the statement is proved. Applying the preceding statement for C and G* we get the last assertion. E x e r c i s e 1.12 Let (xn) C X be a Cauchy sequence. Since \d(x„, x) — d(xm, x)\ < d(xn,xm) for all n, m G N and x € X, we have that the sequence (d(xn,x)) C R is Cauchy, and so it is convergent. Let / : X —} R+ be defined by f(x) := 2\imd(xn,x). From the inequality d(xn,x) < d(xn,x') +d(x',x) for n G N,
303
Exercises  Solutions
x, x' 6 X, we obtain that / is Lipschitz with Lipschitz constant 2. Moreover, we have that (/(x„)) —> 0. By hypothesis, there exists x 6 X such that VxeX
: /(x) < / ( x ) + d ( x , x ) .
Taking x = x n , we get / ( x ) < f(x„)+d(x„,x) for every n € N. Taking the limit, we get / ( x ) <  / ( x ) , and so / ( x ) = 0. This shows that (x„) —> x. E x e r c i s e 1.13 Applying Ekeland's variational principle for xo € d o m / and e = 1, there exists i i g I such that / ( x i ) + d(xo,xi) < /(xo) and / ( x i ) < / ( x ) + d(x,xi) for each x € X \ {xi}. Suppose that xi ^ argmin/. Then there exists x e X \ {xi} such that / ( x ) + d(xi,x) < / ( x i ) < f(x) + d(x, x i ) , whence the contradiction 0 < 0. Therefore xi £ argmin/, and so d(xo, a r g m i n / ) < d(xo,xi) < /(xo) — / ( x i ) = /(xo) — inf/. As this inequality is obvious for xo ^ dom / , the conclusion holds. E x e r c i s e 1.14 As in the proof of Exercise 1.13, there exists xi € X such that / ( x i ) < / ( x ) + d(x,xi) for each x € X \ {xi}. From the hypothesis there exists X2 £ 3l(xi) such that d(x2,xi) + f{xi) < / ( x i ) . If xi / X2 we get the contradiction d(x2,xi) + /(X2) < /(X2) + d(xi,X2). Therefore X2 = xi € 3l(xi). Exercise 1.15 lim
(i) Indeed,
/ ( x ) = c x 3 # V A e R , 3 p > 0, Vx € X, \\x\\ > p : f(x) > A
x>oo
»VAeR, 3p>0
: [ / < A] C D(0,p).
(ii) Let A > inf i e R" f{x). By hypothesis [/ < A] is a closed. By (i) [/ < A] is also bounded, and so it is compact. Applying the wellknown Weierstrass' theorem for /  [ / < A ] we get the desired conclusion. E x e r c i s e 2.1 Let ti,t2 €]0,oo[ and A €]0,1[. Consider t := Ati + (1  \)t2 > 0. Because ^  ( x + t^lu) + (1 ~^)* 2 ( z + t^u) = x + \u, the convexity of / gives
f{x+
i«) < ^f(x+t^u)
+ il=^a/(x +
t^u),
whence ip (Aii + (1 — A)*2) < A'0(ti) + (1 — A)V>(*2). Hence ?/> is convex. E x e r c i s e 2.2 (a) Assume that the conclusion does not hold. Then there exist xi,X2 £ / such that xi < x 2 and / ( x i ) > fixi) Let A := {x 6 [xi.xa]  / ( x i ) < / ( x ) } and take x := sup A Of course, x < x 2 and, from the hypothesis, x > x\. lix $ A then / ( x ) < / ( x i ) . But there exists e > 0 such that f\[xi,x] is nondecreasing, and so f(x) < / ( x ) < / ( x i ) for all x G [x —e,x], contradicting the choice of x. Therefore x € A, and so x < X2. Using again the hypothesis, there exists 0 < £ < X2 — x such that f\[x,x+e] is nondecreasing. Because / ( x i ) < / ( x )
304
Exercises  Solutions
we obtain that f(xi) < f(x + e), and so x + s G A, contradicting again the choice of x. Hence / is nondecreasing on / . (b) 1) Consider fj, := (g(b) — g(a))/(b — a) and h(x) := g(x) — fix. Then h is continuous and there exists h'+ = g'+ — fi. There exists c G [a, b] such that h(c) < h(x) for every x G [a,b]. Because h(a) = h(b) we can take c G [a,b[. It follows that /+(c) = h'+(c) — fi > 0, whence the conclusion. For 2) take c € ]a, 6]. (c) It is sufficient to consider only the case X = K. So A C R is an open interval and for every a G A there exists e > 0 such that the restriction of / to ]a — e,a + s[C A is convex. It follows that there exists /+(6) = limZ4.(, (f(x) — f(b))/(x  b) G R and fL(b) = l i m , t t (/(*)  /(&))/(*  6) < /+(&) for every 6 G]a — e, a + e[. So the functions f'_, f'+ : A —• R are locally nondecreasing. From (a) we obtain that f'_ and f'+ are nondecreasing on A. Because / is locally convex, it is continuous. Let now a, b G A, a < b, and A g]0,1[. Consider c :— (1 — \)a + Xb. Using (b) we obtain x' G [a,c[ and x" € ]c, 6] such that / ( a )  /(c) > f'+(x')(a  c) > fL (c)(o  c) and /(ft)  /(c) > fL (x")(b — c)>fL (c)(b — c). Multiplying the first relation with A and the second one with 1 — A, then adding them we obtain that /(c) < (1 — A)/(a) + A/(6). Therefore / is convex. Exercise 2.3 It is obvious that the implications "=*•" are true. Suppose that fx is use at 0 for every x € Rn and let R 9 A > f(x) = /x(0). Since fx is use at 0, it follows immediately that x £ [f < X]\ The function / being quasiconvex, [/ < M is convex; since dimR" < oo, we obtain that x € int([/ < A]). Therefore / is use at x. Suppose now that fx is lsc at 0 for every x 6 R n . Suppose, by way of contradiction, that / is not lsc at x. Then there exists R 3 A < f(x) such that x G cl[/ < A]. Therefore [/ < A] ^ 0. Let a G ' [ / < A] ( / 0 by Exercise 1.2). Using Exercise 1.2 we obtain that ] a , x [ c l [ / < A] C [/ < A], contradicting the fact that the function fam is lsc at 0. Therefore / is lsc at x. If X is an infinite dimensional normed space, there exists a linear functional ip : X —>• R which is not continuous. It is obvious that ip is quasiconvex (in fact is convex!), but
kXk.
It is clear that ip is well defined and linear. Moreover sup{\
sup{\ip(en)\  n G N} = sup{n  n G N} = oo, which shows that
0 for every i £ R + \ {1}. It follows that ip'p is increasing on R_, and so y p is strictly convex and f'p(0) = 0 for t > 0. Hence y>p is increasing on M+. Let now p e]2,oo[; then p p '(f) < 0 for every i € R+ \ {1}. It follows that ip'p is decreasing on Rj., and so y p is strictly concave and 0. Hence <^p is decreasing on R+. Exercise 2.8 Let D :=]0, oo[2; D is an open convex set and / is twice differentiable on D. Consider ip(x,y) := arctan  for (x,y) € D. Then, for (x,y) e D, we have
Exercises  Solutions
305
E x e r c i s e 2.4 Let x G X be such that / ( x ) < A and take x G X. If / ( x + x) < A we have that x + tx G [/ < A] for t G [0,1[, while if / ( x + x) > A we have that x + tx G [ / < A] for *G [0,i[, where * : = /( _ A  _~^^ ( . Therefore x G [/ < A]'. Let us prove now the converse inclusion. Since A > inf / , there exists xo € X such that /(xo) < A. Let x G [/ < A]1. There exists t > 0 such that x := x + i"(x — xo) 6 [/ < A]. So we obtain that x = JTW + yrj^o a n ( ^ m
< m m
+ i ^ / ( * o ) < T ^ A + ^ A = A,
which proves the converse inclusion. If / is continuous then int[/ < A] D {x G X  / ( x ) < A} # 0, and so, since [/ < A] is a convex set, int[/ < A] = [/ < A]'. E x e r c i s e 2.5 (a) Assume that / is lsc and t G ] inf / , oo[. Since [/ < t] is closed, the inclusion [/ < t] D cl[/ < t] is obvious. Let x G [/ < i\. Take xo 6 [/ < t]. Then x* := (1A)x 0 +Ax G [/ < t] for A G [0,1[, whence x = limxfi x\ G cl[/ < t). Assume now that [/ < t] = cl[/ < t] for t G ] inf / , oo[. If t0 := inf / G K, we have that [/ < to] ~ f]t>t [f < i\. It follows that [/ < t] is closed for every ( e R . Hence / is lower semicontinuous. (b) Assume that / is continuous on dom / and t G ] inf / , oo[. Let x G [/ < £]; since / is continuous at x, [/ < t] is a neighborhood of x, which shows that [/ < <] is a neighborhood of x, too. Hence x G int[/ < t]. It follows that [/ < t] C int[/ < t\. Let now x G int[/ < t] and take x 0 € [/ < t]. As the mapping A H H J := (1  A)x0 + Ax is continuous at 1 G R, there exists A > 1 such that xx € [/ < t]. It follows that x = jX\ + ^j^xo G [/ < t]. Hence [ / < * ] = int[/ < t]. Assume now that [f < t] = int[/ < t] for every t G]inf/, oo[. Consider x G d o m / and take t > f{x). Hence x G int[/ < t], and so / is bounded above on a neighborhood of x. Therefore, by Theorem 2.2.9, / is continuous at x. (c) Assume that / is continuous on X and t e ] i n f / , oo[. Since [/ < t] is closed in our case, from (b) we obtain that b d [ / < t] — (cl[/ < £]) \ (int[/ < £]) = [f
Exercises  Solutions
306
tf'(a,ei) for any t € R Let i £ R n ; then x = xiei + .. . + x„e„ with x\,... ,xn R By the sublinearity of / ' ( a ,  ) it follows that f'(a,x) < f'(a,xiei) + ••• f'(a,x„e„) = xif'(a,ei) +  + xnf'(a,en), and f'(a,—x) < f'(a,—xiei) • •• + f'(a,xne„) = xif'(a,ei)  •••  xnf'(a,e„). Therefore 0 < f'(a,x) f'(a, —x) < 0, and so f'(a,tx) = tf'(a,x) for every t € R It follows that / ' ( a , is linear, and so / is Gateaux differentiate.
S + + + •)
E x e r c i s e 2.7 It is obvious that
' m _ / p ( 2  ( i  * ) p  1  ( i + *) p  1 )
V w
?

1
\ p (2 + (t  I ) ' "  (t + l)
p_1
)
if
*e[o,i[,
if t € ]1, oo[,
,„« m _ / P(P  1) ((1  <)P"2  (1 + *) P_2 )
if * 6 [0,1[,
^ \
if te]i,oo[.
(t)
2
2
pipi)((tir (t+ir j
Because limtfi <£>p(£) = p(2 — 2P~1) = lim^i y p (£), <£ is derivable on [0, oo[ and
df
°Mx,y) =P(iP(x,y)f1 j4== OX
y/xi 1
y(x,y)=P(
and so
~
4 y2 X
y/xz + y2
+ {
+ („(*,„))*
^
y/xz + y2
307
Exercises  Solutions Hence
d2f(x, y)(h, k) = (V(x, y)?> (Pi?  1) + {
{yh X
 pL2
2
(x + y2)yjx2
+ y2
>0
for all (x,y) G D and (/i, &) G R 2 . Using Theorem 2.1.11 we obtain that f\o is convex, and so the function g : R2 —> R defined by g(x,y) := f(x,y) for (x,y) 6 £*, g(x,y) := oo otherwise, is convex. It is obvious that for any a > 0, l i m i n f ^ j , ) . ^ , , ) ^ , ^ ) = ( f ) ' ? a = / ( a , 0 ) and liminf(X>y)_>(o,o)S(a;,y) = 0 = /(
R, ^>(i) = VI +* 2  Then
v t G R :
It follows that
0 for every t G R. By this remark the convexity of / is immediate. Using Taylor's formula for functions of a real variable, we have that for all t, T G R, there exists d G ]0,1[ such that fit + T) 
0, then consider the functions gn,9 defined (a.e.) by the formulas y/l and SdomF = (F*)oo So g<x>(u*) = mmv*eY*(F*)oo(u*,v*) for every u* G X*. Let D := Pry (dom F) and To = linD. It is clear that condition (ii) of Theorem 2.7.1 implies that 0 G lD, and so YQ~ = D1 = D~. The conclusion follows from the last part of Exercise 2.23. Exercise 2.27 We consider that a — b := a + (—6), the sum being taken in the sense of convex analysis: (+oo)+(—oo) — +oo. We have that i n f l € x (f(x) — g{x)) equals inff/^)x£Ji. • (ii) Since if is nondecreasing we obtain that 0. (ii) =>• (iii) This implication is obvious because 0 < f'(x, —x) + f'(x,x). (iii) =>• (i) For x € d o m / and t G 7 we have that tx G d o m / , and so, by hypothesis, pf(tx) < }'{tx,tx) = tf'(tx,x); hence 0 for every t 6 / . But 0 for all s,t G i n t / with s < t. Hence
Let i , « £ X be fixed elements. Replacing t by x(t) and r by u(t), t G [0,1] in the above inequality, then integrating on [0,1], we obtain that
f{x +
u)f{x).fo1;^dt\
This inequality proves that / is Prechet differentiable at x and that V / ( x ) has the announced expression. Let us prove that V / is Prechet differentiable. For all t, r G R there exists 0€]O,1[ such that ¥>'(« + r ) 
9r)r2.
From the expression of tp'" given above we have that y?'"(£) < 3 for every t e R. Therefore V t , r G R : ^'(* + r )  V ( t )  y " ( * ) r  < § r 2 .
Exercises  Solutions
308
Denoting by B(x) the bilinear functional defined by B(x) : X x X > R, W
B(x)(u,v):= v A
UV
f
.
2
Jo (l+x )^/^TP
dt, ^
and using the preceding relation we obtain, as above, that Vx,u,v£X
< I f> u2\v\dt < l\\u\\2\\v\\,
: \(Vf(x + u)Vf(x)B(x)(u,))(v)\
and so Vx,u€X : \\Vf(x + u)Vf(x)B(x)(u,)\\ < w 2 . Therefore V / is Prechet differentiable on X and V 2 / ( x ) = B(x) for every x e X. PYom the expression of V 2 / ( x ) and the expression of tp'" we obtain easily that V 2 / is Lipschitz with Lipschitz constant 3. Therefore V 2 / is continuous on X. E x e r c i s e 2.10 l
Consider X := L 1 (0,1); X is a Banach space for the norm:
Nl •= !0 \u{t)\dtioxuex. Using the function tp defined in the solution of the preceding exercise, taking also into account that tp is convex, / is convex. Moreover V £ , r e R : \T\ < tp'(t)r < tp(t + T) 
+ (X + Tnu)2
9n •=
Tn
 V I + X2
XU ,
9 :
sJ\+X2
It is obvious that lim^nx) gn = g a.e.; from the above inequality we obtain that <7n < w a.e. for every n € N. Using Lebesgue's theorem (see [Precupanu (1976)]) we obtain that lininHx, f0 gn dt = J0 g dt. This proves that / is Gateaux differentiable at x, and the expression of V / ( x ) is that of the statement. Let x € X be arbitrary; we show that / it is not Prechet differentiable at x. We may assume that x is finite a.e. Of course, [0,1] = \JmeNAm, Since limm^xm.eas(Am)
where Am ~ {t € [0,1]  \x{t)\ < m } . = 1, there exists mo G N such that meas(Amo)
> .
Exercises  Solutions
309
But yj\ + (t + r ) 2  yjl + t2 
tr
Vi + t* 1
2
y/i + (t + r) + N/TTF ^
2i 2 + tr N /TTF( 1 /i
+ (t + r)2 + VT+t*y
for T = n and t < mo we have that
y/l + (t + r ) 2 + v T T t 2 ~ 2y/l + (n + mo) 2 2
2i + r t
mo
v / T T t 2 ( N / l + (t + r ) 2 + v / T + 1 2 ) 2
2
'
n + 2m 0
Vl+m2
V 1 + (n ~ ™o)2
mo
%
V 1 + ™o '
the limits being taken for n —• oo. Therefore there exists no > mo and £o > 0 such that VteAm0,
Vn>n0
tn
: y / l + (t + n ) 2  y / l 41 2 
> e 0 n.
vl +* For every n > no there exists a measurable set Bn such that Bn C A m „ and meas(Bn) —ri~2.Let us consider the functions rnl]
,„
Un:[0,l]>R,
f n if t € B„, « „ : =  Q .f f £ ^ ^
^
Then u n  = — —> 0. From the above inequality we have that / ( Vl + (a + M n ) 2  \ / l + a 2  ^L]dt>e0[ Jo \ Vl+a; /
/o
undt = £o\\un\\,
which shows that / it is not Frechet differentiate at x. E x e r c i s e 2.11 The function / : R  > H , / ( t ) := expt is strictly convex. Consequently, for a,b € P, o p ^ 6" and p,q£]1,oo[, ^ + J = 1 we have that ab = / ( ± l n a * + > & * ) < ± / ( l n a ' ) + i/(ln6«) = \c? + \b«. Therefore the statement is true. It is obvious for a = 0 or 6 = 0. The function g : P >• R, g(t) :=  l n t is strictly convex. Let n G N \ {1}, a i , . . . ,an €]0,1[ with a i + . . . + an = 1 and x i , . . . ,x„ € P, not all equal. Then  ln(aixi H
1 anx„) = g (cnxi H < aip(xi) H
1 a „ x n ) l  a „ j ( i j =  I n (x" 1 ••  x " n ) ,
310
Exercises  Solutions
and so aixi H
1 a„xn > x"1 • • • x°n.
It is obvious that the above inequality is still valid when xi,. •., xn € [0, oo[ are not all equal. The final statement is now obvious. Exercise 2.12 The implication => is obvious. Assume that f + x* is quasiconvex for every x* € X* and consider x, y € dom / with x / y and A € ]0,1[; set z :— (1 — \)x + \y. There exists x* € X* such that (x — y,x*) = f(y) — f(x). Since f + x* is quasiconvex, we have that f(z) + (z,x*) < max {/(a;) + (x,x*),f(y)
+ (y,x*)}
= (1  A) (/(*) + (*, x')) + A (/(») + (y, **» , whence / ( ( l  \)x + Xy) < (1  A)/(x) +
\f(y).
E x e r c i s e 2.13 Note that d2f . . ^'""^ Z&(*) oxi
df i \ * , — (a;) = —f(x), OXi to^/M. Xi
a i ( a i  1)L — ^r^f(x), —
=
while for i ^ j , d2f
, . _ otiOtj
(*) = r^/to
The function / is concave (i.e. —/ is convex) if and only if V2f(x) defined for every x G P", i.e. ai{ai 1] 2 Vu = ( t t l l . . . ,u„) 6 R n : r 2 u ^'i=i a;?
is nonnegatively
+ 2V ^ ^ ^^i
< o, _
or, equivalently,
V«eR" : (Y^_ a; Mi) < 5^"_ QiU«2Taking an+i = 1 — ^ " = 1 O i and un+i = 0, the preceding inequality follows from the ( strict) convexity of the function t —> t2 on R; the inequality is strict if ^27=ia' < 1 anc ^ u / 0) a n d so / is strictly concave in this case. Exercise 2.14 We have that df/dxi
— = (expxi)(expxj) (72/ j «C # 7j
= (expa:;) ( S * = i
(V" \*—'« = !
expa;*) /
ex a;
P fc)
>
an<
^
so
Vi, j 6 l , n , i ^ i
Exercises  Solutions So, for all x,ueRn
311
we have that V 2 /(x)(w, u) equals
(EL*"*)"2 ((ELi»0 (EI=1w(~*)a)  (EI =1 ^^) 2 ) where ?/& = expx* for k € IT" Applying the wellknown CauchySchwarz inequality ( E L i a * 6 * ) 2 ^ ( E L i a i ) ( E L i b f c ) f o r
a
* = Jy*
a n d 6fc
=
Vvku^
we obtain that V 2 / ( x ) ( u , it) > 0. Hence / is convex. E x e r c i s e 2.15 One obtains easily that
§«,*) = f/(.,*), ^(,,I)„±/(M), fM = tt ' g « , * ) = ^ / M , 0«,*>=iy/e..>, ^ f t . ) while for i ^ j , 32f
1
CfX% OXj
X%Xj
Since / ( t , x) > 0 for every (t, x) 6 P x P", the fact that / is strictly convex (resp. convex) is equivalent to the fact that the quadratic form
is positively (resp. nonnegatively) defined, or, equivalently, that the quadratic form ?^s2 p
+ Y*n
2u2i  2 V "
*~ii=\
is positively (resp. nonnegatively) defined. quadratic form is / 2=1
/
A =
P
1
 1 2 1 1
V 1
UU
sm + 2 V
*~ii=\
' J
*^l
The matrix associated to the last
1
_1
1 2
1 1 2
\
/
To study the positiveness of the matrix associated to this quadratic form, we have to determine its eigenvalues. Let 6„ := det(At„+i —A). After some computations we obtain that
Exercises  Solutions
312
p
x _
P
P
P
1 1
Al 0
0 Al
••• •••
0 0
1
0
0
•••
Al
Developing with respect to the last row we obtain that 5n = (A  l)6ni
 (A  l ) " " 1 (A + 1/p),
and so 5n = (A  I ) " " 1 * !  (n  1)(A  l ) "  1 (A + 1/p) = (A  I ) " " 1 [A2  (n + 2  1/p) A + 1  (n + l)/p] . Let Ai, A2 be the roots of the polynomial between the square brackets. Because the other roots of the characteristic polynomial are all 1, the quadratic form under consideration is positively (resp. nonnegatively) defined if and only if Ai and A2 are positive (resp. nonnegative). For p < 0 we have that Ai + A2 = n + 2  1/p > 0,
A1A2 = 1  (n + l ) / p > 0,
and so Ai, A2 > 0. When p > 0 the roots Ai, A2 are positive (resp. nonnegative) if and only ifp > n + 1 (resp. p > n + 1 ) . Therefore / is strictly convex if and only if p € ] —oo,0[U]n+l,oo[ and is convex if and only ifp 6 ] —oo,0[U[n+l, oo[. Since g{x) = f((x\c),x) and the function x *¥ (x\c) is linear, it follows immediately that g is (strictly) convex for (p > n + 1) p > n + 1. Since h(x) = / ( l , x) (for p = n + 2), we obtain immediately that h is strictly convex. E x e r c i s e 2.16 We give only the proof for ip'+7 the proof for ip'_ being similax. The function i/> is increasing, while ip'+ is nondecreasing and 0 < ip'(t) < 00 for t > 0 (if V'(*o) = 0 for some t0 > 0 then ip'(t) = 0 for t S [0, to], and so ip{t) = ip{G) = 0 for t € [0, to]) It follows that the mapping P 3 t H 1/ (ip'+ o i/>1) is positive and nonincreasing on P. Let us fix a > 0 and take j3 := ip(a) > 0. From what was said above, the integral jff 1 J*JX •. exists in the sense of Riemann for all 7 6 ]0, f)[. Considering it as a RiemannStieltjes integral, we have that r0
fit
ri>~1(0)
1
[a
l
Exercises  Solutions
313
where S := ip 1(y). Let S = so < «i < . . . < s„ = a be a division A of the interval [5,a] and take at := s;_i for i G l , n . Then ip'(oi) G dip(sii), and so
S(A, (ffi)) = ^" =i ^  y (V>(«i)  tf(*l)) > E L ( S i  S !) = Q  6> because ip(si) — ip(sii) > ip'(sii) • (si — Sj_i) for all i 6 l , n . Taking now
F(x,y):=\ v 'y;
'
M
\ +cx>
* if
1 * = »• Tx ^ y.
It is obvious that F is convex and / is the marginal function associated to F. Therefore, / is convex. Furthermore, for A > 0 and y 6 Y we have that /(Ay) = inf{:r  Tx = Xy} = inf { A • H A " 1 ^  T (A" 1 *) = y} = A i n f {   u    T u = y} = A/(y). If T is open, there exists p > 0 such that pf/y C T(Ux)VyGpf/F, 3 x G f / x
Therefore
: Tx = y,
and so /(y) < 1 for every y 6 pUy; this implies that d o m / = Y and / is continuous. Exercise 2.18 By Exercise 1.15 there exists x G R* such that fix) < f(x) for every x €Rk. We assume, without loss of generality, that x = 0 and f(x) = 0. Let (x,£) G co(epi/). Then there exists a sequence ((xn,tn)) C co(epi/) converging to (x,t). By the Caratheodory theorem (Exercise 1.1), for every n € N there exist Aj, 6 [0,1], ( x ^ f j 6 e p i / for t G l,k + 2 such that E * ^ 2 AB„ = 1 a n d (x„,t„) = S,fc=i2 K(xi,ti). We may assume that for every i G l,k + 2: (A*n) > Ai € [0,1], ( O »• t« 6 [0, oo], ( A X ) _* V G [0, co], (Aj, x l n ) > if G [0, oo] and ( x n) —• /*' G [0,oo]; moreover, if p' < oo, we may assume that {xln) —>• xl G Rk. Of course, £ * + 2 *' = 1 F i r s * of all, because (E?=i 2 AJ,*™) > t = £ * = 2 7% we have that 7 s 6 [0, oo[ for every i G 1, A; + 2. Assume that / / = 00. By our hypothesis we have that (xn 1
1
t'n)
—
* °°>
which implies that t — 00, A = 0 and 77* = 0. Consider the sets / := (i 6 l,fc + 2 I JJ* > 0} and J := l,fc + 2 \ I. Hence (xj,) > x i € Rk \ {0}, A; > 0, and so tl < 00, for i € I; moreover, (x*,i l ) £ e p i / for i G / , because /
314
Exercises  Solutions
is lsc. Let 7 := E ; e j 7* when J is nonempty and A := E ; e j ^' when I is nonempty. It follows that (x,t) = (0,7) if 7 = 0, (x,t) = J2iei *•*(*',**) if J = 0 and (x,t) = J2i€I \'(x',t') + (0,7) if I, J are nonempty. It is obvious that (x,t) £ co(epi/) if 7 or J is empty. If both of them are nonempty then ( M ) = E i g / * ' ( * ' . *') + (1  A) (0, (1  A) _ 1 7) 6 co(epi/) if A ± 1 and (x,t) = E i e / V ( a ; % * i + 7 ) 6 c o ( e p i / ) i f A = l. E x e r c i s e 2.19 If d o m / (~1 dom3 = 0 then ( / + 7 3 ) (x) = 00 for all 7 > 0 and x € X, and the equalities are obvious. So assume that d o m / PI doing ^ 0. Let us begin with the first equality. Since co(/ + ag) 4 fig < f + ag + fig = f + (a + fi)g we have that co (co(/ + ag) + fig) < co ( / + (a + fi)g). In order to show the converse inequality it is sufficient to show that co ( / + (a + fi)g) < co(/ + ctg) + fig, or equivalently, epi (co(/ + ag) + fig) C co (epi ( / + (a + fi)g)). So let (x,£) be in the first set; there exist ti,<2 £ R such that t = t\ + fit2 and {x,ti) £ co(epi(/ + ag)) and (£,£2) 6 epig. Therefore (x,ti) = lim; 6 /(x', x and / £ T(X), we have that / ( * ) < limmf / ( E J I ^ ^ O ^ ^ E j l x ^ / ^ )
^
" ^ E j l x ^ '
and so l i m i e / (/(x) + 7*  E J i i A j s j ) = 0 and ( x , / ( x ) + 7*) 6 e p i / for i € 7, where 7° := max ( 0 , E ? i i ^)s'j:~ f(x)) Because [x), s) + (a + fi)r)) belongs to epi ( / + (a + fi)g) for all i € 7, 1 < j < m, and (x, / ( x ) + y{ + (a + fi)t2) £ epi ( / + (a + fi)g), we have that
£ j l i ^ 8 A> (*' > *' + ( Q + ^ ) + ^
t +
£ /(a:)+7i
=(^' 0 ^( '
(*< '<*> + V + (« + flfc)
^^^
+(a+/3)i2
)
£ c o ( e p i ( / + (a + /3)g)). Taking the limit we obtain that (x, t\ + fit2) — (x, t) £ co (epi ( / + (« + fi)g))Assume that h := co ( / + (a + /?)g) is proper; then, by Theorem 2.3.4, we have that ( / + (a + /?)
Exercises  Solutions
315
SB+C, and so SA + sc < SB + sc Because C is bounded, as noted on page 79, sc(x*) € R for every x* G X*. Prom the preceding inequality we get SA < SBUsing Theorem 2.4.14(vi) we obtain that A C coB. E x e r c i s e 2.21 (i) => (ii) is obvious: / ( x ) < /(0) + L \\x\\. (ii) =*• (iii) Let u € X. Then foo(u) = lim^^o t " 1 (/(t«)  /(0)) < lim^oo r
1
(L \\tu\\ + a  /(0)) = L u.
(iii) =>• (i) Let first x G dom / and take y € X. Then / ( y )  fix) < s u P t > 0 r
l
(/(x + t(y  x))  f{x)) =fooiyx)
\\x  y\\.
This shows that d o m / = X, and so the above inequality holds for all x,y G X. Changing a; and y we obtain that / is LLipschitz. Exercise 2.22 Let u G X be fixed and consider 7 := sup{/(x + u) — f(x) \ x e d o m / } . The inequality /<x,(u) > 7 follows immediately from Eq. (2.28) (see page 74). Hence the conclusion holds if 7 = 00. Assume that 7 < 00. Fix xo 6 d o m / ; then xo + nu 6 d o m / for every n £ M By Eq. (2.28) we have that f{xo + ku) — f(xo + (fc — l)u) < 7 for every k G N. Summing for k from 1 t o n we get f(xo + nu) — /(xo) < nj. Dividing by n and letting n —¥ 00, we obtain that /oo(u) < 7Exercise 2.23 Let x0 € [/ < A] be fixed. Take first u € [f < A]oo Then fixo+tu) < A for every t > 0, and so /<x>(w) = limj.+oo t~1if(xo+tu)—/(xo)) < 0. Hence [/ < A] 00 C [/» < 0]. Let now u £ [/oo < 0]. Then /(xo + 1 « ) — /(xo) < tfooiu) < 0 for every t > 0, and so xo+tu € [/ < A] for t > 0. Hence w € [/ < A]cx). Let x* € d€f(x) be fixed and consider u* G idefix))^. Then x* + tu* £ defix) for every t > 0. It follows that (x' — x,x* + tu*) < f(x') — f(x) + e for all x' G dom / and t > 0. Passing to the limit for t —• 00 we get (x' — x, w*) < 0 for every x' G d o m / , and so u* G iV(dom/;x). Conversely, if u* G N ( d o m / ; x ) then (x' — x,u*) < 0 for every x' G d o m / ; this together with (x' — x,x*) < f(x>) ~ fix) + £ f ° r x> e d o m / gives (x' — x,x* + tu*) < f(x') — / ( x ) + e for all x' G d o m / and t > 0. Therefore x* + <M* G 9 £ / ( X ) for £ > 0, and so
Fix (x*,A) G e p i / * and let (u*,7) G ( e p i / * ) ^ . Then (x* +tu*,\ + t^) G e p i / * , whence (x,x* + tu*) — / ( x ) < A + tj for all x G d o m / and t > 0. Taking the limit for t —• 00 we obtain that (x,u*) < 7 for every x G d o m / ; hence «dom/(w*) < 7, ie. (w*,7) G episaom/ Conversely, if Sdom/(w*) < 7 then (x,u*) < 7 for every x G d o m / ; this together with (x,x*) — f(x) < A for x G d o m / gives (x,x* + tu*) — / ( x ) < A + t 7 for all x G d o m / and t > 0. Therefore /*(x* + t u * ) < A + ty for t > 0, and so (u*,7) G (epi/*)oo Therefore (epi/*)oo = episdom/, which shows that (/*) The formula /oo = Sdom/* follows applying the preceding one for / * .
Exercises  Solutions
316
Let D := P r y ( d o m F ) and take v* £ Y* such by what precedes, SdomF(0,w*) < 0, and so (y,v*) Hence (j/,w*) < 0 for every y € D, i.e. v* € D~. (y>v*) < 0 f° r every y 6 D, and so SdOmF(0,v*) <
that (F*) o o (0,i'*) < 0; then, < 0 for every (x,y) € d o m F . Conversely, if v* € D~ then 0.
Exercise 2.24 (i) Since f<x, is a lsc function, it is clear that if is a closed convex cone and so Xo is a closed linear subspace of X. Let u £ l o and x £ d o m / . Taking into account Eq. (2.27), we have that f(x + u)
f(x) < /«,(«) < 0,
f(x u)
f(x) < fociu)
< 0.
< /«,(«) + foc(u)
< 0,
Adding these two relations side by side, we get 0 < 2 ( ± / ( x + u) + \f{x
u)
f(x))
and so f(x + u) — f(x) = /oo(w) = 0. Let now x £ d o m / and u G XQ. If x + u S d o m / , by what precedes we obtain that x — (x + u) + {—u) S d o m / ; hence x + u £ dom / and so f(x + u) = f(x). (ii) Note first that d o m / * C XQ", and so lin(dom/*) C XQ". Indeed, let x* G d o m / * ; taking xo £ d o m / and u £ Xo, we have (xo + tu,x*) < /(xo + tu) + f(x")
= /(xo) + f*(x*)
Vt € R,
whence (u, x*) = 0. Hence x* 6 XQ". To complete the proof it is sufficient to show that (dom/*) C Xo (<=> X5 1 C (dom/*)"1"1" = lin(dom/*)). So, let u € (dom/*)"1"; then Sdom/*(w) = 0, and so, by the preceding exercise, foo(u) = 0. Therefore (dom/*)" 1 C K, whence (dom/*)J"CXo. (iii) As Xo is a closed linear subspace of X , the quotient space X/Xo is a separated locally convex space, too (X/Xo being endowed with the quotient topology). Moreover, its topological dual is XQ", and its weak* topology is the trace of w* on XQ". When 1 6 I , its class x is x + Xo. From (i) we obtain that / is well defined. Moreover, e p i / = {(x,t) \ (x,t) € e p i / } = 7r(epi/), where TT is the canonical projection of X x R on (X x R)/(Xo x {0}). Hence e p i / is a convex set. Furthermore, because e p i / + Xo x {0} = e p i / , e p i / is also closed. Hence / is a proper lsc convex function. The relation /00(G) = /oo(w) follows immediately from (i) and Eq. (2.27). Using (i) one obtains easily the other two relations. Exercise 2.25 Replacing eventually / by e _ 1 /> w e m a y t a k e e = 1. Because /oo(w) < 0, the mapping R 9 t i  » f(x + tu) € R is nonincreasing for every x € X and d o m / + R+u = d o m / , as seen in Remark 2.2.2. On the other hand, since limt_>oo * _ 1 (f(x — tu) — f(x)) = foo(u) > 0 for x £ d o m / , for every such x
Exercises  Solutions
317
/ ( x  tu) > f{x) + £7 Vt>tx,
(°)
there exists tx > 0 such that
where 7 := §/<x>(—«) > 0. Consider g : X —• K defined by 3(3;) := exp(— t) for a; = tu with £ > 0 and g{x) := 00 elsewhere. Consider f\ := /• 0 such that / i ( x ) = / ( x  ?,«) + exp(t„) < f(x) + fl(0) = / ( x ) + 1;
(°°)
this fact is obvious because, by (°), limt><x> (f(x — tu) + exp(—t)) = 00. Moreover, because f(x) < f(x — tu) for every t > 0, we have that / i ( x ) = inf{/(x  tu) + e x p (  t )  t > 0} > inf{/(x tu)\t>0}
= f(x).
Therefore f < fi < f + 1. Let us show that f\ is lsc. Consider (xi)nsi —> x € X with {fi(xi))ieI —> fi< 00. We may assume that Xi € d o m / i = d o m / for every i 6 7. We must show that / ( x ) < fi. By (°°) we have that fi(xi) = f(xi — Uu) + exp(—U), where U :— tXi > 0. Suppose that (£») —> cx>. Then for every t > 0 there exists it € I such that £i > £ for i X it. Hence /(x< — £;u) + exp(—U) > f(xi — tu), for i y it, and so fi > f(x — tu) for every t > 0. This contradicts (°). Therefore (£i)ig/ has a subnet (t'j)j^j converging to t € R+; hence fi > f (x — tu) + e exp(—£) > / i ( x ) because / is lsc at x — tu. It follows that /1 is lsc. Since inf f\ = inf / + inf g — inf / and / i ( x ) = / ( x — txu) +exp(—t x ) > f(x — txu) > inf / = inf/1 for every x G d o m / i , we have that argmin/i = 0. Exercise 2.26 As proved in Proposition 2.7.2, condition (ii) holds if one of conditions (iii)(viii) is verified. As noted before this proposition, condition (ii) also holds for F, where F(x,y) := F(x,y)  (x,x*) with x* e X*. Taking ip :— F(,0), using Theorem 2.7.1, we have *>*(x*) = sup«x,x*> F(x,0)) xex = min y'&Y*
=  xinf F(x,0) = min €x y*ev
F'(0,y')
F*(x*,y*).
Therefore g is u;*lsc and the values of g are attained. It is obvious that tdomF satisfies condition (ii) of Theorem 2.7.1, too. So, using once again this theorem, we have, as above, that SdomV(u*)
— SUp ( ( X , X * )  tdom v(x)) = SUp ( ( x , X * )  tdom F {x, 0 ) ) x£X x€X =
min (tdomF)*(U*,V*) =
min
SdomF(u*,v*)
Exercises  Solutions
318
for every u* G X*. From Exercise 2.23 we have that Sdom^ = (
=
SU
P {{x^x*)9*{x*)))=mi[f(x)+
x*£X*
, i n t , [5*(a:*)sup ((x,x*)
mf
x^Ji.
 f(x))]
=
(g*(x*) 
(x,x*))]
x tA
inf
(g*(x*) 
f*(x*)).
Exercise 2.28 Let x G dom / be fixed and consider y>: P >• R,
• (iv) Let (x,x*) G g r d / . Then (—x,x") < f'(x,—x) < —pf(x), whence the conclusion. Assume now that / is continuous at 0; of course, 0 G int(dom/) and / is continuous on int(dom/) in this case. Moreover df(x) ^ 0 for every x G int ( d o m / ) , and so the implication (iv) =>• (v) is obvious. (v) =>• (i) Consider x G int(dom/). Taking x* G df(x) furnished by our hypothesis, we have that f'(x,x) > (x,xm) > pf(x). Hence pf(x) < f'(x,x) for every x G int(dom/). Let now x G d o m / . Then tx G i n t ( d o m / ) for every t G int I. As in the proof of the implication (iii) =>• (i) we obtain that the function ip is nondecreasing. Exercise 2.29 Take first x* G ffi=1df{xi). Then {x  X i , x * ) < f(x)  f(xi) for x G d o m / . Multiplying with Ai, then adding side by side the corresponding inequalities, we get {x  x,x*) < / ( x )  c o / ( x ) for x G d o m / . Take now (x,t) = H^i^jfajttj) € co(epi/) with (xj,tj)jeT^ C e p i / and A G A p . From the
Exercises  Solutions
319
preceding inequality we get, as above, (x — x,x*) < t — co/(57). It follows that (x — x,x*)
XiXi + A io x x,x")
whence (a; — x~i0, x*) < f{x)—f(xi0)
< y*
~\if(x~i) +
\i0f(x)cof(x),
because Xi0 > 0. Therefore x* G Pli=i df(x~i).
Exercise 2.30 Let / 0 := f\x0Then dfo{0) = {x* \x0 \ x* 6 9/(0)} and / ' ( 0 , x ) = fo(0,x) for x G Xo, / ' ( 0 , x ) = 00 x e X \ Xo Since /o is continuous at 0 (G XO), we have that /o(0,x) = max{(x,x*)  x* G 9/o(0)} for all x G Xo. Hence, for x G Xo, / ' ( 0 , i ) = m a x { ( x , i * )  x * G 9/o(0)} = sup{(x,x*)  x* G 9/o(0)} = sup{(x,x*x 0 > I a;* € 9/(0)} = sup { f o x ' )  x* G 9 / ( 0 ) } . If x $ Xo = XQ"1, there exists x* G X* such that x"*x0 = 0 and (x,37*) 7^ 0. Taking x*0 G 9/(0) ( / 0 by Theorem 2.4.12), we obtain that sup{(x,x*) I a;* G 9/(0)} > sup {(x,x"0 + tx*) \ t G R} = 00 = /'(0,57). Let now i 6 I o \ Xo Since /o is continuous at 0, by Theorem 2.2.11, /o is Lipschitz at 0, i.e. there exists Vo G N x 0 such that Vo C dom/o and /O(:E) — fo(y)\ < pv0(x — y) for all x,y G Vo, where py 0 is the Minkowski gauge associated to Vo. It follows that /o(0, x) < pv0(x) for all x G X0. Replacing eventually Vo by a subset, we may suppose that Vb = V fl Xo, where V G !Nx • So we have that /o(0, x) < pv(x) for all x € Xo It follows that V i e X o , Vi* G d / ( 0 ) : {x,x*) =
(x,x*\Xo)
As x G Xo, there exists a net (XJ) C Xo converging to x. From the relation above, taking into account the continuity of x* G 9/(0) a n d p v , we obtain that (51, x*) < pv(x) for all x* G 9/(0), and so sup{(x, x*)  x* G 9/(0)} < pv(x) < 00. Exercise 2.31 We take x / y (the other case being obvious). Consider the function ip : R —> R, (^(t) = / ( ( l — t)x + tj/) — t(f(y) — / ( x ) ) . Because
+
(tt0)(yx))~f(z).
320
Exercises  Solutions
Dividing by t — to for t G]£o, 1] and taking the limit for t —> to we get f(y) — f(x) < f'(z,y — x), while by dividing by t — to for t G [0,
= max{(y  x, z*) \ z* G = mm{{yx,z*)
\ z*
df(z)}, €df(z)},
we obtain the conclusion. Exercise 2.32 (i) The equivalence of a) and b) is true for any multifunction T : X =3 X*, and follows immediately. a) =>• c) Suppose that df is single valued on domdf. Assume that there exist distinct Xo,x\ G X* and A G]0,1[ such that [xo,x] C Imdf and f{x\) — (1  A)/*(x3) + Xf'(xl), where x*x := (1  A)xS + \x\ e Imdf. Let x 6 X such that x^ G df{x). Then, by YoungFenchel inequality, o = /(*) + / * ( * ! )  < * , * A > = (i  A) (/(x) + /•(*;) 
(x,x*0)) + A(/(X) + /•(*;>  <*,*:» > o.
It follows that / ( # ) + /*(x*) — (x, x*) = 0 for i G {0,1}, whence the contradiction
{x'0,xl}cdf{x). c) => a) Assume that df is not single valued on domdf. Then there exist x G X and distinct xS,xJ G df(x). Of course, [x3,xl] C df(x) C Imdf. Hence f(x) + f(x'x){x,x*x) = 0 for AG [0,1], where x j := (1  A)xS + \x{. It follows that f*(x*\) = (1 —A)/*(xS) + A/*(xi), which shows that / * is not strictly convex on [zo.aJi]Assume now that / is continuous on D := i n t ( d o m / ) and D / 0. Suppose there exists x G dome*/ \ D. By Theorem 1.1.3 there exists x* ^ 0 such that (x — x,x*) < 0 for every x G d o m / . Taking x* G df(x), we obtain that x*+tx* G df(x) for every t > 0, contradicting a). (ii) Consider the topology w* on X*. Then (/*)* = / and df* = (df)'1. By (i) we obtain that / is strictly convex on every segment [xo,xi] C dom df. But, by Theorem 2.4.9, i n t ( d o m / ) C dom df. The conclusion follows. Exercise 2.33 Let (x*,y*) G df(x0,yo); it is obvious that x* G df(,yo)(xo) and y* G df(xo,)(yo), even without assuming that / is continuous at (xo,yo). Let x* G df(,yo)(xo). Since / is continuous at (xo,yo) G d o m / , the function f'{(xo,yo),) is finite, sublineax and continuous. Furthermore, since x* G df(, yo)(xo), we have that VxGX
: (x,x*) < f{xo + x,y0) 
f(x0,yo),
and so V x G X : (x,x*)
Exercises  Solutions
321
Consider the linear subspace X x {0} of X x Y and the linear functional tpo '• X x {0} —> R, <po(x,0) := (x,x*); taking into account the above inequality and the HahnBanach theorem, there exists a linear functional y i l x Y ^ R such that
: tp(x,y) < f'((x0,yo),
(x,y)).
Since f'({xo, yo), •) is continuous, from this inequality we get that
• 1) is clear. 3) =*> 2) Let 2a := liminfxxx, ip(x)/x > 0; then there exists p > 0 such that ax for every x > p. Let x > 0 be such that ip(x) = 0; hence (x,0) € epitp — co(epi / ) C R 2 . Using Caratheodory's theorem (Exercise 1.1), for every i g {1,2,3} there exist the sequences (AJJ, (x\), (tln) C R+ such that: A> + A2 + A3 = 1, ¥>(*») < ti for all n e N and 1 < i < 3, ( £ 3 = 1 A n < ) * * and ( 2 f = i ^n*n) —> i>ix) = 0 Without loss of generality, we assume that (x\)>x\ ax'n, and so \ntn > a\nxln. So we get the contradiction 0 = T{ > ay{ > 0. Therefore y1 = y2 = y3 = 0, and so x = 0. 1) => 3) Suppose that liminfzKx, ip(x)/x = 0; then there exists (xn) C R+ such that (x„) —¥ oo and ip(x„) = anxn with (a„) —¥ 0. Let x € P; there exists n* € N such that xn > x for n > nx. Set A„ := x/x„ €]0,1[ for n > nx\ then K(xn,f{xn)) (t) = 0 <S> t = 0} the affine hull of the set A (p. 2) the set {xeX\ f[x) = inf / } for / : X + I the set {y € (X, d)  d(y, x) < e} the set J3(0,1) in a normed space (X,  • ) the class of all bounded subsets of the tvs X the set cl A \ int A the adjoint of the convex process C (p. 26) the cone cl (cone(A — a)) the nvs of real continuous functions on the interval [a, b] the set of convex and Lipschitz functions on (X,  • ) the closure of the set A C (X, r ) the convex hull of the set A (p. 2) and co A, respectively the lsc convex hull of the function / (p. 63) the conic hull of the set A (p. 2) and cone A, respectively the upper Dini directional derivative (p. 58) the set {y G (X, d) \ d(y, x) < e} metric or distance 363
F{x,t):={ fo{x) ' \ CO
* if
f {
} *\^\ fl(X) > t,
and its associated marginal function h : R —$• R; h is a decreasing convex function such that /i(0) = v G R and /i**(0) = v*. By Theorem 2.6.1(v) we have that v* = v if and only if h is lsc at 0. Since h is decreasing, we have: v*
= w « . V e > 0 , 35>0,Vte]6,5[\ &Ve>0,
3<5>0
h(t)
>ye
/i(<5) > v  e
^ • V e > 0 , 35 > 0
[fi(x) < 6 => f0(x) > y  e]
^Ve>0,
[fo(,x) < v  e =*• fi(x) > 5]
35>0
& Ve > 0 : inf{/i(z)  f0(x) < v  e} > 0, i.e. the conclusion holds. Exercise 2.35 "=>" If x* < / then /(a;) > (x,x*) >  x • x* for every x € X, and so the conclusion holds with M = \\x*\\. "•$=" Let g : X >• R be defined by g(x) := Mx. The function g being continuous, from the FenchelRockafellar duality formula we have 0 < inf (/(x) + M   x   ) = mj« (  / • ( * ' )  V (  * * ) ) =
max
(/*(**))•
Since —/*(x*) = inf^ex (/(x) — (x,x*)), the conclusion follows. Exercise 2.36 "=>•" If  / i < x* + a < f2 then / i ( x i ) >  (xi,x*>  a and / 2 ( x 2 ) > <x 2 ,x*) + a, and so / i ( x i ) + / 2 ( x 2 ) > (x 2  x i , x * ) >  x 2  x i      x *   for all x i , x 2 6 X, and so the conclusion holds with M = \\x*\\. "<=" Let f,g : X x X > 1 be defined by / ( x i , x 2 ) := / i ( x i ) + / 2 ( x 2 ) and g{xi,X2) := M   x 2 — Xi. The function g being continuous, from the Fenchel
322
Exercises  Solutions
Rockafellar duality formula we have 0<
inf
(f(xi,x2)
+ g(xi,x2))
=
max {f*{x{, x*2)  g*(x\,
x\tX2^X
x\)).
x^,x^€X*
But g*(—x{, —x2) — 0 if x\ = — x2 € Ux*,
=
inf
(fi(xi)
+ f2(x2)
+ (xi,x*)

(x2,x*))
X\,X2&X
= inf (/i(xi) + ( x i , x * ) ) + inf xi£X
(f2(x2)(x2,x*)).
!2fcA
Note that both infima are finite. Taking a := inf^gx (fi{x) + (x,x*)) get the conclusion.
S R, we
E x e r c i s e 2.37 The implication "^=" follows immediately from the obvious inequalities  2 (y,y*) <2\\y\\ • y* <  y  2 + y* 2 . "=>•" Consider the topology cr(X, X') on X; so X is a separated locally convex space and T : X —• Y is a continuous linear operator. Take g : Y —>• R defined by g(y) '•= \\y + 2/o2 The function g being continuous, from the FenchelRockafellar duality formula we have 0 < inf• (f(x) + \\Tx + y0\\)= xzx
max (  / * ( T V ) ~
But g'iV*)
= (2/o,2/*> + i   » *   2 ,
 / * ( T V ) = inf ( / ( * )  < T * , * * ) )
(see eventually Theorems 2.3.1 and 3.7.2); the conclusion follows replacing y* by 2y*. E x e r c i s e 2.38 Taking / : X t K defined by f(x) := \\x — x\\ and g :— ic, we have that d(x,C) = inf x e x (f(x) + g(x)). We notice that f*(x*) = (x,x*) + tux, (x") and g* = t_c+. Because / is continuous on X, applying the FenchelRockafellar duality formula we get the first formula. In order to prove the second formula take / := x* + iux and the same g as above. Notice that f*(x") = \\x* — z* for x* € X*. Because / is continuous at 0 6 d o m / n domp, we can again use the FenchelRockafellar duality formula. Hence i n f x e u x n c (x,x*) — max a ,. g c + (— x* — x*). The conclusion follows. Using the previous formulas we get sup xeuxnc
d(x,D)=
sup xeuxtic
sup x*eux,nD+
(— (x,x*)) —
sup x*eux.r\D+
d(x*,C+).
323
Exercises  Solutions
Exercise 2.39 The inclusions [l,oo[df(x) C df(x) C d< f{x) are obvious < (and true without convexity). Let x* G d f(x) and x € [/ < /(x)]; there exists x0 e X such that f(x0) < f(x). It follows that (1  X)x + Ax0 € [/ < /(x)] for every A €]0,1[, and so ((1  X)x + \x0x,x*)
((1  X)x + Xx0)f(x)
<
Taking the limit for A —> 0 we obtain that (x — x,x")
(lX)f(x)+Xf{x0)f{x). < f(x) — / ( x ) , and so
x* e d±f(x). Let now x* € dfix); this means that x is an optimal solution of the convex problem (P) min f{x)  (x, x*), g(x) := f(x)  / ( x ) < 0. By Theorem 2.9.3, there exists A > 0 such that 0 e d(f  x* + Xg)(x) d{(l + A)/  x*), i.e. x* e (1 + A)3/(x). Therefore x* e

d
Exercise 2.40 1) It is obvious that (c) => (a) and that (c) =£• (b). (b) => (c) Let XQ € d o m / ; therefore ^Cn^iAnlko — o„ 2 < oo. Let x € X be fixed. Then II*  an 2 < (\\x  xo + xo  anil) 2 < 2(x  x 0  2 + x 0  a„ 2 ). So we obtain that f{x) = Y°°
A „   x  a „   2 < 2x  xo 2 + 2Y" 0 0 A n x 0  a„ 2
<2xxo2+2/(xo),
(')
which shows that x e dom / . Taking xo = 0 in the above proof, we have that (a) =£• (c). 2) Suppose that ^ ^ j A n U a n H 2 < 00. By 1) we have that d o m / = X. Consider the function fn • X —¥ R,
f„ := X\dai + • • • + X„dan,
where da := d{a} : X  > R , da(x) := x — a\\. It is obvious that limn»oo / n ( x ) = / ( x ) for every x E X. Since / n is a convex function for every n € N, using Theorem 2.1.3(iii), we obtain that / is convex. Applying relation (*) for xo = 0, we obtain that f{x) < 2 + 2/(0) for all x e Ux Since 00 > / ( x ) > 0 for every x 6 X, it follows that / is finite and continuous on X. 3) Assume that X^nt=iAnan2 < 00. First note that the series ^nt=i ^n ll a "ll is convergent; this follows immediately from the inequality 2An a n  < An + An an 2 .
Secondly, recall that ddl(x) = 2da(x)dda(x) x,a € X.
= 2 x  a\\ • d\\\\ (x  o) for all
324
Exercises  Solutions
Let x G X be fixed and consider x*n G 9 (x — a„) for every n > 1. Since xJi G f/x* and An x — an\\ < \ n \\x\\ + X„ \\an\\, we obtain that the series S n > i 2^™ 11^ — "n x„ is convergent; let a;* be its sum. Because f{u) = lim/„(M) for every u G X and Y^l=x 2 ^* \\x ~ a *ll xl e dfn{x), we obtain immediately that x* £ a / ( x ) . Take now x* G df(x). Consider the function h„ := / — / „ ; the function hn is also convex and continuous. It is clear that hn = A„+id 2 + hn+i, and so dh„{x) = \n+iddln+1(x) + dhn+\{x) for all n > 1. As x* G df(x) and / = Aid„j + hi (and so d / ( x ) = A i S d ^ x ) + dh\(x)), there exists x* G 9 (a; — ai) such that x* — 2 x — ai x\ G dhi(x). As hi = A2d„2 + /12, there exists X2 G 51•  (x — 02) such that x* — 2 x — a\\\x\ — 2 x — a2x2 € dh2(x). Continuing in this way we obtain a sequence (x£) n >i C X* such that x* G 9 (x an),
x*  ^2
2 \\x  ak\\ x*k € dhn(x)
Vn G N.
But for n G N and u G J7x we have that (h„)'(x,u)
< hn(x + u) hn(u)
Afc (x + uajfc 2  x  a j t   2 )
= Y^ *—•'k=n+l
'
< rn \\u\\, where
E
oo
A*(2x + 1 + 2  K   ) . Of course, (r„) * 0. It follows that (hn)'(x,u) < rn \\u\\ for every u £ X, and so dhn(x) C r„Ux« Therefore x*  Y2=i 2 Hx _ a*H x *il ^ r " f o r e v e l T n e N> whence we get x* = 2 X3 n >i Wx ~ a "ll x n Hence the formula for df(x) holds. Exercise 2.41 It is obvious that (d) =>• (e) =>• (a) =>• (b) => (c); moreover these implications hold for an arbitrary function / . (c) => (b) Let xo G X be such that /(xo) < A and A > A. Consider x G [/ < A] and t := (A  A)/(A  / ( x 0 ) ) G ]0,1[. Then f(tx0 Therefore
+ (1  <)x) < tf(x0) _ A—A A  /(xo)
+ (1  t ) / ( x ) < tf(x0)
+ (1  «)A = A.
_ A — /(xo) A  /(xo)
., , T ,
whence we obtain that x G
A — f(xo)
T
t)
r.
^ ri
A—A
' [ /  A ] " r ~ 7 T ^ •Xo
A  f(xo) A  /(xo) Therefore [/ < A] is bounded. The conclusion evidently holds for A < A. (Note that we didn't use the convexity of / . )
Exercises  Solutions
325
(b) =>• (a) The proof is immediate by way of contradiction (without any condition on / ) . (e) => (d) Since (e) is true, there exist a,p > 0 such that f(x) > a\\x\\ for every x G X, \\x\\ > p. Since / G T(X), by Theorem 2.2.6, there exist x* G X* and 7 G R such that f(x) > {x, x*) + 7 for every i f l . Therefore V x G X , \\x\\
: /(x)>x.x*+7>px*+7.
Taking /? := min{0,7  px*}, condition (d) is satisfied. (a) =>• (e) We show it by way of contradiction. Therefore there exists a sequence (x„) C X such that (x„) —• 00 and lim„>oo/(x n )/x n  = p> < 0. Therefore, for every fc G N there exists n'fc G N such that V n e N , n > n ' f c : x„ > fc2 and / ( x n ) /   x n   < 1/fc; hence there exists an increasing sequence (n^kzn C N such that f(x„k)/\\x„k  < 1/fc and   x „ J  > fc2 for every k. WefixxG dom / and consider tk := lxnfc/fc > k and yfc := (1  ±)x + ^ i n t . Then (\\yk\\) »• 00 and /(»*) < (1  **') / ( ^ ) + tk'fixn,)
< 1 + max{0,
f(x)},
a contradiction. Therefore the implication (a) => (e) is true. (d) => (f) Let g : X > R, g(x) := a • x + (3. The function 3 is convex; using Theorem 2.3.1 and Corollary 2.4.16 we have that g*(x*) = Laux, (x*) — j3. Since / > g, we have that /* < g", whence aUxm C dom/*; the conclusion follows. (f) =>• (d) Since /* is u>*lsc, /* is  • lsc, too; since 0 G int(dom/*), using Theorem 2.2.20 it follows that /* is continuous at 0. Therefore there exist a,/3 € R, a > 0, such that /*(x*) < fi for every x* G aUx* Therefore /* < Laux. +P, whence, taking the conjugates, we obtain that / ( x ) > /**(x) > a • \\x\\ — /3 for every x G X. Let dimJf < 00. The implication (b) => (c') is obvious. Assume that (c') holds but (b) does not. Then there exist A € R and (x„) C [/ < A] such that (x„) »• 00. We may assume that (   x n   _ 1 x „ ) > u / 0. It follows that u G [/ ^ M°°, and so, by Exercise 2.23, u G [/ < inf /],*,, contradicting the boundedness of [/ < inf/]. Suppose now that p,q G ]1, co[, 1/p + 1/q = 1. The implications (i) =*• (g) and (j) =^ (h) are obvious, while the proof of the implication (g) => (i) is similar to that of (e) => (d) above. The equivalence (g) •£> (h) follows immediately from the antimonotonicity of the conjugation and using the relation ( i   •  p )* = i   • \\q. Suppose that (h) holds. Then there exist a,p > 0 such that /*(x*) < ax* ? for every x* G X*, x* > p. Since d o m / * is a convex set, the preceding inequality shows that d o m / * = X*. Let
326
Exercises  Solutions
x* e X* \ {0}, a;* < p; consider x* := px7x*. Then
n o
=,. (M,.
+
(i _ M) 0 ) < H r ( f ! , + (i _ M)/(o)
< a p « ^ + f l  J £ ! h / ' ( ( ) ) < ap" + m a x { 0 , / * ( 0 ) } . p V p i Taking 0 := ap« + max{0, /*(0)}, (j) is verified. E x e r c i s e 2.42 Let 7 > 0 be such that  / n ( 0 )  < 7 for every n € N. By the preceding exercise there exists r > 0 such that f(x) > 37 for x € rSx Since rUx is a compact set, by Corollary 2.2.23 we have that (/„) converges uniformly to / on rUx, and so there exists no £ N such that \fn(x) — f(x)\ < 7 for x 6 rUx and n>no; hence fn(x) > 27 for x € rSx and n > no Let x 6 R m be such that x > r and n > no. Then
2lS
' • (iSl' + i1 ~Hi)°) s H A W + ( '  H i ) M 0 ) s H /  ( I , + 7 '
whence fn{x) > r~lry \\x\\. Taking m := inf / ( R m ) , which is in R by the preceding exercise, we obtain that fn(%) > m — 7 for x € r£/x and n > n 0 . It follows that fn(x) > a\\x\\ + [3 for x € R m and n > n 0 , where a := r_1f and /3 := min{m — 7, 0}. E x e r c i s e 2.43 The implication (i) => (ii) is obvious. Assume that (ii) holds. Hence there exists xo 6 X and XQ 6 X* such that <x,x*.>>(x 0 ,x*) +   x  x o   >   x    / 3
Vx 6 C,
(°)
where /? := xo — (xo,xo). If y5 < 0, (i) obviously holds. Assume that f3 > 0. Since 0 $ C, there exist x* £ X* and S > 0 such that (x,x*)>5
VxeC.
(°°)
Multiplying (°°) with 7 := <S1/? > 0 and adding the result to (°) we get {x, XQ + 7X*) > x for every x £ C. Note that conditions (ii)—(iv) are invariant under translations; so we may assume, and we will, that 0 £ C. (i) =>• (iii) Prom (i) we have that / := ic+xo satisfies condition (d) in Exercise 2.41, and so, by the same exercise, 0 € int(dom/*). Since /*(x*) = L*C{X* —XQ) = sc(x* — XQ), we have that d o m / * = XQ + domsc Hence (iii) holds. (iii) =>• (i) Assume that (iii) holds, and take — X*, £ int(domsc). Then 0 6 int(dom/*) with / defined above. By the implication (f) => (d) of Exercise 2.41 we have that (X,XQ) > a x — (5 for every x 6 C, for some a, {3 € R, a > 0.
327
Exercises  Solutions
Multiplying eventually with a  1 , we may suppose that a = 1. As in the proof of the implication (ii) => (i) above, we obtain that (i) holds. (i) => (iv) Let c £ C be fixed and take u e Coo, and t > 0; of course, c+tu € C. From (°) we obtain that (c + tu, x*0) > \\c + tu\\ > t u  c
Vt > 0;
letting t —> oo, we get {u, Xo) > \\u\\. Hence (a) holds. Moreover, if (xn) C C with (H^nll) —• oo and (   x „   _ 1 x n ) ^ u, we have that (   x n   _ 1 xn,Xo) > 1, whence (u,Xo) > 1. T h u s u / 0 . Assume now that X is a reflexive Banach space. (iv) =$• (i) Let Xo G X* be such that (u, XQ) > 0 for every u E Coo \ {0} and x{ 6 X*, 5 > 0 satisfy (°°). It follows that ( u . ^ ) > 0 for every u G OooThere exists i) > 0 such that M ) >  T ? Vxec. (°°°) Otherwise there exists (xn) C C such that (xn,Xo) < —n for every n G N. Since ll^nll • H^oll > «, we have that x„ ¥ oo. Passing to a subsequence if necessary (X being a reflexive space), we may suppose that a;n _1 xn ^y u. It follows that u € Coo (C being closed). By (b) we have that u / 0. Since ( a; n  1 xn, xS) < 0, we obtain that {u, XQ) < 0, contradicting the choice of XQ. Hence (°°°) holds. Replacing XQ by J _ 1 (?j + S)xl + XQ, we have that (°°) still holds and (u, XQ) > 0 for \ {0} (i.e., we may take x\ — XQ). Assume that C does not satisfy (i). Then for every n 6 N* there exists xn € C such that (nS <) (x„,nxo) < \\xn\\. Hence a; n  —> oo. As above, we may assume that   x n   _ 1 xn A w; hence u € Coo, and so, by (b), u ^ 0. But, since ( a;n _ xn,xo) < n1 ,we have that (u, XQ) < 0, a contradiction.
E x e r c i s e 2.44 (a) => (b) Suppose that (a) holds. Consider xo € X such that f(xo) < A. Let x £ X; it is clear that (b) is true for a; < p. Consider that a; > p; then f(x) > A. Since for f(x) — oo the inequality (b) is obviously true, let f(x) € R If f{tx + (1  t)x 0 ) < A for every t e]0,1[ we have that \\tx + (1 — £)xo < p for every t G]0,1[, and so the contradiction x < p. Therefore there exists t\ €]0,1[ such that f(t\x + (1 — ti)xo) > A. By Theorem 2.1.12(vii), there exists t2 €]0,1[ such that f(t2x + (1  t2)xo) < A. Since the function ]0,1[9 t tt f(tx + (1 — t)xo) is convex, it is continuous. So, there exists t e]0,1[ such that A = f(tx + (1  F)x 0 ) < t / ( x ) + (1 
t)f(xo).
But t   i    (1  t )\\x0\\ < \\tx + (1  t )x 0  < p,
Exercises  Solutions
328
and so t < {p+\\xo\\)/(\\x\\
+ \\x0\\).
Therefore
A < t(f(x)  /(*„)) + /(*„) <
II^^+^HII
• (/(*)  /(x°)) + /(*»)•
Thus
/() > /(*o) + J ^ ± M . (A  /(,„)) = M ^ A . (A _ / ( l o ) ) + A P+IMI
P+ll*o
>JI4^(A/(«o)) + A. Ip Since f(xo) can be taken as close to inf / as wanted, we have that /(x)>A+M^(Ainf/)>inf/+M^.(Ainf/), which shows that the conclusion holds. (b) => (c) Set p := (A  inf /)/(2p) and consider g : X >• E, g(x) := p,max{0, x — p}; g is a continuous convex function. For computing g* we could apply the conjugation formulas established in Section 2.8, but we proceed directly. g*(x*) = s u p x g x ((x,x*)  pmax{0, \\x\\  p}) = max sup W < p (a;,x*),sup N  > p ((x,x*)  p(\\x\\  p ) ) } = maxpx*,sup I > p ((x,x*)  p\\x\\) + ppj . But sup x  >p ((a;,a;*) —/ix) = oo if x* > p, while if x* < p then <*,**>  p\\x\\ < * • (z*  p) < p(\\x*\\  p), and so sup ((x,x')p\\x\\)+pp IWI>p
Therefore g*(x*) = max{p • a;*, i^ux.
(x*)}.
Since / > inf/ + g and inf/ = — /*(0), by conjugation we obtain that /* < /*(0) + g*, and so (c) holds. (c) =>• (b) Using (c) we obtain that /* < /*(0) + g*. Since g** = g, by conjugation we obtain that (b) holds. E x e r c i s e 2.45 (a) The mentioned equivalence was established in Exercise 2.41. Let p > 0 be such that liminfuxntoo / ( x ) /   x   > p. Then there exists p > 0 such that f(x) > p • \\x\\ for x > p. Since / S T(X), / is lower bounded on
329
Exercises  Solutions
liUx, and so there exists 7 € R such that f(x) > n]\x\\ + 7 for every i £ l , As in the proof of Exercise 2.41 we obtain that /* < iMc/x» — 7. Therefore /* is upper bounded on [iUx* • Since \i £]0,liminf:E_+(xl /(aO/z[ is arbitrary, we obtain that the inequality "<" of the relation to be proved holds. Suppose now that n > 0 and /* is upper bounded on pUx* • Then there exists 7 £ R such that /* < t M c/ x . — 7 Taking the conjugates, we obtain that f(x) > fJ.\\x\\ + 7 for every x £ X, whence liminf x _j 0 0 /(x)/a; > fx. So we have obtained the inequality ">" of the desired relation. Before beginning effectively the proof of the other assertions let us do some remarks. If /j > 0 and g:X*R,
g(x) := f(x) + ii\\x\\,
then, by using Theorem 2.8.7, we have that g*(0) = min{/*(x*) + ipvx. (—X*) \ x* £ X*} = min{/*(a;*)  x* £ fiUx*}Let fj, > 0 be such that liminfa._KX) /(a;)/a; > — fj,. By what precedes, we get the existence of 7 6 K such that f(x) > — fj,\\x\\ + 7 , i.e. g(x) := f(x) + (J,\\X\\ > 7, for every x € X. Therefore 0 € domg*. By the preceding displayed relation we have that fiUx' D d o m / * ^ 0, and so d<jom/*(0) < /J.; hence ddom/*(0) > min{0, liminf j ; _ > 0 0 /(x)/x} . Let now (x„) C d o m / * be such that (xnll) —^ ddom/*(0). Using the YoungFenchel inequality, for every n 6 N we have that V x e l :
/ ( * ) > (x,x'n)  f(x*n)
> Hsll • Hi'll 
f K ) ,
and so, dividing by i; and then passing to the limit inferior for x —> 00, we get V n e N : liminf B _ M)0 /(*)/a! >   K   . Taking now the limit, we have that liminf a ._ HX> /(a:)/x > d d om/*(0). So we obtained that min{0,liminf a ! _ > o o /(x)/i} = d d om/*(0).
(*)
Let us prove now the other two assertions. (b) If liminf a: _, 00 /(a;)/a; = 0, by (*) we have that 0 € cl(dom/*); using (a) we have that 0 ^ int(dom/*). Therefore 0 £ Bd(dom/*). Conversely, if 0 6 Bd(dom/*) we have 0 ^ int(dom/*), and so, by (a), liminfx_KX, /(x)/a; < 0; using the fact that ddom/*(0) = 0 and (*) we obtain liminf;E_+0O / ( x ) /   x   = 0.
330
Exercises 
Solutions
(c) If liminfii^ntoo / ( x ) /   x   < 0, using (*) we obtain the equality from (c) and 0 < ddom/*(0), i.e. 0 ^ cl(dom/*). Conversely, if 0 $ cl(dom/*) then ddom/*(0) > 0; using again relation (*), it follows that liminfrE_+00 / ( x ) /   x   < 0. Exercise 2.46 Let x = (x„)„>i € £p; then (x„) —• 0, whence there exists ls no > 1 such that \xn\ < \ for n > no Since the series 52n>in/2n convergent and n  x n  n < n / 2 " for n > no, it follows that the series ^2n>1n\xn\n converges; therefore x € d o m / . The function (P B \n £ M is obviously convex. It follows that the function
fn:t?^m,
/»(*):= V "
*—Jk=l
k\xk\\
is convex, too. Since f(x) = lim fn(x) for every x 6 lv', using Theorem 2.1.3(H), we obtain that / convex. Let p 6]0,1[ and x = (xn)n>i € pUtv. Then \xn\ < p for every n > 1. It follows that
MxtpUtv
: }{x)
npn € R.
The function / being convex, using Theorem 2.2.13, we obtain the continuity of / on dom / = V. Since f(e„) = n, where e„ = ( 0 , . . . , 0 , 1 , 0 , . . . ) , 1 being on position n, f is not bounded on pUiv for every p > 1. Applying Exercise 2.45 (a) for f*:£q = (£p)* —t R, where q = p/(pl), since (/*)*  f, we have that liminfiij,^,*, f*(y)/\\y\\ = 1. Exercise 2.47 (a) Let K := A(C); it is obvious that if is a convex set. Using Corollary 1.3.15 we have that K is also a closed set. Since the function Y 3 V '* IMI 2 +1K(V) is convex, lsc and coercive, and the space Y is reflexive, using Theorem 2.5.1(h), there exists y € K such that y 2 < f M 2 for every y S K. Taking x e C such that y = Ax we obtain that x is a solution of problem (P). (b) Let x e C and / : X >• R, f(x) := iAx 2 . By the PshenichnyiRockafellar theorem, x is a solution of (P) if and only if df(x) D (—N(C;x)) ^ 0. Since d(±\\ • \\2)(y) = {y} for y £ Y, from Theorem 2.8.6 we have that df{x) = {A*(Ax)} for x 6 X. Therefore x is a solution of (P) if and only if A*(Ax) e —N(C;x), i.e. (x\v) < (x\v) for every x € C, where i; = A*(Ax). Exercise 2.48 Using Exercise 1.5 (c) we have that C + ker A is a closed set, while using Exercise 2.47 (a) we have that (P) has optimal solutions. Suppose that there exists x with the mentioned property; then Slater's condition is satisfied for the problem min v4x 2 ,
(x, ipi)  on < 0, 1 < i < k.
The conclusion follows applying Corollary 2.9.4.
Exercises  Solutions
331
Exercise 2.49 Consider the function / : X > R, f(x) := \\x — oi + \\x a.211 + \\x — d3. It is obvious that the function / is convex, continuous and limI._>00 f(x) = oo, i.e. f is coercive. The function / is even strictly convex. Indeed, for x,y G X, we have that \\x + y\\ = \\x\\ + \\y\\ if and only if there exists A,n > 0, A + fj, > 0, such that Xx = py (i.e. x, y are on a halfline passing through the origin). Using this fact, if there exist x, y G X, x / y and A G]0,1[ such that f(Xx + (1 — X)y) = Xf(x) + (1 — X)f(y), then 01,02,03 are on the straightline determined by x,y, which is a contradiction. Therefore there exists a unique element x G X such that f(x) < f(x) for every x € X. Moreover x is characterized by 0 € df(x). But f)f/v\f
T,l
if
^Xdi)
u g {ai,a2,a3},
Denote ej := (x — ai)/\\x — aj when x ^ a;. If x ^ (ai,02,03}, using the relation ei + e2 + e3 = 0 we obtain that
V3
"\ 1 v  3
1
^i^WxaiWj
a<
^i=x\\xai\\'
and so x G 00(01,02,03}. If x £ {01,02,03}, of course x G 00(01,02,03}. Therefore the conclusion holds. Prom 0 G df(x) we obtain that x = 03 if and only if e\ 4 ei G Ux, which is equivalent to (eie2) < — \, i.e. ^(01,02,03) > 120°. Suppose that x g {01,02,03}. Then ei + e2 + ez = 0, whence (e; \ej) — — for i / j . Therefore Z(o;, x, aj) = 120° for i / j . Thus we obtained the known properties of Toricelli's point. Exercise 2.50 For every n > 0 and i G {1, 2, 3,4} we consider the function
U.Xi^
R,
/„(*) := J
(tx(t) + py/\ + (x(i))2)
Prom Exercises 2.9 and 2.10 we have that / M is convex; moreover / is a C2 function on Xi and X3 and only Gateaux differentiable on X2 and X j . In every case we have that Vx.ueXi
: V/M(x)(U)= T L w + M  ^ M g ^ )
Jo V
V 1 + (*W)2/
dt.
Moreover v(Pt)
> v(P£),
v(P?) > «(P 4 ").
Let us take i = 2 and ^ > 0. The function to be minimized being convex and Gateaux differentiable, x^ € X2 is a solution of (.Rf) ^ anc ^ o n l y ^ ^fp{ x v) = 0
332
Exercises 
Solutions
It is obvious that V/o(z) / 0 for every x € X2. So, let fj, > 0. The condition V / M ( J ; M ) = 0 is equivalent to
VBEXJ
:
f
(t + n
Jo \
*jj(f)
)u(t)dt = 0,
2
V l + W*)) /
which is, obviously, equivalent to each of the following relations: XjAf)
V l + (*M(*))a
+
i = 0 a.e. t € [0,1],
x^t)
=
"
~*
a.e. t e [0,1].
v / ^ ^
The function xM above is in X2 if and only if /u > 1. Therefore for \i > 1 the problem (P^) has a unique optimal solution, xM, while for /x < 1 the problem (P^) has no optimal solutions. Let us determine ^(P^) when fj, £ [0,1[. For every a e]0,1[ and /3 € ] 1 , oo[ consider the function
^:[O,I]^R,
« a ,(t):={_ l(t _° 1+a) ;[
^L1;;};
An elementary computation shows that = M (l  a) + a/3 (   I + f ^ l + ^ + ^ I n (/? + V ^ + 0 5 ) ) •
U{xa,p)
It follows that fii{xn\n2) —> —00 for // € [0,1[, and so ^(P^) = —00. Let now » = 1. Since x n  i n2 € X\ we obtain that v{P£) = —00 for /z € [0,1[. Doing again the calculations of the case i — 2 we obtain that (P?) has optimal solutions if and only if fi > 1. The solution is unique and is the function xM determined above. Therefore v(Pjl) = v(P£) for /x > 1. For /1 = 1 we have already seen that v(Pi) > ^(P^). Considering for every n 6 N the function Il
[0,1HR'
^ W  l ^(ii)
if
te]ii,i],
we observe that x" € Xi and limnxx, / i ( x " ) = f\(x\) (one can use Lebesgue's theorem). Therefore u ( P / ) = ^(Pf). Let i = 3. Similarly to the case i = 2, j / M is an optimal solution for (P^) if and only if V/M(j/M) = 0. For /j. = 0 this is not possible, and so (P°) has no optimal solutions. Let fj, > 0 and i/M an optimal solution of (P3"). The condition V/M(j/M) = 0 is equivalent to
333
Exercises  Solutions
Recall the following result: if x e C[a,b] (a,b eR, a < b) and fa x(t)u(t)dt = 0 for every u 6 C[a, b] with J u(t)dt = 0, then £ is a constant function. Indeed, let c := (6 — a )  1 / u(t) dt and u := x — c; by hypothesis we have that Ja (u(t) — c)2dt = f (u(t) — c)u(t) dt = 0 and so x(t) = c for every t e [a, b]. Therefore there exists a e R such that V*G [0,1] : n
,
V
^
=+t = a;
it follows that V i e [0,1] : y „ ( t ) =
^  l L = = . ^H2  (a  t)2
Since j / M e C[0,1], we necessarily have that /u > a — t\ for every t e [0,1], i.e. 1 — fi < a < ft. It follows that p > \. In this case, yM e X3 if and only if a =  . So, for (j, < i the problem (P^) has no optimal solutions, while for fi >  the problem (P3") has a unique optimal solution, namely
yM : [0,1] + R,
2
y„(t) =
In a similar manner, but using this time the result: if x e L°°(o,6) (0,6 e R, o < 6) and f x(t)u(t)dt = 0 for every function u e Ll(a,b) with the property / u(t) dt = 0, then x is constant a.e. (the proof being similar to that presented above), we obtain that (P£) has optimal solutions if and only if \i >  ; in this case the unique solution is yM denned above. Let now fj, <  ; for a e]0,  [ , P € P consider the function
r ya,0 : [0,1] J R,
y t t ,^(t):=<
0
if
te[o,a],
0 { p
if if
t€]a,la[, t e [1  a, 1].
ya„3 e X4 and /(y t t l  8 ) = aP
faV^+P2
 1 + a ) + /x(l  2a).
Since / M (y n i,„2) > —00, we have that v(P£) =  c o for n <  . Taking into account that for every (a,P) € ]0,  [ x ]0, oo[ the function yatp can be approximated pointwisely by a sequence {ya,0,k)keN C X3 such that Ij/a.^.jfcl < \ya,p\ and using eventually Lebesgue's theorem, we obtain that v(Pg) = —00 for fi <  . Approximating similarly yi/2, we obtain that v(P3 ) = v(P4' ). Elementary calculations
Exercises  Solutions
334 show that
/* e [0,1[, //G [l,co[, and f —oo
if
M G [0,  [ , ^ G [i,oo[.
The solution is complete. E x e r c i s e 2.51 Recall that x G AC[0,1] if and only if x is derivable at almost every t G [0,1] and x' G L^O, 1). Note that for x G X, J* x(t) dt =  J* tx'(t) dt. Taking into account this fact, to problem (P) we associate the problems (Pi)
min Jo tx(t)dt,
xGXi,
J* y/l + (x(i)) 2 dt < L,
i = 3,4, the spaces X3 and X\ are those introduced in Exercise 2.50. Note that v(P) = —v(P3) if X = C^O, 1], and x is a solution of (P) if and only if x' is a solution of (P 3 ); if X = AC[0,1] then v(P) = v(P4) and x is a solution of (P) if and only if x1 is a solution of [PA). Considering for i G {3,4} the functions f,g:Xi^R,
f(x):=titu(t)dt,
g(x) := J* y/l + (u(t))* dt,
the problem (Pi) becomes (Pi)
mmf(x),
xeXi,
g(x)
The function / is linear and g is convex and Gateaux differentiable; moreover the differential of g is given by Vg(x)(u) = J0 xu/^/1 + x2 dt for i , t i £ l j (see Exercises 2.9 and 2.10). Therefore the problems (Pi), i G {3,4}, are convex problems. If L < 1, then the problem (Pi) has no admissible solutions for every i G {3,4}, while if L = 1, (Pi) has only an admissible solution, x = 0, which is the optimal solution, too. Let L > 1 and i = 4. Since g(0) = 1 < L, Slater's condition is verified for the convex problem (P4). Therefore an admissible solution 1 6 X 4 (g(x) < L) is optimal for (P4) if and only if there exists A > 0 such that \(g(x)  L) = 0,
V / ( x ) + \Vg(x)
= 0.
Let x G X4 be an optimal solution of (P4) and A > 0 satisfying the above relations.
Exercises  Solutions
335
Since V / ( s ) ^ 0 we obtain that A > 0, and so g(x) = L. Therefore
Vnei 4 : f1 ( . Xx^=
+ t]u(t)dt = o.
As seen in the solution of Exercise 2.50, the above relation is possible only when A > 1 and the sole function which satisfies it is i/A : [ 0 , 1 ]  • R,
yx(t)
•t
y/*(ht)*
Since g{yx) = L we have that 2Aarcsin ^ = L. But the function ]0,1] 3 t >> j arcsin t € R is increasing with range ]1, TT/2]. Since in convex programming the necessary condition is also sufficient, we obtain that for L S]l,7r/2] the problem (Pi) has the unique solution y\, where 2Aarcsin ^y = L. The (unique) solution of problem (P) is, in this case, the function *:[0,1]^R,
, ( t ) : = ^  ( f  i p  ^   ,
(•)
with A defined above. For L > 7r/2 the problem (P4) (and (P), too) has no optimal solutions. To compute the value of problem (P4), we associate its dual problem. By Theorem 2.9.3 we have that «(Pi) = v(D4) (and (D4) has optimal solutions), where (£>4)
max i n f x e x 4 ( / ( s ) + A ( p ( x )  L ) ) ,
A > 0.
Taking into account the result obtained in solving Exercise 2.50, we have that
v(Di) = max { A2 arcsin i + \ J\2
 \  \ L \
A> \ \ .
If i = 3, the same argument as for the case i — 4 proves that (P3) has a unique solution for L 6]l,7r/2[, having the same expression, while for L > TT/2 the problem has no optimal solutions. Moreover v(Ps) = v(P4) for every L > 1. In conclusion, problem (P) has a unique solution for L g]l,7r/2[ and has no optimal solutions for L > n/2 when X = C 1 [0,1], and has a unique solution for L 6]l,7r/2] and has no optimal solutions for L > 7r/2 when X = AC[0,1]; in both cases the solution is given by formula (*). Moreover V(P)=S
\SJ^L\\^L
I =^
if if
L€]l,7r/2], ^6W2,oo[,
where \L is the unique positive solution of the equation 2A arcsin ^
Le]l,n/2].
= L when
336
Exercises  Solutions
E x e r c i s e 3.1 First recall that i1 := {(xk)k>i C K  ^2k>1 \xk\ < 00} is a Banach space with respect to the norm \\x\\l := ^2k>1 \xk\ Its dual is £°° := {(xk)k>i C R I (xk) is bounded}, the dual norm being CiCfc)00 := s u p ^ \xk\Let x = (xk)k>i € i1. Since (xk) > 0 we have that \\xk\\ < 1 for k > ko, and so a?fcj1+1'* < \xk\ for k > ko. It follows that the series defining f(x) is convergent, and so / ( x ) € R. Moreover, because \xk\ < 1 (k > 1) for x < 1, the above inequality shows that f(x) < 1 for x £ Uti; hence / is continuous if we succeed to show that / is (strictly) convex. Let us show that / is strictly convex. So, let x,y 6 I1 with x ^ y and A e]0,1[. Of course, there exists ko > 1 such that Xk0 / yk0 Since the mapping ipa : K > R, <^a(<) :=  i  a , is strictly convex for every a > 1 (see the examples after Theorem 2.1.10), we obtain that \\xk + (1  A)j/jfc1+1/* < Ax* 1 + 1 / f c + (1  A) \yk\1+1/k, with strict inequality for k = fco. Summing up these inequalities side by side we get /(Ax + (1  X)y) < A/(x) + (1  X)f(y). Let x* = (x'k)k>i C (^)* = ^°° be in df(x) with x = (xk)k>i Let us fix ko 6 N. Taking J/J, := Xk for k ^ ko and y*;0 = xfco + M in the definition of df(x) we get \xko + «  1 + 1 / * °   X A , 0  1 + 1 A O > ux'kQ for every u € R, i.e. x'^0 6 9yi + i/j : o (xj : o ). Using Theorem 3.7.2(iii) we obtain that x'k0 — (1 + •£) Ix^ol1^*0 sgn(xjt 0 ). Therefore 9 / ( x ) (which is nonempty because / is continuous on i1) has a unique element x* whose components are x'k := (1+ \) \xk\X sgn(xfc). Taking into account Corollary 2.4.10 we obtain that / is Gateaux differentiable. Consider now x = (xk) 6 Z1 and show that / is not Frechet differentiable at x. We distinguish two cases: 1) l i m s u p j . ^ ^ Ix^l 1 '* > 0 and 2) l i m s u p j . ^ ^ Ixfcl1'* = 0. 1) In this case there exist r\ > 0 and an infinite set P C N such that \xk 1/,fc > 77 for every k € P. For n 6 N consider u" 6 i1 defined by u^ := 0 if k < n or k € N \ P and u^ •— —2xk otherwise. It is obvious that (KM"!!!) —• 0 for n —>• 00. But V/(x + u n )  V / ( x )   = 2
sup k£P,k>n
((l + k1)\xk\1/k)>2r)
V
Vn € N. '
Taking into account Corollary 3.3.6 we obtain that / is not Frechet differentiable at x. 2) In this case (\xk\1/k) >• 0. For n € N consider un S ^ defined by uk := fc_2sgn(xj,) this case
for fc > n and u]J := 0 otherwise; it is obvious that (Uu™!!!) + 0. In
  V / ( x + un)  V / ( x )   = sup ( ( 1 + AT1) • \(\xk\ + k~2)1/k k>n
^
I
> Um ((1 + AT1) • \(\xk\ + k~2)1,k

\xk\1/k\) 1'
 x* 1 / f c ) = 1
for every n € N. As above we obtain again that / is not Frechet differentiable at
337
Exercises  Solutions
Exercise 3.2 Because \xn\ < \\x\\ for every n, we have that 0 < f(x) < \\x\\ for every x 6 £°°. Because l°° 3 x >> \x„\ £ R is convex, by Theorem 2.1.3(iii) / is convex (even sublinear). It follows that / is continuous, being bounded above on Ue°°. Let x e Z°° be fixed. If f(x) = 0 consider u := ( 1 , . . . , 1,...) G (°°. Then for any t > 0 we have that fix + tu) =t — f(x — tu), and so limtj.o t~l (/(5" + tu) + f(x  tu)  2/(x)) = 2 > 0. Assume that fix) > 0 and consider an infinite subset P of N such that lim„gpa;n = f(x); without loss of generality we assume that xn >  / ( x ) for n £ P (replacing eventually x by — x). Let P i , P2 be infinite disjoint subsets of P such that P = Pi U Pi Let u € l°° be such that un :— 1 for n € Pi and u„ := 0 for n e N \ P i . Let 0 < t < \f{x). Then \xn + tu\ = xn + t for n € Pi and \x„ + tu\ = \x„\ for n € N \ P i . It follows that f(x+tu) = l i m s u p n £ N \x„ +tu\ > lim n 6 p 1 (x„+<) = f(x)+t. Since \x„— tu\ = \xn\ for n 6 N \ P i D P2, we have that limsup n e N V v P l x„—1« = f(x). Because \xn— tu\ = xn—t for n 6 P i , it follows that limsup n g N \xn — tu\ = }{x). Therefore limtiot1(f(x + tu) + f(xtu)2f(x)) > l i m u o t~^ (f(x) + t + f(x)  2f{x)) = 1 > 0. Hence, in both cases there exists u € £°° such that l i m ^ o < _ 1 ( / ( 5 + tu) + fix — tu) — 2fix)) > 0. By Theorem 3.3.2 / is not Gateaux differentiate at x. Exercise 3.3 Because pf is nondecreasing on [0, oo[, it is obvious that the three situations in (ii) are equivalent. Also for the equivalence of the three situations in (iii) it is sufficient to show that p : [0, oo[—> [0,oo] defined by pis) := i n f { i / ( x ) + \fiy)
 fi\x
+ \y) \ x,y e d o m / , \\x  y\\ = s},
is nondecreasing (of course, inf 0 = +00). Indeed, taking 0 < s' < s and x, y € d o m / with x — y\\ = s, then applying Theorem 2.1.5(vii) for A — 1/2, ipit) := fix + tiy — x)) and 0 < j < 1 we obtain that i / ( i ) + \fix
+ °iiy  x))  fix + £iy  x)) <  / ( x ) + \f{y)
 f(\x
+ \y),
which shows that p(s') < p(s) because ( ^(3/ — aOII = s'. Let now x, y e d o m / with \\x — y\\ = t > 0 and A 6]0,1[; we may assume that A e]0,1/2]. Set z := (1  \)x + Xy; then f(z) = / ((1  2X)x + 2A((x + y))) < (1  2A)/(x) + 2 A / ( i ( x + y)) < (1  2\)f(x)
+ 2A ( i / ( x ) + \fiy)
 p(x 
= ( l  A ) / ( x ) + A/(2/)2Ap(t) <(lA)/(x) + A/(y)2A(lA)^).
W)
338
Exercises  Solutions
Therefore 2p(t) < Pf(t) < 4p(t) for t > 0. These inequalities show that (iii) <=>• (i). Since the implications (i) =S (ii) => (iii) are obvious, the equivalence of these three conditions is proved. (iv) Fixing x 6 dom / in the preceding proof we get the conclusion. E x e r c i s e 3.4 Let x G I1 and t > 0. Consider e n e E1 with e£ := 1 if k = n and e£ := 0 otherwise. Then pftX{t) < 4 {\f{x + ten) + \f(x)  }{x + \ten)) for every n g N , and so 0 < p/,x(t) < 41im„_oo (  x» + t\1+1/n +   * n  1 + 1 / "  \xn +  t  1 + 1 / n ) = 0. Because / is Gateaux differentiable, we have (see page 201) that */,.(*) = inf{r>/(»,i)    »  x   = t }
VxG£\
t>0,
where Df{y, x) := f(y)  f(x)  (y  x, Vf{x)). Note first that \\y — x\\ — y — x and D/(y,x) > Df(y,x) for any y € i1, where y € I1 is denned by yk := yk if sgn(j/*  x*,) = sgn(xfc) and yk := 2xfc  yk otherwise. It is obvious that \yk — xk\ = j/fc — a;fc for every k > 1, and so   v  x   =   »  x   . Then
£>/(Vlx)  Df(y,x) = /(»)  /(y) (yy,
V/(*)> = J ^ fl*.
where 6k := \yk\1+1/k\yk\1+1/k(l + i)sgn(xk)\xk\1/k (ykyk)It is sufficient to show that 6k > 0 for every k > 1. Of course, #& = 0 if sgn(yfc — xk) — sgn(x/t) or xk = 0. So, assume that this is not the case. Then uk := 1 — yk/xk > 0, p* := 1 + £ £ ]1, 2], yk = 2xk  yk and 6k = \xk\Pk (1  uk\Pk  1 +
P( W& =
+ 2p*Ufc) = x, Pfc ^ ( u * ) ,
because yk  yk = 2(yk  xk) and sgn(yfc  xk) = sgn(xfc), where <^p(£) := 1 — t\p — 1 + t\p + 2pt for t > 0. Using Exercise 2.7 we have that tpp is nondecreasing for p S]l,2], and so ipPk(uk) > 0. Hence 6k > 0 for every A; > 1, and so Df(y,x) > Df(y,x). Assume now that x € I1 \ A and t > 0. Of course, l i m s u p j , ^ ^ £jfc = 1. Consider un := sgn(x„) • en. Then #/,*(*) < / ( x + tun)  f(x)  t {un, V / ( x ) ) = (x„ + i ) 1 + 1 / "   x n  1 + 1 / r l  t ( l + i )  x n  1 / n . It follows that 0 < tf/,*(t) < liminfn^oo ( (  x n  + t)l+1/n
 \xn\1+1/n
Fix x € A and take 7 := 1 — l i m s u p j . ^ ^ xji
*:[0,7[R, 0(s):=Zk>_Xvh
 t ( l + ±) M 1 / n ) = 0. > 0. Consider the function
+
lXkll/k k
)
339
Exercises  Solutions
The function 0 is welldefined, continuous, increasing and (f>(0) = \\x\\. Indeed, as l i m s u p (  ^  + M 1 A > ) = s + limsupx )i;  1/ * : < 1
Vs6[0,7[,
by the CauchyHadamard test of convergence we have that the series defining (f>(s) is convergent. Moreover, for any k > 1 the function 4>k defined by <j>k(s) :=
( •j^pi + \xk\
\ k
J is continuous and increasing on K+, and so
[0,7[. Also, for so €]0,7[, we have that 0 < 4>k{s) < (j>k(so) for all s e [0,so] and k > 1, and so the series defining cj> is uniformly convergent on [0, so]. It follows that 0 is continuous on [0,7[. It is obvious that (0) — \\x\\. Let to := 4>(~y/2) — \\x\\ > 0. Take now t € ]0, to]; we have that (f>(0) < t + \\x\\ < 4>{f/2), and so there exists (a unique) s := St G]0,7/2] such that <^>(s) = t + a;. We intend to show that dj,x{t) = Df{w,x), where w := wt 6 C1 is defined by + \xk\1/k]
wk:=sgn(xk)(~^j
V* > 1. /
\ k
First note that sgn(wfc Xk) = sgn(xfc) and \wk  Xk\ = f ^ + x fc  1/A: J  \xk\, whence \\w — x\\ = t. This also implies that w = (w — x) + x € E1. Take y € i1 with \\y — x\\ = t. Replacing, if necessary, y by y as above, we may assume that sgn(yk — Xk) — sgn(xk) for k > 1. Then, by the convexity of / , Df (y, x)  Df (w, x) = f(y)  f{w) (yw,
V / ( x ) ) >{yw,
Vf(w)

Vf{x)).
But ( V / H  Vf(x))k
= (l + l)
^(xk)
( J ^
+ \xk\1/k
 M
1 A
) = ssgn(xfc).
Hence Df{y,
x)  Df(w, x)>s = s
J2k>1
s
&n(xk) {Vk ~ wk)
( E f c > 1 sgn(Xk^ (yk ~ Xk"> ~ E f c > i s S n ( x *) (™«= =s ( E ^ iw  Xfci  E t > x i""=  *fc0 = °
_ x
*))
It follows that i?/,i(t) = Df(wt,x) > 0, the last inequality because / is strictly convex and wt =£ x. As i?flX is nondecreasing we obtain that i?/, x (t) > 0 for every t >0. Exercise 3.5 (i) Taking / „ : X >• R, / n ( x ) := X)fc=i 2 _ f c (*> xife)2> w e h a v e t h a t fn is convex, 0 < fn{x) < \\x\\2 and {fn(x)) > / ( x ) for every x G X. Therefore /
340
Exercises  Solutions
is convex, nonnegative and continuous (being bounded above on Ux) Let x € X, M 6 S x a n d t £ R \ {0}. Then
f{x + tu)  f(x)  V
,  n + l (x,xn)
• {u,xn)
= l*l£ > 2"(«,^>2<*. *•—'rc>l
Hence / is uniformly Frechet difFerentiable and V / ( x ) = S n > i 2~™+1 (x, xn) • x"n for every x € X. It is obvious that g := y/J is quasiconvex and g(Xx) = A g(x) for all x 6 X and A £ R. Using Corollary 2.2.3 we obtain that g is sublineax, and so g is a seminorm. For x 6 X we have that f(x) = 0 4=S> s(x) = 0<S>[Vn£N : (x, x*) = 0] ^ x € A X . Therefore
Because  < Mj^ < \/2, H^ is
equivalent to . Since j is strictly convex, from Theorem 3.7.2(v) we have that (X, H'llj) is strictly convex. Recall that X is separable when X* is separable. Indeed, if X* is separable there exists A := {x*n \ n S N} C Sx* dense in Sx* • For every n S N consider x„ € Sx such that {xn,x*n) > 1/2 and take B := {xn \ n 6 P^}. Let x* 6 Sx*; there exists n € N such that x* — x* < 1/2. Then (x n ,x*) — ( x n , x ^ ) + (xn,x* — x^) > 1/2 — x* — x^ > 0, which shows that x* £ Bx. Therefore B1' = {0}, and so X is separable. Assume now that X is reflexive and separable. Because X* is separable, there exists an equivalent norm 1 x on X such that {X, HHj) is strictly convex. Using Theorems 3.7.3 and 3.7.2(iv) we obtain that 5GHI1)2 is Gateaux differentiable. Taking h{x*) := Y,n>i 2~n (xn,x*)2 , where {xn \ n £ N} C Sx is dense in Sx, by (i) we have that k := h + jflMIJ) 2 is strictly convex and Gateaux differentiable. Using again Theorem 3.7.2, it follows that Vk is a norm on X* equivalent to J such that (X* ,Vk) is smooth and strictly convex. Then the
Exercises 
Solutions
341
dual norm of Vk on X** = X, denoted by  2 , is equivalent to the initial norm and (X, •  2 ) is smooth and strictly convex. Exercise 3.6 The fact that P 9 t H* t~pip(t) is nondecreasing means that ip(ct) < cp(p(t) for all c e]0,1[ and t > 0. Let 0 < si < s 2 and v? e (si) < ^J. Then there exists t < ft such that ip{t) > si. Let c := (si/s2) 1 / ' p €]0,1[. Then si < y(*) = V (c (c  1 *)) < c*V (c  1 *) = —V (c  1 *) , $2
and so S2 <
= sup((x,x*) ipo(dc(x)))
= min(^o(A) +
{\dc)*(x*))
# = min ( (A) + A c ^ ( A  V ) )
= min (V*(A) + A ( ^ ( A " V ) + iUx. (A~ V ) ) ) = min fV"*(A) + L*C{X*) + i\ux, =
(z*)) = <c(x*) +
min
, V"*(A)
ih(x*)+ip#(\\x*\\),
because V* is nondecreasing; for x* = 0 the equality is obvious. Since ip o dc is continuous and convex, there other formula follows from the biconjugate theorem. Exercise 3.8 By Theorem 3.10.1, there exists c > 0 such that f*{x*) = /*(0) + i*s{x*) for all x* € cUx> Because 5 is bounded, t j is continuous, and so subdifferentiable at any x* S X*. From the preceding relation we obtain that df{x*) = diS{x*) C S for every x* e X* with a;* < c. Hence, for x* € cBx* there exists x G S such that x € df(x*), or, equivalently, x* 6 df(x). Therefore cBx C 0 / ( 5 ) .
342
Exercises  Solutions
E x e r c i s e 3.9 The implications (i) =$• (ii) •» (iii) => (iv) are obvious, while the implication (iii) => (i) follows from Corollary 3.4.2 applied to h and xj)(t) := a~1t for t 6 [0,(5]. Assume that S is compact and (iv) holds. Then for every z S S there exist 8z,az > 0 such that ds{x) < azh(x) for every x € D(z,6z). Since S is compact and S C \JZ£sB(z,5z/2), there exists a finite set So C S such that S C Uzes0B(z,6z/2). Let a := max{a z  z € So} > 0 and 5 := min{5 z /2  z £ So} > 0. Consider i 6 l with ds(x) < 5. Since 5 is compact, there exists z e S such that x — z\\ < 5. For this z, there exists zo 6 5o such that z 6 £(20, SZQ/2). Hence \\x — zo\\ < \\x — z\\ + \\z — zo\\ < 5 + SZQ/2 < 5Z0. By the choice of 5Z0 and azo we have that ds(x) < azoh(x) < ah(x). The proof is complete.
E x e r c i s e 3.10 1) Note that / B (  , M ) — /El/i • . Therefore JB{,P) is a convex function and dom/s(,/x) = X. So, if / s (  , / i ) takes the value —oo then it is identical —oo. Suppose that JB{,P) does not take the value —oo; hence the function is finite. Let us prove that }B{,P) is Lipschitz with Lipschitz constant p,. Let x i , X2 £ X. For e > 0 there exists yi € X such that fB{xi,p)
> f{y\) + [J.\\xi — a/i11
e,
and so fB(x2,p)fB(xi,p)
< f(yi)+p\\x2yi\\f{yi)p\\x1y1\\+e
<
/J,\\x2xi\\+e.
Since e > 0 is arbitrary, we obtain that IB(X2,H)
 fB(xi,fi)
< p\\x2 
xi\\.
Interchanging x\ and x2 we obtain that /s(,/*) is Lipschitz, with Lipschitz constant p. It is obvious that for 0 < pi < p2 one has /B(,A*I) < /s(,P2) < /• Let us prove now the last statement of 1). Note that, since / € T(X), there exists x* € X* and 7 6 R such that VxG X : f(x) >
(x,x*)+y.
Let x € X be fixed, and R 3 A < f(x). Since / is lsc at x, there exists p > 0 such that f(x) > A for x € D(x,p). Let ^ := max{x*, (A + x* — 7 — (x,x*))/p}.
Exercises  Solutions
343
For p > p we have fB(x,p)
= mm\
inf
(f{y) + p\\xy\\),
> mkJ
inf
(A + p\\x  y\\),
\_yeD(x,p)
inf
(/(y) + p\\x  y\\) Uy,x*) + 7 + p\\x  y\\) \
y$D(x,p)
> min{A,inf{(x,x*) +t{u,x*) > min {A, inf {(x, x*)+j
inf
)
+ 7 + pt  u G 5, t > p}}
+ t(p
\\x* )  t > p}}
> A. Therefore lim^,^ / B ( X , p ) = f{%) Since x is arbitrary, we have the desired conclusion. 2) Let XQ G X be fixed and g : X ¥ R, ff(x) := / ( x ) + /JX — a?o It is clear that dg{x0) = df{x0) + pUx* when XQ G d o m / . (a) =>• (b) From the relation df(xo)C\pUx* 0 G dg(xo). Therefore
Vx G X •. f(x0) =
# 0 we have that xo G d o m / and
g(xo)
and so / ( x 0 ) = / s ( x 0 , / f ) . (b) =>• (c) By what was proved above and using the hypothesis we have that —00 < /(xo) = /s(xo,/i) < 00. Therefore fB{xo,p) G R. From this, taking into account 1), it follows that /B(,A*) is convex and continuous. Therefore df(xo,p) ^ 0 Since JB{,P) is an exact convolution at xo = xo + 0, using Corollary 2.4.7, we have that dJB{xo,p) = df{xo) f\pUx(c) => (a) Since JB{XO,P) € R, by 1) we have that /s(,/i) is convex and continuous. Therefore df(xo,p) ^ 0, and so (a) is true. E x e r c i s e 3.11 1) Since / € T(X), there exist x* G X* and 7 G R such that V x G X : /(x) > ( x , x * ) + 7 .
(°)
Therefore for every x € X, V2/GX : / ^ ( 2 / ) : = / ( y ) + f   x  2 /   2 > f   x  2 /   2    x *      x  y   + <x,x*>+7; (*) this implies that /M(X,/U) > —00 for every x € X. Since fsi{,p) = / d f II ' II 1 we have that dom/M(,A*) = d o m / + X = X; hence / M (  , M ) is finite. Since / is convex and ^ • 2 is convex and continuous, it follows that fivi{,p) is convex and continuous. The inequalities /M(,A*I) < fiv[{,p2) < / are obvious for 0 < pi < p2 The proof of lim^nx, JM{X,P) = f(x) for every x G X is completely analogue to that of Exercise 3.10.
Exercises  Solutions
344
Let x* e df(x). VySX
Then : (yXjX*)
+ z\\yx\\a<m
+
$\\yx\\2f(x).
Taking the infimum with respect to y in both members of the inequality we obtain — jHx* 2 < / M ( X , P ) — fix), i.e. our conclusion. (In this case relation lim^Kx) / M ( X , / Z ) = fix) is immediate.) 2) Prom (*), the function f^^ is coercive; being also lsc, since X is a reflexive Banach space, there exists an element xM 6 X such that /M(x,/i)=/(^)+fx/ix2.
(")
(If X is strictly convex, using Theorem 3.7.2(v), the function f  •  is strictly convex, whence f^.lX is strictly convex, too; in this case xM is unique.) Furthermore, using Corollary 2.4.7, we have dfM(x,[i)
= df{x^)r\fj.^x{x
x,j.).
(***)
Let us consider the operator J^ : X —> X, J M (x) := xM. Relation (**) proves the formula of the statement. Let x 6 d o m / be fixed, and (i > 0. By (°) and (**) we have that <JM(x),x*> + 7 + IWMx)  x\\2 < fM(x,fi) < f{x), and so, for r\ := fix) — 7 — (x, x*), we have that V^>0 :
fJ^)zll2s*llllWzll^O.
Thus we have
V M > 0 : J M (x) a ;< M  1 (x* + Vlk1 2 + 2 W ) . Taking the limit for n —»• 00 we obtain that limM>co J^ix) = x. By (**) we have that / ( J M ( x ) ) < / M ( X , / / ) < f(x) for every /J, > 0. Since / is lsc and limM_,oo Jji(x) = x, we obtain that fix) < liminf / (J M (x)) < limsup / (J M (x)) < and so l i m ^ o o / (J M (x)) = f(x). 3) Let x* 6 cJ/(x) and x^ e dfu{x,n) is continuous). Using (***) we have that
fix),
for /i > 0 (x^ exists because / M ( , / ^ )
fi{x  J M (x),x*) = [i2\\x  J^(x) 2 = x* 2 and, taking into account 1) and (**), (x  J M (x),x;) < fix)  fiMx))
< ^   x *   2  1 \\x 
J,ix)\\2.
345
Exercises  Solutions
Using the above relations we obtain that a;* < a;*. Since x* € df(x) is arbitrary, we have that x^j < dgf(x)(0). Let now (fj.i)iei —*• oo and x* = w* — limx*^ (such a convergent net exists because Ux is iu*compact). Because x*p € 9 / ( J M ( x ) ) and (JM(a:)) —> x, by Theorem 2.4.2(ix) we have that x* € df(x). Moreover, because the norm of X" is w*\sc, we have that dgf^(0) < \\x*\\ < liminfjg/ xjlj. Prom the first part we get limx* ;  = da/(z)(0) = x*, and so x* € Pgf(x)(0). Therefore lim^^oo x*  = d o / ( l ) ( 0 ) . 4) Assume that X is smooth. By Theorem 3.7.2(iv), the duality mapping $ x is singlevalued. From (***) we have that 5 / M ( X , / J ) = p.$x{x — J^(x)). Using Corollary 2.4.10 we obtain that fu(,p) is Gateaux differentiable and V / M ( X , / I ) = p.$x{x  J^(x)) for every x e X. Assume now that df(x) ^ 0. The space X being reflexive, by Theorem 3.7.3, X* is strictly convex. It follows that Pe/(x)(0) is a singleton {XQ}. From 3) we obtain that XQ is the sole limit point of the net ( V / M ( : K , M ) ) f ° r A* *~• °°> a n d so XQ = w* — limM_>oo V / M ( X , M )  Because (   V / M ( X , M ) I I ) —> xo, we obtain that ( V / M ( X , / J ) ) —> XQ when X* has the KadecKlee property. 5) Let now X be locally uniformly smooth and consider x G X; from 4) we have that / M (  , M ) is Gateaux differentiable and V/Af(x,/i) = pNfi(x — J^(x), where / 2 :=    •  2 . Then for y e X we have 0 < / M ( 2 / , M )  / M ( X , P )  {yx, 2
VfM(x,fj,))
< HMx)) + fHy  J„(z)  f(J,(x))  $*  JM(*)2 (y x,fj.vf2{x  JM(x))) x J x J X = Kf*(y  M( ))  M  »( )) (y«, v/ 2 (x  J M (X)))).
From Theorem 3.7.4 we obtain that /M(,AO is Frechet differentiable at x. Using Theorem 3.3.2 we obtain that V / M (  , / I ) is continuous. Exercise 3.12 Let ip 6 Ao 0 No and if) = coip. It is obvious that 0 < tp <
(**„)•*',
(Ai)»V,
(AUn)^!/',
(AX)>r;,
w i t h x ' . f , ^ , ^ e l + and A* € [0,1] for i € {1,2,3}. Of course, A1 + A2 + A3 = 1. y1 +y2 +y3  x and T1 + T2 + T3 = 0. Therefore T1 = T2  T3 = 0. If A' > 0, since ( A ^ ) ¥ 0, we have that tl — 0, and so (p(xln)) —>• 0. Since ip
Exercises  Solutions
346
is nondecreasing, from our hypothesis we obtain that (xzn) —> x% = 0, which, at its turn, implies that yl — 0. If A' = 0 and y% > 0 then (x'n) ¥ oo. Therefore there exists rij 6 N such that xn > p for every n > n,. Thus, for n > n; we have that £J, >
= (x,anx)
E co(epiy?) C e p i ^ .
Hence (x,0) € epii/'. Therefore ^(s;) = 0 for every x > 0. The solution is complete. Exercise 3.13 Let 1 6 X and x\ e A". By Theorem 3.8.4(vi) we have that xi=
PK(x)&\/y
eK
: (x  Xi\xi
 y) >0
•«Vt/€ftT : (x — xi  Xi) > (x — Xi \y) <&VyeK
:
(xxi\xi)>0>(xxi\y)
<$ x — x\ G K~, (x — xi  xi) = 0. Let xi = P K ( S ) . By the above characterization we have that X2 := x — xi S fiT, x — X2 = x\ 6 (K~)~ = K, (x — xi \ X2) = (x\\x
— X2) = 0.
By the same characterization we have that X2 = PK (X) Therefore the conclusion is true. Suppose now that x = xi +X2 with xi S K, x% € K~ and (xi  X2) = 0. Then xi E K, x — xi € K~ and (x — xi  x i ) = 0. Therefore, using again the above characterization, we have that xi = PK(X), and so X2 = PK (X). Exercise 3.14 (a) Using the equivalence (i) &• (ii) of Theorem 1.3.16 with T = Idy and C = A, there exists m > 0 such that \\A*y\\ > m\\y\\ for every y e Y* = Y. It follows that \\Ty\\ • Hvll > (Ty,y) = (A'y,Amy)
= \\A'y\\2 > m2 \\y\\2.
So, \\Ty\\ > m2 \\y\\ for every y G Y. Of course, this relation implies that T is injective. Since T* = T, the preceding inequality can be written as T*j/ > m2 \\y\\ for every y € Y. Using again the equivalence (i) •«• (ii) of Theorem 1.3.16 with T replaced by Idy and C replaced by our T, we obtain that T is surjective, and so T is bijective.
Exercises  Solutions
347
(b) Consider the function / : X >• R, / ( s ) = \ \\x\\2 + tc{Ax + y0). It is obvious that / is coercive and strictly convex. Using Theorem 2.5.1 and Proposition 2.5.6 we obtain that / has a unique minimum point x G X. By Corollary 2.8.5 there exists v 6 N{C\Ax + yo) such that x = — A*v. Of course, y := Ax + j/o G C, and so Ax = —{A o A*)v = — Tv, whence y — yo = —Tv. It follows that T _ 1 (j7  y0) — v, and so {A* o T)(y  y0) = —A*v = x. Since (y — y,v) < 0 for every y € C, we have that
Vj/GC : (yy^iypT'y
+
pT'yo^^O,
where p > 0. By Theorem 3.8.4(vi) we have that y = Pc{y  pT~ly + pT~1y0). Conversely, if y = Pc(y  p T " l y + pT _ 1 yo) for p > 0 and x = (A* o T _ 1 ) ( j / yo), taking v := —T_1(j7 — yo) we have that x = — A*v, Ax + j/o = y € C and w £ iV(C; Ax + j/o) Using again Corollary 2.8.5 we obtain that x is the solution of (P). E x e r c i s e 3.15
Applying Exercise 2.41 we find a, /3 G M, a > 0, such that
g(x) > a\\x\\ + 0 for every x G X. Consider ( ( a ^ ^ n ) ) C gcdg with (x*) 4 0. Let s G dom g be fixed. Then " I W I + 0  \\xn x\\
•  K  
There exists no G N such that x* < a / 2 for n > no, and so \\xn — x\\ < 2a~1(g(x) + a \\x\\ — fi) for n > no Hence the sequence (xn) is bounded. Assume that the conclusion is false. Then there exist an increasing sequence (rik) C N and t > infg such that g{xnk) > t for every k € N. Because X is reflexive and (xn) is bounded, we may assume that ( x „ J 4 i 6 l As (xj;) M! 0, by Theorem 2.4.2(ix), we have that 0 € dg(x), and so p(x) = inf g. The inequality £ + (x — x„k,x*nk) < g(x„k) + (x — Xnk,x*nk) < g{x) yields the contradiction t < inf p. E x e r c i s e 3.16 (a) Because g* is iu*lsc, g* is wlsc, too. Since X* is a Banach space it follows that g* is continuous at 0, and so 0 G int(dom 0 satisfying the conclusion (maybe not our a ) . In fact, for x G d o m / \ C = ( A n doing) \ C and
348
Exercises  Solutions
t := a fix) we have that / (x + tu) < fix) + t/ 0 0 (u) = 0. So x + tuE C, whence dcix) < \\x + tu — x\\ = t = a • fix). E x e r c i s e 3.18 It is obvious that inf
= (  ^ i _  1 ,  ^ J L _ ) for (x,y)
/
(0,0) and Vg2ix,y)
=
2
(2,2y) for (x,y) £ R . Using Corollary 3.10.9 we obtain that »•«(*) = kgi(t) •
f
= inf {\\Vgxix,y)\\
1 , 2t + 2 x + l+t
•• i n f •
\ y/x2 + y2  x  1 = t)
x> t±l }=Q
W >l,
and *•«(«) = *«(*) = inf {V 5 2(x, j/) I y 2  2x  1 = t} = {2y/l
+ y»  y € R } = 2
for any t € R. Note that giix,y) = giix,y) • c(x,y), where e(x, «/) := ^ a ; 2 + y 2 + x + 1 > 1. It follows that 93(3;, 1/) = 32(2;, y) > 0 if y 2 — 2a; > 1 and 33(2;, y) = giix,y) < 0 if 2/2  2x < 1; moreover, [31 = 0] = [32 = 0] = [33 = 0] = {(x,y) € R 2 I y 2 = 2x + 1}. Prom the very definition of r / = rj and A;/ we obtain that »M(0) = r 9 1 (0) and fcM(0) = fcOT(0).
Bibliography
Adly, S., Ernst, E. and Thera, M. (2001a) "A characterization of convex and semicoercive functionals", J. Convex Anal. 8(1), 127148. Adly, S., Ernst, E. and Thera, M. (2001b) "Stabilite de l'ensemble des solutions d'une inequation variationnelle non coercive", C. R. Acad. Sci. Paris Ser. I Math. 333, 409414. Amara, C. (1998) Direction de majoration, ensembles sousseriellement convexes. Applications a l'optimisation, PhD thesis, University of Avignon, Avignon, France. Amara, C. and CiligotTravain, M. (1999) "Lower CSclosed sets and functions", J. Math. Anal. Appl. 239(2), 371389. Anger, B. and Lembcke, J. (1974) "HahnBanach type theorems for hypolinear functionals", Math. Ann. 209, 127151. Asplund, E. and Rockafellar, R. T. (1969) "Gradients of convex functions", Trans. Amer. Math. Soc. 139, 443467. Attouch, H. and Beer, G. (1993) "On the convergence of subdifferentials of convex functions", Arch. Math. (Basel) 60(4), 389400. Attouch, H. and Brezis, H. (1986) Duality for the sum of convex functions in general Banach spaces, in "Aspects of mathematics and its applications", NorthHolland, Amsterdam, pp. 125133. Attouch, H. and Thera, M. (1996) "A general duality principle for the sum of two operators", J. Convex Anal. 3(1), 124. Aubin, J.P. and Frankowska, H. (1990) Setvalued analysis, Birkhauser Boston Inc., Boston, MA. Auslender, A., Cominetti, R. and Crouzeix, J.P. (1993) "Convex functions with unbounded level sets and applications to duality theory", SIAM J. Optim. 3(4), 669687. Auslender, A. and Crouzeix, J.P. (1988) "Global regularity theorems", Math. Oper. Res. 13(2), 243253. Auslender, A. and Crouzeix, J.P. (1989) "Wellbehaved asymptotical convex 349
350
Bibliography
functions", Ann. Inst. H. Poincare Anal. Non Lineaire 6(suppl.), 101121. Analyse non lineaire (Perpignan, 1987). Aussel, D., Corvellec, J.N. and Lassonde, M. (1995) "Mean value property and subdifferential criteria for lower semicontinuous functions", Trans. Amer. Math. Soc. 347(10), 41474161. Aze, D. (1994) "Duality for the sum of convex functions in general normed spaces", Arch. Math. (Basel) 62(6), 554561. Aze, D. (1997) Elements d'analyse convexe et variationnelle, Ellipses, Paris. Aze, D. and Penot, J.P. (1995) "Uniformly convex and uniformly smooth convex functions", Ann. Fac. Sci. Toulouse Math. (6) 4(4), 705730. Aze, D. and Rahmouni, A. (1996) "On primaldual stability in convex optimization", J. Convex Anal. 3(2), 309327. Barbu, V. and Da Prato, G. (1985) "HamiltonJacobi equations in Hilbert spaces: variational and semigroup approach", Ann. Mat. Pura Appl. (4) 142, 303349 (1986). Barbu, V. and Precupanu, T. (1986) Convexity and optimization in Banach spaces, second edn, D. Reidel Publishing Co., Dordrecht. Bauschke, H. H., Borwein, J. M. and Combettes, P. L. (2001) "Essential smoothness, essential strict convexity, and Legendre functions in Banach spaces", Commun. Contemp. Math. 3(4), 615647. Bauschke, H. H., Borwein, J. M. and Tseng, P. (2000) "Bounded linear regularity, strong CHIP, and CHIP are distinct properties", J. Convex Anal. 7(2), 395412. Borwein, J. and Kortezov, I. (2001) "Some generic results on nonattaining functionals", SetValued Anal. 9(12), 3547. Wellposedness in optimization and related topics (Gargnano, 1999). Borwein, J. M. (1983) "Adjoint process duality", Math. Oper. Res. 8(3), 403434. Borwein, J. M. and Lewis, A. S. (1992) "Partially finite convex programming. I. Quasi relative interiors and duality theory", Math. Programming 57(1, Ser. B), 1548. Borwein, J. M. and Lewis, A. S. (2000) Convex Analysis and Nonlinear Optimization. Theory and Examples, SpringerVerlag, New York. Borwein, J. M. and Preiss, D. (1987) "A smooth variational principle with applications to sub differentiability and to differentiability of convex functions", Trans. Amer. Math. Soc. 303(2), 517527. Borwein, J. M. and Vanderwerff, J. D. (1995) "Convergence of Lipschitz regularizations of convex functions", J. Funct. Anal. 128(1), 139162. Borwein, J. M. and Vanderwerff, J. D. (2000) Convex functions of Legendre type in general Banach spaces, Technical report, CECM, Simon Fraser University, Barnaby. Bourbaki, N. (1964) Elements de mathematique. Topologie generale, Hermann, Paris. Burke, J. V. and Ferris, M. C. (1993) "Weak sharp minima in mathematical programming", SIAM J. Control Optim. 31(5), 13401359.
Bibliography
351
Butnariu, D. and Iusem, A. N. (2000) Totally convex functions for fixed points computation and infinite dimensional optimization, Kluwer, Dordrecht. Caristi, J. (1976) "Fixed point theorems for mappings satisfying inwardness conditions", Tran. Am. Math. Soc. 215, 241251. Carja, O. (1989) "Range inclusion for convex processes on Banach spaces; applications in controllability", Proc. Amer. Math. Soc. 105(1), 185191. Carja, O. (1998) Elements of nonlinear functional analysis, Editura Universita^ii "Al.I.Cuza", Ia§i. Romanian. Castaing, C. and Valadier, M. (1977) Convex analysis and measurable multifunctions, SpringerVerlag, Berlin. Lecture Notes in Mathematics, Vol. 580. Cioranescu, I. (1990) Geometry of Banach spaces, duality mappings and nonlinear problems, Kluwer Academic Publishers Group, Dordrecht. Clarke, F. H. (1983) Optimization and nonsmooth analysis, John Wiley & Sons Inc., New York. A WileyInterscience Publication. Combari, C , Laghdir, M. and Thibault, L. (1994) "Sousdifferentiels de fonctions convexes composees", Ann. Sci. Math. Quebec 18, 554561. Combari, C , Laghdir, M. and Thibault, L. (1999) "On subdifferential calculus for convex functions defined on locally convex spaces", Ann. Sci. Math. Quebec 23(1), 2336. Cominetti, R. (1994) "Some remarks on convex duality in normed spaces with and without compactness", Control Cybernet. 23(12), 123138. Parametric optimization. Coodey, M. and Simons, S. (1996) "The convex function determined by a multifunction", Bull. Austral. Math. Soc. 54(1), 8797. Cornejo, O., Jourani, A. and Zalinescu, C. (1997) "Conditioning and upperLipschitz inverse subdifferentials in nonsmooth optimization problems", J. Optim. Theory Appl. 95(1), 127148. Correa, R., Jofre, A. and Thibault, L. (1994) "Subdifferential monotonicity as characterization of convex functions", Numer. Funct. Anal. Optim. 15(5& 6), 531535. Crouzeix, J.P. (1980) "Conditions for convexity of quasiconvex functions", Math. Oper. Res. 5(1), 120125. Crouzeix, J.P. (1981) Continuity and differentiability properties of quasiconvex functions on R n , in S. Schaible and W. Ziemba, eds, "Generalized Concavity in Optimization and Economics", Academic Press, pp. 109130. Crouzeix, J.P., Ferland, J. A. and Schaible, S. (1992) "Generalized convexity on affine subspaces with an application to potential functions", Math. Programming 56(2, Ser. A), 223232. Deng, S. (1997) "Computable error bounds for convex inequality systems in reflexive Banach spaces", SIAM J. Optim. 7, 274279. Deng, S. (1998) "Global error bounds for convex inequality systems in Banach spaces", SIAM J. Control Optim. 36(4), 12401249. Deville, R., Godefroy, G. and Zizler, V. (1993) Smoothness and renormings in Banach spaces, Longman Sci. Tech., Harlow. Lecture Notes in Mathematics,
352
Bibliography
Vol. 580. Diestel, J. (1975) Geometry of Banach spaces, Vol. 485 of Lecture Notes in Mathematics, SpringerVerlag, Berlin. Dieudonne, J. (1966) "Sur la separation des ensembles convexes", Math. Ann. 163, 13. Dolecki, S. and Angleraud, P. (1996) "When a well behaving convex function is wellconditioned", Southeast Asian Bull. Math. 20(2), 5963. Dontehev, A. L. and Zolezzi, T. (1993) Wellposed optimization problems, SpringerVerlag, Berlin. Efimov, N. V. and Stechkin, S. B. (1959) "Support properties of sets in Banach spaces and Chebyshev sets", Dokl. Akad. Nauk SSSR 127(2), 254257. Ekeland, I. (1974) "On the variational principle", J. Math. Anal. Appl. 47, 324353. Ekeland, I. and Temam, R. (1974) Analyse convexe et problemes variationnels, Dunod. Collection Etudes Mathematiques. Eremin, I. I. and Astafiev, N. N. (1976) Introduction to the theory of linear and convex programming, Nauka, Moskow. (Russian). Giles, J. R. (1982) Convex analysis with application in the differentiation of convex functions, Pitman (Advanced Publishing Program), Boston, Mass. Hamel, A. (1994) "Remarks to equivalent formulation of Ekeland's variational principle", Optimization 3 1 , 233238. HiriaxtUrruty, J.B. (1982) esub differential calculus, in "Convex analysis and optimization (London, 1980)", Pitman, Boston, Mass., pp. 4392. HiriaxtUrruty, J.B. (1998) Optimisation et analyse convexe, Presses Universitaires de France, Paris. HiriartUrruty, J.B. and Lemarechal, C. (1993) Convex analysis and minimization algorithms. I, SpringerVerlag, Berlin. Fundamentals. HiriartUrruty, J.B., Moussaoui, M., Seeger, A. and Voile, M. (1995) "Subdifferential calculus without qualification conditions, using approximate subdifferentials: a survey", Nonlinear Anal. 24(12), 17271754. HiriartUrruty, J.B. and Phelps, R. R. (1993) "Subdifferential calculus using esubdifferentials", J. Funct. Anal. 118(1), 154166. Holmes, R. B. (1972) A course on optimization and best approximation, SpringerVerlag, Berlin. Lecture Notes in Mathematics, Vol. 257. Holmes, R. B. (1975) Geometric functional analysis and its applications, SpringerVerlag, New York. Graduate Texts in Mathematics, No. 24. Hormander, L. (1955) "Sur la fonction d'appui des ensembles convexes dans un espace localement convexe", Ark. Mat. 3, 181186. Ioffe, A. and Tikhomirov, V. (1974) Theory of extremal problems, Nauka, Moskow. Jameson, J. (1972) "Convex series", Proc. Cambridge Philos. Soc. 72, 3747. Jeyakumar, V. and Wolkowicz, H. (1992) "Generalizations of Slater's constraint qualification for infinite convex programs", Math. Program. 57, 85101. Joly, J.L. and Laurent, P.J. (1971) "Stability and duality in convex minimization
Bibliography
353
problems", R.I.R.O. 2, 342. Jourani, A. (2000) "Hoffman's error bound, local controllability, and sensitivity analysis", SIAM J. Control Optim. 38(3), 947970. Kassara, K. (2000) "Feedback spreading control laws for semilinear distributed parameter systems", Systems Control Lett. 40(4), 289295. Kelley, J. (1955) General topology, Van Nostrand, Princeton. Klatte, D. and Li, W. (1999) "Asymptotic constraint qualifications and global error bounds for convex inequalities", Math. Programming 84(5), 137160. Kosmol, P. and MiillerWichards, D. (2001) Homotopy methods for optimization in orlicz spaces, in V. Bulatov and V. Baturin, eds, "Proceedings of the 12th Baikal International Conference on Optimization Methods and their Applications", Institute of System Dynamics and Control Theory of SB RAS, Irkutsk, pp. 224230. Kusraev, A. G. and Kutateladze, S. S. (1995) Subdifferentials: theory and applications, Kluwer Academic Publishers Group, Dordrecht. Translated from the Russian. Kutateladze, S. S. (1977) "Formulas for the computation of subdifferentials", Dokl. Akad. Nauk SSSR 232(4), 770772. Kutateladze, S. S. (1979a) "Convex operators", Uspekhi Mat. Nauk 34(1(205)), 167196. Russian. Kutateladze, S. S. (1979b) "Convex eprogramming", Dokl. Akad. Nauk SSSR 245(5), 10481050. Laurent, P. (1972) Approximation et optimisation, Hermann, Paris. Lemaire, B. (1985) Subdifferential of a convex composite functional. Application to optimal control in variational inequalities, in "Nondifferentiable Optimization", SpringerVerlag, Sopron, 1984. Lemaire, B. (1989) "Quelques resultats recents sur 1'algorithme proximal", Seminaire d'Analyse Numerique, Toulouse. Lemaire, B. (1992) "Bonne position, conditionnement et bon comportement asymptotique", Semin. Anal. Convexe, Univ. Sci. Tech. Languedoc, Expo. 5. Levitin, E. S. and Polyak, B. T. (1966) "Convergence of minimizing of sequences in the conditionalextremum problem", Dokl. Akad. Nauk SSSR 168, 9971000. Russian. Lewis, A. S. and Pang, J.S. (1998) Error bounds for convex inequality systems, in J.P. Crouzeix, J.E. MartinezLegaz and M. Voile, eds, "Proceedings of the 5th International Symposium on Generalized Convexity held in Luminy, June 1721, 1996", Kluwer Academic Publishers, Dordrecht, pp. 75110. Li, W. and Singer, I. (1998) "Global error bounds for convex multifunctions and applications", Math. Oper. Res. 23(2), 443462. Lifsic, E. A. (1970) "Ideally convex sets", Funkcional. Anal, i Prilozen. 4(4), 7677. Lions, P.L. and Rochet, J.C. (1986) "Hopf formula and multitime HamiltonJacobi equations", Proc. Amer. Math. Soc. 96(1), 7984.
354
Bibliography
Luc, D. T. (1993) "On the maximal monotonicity of subdifferentials", Acta Math. Vietnam. 18(1), 99106. Luc, D. T. and Swaminathan, S. (1993) "A characterization of convex functions", Nonlinear Anal. 20(6), 697701. Marti, J. T. (1977) Konvexe Analysis, Birkhauser Verlag, Basel. Moreau, J. (1965) "Proximite et dualite dans un espace de Hilbert", Bull. Soc. Math. France 93, 273299. Moussaoui, M. and Seeger, A. (1994) "Sensitivity analysis of optimal value functions of convex parametric programs with possibly empty solution sets", SIAM J. Optim. 4(3), 659675. Moussaoui, M. and Voile, M. (1997) "Quasicontinuity and united functions in convex duality theory", Comm. Appl. Nonlinear Anal. 4(4), 7389. Penot, J.P. (1996a) "Conditioning convex and nonconvex problems", J. Optim. Theory Appl. 90(3), 535554. Penot, J.P. (1996b) "Subdifferential calculus without qualification assumptions", J. Convex Anal. 3(2), 207219. Penot, J.P. (1998a) Are generalized derivatives useful for generalized convex functions?, in "Generalized convexity, generalized monotonicity: recent results (Luminy, 1996)", Kluwer Acad. Publ., Dordrecht, pp. 359. Penot, J.P. (1998b) Wellbehavior, wellposedness and nonsmooth analysis, in "Proceedings of the 4th International Conference on Mathematical Methods in Operations Research and 6th Workshop on Wellposedness and Stability of Optimization Problems (Sozopol, 1997)", Vol. 12, pp. 141190. Penot, J.P. and Bougeard, M. L. (1988) "Approximation and decomposition properties of some classes of locally D.C. functions", Math. Programming 41(2 (Ser. A)), 195227. Penot, J.P. and Voile, M. (1990) "Inversion of realvalued functions and applications", Z. Oper. Res. 34(2), 117141. Penot, J.P. and Zalinescu, C. (2000) "Harmonic sum and duality", J. Convex Anal. 7(1), 95113. Phelps, R. R. (1989) Convex functions, monotone operators and differentiability, Vol. 1364 of Lecture Notes Math., SpringerVerlag, Berlin. Phelps, R. R. (1993) Convex functions, monotone operators and differentiability, Vol. 1364 of Lecture Notes Math., second edn, SpringerVerlag, Berlin. Poliquin, R. A. (1990) "Subgradient monotonicity and convex functions", Nonlinear Anal. 14, 305317. Polyak, B. T. (1966) "Existence theorems and convergence of minimizing of sequences in extremum problems with constraints", Dokl. Akad. Nauk SSSR 166, 287290. Russian. Polyak, B. T. (1980) Sharp minimum, Technical report, Institute of Control Sciences, Moskow, USSR. Precupanu, A. (1976) Mathematical Analysis. Real functions, Editura Didactica §i Pedagogical, Bucharest. Romanian. Precupanu, T. (1992) Linear topological spaces and elements of convex analysis,
Bibliography
355
Editura Academiei Romane, Bucharest. Romanian. Robinson, S. M. (1976) "Regularity and stability for convex multivalued functions", Math. Oper. Res. 1(2), 130143. Rockafellar, R. T. (1966) "Characterization of the subdifferentials of convex functions", Pacific J. Math. 17, 497510. Rockafellar, R. T. (1969) "Local boundedness of nonlinear, monotone operators", Michigan Math. J. 16, 397407. Rockafellar, R. T. (1970) Convex analysis, Princeton University Press, Princeton, N.J. Princeton Mathematical Series, No. 28. Rockafellar, R. T. (1974) Conjugate duality and optimization, Society for Industrial and Applied Mathematics, Philadelphia, Pa. Lectures given at the Johns Hopkins University, Baltimore, Md., June, 1973, Conference Board of the Mathematical Sciences Regional Conference Series in Applied Mathematics, No. 16. Rockafellar, R. T. (1976) "Monotone operators and the proximal point algorithm", SIAM J. Control Optimization 14(5), 877898. Rockafellar, R. T. (1981) The theory of subgradients and its applications to problems of optimization, Heldermann Verlag, Berlin. Convex and nonconvex functions. Seeger, A. and Voile, M. (1995) "On a convolution operation obtained by adding level sets: classical and new results", RAIRO Rech. Oper. 29(2), 131154. Shioji, N. (1995) "On uniformly convex functions and uniformly smooth functions", Math. Japonica 4 1 , 641655. e
Simons, S. (1990) "The occasional distributivity of o over + and the change of variable formula for conjugate functions", Nonlinear Anal. 14(12), 1111— 1120. Simons, S. (1994a) "Subtangents with controlled slope", Nonlinear Anal. 22(11), 13731389. Simons, S. (1994b) "Swimming below icebergs", SetValued Anal. 2(12), 327337. Set convergence in nonlinear analysis and optimization. Simons, S. (1998a) Minimax and monotonicity, Vol. 1693 of Lecture Notes Math., SpringerVerlag, Berlin. Simons, S. (1998b) "Sum theorems for monotone operators and convex functions", Trans. Amer. Math. Soc. 350(7), 29532972. Singer, I. (1979) "A FenchelRockafellar type duality theorem for maximization", Bull. Austral. Math. Soc. 20(2), 193198. Soloviov, V. (1993) "Duality for nonconvex optimization and its applications", Anal. Math. 19(4), 297315. Soloviov, V. (1997) "Dual extremal problems and their applications to minimax estimation problems", Russian Math. Surveys 52(4), 685720. Stoer, J. and Witzgall, C. (1970) Convexity and Optimization in Finite Dimensions I, Vol. 163 of Die Grundlehren der Mathematischen Wissenschaften, Springer Verlag, Berlin. Sullivan, F. (1981) "A characterization of complete metric spaces", Proc. Am.
356
Bibliography
Math. Soc. 83, 345346. Thibault, L. (1995a) A generalized sequential formula for sub differentials of sums of convex functions defined on Banach spaces, in "Recent developments in optimization (Dijon, 1994)", Springer, Berlin, pp. 340345. Thibault, L. (1995b) "A note on the Zagrodny mean value theorem", Optimization 35, 127130. Thibault, L. and Zagrodny, D. (1995) "Integration of subdifferentials of lower semicontinuous functions on Banach spaces", J. Math. Anal. Appl. 189(1), 3358. Toland, J. F. (1978) "Duality in nonconvex optimization", J. Math. Anal. Appl. 56, 399415. Ursescu, C. (1975) "Multifunctions with convex closed graph", Czechoslovak Math. J. 25(100)(3), 438441. Vainberg, M. M. (1964) Variational methods for the study of nonlinear operators, HoldenDay Inc., San Francisco, Calif. With a chapter on Newton's method by L. V. Kantorovich and G. P. Akilov. Translated and supplemented by Amiel Feinstein. Vladimirov, A. A., Nesterov, Y. E. and Chekanov, Y. N. (1978) "On uniformly convex functionals", Vestnik Moskov. Univ. Ser. XV Vycisl. Mat. Kibernet. 3, 1223. Russian. Vlasov, L. P. (1972) "Several theorems on Chebyshev sets", Mat. Zametki 11, 135144. Voile, M. (1992) "Some applications of the AttouchBrezis condition to closedness criterions, optimization, and duality", Sem. Anal. Convexe 22, Exp. No. 16, 15. Voile, M. (1993) "Sousdifferentiel d'une enveloppe superieure de fonctions convexes", C. R. Acad. Sci. Paris Ser. I Math. 317(9), 845849. Voile, M. (1994) "Sur quelques formules de dualite convexe et non convexe", SetValued Anal. 2(12), 369379. Set convergence in nonlinear analysis and optimization. Walkup, D. W. and Wets, R. J.B. (1967) "Continuity of some convexconevalued mappings", Proc. Amer. Math. Soc. 18, 229235. Willard, S. (1971) General topology, AddisonWesley, Reading. Zagrodny, D. (1988) "Approximate mean value theorem for upper subderivatives", Nonlinear Anal. 12, 14131428. Zalinescu, C. (1978) "A generalization of the Farkas lemma and applications to convex programming", J. Math. Anal. Appl. 66(3), 651678. Zalinescu, C. (1980) "On an abstract control problem", Numer. Funct. Anal. Optim. 2(6), 531542 (1981). Zalinescu, C. (1983a) "Duality for vectorial nonconvex optimization by convexification and applications", An. §tiinl;. Univ. "Al. I. Cuza" Ia§i Sec^. I a Mat. (N.S.) 29(3, suppl.), 1534. Zalinescu, C. (1983b) "On uniformly convex functions", J. Math. Anal. Appl. 95(2), 344374.
Bibliography
357
Zalinescu, C. (1984) Duality for vectorial convex optimization, conjugate operators and sub differentials. The continuous case. Unpublished. Presented at the Conference "Mathematical Programming — Theory and Applications", Eisenach. Zalinescu, C. (1986) "Letter to the editor: on J. M. Borwein's paper: "Adjoint process duality"", Math. Oper. Res. 11(4), 692698. Zalinescu, C. (1987) "Solvability results for sublinear functions and operators", Z. Oper. Res. Ser. AB 31(3), A79A101. Zalinescu, C. (1989) Stability for a class of nonlinear optimization problems and applications, in "Nonsmooth optimization and related topics (Erice, 1988)", Plenum, New York, pp. 437458. Zalinescu, C. (1992a) "A note on dstability of convex programs and limiting Lagrangians", Math. Programming 53(3, Ser. A), 267277. Zalinescu, C. (1992b) "On some open problems in convex analysis", Arch. Math. (Basel) 59(6), 566571. Zalinescu, C. (1998) Mathematical programming in infinite dimensional normed spaces, Editura Academiei, Bucharest, Romania. (Romanian). Zalinescu, C. (1999) "A comparison of constraint qualifications in infinitedimensional convex programming revisited", J. Austral. Math. Soc. Ser. B 40(3), 353378. Zalinescu, C. (2001) Weak sharp minima, wellbehaving functions and global error bounds for convex inequalities in Banach spaces, in V. Bulatov and V. Baturin, eds, "Proceedings of the 12th Baikal International Conference on Optimization Methods and their Applications", Institute of System Dynamics and Control Theory of SB RAS, Irkutsk, pp. 272284.
This page is intentionally left blank
Index
function Bdifferentiable, 191 Ssmooth, 191 bcscomplete, 68 coercive, 100 strongly, 214 concave (strictly), 40 conjugate (Fenchel), 75 convex, 39 strictly, 40 strongly, 203 cost, 99 csclosed, 68 cscomplete, 68 csconvex, 67 distance, 237 ideally convex, 68 indicator, 41 Lagrange, 138 lcsclosed, 68 liconvex, 68 Lipschitz, 66
best approximation, 237 bornology, 190 Frechet, 190 Gateaux, 190 Hadamard, 190 cone, 1 dual, 7 normal, 87 recession, 6 tangent (of Clarke), 169 convolution, 43 directional derivative, 56 ClarkeRockafellar, 171 e, 58
upper Dini, 58 domain of function, 39 multifunction, 12 operator, 42 duality strong, 107 weak, 107 duality mapping, 230
locally, 66 marginal, 43 objective, 99 performance, 107 perturbation, 106 proper, 39 ^conditioned, 196 with respect to, 196
epigraph of function, 39 strict, 39 operator, 42 359
360
Qdecreasing, 42 Qincreasing, 42 quasicontinuous, 67 quasiconvex, 41 recession, 74 pconvex, 203 crsmooth, 204 on a set, 204 subdifferentiable, 80 sublinear, 4 support, 79 uniformly convex, 203 at a point, 201 on a set, 203 on bounded sets, 221 uniformly Frechet differentiable, 207 uniformly smooth on a set, 207 on bounded sets, 221 value, 107 weight, 227 wellbehaved (asymptotically), 259 wellconditioned, 197 gage of conditioning, 196 smoothness (midpoint), 205 uniform convexity, 203 at a point, 201 uniform smoothness, 205 of the norm, 231 on a set, 204 global error bound, 262, 265 graph of multifunction, 12 halfspace closed, 5 open, 5 hull affine, 2 conic, 2 closed, 7 convex, 2
Index closed, 7 linear, 2 lsc convex, 63 hyperplane closed, 5 supporting, 5 image of multifunction, 12 interior algebraic, 3 relative, 3 with respect to, 2 quasi relative, 15 KadecKlee property, 233 weak*, 233 Lagrange multiplier, 139, 140 Lagrangian, 138 lsc envelope, 62 lsc regularization, 62 maxconvolution, 43 metrically regular system, 269 Minkowski gauge, 4 multifunction, 12 bcscomplete, 13 closed, 13 cscomplete, 13 csconvex, 13 ideally convex, 13 inverse, 12 lcsclosed, 13 liconvex, 13 locally bounded, 286 at a point, 286 monotone, 82 cyclically, 84 maximal, 82 strictly, 82 normed space smooth, 227 strictly convex, 227
Index uniformly convex, 234 locally, 234 uniformly smooth, 231 at a point, 231 locally, 231 operator Qconcave, 42 Qconvex, 42 relatively open, 119 point local maximum, 105 local minimum, 105 saddle, 138 sharp minimum, 248 support, 5 problem convex programming, 99 dual, 107 normal, 109 primal, 107 stable, 109 process, 26 adjoint, 26 closed, 26 convex, 26 quasiinverse greatest, 188 lowest, 188 seminorm, 4 separation of two sets, 5 proper, 5 strict, 5 series bconvex, 9 Cauchy, 9 convergent, 9 convex, 9 set absorbing, 4 admissible solutions, 99
affine, 1 balanced, 1 bcscomplete, 9 Chebyshev, 237 constraints, 99 convex, 1 csclosed, 9 cscomplete, 9 ideally convex, 9 level, 39 strict, 39 lower csclosed (lcsclosed), 11 lower ideally convex (liconvex monotone, 82 maximal, 82 strictly, 82 polar, 7 quasicontinuous, 67 starshaped, 1 symmetric, 1 weak sharp minima, 248 sets united, 17 Slater's condition, 138 solution admissible, 137 e, 106 optimal, 99 space algebraic dual, 3 barreled, 9 complete, 8 first countable, 8 Prechet, 8 linear, parallel to, 2 orthogonal, 7 quasicomplete, 8 topological dual, 4 subdifferential, 80 abstract, 178 Clarke, 175 e,82 Fenchel, 80 subgradient, 80 e,82
362
support functional, 5 point, 5 theorem AlaogluBourbaki, 8 Baire I, 34 Baire II, 34 BanachSteinhaus, 25 biconjugate, 77 bipolar, 7 BishopPhelps, 166 Borwein, 159 BorweinPreiss, 31 Br0ndstedRockafellar, 161 closed graph, 25 Dieudonne, 7 Eidelheit, 5 Ekeland, 29 IoffeTikhomirov, 97 open mapping, 25 PshenichnyiRockafellax, 136 Robinson, 23 Rockafellar, 169, 288 Simons, 22, 167 Ursescu, 23 Zagrodny, 179 value of a problem, 99 saddle, 144 YoungFenchel inequality, 75
Symbols and Notations
A denotes A + B, AB — Af A1 — Aoo
A+, A++ A0, A00 Ax, A x x { A 'bA ic A A A0 aff A argmin / B[x,e) Bx Kx BdA
— — — — — — — — — — —
e*
—
C(A;a) C[a, b] C(X) clA co A, co A co/ cone A, cone A Df{x,) D(x,s) d
— — — — — — — ~
the closure of the set A C (X, r) the sets {a + b  a € A, b€ B}, {a  b  a € A, b £ B} the function defined by (Af)(y) = inf{/(x)  Ax = y} the algebraic interior of A (p. 3) the cone (~]t>0 t(A — a) with a € A when A = co A the sets {x* G X*  Vx £ A : {x,x") > 0} and (A+)+ thesets {x* e X* j Vx e A (x,x*) >  1 } and (A 0 ) 0 the sets {x* € X* \Vx e A <x,x*) = 0} and (A1)1 the relative algebraic interior of A (p. 3) 'A if the parallel subspace to aff A is barreled, 0 otherwise 'A if aff A is closed, 0 otherwise the set {(p : R+ + R+ \
364 diam^l — dim A — dom/ — domH — domK — dyi(x), d(x, ^4) — epi / — epi s / — epi# — f\A — / — /oo — /+ — f'(t), f"(t) — /+(£), /(£) — / ' ( a , •) — / T ( a , •) — f'e(a, •) — /*,/** — /iD/2 — /1O/2 — [/ < A], [/ < A] f,p — gr ft — grT — (Hx), (Hwx) — HX',a — H x*,a>Hx*,a — H }*,a>Ht*,a — Idx — ImT — Imft — inf 3, infx 3 — int yl — inty A — kerT — lcs — L 1 (0,1) — L(X, Y) — £>(X, Y) — liminfig/ A, liminf^^a f(x) l i m s u p i € / A;
Symbols and Notations the diameter of A C (X, d), i.e. swp{d(x,y) \ x, y
• Y see p. 14 the set {x G X  (x,x*) = a } the sets { x G X  <x,x*) < a } , {x G X  (x,x*) > a } the sets { x G X  (x,x*> > a } , {x G X  (X,X*) < a } the function from X into X defined by Idx (x) = x the set {T[x)  x G X } for the operator T : X +Y the set \Jx€X ft(x) for the multifunction ft : X =4 Y the number inf^gx g(x), where g : X —> R the interior of the set A C (X, r ) the interior of A C Y w.r.t. the topology on Y C (X, T) the set {x G X  T(x) = 0} for T G L(X, Y) separated locally convex space the space of (classes of) Lebesgue integrable functions on ]0,1[ the space of linear operators T : X —> Y the space of continuous linear operators from X into Y — s u p J g / infi>j A, G R for the net (Xi)iei C R — s u p V € X ( a ) inf x e y / ( x ) G R for / : (X, r ) >• R — infje/ s u p i t • Aj G R for the net (\i)iei C R
Symbols and Notations
365
l i m s u p ^ , , f(x) lin A — lsc — £p(p>l) — £°° — N — N(A; a) — No, Ni, N2 — N(x), Jfx{x) — ySx — yfx — P — V — Pc(x) — Prx — PA{X) — p, q, r — qri A — R, R — R+, R+ — RE, R — D? : X =£ Y —
•R1
— inf v e W ( o ) sup.,. ev f(x) 6 R for / : (X, T) > R the linear hull of the set A (p. 2) lower semicontinuous thespace{(a;n)cR(x„)p:=(£n>iMP)1/p
— or1 : y = t x , or^y) — ^ yeft(z)}forft:X =4 Y
ft o S ft + S ft(A) ft_1(B) rec A ri A rint A Sx S(f, C) S(P) Se(f,C) Se(P) SA supp/i T* To(M, x) tvs (t n )40 Ux use v(f, C)
— — — — — — — — — — — — — — — — — — — — —
the composition of the multifunctions ft and S (p. 12) the sum of the multifunctions ft and S (p. 12) the set {y € Y  3 x € A : (x, y) € ft} the set {x £ X  3 y eB : (x,y)eX) the set {u £ X \ Va 6 A : a + u e A} the set rint A if aff A is closed, 0 otherwise the relative interior of A, i.e. intaff.4 A the set Ux \ Bx the set {x £ C  V i € C : f(x) < f(x)} the set 5 ( / , C) for the problem: (P) min f(x), xeC the set {xeC\\/xsC : f{x)
Symbols and Notations
366 v(P) — w — w* — w.r.t. — X', X* — X/Xo — (X, d) — (X, r ) — Xi —¥ x, (xi) —> x, (xi) —* x — (x^) ^> x* — {xnk)ke®t {xnk) [x,y],[x,y[ — }x,y[ — (x,x*) — y* — 2/i
v(f, C) for the problem: (P) min f(x), xeC the topology <x(X, X*) on the lcs X the topology a(X*,X) on the topological dual X* of X with respect to the spaces L{X,R) and £ ( X , R ) , respectively the quotient space of X w.r.t. the linear subspace Xo C X a metric space a topological space x = \imxi — the net (xi) converges to x the net (xi) converges to x for the topology w of X
the net (x*) converge to x* for the topology w* of X* — a subsequence of (xn) {(1  X)x + Xy  A G [0,1]} and {(1  X)x + Xy  A G [0,1[} the set {(1  A)x + Ay  A G ] 0 , l[} the image of x G X by x* G X* the set Y U {oo} — the fact that J/2 — 2/i £ Q,for Q CY convex cone the function tdom / the set { 1 , . . . , n} for n G N a norm, the norm of the element x the Fenchel subdifferential of the function / (p. 80) the esubdifferential of the function / (p. 82) the Clarke subdifferential of the function / (p. 175) abstract subdifferential (p. 178) the partial derivative of the function / with respect to x, d(df/dxi)/dxj the class of lsc convex functions / G A the sets TnAo, {
0}_ the class of lsc proper convex functions / : X —> R the class of tu*lsc proper convex functions h : X* —> R t h e s e t {(Ai,...,A„) € R +  Ai +  + A„ = 1} for n e N the element of C.E for which Su(x) := 1 if x — u, 0 otherwise the duality mapping of the normed space (X,  • ) (p. 230) — the function defined by V>A{X) := inf{<  (x,t) G A} the conjugate of ip G A :
Symbols and
Notations
367
the gage of uniform convexity of / on rUx (p 221) the gage of uniform convexity of / at x (p. 201) the sets r l~l fii, {a € Si  \imsupt^root~2ifi(t) < oo} gages of uniform smoothness of / (p. 204) see p. 210 gages of uniform smoothness of / on A (p. 204) gages of uniform smoothness of / on rUx (p 221) the gage of smoothness of / at x (p. 191) the gage of smoothness of the norm at x (p. 231) the gage of uniform smoothness of the norm of X (p. 231) the set {/i: E ¥ [0, oo[  supp/i is finite, ^2x€EKx) — 1} the set { u : I >• [ji€I Xi\ V i e / : u(i) € X{} the topology associated with a metric d the topology generated by the family of seminorms V function of the form Qp(x) := X2 n>0 /in z — Mnp (p 31) see pages 201, 225, respectively the gradient (the differential) of the function / at a the second order differential of the function / at a for all, for every there exists (at least one) maximum of extended real numbers or logical "or" minimum of extended real numbers or logical "and" a := b, b =: a means a is equal to b by definition the end of a proof or of a theorem without proof