This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
that evaluates a linear map (p : X at x G X is a duality.
^^K
1. Adjoint or dual maps Let X, Y be vector spaces, X* and F* their duals and < , >x and < , >Y the evaluation maps on X* x X and Y* x Y. For every linear map £ : X ^^ y , one also has a map ^* : F* -^ X* defined by < r(2/*),x > : = < y\e{x)
>
Vx G X,V2/* G F*.
It turns out that £* is linear. Now if (ei, e2,. •., Cn) and ( / i , . . . , fm) are bases in X and Y respectively, and (e^, e ^ , . . . , e^) and (/^, / ^ , . . . , / ^ ) are the dual bases in X* and F*, then the associated matrices L = [L^] G Mm,n(If^) and M = [Mj] G Mn,m(IK), associated respectively to £ and ^*, are defined by m
n
h=l
2=1
By duality, i.e., M = L-^. Therefore we conclude that i / L is the matrix associated to £ in a given basis, then L^ is the matrix associated to £* in the dual basis. We can now discuss how coordinate changes in X reflect on the dual space. Let X be a vector space of dimension n, X* its dual space, ( e i , . . . , e^), (ei, 62,..., Cn) two bases on X and ( e \ e^,..., e^) and (e^ e^,..., e") the corresponding dual bases on X*. Let ^ : X -^ X be the linear map defined by £{ei) := e^ Vi = 1 , . . . , n. Then by duality <£*{e'),ej
> = <e\£{ej)
> = < e\ej > = Sij =< e\ej
>
Vz,j,
^*(e*) = e* Vz = 1 , . . . , n. If L and L^ are the associated matrices to £ and ^*, L changes basis from (ei, e 2 , . . . , e-n) to (ei, 62,..., en) in X, and L^ changes basis in the dual space from (e^, e^,..., e") to (e^, e^,..., e").
2.2 Eigenvectors and Similar Matrices
57
Now if (/? G X*, we have (/? = Zir=i ^i^^ ~ Y17=i ^^^^ hence n
n
i=l
n
n
z=l j = l
z=l
Thus, if a := (ai, a 2 , . . . , a^), b := (61, 62, • • •, bn)^ we have a^ = L ^ b ^
or
a = bL.
In other words, the coordinates in X* change according to the change of basis. We say that the change of coordinates in X* is covariant
2.2 Eigenvectors and Similar Matrices Let A : X —^ X he a. hnear operator on a vector space. How can we describe the properties of A that are invariant by isomorphisms? Since isomorphims amount to changing basis, we can put it in another way. Suppose X is finite dimensional, dim X = n, then we may consider the matrix A associated to A using a basis (we use the same basis both in the source and the target X). But how can we catch the properties of A that are independent of the basis? One possibihty is to try to choose an "optimal" basis in which, say, the matrix A takes the simplest form. As we have seen, if we choose two coordinate systems £ and ^ on X, and S is the matrix that changes coordinates from £ to ^ , then the matrices A and B that represent A respectively in the basis £ and J^ are related by B = SAS-^ Therefore we are asking for a nonsingular matrix S such that S~^AS has the simplest possible form: this is the problem of reducing a matrix to a canonical form. Let us try to make the meaning of "simplest" for a matrix more precise. Suppose that in X there are two supplementary invariant subspaces under A X = Wi® W2,
A{Wi) C Wi, A{W2) C W2.
Then every x e X splits uniquely as x = xi + X2 with xi G VFi, X2 G W25 and A{x) = A{xi) + A{x2) with A(xi) G Wi and A{x2) G VF2. In other words, A splits into two operators Ai : Wi ^^ Wi^ A \ W2 -^ W2 that are the restrictions of A to Wi and W2- Now suppose that d i m X = n and let (ei, 6 2 , . . . , Ck) and (/i, /2, •. •, fn-k) be two bases respectively of Wi and W2. Then the matrix associated to A in the basis (ei, 6 2 , . . . , efc, / i , / 2 , . . •, fn-k) of X has the form
58
2. Vector Spaces and Linear Maps
A =
Ai 0
where some of the entries are zero. If we pursue this approach, the optimum would be the decomposition of X into n supplementary invariant subspaces Wi, W2,.. •, Wn under A of dimension 1,
X = T^i e H^2 e • • • e T^n,
A(Wi) c Wi.
In this case, A acts on each Wi as a dilation: A{x) = Xix Vx G Wi for some A^ G K. Morever, if (ei, 6 2 , . . . , Cn) is a basis of X such that ei G W^ for each 2, then the matrix associated to A in this basis is the diagonal matrix A = diag(Ai,A2,...,An).
2.2-1 Eigenvectors a. Eigenvectors and eigenvalues As usual, K denotes the field R or C. 2.33 Definition. Let A : X -^ X be a linear operator on a vector space X over K. We say that x E X is an eigenvector of A if Ax = Xx for some X E K. If X is a nonzero eigenvector, the number X for which A{x) = Xx is called an eigenvalue of A, or more precisely, the eigenvalue of A relative to X. The set of eigenvalues of A is called the spectrum of A. / / A G Mn,nO^), we refer to eigenvalues and eigenvectors of the associated linear operator A : W^ —> W^, A(x) := Ax, as the eigenvalues and the eigenvectors of A. Prom the definition, A is an eigenvalue of A if and only if ker(A Id — A) 7^ {0}, equivalently, if and only if Aid — >1 is not invertible. If A is an eigenvalue, the subspace of all eigenvectors with eigenvalue A FA := {^ e X I A{x) = Xx\ = ker(AId - A) is called the eigenspace of A relative to A. 2.34 E x a m p l e , let X = C°° ([0, n]) be the linear space of smooth functions that vanish at 0 and n and let £>^ : X —> X he the linear operator D'^{f) := j " that maps every function / into its second derivative. Nonzero eigenvectors of the operator D^ ^ that is, the nonidentically zero functions y € C°*^[0,1] such that D'^y{x) = \y{x) for some A G M, are called eigenfunctions. 2.35 E x a m p l e . Let X be the set Pn of polynomials of degree less than n. Then, each Pfc d Pn A; = 0 , . . . , n is an invariant subspace for the operator of differentiation. It has zero as a unique eigenvalue.
2.2 Eigenvectors and Similar Matrices
59
2.36 %, Show that the rotation in E^ by an angle 6 has no nonzero eigenvectors if 9 ^ 0,n, since in this case there are no invariant lines.
2.37 Definition. Let A : X ^^ X be a linear operator on X. A subspace W C X is invariant (under A) if A{W) C W. In the following proposition we collect some simple properties of eigenvectors. 2.38 Proposition. Let A : X -^ X be a linear operator on X. (i) X 7^ 0 is an eigenvector if and only if Span {x} is an invariant subspace under A. (ii) Let A be an eigenvector of A and let Vx be the corresponding eigenspace. Then every subspace W C Vx is an invariant subspace under A, i.e., A{W) C W. (iii) dimker(AId — A) > 0 if and only if A is an eigenvalue for A. (iv) Let W C X be an invariant subspace under A and let A be an eigenvalue for A^^r. Then A is an eigenvalue for A : X ^ X . (v) A is an eigenvalue for A if and only ifO is an eigenvalue for Aid —^. (vi) Let if : X —^ Y be an isomorphism and let A : X -^ X be an operator. Then x E X is an eigenvector for A if and only if (p{x) is an eigenvector for (p o A o (p~^, and x and ip{x) have the same eigenvalue. (vii) Nonzero eigenvectors with different eigenvalues are linearly independent. Proof, (i), . . . , (vi) are trivial. To prove (vii) we proceed by induction on the number k of eigenvectors. For A; = 1 the claim is trivial. Now assume by induction that the claim holds for fc — 1 nonzero eigenvectors, and let e i , 62, • • •, e^ be such that ej / 0 Vi = 1 , . . . , fc, A{ej) = XjCj Vj = 1 , . . . , fc with Xj ^ Ai Vi 7^ j . Let aiei-fa2e2H
\-akCk = 0,
(2.8)
be a linear combination of e i , 6 2 , . . . , e^. From (2.8), multiplying by Ai and applying A we get aiXiei
+ a2Aie2 H
h afcAie^ = 0,
a i A i e i -h a2A2e2 H
h OfcAfcefc = 0,
consequently k
^{Xj
-Xi)ajej
=0.
j=2
By the inductive assumption, aj{Xj — Ai) = 0 Vj = 2 , . . . , n, hence aj = 0 for all j > 2. We then conclude from (2.8) that we also have a i = 0, i.e., that e i , 6 2 , . - . , Cfc are linearly independent. D
Let A : X ^^ X he a. linear operator on X of dimension n, and let A be the associated matrix in a coordinate system f : X —> W^. Then (vi) implies that x G X is an eigenvector of A if and only if x := £{x) is an eigenvector for x —> A x and x and x have the same eigenvalue. Prom (vii) Proposition 2.38 we infer the following.
60
2. Vector Spaces and Linear Maps
2.39 Corollary. Let A : X -^ X be a linear operator on a vector space X of dimension n. If A has n different eigenvalues, then X has a basis formed by eigenvectors of A. b. Similar matrices Let A : X ^^ Xhea, linear operator on a vector space X of dimension n. As we have seen, if we fix a basis, we can represent A by an n x n matrix. If A and A' G Mn^ni^) ^^^ ^wo such matrices that represent A in two different bases (ei, e 2 , . . . , e^) and (ei, 62,..., en), then by Proposition 2.28 A' = S~^AS where S is the matrix that changes basis from (ei, ^ 2 , . . . , e^) to (ei, € 2 , . . . ,
€n)-
2.40 Definition. Two matrices A , B G Mn n(K) are said to be similar if there exists S G GL(n,K) such that B = S~ AS. It turns out that the similarity relation is an equivalence relation on matrices, thus nxn matrices are partitioned into classes of similar matrices. Since matrices associated to a linear operator A : X ^^ X^ dimX = n, are similar, we can associate to A a unique class of similar matrices. It follows that if a property is preserved by similarity equivalence, then it can be taken as a property of the linear operator to which the class is referred. For instance, let A : X -^ X he a. hnear operator, and let A, B be such that B = S~^AS. By Binet's formula, we have det B = det S"^ det A det S = —— det A det S = det A. detS Thus we may define the determinant of the linear map A : X —^ X hy det A := det A where A is any matrix associated to A. c. The characteristic polynomial Let X be a vector space of dimension n, and let A : X ^^ X he a, hnear operator. The function ^-^PAW
'•= det(AId -A),
AG K,
is called the characteristic polynomial of A. It can be computed by representing A by a matrix A in a coordinate system and computing PAW as the characteristic polynomial of any of the matrices A representing A, PA{X)=PA{X)
=
detiXld-A).
In particular, it follows that p^( ) : K —> IK is a polynomial in A of degree n, and that the roots of p^(A) are the eigenvalues of A or of A. Moreover, we can state
2.2 Eigenvectors and Similar Matrices
61
2.41 Proposition. We have the following. (i) Two similar matrices A, B have the same eigenvalues and the same characteristic polynomials. (ii) / / A has the form
([M\ 0
0 A2
u
0
0 \ 0
AfeU
where for z = 1 , . . . , fc, each block A^ is a square matrix of dimension ki with principal diagonal on the principal diagonal of A, then PA{S) = PAA^) ' PA2{s). • .PAfc(s).
(iii) We have
det(5ld - A) = 5^ - tr A5^-i + • • • + (-l)^det A n k=l
where t r A := X]^=i A^ is the trace of the matrix A, and ak is the sum of the determinants of the kx k submatrices of A with principal diagonal on the principal diagonal of A. Proof, (i) If B = S A S - i , S G GL(n,K), then s I d - B = S ( s l d - A ) S - i , hence det(s Id - B ) = det S det(s Id - A)(det S ) - ^ = det(s Id - A), (ii) The matrix 5 Id — A is a block matrix of the same form
sId-A2
5 Id - Afc
V hence det(s Id - A) = Y[i=i det{s Id - A^). (iii) We leave it to the reader.
Notice that there exist matrices with the same eigenvalues that are not similar, see Exercise 2.73.
62
2. Vector Spaces and Linear Maps
d. Algebraic and geometric multiplicity 2.42 Definition. Let A : X -^ X be a linear operator, and let X GK be an eigenvalue of A. We say that A has geometric multiplicity k e N if dimker(AId - A) = k. Let PA{S) be the characteristic polynomial of A. We say that A has algebraic multiplicity k if PA{S) = {S- X)^q{s),
where q{X) ^ 0.
2.43 Proposition. Let A : X -^ X be a linear operator on a vector space of dimension n and let A be an eigenvalue of A of algebraic multiplicity m. Then dimker(AId - A) <m. Proof. Let us choose a basis (ei, 6 2 , . . •, Cn) in X such that (ei, e2, • •., e^) is a basis for Vx '•= ker(AId — A). The matrix A associated to A in this basis has the form
|\
(\ Aid
C
0
D
A =
vl
1/
where the first block, Aid, is a fc x /c matrix of dimension k = dimV^. Thus Proposition 2.41 (ii) yields PA{S) = det(s Id — A) = (s — \)^pr>(s), and the multiplicity of A is at least k. D
e. Diagonizable matrices 2.44 Definition. We say that A G Mn,n{^) is diagonizable, if A is similar to a diagonal matrix. 2.45 Theorem. Let A : X -^ X be a linear operator on a vector space of dimension n, and let (ei, 6 2 , . . . , Cn) be a basis of X. The following claims are equivalent. (i) ei, 6 2 , . . . , Cn are eigenvectors of A and Ai, A2,..., An are the relative eigenvalues. (ii) We have A{x) = Yl^=i K^^^i /^^ all x e X if x = Yl7=i ^^^i(iii) The matrix that represents A in the basis (ei, ^ 2 , . . . , e^) is diag(Ai, A2,..., An). (iv) If A is the matrix associated to A in the basis (/i, / 2 , . . . , fn), then S-^AS = dia^(Ai, A2,..., An) where S is the matrix that changes basis from (/i, /25 • • • ? /n) io (ei, 6 2 , . . . , Cn), i-e., the ith column of S is the coordinate vector of the eigenvector ei in the basis (/i, 7 2 , . . . , /n)-
2.2 Eigenvectors and Similar Matrices
63
Proof, (i) ^ (ii) by linearity and (iii) <=> (i) since (iii) is equivalent to A{ei) = Xiei. D Finally (iii) ^ (iv) by Corollary 2.29.
2.46 Corollary. Let A : X ^^ X be a linear operator on a vector space of dimension n. Then the following claims are equivalent. (i) X splits as the direct sum of n one-dimensional invariant subspaces (under A), X = Wie • • - ^ Wn. (ii) X has a basis made of eigenvectors of A. (iii) Let Ai, A2,..., A^ be all distinct eigenvalues of A, and let Vx^, • • •, Vx,^ be the corresponding eigenspaces. Then y ^ dim Vxi = n. (iv) / / A is the matrix associated to A in a basis, then A is diagonizable. Proof, (i) implies (ii) since any nonzero vector in any of the Wis is an eigenvector. Denoting by (ei, e2,. • •, en) a basis of eigenvectors, the spaces Wi := Spanje^} are supplementary spaces of dimension one, hence (ii) implies (i). (iii) is a rewriting of (i) since for each eigenvalue A, Vx is the direct sum of the Wi^s that have A as the corresponding eigenvalue. Finally (ii) and (iii) are equivalent by Theorem 2.45. D
2.47 Linear equations. The existence of a basis of X of eigenvectors of an operator A : X —^ X makes solving the Unear equation A{x) — y trivial. Let (ei, e 2 , . . . , Cn) be a basis of X of eigenvectors of A and let Ai, A2,..., An be the corresponding eigenvalues. Writing x^y e X in this basis, n
n
x = Y^x'ei,
^y'ei,
i=l
i=l
we rewrite the equation A{x) = y as
5^(A,x^-2/%=0, z=l
i.e., as the diagonal system Aix^ =y^, ..., (AnX
Therefore
—y
.
64
2. Vector Spaces and Linear Maps
(i) suppose that 0 is not an eigenvalue, then A{x) = y has a unique solution
=E
•ei.
Z=l
(ii) let 0 be an eigenvalue, and let VQ — Span{ei, e 2 , . . . , e^}. Then A{x) = 2/ is solvable if and only \i y^ = --- — y^ — ^ and a solution of A{x) = 7/ is xo := Xir=fc+i A"^^- -^^ linearity, the space of all solutions is the set | X G X X - X O G
kerA =
VQ
[•
2.48 % Let A : X —>• X be a linear operator on a finite-dimensional space. Show that A is invertible if and only if 0 is not an eigenvalue for A. In this case show that 1/A is an eigenvalue for A"^ if and only if A is an eigenvalue for A.
f. Triangularizable matrices First, we notice that the eigenvalues of a triangular matrix are the entries of the principal diagonal. We can then state the following. 2.49 Theorem. Let A G Mn,n(IK). / / the characteristic polynomial decomposes as a product of factors of first degree, i.e., if there are (not necessarily distinct) numbers Ai, A2,..., An G K such that det(AId - A) = (A - A i ) . . . (A - An), then A is similar to an upper triangular matrix. Proof. Let us prove the following equivalent claim. Let A : X -^ X he a linear operator on a vector space of dimension n. If PAW factorizes as a product of factors of first degree, then there exists a basis (iti, it2, • • •, Un) of X such that Span{wi}, Span {it 1,1*2}, Span {141,1*2,^x3}, . . . ,Span {1*1,1*2, • • itn} are invariant subspaces under A. In this case we have for the linear operator >l(x) = A x associated to A A u i = A(ui)
= a}ui,
Au2 = ^ ( u 2 ) = a2Ui H- a2U2,
[AUn
= A(Un)
= a^Ui
+ a^U2
H
(- a^JUn,
i.e., the matrix A associated to A using the basis (t*i, 1*2, • • •, Un) is upper triangular, /a\
A =
0
Vo
4\
4
„2
0
a^/
2.2 Eigenvectors and Similar Matrices
65
We proceed by induction on the dimension of X . If d i m X = 1, the claim is trivial. Now suppose that the claim holds for any linear operator on a vector space of dimension n — 1, and let us prove the claim for A. Prom PA (A) = det(AId - A) = (A - A i ) . . . (A - An), Ai is an eigenvalue of A, hence there is a corresponding nonzero eigenvalue ui and Span {ui} is an invariant subspace under A. Now we complete {ui} as a basis by adding vectors f2, • • • i^n, and let B be the restriction of the operator A to Span {v2,.. .Vn}Let B be the matrix associated to B in the basis {v2,. • • ,Vn), and let A be the matrix associated to A in the basis (iti, W2, • • •, Wn)- Then
/«!
where ai = Ai.
A =
\ Thus PA(X)
= PA (A) = (A - A I ) P B ( A ) = (A -
AI)PB(A).
It follows that the characteristic polynomial of B is P B ( A ) = (A — A 2 ) . . . (A — An). By the inductive hypothesis, there exists a basis (u2,... itn) of Span {v2,..., fn} such that Span{1x2}, Span{u2,W3}, . . . , S p a n { i i 2 , . . •,Wn} are invariant subspaces under B, hence S p a n { u i } , Span{iti,ii2}, Span{141,1x2,^3}, . . . ,Span{iti,tX2,... Wn} are invariant subspaces under A.
D
2.2.2 Complex matrices When K = C, a significant simplification arises. Because of the fundamental theorem of algebra, the characteristic polynomial PA (A) of every linear operator A : X ^^ X over a complex vector space X of dimension n, factorizes as product of n factors of first degree. In particular, A has n eigenvalues, if we count them with their multiplicities. Prom Theorem 2.49 we conclude the following at once. 2.50 Corollary. Let A e Mn,n(C) be a complex matrix. Then A is similar to an upper triangular matrix, that is, there exists a nonsingular matrix S G Mn,n(C) such that S~-^AS is upper triangular. Moreover, 2.51 Corollary. Let A G Mn,n(C)6e a matrix. Then A is diagonizahle (as a complex matrix) if and only if the geometric and algebraic multiplicities of each eigenvalue agree.
66
2. Vector Spaces and Linear Maps
Proof. Let Ai, A 2 , . . . , Afc be the distinct eigenvalues of A , for each i = 1^.. .k,\et and Vx^ respectively be the algebraic multiplicity and the eigenspace of A^. If dim V\^ = rrii Vi, then by the fundamental theorem of algebra k
rrii
n
^ d i m V ^ . =^mi i=l
= n.
i=l
Hence A is diagonizable, by Corollary 2.46. Conversely, if A is diagonizable, then Yli=i dim Vx^ = n, hence by Proposition 2.43 dimVxi ^ i^i^ hence k
n = ^rrii
k
> ^ d i m V ^ i = n. D
2.52 Remark (Real and complex eigenvalues). If A G Mn,n{^), its eigenvalues are by definition the real solutions of the polynomial equation det(AId — A) = 0. But A is also a matrix with complex entries, A G Mn,n{C) and it has as eigenvalues which are the complex solutions of det(AId — A) = 0. It is customary to call eigenvalues of A the complex solutions of det(AId — A) = 0 even if A has real entries, while the real solutions of the same equation, which are the eigenvalues of the real matrix A following Definition 2.33, are called real eigenvalues. The further developments we want to discuss depend on some relationships among polynomials and matrices that we now want to illustrate. a. The Cayley-Hamilton theorem Given a polynomial f{t) = ^^=1 CLk^^•> to every n x n matrix A we can associate a new matrix / ( A ) defined by n
/ ( A ) := ao Id + ^ k=l
n
akA^ =: ^
a^A k
k=0
if we set A^ := Id. It is easily seen that, if a polynomial f{t) factors as f{t) = p{t)q{t), then the matrices p{A) and q{A) commute, and we have fiA)=p{A)q{A)=q{A)p{A). 2.53 Proposition. Let A E Mn{C), and let p{t) be a polynomial. Then (i) if X is an eigenvalue of Ay then p{\) is an eigenvalue ofp{A), (ii) if fi is an eigenvalue of p{A), then fi = p{X) for some eigenvalue A
of A. Proof, (i) follows observing that A'^ , /c G N, is an eigenvalue of A'^ if A is an eigenvalue of A . (ii) Since ^ is an eigenvalue of p ( A ) , the matrix p{A) — /xld is singular. Let p{t) = J2i=i ^i** be of degree fc, a/e 7^ 0. By the fundamental theorem of algebra we have
2.2 Eigenvectors and Similar Matrices
67
p{t) ~ iJ, = ak n(*~'^^)' i=l
hence p(A) — / i l d = a^ H i ^ i C A — r^ Id) and, since p(A) — /xld is singular, at least one of its fa<:tors, say A — ri Id, is singular. Consequently, r i is an eigenvalue of A and trivially, p{ri) — /x = 0. D
Now consider two polynomials P{t) := J^- PjP and Q(t) := ^ ^ Qkt^ with n X n matrices as coefficients. Trivially, the product polynomial R(t) := P{t)Q{t) is given by
2.54 Lemma. Using the previous notation, if A G Mn,n{C) with the coefficients ofQ{t), then R ( A ) = P ( A ) Q ( A ) .
commutes
Proof. In fax;t,
R(A) = ^P,QfcA>+'= = 5^(P,A^)(QfeA'=) = (^P,A^)(^Q,A^)
=P(A)Q(A).
2.55 Theorem (Cayley-Hamiilton). Let A G Mn,n{C) 0,'^d let PA{S) be its characteristic polynomial, PA{S) '= det(sld —A). Then PA{A) = 0. Proof. Set Q(5) := s I d — A , s G C, and denote by cof Q(s) the matrix of cofactors of Q(s). By Laplace's formulas, see (1.22), cof Q(s) Q(s) = Q(s) cof Q(s) = det Q(s) Id =
PA{S)
Id.
Since A trivially commutes with the coefficents Id and A of Q ( s ) , Lemma 2.54 yields PA (A) = PA (A) Id = cof Q ( A ) Q ( A ) = cof Q ( A ) -0 = 0. D
b. Factorization and invariant subspaces Given two polynomials Pi,P2 with degPi > degP2, we may divide Pi by P2, i.e., uniquely decompose Pi as Pi = QP2 + R where d e g P < degP2. This allows us to define the greatest common divisor (g.c.d.) of two polynomials that is defined up to a scalar factor, and compute it by Euclid's algorithm. Moreover, since complex polynomials factor with irreducible factors of degree 1, the g.c.d. of two complex polynomials is a constant polynomial if and only if the two polynomials have no common root. We also have 2.56 Lemma. Let p{t) and q{t) be two polynomials with no common zeros. Then there exist polynomials a{t) and b{t) such that a{t)p{t) + b{t)q{t) = 1 V t e C .
68
2. Vector Spaces and Linear Maps
We refer the readers to [GM2], but for their convenience we add the proof of Lemma 2.56 Proof. Let V := (r{t) := a{t)p{t) -h b{t)q(t) I a{t), b{t) are polynomials} and let d = ap-\-/3q he the nonzero polynomial of minimum degree in V. We claim that d divides both p and q. Otherwise, dividing p by d we would get a nonzero polynomial r := p — md and, since p and d are in V, r = p — md G V also, hence a contradiction, since r has degree strictly less than d. Then we claim that the degree of d is zero. Otherwise, d would have a root that should be common to p and q since d divides both p and q. In conclusion, d is a nonzero • constant polynomial.
2.57 Proposition. For every polynomial p, the kernel of p{A) is an invariant subspace for A G Mn,n{C)Proof. Let w G kerp(A).
Since tp{t) = p{t) t, we infer Ap{A) = p{A)A.
Therefore
p ( A ) ( A w ) = (p(A) A ) w = ( A p ( A ) ) w = A p ( A ) w := AO = 0. Hence Aw G kerp(A).
•
2.58 Proposition. Let p be the product of two coprime polynomials, p{t) =Pi{t)p2{t), and let A G Mn,n(C). Then kerp(A) := kerpi(A) 0ker;?2(A). Proof By Lemma 2.56, there exist two polynomials a i , a 2 such that ai{t)pi(t) + a2(t)p2(t) = 1. Hence (2.9) ai (A)pi (A) + a2 (A)p2 (A) = Id. Set Wi := k e r p i ( A ) ,
W2 := kerp2(A),
W := kerp(A).
Now for every x G W, we have ai {A)pi ( A ) x G W2 since P 2 ( A ) a i ( A ) p i ( A ) x = P2(A)(Id - a2(A)p2(A))x = (Id - a2(A)p2(A))p2(A)x = a i ( A ) p i ( A ) p 2 ( A ) x = a i ( A ) p ( A ) x - 0. and, similarly, a2(A)p2(A)x G Wi. Thus W = Wi + W2. Finally W = WieW2fact, if y G Wi n W2, then by (2.9), we have
In
y = a i (A)pi (A)y -h 02 (A)p2 (A)y = 0 + 0 = 0.
c. Generalized eigenvectors and the spectral theorem 2.59 Definition. Let A G Mn,n(C), and let A be an eigenvalue of A of multiplicity k. We call generalized eigenvectors of A relative to the eigenvalue A the elements of W:=ker{Xld-A)^. Of course, (i) eigenvectors relative to A are generalized eigenvectors relative to A, (ii) the spaces of generalized eigenvectors are invariant subspaces for A.
2.2 Eigenvectors and Similar Matrices
69
2.60 T h e o r e m . Let A G Mn,n(C). Let Ai, A2,..., A^ be the eigenvalues of A with multiplicities m i , 7712,..., ruk and let VFi, VF2,..., W^ 6e the subspaces of the relative generalized eigenvectors, Wi := ker(AiId — A). Then (i) the spaces VFi, VF2,..., Wk are supplementary, consequently there is a basis of C^ of generalized eigenvectors of A, (ii) dimWi = mi. Consequently, if we choose A ' G Mn,n(^) using a basis (ei, 6 2 , . . . , e^) where the the first rui elements span Wi, the following m2 elements span W2 and the last m^ elements span Wk- We can then write the matrix A ' in the new basis similar to A where A' has the form 0
Ai
0 \ 0
A2
A' = 0
0
where for every i = 1,... ,k, the block Ai is ami x ^ i matrix with Xi as the only eigenvector with multiplicity mi and, of course, (A^ Id — A^)"^^ = 0. Proof, (i) Clearly the polynomials pi{s) := (Ai - s)"^i, ^2(5) := (A2 - s)"^^, . . . , Pk{s) := (Afe — s)'^f^ factorize pA and are coprime. Set N^ := p i ( A ) and notice that Wi = k e r N i . Repeatedly applying Proposition 2.58, we then get kerpA(A) = k e r ( N i N 2 . . . N ^ ) = k e r ( N i ) © ker(N2N3 • • • N ^ )
= --- =
WieW2e"-®Wk.
(i) then follows from the Cayley-Hamilton theorem, kerpA(A) = C^. (ii) It remains to show that dim Wi = rrii VI Let (ei, 62, • • •, e-n) be a basis such that the first hi elements span VTi, the following /12 elements span W2 and the last h^ elements span W^. A is therefore similar to a block matrix Ai
0
0
0
A2
0
0
0
\
A' =
Afc h
where the block A^ is a square matrix of dimension hi := dim W^. On the other hand, the Qi X Qi matrix (A^ Id — Aj)"^^ = 0 hence all the eigenvalues of Xi Id — A i are zero. Therefore A i has a unique eigenvalue Ai with multiplicity /li, and p^i (s) := (s — Xi)^^. We then have k
PA{S) = PA'is) = UPAM i=l
k
= Yl(s ~ A)^S i=l
and the uniqueness of the factorization yields hi = rrii. The rest of the claim is trivial. D
70
2. Vector Spaces and Linear Maps
Another proof of dim Wi = ruj goes as follows. First we show the following. 2.61 L e m m a . IfO is an eigenvalue ofB^ eigenvalue for B"^ with multiplicity m.
Mn,n(C) with multiplicity
m, the 0 is an
Proof. The function 1 - A'^, A € C, can be factorized as 1 - A"^ = I l i l o ^ ( ^ ' ~ ^) where (jj := e*27r/m jg ^ j.QQ^ Q£ unity (the two polynomials have the same degree and take the same values at the m roots of the unity and at 0). For z,t E C m—l
.
m —1
---*'"=.™(i - (1)-)=.- n (-^ - ^) = n (-*- - *). i=0
i=0
hence m— l
2^Id-B^ = 2^Id-B^-
J|(u;^2ld-B). i=0
If we set qo(z) := H l i ^ ^ Q{^-^z)^ we have qo{0) ^ 0, and m—l
m—l
P B - (z"^) ••= detiz"^ Id - B"^) = Yl PB(UJ'Z) = JJ (uj^z^q{uj^z) i=0
=
z'^\o{z).
i=0
(2.10) On the other hand p-B^ — s'^qi(r) for some qi with qi{0) ^ 0 and some r > 1. Thus, following (2.10) PBm(s) = s'^qi(s), i.e., 0 is an eigenvalue of multiplicity m for B*^.
•
i4noi/ier proof that dim Wj = m,i in Theorem, 2.60. Since
y] m,i = y^ dim Wi = dim X, it suffices to show that dim Wi < rui Vi. Since 0 is an eigenvalue of B := Aj Id —A of multiplicity m := r/ij, 0 is an eigenvalue of multiplicity m for B"^ by Lemma 2.61. Since Wi is the eigenspace corresponding to the eigenvalue 0 of B ' ^ , it follows from Proposition 2.43 that dim Wj < m. D
d. Jordan's canonical form 2.62 Definition. A matrix B E Mn^n^) exists k >0 such that B'^ = 0.
^s said to he nilpotent if there
Let B G Mq^q{C) be a nilpotent matrix and let k be such that B^ = 0, but gfc-i _^ Q p^^ g^ basis (ei, 6 2 , . . . , Cg) of kerB, and, for each z = 1 , . . . , s, set ej := Ci and define ef, ef,..., ef' to solve the systems Be^ '-= ^i~ for j = 2 , 3 . . . as long as possible. Let {e^}, j = 1 , . . . ,fc^,z = 1 , . . . , ^, be the family of vectors obtained this way. 2.63 Theorem (Canonical form of a nilpotent matrix). Let "B be a q X q nilpotent matrix. Using the previous notation, {e]} is a basis ofC^. Consequently, if we write B with respect to this basis, we get a qxq matrix B ' similar to B of the form
2.2 Eigenvectors eind Similar Matrices
/K
71
0 \ 0 (2.11)
B' B
V
AJ
where each block Bj has dimension ki and, if ki > 1, it has the form 0 1 0 0 0 1 B,= 0 0 0
. . . . . .
0 0 0
0 0 0
. . .
1 0
(2.12)
The reduced matrix B ' is called the canonical Jordan form of the nilpotent matrix B . Proof. The kernels Hj := ker B-^ of B-^, j = 1,... ,k, form a strictly increasing sequence of subspaces {0} = i/o C / / i C i/2 C • • • C Hk-i C Hk := C*?. The claim then follows by iteratively applying the following lemma.
D
2.64 Lemma. For j = 1,2,... ,k — l, let (ei, e 2 , . . . , Cp) be a basis of Hj and let xi, X2,..., x^ be all possible solutions of Bxj = Cj, j = 1,... ,p. Then (ei, 6 2 , . . . , ep,xi, X2,..., Xr) is a basis for Hj^\. Proof. In fact, it is easily seen that o the vectors e i , e2, • . . , Cp, x i , 0:2, • • •, Xr are linearly independent, o {ei, 6 2 , . . . , ep,a;i, X 2 , . . . , Xr} C Hj^i, o the image of Hj^i by B is contained in Hj. Thus r, which is the number of elements ei in the image of B , is the dimension of the image of i / j + i by B . The rank formula then yields dim Hj^i
= dim Hj -\- dim f Im B PI ifj + i ) =
p-\-r.
Now consider a generic matrix A G M„,n(C). We first rewrite A using a basis of generalized eigenvectors to get a new matrix A ' similar to A of the form A^ A'
0
0
...
0 \ 0
(2.13)
72
2. Vector Spaces and Linear Maps
where each block A^ has the dimension of the algebraic multiplicity rrii of the eigenvalue A^ and a unique eigenvalue A^. Moreover, the matrix Ci := A J d - Ai is nilpotent, and precisely C^' = 0 and C^"''^ •=/- 0. Applying Theorem 2.63 to each Cj, we then show that A^ is similar to \i Id + B ' where B ' is as (2.11). Therefore, we conclude the following. 2.65 Theorem (Jordan's canonical form). Lei Ai, A 2 , . . , A^; he all distinct eigenvalues of A e Mn,n(C). For every z = 1 , . . . , A: (i) let (^2,1,..., Ui^p.) be a basis of the eigenspace Vx. (as we know, pi < rii),
(ii) consider the generalized eigenvectors relative to Ai defined as follows: for any j = 1,2,...,pi, a) set ejj := Uij, b) set efj to be a solution of a-l
iA-Xild)efj=e
(2.14)
a
as long as the system (2.14) is solvable, c) denote by OL{i,j) the number of solved systems plus 1. Then for every i = 1,... ,k the list (efj) with j = 1,... ,pi and a = 1,..., a(i,j) is a basis for the generalized eigenspace Wi relative to \i. Hence the full list Kj)
i = l....,kj
= l,...,pi,a
is a basis ofC^. By the definition of the {efj}, S:=
(2.15)
= l,...,a(z,j) if we set
1 2 1 2 1 2 ^ l , l 5 ^ I , l 5 • • • 5 ^ 1 , 2 ' ^1,25 • • • 5 ^ 2 , 1 ' ^ 2 , 1 ' • • •
the matrix J := S ^AS, that represents x —> A x in the basis (2.15), has the form
A
j
=
'1,1
0
0
0
0
Ji,)
0
0
0
0
\
'1,2
0
0
'k,pk
\ where i = 1,... ,k, j = 1,... ,pi, 3ij has dimension a{i,j)
and
2.2 Eigenvectors and Similar Matrices
73
if dim 3ij = 1
1
0
0
o\
0
\i
1
0
0
0
0
\i
1
0
0
0
0
...
\i
0
0
...
0
(\ ^iJ ~~ \
\ Q
otherwise.
1 \)
A basis with the properties of the basis in (2.15) is called a Jordan basis^ and the matrix J that represents A in a Jordan basis is called a canonical Jordan form of A. 2.66 E x a m p l e . Find a canonical Jordan form of ^ 2 0 0 0 0 1 2 0 0 A = 0 1 2 0 0 0 1 3
Vl
0
0
A is lower triangular hence the eigenvalues of multiplictiy 2. We then have /o 0 1 0 A-2Id = 0 1 0 0 Vl 0
^ 0 0 0
1
3/
A are 2 with multiplicity 3 and 3 with
0 0 0 1 0
0 0 0 1 1
o\ 0 0 0 1/
A — 2Id has rank 4 since the columns of A of indices 1, 2, 3 and 5 are linearly independent. Therefore the eigenspace V2 has dimension 5 — 4 == 1 by the rank formula. We now compute a nonzero eigenvalue,
/
(x\ y ( A - 2 Id)
0
z t
\
/0\ 0 0 0
y z-\-t
Voy For instance, one eigenvector is si := ( 0 , 0 , 1 , - 1 , 1 ) ^ . We now compute the Jordan basis relative to this eigenvalue. We have e\ -^ = s\ and it is possible to solve
\
/ 0 \ 0 1
z+t \x-ht + uj
-1
/
0 X
y
for instance, S2
^1,1
Vl/
(0,1,0, —1, 2 ) ^ is a solution. Hence we compute a solution of
74
2. Vector Spaces and Linear Maps
(
0
\ 1 0 -1
X
y z-ht \x + t-\-u/
\2j
hence S3 := ef ^^ = (1,0, 0, - 1 , 2 ) ^ . Looking now at the other eigenvalue,
A-3Id =
0 0 -1 1 0
0 -1 1 0 0
(''1 0 0
Vi
o\
0 0 0 0 1
0 0 0 0/
A is of rank 4 since the columns of indices 1, 2, 3 and 4 are linearly independent. Thus by the rank formula, the eigenspace relative to the eigenvalue 2 has dimension 1. We now compute an eigenvector with eigenvalue 2. We need to solve /x\
/
( A - 3 Id)
- x \
t
-y y- z z
\u)
\x + t/
y z
=
('\ 0 0 0
=
\oJ
and a nonzero solution is, for instance, 54 := (0,0,0, 0,1)-^. Finally, we compute Jordan's basis relative to this eigenvalue. A solution of /
- x \ 0 0 0
-y
y-z z
\x + t) is given by S5 = e^ ^ = (0, 0 , 0 , 1 , 0)-^. Thus, we conclude that the matrix
/o S = Ul
S2 S3 S4 S5
0 1 -1
=
Vi
0 1 0 -1 2
1 0 0 -1 2
0 0 0 0 1
o\ 0 0 1 0/
is nonsingular, since the columns are linearly independent, and by construction /2 0 S-^AS^: 0 0
1 2 0 0
0 0 1 0 2 0 0 3
0\ 0 0 1
Vo 0 0 0 3/
2.2 Eigenvectors and Similar Matrices
75
e. Elementary divisors As we have seen, the characteristic polynomial det(5ld-A),
seK,
is invariant by similarity transformations. However, in general the equality of two characteristic polynomials does not imply that the two matrices be similar.
2.67 E x a m p l e . The unique eigenvalue of the matrix A ^ = I
V /i
I is AQ and has
XoJ
multiplicity 2. The corresponding eigenspace is given by the solutions of the system
{
O'x'^ +0'X^
=0,
fjLX^ + 0 • X ^ = 0 .
If /x 7^ 0, then VXO,M ^ ^ dimension 1. Notice that AQ is diagonal, while A^ is not diagonal. Moreover, AQ and A^ with fi ^ 0 are not similar.
It would be interesting to find a complete set of invariants that characterizes the class of similarity of a matrix, without going explictly into Jordan's reduction algorithm. Here we mention a few results in this direction. Let A e Mn,n(C). The determinants of the minors of order k of the matrix 5 Id — A form a subset T>k of polynomials in the s variable. Denote by Dk{s) the g.c.d. of these polynomials whose coefiicient of the maximal degree term is normalized to 1. Moreover set Do{s) := 1. Using Laplace's formula, one sees that Dk-i{s) divides Dk{s) for all k = l , . . . , n . The polynomiSfe
are called the elementary divisors of A. They form a complete set of invariants that describe the complex similarity class of A. In fact, the following holds. 2.68 Theorem. The following claims are equivalent (i) A and B are similar as complex matrices, (ii) A and B have the same Jordan's canonical form (up to permutations of rows and columns), (iii) A and B have the same elementary divisors.
76
2. Vector Spxaces and Linear Maps
2.3 Exercises 2.69 f. Write a few 3 x 3 real matrices and interpret them as linear maps from M^ into E^. For each of these linear maps, choose a new basis of R^ and write the associate matrix with respect to the new basis both in the source and the target R^. 2.70 %. Let Vi, V 2 , . . . , Vri be finite-dimensional vector spaces, and let / o , / i , - • • ? / n be linear maps such that {0}:^yiAy2^
. . . ^ ^ ' V n _ / ^ ' Vn ^ {0}.
Show that, if I m ( / i ) = ker(/i+i) Vi = 0 , . . . , n - 1, then E ? = i ( - 1 ) ' d i m Vi = 0 . 2.71 f. Consider R as a vector space over Q. Show that 1 and ( are linearly independent if and only if ^ is irrational, ? ^ Q. Give reasons to support that R as a vector space over Q is not finite dimensional. 2.72 ^ L a g r a n g e m u l t i p l i e r s . Let X, Y and Z be three vector spaces over K and let f : X ^>'Y, g : X —^ Z he two linear maps. Show that ker p C ker / if and only if there exists a linear map £ : Z -^ Y such that / := io g. 2.73 f.
Show that the matrices
;:)• 0°)' have the same eigenvalues but are not similar. 2.74 ^ . Let Ai, A 2 , . . . , An be the eigenvalues of A € Mn,n(C), possibly repeated with their multiplicities. Show that tr A = Ai + • • -f An and det A = Ai • A2 • • • An. 2.75 %. Show that p{s) = s'^ -\- an-is^~^ the n X n matrix /
H
0 0
1 0
0 1
-ao
—ai
—a2
2.76 %, Let A G Mfc,fc(K), B € Mn,n{^), polynomial of the matrix
h ao is the characteristic polynomial of ... ...
0 0
\
-fln-i
/
C 6 Mk,n{^)-
VO
Compute the characteristic
B
2.77 % L e t ^ r C ^ -^ C^ be defined by ^(ei) := ei^i if i = 1 , . . . , n - l and ^(en) = e i , where e i , e2, • . . , en is the canonical basis of C^. Show that the associated matrix L is diagonizable and that the eigenvalues are all distinct. [Hint: Compute t h e characteristic polynomial.]
2.3 Exercises
2.78 %. Let A 6 Mn,n{^)
77
and suppose A^ = Id. Show that A is similar to
/T
for some k, 1 < k < n. [Hint: Consider the subspeices V+ := {a^ I A x = x } and V- := {x I A x = - x } and show that V+ 0 y_ = R^. ] 2.79 i[. Let A, B G Mn,nW be two matrices such that A^ = B ^ = Id and tr A = tr B . Show that A and B are similar. [Hint: Use Exercise 2.78.] 2.80 f. Show that the diagonizable matrices span Mn,n{^)- [Hint: Consider the matrices Mij = diag (1, 2 , . . . , n ) + 'Eij where Eij has value 1 at entry (i, j ) and value zero otherwise.] 2.81 %. Let A , B e Mn,n(^) and let B be symmetric. Show that the polynomial t -^ det(A -I- t B ) has degree less than R a n k B . 2.82 %, Show that any linear operator A : W^ dimension 1 or 2.
V^ has an invariant subspace of
2.83 f F i t t i n g d e c o m p o s i t i o n . Let / : X —>• X be a linear operator of a finitedimensional vector space and set f^ := / o • • • o / /c-times. Show that there exists k, 1 < k
ker(/'=) = ker(/'=+i), Im(/'=) = Im (/«=+!), /|iin(/fe) • I n i ( / ^ ) -^ Im(f^) /(ker/*=)cker(/^), /|ker(/fc) • ker(/'^) -^ kerif^)
is an isomorphism, is nilpotent,
(vi) y = k e r ( / ' = ) e l m ( / ' = ) . 2.84 ^ . A is nilpotent if and only if all its eigenvalues are zero. 2.85 %, Consider the linear operators in the linear space of polynomials B(P){t) = tP(t). A(P){t) := P'{t), Compute the operator AB — BA. 2.86 f. Let A,B
be linear operators on R^. Show that
(i) tT(AB)=tv{BA), (ii) AB-BA:^ Id. 2.87 t . Show that a linear operator C : R^ -> R^ can be written as C = AB where A,B : R-^ -^ R-^ are linear operators if and only if tr C = 0.
BA
78
2. Vector Spaces and Linear Maps
2.88 %. Show that the Jordan canonical form of the matrix fa al al 0 a al 0 0 a \0
0
4\ r.2
0
a)
with alai ...an ^ # 0 : /a 1 0 0 o 1 0 0 a
0\ 0 0
\0
a)
0 0
3- Euclidean and Hermitian Spaces
3.1 The Geometry of Euclidean and Hermitian Spaces Until now we have introduced several different languages, linear independence, matrices and products, linear maps that are connected in several ways to linear systems and stated some results. The structure we used is essentially hnearity. A new structure, the inner product, provides a richer framework that we shall illustrate in this chapter. a. Euclidean spaces 3.1 Definition. Let X be a real vector space. An inner product on X is a map ( | ) : X x X —^W which is o
(BILINEAR) (X,2/)
—> {x\y) is linear in each factor, i.e., {Xx + fiy\z) = X{x\z) + fi{y\z), {x\Xy -h /iz) = X{x\y) + fi{x\z),
for all x,y,z
e X, for all A, // G M.
o (SYMMETRIC) {x\y) = {y\x) o (POSITIVE DEFINITE) {X\X)
for
> 0
all x,y e X. VX and {x\x)
=0 if and only if x = 0.
The nonnegative real number \x\ := y/{x\x) is called the norm of x e X. A finite-dimensional vector space X with an inner product is called an Euclidean vector space, and the inner product of X is called the scalar product of X. 3.2 Example. The map ( | ) : R^ x E'^ -^ R defined by (x|y) := x . y = f ^ ^ V ,
x := {x\ x^,..., x^), y := {y\ y\ ... ^ y^)
80
3. Euclidean and Hermitian Spaces
is an inner product on W^, called the standard scalar product of R^, and W^ with this scalar product is an Euclidean space. In some sense, as we shall see later, see Proposition 3.25, it is the unique Euclidean space of dimension n. Other examples of inner products on M^ can be obtained by weighing the coordinates by nonnegative real numbers. Let Ai, A2, •. •, An be positive real numbers. Then (x|y) •.= ^ \ i x Y ,
x = ( x i , x 2 , . . . , x " ) , y = {y\
y\...,y")
i=l
is an inner product on R^. Other examples of inner products in infinite-dimensional vector spaces can be found in Chapter 10.
Let X be a vector space with an inner product. Prom the bihnearity of the inner product we deduce that \x + 2/p = {x + y\x + y) = {x\x + t/) + {y\x + y) = {x\x) + 2{x\y) + {y\y) = \x\^ + 2{x\y) + l^l^
(3.1)
from which we infer the following. 3.3 Theorem. The following hold. (i)
(PARALLELOGRAM IDENTITY)
\x + y\^ + \x-yf (ii)
( P O L A R I T Y FORMULA)
We have
= 2 (|x|2 + \yf)
Vx,
yeX.
We have
{x\y) = 4 ( k + 2/P - k - y\^)
Vx, yeX,
hence we can get the scalar product of x and y by computing two norms. (iii) (CAUCHY-SCHWARZ INEQUALITY) The following inequality holds \{x\y)\<\x\
\yl
VX,7/GX;
moreover^ {x\y) = \x\\y\ if and only if either y = 0 or x = Xy for some A G M, A > 0. Proof, (i), (ii) follow trivially from (3.1). Let us prove (iii). If y = 0, the claim is trivial. If 2/ 7^ 0, the function < -^ |x -f i y p , i 6 E, is a second order nonnegative polynomial since 0<\x-\-
ty\'^ = ix + ty\x + ty) = {x-\- ty\x) -f (x + ty\ty) = \x\'^ + 2{x\y) t + |y|^ *^;
hence its discriminant is nonpositive, thus ((x\y))'^ — |xp|2/p < 0. If {x\y) = \x\ \y\, then the discriminant of t —>• |x -f ty\^ vanishes. If t/ 7^ 0, then for some t G M we have \x + *2/P = 0, i.e., x = —ty. Finally, —t is nonnegative since D -t{y\y) =: ix\y) = \x\ \y\ > 0.
3.4 Definition. Let X be a vector space with an inner product. Two vectors x,y G X are said to be orthogonal, and we write x Ly, if {x\y) = 0.
3.1 The Geometry of Euclidean and Hermitian Spaces
81
Prom (3.1) we immediately infer the following. 3.5 Proposition (Pythagorean theorem). Let X be a vector space with an inner product. Then two vectors x^y G X are orthogonal if and only if I
I
|2
I
|2
, I
|2
1^ + 2/1 = m + \y\ ' 3.6 Carnot's formula. Let x , y G M^ be two nonzero vectors of R^, that we think of as the plane of Euclidean geometry with an orthogonal Cartesian reference. Setting x := (a, 6), y := (c, d), and denoting by 6 the angle between Ox and Oy, it is easy to see that |x|, |y| are the lengths of the two segments Ox and Oy, and that x « y := ac-\- bd = |x| |y| cos^. Thus (3.1) reads as Carnot's formula |x + y p = |xp -h |yp H- 2|x| |y| cos^. In general, given two vectors x , y G R", we have by Cauchy-Schwarz inequality | x » y | < | x | | y | , hence there exists a ^ G R such that x»y
m \y\
. =: cos^.
6 is called the angle between x and y and denoted by xy. In this way (3.1) rewrites as Carnot 's formula |x -f y p = |xp + |yp + 2|x| |y| cosl9. Notice that the angle 6 is defined up to the sign, since cos^ is an even function. 3.7 Proposition. Let X be a Euclidean vector space and let { \ ) be its inner product. The norm of x G X , |x| :=
y/{x\x)
is a function \ | : X -^ R with the following properties (i) \x\ G R4. Vx G X. (ii) (NONDEGENERACY) (iii)
(iv)
\X\ =0 if and only ifx = 0. (1-HOMOGENEITY) |AX| = |A| |x| VA G R, Vx G X. (TRIANGULAR INEQUALITY) \x-\-y\ < \x\ + \y\ \/x,y
G X.
Proof, (i), (ii), (iii) are trivial (iv) follows from the Cauchy-Schwarz inequality since \x + 2/|2 = |x|2 + |t/|2 + 2{y\x) < |x|2 + |y|2 + 2 \{y\x)\
<|xp + |2/|2+2|x|M = (N + M)2). D
Finally, we call the distance between x and y G X the number d{x,y) := \x — y\. It is trivial to check, using Proposition 3.7, that the distance function d : X x X ^^ R defined by d{x,y) := \x — y\^ has the following properties
82
3. Euclidean and Hermitian Spaces
(i)
(NONDEGENERACY)
d(x, y) >0^x,y
e X and d{x, y) = 0 ii and only
ii X = y. (ii) (SYMMETRY) d{x,y) = d{y,x) Vx,?/ € X. (iii) (TRIANGULAR INEQUALITY) d{x,y) < d(x,z) + d{z,y) Vx,y,z e X.
We refer to d as the distance in X induced by the inner product. 3.8 Inner products in coordinates. Let X be a Euclidean space, denote by ( I ) its inner product, and let (ei, 6 2 , . . . , e^) be a basis of X. If ^ = E l L i ^'^i^ y = E I L i y'^i ^ ^i then by linearity {x\y) =
Y^x'y^{ei\ej).
The matrix G = [gij],
Qij = {ei\ej)
is called the Gram matrix of the scalar product in the basis (ei, eg, • • •, ^n)Introducing the coordinate column vectors x = (x^, x ^ , . . . , x'^)'^ and y = (2/^, y ^ , . . . , 2/"^)^ G R"^ and denoting by • • • the standard scalar product in R"^, we have {x\y) = x . G y = x^Gy rows by columns. We notice that (i) G is symmetric, G^ — G, since the scalar product is symmetric, (ii) G is positive definite, i.e., x^Gx > 0 Vx G R"' and x^Gx = 0 if and only if X — 0, in particular, G is invertible. b. Hermit ian spaces A similar structure exists on complex vector spaces. 3.9 Definition. Let X be a vector space over C. A Hermitian product on X is a map {\):XxX-^C which is (i) (SESQUILINEAR),
i.e.,
{av -h (iw\z) = a{v\z) + l3{w\z), {v\aw -f /3z) = a{v\w) -h l3{v\z) (ii) (iii)
(HERMITIAN) {Z\UJ)
yzex.
= (wlz) "iw.z £ X, in particular {z\z) G \
(POSITIVE DEFINITE) (Z\Z)
> 0 and {z\z) = 0 if and only if z = 0.
The nonnegative real number \z\ :=^ y^{z\z) is called the norm of z E X. 3.10 Definition. A finite-dimensional complex space with a Hermitian product is called a Hermitian space.
3.1 The Geometry of Euclidean and Hermitian Spaces
83
3.11 E x a m p l e . Of course the product (z,w) -^ {z\w) := wz is a Hermitian product on C. More generally, the map ( | ) : C^ x C " —^ C defined by n
{z\w) := z » w := ^z^w^ J=i
Mz = (z^, z^,...,
^ " ) , w = {w'^, w'^,...,
w'^)
is a Hermitian product on C^, called the standard Hermitian product of C^. As we shall see later, see Proposition 3.25, C^ equipped with the standard Hermitian product is in a sense the only Hermitian space of dimension n.
Let X be a complex vector space with a Hermitian product ( | ). Prom the properties of the Hermitian product we deduce \z -h w\'^ = (z -{• w\z -\-w) = (z\z -h K;) + (w\z -\-w) I
I
\
I
/
V I
/
\
I
, (3.2)
/
= {z\z) + {z\w) + {w\z) -h {w\w) = \z\'^ -h \w\'^ + 2di{z\w) from which we infer at once the following. 3.12 T h e o r e m ,
(ii)
(i) We have
(PARALLELOGRAM IDENTITY)
\z + w\'^ -\-\z-wf (iii)
( P O L A R I T Y FORMULA)
We have
= 2 (|zp + \wf)
\/z, w
eX.
We have
— iw\ 4:{z\w) =: (\z -\- w\'^ — \z — w;p 1 4- if 1^: -h iw\'^ — \z 2^ for all z^w G X. We therefore can compute the Hermitian product of z and w by computing four norms. (iv) (CAUCHY-SCHWARZ INEQUALITY) The following inequality holds \{z\w)\ < \z\ \wl
yz.weX;
moreover {z\w) = \z\ \w\ if and only if either w = 0, or z = Xw for some A G M, A > 0. Proof, (i), (ii), (iii) follow trivially from (3.2). Let us prove (iv). Let z, w E X and A = te*^, t,e eR. From (3.2) 0 < |z + \w\'^ = t'^\w\'^ + \z\^ + 2t^{e-'yz\w))
Vt G M,
hence its discriminant is nonpositive, thus me-'»{z\w))\
< \z\ \w\.
Since 0 is arbitrary, we conclude |(2;|ii;)| < \z\ \w\. The second part of the claim then follows as in the real case. If {z\w) = \z\ \w\, then the discriminant of the real polynomial t —> |2 + ttyp, t e R, vanishes. If -w; ^i^ 0, for some t G M we have \z + tw\'^ = 0, i.e., z = —tw. Finally, —t is nonnegative since —t{w\w) = {z\w) = \z\ \w\ > 0. D
,
84
3. Euclidean and Hermitian Spaces
3.13 i[. Let A" be a complex vector space with a Hermitian product and let z^w ^ X. Show that K^^lif)! = \z\ \w\ if and only if either it; = 0 or there exists A 6 C such that z = \w.
3.14 Definition. Let X be a complex vector space with a Hermitian product ( I ). Two vectors z^w e X are said to be orthogonal, and we write z 1.W, if {z\w) = 0. Prom (3.2) we immediately infer the following. 3.15 P r o p o s i t i o n ( P y t h a g o r e a n t h e o r e m ) . Let X be a complex vector space with a Hermitian product ( | ). If z^w E X are orthogonal, then
We see here a diiference between the real and the complex cases. Contrary to the real case, two complex vectors, such that |z + i(;p = |2:p + |if;p holds, need not be orthogonal. For instance, choose X := C, {z\w) := wz, and let z = 1 and w = i. 3.16 P r o p o s i t i o n . Let X be a complex vector space with a Hermitian product on it. The norm of z e X, \z\ :=
^/(z\z),
is a real-valued function \ \ : X —^R with the following properties (i) \z\ G R+ Vz G X. (ii) (NONDEGENERACY) (iii) (iv)
\Z\=0 if and only if z = 0. (1-HOMOGENEITY) \XZ\ = |A| \z\ VA G C, Vz G X. (TRIANGULAR INEQUALITY) \Z-\-W\ < \z\ + \w\ \/z,w
G X.
Proof, (i), (ii), (iii) are trivial, (iv) follows from the Cauchy-Schwarz inequality since \z + w\^ = \z\^ -f k | 2 + 2^(z\w)
< \z\^ + |i/;|2 + 2 \(z\w)\
<\z\^ + \w\^ + 2\z\\w\
= (\z\ +
\w\n D
Finally, we call distance between two points z^w oi X the real number d{z,w) := \z — w\. It is trivial to check, using Proposition 3.16, that the distance function d : X x X ^^ R defined by d{z,w) := \z — w\ has the following properties (i)
(NONDEGENERACY) d{z,w) > 0 yz,w e X and d{z,w) = 0 if and only ii z = w. (ii) (SYMMETRY) d{z,w) = d{w,z) ^z.w G X. (iii)
(TRIANGULAR INEQUALITY) d{z, w) < d(z, x) + d{x, w) Ww, x,z
e
X.
We refer to d as to the distance on X induced by the Hermitian product.
3.1 The Geometry of Euclidean and Hermitian Spaces
85
3.17 Hermitian products in coordinates. If X is a Hermitian space, the Gram matrix associated to the Hermitian product is defined by setting G=
[gij],
9ij :=
{ei\ej).
Using Hnearity {z\w) = Y2 {ei\ej)z'w^ = z^Gw ',3 = 1
if z = (z^, z ^ , . . . , z'^), w = (it;^, w'^,..., w'^) G C^ are the coordinate vector columns of z and w in the basis (ei, e 2 , . . . , Cn)- Notice that (i) G is a Hermitian matrix, G = G, (ii) G is positive definite, i.e., z^Gz > 0 Vz G C^ and z^Gz = 0 if and only if z = 0, in particular, G is invertible. c. Orthonormal basis and the Gram—Schmidt algorithm 3.18 Definition. Let X be a Euclidean space with scalar product { \ ) or a Hermitian vector space with Hermitian product ( | ). ^4 system of vectors {^a}aeA ^ ^ '^s called orthonormal if iea\ep) =Sap
Va,/3 G A
Orthonormal vectors are hnearly independent. In particular, n orthonormal vectors in a Euclidean or Hermitian vector space of dimension n form a basis, called an orthonormal basis. 3.19 E x a m p l e . The canonical basis ( e i , e 2 , . . . , Cn) of E"^ is an orthonormal basis for the standard inner product in E " . Similarly, the canonical basis ( e i , 6 2 , . . . , en) of C^ is an orthonormal basis for the standard Hermitian product in C"^. 3.20 %. Let ( I ) be an inner (Hermitian) product on a Euclidean (Hermitian) space X of dimension n and let G be the associated Gram matrix in a basis (ei, 6 2 , . . . , en)Show that G = Idn if and only if (ei, e2,. •., en) is orthonormal.
Starting from a denumerable system of linearly independent vectors, we can construct a new denumerable system of orthonormal vectors that span the same subspaces by means of the Gram-Schmidt algorithm, 3.21 Theorem (Gram-Schmidt). Let X be a real (complex) vector space with inner (Hermitian) product ( | ). Let t'l, t;2,..., t'jt,... be a denumerable set of linearly independent vectors in X. Then there exist a set of orthonormal vectors wi, W2,. - -, Wk,- • - such that for each fc = 1,2,... Span|it;i, W2,..., wA = Spanjt'i, t'2,.--, Vkj.
86
3. Euclidean and Hermitian Spaces
Proof. We proceed by induction. In fact, the algorithm W[ = VI,
wi := -—f-, '^p='^pYl^j=lMwj)wj p -P A^j= w Wp:=
—-
never stops since Wp ^ 0 "ip = 1,2,3,...
and produces the claimed orthonormal basis. D
3.22 Proposition (Pythagorean theorem). LetX be a real (complex) vector space with inner (Hermitian) product ( | ). Let (ei, e2, • . . , e^) be an orthonormal basis of X. Then k X =
Y^{x\ej)ej
xeX,
2=1
that is the ith coordinate of x in the basis (ei, e2,. •., Cn) is the cosine director {x\ei) of x with respect to ei. Therefore we compute k
{x\y) = Y^(x|ei) {y\ei)
if X is Euclidean^
i=l k
{x\y) = 2_\{^\^i) {y\^i)
^ / ^ ^-^ Hermitian,
i=l
so that in both cases Pythagoras's theorem holds: k
\x\' = {x\x) = Y;^\ix\ei)\-'. i=l
Proof. In fact, by linearity, for j = 1 , . . . , A; and x — ^Y^=\ ^^^i ^® have n
n
n
i=l
i=\
i=\
Similarly, using linearity and assuming X is Hermitian, we have {x\y) = (Y^x'ei
I (jZv^ej)
i=l n
= f^ i,3 = ^
j=\ k
hence, by the first part, n
{x\y) =
^{x\ei){y\ei).
x*^(e,|e^)
3.1 The Geometry of Euclidean and Hermitian Spaces
87
d. Isometries 3.23 Definition. Let X^Y he two real (complex) vector spaces with inner (Hermitian) products ( | )x and ( | )y. We say that a linear map A : X -^ Y is an isometry if and only if \A{X)\Y
= \x\x
Vx e
X,
or, equivalently, compare the polar formula, if {A{x)\A{y))Y
= {x\y)x
^x,yeX.
Isometries are trivially injective, but not surjective. If there exists a surjective isometry between two Euclidean (Hermitian) spaces, then X and Y are said to be isometric. 3.24 %. Let X,Y be two real (complex) vector spaces with inner (Hermitian) products { \ )x and { \ )Y and let A : X —>• V be a linear map. Show that the following claims are equivalent (i) A is an isometry, (ii) B C X is am orthonormal basis if and only if A{B) is an orthonormal basis for A{X).
Let X be a real vector space with inner product ( | ) or a complex vector space with Hermitian product ( | ). Let (ei, e 2 , . . . , e^) be a basis in X and f : X -^ K^, (K = R of K = C) be the corresponding system of coordinates. Proposition 3.22 implies that the following claims are equivalent. (i) (ei, 6 2 , . . . , Cn) is an orthonormal basis, (ii) £{x) = ((x|ei),...,(x|en)), (iii) £ is an isometry between X and the Euclidean space W^ with the standard scalar product (or C^ with the standard Hermitian product). In this way, the Gram-Schmidt algorithm yields the following. 3.25 Proposition. Let X be a real vector space with inner product ( | ) (or a complex vector space with Hermitian product { \ )) of dimension n. Then X is isometric to R^ with the standard scalar product (respectively, to C^ with the standard Hermitian product), the isometry being the coordinate system associated to an orthonormal basis. In other words, using an orthonormal basis on X is the same as identifying X with R"^ (or with C") with the canonical inner (Hermitian) product. 3.26 I s o m e t r i e s in c o o r d i n a t e s . Let us compute the matrix associated to an isometry R : X -^ Y between two Euclidean spaces of dimension n and m respectively, in an orthonormal basis (so that X and Y are respectively isometric to R^ (C^) and W^ ( C ^ ) by means of the associated coordinate system). It is therefore sufficient to discuss real isometries i? : E^ -^ E"^ and complex isometries RiC^ -^C^. Let i? : E^ —)• E ^ be linear and let R € Mm,nW be the associated matrix, il(x) = R x , X G E"^. Denoting by ( e i , e 2 , . . •, en) the canonical basis of E " ,
3. Euclidean and Hermitian Spaces
R =
ri
r2
...
Fn ,
Ti = R e j Vi.
Since (ei, e 2 , . . . , Cn) is orthonormal, R is an isometry if and only if ( r i , rg, • •., Tn) are orthonormal. In particular, m > n and Tj Ti = Ti* Tj = 5ij
i.e., the matrix R is an orthogonal matrix^ R ^ R = Idn. When m — n, the isometries i l : R'^ —» R'^ are necessarily surjective being injective, and form a group under composition. As above, we deduce that the group of isometries of R'^ is isomorphic to the orthogonal group 0(n) defined by 0{n) := | R e Mn,n(R) | R ^ R = I d n } . Observe that a square orthogonal matrix R is invertible with R~^ = R-^. If follows that R R ^ = Id and | det R | = 1. Similarly, consider C^ as a Hermit ian space with the standard Hermitian product. Let R:C -^C^ he linear and let R € Mm,n(C) be such that R{z) = R z . Denoting by ( e i , e 2 , . . . , Cn) the canonical basis of R^, R =
r i r2
...
Fn ,
Ti = R e i Vi = 1 , . . . , m.
Since ( e i , e 2 , . . . , en) is orthonormal, R is an isometry if and only if r i , r 2 , . . •, rn are orthonormal. In particular, m > n and
i.e., the matrix R is a unitary
matrix, R^R=
Idn.
When 171 = 71, the isometries R : C^ -^ C^ are necessarily surjective being injective, moreover they form a group under composition. From the above, we deduce that the group of isometries of C^ is isomorphic to the unitary group U(n) defined by U{n) := | R € Mn,n(C) | R ^ R = I d n } . Observe that a square unitary matrix R is invertible with R R R ^ = Id and | det R | = 1.
^= R
. I t follows that
e. The projection theorem Let X be a real (complex) vector space with inner (Hermitian) product ( I ) that is not necessarily finite dimensional, let F C X be a finitedimensional linear subspace of X of dimension k and let (ei, 6 2 , . . . , e^) be an orthonormal basis of V. We say that x G X is orthogonal to V if {x\v) = 0 Vz; G 1^. As (ei, e 2 , . . . , ek) is a basis of V, x 1.V if andonly if (x|ei) = 0 Vi = 1 , . . . ,fc. For all a; G X, the vector Py{x) :=^{x\ei)ei i=i
eV
3.1 The Geometry of Euclidean and Hermitian Spaces
89
is called the orthogonal projection of x in F , and the map Py : X -^ V, X —> Pv{x), the projection map onto V. By Proposition 3.22, Py(x) = x if X G F , hence ImP = V and P^ = P. By Proposition 3.22 we also have |Py(x)p = Zli=i l(^ki)P- The next theorem explains the name for Pv{x) and shows that in fact Pv{x) is well defined as it does not depend on the chosen basis (ei, e2, • •., e/c). 3.27 Theorem (of orthogonal projection). With the previous notation, there exists a unique z G V such that x — z is orthogonal to V, i.e., {x — z\v) = 0 \fv e V. Moreover, the following claims are equivalent. (i) X — z is orthogonal to V, i.e., {x — z\v) = 0 ^v e V, (ii) z GV is the orthogonal projection of x onto V, z = Pv{x), (iii) z is the point in V of minimum distance from x, i.e., \x — z\ < \x — v\
Mv
GV^
V ^ z.
In particular, Pv{x) is well defined as it does not depend on the chosen orthonormal basis and there is a unique minimizer of the function v -^ \x — v\, V e V, the vector z = Pv{x). Proof. We first prove uniqueness. U zi,Z2 £V are such that (x — Zi\v) = 0 ioT i = I, 2, then {zi — Z2\v) = 0 Vt; G V, in particular \zi — Z2\'^ = 0. (i) => (ii). From (i) we have {x\ei) = {z\ei) Vi = 1 , . . . , fc. By Proposition 3.22 k
k
z = Y^{z\ei)ei
= '^{x\ei)ei
=
Pv{x).
1=1
i=i
This also shows existence of a point z such that x — z is orthogonal to V and that the definition of Pv{x) is independent of the chosen orthonormal basis (ei, 6 2 , . . . , e^). (ii) =^ (i). Ii z = Py{x),
we have for every j = 1,...
,k
k
{x - z\ej) = {x\ej) - '^{x\ei)(ei\ej)
= {x\ej) - {x\ej) = 0,
i=l
hence (x — z\v) = 0 Vi'. (i) => (iii). Let v EV.
Since {x — z\v) = 0 v/e have
\x-v\'^
= \x-
z-\- z-vl"^
= \x-
z\'^ -\-\z-
i;p,
hence (iii). (iii) =^ (i). Let v e V. The function t ^y \x — z + t v p , t G M, has a minimum point at t = 0. Since \x- z + tvl"^ = \x - z\'^ + 2t^{x - z\v) -\- t'^\v\'^, necessarily 3f?(x — z\v) = 0. If X is a real vector space, this means {x — z\v) = 0, hence (i). If X is a complex vector space, from R{x — z\v) = 0 \/v e V, we also have 3f?(e-^^(x - z\v)) = 0 V(9 € M Vv G V, hence {x - z\v) = 0 Vi; € V and thus (ii). D
We can discuss linear independence in terms of an orthogonal projection. In fact, for any finite-dimensional space V C X, x e V ii and only if X — PY{X) = 0, equivalently, the equation x — Pv{x) = 0 is an implicit equation that characterizes V as the kernel of Id- Py.
90
3. Euclidean and Hermitian Spaces
3.28 %, Let W = Span { v i , V 2 , . . . , v^} be a subspace of K^. Describe a procedure that uses the orthogonal projection theorem to find a basis of W. 3.29 %, Given A G Mm,n(^), describe a procedure that uses the orthogonal projection theorem in order to select a maximal system of independent rows and columns of A. 3.30 %. Let A G Mm^nC^)- Describe a procedure to find a basis of ker A. 3.31 ^. Given k linear independent vectors, choose among the vectors (ei, e2, • . . , en) of R"" (n — k) vectors that together with v i , V 2 , . . . , v^ form a basis of R"^. 3.32 P r o j e c t i o n s in c o o r d i n a t e s . Let X be a Euclidean (Hermitian) space of dimension n and let F C X be a subspace of dimension k. Let us compute the matrix associated to the orthogonal projection operator Py : X -^ X in an orthonormal basis. Of course, it suffices to think of Py as of the orthogonal projection on a subspace of R^ (C^ in the complex case). Let (ei, e 2 , . . . , en) be the canonical basis of R^ and V C R^. Let v i , V 2 , . . . , v^ be an orthonormal basis of V and denote by V = [vM the n x k nonsingular matrix
V : = ^vi I V2 I ... I VfcJ so that Vj = Z ^ i L i ^ j e j - Let P be the n x n matrix associated to the orthogonal projection onto V, Py(x) = P x , or, Pi = P e i , z = l , . . . , n .
' = [pi |P2 I ••• | p n j , Then Pi = Py{ei)
= ^{ei.Wj)wj
=
Y^v]wj
3=1
j=l
j=lh=l
h=l
(3.3)
I.e.,
P = VV^. The complex case is similar. With the same notation, instead of (3.3) we have k
Pi = Py{ei)
k
= ^{ei
.Vj )wj = ^
v^Wj
J=l
3=1
3=1h=l
h=l
(3.4)
i.e., P = VV^.
f. Orthogonal subspaces Let X be a real vector space with inner product ( | ) or a complex vector space with Hermitian product ( | ). Suppose X is finite dimensional and let W^ C X be a linear subspace of X. The subset ly-^ := {x G X I {x\y) is called the orthogonal of W in X.
=OyyeW^
3.1 The Geometry of Euclidean and Hermitian Spaces
91
3.33 Proposition. We have (i) W-^ is a linear subspace of Xj
(ii) wnw-^ = {o}, (iii) (W^)^ = W, (iv) W and W-^ are supplementary, hence dim W + dim W-^ = n, (v) if Pw dnd Pw^ tt^^ respectively, the orthogonal projections onto W and W-^ seen as linear maps from X into itself, then Pw^ = ^^x — PwProof. We prove (iv) and leave the rest to the reader. Let {vi, V2, •.., Vk) he a basis of W. Then we can complete (vi, V2,... •, v^) with n — k vectors of the canonical basis to get a new basis of X. Then the Gram-Schmidt procedure yields an orthonormal basis {wi, W2^" ", Wn) of X such that W = Span iwi, W2,... •, Wk\- On the other hand Wk-\-i,..., Wn € W-^, hence dim W-^ = n — k. •
g. Riesz's theorem 3.34 Theorem (Riesz). Let X be a Euclidean or Hermitian space of dimension n. For any L e X* there is a unique XL G X such that L{X) = {X\XL)
VXGX.
'
(3.5)
Proof. Assume for instance, that X is Hermitian. Suppose L ^ 0, otherwise we choose XL = 0, and observe that d i m l m L = 1, and V := kerL has dimension n — 1 if d i m X = n. Fix XQ G V-^ with |a:o| = 1, then every x E X decomposes as X = x' -\- XxQ,
x' 6 kerL, A = (x|a:o)-
Consequently, L{x) = I/(a:') -f AL(xo) = (x|a;o)I/(a:o) = (a;|L(xo)xo) and the claim follows choosing x^ '= L{xo)xo.
•
The map (3 : X* ^^ X, L ^^ XL defined by the Riesz theorem is called the Riesz map. Notice that /? is linear if X is Euclidean and antilinear if X is Hermitian. 3.35 T h e Riesz m a p in c o o r d i n a t e s . Let X be a Euclidean (Hermitian) space with inner (Hermitian) product ( | ),fixa basis and denote by x = (x^, x ^ , . . . , x'^) the coordinates of x, and by G the Gram matrix of the inner (Hermitian) product. Let L G X* and let L be the associated matrix, L{x) = L x . From (3.5) L x = L{x) = {X\XL)
= X-^GXL
if X is Euclidean,
L x = L{x) = {X\XL)
= X"^G5CL"
if X is Hermitian,
Gx£, = L-^
or XL = G~^L-^
if X is Euclidean,
7^
1 'p
i.e.,
Gxx, = L
or x/, = G
L
if X is Hermitian.
In particular, if the chosen basis (ei, 62, • • •, en) is orthonormal, then G = Id and XX, = L-^
if X is Euclidean,
—T
XL = L
if X is Hermitian.
92
3. Euclidean and Hermitian Spaces
Figure 3.1. Dynamometer.
3.36 E x a m p l e (Work and forces). Suppose a mass m is fixed to a dynamometer. If 9 is the inclination of the dynamometer, the dynamometer shows the number L = mg cos 0,
(3.6)
where p is a suitable constant. Notice that we need no coordinates in R"^ to read the dynamometer. We may model the lecture of the dynamometer as a map of the direction V of the dynamometer, that is, as a map L : S"^ —* R from the unit sphere 5"^ = {x € E^ Ma;I = 1} of the ambient space V into R. Moreover, extending L homogeneously to the entire space V by setting L{v) := \v\ L{v/\v\), v e R^ \ {0}, we see that such an extension is linear because of the simple dependence of L from the inclination. Thus we can model the elementary work done on the mass m, the measures made using the dynamometer, by a linear map L : V -^ R. Thinking of the ambient space V as Euclidean, by Riesz's theorem we can represent L as a scalar product, introducing a vector F := XL EV such that {v\F) = L(y)
Vt; G V.
We interpret such a vector as the force whose action on the mass produces the elementary work L{v). Now fix a basis (ei,62,63) of V. li F = (F^^F"^, F^)^ is the column vector of the force coordinates and L = (Li,L2,I/3) is the 1 x 3 matrix of the coordinates of L in the dual basis, that is, the three readings Li = L{ei), i = 1,2,3, of the dynamometer in the directions 61,62,63, then, as we have seen.
In particular, if (61,62,63) is an orthonormal basis.
h. The adjoint operator Let XyY be two vector spaces both on K = M or K = C with inner (Hermitian) products ( | )x and ( | )y and let A : X -^Y he a, hnear map. For any y eV the map X -> {A{x)\y)Y
3.1 The Geometry of Euclidean and Hermitian Spaces
93
defines a linear map on X, hence by Riesz's theorem there is a unique A*{y) e X such that {A{x)\y)Y = {y\A%x))x
Vx G X, Vy e Y,
(3.7)
It is easily seen that the map y -^ A*{y) from Y into X defined by (3.7) is linear: it is called the adjoint of A. Moreover, (i) let A,B : X -^ Y he two linear maps between two Euchdean or Hermitian spaces. Then {A + B)* = A* ^ B*, (ii) (XA)* = XA* if A G M and A : X ^ Y is SL hnear map between two Euclidean_spaces, (iii) (XA)* = XA"" if A G C and A : X ^ F is a hnear map between two Hermitian spaces, (iv) ( 5 o A)* = A* o 5 * if A : X ^ F and B : F ^ Z are hnear maps between Euclidean (Hermitian) spaces, (v) (A*)* = ^ i f A : X - ^ y i s a linear map. 3.37 %, Let X, Y be vector spaces. We have already defined an adjoint A : Y* —^ X* with no use of inner or Hermitian products, < A{y*),x>=<
y*,A(x)
>
\/x e X, My* e
Y\
If X and Y are Euclidean (Hermitian) spaces, denote by /3x : X* —^ X, /Sy : Y* -^ Y the Riesz isomorphisms and by A* the adjoint of A defined by (3.7). Show that A* =
/3xoAo0-\ 3.38 T h e adjoint o p e r a t o r in c o o r d i n a t e s . Let X, Y be two Euclidean (Hermitian) spaces with inner (Hermitian) products { \ )x and ( | ) y . Fix two bases in X and y , and denote the Gram matrices of the inner (Hermitian) products on X and Y respectively, by G and H . Denote by x the coordinates of a vector x. Let A : X -^ Y be a linear map. A* be the adjoint map and let A, A* be respectively, the associated matrices. Then we have (A(x)\y)Y
= x^A^Hy,
{x\A*{y))
= x^'GA'y,
{x\A*iy))
= x^GA^^y,
if X and Y are Euclidean and (A{x)\y)Y
= x^A^Hy,
if X and Y are Hermitian. Therefore GA* = A ^ H
if X and Y are Euclidean,
GA* = A ^ H
if X and Y are Hermitian,
or, recalling that G ^ = G, ( G ~ i ) ^ = G - \ H ^ = H if X and Y are Euclidean and that G ^ = G, ( G - i ) ^ = G - i , and H ^ = H if X and Y are Hermitian, we find A* = G - ^ A ^ H
if X and Y are Euclidean,
A* = G~^ A ^ H
if X and r are Hermitian.
In particular. A* = A ^ in the Euclidean case, _r A* = A in the Hermitian case if and only if the chosen bases in X and Y are orthonormal.
(3.8)
94
3. Euclidean and Hermitian Spaces
3.39 Theorem. Let A: X -^Y be a linear operator between two Euclidean or two Hermitian spaces and let A* : Y -^ X be its adjoint. Then Rank^* = R a n k ^ . Moreover, (Im^)-^=ker^*,
Im^=(ker^*)^,
{lmA*)-^=kerA,
IinA* = (ker^)-^.
Proof. Fix two orthonormal bases on X and Y, and let A be the matrix associated to A using these bases. Then, see (3.8), the matrix associated to A* is A-^, hence Rank A* = Rank A ^ = Rank A = Rank A, and dim(ker A*)-^ = dim Y - dim ker A* = Rank A* = Rank A = dim Im A. On the other hand, Im A C (ker A*)-*- since, if t/ = A{x) and A*{v) = 0, then {y\v) = (A{x)\v) = {x\A*{v)) = 0. We then conclude that (ker A*)-*- = ImA. The other claims easily follow. In fact, they are all equivalent to I m A = (kerA*)-*-. D
As an immediate consequence of Theorem 3.39 we have the following. 3.40 Theorem (The alternative theorem). Let A : X -^ Y be a linear operator between two Euclidean or two Hermitian spaces and let A* : Y -^ X be its adjoint. Then A|kerA-L * (ker^)-^ -^ ImA and At^ : ImA —^ (ker^)-^ are injective and onto, hence isomorphisms. Moreover, (i) A(x) = y has at least a solution if and only if y is orthogonal to kerA^ (ii) y is orthogonal to ImA if and only if A*{y) = 0, (iii) A is injective if and only if A* is surjective, (iv) A is surjective if and only if A* is injective. 3.41 . A more direct proof of the equaUty ker A = (Im^*)-^ is the following. For simplicity, consider the real case. Clearly, it suffices to work in coordinates and by choosing an orthonormal basis, it is enough to show that Im A = (ker A^)-^ for every matrix A € Mm,n{^)Let A = (a'j) G Mm„n{^) and let a^, a^,..., a"^ be the rows of A, equivalently the columns of A ^ , /ai\
A =
\a^J Then,
3.2 Metrics on Real Vector Spaces
Ax
/ a\x^ + alx^ + • • • + al^x"" \ ajx^ + ajx^ + • • • + alx"" \ a f x^ + afx^ + • • • + a ^ x ^ /
95
2
\ a^ • x/
Consequently, x G ker A if and only if a* • x = 0 Vi = 1 , . . . , m, i.e., kerA = S p a n U \ s?,...,
a^}
= (ImA^)"^.
(3.9)
3-2 Metrics on Real Vector Spaces In this section, we discuss bilinear forms on real vector spaces. One can develop similar considerations in the complex setting, but we refrain from doing it. a. Bilinear forms and linear operators 3.42 Definition. Let X he a real linear space. A bilinear form on X is a map 6 : X X X —> M that is linear in each factor, i.e., b{ax + /3y, z)=a 6(x, z)-\-(3b{y, z), 6(x, ay -\- f3z) = a 6(x, y) + 0 b{z, z). for all X, y, X G X and all forms on X by B{X).
a,l3e
We denote the space of all bilinear
Observe that, if 6 G B{X), then 6(0, x) - b{0,y) = 0 Vx,?/ G X. The class of bihnear forms becomes a vector space if we set (6i + 62)(x,y) := 6i(x,y) + b2{x,y), (A6)(x, y) := 6(Ax, y) =- 6(x, A, y). Suppose that X is a linear space with an inner product denoted by ( I ). If 6 G S(X), then for every y e X the map x -^ b{x,y) is linear, consequently, by Riesz's theorem there is a unique B := B{y) G X such that b{x,y) = {x\B{y)) Vx G X. (3.10) It is easily seen that the map y —> B{y) from Y into X defined by (3.10) is linear. Thus (3.10) defines a one-to-one correspondence between B{X) and the space of linear operators £(X, X), and it is easy to see this correspondence is a linear isomorphism between B{X) and £(X, X).
3. Euclidean and Hermitian Spaces
96
Ueber
die Hjpothe8eQ,welohe derGeometne zu Gruad^ liegen.
d ie
Hypothesen, B. R i e m a n n.
welche der Oeometrie za 6nmde liegen. iBcljung. P l » n d«r Un BekaantUoh wut dte Qmmctrte to . den Begritf de« iUittmea, ab die «r*««n Grandbagriib tta die Baume al* etwaa Gegebene* wnxa. Sie giebt ton iha NomlnaldefinitioneD, w«hrend die wcMBtUchen BeetimmuBgeQ in Form TOO Axiomen auftretesi. D u VerMltaiM dieeer VoreuMetrangen bleibt dabei im Dankeln; man •ieht weder «in, ob uod in wie wdt "ihte Verbindung nothwendig, fiocli a priori, ob sie mflgHch Ut. Uiese Dankelheit wntde such von Ettklid bi« auf L e g e n d r e . der Oeometrie ni nennen. urn den berOl>mte»ten neoeren Bei weder Ton den MathenwUkem, noch on den PhiloMiphen, welch* sich te die* aeinen Oiund wohl dariu, damit betcbiitigttD. geboben. £ • h daM der allgemeine Begriff mehr£uh tttgedehnter i . ganz onbearbeitet blieb. Icli cliein die babe mir daher tunichet die A n ^ b e gesteilt. den Begriff einer mehrftch auigedehateo GrBMe au< aUgemrinei» Ght««»«nbegri«»n lu conetniimi. £ • wird daraos hervoigehea. dan ein« m«hrfe«h auigedehnte GrOwe ver-
B. R i e m a n n.
t) Dine AMModlooc H tm 10. Jani 18&4 ran d«m 2«Mk MiMT HaUiUtioa Ttnaitaltelen CoUoqiiam Iflerans eridlrt Mch die Form der D«nMlni(,
d«m dreix«bs«m B«nd« 4*r AUiandlaag«n der KSni|^ieiicD OtMllwiwrt der WiMMtehaftcn tn OSttingen.
Gdttingen, in der Diet«riolitoli»ii 1867,
BoohhandUng.
BmanMbwrig, im JuU 1M7.
Figure 3.2. Frontispiece and a page of the celebrated dissertation of G. F. Bernhard Riemann (1826-1866).
3.43 Bilinear forms in c o o r d i n a t e s . Let X be a finite-dimensional vector space and let (ei, 6 2 , . . . , e-n) be a basis of X. Let us denote by B the nxn matrix, sometimes called the Gram matrix of b. B = [bij]
bij =
b{ei,ej).
Recall that the first index from the left is the row index. Then by linearity, if for every x, 2/, X = {x^, cc^,..., ic^)^ and y = (y^, 2/^, • • •, y^)^ € M^ are respectively, the column vectors of the coordinates of x and y, we have
bix,y) = J2 ^ij^'y' = x^-lBy) =x^By. In particular, a coordinate system induces a one-to-one correspondence between bilinear forms in X and bilinear forms in W^. Notice that the entries of the matrix B have two lower indices that sum with the indices of the coordinates of the vectors x, y that have upper indices. This also reminds us that B is not the matrix associated to a linear operator B related to B . In fact, if instead N is the associated linear operator to 6, bix.y) = (x\N{y))
Vx,y€X,
then y ^ B x - b(x,y) = {x\N{y))
= y^N^Gx
where we have denoted by G the Gram matrix associated to the inner product on X , G = [gij], gij — (ei|ej), and by N the n v. n matrix associated to A^ : X —^ X in the basis (ei, 6 2 , . . . , en). Thus N^G = B or, recalling that G is symmetric and invertible, N = G-iB^.
3.2 Metrics on Real Vector Spaces
97
b. Symmetric bilinear forms or metrics 3.44 Definition. Let X be a real vector space. A bilinear form b G B{X) is said to be (i) symmetric or a metric, ifb{x^y) = b{y,x) Vx,?/ G X, (ii) antisymmetric ifb{x,y) = —b{y,x) ^x,y G X. The space of symmetric bilinear forms is denoted by Syra{X). 3.45 %, Let b G B(X). Show that bs{x,y) := ^{b{x,y) + b{y,x)), x,y € X , is a symmetric bilinear form and bA{x,y) :— ^{b{x,y) — b{y,x)), x,y £ X, is an antisymmetric bilinear form. In particular, one has the natural decomposition 6(x, y) = bs {x, y) + 6^ (x, y) of b into its symmetric and antisymmetric parts. Show that b is symmetric if and only if 6 = 65, and that b is antisymmetric if and only if 6 = 6^^. 3.46 %. Let 6 G B(X) be a symmetric form, and let B be the associated Gram matrix. Show that 6 is symmetric if and only if B ^ = B . 3.47 %, Let b e B(X) and let N be the associated linear operator, see (3.10). Show that AT is self-adjoint, N* = AT, if and only if 6 G Sym{X). Show that N* = -N if and only if b is antisymmetric.
c. Sylvester's theorem 3.48 Definition. Let X be a real vector space. We say that a metric on X, i.e., a bilinear symmetric form g : X x X ^^^R is (i) nondegenerate if^xeX,x^O there is y e and Wy e Xy y y^ 0 there is x e X such that (ii) positively definite ifb{x^x) > O'^x ^ X, x ^ (iii) negatively definite ifb{x,x) < 0 \/x e X, x
X such that 6(x, y) ^ 0 6(x, y) j^ 0, 0, ^0.
3.49 %. Show that the scalar product {x\y) on X is a symmetric and nondegenerate bilinear form. We shall see later, Theorems 3.52 and 3.53, that any symmetric, nondegenerate and positive bilinear form on a finite-dimensional space is actually an inner product.
3.50 Definition. Let X be a vector space of dimension n and let g G Sym{X) be a metric on X. (i) We say that a basis (ei, e 2 , . . . , Cn) is g-orthogonal if g{ei,ej) = 0 Vz,j = l,...,n, i^j. (ii) The radical of g is defined as the linear space md{g) := | x G X\g{x,y)
= 0\fye
x}.
(iii) The range of the metric g is r{g) := n — dimrad^.
98
3. Euclidean and Hermitian Spaces
Figure 3.3. Jorgen Gram (1850-1916) and James Joseph Sylvester (1814-1897).
(iv) The signature of the metric g is the triplet of numbers {i^{g)J.{g),io{g)) where i^{g) := maximum of the dimensions of the subspaces V C X on which g is positive definite, g{v^v) > O^v EV, V ^^ 0, i-{g) := maximum of the dimensions of the subspaces V C X on which g is negative definite, g{v,v)
and
i+{g) + i-{g) + io{g) = n.
3.2 Metrics on Real Vector Spaces
Proof. Suppose that g(ei,ei)
99
> 0 for i = 1 , . . . , n-j.. For each v = X^i^i '^^^i^ we have 9{v,v) = Y^\v'\'^g{ei,ei)
> 0,
1=1
hence dim Span {ei, 6 2 , . . . , en^} < H ( P ) - On the other hand, if l y C X is a subspace of dimension i-^{g) such that g{v, i;) > 0 Vv € W, we have V y n S p a n { e n ^ + i , . . . , e n } = {0} since g{v,v) < 0 for all v G S p a n { e n , + i , . . . , e n } . Therefore we also have i-\-{g) < n — {n — n-|_) = n.^. Similarly, one proves that n _ = i-{g). Finally, since G := [^(ei,ej)] is the matrix associated to g in the basis (ei, 6 2 , . . . , en), we have io{g) = d i m r a d ( y ) = d i m k e r G , and, since G is diagonal, d i m k e r G = UQ. D
d. Existence of ^-orthogonal bases The Gram-Schmidt algorithm yields the existence of an orthonormal basis in a Euclidean space X. We now see that a slight modification of the GramSchmidt algorithm allows us to construct in a finite-dimensional space a ^-orthogonal basis for a given metric g. 3.53 Theorem (Gram-Schmidt). Let g be a metric on a finite-dimensional real vector space X. Then g has a g-orthogonal basis. Proof. Let r be the rank of gf, r := n—dimrcid (^), and let {wi, W2,. •., Wn-r) be a basis of rad (g). If V denotes a supplementary subspace of rad (^f), then V is p-orthogonal to radg and d i m F = r. Moreover, for every v £ V there is z £ X such that g{v, z) ^ 0. Decomposing zasz = w-\-t^wEV^t£ r a d ( ^ ) , we then have g{v,w) = g{v,w) + g{v, t) = g(v, z) ^ 0, i.e., g is nondegenerate on V. Since trivially, (i^i, i i ; 2 , . . . , Wn-r) is ^-orthogonal and V is ^-orthogonal to (i^i, 1^2, • • •, '^n-r)-, in order to conclude it suffices to complete the basis {w\^ W2,. - • -, Wn-r) with a ^f-orthogonal basis of V; in other words, it suffices to prove the claim under the further assumption that g be nondegenerate. We proceed by induction on the dimension of X. Let ( / i , / 2 , . . . , / n ) be a basis of X. We claim that there exists ei G X with g{ei,ei) / 0. In fact, if for some fi we have gifiifi) 7^ O5 we simply choose ei := / i , otherwise, if g{fi,fi) = 0 for all i, for some k ^ 0 we must have g{fi, fk) ¥" 0, since by assumption rad (g) = {0}. In this case, we choose ei := / i -|- /^ as
g{fi + fkji + fk) = gifufi) + 2g{fiJk) -f gifkJk) = O + 2g{fiJk) + 0 / 0 . Now it is easily seen that the subspace Vi:=[vex\g{euv)
= 0]
supplements S p a n { e i } , and we find a basis (t'2, • • • ,^n) of Vi such that g{vj,ei) for all j = 2 , . . . , n by setting ._.
= 0
fl(/j,ei) P(ei,ei)
Since g is nondegenerate on Vi, by the induction assumption we find a p-orthogonal basis ( e 2 , . . . , Cn) of Vi, and the vectors (ei, 6 2 , . . . , Cn) form a p-orthogonal basis of X. D
100
3. Euclidean and Hermitian Spaces
A variant of the Gram-Schmidt procedure is the following one due to Carl Jacobi (1804-1851). Let g : X X X -^ R he Si metric on X. Let (/i, / 2 , . •., fn) be a basis of X, let G be the matrix associated to g in this basis, G = [gij], gij = 9{fu fj)- Set Ao = 1 and for A: = 1 , . . . , n Ak :=detGA: where G^ is the k x k submatrix of the first k rows and k columns. 3.54 Proposition (Jacobi). / / A^ 7^ 0 for all k = 1 , . . . , n, there exists a g-orthogonal basis (ei, 6 2 , . . . , e^) of X; moreover g[ek,ek) := —7—. Proof. We look for a basis (ei, e2, • . . , en) so that I ei
=a\fi,
62 = 0 2 / 1 +«2/2»
or, equivalently, ek
•=Yl^kfi^
(3.11)
k = l,..-,n,
as in the Gram-Schmidt procedure, such that g(ei,ej) = 0 for i ^ j . At first sight the system giei^ej) = 0, i :^ j , is a system in the unknowns aj.. However, if we impose that for all fc's p(efc,/i) = 0 Vi = l , . . . , A ; - l , (3.12) by linearity g{ek,ei) = 0 for i < /c, and by symmetry gie^^ei) = 0 for i > k. It suffices ,a^ then to fulfill (3.12) i.e., solve the system offc— 1 equations in k unknowns al.,a^,... k
Yl^ifjJiK
=0,
Vi = 1,... ,^ - 1.
(3.13)
j=i
If we add the normalization condition k
Yl9{fjJk)ai
= l,
(3.14)
j=i
we get a system of k equations in k unknowns of the type G ^ x = b , where Gk = [9ij]y 9ij '•= gifiJj)^ X = (oj^,...,aj^)^ and b = ( 0 , 0 , . . . , 1)^. Since det Gfc = Afc and Afc 7^ 0 by assumption, the system is solvable. Due to the arbitrarity of fc, we are able to find a gf-orthogonal basis of type (3.11). It remains to compute ^(6^,6^). Prom (3.13) and (3.14) we get
9iek,ek) = Yl ^O'kOUiJj) = J2'^k{Y^9(fiJj)4)
=Yl/'k^Jk=cit j=i
and we compute a^ by Cramer's formula, ^k —
hence giek.Ck) =
Ak-i/Ak-
Afc
'
3.2 Metrics on Real Vector Spaces
101
3.55 Remark. Notice that Jacobi's method is a rewriting of the GramSchmidt procedure in the case where g{fi^fi) 7^ 0 for all i's. In terms of Gram's matrix G := [gi^i^ej)]^ we have also proved that
T-GT = d i a g { ^ } for a suitable triangular matrix T. 3.56 Corollary (Sylvester). Suppose that Ai,...,Afc ^ 0. Then the metric g is nondegenerate. Moreover, i-{g) equals the number of changes of sign in the sequence (1, Ai, A 2 , . . . , A^). In particular, if Ak > 0 for all k 's, then g is positive definite. Let (ei, 6 2 , . . . , en) be a ^-orthogonal basis oi X. By reordering the basis in such a way that > 0 if j = l , . . . , 2 + ( ^ ) , < 0 if j = 2+(p) + l,...,2+(^) + *-(^), = 0 otherwise;
9{ej,ej)\ and setting (
fr-=^
[ Cj
^
if 7 - 1
i^(a)-\-i
(a)
otherwise
we get
9{fjJj)
( 1 if j = l , . . . , i + ( ^ ) , = < - 1 if j = n ( ^ ) + l , . . . , i + ( ^ ) + z_(c/), I 0 otherwise.
e. Congruent matrices It is worth seeing now how the matrix associated to a bilinear form changes when we change bases. Let (ei, 6 2 , . . . , Cn) and (/i, /2, • • •, fn) be two bases of X and let R be the matrix associated to the map R : X -^ X, R{ei) := fi in the basis (ei, 6 2 , . . . , e^), that is
R := [ri I r2 where r^ is the column vector of the coordinates of fi in the basis (ei, 6 2 , . . . , Cn)' As we know, if x and x' are the column vectors of the coordinates of X respectively, in the basis (ei, 6 2 , . . . , en) and (/i, /2, • • •, /n), then x = R x ' . Denote by B and B ' the matrices associated to b respectively, in the coordinates (ei, 6 2 , . . . , e^) and (/i, / 2 , . • •, fn)- Then we have
102
3. Euclidean and Hermitian Spaces
b{x,y)
x'^BV
b{x, y) = x^By = x "T-DTJ R^BRy' hence (3.15)
B' = R^BR.
The previous argument can be of course reversed. If (3.15) holds, then B and B ' are the Gram matrices of the same metric h on W^ in different coordinates 6(x,2/)=x^bV = (RxfB(Ry). 3.57 Definition. Two matrices A , B G Mn^n{ if there exists a nonsingular matrix R E Mn,n{
are said to he congruent such that B = R ^ A R .
It turns out that the congruence relation is an equivalence relation on matrices, thus the nxn matrices are partitioned into classes of congruent matrices. Since the matrices associated to a bilinear form in different basis are congruent, to any bilinear form corresponds a unique class of congruent matrices. The above then reads as saying that two matrices A, B G Mn,n{^) are congruent if and only if they represent the same bilinear form in different coordinates. Thus, the existence of a ^r-orthogonal basis is equivalent to the following. 3.58 T h e o r e m . A symmetric matrix A G Mn^n{ diagonal matrix.
is congruent to a
Moreover, Sylvester's theorem reads equivalently as the following. 3.59 T h e o r e m . Two diagonal matrices I, J G Mn,n(^) o,f^ congruent if and only if they have the same number of positive, negative and zero entries in the diagonal. If, moreover, a symmetric matrix A G Mji_^ji{M^ is congruent to (\ 0 0 Ida 0
V 0
-Idfc
0
0
0/
then (a^b^n — a — b) is the signature of the metric y ^ A x . Thus the existence of a ^-orthogonal matrix in conjunction with Sylvester's theorem reads as the following. 3.60 T h e o r e m . Two symmetric matrices A , B G Mn,nO^) o,re congruent if and only if the metrics y-^ A x and y-^Bx on W^ have the same signature (a, 6, r). In this case, A and B are congruent to
3.2 Metrics on Real Vector Spaces
V
103
\
Ida
0
0
0
-Idfc
0
0
0
0/
f. Classification of real metrics Since reordering the basis elements is a linear change of coordinates, we can now reformulate Sylvester's theorem in conjunction with the existence of a ^-orthonormal basis as follows. Let X, Y be two real vector spaces, and let g, h be two metrics respectively, on X and Y. We say that {X,g) and (F, h) are isometric if and only if there is an isomorphism L : X —^ Y such that h{L{x),L{y)) = g{x^y) Wx,y G X. Observing that two metrics are isometric if and only if, in coordinates, their Gram matrices are congruent, from Theorem 3.60 we infer the following. 3.61 Theorem. (X, g) and (y, h) are isometric if and only if g and h have the same signature, (hid) ^'^-{9)^0(9)) = (i+(/i),i_(/i),ioW). Moreover, if X has dimension n and the metric g on X has signature (a^b.r), a + b + r = n, then (X^g) is isometric to (lR^,/i) where /i(x,y) := x^Hy and
/I H
V
Ida
0
0
0
-Idb
0
0
0
0/
According to the above, the metrics over a real finite-dimensional vector space X are classified, modulus isometrics, by their signature. Some of them have names: (i) The Euclidean metric: i-^{g) = n, i-{g) = io{g) — 0; in this case g is a scalar product. (ii) The pseudoeuclidean metrics: i-{g) = 0. (iii) The Lorenz metric or Minkowski metric: i-\-{g) = n — 1, i-{g) — 1, ^o(^) = 0. (iv) The Artin metric i-\-{g) = i-{g) = P, ^o(p) = 03.62 %. Show that a biUnear form ^ on a finite-dimensional space X is an inner product on X if and only if g is symmetric and positive definite.
104
3. Euclidean and Hermitian Spaces
g. Quadratic forms Let X be a finite-dimensional vector space over M and let b G B{X) be a bilinear form on X. The quadratic form 0 : X ^ R associated to b is defined by 0(x) = b{x,x)^ X e X. Observe that 0 is fixed only by the symmetric part of b bs{x,y) :=
-{b{x,y)-\-b{y,x))
since b{x, x) = bs{x, x) \fx G X. Moreover one can recover bs from (/) since bs is symmetric, bs{x, y) = 2 v^^ -^y)-
^ W - ^(^)) •
Another important relation between a bilinear form b G B{X) and its quadratic form 0 is the following. Let x and v £ X. Since (j){x -f- tv) = (f){x) + M b(x^ v) -f b{v, we have j^(t>{x + tv)^t=o = 2bs{x,v).
(3.16)
We refer to (3.16) saying that the symmetric part bs ofb is the first variation of the associated quadratic form. 3.63 Homogeneous polynomials of degree two. Let B = [bij] € Mn,nW and let n
6(x,y) := x^By = ^
bijx'y^
be the bilinear form defined by B on R*^, x = ( x \ x ^ , . . . , x"), y = ( y \ ?/2,..., 2/^). Clearly, n
(/)(x) = 6(x, x) = x^Bx = ^
bijx'^x^
is a homogeneous polynomial of degree two. Conversely, any homogeneous polynomial of degree two P{x) = ^
bijx'x^ = x^Bx
i,j = l,n i<3
defines a unique symmetric bilinear form in W^ by 6(x,y):=^(p(x + y ) - P ( x ) - P ( y ) ) with associated quadratic form P .
3.2 Metrics on Real Vector Spaces
105
3.64 E x a m p l e . Let {x,y) be the standard coordinates in M."^. The quadratic polynomial ax^ + 6xs/ + cy^ = ( x , j , ) ( ^ J ^
' f ) ( y
is the quadratic form of the metrics g{{x,y),{z,w))
'a := {z,w) L \b/2
b/2\
(x^
c J
\yj
3.65 D e r i v a t i v e s of a quadratic form. Prom (3.16) we can compute the partial derivatives of the quadratic form (f){x) := x ^ G y . In fact, choosing u = e^, we have -^{x)
:= -(t>{x + teh) = 2bs(x,eh)
= x^(G + G^)e^
hence, arranging the partial derivatives in a 1 x n matrix, called the Jacobian oi(t>,
matrix
•'*'-[^w||^w|-lSH Idx^^ ^\ dx^^'^'l '" \ dx^ we have D>(a;)=x^(G-hG^), or, taking the transpose, V(/)(x) := (D)(x))^ = (G + G ^ ) x .
h. Reducing to a sum of squares Let ^ be a metric on a real vector space X of dimension n and let (j) be the associated quadratic form. Then, choosing a basis (ei, 6 2 , . . . , en) we have (t){x) = g{x,x) =
^{x'fg[euei) 2=1
if and only if (ei, 6 2 , . . . , e^) is ^-orthogonal, and the number of positive, negative and zero coefficients is the signature of g. Thus, Sylvester's theorem in conjunction with the fact that we can always find a ^-orthogonal basis can be rephrased as follows. 3.66 Theorem (Sylvester's law of inertia). Let (j){x) = g{x,x) be the quadratic form associated to a metric g on an n-dimensional real vector space. (i) There exists a basis (/i, / 2 , . . . , fn) of X such that
2=1
where {i^{g),i-{g),io{g))
2=1
is the signature of g.
2=1
106
3. Euclidean and Hermitian Spaces
(ii) If for some basis (ei, 6 2 , . . . , en) n
n
x:=Y,^'ei,
(3.17)
then the numbers n^^n^ andno respectively, of positive, negative and zero ^{ei)^s are the signature {i^{g),i-{g),io{g)) of g. 3 . 6 7 E x a m p l e . In order to reduce a quadratic form cf) to the canonical form (3.17), we may use Gram-Schmidt's algorithm. Let us repeat it focusing this time on the change of coordinates instead of on the change of basis. Suppose we want to reduce to a sum of squares by changing coordinates, the quadratic form n
where at least one of the aij 's is not zero. We first look for a coefficient akk that is not zero. If we find it, we go further, otherwise if all akk vanish, at least one of the mixed terms, say a i 2 , is nonzero; the change of variables x^ = i / i -f-2/2,
yx^ = y^
for j = 3 , . . . , n,
transforms ai2X^x'^ into ai2{(y^)'^ — (y^)^), and since a n = 022 = 0, in the new coordinates (y^, j / ^ , . . . , y^) the coefficient of (y^)^ is not zero. Thus, possibly after a linear change of variables, we write 0 as ^(x) = — ( a n j / i ) 2 + y ^ azjyiyJ + B ( s / 2 , . . . , J/"). We now complete the square and set
iY'=a^^y^ +
ZU^y'^
\Y^ =y^
fori = 2 , . . . , n .
so that
^ix) =
«ii ^
^(a^^y'-^^^yA\c=^{Y'f-^C
"Hi 2
j=2
/
an
where C contains only products of Y ^ , . . . , Y^. The process can then be iterated. 3.68 E x a m p l e . Show that Jacobi's method in Proposition 3.54 transforms (p in Ai
A2
A3
if X = Y27=i ^*^i' ^^^ ^ suitable p-orthogonal basis (ei, 6 2 , . . . , en)3.69 E x a m p l e (Classification of conies). The conies in the plane are the zeros of a second degree polynomial in two variables P{x,y)
:=ax'^ -{-2bxy-hcy^
-\-dx + ey + f = 0,
{x,y) G M^,
(3.18)
3.2 Metrics on Real Vector Spaces
107
where a, 6, c, d,e,f£ R. Choose a new system of coordinates (X, Y), X = ax-{- l3y, Y = 7X + 6y in which the quadratic part of P transforms into a sum of squares ax^ + bxy + cy^ = pX"^ + g^^, consequently, P into pX'^ + qY'^ + 2rX + 2 s y + / = 0. Now we can classify the conies in terms of the signs of p, q and / . If p, q are zero, the conic reduces to the straight line 2rX + 2 s y + / = 0. If p 7»^ 0 and g == 0, then, completing the square, the conic becomes p{X - Xof
+ 25y -f / = 0,
Xo = - ,
i.e., a parabola with vertex in (Xo,0) and axis parallel to the axis of Y. Similarly, if p = 0 and q ^ 0, the conic is a parabola with vertex in {0,Yo), YQ := s/q, and axis parallel to the X axis. Finally, if pq ^ 0, completing the square, the conic is p{X - Xo? -h q{Y - Yof
+ / = 0,
Xo=
r/p,
FQ =
s/q,
i.e., it is o a hyperbola if / 7^ 0 and pg < 0, o two straight lines if / = 0 and pg < 0, o an ellipse if sgn (/) = —sgn (p) and pq > 0, o a point if / = 0 and pq > 0, o the empty set if sgn (/) = sgn (p) and pq > 0. Since we have operated with linear changes of coordinates that map straight lines into straight lines, ellipses into ellipses, and hyperbolas into hyperbolas, we conclude the following. 3.70 P r o p o s i t i o n . The conies in the plane are classified in terms of the signature of their quadratic part and of the sign of the zero term. 3.71 %. The equation of a quadric i.e., of the zeros of a second order polynomial in n variables, see Figure 3.4 for n — 3, has the form )(x) : = x ^ A x + 2 b « x + c = 0 where A G Mn,n{R) is symmetric, h ^ X and c ^R. Prove the following claims, (i) 0 is a center of symmetry, i.e., )(x) = >(—x), if and only if b = 0. (ii) xo is a center of symmetry, i.e., 0(xo — x) = )(xo + x), if and only if AXQ = —b. (iii) If det A 7^ 0, then there is a center of symmetry XQ; letting x = z + XQ, we have (f)(x.) = z ^ A z -f ci = 0 for a suitable ci 6 M. (iv) By Sylvester's law of inertia, z ^ A z - f c i transforms with a suitable linear change of coordinates into i=l
i=p-\-l
(v) Suppose det A = 0. Since A = A ^ , we have ker A — (Im A)-^. Choosing a basis in which the first k elements generate Im A and the last n — k ker A, then A writes as
I
f\ A'
V
0
0'
108
3. Euclidean and Hermitian Spaces
(b)
ic),{k),iQ)
^
^
/^\if)
ii)
(9)
iJ)
^
(0
Figure 3.4. Quadrics: (a) ellipsoid: a^x'^-\-b^y'^-\-c'^z'^ = 1; (b) point: a^x'^-\-b'^y^-\-c^z'^ = 0; (c) imaginary ellipsoid: a^x"^ -f- b'^y^ -}- c^z^ = — 1; (d) hyperboloid of one sheet: a^x^ -h b'^y'^ — (P'z^ = 1; (e) cone: a^x^ + b^y^ — cP'z'^ = 0; (f) hyperboloid of two sheets: -a^x^ - b'^y^ + c^z^ = 1; (g) paraboloid: a'^x'^ + b'^y^ - 2cz = 0, c> 0; (h) saddle a^x^ - b'^y^ — 2cz = 0, c > 0; (i) elliptic cyhnder: a^x^ + b'^y^ — 1; (j) straight line: a^x^ + 6^2/^ = 0; (k) imaginary straight line: c?x^ -\-b^y^ = —1; (1) hyperbolic cylinder cP'x^ — b^y^ — 1; (m) nonparallel planes: d^x^ — b'^y^ = 0; (n) parabolic cylinder a?x'^ — 2cz, c > 0; (o) parallel planes: a?x'^ = 1; (p) plane: a^x^ = 0; (q) imaginary plane: a?x'^ = —1.
3.3 Exercises
109
in this new basis and the quadric can be written as (/>(x) = ( x O ^ A ' x ' + 2(b'|xO + 2 ( b ' ' | x ' 0 + C2 = 0 where x ' , b ' G Im A, x ' ' , b ' ' G ker A, x = x ' + x ' ' , b = b ' + b ' ' and det A ' / 0. Applying the argument in (iii) to ( x ' ) ^ A ' x ' + 2 b ' « x ' +C2, we may further transform the quadric into (l){x) = ( x O ^ A ' x ' + C3 + 2 b ' ' . x ' ' = 0, and, writing j / ' := —2 b " • x ' ' — C3, that is, by means of an affine transformation that does not change the variable x', we end up with (f){x) = ( x ' ) ' ^ A ' x ' — y' = 0.
3.3 Exercises 3.72 %, Starting from specific lines or planes expressed in parametric or implicit way in M^, write o the straight line through the origin perpendicular to a plane, o the plane through the origin perpendicular to a straight line, o the distance of a point from a straight line and from a plane, o the distance between two straight lines, o the perpendicular straight line to two given nonintersecting lines, o the symmetric of a point with respect to a line and to a plane, o the symmetric of a line with respect to a plane. 3.73 %. Let X, Y be two Euclidean spaces with inner products respectively, ( | ) x and ( I ) y . Show that X X y is an Euclidean space with inner product {xi\yi)-\-{x2\y2), (xi,X2), (2/1,2/2) E X xY. of X X y .
Notice that X x {0} and {0} x Y are orthogonal subspaces
3.74 If. Let X, 2/ G M"". Show that x ± y if and only if \x - ay\ > \x\ Va G M. 3.75 %, The graph of the map A{x) := Ax, A G Mm,n{^) GA := { ( x , y ) | a : G R ' ' , yeR^,
y = A{x)\
is defined as C R"" x R'^.
Show that GA is a linear subspace of M'^+'^ of dimension n and that it is generated by the column vectors of the {k -\- n) x n
1\
A Id„
110
3. Euclidean and Hermitian Spaces
Also show that the row vectors of the k x {n + k) matrix
-Idfe
A generates the orthogonal to GA-
3.76 %. Write in the standard basis of R^ the matrices of the orthogonal projection on specific subspaces of dimension 2 and 3. 3.77 %, Let X be Euclidean or Hermitian and let V, W be subspaces of X. Show that
v-^nw^ = {v-\-w)^. 3.78 If. Let / : Mn,n{K) -> K be a linear map such that / ( A B ) = / ( B A ) V A , B € Mn,niK). Show that there is A 6 K such that / ( X ) = A t r X for all X E Mn,n{K) where t r X : = E ^ i C c ^ i f X = [a.}]. 3.79 f. Show that the bilinear form b : Mn,n{^)
x Mn,n{R) -^ K given by n
6(A,B):=tr(A^B):=53(A^B)| i=l
defines an inner product on the real vector space Mn,n{R)- Find the orthogonal of the symmetric matrices. 3.80 f. Given n + 1 points zi, Z2,.-., Zn+i in C, show that there exists a unique polynomial of degree at most n with prescribed values at zi, ^2, • • ? Zn+i- [Hint: If Vn is the set of complex polynomials of degree at most n, consider the map >: Vn -^ C^"*" given by (/>(P) := ( P ( 2 i ) , P f e ) , • • • ,P(^n)).] 3 . 8 1 % D i s c r e t e i n t e g r a t i o n . Let ti, t2, • - •, tn he n points in [a,b] C M. Show that there are constants a i , a 2 , . . •, an such that b
/
n
P{t)dt
=
^ajP{tj)
for every polynomial of degree at most n — 1. 3.82 If. Let g := fO, 1]^ = la; G M"" I 0 < a:i < 1,
i=l,...,n\
be the cube of side one in R'^. Show that its diagonal has length y/n. Denote by x i , . . . ,X2'n the vertices of Q and by x := ( 1 / 2 , 1 / 2 , . . . , 1/2) the center of Q. Show that the balls around x that do not intersect the balls B{xi, 1/2), i = 1 , . . . , 2^, necessarily have radius at most Rn '-= {y/n — 2)/2. Conclude that for n > 4, B(x, Rn) is not contained in Q. 3.83 f.
Give a few explicit metrics in M^ and find the corresponding orthogonal bases.
3.84 f. Reduce a few explicit quadratic forms in R^ and R'* to their canonical form.
4. Self-Adjoint Operators
In this chapter, we deal with self-adjoint operators on a Euclidean or Hermitian space, and, more precisely, with the spectral theory for self-adjoint and normal operators. In the last section, we shall see methods and results of linear algebra at work on some specific examples and problems.
4.1 Elements of Spectral Theory 4.1.1 Self-adjoint operators a. Self-adjoint operators 4.1 Definition. Let X be a Euclidean or Hermitian space X. A linear operator A : X -^ X is called self-adjoint if A* = A. As we can see, if A is the matrix associated to A in an orthonormal basis, then A ^ and A are the matrices associated to A* in the same basis according to whether X is Euclidean or Hermitian. In particular, A is self-adjoint if and only if A = A"^ in the Euclidean case and A = A^ in the Hermitian case. Moreover, as a consequence of the alternative theorem we have X = ker A 0 Im A,
ker A
±lmA
ii A : X -^ X is self-adjoint. Finally, notice that the space of self-adjoint operators is a subalgebra of £(X, X). Typical examples of self-adjoint operators are the orthogonal projection operators. In fact, we have the following. 4.2 Proposition. Let X be a Euclidean or Hermitian space and let P : X -^ X be a linear operator. P is the orthogonal projection onto its image if and only if P* = P and P o P = P.
112
4. Self-Adjoint Operators
Proof. This follows, for instance, from 3.32. Here we present a more direct proof. Suppose P is the orthogonal projection onto its image. Then for every y € X {y~P{y)\z) = 0 V2 € I m P . Thus y = P{y) if y € I m P , that is P{x) = PoP(a:) = P'^{x) Vx G X. Moreover, for x,y E X 0 = (a; - P{x)\P{y))
= (x\P{y))
-
(P{x)\P(y)),
0 = (P{x)\y - P(y)) = (P(x)\y)
-
(P(x)\P{y)),
hence, {P{x)\y) = (x|P(j/)),
i.e.,
P * = P.
Conversely, if P * = P and P^^ = P we have {x - P{x)\Piz))
= {P*(x - P{x))\z)
= (P{x) - P''(x)\z)
= (P(x) - P{x)\z)
=0
for all z^X.
D
b. The spectral theorem The following theorem, as we shall see, yields a characterization of the self-adjoint operators. 4.3 Theorem (Spectral theoremi). Let A : X -^ X be a self-adjoint operator on the Euclidean or Hermitian space X. Then X has an orthonormal basis made of eigenvectors of X. In order to prove Theorem 4.3 let us first state the following. 4.4 Proposition. Under the hypothesis of Theorem 4-3 we have (i) A has n real eigenvalues, if counted with multiplicity, (ii) ifV CW^ is an invariant subspace under A, then V-^ is also invariant under A, (iii) eigenvectors corresponding to distinct eigenvalues are orthogonal. Proof, (i) Assume X is Hermitian and let A € Mn,n(^) be the matrix associated to A in an orthonormal basis. Then A = A''", and A has n complex eigenvalues, if counted with multiplicity. Let z 6 C"^ be an eigenvector with eigenvalue A G C. Then A z = A z = Az = A z. Consequently, if A = (a*), z = (z^, z"^,...,
z^), we have
1 A|z|2 = Er=i A z* z* = E«"=i ^' A? = E«",,=i "5 ^^ ^\ = E",=i 4 ^' ^'Since a* = a^ for a l H , j = 1 , . . . , n, we conclude that (A-A)|zp = 0
i.e.,
AGM.
In the Euclidean case, A ^ = A = A , also. (ii) Let w € V^. For every v £ V -we have {A{'w)\v) = (w\A{v)) = 0 since A{v) G V and w eV-^. Thus A{w) ± V. (iii) Let x, y be eigenvectors of A with eigenvalues A, /x, respectively. Then A and /x are real and (A - ^i){x\y) = iXx\y) - {x\ny) = (A(x)\y) - {x\A{y)) = 0. Thus (x\y) = 0 if A 5^ ^ .
D
4.1 Elements of Spectral Theory
113
Proof of Theorem 4-3. We proceed by induction on the dimension n oi X. On account of Proposition 4.4 (i), the claim trivially holds if d i m X = 1. Suppose the theorem has been proved for all self-adjoint operators on H when dim H = n — 1 and let us prove it for A. Because of (i) Proposition 4.4, all eigenvalues of A are real, hence there exists at least an eigenvector ui of A with norm one. Let H := Span {txi}"*" and let B := A^fj be the restriction of A to H. Because of (ii) Proposition 4.4, B{H) C H, hence B \ H ^^ H is a linear operator on H (whose dimension is n — 1); moreover, B is self-adjoint, since it is the restriction to a subspace of a self-adjoint operator. Therefore, by the inductive assumption, there is an orthonormal basis {u2^... ,Un) oi H made by eigenvectors of B , hence of A. Since U2,... ,Un are orthogonal to u i , (txi, W2, • • •, ttn) is an orthonormal • basis of X made by eigenvectors of ^ .
The next proposition expresses the existence of an orthonormal basis of eigenvectors in several diflFerent ways, see Theorem 2.45. We leave its simple proof to the reader. 4.5 Proposition. Let A : X ^^ X be a linear operator on a Euclidean or Hermitian space X of dimension n. Let (wi, U2, • • •, Un) be a basis of X and let Xi^ A2,..., An be real numbers. The following claims are equivalent (i) (lAi, 1^2, •. •, Un) is an orthonormal basis of X and each Uj is an eigenvector of A with eigenvalue Xj, i.e., A{uj) = XjUj,
(^il'^j) = ^ij
Vi, j = 1 , . . . , n,
(ii) {ui, U2',..., Un) is an orthonormal basis and n
J= l
(iii) (lii, 1/2,..., Un) is an orthonormal basis and for all x^y £ X (A(x)\y) = / ^ ^ = i ^ji^M (y\^j) ^f^ ^^ Euclidean, IX]?= 1 ^3i^Wj) iy\%•) if X is Hermitian. Moreover, we have the following, compare with Theorem 2.45. 4.6 Proposition. Let A : X —^ X be a self-adjoint operator in a Euclidean or Hermitian space X of dimension n and let A € Mn^n{^) be the matrix associated to A in a given orthonormal basis. Then A is similar to a diagonal matrix. More precisely, let {ui, 1*2,..., Un) be a basis of X of eigenvectors of A, /et Ai, A2,.. •, An E M be the corresponding eigenvalues and let S G Mn,n{^) be the matrix that has the n-tuple of components of Ui in the given orthonormal basis as the ith column. S :=
U i U2
Ur,
Then S^S = Id
and
S^AS = diag (Ai, A2,..., An)
114
4. Self-Adjoint Operators
if X is Euclidean, and S^S = Id
and
S^AS = diag (Ai, A2,..., A^).
if X is Hermitian. Proof. Since the columns of S are orthonormal, it follows that S ^ S = Id if X is Euclid—T
ean or S S = Id if X is Hermitian. The rest of the proof is contained in Theorem 2.45. D
c. Spectral resolution Let A : X —^ X he 3. self-adjoint operator on a Euclidean or Hermitian space X of dimension n, let (i^i, 1*2,..., Un) be an orthonormal basis of eigenvectors of A, let Ai, A2,..., A^; be the distinct eigenvalues of A and Vi, V2,..., Vk the corresponding eigenspaces. Let Pi : X -^ Vi he the projector on Vi so that
UjEVi
and by (ii) Proposition 4.4 k
A{x) = J2XiPi{x). i=l
As we have seen, by (iii) Proposition 4.4, we have Vi L Vj ii i •=^ j and, by the spectral theorem, Y2i=i dimV^ = n. In other words, we can say that {Vi}i is a decomposition of X in orthogonal subspaces or state the following. 4.7 Theorem. Let A : X -^ X be self-adjoint on a Euclidean or Hermitian space X of dimension n. Then there exists a unique family of projectors Pi, P25. • • 5 Pk CL'^d distinct real numbers Ai, A2,. •., A^^ such that k
p. o Pj = 6ijPj,
^
k
Pi = Id
and A = ^
XiPi.
Finally, we can easily complete the spectral theorem as follows. 4.8 Proposition. Let X be a Euclidean or Hermitian space. A linear opertor A : X -^ X is self-adjoint if and only if the eigenvalues of A are real and there exists an orthonormal basis of X made of eigenvectors of A.
4.1 Elements of Spectral Theory
115
d. Quadratic forms To a self-adjoint operator A : X —^ X we may associate the bilinear form a:X xX ^K, a{x,y) := {A{x)\y),
x,y e X,
which is symmetric if X is EucUdean and sesquilinear^ tt(^?y) = o.{y^x)^ if X is Hermitian. 4.9 Theorem. Let A : X —^ X be a self-adjoint operator, (ei, e2, • . . , e^) an orthonormal basis of X of eigenvectors of A and Ai, A2,. •., A^ be the corresponding eigenvalues. Then n
{A{x)\x) =Y.Xi\{x\ei)f
Vx e X,
(4.1)
2=1
In particular, if Amin ^nd Amax CiT'e respectively, the smallest and largest eigenvalues of A, then A m i n k l ^ < {A{x)\x)
< A max 1^1
Vx G X.
Moreover, we have {A{x)\x) = Amin |^P (resp. {A{x)\x) = Amax kPy^ if and only if x is an eigenvector with eigenvalue Amin (resp. Xmax)Proof. Proposition 4.5 yields (4.1) hence n
n
i=l
2=1
and, since l^p = J27=i l ( ^ k i ) P ^^ G X, the first part of the claim is proved. Let us prove that {A(x)\x) = Amin 12^ P if and only if x is an eigenvector with eigenvalue Amin- If x is an eigenvector of A with eigenvalue Amin) then A{x) = Amin 2: hence {A{x)\x) = (Amin^:^!^) = Amin k p . Conversely, suppose (ei, 6 2 , . . . , Cn) is a basis of X made by eigenvectors of A and the eigenspace Vx^-^^ is spanned by (ei, 6 2 , . . . , e^). Prom {A(x)\x) = AminkP^ we infer that 0 = iA(x)\x)
n - AminkP = E ( ^ ^ i=l
^min)|(x|ei)P
and, as AjAmin ^ 0, we get that (x|ei) = 0 Vz > A:, thus x G V\^.^. We proceed similarly for Amax.
D
All eigenvalues can, in fact, be characterized as in Theorem 4.9. Let us order the eigenvalues, counted with their multiplicity, as Ai < A2 < • • • < An and let (ei, e 2 , . . . , en) be an orthonormal basis of corresponding eigenvectors (ei, 6 2 , . . . , en), A{ei) = XiCi Vz = 1 , . . . , n; finally, set Vk := Span{ei, e 2 , . . . , e^},
Wk := {efc,efc+i,... ,en}.
116
4. Self-Adjoint Operators
Since T4, Wk are invariant subspaces under A and Vj^ = Wk-\-i, by applying Theorem 4.9 to the restriction of {A{x)\x) on 14 and VF^, we find Ai = mm{A{x)\x), 1x1=1 Xk = max< {A{x)\x)
(4.2) |x| = 1, x G Vjt >
= min< {A{x)\x)
|a:| = 1, x e Wk \
if A: = 2 , . . . , n — 1,
An = max(A(x)|x). kl=i Moreover, if 5 is a subspace of dimension n—fc-fl, we have 5014 ¥" {0}? then there is XQ ^ S ilVk with |a;o| = 1; thus min|(A(x)|x) |x| = 1, x e s\ < {A{xo)\xo) < max< {A{x)\x)
|x| = 1, x eVk\
= Xk-
Since dim W4 = n — k -\-1 and mmxeWk{^{^)\^) = A^, we conclude with the min-max characterization of eigenvalues that makes no reference to eigenvectors. 4.10 Proposition (Courant). Let A he a self-adjoint operator on a Euclidean or Hermitian space X of dimension n and let Xi < X2 < - - - < Xn be the eigenvalues of A in nondecreasing order and counted with multiplicity. Then Xk =
max
min<^ (A(x)\x)
dimS=n-k-\-l
=
\x\ = 1. x e S>
l^v V / I / I I I
min m a x n A ( x ) b ) dimS=k
J
\x\ =
l^v V / I / I I I
l,xeS>. J
4.11 A variational algorithm for the eigenvectors. Prom (4.2) we know that Afc :=min{(A(x)|x)| |x| = 1, x G 14-^1},
A: = l . . . , n ,
(4.3)
where F_i = {0}. This yields an iterative procedure to compute the eigenvalues of A. For j = 1 define Ai =
m.m{A{x)\x), kl=i
and for j = 1 , . . . , n — 1 set Vj := eigenspace of Aj, Aj+i := min< {A{x)\x) | |x| = 1, x e Wj >.
4.1 Elements of Spectral Theory
117
Notice that such an algorithm yields an alternative proof of the spectral theorem. We shall see in Chapter 10 that this procedure extends to certain classes of self-adjoint operators in infinite-dimensional spaces. Finally, notice that Sylvester's theorem, Gram-Schmidt's procedure or the other variants for reducing a quadratic form to a canonical form, see Chapter 3, allow us to find the numbers of positive, negative and null eigenvalues (with multiplicity) without computing them explicitly. e. Positive operators A self-adjoint operator A : X -^ X is called positive (resp. nonnegative) if the quadratic form (j){x) := (Ax|x) is positive for x ^ 0 (resp. nonnegative). Prom the results about metrics, see Corollary 3.56, or directly from Theorem 4.9, we have the following. 4.12 Proposition. Let A: X ^^ X be self-adjoint. A is positive (nonnegative) if and only if all eigenvalues of A are positive (nonnegative) or iff there is X> 0 (X>0) such that {Ax\x) > A|xp. 4.13 Corollary. A : X ^^ X is positive self-adjoint if and only if a{x,y) = {A{x)\y) is an inner (Hermitian) product on X. 4.14 Proposition (Simultaneous diagonalization). Let A^M : X ^>X be linear self-adjoint operators on X. Suppose M is positive. Then there exists a basis (ei, e 2 , . . . , e^) of X and real numbers Ai, A2,..., An such that {M{ei)\ej) = 6ij, A{ej) = XjMcj \/iJ = 1 , . . . ,n. (4.4) Proof. The metric g{x,y) := (M(x)\y) is a scalar (Hermitian) product on X and the Unear operator M~^A : X —> X is self-adjoint with respect to g since g{M-^A{x),y)
= {MM-''A(x)\y)
= {A{x)\y) =
= {x\MM-'^A(y))
= (Mx\M-'^A(y))
{x\A{y)) =
g{x,M-'^A{y)).
Therefore, M~^A has real eigenvalues and, by the spectral theorem, there is a gorthonormal basis of X made of eigenvectors of M ~ ^ A , g{ei,ej)
= {M{ei)\ej)
= Sij,
M~^A(ej)
= XjCj \/i,j = 1 , . . . , n .
4.15 Remark. We cannot drop the positivity assumption in Proposition 4.14. For instance, if
we have det(AId — M~^A) = A^ + 1, hence M~^A has no real eigenvalue.
118
4. Self-Adjoint Operators
4.16 %. Show the following. P r o p o s i t i o n . Let X be a Euclidean space and let g,b : X x X —^R be two metrics on X. Suppose g is positive. Then there exists a basis of X that is both g-orthogonal and b-orthogonal. 4 . 1 7 ^ . Let A,M be linear self-adjoint operators and let M be positive. Then M~^A is self-adjoint with respect to the inner product g{x,y) := (M{x)\y). Show that the eigenvalues Ai, A2, • •., An of M~^^A are iteratively given by Ai =
min
g(x,x) = l
g{M
A(x))x
= min
Xyt^O
{M{x)\x)
and for J = 1 , . . . , n — 1 I V^ := eigenspace of M~^A IWj
relative to Aj,
:=iVi®V2e'-'eVj)-^,
[Xj+1 := mm{{A{x)\x)
\ {M{x)\x)
= 1, x e
Wj},
where V-*- denotes the orthogonal to V with respect to the inner product g. 4 . 1 8 f.
Show the following.
P r o p o s i t i o n . Let T be a linear operator on K^. IfT-\-T* is positive then all eigenvalues of T have positive (nonnegative) real part.
(nonnegative),
f. The operators A* A and A A* Let A : X -^Y he Si linear operator between X and Y that we assume are either both Euclidean or both Hermitian. Prom now on we shall write Ax instead of A{x) for the sake of simplicity. As usual, A* : Y ^ X denotes the adjoint of A. 4.19 Proposition. The operator A* A : X -^ X is (i) self-adjoint, (ii) nonnegative, (iii) Ax, A*Ax and {A*Ax\x) are all nonzero or all zero, in particular A* A is positive if and only if A is infective, (iv) if ui^ i/2,..., Un are eigenvectors of A* A respectively, with eigenvalues Ai, A2,..., An; then {Aui\Auj) =
\i{ui\uj),
in particular, if ui, U2,. -., Un are orthogonal to each other, then Au\,..., Aun are orthogonal to each other as well. Proof
(i) In fact, {A*A)* = A*A** = A*A.
(ii) and (iii) If Ax = 0, then trivially A*Ax = 0, and if A*Ax = 0, then (A*Ax\x) = 0. On the other hand, {A*Ax\x) = {Ax\Ax) = | ^ x p hence Ax = 0 if (A*Ax\x) = 0. (iv) In fact, {Aui\Auj)
= {A*Aui\uj)
= \i{ui\uj)
= Xi\ui\'^Sij.
D
4.1 Elements of Spectral Theory
4.20 Proposition. The operator A A* :Y ^^Y
119
is
(i) self'adjoint, (ii) nonnegative, (iii) A*x, AA*x and {AA*x\x) are either all nonzero or all zero, in particular AA* is positive if and only if ker A* = {0}, equivalently if and only if A is surjective. (iv) if ui, U2j' >., Un are eigenvectors of AA"^ with eigenvalues respectively, Ai, A2,..., Xny then {A''ui\A*Uj) = Xi{ui\uj), in particular, if ui, U2,.. - ^ Un are orthogonal to each other, then A*ui^..., A*Un are orthogonal to each other as well Moreover, A A* and A* A have the same nonzero eigenvalues and Rank^Tl* = R a n k ^ M = Rank A = Rank A*. In particular, Rank A A* = Rank A* A < min(dim X, dim F ) . Proof. The claims (i) (ii) (iii) and (iv) are proved as in Proposition 4.19. To prove that A* A and A A* have the same nonzero eigenvalues, notice that if X € A", x 7«^ 0, is an eigenvalue for A*A with eigenvalue A 7^ 0, A*Ax = Xx, then Ax 7^ 0 by (iii) Proposition 4.19 and AA*{Ax) = A{A*Ax) = A{Xx) = XAx, i.e., Ax is a nonzero eigenvector for A A* with the same eigenvalue A. Similarly, one proves that if 2/ 7^ 0 is an eigenvector for A A* with eigenvalue X ^ 0, then by (iii) A*y ^ 0 and A*y is an eigenvector for A* A with eigenvalue A. Finally, from the alternative theorem, we have Rank A A* = Rank A* = Rank A = Rank A M .
g. Powers of a self-adjoint operator Let A : X -^ X he self-adjoint. By the spectral theorem, there is an orthonormal basis (ei, 6 2 , . . . , e-n) of X and real numbers Ai, A2,..., A^ such that n
Ax = 2_[^j{^\^j)^j
^^ ^ ^'
By induction, one easily computes, using the eigenvectors ei, 6 2 , . . . , Cn and the eigenvalues Ai, A2,..., An of A the /c-power oi A, A^ :=^ Ao- - -oA k times, V/c > 2, as n
A^x = Y,i^i)H^\ei)
ei
Vx e X
(4.5)
i=l
4.21 Proposition. Let A: X ^^ X be self-adjoint and fc > 1. Then (i) A^ is self-adjoint, (ii) A is an eigenvalue for A if and only if A^ is an eigenvalue for A^,
120
4. Self-Adjoint Operators
(iii) X E X is an eigenvector of A with eigenvalue A if and only if x is an eigenvector for A^ with eigenvalue X^. In particular, the eigenspaces of A relative to A and of A^ relative to X^ coincide. (iv) / / A is invertihle, equivalently, if all eigenvalues of A are nonzero, then 1
A~^x = 22 T~(^kO ^i
^^ ^ ^'
.=1 ^^ 4.22 % Let A: X -^ X he self-adjoint. Show that
li p{t) = YlT=i ^kt^ ^^ ^ polynomial of degree m, then (4.5) yields m
m
P{A){x) = Y^akA^x)
n
n
= ^5^afcA^^(x|e,)e,- = 5^p(A,)(x|e,)e,-. (4.6)
k=l
k=lj=l
j=l
4.23 Proposition. Let A : X -^ X be a nonnegative self-adjoint operator and let k E N, k > 1. There exists a unique nonnegative self-adjoint operator B : X -^ X such that B'^^ = A. Moreover, B is positive if A is positive. The operator B such that B'^^ = A \s called the 2A;th root of A and is denoted by ^ \ / A . Proof. If A{x) ^ X;7=i ^jiA^j^j^
(4.5) yields B^^ == A for n
Uniqueness remains to be shown. Suppose B and C are self-adjoint, nonnegative and such that A = B"^^ = C^'^. Then B and C have the same eigenvalues and the same eigenspaces by Proposition 4.21, hence B = C. •
In particular, if A : X —^ X is nonnegative and self-adjoint, the operator square root of A is defined by n \/A{X)
:= ^2 V ^ ( ^ l ^ i ) ^ J '
X e X,
i=l
if A has the spectral decomposition Ax = X]^=i
^j{^\^j)^j'
4 . 2 4 %. Prove Proposition 4.14 by noticing that, if A and M are self-adjoint and M is positive, then M~'^/^AM~'^/^ : X —>• X is well defined and self-adjoint. 4 . 2 5 ^ . Let A,B be self-adjoint and let A be positive. Show that B is positive if S := AB -h BA is positive. [Hint: Consider A~^/^BA~^/2 and apply Exercise 4.18.]
4.1 Elements of Spectral Theory
121
4.1.2 Normal operators a. Simultaneous spectral decompositions 4.26 Theorem. Let X be a Euclidean or Hermitian space. If A and B are two self-adjoint operators on X that commute, A = A\
B = B\
AB = BA,
then there exists an orthonormal basis (ei, 6 2 , . . . , Cn) on X of eigenvectors of A and B, hence n
n
z = ^{z\ei)ei,
Az = ^Xi{z\ei)ei,
2=1
1=1
n
Bz =
^fii{z\ei)ei, i—1
Ai, A2,..., An G M and /ii, /Li2,..., /in ^ I^ being the eigenvalues respectively of A and B. This is proved by induction as in Theorem 4.3 on account of the following. 4.27 Proposition. Under the hypoteses of Theorem 4-26, we have (i) A and B have a common nonzero eigenvector, (ii) if V is invariant under A and B, then V-^ is invariant under A and B as well. Proof, (i) Let A be an eigenvalue of A and let V\ be the corresponding eigenspace. For all y € Vx we have ABy = BAy = XBy, i.e., By G V^. Thus V^ is invariant under B , consequently, there is an eigenvector w £ Vx oi B^y^, i.e., common to A and B. (ii) For every w G V-^ and z £ V, we have Az,Bz G V and {Aw\z) = {w\Az) = 0, {Bw\z) = {w\Bz) = 0, i.e.. Aw, Bw eV-^. • 4 . 2 8 %. Show that two symmetric matrices A, B are simultaneously diagonizable if and only if they commute A B = B A .
b. Normal operators on Hermitian spaces A linear operator on a Euclidean or Hermitian space is called normal if NN* = N*N. Of course, if we fix an orthonormal basis in X, we may represent N with an n x n matrix N 6 Mn,n(C) and N is normal if and only if N N ^ = N ^ N if X is Hermitian or N N ^ = N ^ N if X is Euclidean. The class of normal operators, though not trivial from the algebraic point of view (it is not closed for the operations of sum and composition), is interesting as it contains several families of important operators as subclasses. For instance, self-adjoint operators A^ = A^*, anti-self-adjoint operators N* = —N, and isometric operators, N*N = Id, are normal operators. Moreover, normal operators in a Hermitian space are exactly the ones that are diagonizable. In fact, we have the following.
122
4. Self-Adjoint Operators
4.29 Theorem (Spectral theorem). Let X be a Hermitian space of dimension n and let N : X -^ X he a linear operator. Then N is normal if and only if there exists an orthonormal basis of X made by eigenvectors ofN. Proof. Let (ei, 6 2 , . . . , Cn) be an orthonormal basis of X made by eigenvectors of N. Then for every z £ X n
n
Nz = ^Xj{z\ej)ej,
N*z =
j= l
^^{z\ej)ej 3= 1
hence NN*z = XI^^i |AjP(^|e-,)ej- = N*Nz. Conversely, let N + N* N -N* A:= — , B := . 2 2i It is easily seen that A and B are self-adjoint and commute. Theorem 4.26 then yields a basis of orthonormal eigenvectors of A and B and therefore of eigenvectors oi N := A+ iB and N* = A- iB. D 4.30 1 . Show that AT : C"^ -^ C"^ is normal if and only if N and N* have the same eigenspaces.
c. Normal operators on Euclidean spaces Let us translate the information contained in the spectral theorem for normal operators on Hermitian spaces into information about normal operators on Euclidean spaces. In order to do that, let us first make a few remarks. As usual, in C^ we write z — x+iy, x,y EW^ for z — {x\ -\-iyi,..., Xn + iyn)' If VF is a subspace of W^, the subspace of C"^ WeiW
:= (zeC'^\z
= x-h iy, x,y
is called the complexified ofW. Trivially, A\mc{W®iW) if F is a subspace of C^, set V
ew\ = dim^ W. Also,
:={ZGC^|ZGF}.
4.31 Lemma. A_subspace V C C^ is the complexified of a real subspace W if and only ifV = V. Proof. liV vectors
= W^
iW, trivially V = V. Conversely, if 2 € F is such that z e V, the
''have real coordinates. Set
z -\- z 2 '
^ -
W :=^xeW\x=
z — z {z/i) -h z/i 2i 2
^ ^ , 2€ V};
then it is easily seen that V = W ® iW li V = V.
4.1 Elements of Spectral Theory
123
For N : M^ ^ W^ we define its complexified as the (complex) linear operator Nc : C -^ C defined by Nc{z) := Nx + iNy iiz = x-\-iy. Then we easily see that (i) A is an eigenvalue of N if and only if A is an eigenvalue of Nc^ (ii) N is respectively, a self-adjoint, anti-self-adjoint, isometric or normal operator if and only if Nc is respectively, a self-adjoint, anti-selfadjoint, isometric or normal operator on C"^, (iii) the eigenvalues of N are either real, or pairwise complex conjugate; in the latter case the conjugate eigenvalues A and A have the same multiplicity. 4.32 Proposition. Let N : W^ ^^ W^ be a normal operator. Every real eigenvalue \ of N of multiplicity k has an eigenspace Vx of dimension k. In particular, V\ has an orthonormal basis made of eigenvectors. Proof. Let A be a real eigenvalue for NQ, NQZ = Xz. We have NQZ
— Nx - iNy = N^z = Xz = Xz,
i.e., z € C^ is an eigenvector of N^ with eigenvalue A if and only if 'z is also an eigenvector with eigenvalue A. The eigenspace Ex of Nc relative to A is then closed under conjugation and by Lemma 4.31 Ex '•= Wx © iWx, where
Wx:={xeR''\x=^,
z-\- z
zeEx],
and dimR Wx — dime ^x • Since N^ is diagonizable in C and W\ C VA ? we have k — dime Ex = dim Wx < dimR V^. As dimV^ ^ k, see Proposition 2.43, the claim follows.
D
4.33 Proposition. Let X be a nonreal eigenvalue of the normal operator N :W^ —^W^ with multiplicity k. Then there exist k planes of dimension 2 that are invariant under N. More precisely, i / e i , e 2 , . . . , en G C"^ are k orthonormal eigenvectors that span the eigenspace Ex of Nc relative to A and we set U2J-1 : =
Cj -\- Cj 7=—,
V2 '
U2j :=
'''
^.— ^.
V2i '
then lii, ii2,. • •, U2k Q'Te orthonormal in W^, and for j = 1,... ,k the plane Span{iA2j_i5^2j}? is invariant under N; more precisely we have
{
N{u2j-l)
= OLU2J-1 - fiU2j,
N{u2j) = /3u2j-i + au2j
where X=: a-\- i/3. Proof. Let Ex, Ej be the eigenspax:es of Nc relative to A and A. Since Nc is diagonizable on C, then ^A -L ^Jdime ^A = dime -^X ~ ^' On the other hand, for z ^ Ex
124
4. Self-Adjoint Operators
Ncz = Nx — iNy = N^z =
\z.
Therefore, z ^ Ex'ii and only if 2 G E-^. The complex subspgice F\ := Ex®Ej of C^ has dimension 2k and is closed under conjugation; Lemma 4.31 then yields Fx = Wx ^iWx where Wx:=[xeR''\x=
^ ^ ,
zeEx\
and
dimR Wx = dime E = 2k.
(4.7)
If (ei, 6 2 , . . . , efc) is an orthonormal basis of Ex, (ei, 6 2 , . . . , e^) is an orthonormal basis of Ej; since y/2ej
=: U2J-1
-\-iu2j,
V^ej = : U2j-1
• ^y'2j,
we see that {uj} is an orthonormal basis of Wx- Finally, if A := a -f- z/?, we compute
= ^+ Ae-
(Niu2j-i) = Nc{^) \N{U2J)
= Af ( ^ )
= ^ ^ ^
• = aU2j-l
- 0U2j,
= • • • = /3«2,-l + a « 2 „
i.e., Span {tt2j-1, W2j} is invariant under N.
D
Observing that the eigenspaces of the real eigenvalues and the eigenspaces of the complex conjugate eigenvectors are pairwise orthogonal, from Propositions 4.32 and 4.33 we infer the following. 4.34 Theorem. Let N be a normal operator on M.^. Then R^ is the direct sum of 1-dimensional and 2-dimensional subspaces that are pairwise orthogonal and invariant under N. In other words, there is an orthonormal basis such that the matrix N associated to N in this basis has the block structure 0 \ 0 A Ai N'
0
0
0
0
To each real eigenvalue A of multiplicity k correspond k blocks A of_dimension 1 x 1 . To each couple of complex conjugate eigenvalues A, A of multiplicity k correspond fc 2 x 2 blocks of the form a -a
(3 a
where a + if3 := A. 4.35 Corolleiry. Let A/^: M" —> R" be a normal operator. Then (i) N is self-adjoint if and only if all its eigenvalues are real, (ii) A'' is anti-self-adjoint if and only if all its eigenvalues are purely imaginary (or zero), (iii) N is an isometry if and only if all its eigenvalues have modulus one. 4 . 3 6 % Show Corollary 4.35.
4.1 Elements of Spectral Theory
125
4.1.3 Some representation formulas a. The operator A* A Let yl: X —> y be a linear operator between two Euclidean spaces or two Hermitian spaces and let ^* : y ^ ' X be its adjoint. As we have seen, yl*A : X —> X is self-adjoint, nonnegative and can be written as n
A" Ax =
Y^Xi{x\ei)ei 2=1
where (ei, 6 2 , . . . , Cn) is a basis of X made of eigenvectors of A*A and for each 2 = 1 , . . . , n Ai is the eigenvalue relative to e^; accordingly, we also have {A*Ay^^x
:= ^
fJ.i{x\ei) e^.
2=1
where /Xi := ^/Xi. The operator (A*A)^/^ and its eigenvalues / i i , . . . ,/Xn, called the singular values of A, play an important role in the description of A 4 . 3 7 f. Let A G Mm,n{^)' Show that ||A|| := sup|a.|^i |Ax| is the greatest singular value of A. [Hint: | A x p = (A* Ax) •x .]
4.38 Theorem (Polar decomposition). Let A : X ^^Y between two Euclidean or two Hermitian spaces.
be an operator
(i) If dimX < d i m y , then there exists an isometry U : X -^ Y, i.e., tf'U = Id, such that Moreover, if A = US with f/*C/ = Id and S* = S, then S = (A* A)^/^ and U is uniquely defined on ker S-^ = ker A-^. (ii) If dimX > dim.Y, then there exists an isometry U : Y ^y X, i.e., U*U = Id such that A = {AA*y^^U\ Moreover, ifA = SU with U*U = Id and 5* = S, then S = {AA*)^^'^ and U is uniquely defined on ker 5-^ = I m ^ . Proof. Let us show (i). Set n := d i m X and N := d i m y . First let us prove uniqueness. If A = 175 where U*U = Id and 5* = S, then A*A = S*U*US = S*S = ^ 2 , i.e., S = (A*A)i/2. Now from A = U(A*A)^/^, we infer for i = 1 , . . . , n Aid)
= t/(A*A)i/2(ei) = Uimei)
=
fiiU{ei),
if (ei, 6 2 , . . . , en) is an orthonormal basis of X of eigenvectors of {A*A)^^^ with relative eigenvalues ^ 1 , /JL2, . • •, /Xn- Hence, U(ei) = —A(ei) if/Ltj ^ 0, i.e., U is uniquely defined by A on the direct sum of the eigenspaces relative to nonzero eigenvalues of (A* A)^/^, that is, on the orthogonal of ker(A*A)^/2 = ker A. Now we shall exhibit U. The vectors A{ei),..., A{en) are orthogonal and |A(ei)| = Mi as
126
4. Self-Adjoint Operators
{A{ei)\A{ej))
= {A*A{ei)\ej)
= fJLi{ei\ej) = f^i6ij.
Let us reorder the eigenvectors and the corresponding eigenvalues in such a way that for some k, 1 < k < n, the vectors A{ei),..., A{ek) are not zero and A(ek-\-i) = - • - = A{en) = 0. For i = 1 , . . . , A: we set i;i := , . r ^ ^ and we complete t;i, t ; 2 , . . . , ^^fc to form a new orthonormal basis (vi^ V2,-.., VN) oiY. Now consider U : X -^Y defined by U(ei) :=Vi
i=l,...,n.
By construction {U{ei)\U{ej)) — Sij, i.e., U*U = Id, and, since fXi = \A{ei)\ = 0 for i > k, we conclude for every i = 1,... ,n
I yrt^t;^ = Ovi = 0
if k < i < n
(ii) follows by applying (i) to ^ * .
D
b. Singular value decomposition Combining polar decomposition and the spectral theorem we deduce the so-called singular value decomposition of a matrix A. We discuss only the real case, since the complex one requires only a few straightforward changes. Let A G MM,nW with n < N. The polar decomposition yields A = U(A^A)i/2
^i^j^
uTu _ jj
On the other hand, since A-^A is symmetric, the spectral theorem yields S e Mn.nW such that (A^A)i/2 = S^diag(/ii, / i 2 , . . . , /in)S,
S^S = Id,
where /ii, /X25 • • •, /^n are the squares of the singular values of A. Recall that the ith column of S is the eigenvector of (A*A)^/^ relative to the eigenvalue fii. then T ^ T = Id, In conclusion, if we set T := U S ^ G MNA^), S^S = Id and A = Tdiag(/ii, / i 2 , . . . , /in)S. This is the singular value decomposition of A, that is implemented in most computer hbraries on linear algebra. Starting from the singular value decomposition of A, we can easily compute, of course, (A-^A)^/^, and the polar decomposition of A. 4.39 . We notice that the singular value decomposition can be written in a more symmetric form if we extend T to a square orthogonal matrix V G MN,N(^), V ^ V = Id and extending diag (/xi, /i2, • • •, /in) to a A/^ x n matrix by adding N — n null rows at the bottom. Then, again
A = VAS where V G MATXATW, V ^ V = Id, S G Mn,n(^),
S ^ S = Id and
4.1 Elements of Spectral Theory
0
/MI 0
/i2
0
0 0
0 0
Mn
\0
0
A =
127
0
0/
c. The Moore-Penrose inverse Let A : X —^Y he 3. linear operator between two Euclidean or two Hermitian spaces of dimension respectively, n and m. Denote by P:X
^kevA^
Q:Y-^lmA
and
the orthogonal projections operators to kevA-^ and 1mA. Of course Ax = Qy has at least a solution x G X for any y ^Y. Equivalently, there exists X E X such that y — Ax ± Im ^ . Since the set of solutions of Ax — Qy is a translate of ker A, we conclude that there exists a unique x := A'^y E X such that y — Ax 1. Im A, [x e
Ax = Qy,
equivalently,
(4.8)
X = Px.
kerA^,
The linear map A"^ :Y ^^ X, y —^ A^y, defined this way, i.e.,
is called the Moore-Penrose inverse oi A: X ^^Y. {AA^
From the definition
=Q,
A^A = P, ker A+ =lmA^ ImA^ =
= kerQ,
keiA^.
4.40 Proposition. A^ is the unique linear map B :Y -^ X such that AB = Q,
BA = P
and
kerB = keTQ',
(4.9)
moreover we have A^AA"^ =A^AA''
=A\
(4.10)
128
4. Self-Adjoint Operators
Proof. We prove that B = A^ by showing for s\\ y £ Y the vector x := By satisfies (4.8). The first equaUty in (4.9) yields Ax = ABy = Qy and the last two imply x = By = BQy = BAx = Px. Finally, from AA^ = Q and A^A = P^ we infer that A*AA^ = A*Q = A\
A^AA* = PA* = A*,
using also that A*Q = A* and PA* = A* since A and A* are such that ImA (kerA*)-^ and ImA* = kerA-^.
= D
The equation (4.10) allow us to compute A^ easily when A is injective or surjective. 4.41 Corollary. Let A : X ^^ Y be a linear map between Euclidean or Hermitian spaces of dimension n and m, respectively. (i) If ker A = {0}, then n <m, A* A is invertible and A^ =
{A*A)-^A*;
moreover^ if A = [/(A*A)^/^ is the polar decomposition of A, then A^ = {A*A)-^/^U\ (ii) If ker A* = {0}, then n>m, AA* is invertible, and
moreover, if A = {AA*)^^'^U* is the polar decomposition of A, then At = C/(AA*)-V2. For more on the Moore-Penrose inverse, see Chapter 10.
4.2 Some Applications In this final section, we illustrate methods of linear algebra in a few specific examples.
4.2.1 The method of least squares a. The method of least squares Suppose we have m experimental data yi, 2/2, • • •, Vm when performing an experiment of which we have a mathematical model that imposes that the data should be functions, (/>(x), of a variable x e X. Our problem is that of finding x G X in such a way that the theoretical data 0(x) be as close as possible to the data of the experiment. We can formahze our problem as follows. We list the experimental data as a vector y = (yi, 2/2, • • •, 2/m) G W^ and represent the mathematical
4.2 Some Applications
129
model as a map 0 : X —> W^. Then, we introduce a cost function C = C{(j){x)^y) that evaluates the error between the expected result when the parameter is x, and the experimental data. Our problem then becomes that of finding a minimizer of the cost function C. If we choose (i) the model of the data to be linear^ i.e., X is a vector space of dimension n and (j) = A\ X -^ W^ is a linear operator, (ii) as cost function, the function square distance between the expected and the experimental data, C{x) = \Ax - 2/|2 = {Ax - y\Ax - y),
(4.11)
we talk of the {linear) least squares problem. 4.42 Theorem. Let X and Y he Euclidean spaces, A \ X ^^ Y a linear map, y EY and C : X ^^R the function C{x) := \Ax-y\Y^
x e X.
The following claims are equivalent (i) X is a minimizer of C, (ii) y - Ax ± 1mA, (iii) X solves the canonical equation A*{Ax-y)
= 0.
(4.12)
Consequently C has at least a minimizer in X and the space of minimizers of C is a translate o/ker A. Proof. Clearly minimizing is equivalent to finding z = Ax G I m A of least distance from y. By the orthogonal projection theorem, x is a minimizer if and only if Ax is the orthogonal projection of y onto Im A. We therefore deduce that a minimizer x G X for C exists, that for two minimizers xi,X2 of C we have Axi = Ax2, i.e., x i — X2 6 ker A and that (i) and (ii) are equivalent. Finally, since ImA-*- = kerA*, (ii) and (iii) are clearly equivalent. • 4 . 4 3 R e m a r k . The equation (4.12) expresses the fa-ct that the function x —>• | Aa: — 6p is stationary at a minimizer. In fact, compare 3.65, since Vx{z\x) = z and Vx(^x\x) = 2La; if L is self-adjoint, we have \Ax - 6|2 = |6|2 _ 2(6|Ax) + |Ax|2, V(6|Ax) = V(A*6|x) = A*6, Vx|Ax|2 = Vx{A*Ax\x)
=
2A*Xx
hence Vx\h-Ax\^
=
2A*{Ax-h).
As a consequence of Theorem 4.42 on account of (4.8) we can state the following 4.44 Corollary. The unique minimizer of C{x) = \Ax — y|y in Im A* = ker A-^ is X = A^y.
130
4. Self-Adjoint Operators
b. The function of linear regression Given m vectors xi, X2,. • •, Xm in a Euclidean space X and m corresponding numbers yi, 2/2, • • •, 2/m, we want to find a linear map L : X -^R that minimizes the quantity m
nL):='£\yi-Lixi)\\ i=l
This is in fact a dual formulation of the linear least squares problem. By Riesz's theorem, to every linear map L : X —> R corresponds a unique vector WL ^ X such that L{y) := {y\wL)j and conversely. Therefore, we need to find w G X such that m
C{w) := ^Ivi
- {xi\w)\'^ -^ min.
2=1
If y := (2/1, 2/2, • • •, 2/m) ^ R"^ and A: X —^ W^ is the linear map Aw := [{xi\w), {X2\w),...
{xn\w)j,
we are again seeking a minimizer of C : X —> M C{w) :=\y-Aw\^,
w e X.
Theorem 4.42 tells us that the set of minimizers is nonempty, it is a translate of ker A and the unique minimizer of C in ker A-^ = Im ^* is if; := A^y. Notice that n
A*a=j2
^i^i^
^=(«^
^^ • • -«"") ^ ^"^
2=1
hence, w £ IxnA'' = ker A"^ if and only if if; is a linear combination of xi, X2,..., Xm- We therefore conclude that A'^y is the unique minimizer of the cost function C that is a linear combination o / x i , 0:2,..., Xm- The corresponding linear map L{x) := {x\A^y) is called the function of linear regression.
4.2.2 Trigonometric polynomials Let us reconsider in the more abstract setting of vector spaces some of the results about trigonometric polynomials, see e.g.. Section 5.4.1 of [GM2]. Let Vn,2Tr be the class of trigonometric polynomials of degree m with complex coefficients n
Vn,2ir ••= [Pix) = ^
Cke''"' I Cfc e C, A; = - n , . . . , n } .
4.2 Some Applications
131
Recall that the vector {c-n,. "•,Cn) G C^"^"^^ is called the spectrum of P{^) — Z]fc=-n ^k^^^^' Clearly, Pn,27r is a vector space over C of dimension at most 2n + 1. The function (P|Q) : VU^-K X ^n,27r -^ C defined by
is a Hermitian product on Vn,2T^ that makes Pn,27r a Hermitian space. Since
(gifcx|^z/.x)=1. r ^i{k-h)x ^^ ^ ^^^^ see Lemma 5.45 of [GM2], we have the following. 4.45 Proposition. The trigonometric polynomials {e^^^}k=-n,n form an orthonormal set of2n-\-l vectors in Pn,27r o,nd we have the following. (i) T^n,27T is a Hermitian space of dimension 2n + 1. (ii) The map ^: 'Pn,27r -^ C^^"^^, that maps a trigonometric polynomial to its spectrum is well defined since it is the coordinate system in Vn,27r relative to the orthonormal basis {e'^^^}. In particular^ : Vn,2n -^ £2n-\-i j^g ^ (complex) isometry. (iii) (FOURIER COEFFICIENTS) For k = - n , ...,n we have
1 n
Cfe = (P|e"=") = — / (iv)
P{t)e-''''dt.
( E N E R G Y IDENTITY)
i- r \P{t)fdt = \\P\f := (P|P) = J2 |(P|e''=-)|2 = f; \Ck?. k=—n
k=—n
a. Spectrum and products Let P{x) = Y2=-n ^ke'^"" and Q{x) = Y2=-n dke'^'' be two trigonometric polynomials of order n. Their product is the trigonometric polynomial of order 2n
k=—n
/e=—n
h,k=—n
2n
= E ( E -Hd,)e'''. p=-2n
h-\-k=p
If we denote by {ck} * {dk} the product in the sense of Cauchy of the spectra of P and Q, we can state the following.
132
4. Self-Adjoint Operators
4.46 Proposition. The spectrum of P{x)Q{x) is the product in the sense of Cauchy of the spectra of P and Q (PQ)k = Pk^Qk4.47 Definition. The convolution product of P and Q is defined by p * Q{x) := ^
r
P{x +
t)Q{i)dt
^TT J-n
Notice that the operation (P^Q) ^^ P *Q is hnear in the first factor and antihnear in the second one. We have 4.48 Proposition. P^Q is a trigonometric polynomial of degree n. Moreover the spectrum of P^Q is the term-by-term product of the spectra of P and Q, {p7Q)^:=P,Qk. Proof. In fact for h, k = — n , . . . , n, we have
27r y_7r
27r J - T T
hence, if P{x) = Efc=_n Cfce^^=" and Q{x) = E f c = - n ^fce^''^, we have
P^Q{x)=
f^
f2 ^hdi^^hke'"''= Y.
h=—nk=—n
^fc^^'""-
k=—n
b. Sampling of trigonometric polynomials A trigonometric polynomial of degree n can be reconstructed from its values on a suitable choice of its values on 2n + 1 points, see Section 5.4.1 r - := — o^+li? 27r . J = "~^?..., n, then the sampling map of [GM2]. Set Xj C : Vn,2n
-- C ^ ^ + l ,
C{P)
:= ( P ( x _ n ) , • • • ,
is invertible, in fact, see Theorem 5.49 of [GM2], 1 '^ ^^""^ • " 2 ; r f T ^
P{xj)Dn{x-Xj)
3 = -n
where Dn{t) is the Dirichlet kernel of order n Dn{t)—
Yl
e^^* = l 4 - 2 ^ c o s A : t .
P{Xn))
4.2 Some Applications
133
Spectrum
Trigonometric polynomials of degree n
£2n-\-l
E
n
ikt
Samplings £2n+l
IDFT
Figure 4.1. The scenario of trigonometric polynomials, spectra and samples.
4.49 Proposition. K,27r given by
J^^^C
V2n-\-lC~'^(z)(x)
and its inverse yjin + 1C~^ : C^"''^^
:= , \^ ZiDJx - Xj) >/2nTT .^1^ -^ '
are isometries between Vn,2n O'Tid C^"^"^^. Proof. In fact, C maps e* '^*, k = — n , . . . , n, to an orthonormal basis of C^'^"'"^:
Prom the samples, we can directly compute the spectrum of P, 4.50 Proposition. Let P{x) e Vn,2n CL'^^d Xj := 2 ^ ^ j ? j = —n, . . . , n . Then
^ f^ P(t)e-'^' dt = ^ ^ J2 P{xj)e-''^^.
(4.13)
Proof. Since (4.13) is linear in P , it suffices to prove it when P{x) = e*'^*, h = — n , . . . , n. In this case, we have ^ J^^ P{t)e~'^ ^^ dt = 6hk ^^^
3=-n
since Dn{xj)
J=-n
= 0 for j ^ 0, j e [-n,n] and DnCO) = 2n + 1.
D
134
4. Self-Adjoint Operators
c. The discrete Fourier transform The relation between the values {P{tj)} of P e 'Pn,27r at the 2n + 1 points tj and the spectrum P of P in the previous paragraph is a special case of the so-called discrete Fourier transform. For each positive integer N^ consider the 27r-periodic function EN{t) : R -^ C given by „ /^N v^^ ikt 1 ^ EN{t):=2^e'^' = \ , k=o [ i-eit
if H s a multiple of 27r,
. . (4.14)
Otherwise.
Let uj = e*i5^ and let l,a;,c<;^,... ^uj^~'^ be the ATth roots of one. For h GZ we have — V cj^^ = < ^ if /i is a multiple of TV, -^ ^^Q [O otherwise,
,^ ^^.
in particular, iV-l
_ ^
u;^^ = (^;,fc
a -N
(4.16)
fc=0
The discrete Fourier transform of order N, DFTN defined by DPT/v(y) := U y rows by column, where U = [£/]],
C/j:=-iw-'^
Vi,i = 0 , . . . , J V - l .
The inverse discrete Fourier transform of order AT, := Vz where is defined by IDFTM{Z) V = [Vi],
V; = Ji,
- C ^ -^ C ^ , is
IDFTN
'- C^ —^ C ^ ,
Vz,j = 0 , 7 V - l .
4.51 Proposition. IDFTN is the inverse of DFTj\f. Moreover, the operators y/N DFTN and -4^ IDFT^ are isometrics ofC^. Proof. In fact, by (4,16)
fc=0 fc=0 i.e., V = U - i and, by the definition of U and V , U ^ = ; ^ V , hence U ^ = ^ V =
4.2 Some Applications
135
Notice that, according to their definitions, we need N'^ multiphcations to compute DFTN (or IDFTjsj). There is an algorithm, that we shall not describe here, called the Fast Fourier Transform that, using the redunwith only N dance of some multiphcations, computes DFTN (or IDFTN) multiplications with a performance of 0{N log N). Let P{t) = E L - n ^k^'^'' ^ ^n,27r and let AT > 2n + 1. A computation similar to the one in Proposition 4.50 shows that
^ ' ^ h £
^^^^'~'' ^^ ^ ^ ^
n^j^-'''^'
=
DFTMV
(4.17)
and Xj := ^ j , -N < j < N. Thus the where y := {P{xo),... ,P{XN)) spectrum of P is the DFT^ of its values at Xj if n < N/2. On the other hand, if z := (ZQ, • • •, ZN-I) is the vector defined by
Zk '=
and we recall that
Pk
if 0 < fc < n,
0
if n < A: < iV/2,
Pk-N
if N/2
0
if iV/2 + n < fc < TV
<
IDFTN
is the inverse of P{xj)
i.e., the values of P at
Xj are
:=
the
N/2 -f n,
DFTN,
we have
{IDFTNZ)J,
IDFTN
of the spectrum of P.
4.52 Frequency spectrum. In applications, the DFTN and IDFTN may appear slightly differently. If / is a To-periodic function, one lets To/AT be the period of sampling, so that tj := ^j = jT^ j = 0 , 1 , . . . , iV — 1, are the sampling points and DTFN produces the sequence
cfc := ^ E fiJT)e''^'''In other words, the values of {ck} are regarded as the values of the component of frequency Vk '•= k^- = - ^ , i.e., as the samples of the so-called frequency spectrum / : E ^^ C of / , defined by
\0
otherwise.
The discrete Fourier transform and its inverse then rewrite as
136
4. Self-Adjoint Operators
f{4f)'h"ff<-'^>'-'^" j=0 N-1
f('^^)=Ef{iky^^''j=o
4.2.3 Systems of difference equations Linear difference equations of first and second order are discussed e.g., in [GM2]. Here we shall discuss systems of linear difference equations. a. Systems of linear difference equations First let us consider systems of first order. Let A G Mk,k{C). The homogeneous linear recurrence for the sequence in C'^
{
Xn+i = AXn,
n > 0,
Xo given
has the unique solution Xn := A'^XQ Vn, as one can easily check. 4.53 Proposition. Given {Fn} in C^, the recurrence f X n + l = AXn
-h F n + i ,
n > 0,
[Xo given has the solution n
Xn := A^Xo + Y. ^""''^3
'^n>{)
3=0
where we assume FQ := 0. Proof. In fact, for n > 0 we have n+l
n
3=0
3=0
n
= A A^Xo + A ^ j=0
n
A ^ - ^ F , - + F n + i = A ( A ^ X O -h ^
A ^ - ^ F ^ ) + Fn+i
3=0
= AXn + Fn-l-l. D
4.2 Some Applications
137
4.54 Higher order linear difference equations. Every equation Xn-[-k H- dk-lXn-^k-l
H
h Go^n = /n+1,
n > 0
(4.18)
can be transformed into afcx A: system of difference equations of first order. In fact, if Xn := (Xn,Xn+l,...,Xn+fc-l)^ G C'',
Fn:=(0,0,...,0,/n^€C^ and A is the k x k matrix / 0 0
1 0
0 1
0 0
\ (4.19)
A:= 0 -ao
0 0 —oi —a2
-flfe-l
/
it is easily seen that ^n+1 — AX„ + Fn-\-l
(4.20)
and conversely, if {Xn} solves (4.20), then {xn}, Xn '-= X^ Vn, solves (4.18). In this way the theory of higher order linear difference equations is subsumed to that of first order systems. In this respect, one computes for the matrix A in (4.19) that k-i
det(AId - A) = A'^ + ^ a j X ^ . j=0
This polynomial in A is called the characteristic polynomial of the difference equation (4.18). b. Power of a matrix Let us compute the power of A in an efficient way. To do this we remark the following. (i) If B is similar to A, A = S'^BS for some S with det S 7^ 0, then A^ = S-^BSS-^BS = S-^B^S and, by induction, A " = S-^B"S
Vn.
138
4. Self-Adjoint Operators
(ii) If B is a block matrix with square blocks in the principal diagonal 0
Bi
0
Bo
B =
then
B" =
v
By
0
0
B?
0
0
B:
Let Ai, A2,..., A/j be the distinct eigenvalues of A with multiphcities mi, m 2 , . . . , mfc. For every k, let pk be the dimension of the eigenspace relative to A^ (the geometric multiplicity). Then, see Theorem 2.65, there exists a nonsingular matrix S € Mfe,fc(C) such that J := S~^AS has the Jordan form Ji,i
J==
v
0
0
0
Jl,2
0
...
0
0
...
\
'k,pk
where i = 1 , . . . , A;, j = 1 , . . . ,pi and /Ai 0
1 0 Ai 1
Xi 0 0
if Jij has dimension 1, ... ...
0\ 0
'^ij
otherwise. 0 Vo
0 0
0 0
... ...
Ai 0
1 Ai/
4.2 Some Applications
139
Consequently A^ = S J ^ S - \ and -tn
0
0
Tn •^1,2
0
0
J^ =
Tn
0
V It remains to compute the power of each Jordan block. If J ' = Jij = (A) has dimension one, then J'^ = A'^. If instead J ' = Jij is a block of dimension q at least two, /A 0
1 A
0 1
... ...
0\ 0
0 ... VO 0
0 ...
A 0
1 A/
J^ =
then J ' = AId + B ,
B}
:=Si+ij.
Since I Sr-^ij
C^)]
if r < g.
[0 ifr>9, we have B^ = 0. Thus Newton's binomial formula yields
J'^ = (Aid + B)^ = J2 ("")A"-^B^ 3=0 ^-^^
I.e.
j=0
^-^ ^
0
1
0 VO
3=0 ^-^^
n\
...
( " ) ^
0
1
n\ A
0
0
1
J ' " = A"
/
140
4. Self-Adjoint Operators
Notice that each element of A " = SJ^S~^ has the form k
3=1
where Ai, A2,..., A^ are the eigenvalues of A and Pj{t) is a polynomial of degree at most Pj — 1, where pj is the algebraic multiplicity of \j. It follows that for p > maxi{\\i\) there is a constant Cp such that every solution of Xn-^i = AXfi satisfies \Xn\ = sup < Cpp"" \Xo\
Vn.
In particular we have the following. 4.55 Theorem. If all eigenvalues of A have modulus less than one, then every solution of Xn^i = AXn converges to zero as n -^ +00. Proof. Fix cr > 0 such that maxi=i,n \^i\ < cr < 1. As we have seen, there exists a constant Ca such that if Xn is a solution of X n + i = AXn Vn, then \Xn\ < CaCT'' \Xol
Vn.
Since 0 < cr < 1, a"^ —^ 0, and the claim is proved.
D
4.56 E x a m p l e (Fibonacci n u m b e r s ) . Consider the sequence of Fibonacci numbers
(
/n+2 = /n+1 + / n
n > 0,
.^ ^^.
/O = 0, / i = 1, that is given by 1 / / l + \/5\n
/l-y/5\n
(4.22)
see e.g., [GM2]. Let us find it again as an application of the above. Set fn Un-i-1 then
F.,.H^^^M= /n+2/
'-''
\/n+/n+l7
0^ ' 1 and, F„=A»(; where 0 1
1 1
]=r
\1
^ ^n, 1
4.2 Some Applications
141
The characteristic polynomial of A is det(AId — A) = A(A — 1) — 1, hence A has two distinct eigenvalues l-hv/5 l-v/5 2 ' ^ 2 An eigenvalue relative to A is (1, A) and an eigenvector relative to /x is (1, /x). The matrix A is diagonizable a s A = S A S ~ ^ where
^X
IJLJ
A -/i
y A
-ly
\0
/x
It follows that
1
A
A — /x \^A
l \ / A'^ ii)
\—iJL^
Consequently,
'" = I^(--''") = 7!((H^)"-(^)") 4,2.4 An ODE system: small oscillations Let x i , X 2 , . . . , xjv be N point masses in M^ each respectively, to a nonzero mass m i , m 2 , . . . , TRN. Assume that each point exerts a force on the other points according to Hookers law^ i.e., the force exerted by the mass at Xj on Xi is proportional to the distance of Xj from x^ and directed along the line through x^ and Xj,
By Newton's reaction law, the force exerted by x^ on Xj is equal and opposite in direction, fji = ~fij^ consequently the elastic constants fc^j, i ^ j , satisfy the symmetry condition kij = kji. In conclusion, the total force exerted by the system on the mass at x^ is N X,j = l,N i^3
3 = 1,N i^j
where we set ku := — ^J=I,N
3 = 1,N iy^J
j=l
kij. Newton's equation then takes the form
mix'/-fi=0,
z = l,...,Ar,
(4.23)
with the particularity that the j t h component of the force depends only on the j t h component of the mass. The system then splits into 3 systems of N equations of second order, one for each coordinate. If we use the matrix notation, things simplify. Denote by M := diag {mi, m2,. • •, TUN}
142
4. Self-Adjoint Operators
the positive diagonal matrix of masses, by K := (fc^j) G MN,N{^) the symmetric matrix of elastic contants, and by X{t) € M^ the jth coordinates of the points x i , . . . , x^v A = (Xj^,..., x ^ j ,
x^ =: [Xj^, a^^, x^ j ,
i.e., the columns of the matrix X(t) := [xi(t) I X2(t) I . . . I Xiv(t)]
e Miv,3(M).
Then (4.23) transforms into the system of equations MX'\t)-\-KX{t) = 0
(4.24)
where the product is the product rows by columns. Finally, if X"(t) denotes the matrix of second derivatives of the entries of X(^), the system (4.23) can be written as X'\t) + M - ^ K X ( t ) = 0, (4.25) in the unknown X : R -^ Miv,n(R)Since M~^K is symmetric, there is an orthonormal basis of R ^ made by eigenvalues of M~^K and real numbers Ai, A2,..., AA/^ such that {ui\uj) = 5ij
and
M.~^Kuj := \jUj\
notice that i^i, ?X2? • • •, ^iv are pairwise orthonormal vectors since M is diagonal. Denoting by Pj the projection operator onto Span{uj} we also have N
Id = 5^P„
N
M-^K = X^A,P,.
3=1
j=i
Thus, projecting (4.25) onto SpanjiXj} we find 0 = Pj{0) = P , ( X " + M - ^ K X ) = {PjXy
+ A^(P,X),
Vj = 1 , . . . , iV,
i.e., the system (4.25) splits into A^ second order equations each in the unknowns of the matrix PjX.{t). Since K is positive, the eigenvalues are positive, consequently each element of the matrix PjX(t) is a solution of the harmonic oscillator
hence PjX{t) = cos(yA~t)P,X(0) + ! ^ ^ ^ i ^ P , . X ' ( 0 ) . In conclusion, since Id = ^j^i Pj-, we have
4.3 Exercises
143
N
x{t) = J2PiMt) ' '
(4.26)
The numbers \/A7/(27r),... \/A^/(27r) are called the proper frequencies of the system. We may also use a functional notation ^
A2n+1
^
\2n
and we can write (4.26) as X(i) = cos{t^/A)X{0)
+ ?H^i^^X'(0), vA
where A := M~^K.
4.3 Exercises 4 . 5 7 %. Let A be an n X n matrix and let A be its eigenvalue of greatest modulus. Show that |A| < sup^dail + | 4 | + • • • + |aj,|). 4.58 % G r a m m a t r i x . Let {/i, / 2 , . . . , fm} be m vectors in M^. The matrix G = [gij] G Mm,n(IR) defined by gij = {fi\fj) is called Gram's matrix. Show that G is nonnegative and it is positive if and only if / i , 72, • • • ? fm are linearly independent. 4.59 t . Let A,B : C^ -^ C^ be self-adjoint and let A be positive. Show that the eigenvalues of A~^B are real. Show also that A~^B is positive if B is positive. 4 . 6 0 %. Let A = [a^] G Mn,n(K) be self-adjoint and positive. Show that det A < (trA/n)"^ and deduce det A < n?=i<^i- [Hint: Use the inequality between geometric and aritmethic means, see [GMl].] 4.61 %, Let A e Mn,nOQ and let a i , a 2 , . . . , an G K"^ be the columns of A. Prove Hadamard's formula det A < f l i L i \^i\- [Hint: Consider H = A* A.] 4.62 f. Let A, B G Mn,n{^) be symmetric and suppose that A is positive. Then the number of positive, negative and zero eigenvalues, counted with their multiplicity, of A B and of B coincide. 4.63 1 . Show that ||Ar*Ar|| = ||7V||2 if N is normal.
144
4. Self-Adjoint Operators
4 . 6 4 % D i s c r e t e Fourier transform. Let T : C^ —• C ^ be the cycling forward shifting operator T{{zo,zi,... ,ZN-I)) '•= {zi,Z2,.. • ,ZN-I,ZO). Show that (i) T is self-adjoint, (ii) the N eigenvalues of T are the ATth roots of 1, (iii) the vectors Ufc : - - i = ( l , a ; ^ , u ; 2 f e , . . . , a ; ^ ( ^ - l ) ) ,
u := e'^, fc = 0 , . . . , AT - 1,
form an orthonormal basis of C ^ of eigenvectors of T; finally the cosine directions (z|ufc) of z G C-^ with respect to the basis ( u o , . . . , u^^^) are given by the Discrete Fourier transform of z. 4.65 ^ . Let A, B : X -^ X he two self-adjoint operators on a Euclidean or Hermitian space. Suppose that all eigenvalues of A- B are strictly positive. Order the eigenvalues Ai, A 2 , . . . , An of A and /ii, /i2, • • •, /in of B in a nondecreasing order. Show that Xi < fj,i Vi = 1 , . . . , n. [Hint: Use the variational characterization of the eigenvalues.] 4.66 f. Let A : X -^ X he self-adjoint on a Euclidean or Hermitian space. Let Ai, A 2 , . . . , An and /xi, /X2,..., /Xn be respectively, the eigenvalues and the singular values of A that we think of as ordered as |Ai | < IA2I < • • • < |An| and A*i < /i2 < • • • < MnShow that |Ai| = /ii Vi = 1 , . . . , n . [Hint: A*A = A^.] 4 . 6 7 %, Let A : X -^ X he a. linear operator on a Euclidean or Hermitian space. Let m, M he respectively the smallest and the greatest singular value of A. Show that ' ^ ^ l-^l ^ ^ for any eigenvalue X oi A. 4.68 f. Let A : X -^ Y he a, linear operator between two Euclidean or two Hermitian spaces. Show that (i) ( ^ M ) i / 2 maps ker A to {0}, (ii) (A*A)^^'^ is an isomorphism from ker A-^ onto itself, (iii) {AA*y^'^ is an isomorphism from ImA onto itself. 4.69 %, Let A : X -^ Y he a. linear operator between two Euclidean or two Hermitian spaces. Let (wi, 1*2, • • •, Un) and /ii, / i 2 , . . . , /in, fJ'i > 0 be such that (wi, U2, • •, Un) is an orthonormal basis of X and {A* A^^'^x = X]i A*i(^l^i)^i- Show that (i) AA'^y = E ^ , # o / ^ i ( 2 / l ^ ^ i ) ^ ^ i ^2/ e ^ ' (ii) If B denotes the restriction of [A*A)^^'^ to kei A^, see Exercise 4.68, then B ^x=
y ^ —{x\ui)ui • ^ — '
1
1
Vj;€kerA"'",
1
>^i^0 " *
(iii) If C denotes the restriction of (AA*)^^"^ to ImA, see Exercise 4.68, then C~^2/= ^
—{y\Aui)Aui
VyGlmA
^i^O ^'
4.70 %. Let A 6 Miv,n(^K)i ^ > ^? with Rank A = n. Select n vectors wi, 1^2, • • •, Un € K** such that A w i , . . . , A u n G K ^ are orthonormal. [Hint: Find U G Mn,n(K) such that A U is an isometry.]
4.3 Exercises
4.71 %. Let A e MN,n{^) and A = U A V , where U G 0(N), to 4.39, show that A+ = V ^ A ' U ^ where
/^
0
0
0\
0
-^
0
0
A' =
V G 0{n).
145
According
1
Mfc
0
\ 0
0
...
0
0/
Ml) M2, • • •, Mfe being the nonzero singular values of A. 4.72 %, For u : R —> R^, discuss the system of equations
(Pu —-+ dt
2
. V-l
-l'
u = 0.
4.73 If. Let A G Mn,n(R) be a symmetric matrix. Discuss the following systems of ODEs x ' ( t ) + A x ( t ) = 0, - i x ' ( t ) + Ax(t) = 0 , x " ( t ) + Ax(t) = 0,
where Ais positive definite
and show that the solutions are given respectively by e-*^x(0),
e-^*^x(0),
cos(tVA)x(0) +
sin(t\/A)
^
, x'(0).
4.74 ^ . Let A be symmetric. Show that for the solutions of x''(t) + Ax(t) = 0 the energy is conserved. Assuming A positive, show that |x(t)| < E/Xi where E is the energy of x(t) and A the smallest eigenvalue of A. 4.75 %. Let A be a Hermitian matrix. Show that |x(t)| = const if x(t) solves the Schrodinger equation i x ' -|- A x = 0.
Part II
Metrics and Topology
Felix Hausdorff (1869-1942), Maurice Frechet (1878-1973) and Rene-Louis Baire (18741932).
5. Metric Spaces and Continuous Functions
The rethinking process of infinitesimal calculus, that was started with the definition of the limit of a sequence by Bernhard Bolzano (1781-1848) and Augustin-Louis Cauchy (1789-1857) at the beginning of the XIX century and was carried on with the introduction of the system of real numbers by Richard Dedekind (1831-1916) and Georg Cantor (1845-1918) and of the system of complex numbers with the parallel development of the theory of functions by Camille Jordan (1838-1922), Karl Weierstrass (18151897), J. Henri Poincare (1854-1912), G. F. Bernhard Riemann (18261866), Jacques Hadamard (1865-1963), Emile Borel (1871-1956), ReneLouis Baire (1874-1932), Henri Lebesgue (1875-1941) during the whole of the XIX and beginning of the XX century, led to the introduction of new concepts such as open and closed sets, the point of accumulation and the compact set. These notions found their natural collocation and their correct generalization in the notion of a metric space, introduced by Maurice Frechet (1878-1973) in 1906 and eventually developed by Felix Hausdorff (1869-1942) together with the more general notion of topological space. The intuitive notion of a "continuous function" probably dates back to the classical age. It corresponds to the notion of deformation without "tearing". A function from X to F is more or less considered to be continuous if, when x varies slightly, the target point y = f{x) also varies slightly. The critical analysis of this intuitive idea also led, with Bernhard Bolzano (1781-1848) and Augustin-Louis Cauchy (1789-1857), to the correct definition of continuity and the limit of a function and to the study of the properties of continuous functions. We owe the theorem of intermediate values to Bolzano and Cauchy, around 1860 Karl Weierstrass proved that continuous functions take on maximum and minimum values in a closed and limited interval, and in 1870 Eduard Heine (1821-1881) studied uniform continuity. The notion of a continuous function also appears in the work of J. Henri Poincare (1854-1912) in an apparently totally different context, in the so-called analysis situs, which is today's topology and algebraic topology. For Henri Poincare, analysis situs is the science that enables us to know the qualitative properties of geometrical figures. Poincare referred to the properties that are preserved when geometrical figures undergo any kind of deformation except those that introduce tearing and glueing of points. An intuitive idea for some of these aspects may be provided by the following examples.
150
5. Metric Spaces and Continuous Functions
GBUNDZCGE
MENaENLEHEE ESPACES ABSTRAITS INTRODUCTION A L'ANALYSE GfiNfiRALB
FELIX H1U8D0RPP
a FIOCRES: (M TKXr Xatirice MUlECHBT
PARIS GA0THIBU-VJLLAR8 »i CI, tOlTEURS H aniHl»-A«fil(li««, H
LEIPZia VEBLAO VON VEIT A COilP,
Figure 5.1. Frontispieces of Les espaces abstraits by Maurice Frechet (1878-1973) and of the Mengenlehre by Felix Hausdorff (1869-1942).
o Let US draw a disc on a rubber sheet. No matter how one pulls at the rubber sheet, without tearing it, the disc stays whole. Similarly, if one draws a ring, any way one pulls the rubber sheet without tearing or glueing any points, the central hole is preserved. Let us think of a loop of string that surrounds an infinite pole. In order to separate the string from the pole one has to break one of the two. Even more, if the string is wrappped several times around the pole, the linking number between string and pole is constant, regardless of the shape of the coils. o We have already seen Euler's formula for polyhedra in [GMl]. It is a remarkable formula whose context is not classical geometry. It was Poincare who extended it to all surfaces of the type of the sphere, i.e., surfaces that can be obtained as continuous deformations of a sphere without tearing or glueing. o E, R^, R^ are clearly different objects as linear vectorspaces. As we have seen, they have the same cardinality and are thus undistinguishable as sets. Therefore it is impossible to give meaning to the concept of dimension if one stays inside the theory of sets. One can show, instead, that their algebraic dimension is preserved by deformations without tearing or glueing. At the core of this analysis of geometrical figures we have the notion of a continuous deformation that corresponds to the notion of of a continuous one-to-one map whose inverse is also continuous, called homeomorphisms. We have already discussed some relevant properties of continuous functions / : R - ^ R e / : R 2 - ^ R i n [GMl] and [GM2]. Here we shall discuss continuity in a sufficiently general context, though not in the most general.
5.1 Metric Spaces
151
Poincare himself was convinced of the enormous importance of extending the methods and ideas of his analysis situs to more than three dimensions. ... L'analysis situs a plus de trois dimensions presente des difficultes enormes; il faut pour tenter de les surmonter etre bien persuade de Vextreme importance de cette science. Si cette importance n'est pas bien comprise de tout le monde, c 'est que tout le monde n'y a pas suffisamment reflechi^ In the first twenty years of this century with the contribution, among others, of David Hilbert (1862-1943), Maurice Prechet (1878-1973), FeUx Hausdorff (1869-1942), Pavel Alexandroff (1896-1982) and Pavel Urysohn (1898-1924), the fundamental role of the notion of an open set in the study of continuity was made clear, and general topology was developed as the study of the properties of geometrical figures that are invariant with respect to homeomorphisms, thus linking back to Euler who, in 1739, had solved the famous problem of Konigsberg's bridges with a topological method. There are innumerable successive applications, so much so that continuity and the structures related to it have become one of the most pervasive languages of mathematics. In this chapter and in the next, we shall discuss topological notions and continuity in the context of metric spaces.
5.1 Metric Spaces 5.1.1 Basic definitions a. Metrics 5.1 Definition. Let X be a set. A distance or metric on X is a map d : X X X -^ R+ for which the following conditions hold: (i) (IDENTITY) d{x,y) >Oifx^yeX, and d{x,x) = 0 Vx G X . (ii) (SYMMETRY) d{x,y) = d{y,x) ^x,y e X. (iii) (TRIANGLE INEQUALITY) d{x,y) < d{x,z) -\-d{z,y), \/ x,y,z e X. A metric space is a set X with a distance d. Formally we say that (X, d) is a metric space if X is a set and d is a distance on X. The properties (i), (ii) and (iii) are often called metric axioms.
^ The analysis situs in more than three dimensions presents enormous difficulties; in order to overcome them one has to be strongly convinced of the extreme importance of this science. If its importance is not well understood by everyone, it is because they have not sufficiently thought about it.
152
5. Metric Spaces and Continuous Functions
\
T
B
Figure 5.2. Time as distance.
5.2 E x a m p l e . The Euclidean distance d{x, y) := \x — y\, x,y £R, is a, distance on R. On M? and R^ distances are defined by the Euclidean distance, given for n = 2,3 by _
^ 1/2
Irtixi-y^A
where X := (xi,a;2),y := (yi,2/2) if n = 2, or x := (xi,X2,X3),y := (2/1,2/2,2/3) if n = 3. In other words, R, R'^, R^ are metric spaces with the EucHdean distance. 5.3 E x a m p l e . Imagine R^ as a union of strips En := {(a:i,a;2,iC3) \n < X3 < n -\- 1}, made by materials of different indices of refractions Vn- The time t{A, B) needed for a light ray to go from A to B in R^ defines a distance on R^, see Figure 5.2.
5.4 E x a m p l e . In the infinite cylinder C = {{x,y,z)\x'^ -\-y"^ = 1} c R^, we may define a distance between two points P and Q as the minimal length of the line on C, or geodesic, connecting P and Q. Observe that we can always cut the cylinder along a directrix in such a way that the curve is not touched. If we unfold the cut cylinder to a plane, the distance between P and Q is the Euclidean distance of the two image points.
5.5 1. Of course 1001a; — 2/1 is also a distance on R, only the scale factor has changed. More generally, if / : R —• R is an injective map, then d(x, y) := \f(x) — f{y)\ is again a distance on R.
5.6 Definition. Let (X, d) be a metric space. The open ball or spherical open neighborhood centered at XQ e X of radius p > 0 is the set B{xo,p) := Ix e X\ d{x, XQ) < p\.
Figure 5.3. Metrics on a cylinder and on the boundary of a cube.
5.1 Metric Spaces
153
Notice the strict inequality in the definition of B{x,r). In M, R^, R^ with the EucHdean metric, B{xo^ r) is respectively, the open interval ]xo — r, xo + r[, the open disc of center XQ G R^ and radius r > 0, and the ball bounded by the sphere of R^ of center XQ G R^ and radius r > 0. We say that a subset £' C X of a metric space is bounded if it is contained in some open ball. The diameter oi E C X is given by d i a m ^ := sup< d(a:,y) x, ?/ G £^L and, trivially, E is bounded iff didiXnE < +oo. Despite the suggestive language, the open balls of a metric space need not be either round nor convex; however they have some of the usual properties of discs in R^. For instance (i) B{xo, r) C B{xo, s) VXQ G X and 0 < r < 5, (ii) Ur>oB{xo,r) = X Vxo G X, (iii) nr->o^(^o,^) = {^o} Vxo G X ,
(iv) Va:o G X and Vz G B{xo^r) the open ball centered at z and radius p :=z r — d(xo, z) > 0 is contained in B{xo, r), (v) for every couple of balls B{x,r) and B{y,s) with a nonvoid intersection and Vz G B{x,r) n ^(y, s), there exists t > 0 such that B{z, t) C B{x, r) n B{y, s), in fact t := min(r — d(x, z), s — d{y, z)), (vi) for every x,y e X with x ^ y the balls 5 ( x , r i ) and B{y^r2) are disjoint if ri + r2 < G!(X, y). 5.7 ^ . Prove the previous claims. Notice how essential the strict inequality in the definition of B(xo, p) is.
b. Convergence A distance d on a set X allow us to define the notion of convergent sequence in X in a natural way. 5.8 Definition. Let (X, c!) he a metric space. We say that the sequence {xn} C X converges to x e X, and we write Xn -^ x, if d{Xn,x) -^ 0 in R, that is , if for any r > 0 there exists n such that d{xn^x) < r for all n>n. The metric axioms at once yield that the basic facts we know for limits of sequences of real numbers also hold for limits of sequences in an arbitrary metric space. We have (i) the limit of a sequence {xn} is unique, if it exists, (ii) if {xn} converges, then {xn} is bounded, (iii) computing the limit of {xn} consists in having a candidate x G X and then showing that the sequence of nonnegative real numbers {d{xn',x)} converges to zero, (iv) if Xn -^ X, then any subsequence of {xn} has the same limit x.
154
5. Metric Spaces and Continuous Functions
Thus, the choice of a distance on a given set X suffices to pass to the limit in X (in the sense specified by the metric d). However, given a set X, there is no distance on X that is reasonably absolute (even in R), but we may consider different distances in X. The corresponding convergences have different meanings and can be suited to treat specific problems. They all use the same general language, but the exact meaning of what convergence means is hidden in the definition of the distance. This flexibility makes the language of metric spaces useful in a large context.
5-1.2 Examples of metric spaces Relevant examples of distances are provided by linear vector spaces on the fields K = E or C in which we have defined a norm. 5.9 Definition. Let X be a linear space over K = R or C. A norm on X is a function \\ \\ : X -^ R+ satisfying the following properties (i) (FiNiTENESS) ||a;|| eR\/x e X. (ii) (IDENTITY) ||X|| > 0 and \\x\\ =0 if and only if x — 0. (iii)
(iv)
(1-HOMOGENEITY) ||AX|| = |A| ||X|| V X G X, (TRIANGLE INEQUALITY) ||X + 2/|| < ||x|| +
VA G K.
||2/|| \J x,y e X.
/ / II • II is a norm on X, we say that (X, || ||) Z5 a linear normed space or simply that X is a normed space with norm \\\\. Let X be a linear space with norm || ||. It is easy to show that the function d: X x X -^ R+ given by d{x,y) := \\x-y\\,
x,y e X,
satisfies the metric axioms, hence defines a distance on X, called the natural distance in the normed space (X, || ||). Obviously, such a distance is translation invariant, i.e., d{x -\- z,y -\- z) — d{x, y) Vx, y,z ^ X. Trivial examples of metric spaces are provided by the nonempty subsets of a metric space. If A is a subset of a metric space (X, c?), then the restriction of d to ^ x A is trivially, a distance on A. We say that ^ is a metric space with the induced distance from X. 5.10 If. For instance, the cylinder C := {{x, y, 2;) G M^ | x^ 4- y^ = 1} is a metric space with the Euclidean distance that, for x, y G C, yields d(x, y) :=:length of the chord joining x and y. The geodesic distance dg of Example 5.4, that is the length of the shortest path in C between x and y, defines another distance. C with the geodesic distance dg has to be considered as another metric space different from C with the Euclidean distance. A simple calculation shows that l|x-y||
We shall now illustrate a few examples of metric spaces.
5.1 Metric Spaces
,-'^/ / //^/ / :: // / / / V ^
\
^
^
155
V
\\ \ \ \N • \ \ '» : \\^: Ni
/Ti,
= \ \ \\ \ : X ^s \V\V
//'i // /: / / /' . v \ /^'/ / / '
Figure 5.4. The ball centered at (0,0) of radius 1 in R^ respectively, for the metrics d\, di.3, d2 and d^. The unit ball centered at (0,0) of radius one for the metric doo is the square ] — 1, l[x] — 1,1[.
a. Metrics on
finite-dimensional
vector spaces
5.11 %, As we have already seen, E^ with the Euclidean distance |x — y | is a metric space. More generally, any Euclidean or Hermitian vector space X is a normed space with norm given by
\\x\\:=^/W) cf. Chapter 3. X is therefore a metric space with the induced distance d{x,y) :=
\\x-y\\,
called the Euclidean distance of X. 5.12 % oo-distance. Set for x = (a^i, 0:2,..., Xn) G M.'^ ||x||cx) := max(|xi|, | x 2 | , . . . , \xn\)Show that X —> ||a:||oo is a norm on R'^. Hence, R^, equipped with the distance doo(x,y) :=
\\x-y\\oo,
is a metric space different from the standard R^ with the Euclidean distance of Exercise 5.11. In R"^, the unit ball centered at (0,0) of radius one for the metric doo is the square ] — 1, l[x] — 1,1[, see Figure 5.4.
5.13 % p-distance. Given a real number p > 1, we set for x € R*^
:=(Ekil
l/p
Show that ||x||p is a norm on R"^, hence dp(x,y) := | | x - 2 / | | p is a distance on R*^. Observe that || II2 and ^2 are the Euclidean norm and distance in R"^. In R^, the unit ball centered at (0,0) of radius one for the metric dp for some values of p is shown in Figure 5.4. [Hint: The triangle inequality for the p-norm is called Minkowski's discrete inequality
156
5. Metric Spaces and Continuous Functions
| | a + b | | p < ||a||p+||b||p,
Va := (ai, a2, • . . , an), h={bi,
6 2 , . . . , 6n),
which follows for instance from Minkowski's inequality for integral norms, see [GMl]. Alternatively, we can proceed as follows. Suppose a and b are nonzero, otherwise the inequality is trivial, apply the convexity inequality f{Xx+{l — X)y) < Xf{x)-\-{l — X)f{y) to f{t) = tP with X := a i / | | a | | p , y := 6i/||b||p, A := ||a||p/(||a||p + ||b||p), and sum on i from 1 to n, to get |a + b||p
<1.]
^llp-M|b|b
5.14 t P r o d u c t spaces. Let (Xi,^^^)), {X2,d^'^^), ... (Xn,d(^)) be n metric spaces and let y = X i X ^ 2 X • • • X Xn be the Cartesian product of X i , . . . , Xn- Show that each of the functions defined on X x K by
fcip(x,y) := (EILW('H^i,2/iF)'^^ (^doo(x,y) :=m3X^S'-\xi,yi)
ifp> 1,
U == l , . . . , n |
for X = (rr^, x ^ , . . . , x^), y = (t/^, 2 / ^ , . . . , y^) G Y, are distances on Y. Notice that if X i = • • • = Xn = M with the Euclidean distance, Y is R^, then the distances dp(x, y) are just the distances in Exercises 5.13 and 5.12. Also show that if {x^} C Y, x^ := (x^, x | , . . . x^) Vfc, and x = (x^, x ^ , . . . , x ^ ) , then the following claims are equivalent. (i) There exists p > 1 such that dp(xfc,x) —^ 0, (ii) d p ( x f c , x ) - ^ O V p > 1, (iii)
doo(Xfc,x)-^OVp>l,
(iv) Vz = l , . . . , n d i ( 4 , x O - ^ 0 . 5.15 % D i s c r e t e d i s t a n c e . Let X be any set. The discrete distance on X is given by \l d{x, y)= < I 0
ifx^y, if X = y.
Show that the balls for the discrete distance are B{x,r)
= {y ex\d{x,y)
^'
'
-^
= {^''^
[X
'^"" - ^'
ifr>l,
and that convergent sequences with respect to the discrete distance reduce to sequences that are definitively constant. 5.16 % C o d e s d i s t a n c e . Let X be a set that we think of as a set of symbols, and let X ^ = X x X x - - - x X the space of ordered words on n symbols. Given two words X = (xi, X2,..., Xn) and 2/ = (t/i, 2/2, • • •, 2/n) € X ^ , let dix,y)
: = # | i | a ; i ^ yij
be the number of bits in x and y that are diff^erent. Show that d(x, y) defines a distance in X ^ . Characterize the balls of X ^ relative to that distance. [Hint: Write d{x,y) — X)?=i d(xi,yi) where d is the discrete distance in X.]
5.1 Metric Spaces
157
RENDICONTI MEMOfilE E COMUHiCAZiONI.
CIRCOLO MATEMATICO
SUR OOEUHIES POLXTS DU CALCUL FOKCTIOSKEL; P«r M. M i n r l o t fritkti
( P m ) •>
DI PALERMO IXTRODUCTIOM.
DiHiTOKt: G. B. GUCCIA. U vitnHe X, qiund i tauw vakur aumirfqiw de x amrMfond OM n h u r bin Mwr-
TOMO XXII ^irectkiiu ((ar eumpk «i c« ^ui coacemt ruaifsnniU) « eo pinkulhr au poiot ik vw ik «e <)u'on doit jXtndre pwr varubk. Oepuk looficmpt, oo t cooaMM te ruKdiNU ik liow, il* tni^ ou mOuw Jk n v u M l n iramiti^ut*. U s l u m t (lainl». ndoui Kwt pitu li'caun. Akui, M. U: Koi-x > M uiwni i CtwUcf l a ibiKiiQiu
PALERMO. SEDM DMllA SOCISTA
Figure 5.5. The first page of the These at the Faculty of Sciences of Paris by Maurice Prechet (1878-1973), published on the Rendiconti del Circolo Matematico di Palermo.
b. Metrics on spaces of sequences We now introduce some distances and norms on infinite-dimensional vector spaces. 5.17 E x a m p l e (^oo s p a c e ) . Consider the space of all real (or complex) sequences X := ( a ; i , . . . ) - For x = {xn}, y := {yn}, set ||x||oo : = s u p | a : n | ,
doo(x,y) := | | x - y | | o o .
n
It is easy to show that x —• ||a;||oo satisfies the axioms of a norm apart from the fact that it may take the value +oo. Thus a: —>• ||a:||oo is a norm in the space ^C50 : = | x = {Xn} I ||x||oo < +CXD j ,
that is, on the linear vector space of hounded sequences. Consequently,
doo{x,y) := | | x - y | | o o ,
x,y eioo,
is a distance on ^oo J called the unifoTm distance. Convergence of "{x/j. }• (Z •^oo to x G •^oo in the uniform norm, called the uniform convergence, amounts to ||Xfc - x | | o o = S U p | x J . - X '
as fc —• oo.
(5.1)
where Xfc = ( a ; ^ , a ; | , . . . ) and x = (x^, x ^ , . . . ) . Notice that the uniform convergence in (5.1) is stronger than the pointwise convergence
Vi
xi
as fc —>^ oo.
For instance, let (f{t) := te~*, t € M+, and consider the sequence of sequences {x^} where x^ := {x^}n, xV- := < ^ ( ^ ) . Then \fi we have xj. = f e"^/'' -^ 0 as fc -^ oo, while llxfc - Oiloo = s u p j ^ e - ^ / ^ I 2 = 0 , 1 , . . . j = - 7^ 0.
158
5. Metric Spaces and Continuous Functions
Of course W^ with the metric doo in Exercise 5.12 is a subset of ^oo endowed with the induced metric doo- This follows from the identification (a:\...,x^) ^
(x\...,x",0,...,0,...).
5.18 E x a m p l e (£p spaces, p > 1). Consider the space of all real (or complex) sequences X := ( x i , . . . ) . For 1 < p < oo, x = {xn} and y := {y-n} set 1 /
Trivially, ||x||^p = 0 if and only if any element of the sequence x is zero, moreover Minkowski's inequality
l|x + y|Up<||x||«^ + ||y|U^, holds as it follows from Exercise 5.13 (passing to the limit as n -^ oo in Minkowski's inequality in E'^). Thus || ||^ satisfies the metric axioms apart from the fact that it may take the value -f-oo. Hence, || H^ is a norm in the linear space of sequences ^p-{x={Xn}|||x|Up<+Oo}.
Consequently, d^ (x, y) := ||x — yWcp is a distance on ip. Convergence of {x^} C ^p to x G ^p amounts to oo
where x^ = i^\^^^^ • • •) ^^^ ^ = (x^, x ^ , . . . ) . Notice that W^ with the metric dp in Exercise 5.13 is a subset of £p endowed with the induced metric d^p. This follows for instance from the identification (x\...,x'^) ^
(x\...,x'',0,...,0,...).
Finally, observe that ||x||^g < ||x||£p Vx if 1 < p < g, hence iiCipC
iq.
(5.2)
Since there exist sequences x = {xn} such that HxH^^ < +oo while ||x||£p = +00 if p < g, as for instance r 1 1i/p the inclusions (5.2) are strict if 1 < p < g. The case p = 2 is particularly relevant since the ^2 norm is induced by the scalar product ( x | y ) , , := £ x * j / * ,
||x|U, = ^ ( x l x ) , , .
Z = l
^2 is called the Hilbert coordinate space, and the set { x - ( x \ x 2 , . . . , ) € ^ 2 | | x i | < i Vi} the Hilbert cube.
5.1 Metric Spaces
159
Figure 5.6. Tubular neighborhood of the graph of / .
c. Metrics on spaces of functions The language of metric spaces is particularly relevant in dealing with different types of convergences of functions. As examples of metric spaces of functions, we then introduce a few normed spaces that are relevant in the sequel. 5.19 E x a m p l e ( C o n t i n u o u s f u n c t i o n s ) . Denote by C°([0,1]) the space of all continuous functions / : [0,1] -^ M. For / : [0,1] -> E set ll/||oo,[0,l] —
sup x€[0,l]
\f(x)\.
We have (0 ll/llcxD,[o,i] ^ ~^^^ ^y Weierstrass's theorem, (ii) ll/lloc!lo!i]=Oifr/(x) = OVx, (iii) l|A/||oc,[o,i] = |A|||/||oo,[o,i). (iv) | | / + 9 | | c » , [ 0 , l l < ll/llcx,,[0,ll + IMIoo,lO,ll-
To prove (iv) for instance, observe that for all x € [0,1], we have \f(x)+g{x)\ < \f(x)\ + \g{x)\ < ||/||oc,[o,i] + IMIoc,[o,i] hence the right-hand side is an upperbound for the values of f -\- g. The map / —* ||/||oo,[o,i] ^^ *^^^ ^ norm on C^([0,1]), called the uniform or infinity norm. Consequently C^([0,1]) is a normed space and a metric space with the uniform distance ^
^
te[o,i]
In this space, the ball B{f, e) of center / and radius e > 0 is the set of all continuous functions g G ^^([0,1]) such that \gix) - f{x)\ < e
VxG[0,l]
or the family of all continuous functions with graphs in the tubular neighborhood of radius e of the graph of / t / ( / , e ) : = { ( a ; , j / ) | x e [ 0 , l ] , y eR,\y
- f{x)\ < e],
(5.3)
see Figure 5.6. The uniform convergence in C^([0,1]), that is the convergence in the uniform norm, of {fk} C C^([0,1]) to / G C^([0,1]) amounts to computing Mk '•= ll/fc - /lloc,[o,i] = ^ ^ ^ , j l/fc W - / W l for every fc = 1, 2 , . . . and to showing that Mk —* 0 as /c -^ -f-oo.
160
5. Metric Spaces and Continuous Functions
-1
-1/A;2
Figure 5.7. The function / ^ in (5.4).
5.20 E x a m p l e ( F u n c t i o n s o f class C^dO, 1])). Denote by C^([0,1]) the space of all functions / : [0,1] - ^ R of class C \ see [GMl]. For / 6 C^{[0,1]), set llci([0,i]) ' =
sup | / ( a : ) | + xG[0,l]
sup
\f{x
||/lloo,[0,l] + ll/'lloo,[0,l]-
xG[0,l]
It is easy to check that / —• Consequently,
Ci(fo 11) ^s ^ norm in the vector space C^([0,1]).
dc^([o,i])if^9) •= ll/-pllci([o,i]) defines a distance in C^([0,1]). In this case, a function g E C^ has a distance less than e from / if | | / - S/||oo,[o,i] + 11/' - P1IOO,[O,I] < ^i equivalently, if the graph of g is in the tubular neighborhood t / ( / , ei) of the graph of / , and the graph of g' is in the tubular neighborhood C/(/',e2) of f with ei + 62 = e, see (5.3). Moreover, convergence in the Ci([0,l])-norm of { A } C CH[0,1]) to f E C i ( [ 0 , l ] ) , \\fk - /Ilci([0,i]) ^ 0, amounts to / uniformly in [0,1], fk •/'
uniformly in [0,1].
Figures 5.8 and 5.9 show graphs of Lipschitz functions and functions of class C^([0,1]) that are closer and closer to zero in the uniform norm, but with uniform norm of the derivatives larger than one. 5.21 E x a m p l e (Integral m e t r i c s ) . Another norm and corresponding distance in C^([0,1]) is given by the distance in the mean
II/IIL1([O,I]) '= J
l/WN*,
dLmo,i])(f,g)
•= \\f -g\\LHo,i)
•= j
\f-9\dx.
5.22 ^ . Show that the L -norm in C0([0,1]) satisfies the norm axioms. Convergence with respect to the L^-distance differs from the uniform one. For instance, for A: = 1, 2 , . . . set (5.4)
\o
if]^
We have ||/fc||cx>,[o,i] = /(O) = fc - ^ H-oo while ||/fc||Li([o,i]) = V(2fc) - ^ 0, cf. Figure 5.7. More generally, the I / P ( [ 0 , l])-norm, 1 < p < oo, on C°([0,1]), is defined by
5.1 Metric Spaces
161
Figure 5.8. The Lebesgue example.
\\f\\LP(o,i) ••= (^£ \fixw dxy \ It turns out that / -^ | | / | | L P ( O , I ) satisfies the axioms of a norm, hence 1
dLP{[0,i]){f,9)
'= \\f - 9\\LPi[0,i]) '= [
i/p
\f-9\^^^] 0
is a distance in C^([0,1]); it is called the L P ( [ 0 , 1])-distance. 5.23 ^ . Show that the L P ( [ 0 , l])-norm in C^([0,1]) satisfies the norm axioms. [Hint: The triangle inequality is in fact Minkowski's inequality, see [GMl].]
5-l,3 Continuity and limits in metric spaces a. Lipschitz-continuous maps between metric spaces 5.24 Definition. Let (X^dx) and (Y^dy) be two metric spaces and let 0 < a < 1. We say that a function f : X ^^ Y is a-Holder-continuous if there exists L > 0 such that dY{f{x)J{y)) 1-Holder-continuous smallest constant L of / , often denoted called the Lipschitz
yx.yeX.
< Ldxix.vr,
(5.5)
functions are also called Lipschitz continuous. The for which (5.5) holds is called the a-Holder constant by [f]a' When a = 1, the l-Holder constant is also constant of f and denoted by [f]i, L i p / or L i p ( / ) .
5.25 E x a m p l e ( T h e d i s t a n c e f u n c t i o n ) . Let {X, d) be a metric space. For any XQ G X , the function f{x) := d{x,xo) : X —> R is a Lipschitz-continuous function with Lip (/) = 1. In fact, from the triangle inequaUty, we get \fiy) - f{x)\ = \d(y,xo) -d{x,xo)\
< d{x,y)
yx,y G X,
hence / is Lipschitz continuous with Lip (/) < 1. Choosing a; = XQ, we have \f{y) - / ( ^ o ) | = \d{y, xo) - d{xo, xo)\ = d{y, XQ), thus L i p ( / ) > 1.
162
5. Metric Spaces and Continuous Functions
Figure 5.9. On the left, the sequence fkix) := k~^ cos{kx) that converges uniformly to zero with slopes equibounded by one. On the right, gk(x) '•= k~^ cos{k'^x), that converges uniformly to zero, but with slopes that diverge to infinity. Given any function / G C'^([0,1]), a similar phenomenon occurs for the sequences fk{x) := f(kx)/k, gk{x) =
f{k'^x)/k.
5.26 ^ D i s t a n c e from a s e t . Let (X, d) be a metric space. The distance function from cc € A" to a nonempty subset A C X is defined by d{x, A) := inf|d(a:, y) U € A \ . It is easy to show that f{x) := d(x, A) : X —^R is a, Lipschitz-continuous function with
Li (f)^
Jo iid(x,A) = OWx, I1
otherwise.
If d{x, A) is identically zero, then the claim is trivial. On the other hand, for any x,y £ X and z E A we have d{x, z) < d{x, y) -\- d{y, z) hence, taking the infimum in z^
d{x,A)-d{y,A)
and interchanging x and y,
\d{x,A)-d{y,A)\
= d{x, A) >
d(x, Xn), n -\-1 from which we infer that the Lipschitz constant of a: —• d{x^ A) must not be smaller than one.
b. Continuous maps in metric spaces The notion of continuity that we introduced in [GMl], [GM2] for functions on one real variable can be extended in the context of the abstract metric structure. In fact, by paraphrasing the definition of continuity of functions / : R -^ R+ we get
5.1 Metric Spaces
163
5.27 Definition. Let (X^dx) and {Y^dy) be two metric spaces. We say that f : X ^^ Y is continuous at XQ Z/ Ve > 0 there exists S > 0 such that dy(/(x),/(xo)) < e whenever dx{x,xo) < S, i.e., S)) C 5 y ( / ( X Q ) , e).
Ve > 0 3(5 > 0 such that f{Bx{xo,
(5.6)
We say that f : X ^^ Y is continuous in E C X if f is continuous at every point XQ ^ E. When E = X and f : X ^^ Y is continuous at any point of X, we simply say that f : X ^^Y is continuous. 5.28 1[. Show that a-Holder-continuous functions, 0 < a < 1, in particular Lipschitzcontinuous functions, between two metric spaces are continuous.
Let (X, dx) and (F, dy) be two metric spaces and E C X. Since E is a, metric space with the induced distance of X, Definition 5.27 also appHes to the function f : E —^Y. Thus f : E —^Y is continuous dit XQ E E if \/e>03S>0
such that f{Bx{xo,
S) H E) C By{f{xo),
e)
(5.7)
and we say that / : E" ^ y is continuous ii f : E ^^ Y is continuous at any point XQ E E. 5.29 Remark. As in the case of functions of one real variable, the domain of the function / is relevant in order to decide if / is continuous or not. For instance, f : X -^Y is continuous in £" C X if Vxo G E V€ > 0 35 > 0 such that f{Bx{xo,
5)) C By{f{xo),
e),
(5.8)
while the restriction f\E - E -^ Y oi f to E is continuous in E if yxoeE\/e>03S>0
such that f{Bx{xo,S)nE)
C By{f{xo),e).
(5.9)
We deduce that the restriction property holds: if f : X ^^Y is continuous in E, then its restriction f\E '- ^ ~^^ ^o E is continuous. The opposite is in general false, as simple examples show. 5.30 Proposition. Let X,Y,Z be three metric spaces, and XQ E X. If f : X ^^ Y is continuous at XQ and g : Y -^ Z is continuous at f{xo), then g o f : X ^^ Z is continuous at XQ • In particular, the composition of two continuous functions is continuous. Proof. Let e > 0. Since g is continuous at f{xo), there exists a > 0 such that g{BY{f{xo),(T)) C Bz{g{f{xo)),e). Since / is continuous at XQ, there exists 5 > 0 such that f(BxixoyS)) C By(/(a;o),cr), consequently go f{Bx{xo,S))
C g{BYif{xo),a))
C Bzigo
f{xo),e). D
Continuity can be expressed in terms of convergent sequences. As in the proof of Theorem 2.46 of [GM2], one shows 5.31 Theorem. Let (X, dx) and (y, dy) be two metric spaces, f : X -^Y is continuous at XQ E X if and only if f{xn) -^ f{xo) in iy^dy) whenever {Xn}
d X,
Xn-^
XQ in ( X , d x ) .
164
5. Metric Spaces and Continuous Functions
c. Limits in metric spaces Related to the notion of continuity is the notion of the hmit. Again, we want to rephrase f{x) —^yo as x ^ XQ. For that we need / to be defined near XQ, but not necessarily at XQ. For this purpose we introduce the 5.32 Definition. Let X be a metric space and A C X. We say that XQ G X is an accumulation point of A if each ball centered at XQ contains at least one point of A distinct from XQ , Vr>0
B(xo,r)nA\{xo}^0.
Accumulation points are also called cluster points. 5.33 %, Consider R with the EucUdean metric. Show that (i) the set of points of accumulation of A :=]a, 6[, B = [a,b], C = [a, b[ is the closed interval [a, 6], (ii) the set of points of accumulation of A :=]0,1[U{2}, B = [0,1]U{2}, C = [0,1[U{2} is the closed interval [0,1], (iii) the set of points of accumulation of the rational numbers and of the irrational numbers is the whole R.
We shall return to this notion, but for the moment the definition suffices. 5.34 Definition. Let (X, dx) and (Y, dy) be two metric spaces, letEcX and let XQ G X be a point of accumulation of E. Given f : E\ {XQ} -^ Y, we say that y^ £Y is the limit of f{x) as x -^ XQ, X E E, and we write f{x) -^yo as x-^
XQ,
or
lim /(x) = yo X
'XQ
xeE
if for any e > 0 there exists 6 > 0 such that dy(/(x),2/o) < e whenever X e E and 0 < dx{x,xo) < S. Equivalently, Ve > 0 3(5 > 0 such that f{Bx{xo,
S)r]E\
{XQ}) C Byiyo, e).
Notice that, while in order to deal with the continuity of / at xo we only need / to be defined at XQ; when we deal with the notion of limit we only need that XQ be a point of accumulation of E. These two requirements are unrelated, since not all points of E are points of accumulation and not all points of accumulation of E are in E, see, e.g.. Exercise 5.33. Moreover, the condition 0 < dxix^xo) in the definition of limit expresses the fact that we can disregard the value of / at XQ (in case / is defined at XQ). Also notice that the limit is unique if it exists^ and that limits are preserved by restriction. To be precise, we have 5.35 Proposition. Let (X, dx) and (F, dy) be two metric spaces. Suppose F C E C X and let XQ G X be a point of accumulation for F. If f{x) —^y as X -^ xo; X £ E, then / ( x ) -^ y as x -^ XQ, X £ F. 5.36 ^ . As for functions of one variable, the notions of limit and continuity are strongly related. Show the following.
5.1 Metric Spaces
165
P r o p o s i t i o n . Let X and Y be two metric spaces, E C X and XQ ^ X. (i) / / XQ belongs to E and is not a point of accumulation of E, then every function f : E —^Y is continuous at XQ. (ii) Suppose that XQ belongs to E and is a point of accumulation for E. Then a) f : E -^ Y is continuous at XQ if and only if f{x) —>• f{xo) as x ^^ XQ, xe E, b) f(x) —> y as X -^ XQ, X ^ E, if and only if the function g : EU {XQ} -^ Y defined by \fix)
ifxeE\{xo}, if X =
is continuous
XQ
at XQ.
We conclude with a change of variable theorem for limits, see e.g., Proposition 2.27 of [GMl] and Example 2.49 of [GM2]. 5.37 Proposition. Let X^Y,Z be metric spaces, E C X and let XQ be a point of accumulation for E. Let f : E —^ Y, g : f{E) -^ Z be two functions and suppose that /(XQ) is an accumulation point of f{E). If (i) 9{y) -^ L as y -^ yo, y e f{E), (ii) f(x) -^ yo as X -^ xoy X e E, (iii) either f{xo) = yO) or f{x) ^ yo for all x e E and x ^ XQ, then g{f{x)) -^ L as x —^ XQ, X E E. d. The junction property A property we have just hinted at in the case of real functions is the junction property^ see Section 2.1.2 of [GMl], which is more significant for functions of several variables. Let X be a set. We say that a family {[/«} of subsets of a metric space is locally finite at a point XQ G X if there exists r > 0 such that B(xo, r) meets at most a finite number of the C/a's. 5.38 Proposition. Let {X^dx), (1^,dy) be metric spaces, f : X —^ Y a function, XQ E X, and let {Ua} be a family of subsets of X locally finite at XQ.
(i) Suppose that XQ is as X -^ XQ, X eUa, (ii) / / XQ G HaUa and then f : X ^^Y is
a point of accumulation of Ua and that f[x) -^ y for all a. Then f{x) —^y as x ^^ XQ, X £ X. f : Ua C X —^ Y is continuous at xo for all a, continuous at XQ .
5.39 t - Prove Proposition 5.38. 5.40 E x a m p l e . An assumption on the covering is necessary in order that the conclusions of Proposition 5.38 hold. Set A := {(x, y)\x'^
{
1
if I
0
otherwise. otl
166
5. Metric Spaces and Continuous Functions
The function / is discontinuous at XQ := (0,0), since its oscillation is one in every ball centered at XQ. Denote by Um the straight line through the origin Um '•= { ( x , y ) \ y = m x } ,
m G M,
Uoo '-= {{x,y)\x
= 0}.
The C/a's, a G R U {oo} form a covering of E^ that is not locally finite at XQ and for any a G M U oo, the restriction of / to e£ich Ua is zero near the origin. In particular, each restricition f^u^ : [/« —^ M is continuous at the origin.
5-1-4 Functions from R^ into
om
It is important to be acquainted with the Umit notion we have just introduced in an abstract context. For this purpose, in this section, we shall focus on mappings between Euclidean spaces and illustrate with a few examples some of the abstract notions previously introduced. a. The vector space C^{A,W^) Denote by eV R"^ -^ R the linear map that maps x = (x^, x ^ , . . . , x^) G R'^ into its ith component, e*(x) := x \ Any map f : X -^ W' from a set X into R^ writes as an n-tuple of real-valued functions / ( x ) = ( / ^ ( x ) , . . . , /'^(x)), where for any i = 1 , . . . , n the function /* : X -^ R is given by p{x) := e^(/(x)). Prom n
|yi|,|y2|,...,i2/„l<|y|<X^|j/i|
yeM"
z=l
we readily infer the following. 5.41 Proposition. The following claims hold. (i) The maps eV R^ —> R, i = 1 , . . . , n, are Lipschitz continuous. (ii) Let {X, d) he a metric space. Then a) f : X -^ W^ is continuous at XQ e X if and only if all its components f^, / ^ , . . . , / ^ are continuous at XQ, b) '^f fi9 ' ^ -^^^ ^^6 continuous at XQ, then f -\- g : X —> W^ is continuous at XQ , c) if f : X -^ W^ and A : X —> R are continuous at XQ then the map Xf : X —^ W^ defined by A/(x) := A(x)/(x), is continuous at
XQ.
5.42 E x a m p l e . The function / : R^ —>• R, / ( x , y, x) := sin(x^y) + x^ is continuous at R^. In fact, if xo := (xo,yo,zo), then the coordinate functions x = (x, y,%) -^ x, x —^ y, X —>• z are continuous at XQ by Proposition 5.41. By Proposition 5.41 (iii), x -^ x'^y and X -^ z'^ are continuous at xo, and by (ii) Proposition 5.41, x -^ x'^y + z"^ is continuous at XQ. Finally sin(x^2/ -^ x^) is continuous since sin is continuous.
5.43 Definition. Let X and Y he two metric spaces. We denote hy C^{X, Y) the class of all continuous function f : X -^Y.
5.1 Metric Spaces
167
As a consequence of Proposition 5.41 C^(X,R"^) is a vector space. Moreover, if A G C^{X,R) and / G CO(X,R^), then A / : X -^ R^ given by Xf{x) := A(x)/(x), xeX, belongs to C^(X,R^). In particular, 5.44 Corollary. Polynomials in n variables belong to C^(R^,R). Therefore, maps f '.W^ —^ W^ whose components are polynomials of n variables are continuous. In particular, linear maps L G £(R^,R'^) are continuous. It is worth noticing that in fact 5.45 Proposition. Let L :W^ ^^ W^ be linear. Then L is Lipschitz continuous in R^. Proof. As L is linear, we have Lip (/) : ==
sup x,yeR'^
\\X-y\\Rn
Xy^y
=
sup — x,y£R^ \\x-y\\un
=
sup —— o^zeR^ IPIIR'^
= : ||L||.
xj^y
Let us prove that ||L|| < H-CXD. Since L is continuous at zero by Corollary 5.44, there exists S > 0 such that ||L(ii;)|| < 1 whenever \\w\\ < S. For any nonzero 2 € M^, set w := 2Jnr\- Since ||ti;|| < 6, we have ||L(i(;)|| < 1. Therefore, writing z = ^y^w and using the linearity of L
||L(.)|| = | | « L W | | = « | 1 L H | | < ^ | N 1 hence
||L||<+oo.
For a more detailed description of linear maps in normed spaces, see Chapters 9 and 10. b. Some nonlinear continuous transformations from R^ into R ^ We now present a few examples of nonlinear continuous transformations between Euclidean spaces. 5.46 E x a m p l e . For fc = 0 , 1 , . . . consider the map w^ :] — 1,1[—> M^ given by , ,
J (cos kt, sin kt)
if t G]0, 27r/fc[,
1(1,0)
otherwise.
This is a Lipschitz function whose graph is given in Figure 5.10. Notice that the graph of Uk = {{t,Uk{t))} is a curve that "converges" as fc —>• oo to a horizontal Une plus a vertical circle at 0. Compare with the function sgn x from M to R.
168
5. Metric Spaces and Continuous Functions
2Tv/k
Figure 5.10. The function u^ in Example 5.46.
5.47 E x a m p l e ( S t e r e o g r a p h i c p r o j e c t i o n ) . Let
be the unit sphere in E^"*"^. If x = ( x i , . . . Xn.Xn^i) € R^"*"^, let us denote the coordinates of x by (2/, z) where y = (xi, X2,..., Xn) € M" and z = Xn-\-i € R. With this notation, S"^ = {(y, z) e M"" X M 112/|2 -j-z'^ = 1}. Furthermore, denote by Ps = (0, - 1 ) € 5 ^ the South pole of S"^. The stereographic projection (from the South pole) is the map that projects from the South pole the sphere onto the {z = 0} plane, cT : S " \ {Ps} C K " + i ^ R",
(y, z) -^
~ ^ . L -j- Z
It is easily seen that a is injective, surjective and continuous with a continuous inverse given by
that maps x E M.'^ into the point of S'^ lying in the segment joining the South pole of 5"' with X, see Figure 5.11. 5.48 E x a m p l e (Polar c o o r d i n a t e s ) . The transformation (T:E:=
Up,e)\p>0,
0 < ^ < 27r}-^R2,
(p,^) ^
(pcos(9,psin(9)
defines a map that is injective and continuous with range R^ \ {0}. The extension of the map to the third coordinate a : E X R - ^ R 2 xR--R^,
(p.O.z) -^
(pcose.psmO.z)
defines the so-called cylindrical coordinates in R^. 5.49 E x a m p l e (Spherical c o o r d i n a t e s ) . The representation of points {x,y, z) € R^ as I X — psinipcosO, I y = p simp sin 6, ^Z =
pCOSip,
see Figure 5.12, defines the spherical coordinates in R^. This in turn defines a continuous transformation (p,0,(p) —> {x,y,z) from E := | ( p , ^, (^) I /9 > 0, 0 < ^ < 27r, 0 < <^ < TT} = ] 0 , +OO[X [0,2n[x [0, n]
into R 3 \ {0}.
5.1 Metric Spaces
169
Figure 5.11. The stereographic projection from the South pole.
Complex-valued functions of one complex variable provide examples of tranformations of the plane. 5.50 E x a m p l e {w = z'^). The map z ^^ z'^ defines a continuous transformation of C to C. The inverse image of each nonzero n; G C is made by n distinct points, given by the n roots of w, those n points collapse to zero when it; = 0. If we write the transformation w = z"^ as \w\ == \z\'^ Argti; = n A r g z , we see, identifyng C with M^, that the circle of radius r and center 0 is mapped into the circle of radius r'^ and center 0. Moreover, if a point goes clockwise along the circle, then the normalized point image -^jp^ goes along the unit circle clockwise n times. 5.51 f. The map z —^ w = z'^ restricted to ipo < Argz < c^i with 0 < c^i — v?o < 27r/n is injective and continuous. 5.52 %. Show that the map z -^ w = z"^ maps the family of parallel lines to the axes (but the axes themselves) into two families of parabolas with the common axis as the real axis and the common foci at the origin, see Figure 5.13.
5.53 E x a m p l e ( T h e Joukowski f u n c t i o n ) . This is the map
A(.):=i(z+1),
P==
Figure 5.12. Spherical coordinates.
z/O,
{x,y,z)
170
5. Metric Spaces and Continuous Functions
Figure 5.13. The transformation w = z'^ maps families of lines parallel to the axes, except for the axes, into two families of parabolas with the common axis as the real axis and the common foci at the origin.
which appears in several problems of aerodynamics. It is a continuous function defined every point w ^ ± 1 , 0 has at most, and, in fact, exactly in C \ {0}. Since X{z) = \(l/z), two distinct inverse images zi,Z2 satisfying ziZ2 = 1. is one-to-one from {\z\ < 1, 2 / 0} or {\z\ > 1} 5.54 If. Show that X(z) = l/2{z-\-l/z) into the complement of the segment {w\ — 1 < dlw < 1}. A maps the family of circles {z I \z\ = r}ry 0 < r < 1, into a family of co-focal ellipses and maps the diameters z = te^°^, — 1 < ^ < 1 , 0 < a < 7 r , i n a family of co-focal hyperbolas, see Figure 5.14. 5.55 E x a m p l e ( T h e M o b i u s t r a n s f o r m a t i o n s ) . These maps, defined by L(^):==^i±^, cz -\- d
ad-bc^O
(5.10)
are continuous and injective from C \ {—d/c} into C \ {a/c} and have several relevant properties that we list below, asking the reader to show that they hold. 5.56 %. Show the following. (i) L(z) —> a/c as \z\ -^ oo and \L{z)\ -^ oo as z -^ —d/c. Because of this, we write L(oo) = a/c, L(—d/c) = oo and say that L is continuous from CU{oo} into itself. (ii) Show that every rational function, i.e., the quotient of two complex polynomials, defines a continuous transformation of C U {oo} into itself, as in (i). (iii) The Mobius transformations L{z) in (5.10) are the only rational functions from C U {oo} into itself that are injective. (iv) The Mobius transformations (aiZ -\- bi)/(ciZ -|- di), i = 1,2, are identical if and only if ( a i , 6i, ci, di) is a nonzero multiple of (a2, 62, C2, ^2)(v) The Mobius transformations form a group G with respect to the composition of maps; the subset H C G, H := {z, 1 — z, l/z, 1/(1 — z), (z — l)/z} is a subgroup
of a (vi) A Mobius transformation maps straight lines and circles into straight lines and circles (show this first for the map l/z, taking into account that the equations for straight lines and circles have the form A{x'^ + V^) + 2Bx + 2Cy + D = 0 if z = x-\-iy). (vii) The map in (5.10) maps circles and straight lines through —c/d into a straight line and any other straight line or circle into a circle.
5.1 Metric Spax:es
171
\ \ Figure 5.14. The Joukowski function maps circles \z\ = r, 0 < r < 1, and diameters z = ±6=*=*", 0 < t < l , 0 < a < 27r, respectively into a family of ellipses and of cofocal hyperbolas.
(viii) The only Mobius transformation with at least two fixed points is z. Two Mobius transformations are equal if they agree at three distinct points. There is a unique Mobius transformation that maps three distinct points 2:1,22, ^3 € C U {00} into three distinct points wi,W2jWs G C U {00}.
5.57 E x a m p l e ( E x p o n e n t i a l a n d l o g a r i t h m ) . The complex function z —> exp^;, see [GM2], is continuous from C —• C, periodic of period 27Ti with image C \ {0}. In particular e^ does not vanish, and every nonzero w has infinitely many preimages. 5.58 f. Taking into account what we have proved in [GM2], show the following. (i) ii; = e^ is injective with a continuous inverse in every strip parallel to the real axis of width /i < 27r, and has an image as the interior of an angle of radiants h and vertex at the origin; (ii) w = e^ maps every straight line which is not parallel to the axes into a logarithmic spiral, see Chapter 7.
c. The calculus of limits for functions of several variables Though we may have appeared pedantic, we have always insisted in specifying the domain E C X in which the independent variables varied. This is in fact particularly relevant when dealing with limits and continuity of functions of several variables, as in this case there are several reasonable ways of approaching a point XQ. Different choices may and, in general, do lead to different answers concerning the existence and/or the equality of the limits lim fix) and lim fix). xeE xeF Let (X, dx) and (F, dy) be two metric spaces, f : X -^Y a point of accumulation of X.
and XQ e X
172
5. Metric Spaces and Continuous Functions
Figure 5.15. The function in Example 5.59.
(i) If we find two sets Ei^E2 such that XQ is an accumulation point of both El and E2, and the restrictions f : Ei C X -^ Y and f : E2 C X -^Y oi f have diflFerent Hmits, then / has no Hmit when X ^y
XQ,
X e El U E2.
(ii) if we want to show that f{x) has hmit as a: ^ XQ, we may a) guess a possible limit yo £Y^ for instance computing the limit yo of a suitable restriction of / , b) show that the real-valued function x -^ dy(/(x),2/o) converges to zero as X -^ xo, for instance proving that dy (/(x), yo) < h{x) for all x G X, x ^ XQ, where h : X -^R is such that h{x) -^ 0 as x -^ XQ. 5.59 E x a m p l e . Let / : E^ \ {(0,0)} -^ M be defined by f(x,y) := xy/{x^ + 2/^) for {x,y) 7^ (0,0). Let us show that / has no limit as (x^y) —• (0,0). By contradiction, suppose that f{x,y) —> L G R as (x,y) —>• (0,0). Then for any sequence {(xn,2/n)} C R^\{(0,0)} converging to (0,0) we find f{xn,yn) —>• L- Choosing {xn.yn) '-— (l/'^, ^7^)? we have
hence, a s n - ^ 0 0 , L = k/{l-\-k?). Since k is arbitrary, we have a contradiction. This is even more evident if we observe that / is positively homogeneous of degree 0, i.e., /(Ax, \y) = f{x, y) for all A > 0, i.e., / is constant along half-lines from the origin, see Figure 5.15. It is then clear that / has limit at (0,0) if and only if / is constant, which is not the case. Notice that from the inequality 2xy < x^ -^y^ we can easily infer that \f{x, y)\ < 1/2 V(x, y) € R^ \ {(0, 0)}, i.e., that / is a bounded function. 5.60 E x a m p l e . Let fix,y) := sm(x'^y)/{x'^ +2/^) for {x,y) ^ (0,0). In this case ( l / n , 0 ) -» (0,0) and / ( l / n , 0 ) = 0 - ^ 0 . Thus 0 is the only possible limit as (x,y) -^ (0,0); and, in fact it is, since \ff^ o.\ n\ |sin(x2y)| ^ |x| |y| ,^, ^ 1 ,^, ^ |/(x,2/)-0 = < \x\ < - x| - ^ 0 x^ + y^ x^ -\-y^ 2 as (x,2/) —^ (0,0). Here we used | sin t\ < \t\ Vt, 2|x| |y| < x2 -h 2/^ V(x,2/) and that (x,2/) -^ |x| is a continuous map in R^, see Proposition 5.41.
5.1 Metric Spaces
173
We can also consider the restriction of / to continuous paths from XQ, i.e., choose a map (f : [0,1] -^ R^ that is continuous at least at 0 with (p{0) = XQ and ^{t) ^ XQ ior t ^ 0 and compute, if possible
Such limits may or may not exist and their values depend on the chosen path, for a fixed / . Of course, if xeE then, on account of the restriction property and of the change of variable theorem,
i i S , /(^) = ^
and
lim f{^{t)) = L
xeF respectively for any F C E oi which XQ remains a point of accumulation and for any continuous path in E, (^([0,1]) C E. 5.61 E x a m p l e . Let us reconsider the function / :
R2
\ {(0, 0)} ^ M,
fix, y) :=
- ^ ^ x^ + 2/^
which is continuous in R-^ \ {(0, 0)}. Suppose that we move from zero along the straight Hne {(x,y)\y — mx,a: € M} that we parametrize \yy x -^ (x,mx). Then f{(p{x)) = f{x,mx)
= —-^ — -, as X -> 0, 1 + m^ 1 + m^ in particular, the previous limit depends on m, hence f{x, y) has no limit as (x, y) -^ (0,0). Set E := {{x,y) | x € M, 0
= 0.
In fact, in this case
0 < J ^ < ^ = A N X + y^
X'^
i„E.
5.62 E x a m p l e . The function
fi^,y) = { ^
if(x,v)eEM{(0,0)}, if (x, 3/) = (0,0)
is continuous in M? \ {(0,0)} but is not continuous at (0,0). Restricting / to a straight line through (0,0) parametrized as (p{t) = (ta, tb), t € K, gives
f(^it)) = fiat,bt) = - ^ ^ ^ = J^t^O
ast^O.
However, restricting / along the graph of the function y = ax^ parametrized as (^(x) := (x,ax'^), gives / ( x , a x ) = -—-—r— x^ + a^x^
-> — - — - , 1 + a^
asx-^0,
174
5. Metric Spaces and Continuous Functions
MONOGRARTE
MATEMATYCZNE
|
KOMITET REDAKCYJNY: S. B A M A C H . B. KKA5T£il. K. KV&ATOWSKI. s. MAatoBKiBWicz, w. saM.ntiitit
i «.
srtwHAVs
TOMin
TOPOLOGIE I E5PACE5 M £ T R I S A B L E 5 . &SPACE5 COMPLETE
C A S I M I R
K U R A T O W S K I
. X I'CCOie POtTTeCHNIQI,'! OZ IWOv
Z SUBWEMCJI FU»Z)0*ZU KCtTgKT K A * 0 » 0 W E ; W A R S Z A W A - L W d W
iqSS
Figure 5.16. Kazimierz Kuratowski (1896-1980) and the frontispiece of the first volume of his Topologie.
thus / has no Hmit as {x, y) —> (0, 0). Let us now consider the restriction of / to the set
E:=\{x,y)\x>0,
\y\ < x^}.
We have lim
/ ( x , y) — 0.
{x,y)eE In fact,
I x^y
•0
=
J.4 -1-2/2
x^ -\' 2/2
< kl -^ 0,
since (x, y) G £?.
We conclude by observing that for functions / : R^
the expression
lim f{x) = +00 means that VM G R there is J > 0 such that f{x) > MWxe
B{xo,
5)\{XQ}.
5.2 The Topology of Metric Spaces In this section we introduce some families of subsets of a metric space X that are defined by the metric structure, namely the families of open and closed sets. Recall that if X is a set, V{X) denotes the set of all subsets of
X: Ae V{X) if and only
liAdX.
5.2 The Topology of Metric Spaces
175
5.2.1 Basic facts a. Open sets 5.63 Definition. A subset A of a metric space (X, d) is called an open set if for all x £ A there exists a ball centered at x contained in A, i.e., Vx G A 3ra, > 0 such that B{x,rx)
C A.
(^-H)
5.64 Proposition. A subset A of a metric space X is open if and only if either A is empty or is a union of open balls. Proof. Let A be open. Then either A is empty or A is trivially a union of open balls, A — \Jr^^AB(x,rx). Conversely, (5.11) trivially holds if A = 0. If instead x G A 7^ 0, since we assume that A is union of balls, there is 2/ 6 X and p > 0 such that x G ^(2/5 p) G A. Thus y ^ A and, setting r := p — d{x, y), we have r > 0 and by the triangle inequality B{x^r) C B{y,p) C A. •
In particular, 5.65 Corollary. The open balls of a metric space X are open sets. 5.66 %. Let (X, d) be a metric space and r > 0. Show that {y E X \ d{y, x) > r} is an open set in X. 5.67 %, Let (X, d) be a metric space. Show that {xn} C X converges to x G X if and only if, for any open set A such that x £ A, there exists n such that Xn E A for all n >n.
The following is also easily seen. 5.68 Proposition. Let {X, d) be a metric space. Then (i) 0 and X are open sets, (ii) if {Aa} is a family of open sets, then UaA^ is an open set, too, (iii) zf 74i, A 2 , . . . , An are finitely many open sets, then n^^^A^ is open. 5.69 % By considering the open sets {] — - , ^ [ | n G N}, show that the intersection of infinitely many open sets needs not be an open set.
b. Closed sets Recall that the complement of A C X is the set A^ := X \ A. 5.70 Definition. Let X be a metric space. F C X is called a closed set is an open set. if F^ — X\F
176
5. Metric Spax^es and Continuous Functions
The de Morgan formulas
a
a
together with Proposition 5.68 yield at once the following. 5.71 Proposition. Let X be a metric space. Then (i) 0 and X are closed sets, (ii) the intersection of any family of closed sets is a closed set, (iii) the union of finitely many closed sets is a closed set. 5.72 %. Show that [a, 6], [a,+oo[ and [—a, H-oo[ are closed sets in E, while [0,1[ is neither closed nor open. 5.73 if. Show that the set { - | n = 1,2,... } is neither closed nor open. 5.74 %, Show that any finite subset of a metric space is a closed set. 5.75 ^ . Show that the closed ball {x € X\d{x,xo) X I d(x, xo) = r} are closed sets.
< r}, and its boundary {x €
One may characterize closed sets in terms of convergent sequences. 5.76 Proposition. Let {X,d) be a metric space. A set F C X is a closed set if and only if every convergent sequence with values in F converges to a point of F. Proof. Suppose that F is closed and that {x^} C F converges to x € X. Let us prove that x E F. Assuming on the contrary, that x f F, there exists r > 0 such that B(x,r) fl F = 0. As {xn} C F , we have d{xn,x) > r Vn, a contradiction since d{xn,x) —> 0. Conversely, suppose that, whenever {xk} C F and Xk -^ x, we have a: G F , but F is not closed. Thus X \F is not open, hence there exists a point x £ X \F such that Vr > 0 B{x, r) n F ^ 0. Choosing r = l , | , | , . . . , w e inductively construct a sequence {xn} C F such that d{xn,x) < ^, hence converging to x. Thus x e F by assumption, but X £ X \F by construction, a contradiction. •
c. Continuity 5.77 Theorem. Let {X,dx) and {Y^dy) be two metric spaces and f : X —^Y. Then the following claims are equivalent (i) (ii) (iii) (iv)
/ is continuous, f~^{B) is an open set in X for any open ball B ofY, f~^{A) is an open set in X for any open set A inY, f~^(F) is a closed set in X for any closed set F inY.
5.2 The Topology of Metric Spaces
177
Proof, (i) => (ii). Let B be an open ball in Y and let x be a point in f~^{B). Since f(x) E B, there exists a ball BY{f{x),e) C B. Since / is continuous at x, there exists (5 > 0 such that f{Bx{x,S)) C B y ( / ( x ) , e ) C B that is B x ( x , 5 ) C / " ^ ( B ) . As x is arbitrary, f~^{B) is an open set in X. (ii) =^ (i) Suppose / " ^ B ) is open for any open ball B of y . Then, given XQ, / ~ ^ ( B y ( / ( x o , 6 ) ) ) is open, hence there is (5 > 0 such that Bx{xo,S) is contained in /~-^(By(/(xo,6))), i.e., f(Bx{xoyS)) C B y ( / ( x o ) , € ) , hence / is continuous at XQ. (ii) and (iii) are equivalent since f~^{UiAi) ofX.
= Uif~^{Ai)
for any family {Ai} of subsets
(iii) and (iv) are equivalent on account of the de Morgan formulas.
•
5.78 %. Let / , p : X —> y be two continuous functions between metric spaces. Show that the set {x £ X \ / ( x ) = 9{x)} is closed. 5.79 ^ . It is convenient to set Definition. Let (X, d) be a metric space. U C X is said to be a neighborhood of XQ £ X if there exists an open set A of X such that XQ £ A C U. In particular o B(xo,r) is a neighborhood of any x € B ( x o , r ) , o A is open if and only if A is a neighborhood of any point of A. Let {X, d), (y, d) be two metric spaces let XQ e X and let / : X -^ y . Show that / is continuous at XQ if and only if the inverse image of an open neighborhood of / ( X Q ) is an open neighborhood of XQ .
Finally, we state a junction rule for continuous functions, see Proposition 5.38. 5.80 Proposition. Let (X, d) be a metric space, and let {Ua} be a covering of X. Suppose that either all Ua ^s are open sets or all Ua 's are closed and for any x £ X there is an open ball that intersects only finitely many Ua- Then (i) A
d. Continuous real-valued maps Let (X, d) be a metric space and / : X ^> R. Prom Theorem 5.77 we find that / : X ^ ' R is continuous if and only if /~^(]a, b[) is an open set for every bounded interval ]a, fe[c M. Moreover, 5.82 Corollary. Let f : X ^^ R be a function defined on a metric space X and let t e R. Then
178
5. Metric Spaces and Continuous Functions
(i) {x e X\ f{x) > t}, {x e X\ f{x) < t} are open sets, (ii) {x e X\ f{x) >t},{xeX\ f{x) < t} and {x £ X \ f{x) = t} are closed sets. 5.83 Proposition. Let (X^d) be a metric space. Then F C X is a closed set of X if and only if F = {x\ d(x, F) =0}. Proof. By Corollary 5.82, {x \ d{x, F) = 0} is closed, x -> d{x, F) being Lipschitz continuous, see Example 5.25. Therefore F = {x\ d(x, F) = 0} implies that F is closed. Conversely assume that F is closed and that there exists x ^ F such that d{x, F) = 0. Since F is closed by assumption, there exists r > 0 such that B{x, r) D F = 0. But then (i(x, F ) > r > 0, a contradiction. D 5.84 f. Prove the following P r o p o s i t i o n . Let (X, d) he a metric space. (i) F C such (ii) A C such
X is that X is that
a closed set if and F = {x e X\ f{x) an open set if and A = {x e X\ f{x)
Then
only if there exists a continuous function f : X -^R < 0}, only if there exists a continuous function f : X ^>-R < 0}.
Actually f can be chosen to be a Lipschitz-continuous [Hint: If F is closed, choose f{x) := d{x,F)^
function.
while if A is an open set, choose f{x)
=
-d(x,X\A).]
e. The topology of a metric space 5.85 Definition. The topology of a metric space X is the family TX C V{X) of its open sets. It may happen that different distances di and ^2 on the same set X that define diff'erent famihes of balls produce the same family of open sets for the same reason that a ball is union of infinitely many squares and a square is union of infinitely many balls. We say that the two distances are topologically equivalent if {X,di) and (X, ^2) have the same topology, i.e., the same family of open sets. The following proposition yields necessary and sufficient conditions in order that two distances be topologically equivalent. 5.86 Proposition. Let di, d2 be two distances in X and let Bi{x,r) and B2(x^r) be the corresponding balls of center x and radius r. The following claims are equivalent (i) di and ^2 ^^^ topologically equivalent, (ii) every ball Bi{x,r) is open for 0^2 OL^d every ball B2{x^r) is open for di. (iii) \/x e X and r > 0 there are r^^Px > 0 such that B2{x, r^) C Bi{x, r) and Bi[x,px) C B2[x,r), (iv) the identity map i : X -^ X is a homeomorphism between the metric spaces (X^di) and (X, 0J2).
5.2 The Topology of Metric Spaces
179
Figure 5.17. x is an interior point to A, y is s. boundary point to A and z is an exterior point to A. X and y are adherent points to A and z is not.
5.87 ^ . Show that the distances in R^ doo and dp Vp > 1, see Exercise 5.13, are all topologically equivalent to the Euclidean distance d2- If we substitute W^ with the infinitely-dimensional vector space of sequences ii, the three distances give rise to different open sets.
We say that a property of X is a topological property of X if it can be expressed only in terms of set operations and open sets. For instance, being an open or closed set, the closure of or the boundary of, or a convergent sequence are topological properties of X, see Section 5.2.2 for more. As we have seen, / is continuous if and only if the inverse image of open sets is open, A trivial consequence, for instance, is that the composition of continuous functions is continuous, see Proposition 5.30. Also we see that the continuity of / : X —> y is strongly related to the topologies Tx '•= {Ac
X\A
open in X } ,
ry := {AcY\A
open in F } ,
respectively on X and y , and in fact it depends on the metrics only through Tx and Ty. In other words being a continuous function f : X ^^ Y is a, topological property of X and Y. f. Interior, exterior, adherent and boundary points 5.88 Definition. Let X be a metric space and A C X. We say that XQ G X is interior to A if there is an open ball B{xQ,r) such that B{xo^r) C A; we say that XQ is exterior to A if XQ is interior to X \ A; we say that XQ is adherent to A if it is not interior to X \ A; finally, we say that XQ is a X\A. boundary point of A if XQ is neither interior to A nor interior to o
The set of interior points to A is denoted by A or by int A, the set of adherent points of A, called also the closure of A, is denoted by A or by cl (A), and finally the set of boundary points to A is called the boundary of A and is denoted by dA. 5.89 ^ . Let (X, d) be a metric space and B{xo,r) be an open ball of X. Show that (i) every point of B{xo,r) is interior to B{xo,r), i.e., intB{xo,r) = B{xo,r), (ii) every point x such that d(x,xo) = r is a boundary point to B{xo,r), i.e., dB{xo,r) = {x I d{x, XQ) = r } , (iii) every point x with d{x,xo) > r is exterior to B{xo,r),
180
5. Metric Spaces and Continuous Functions
(iv) every point x such that d{x^xo) < r is adherent to B{XO^T)^ {x\d{x,XQ) < r } . 5.90 %, Let X be a metric space and Ac (i) int Ac A,
i.e., c\{B{xo^r))
=
X. Show that
(ii) int A is an open set and actually the largest open set contained in A, i n t ^ = u | c / I U open U C A \ (iii) A is open if and only if >1 = int v4. 5.91 %. Let X be a metric space and A (Z X. Show that (i) ^ C A , (ii) A is closed and actually the smallest closed set that contains A, i.e., cl {A) = n J F I F closed , F D
A^
(iii) A is closed if and only if A = A, (iv) A = {xeX\d(x,A) = 0}. 5.92 (i) (ii) (iii)
%, Let X be a metric space and A C X. Show that dA = d{X\A),_ dAnmtA = i/}, A = dAU int A,dA = A\ int A, ^A = An A^, in particular dA is a closed set,
(iv) ddA = 0, A = A, int int A = int A, (v) >l is closed if and only if dA C 4 , (vi) A is open if and only if dA fl A = 0. 5.93 %, Let {X,dx) and ( F , d y ) be metric spaces and / : X —>• V. Show that the following claims are equivalent (i) / : X —>^ y is continuous, (ii) f(A) C JjA) foTjll ACX, for all BcY. (iii) / - i ( ^ ) C f-'^(B)
g. Points of accumulation Let A C Xhesi subset of a metric space. The set of points of accumulation, or cluster points, of A, denoted by DA, is sometimes called the derived of A. Trivially T>A C A, and the set of adherent points to A that are not points of accumulation of A, X{A) := ^ \ DA, are the points x e A such that J5(x, r) r\A = {x} for some r > 0. These points are contained in A, I{A)=A\VAcA and are called isolated points of A. 5.94 % Show that X>A C A and that A is closed if and only if VA C A.
5.95 Proposition. Let (X,d) be a metric space, F C X and x e X. We have
5.2 The Topology of Metric Spaces
181
(i) X is adherent to F if and only if there exists a sequence {xn} C F that converges to x, (ii) X is an accumulation point for F if and only if there exists a sequence {xn} C F taking distinct values in F that converges to x; in particular, a) X is an accumulation point for F if and only if there exists a sequence {xn} C F\ {x} that converges to x, b) in every open set containing an accumulation point for F there are infinitely many distinct points of F. Proof, (i) If there is a sequence {xn} C F that converges to x G X , in every neighborhood of X there is at least a point of F , hence x is adherent to F. Conversely, if x is adherent to F, there IS a XYI G B{x, ^ ) n F for each n, hence {xn} C F and Xfi ^ X. (ii) If moreover x is a point of accumulation of F, we can choose Xn E F \ {x} and moreover Xn G B{x,rn), Vn •= mm{d{x,Xn-i), - ) • The sequence {xn} has the desired properties. D
h. Subsets and relative topology Let (X^d) be a metric space and Y C X. Then (Y^d) is a metric space, too. The family of open sets in Y induced by the distance d is called the relative topology of Y, We want to compare the topology of X and the relative topology of Y. The open ball in Y with center x ^Y and radius r > 0 is Bvix.r)
:= ^yeY^d{y,x)
< r|
=Bx{x,r)nY.
5.96 Proposition. Let {X,d) be a metric space and let Y C X.
Then
(i) B is open in Y if and only if there exists an open set A C X in X such that B = AnY, (ii) B is closed in Y if and only if there exists a closed set A in X such that B = AnY. Proof. Since (ii) follows at once from (i), we prove (i). Suppose that A is open in X and let x be a point in An Y. Since A is open in X , there exists a ball Bx{x,r) C A, hence By(x,r) = Bx{x,r)nY C AnY. Thus AnY is open in Y. Conversely, suppose that B is open in Y. Then for any x e B there is a ball Byix^rx) = Bx{x,rx) n B C B. The set A := U{Bxix,rx) | a: G B } is an open set in X and AnY = B. D
Also the notions of interior, exterior, adherent and boundary points, in (y, d) are related to the same notions in (X, d), and whenever we want to emphasize the dependence on_y' of the interior, closure, derived and boundary sets we write inty(yl), Ay, VyA, dyA instead of int A, A, DA, dA, 5.97 Proposition. For any A CY (i) mty{A)_=mtx{A)nY, (ii) Ay = AxnY,
we have
182
5. Metric Spaces and Continuous Functions
dyA Figure 5.18. dA and
(iii) VYA = (iv) dYA =
dyA.
VxAr]Y, dxA\dxY.
5.98 f. Let Y := [0,1[C R. The open balls of Y are the subsets of the type {y G [0,1[ I I?/ — a:;| < r } . If x is not zero and r is sufficiently small, {y | I?/ — ic| < r } fl [0,1[ is again an open interval with center x,]x — r^x -\- r[. But, if x = 0, then for r < 1 B y ( 0 , r ) := [0,r[. Notice that x = 0 is an interior point of Y (for the relative topology of Y), but it is a boundary point for the topology of X. This is in agreement with the intuition: in the first case we are considering y as a space in itself and nothing exists outside it, every point is an interior point and dyY = i/}; in the second case y is a subset of R and 0 is at the frontier between Y and M \ y . 5.99 ^ . Prove the claims of this paragraph that we have not proved.
5.2.2 A digression on general topology a. Topological spaces As a further step of abstraction, following Felix Hausdorff (1869-1942) and Kazimierz Kuratowski (1896-1980), we can separate the topological structure of open sets from the metric structure, giving a set-definition of open sets in terms of their properties. 5.100 Definition. Let X be a set. A topology in X is a distinct family of subsets T C V{X), called open sets, such that o 0, X G r, o if {Aoc} C T, then Ua^a ^ T, o if Ai,A2,...,AneT, thenOl^^Ak^T. A set X endowed with a topology is called a topological space. Sometimes we write it as (X^r). 5.101 Definition. A function f : X -^ Y between topological spaces (X,Tx) and {Y,TY) is said to be continuous if f~^{B) e TX whenever
5.2 The Topology of Metric Spaces
183
B e TY' f : X ^^ Y is said to be a homeomorphism if f is both injective and surjective and both f and f~^ are continuous, or, in other words A G Tx if and only if f{A) G ryTwo topological spaces are said to be homeomorphic if and only if there exists a homeomorphism between them. Proposition 5.68 then reads as follows. 5.102 Proposition. Let (X, d) be a metric space. Then the family formed by the empty set and by the sets that are the union of open balls of X is a topology on X, called the topology on X induced by the metric d. The topological structure is more flexible than the metric structure, and allows us to greatly enlarge the notion of the space on which we can operate with continuous deformations. This is in fact necessary if one wants to deal with qualitative properties of geometric figures, in the old terminology, with analysis situs. We shall not dwell on these topics nor with the systematic analysis of different topologies that one can introduce on a set, i.e., on the study of general topology. However, it is proper to distinguish between metric properties and topological properties. According to Felix Klein (1849-1925) a geometry is the study of the properties of figures or spaces that are invariant under the action of a certain set of transformations. For instance, Euchdean plane geometry is the study of the plane figures and of their properties that are invariant under similarity transformations. Given a metric space (X, d), a property of an object defined in terms of the set operations in X and of the metric of X is a metric property of X, for instance whether {xn} C X is convergent or not is a metric property of X. More generally, in the class of metric spaces, the natural transformations are those h : (X, dx) -^ {Y, dy) that are one-to-one and do not change the distances dy{h{x)^h{x)) = dx{x,y). Also two metric spaces (X,d) and (y, d) are said to be isometric if there exists an isometry between them. A metric invariant is a predicate defined on a class of metric spaces that is true (respectively, false) for all spaces isometric with (X, d) whenever i