MATRIX PARTIAL ORDERS, SHORTED OPERATORS AND APPLICATIONS
SERIES IN ALGEBRA Series Editors: Derek J S Robinson (Unive...
63 downloads
1163 Views
2MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
MATRIX PARTIAL ORDERS, SHORTED OPERATORS AND APPLICATIONS
SERIES IN ALGEBRA Series Editors: Derek J S Robinson (University of Illinois at Urbana-Champaign, USA) John M Howie (University of St. Andrews, UK) W Douglas Munn (University of Glasgow, UK)
Published Vol. 1
Infinite Groups and Group Rings edited by J M Corson, M R Dixon, M J Evans and F D Röhl
Vol. 2
Sylow Theory, Formations and Fitting Classes in Locally Finite Groups by Martyn R Dixon
Vol. 3
Finite Semigroups and Universal Algebra by Jorge Almeida
Vol. 4
Generalizations of Steinberg Groups by T A Fournelle and K W Weston
Vol. 5
Semirings: Algebraic Theory and Applications in Computer Science by U Hebisch and H J Weinert
Vol. 6
Semigroups of Matrices by Jan Okninski
Vol. 7
Partially Ordered Groups by A M W Glass
Vol. 8
Groups with Prescribed Quotient Groups and Associated Module Theory by L Kurdachenko, J Otal and I Subbotin
Vol. 9
Ring Constructions and Applications by Andrei V Kelarev
Vol. 10 Matrix Partial Orders, Shorted Operators and Applications by Sujit Kumar Mitra*, P Bhimasankaram and Saroj B Malik
*
Deceased
SERIES
MATRIX PARTIAL ORDERS, SHORTED OPERATORS AND APPLICATIONS
IN
ALGEBRA VOLUME 10
Sujit Kumar Mitra Indian Statistical Institute, India
P Bhimasankaram University of Hyderabad, India
Saroj B Malik Hindu College, University of Delhi, India
World Scientific NEW JERSEY
•
LONDON
•
SINGAPORE
•
BEIJING
•
SHANGHAI
•
HONG KONG
•
TA I P E I
•
CHENNAI
Published by World Scientific Publishing Co. Pte. Ltd. 5 Toh Tuck Link, Singapore 596224 USA office 27 Warren Street, Suite 401-402, Hackensack, NJ 07601 UK office 57 Shelton Street, Covent Garden, London WC2H 9HE
British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library.
Series in Algebra — Vol. 10 MATRIX PARTIAL ORDERS, SHORTED OPERATORS AND APPLICATIONS Copyright © 2010 by World Scientific Publishing Co. Pte. Ltd. All rights reserved. This book, or parts thereof, may not be reproduced in any form or by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system now known or to be invented, without written permission from the Publisher.
For photocopying of material in this volume, please pay a copying fee through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA. In this case permission to photocopy is not required from the publisher.
ISBN-13 978-981-283-844-5 ISBN-10 981-283-844-9
Printed in Singapore.
To Asha Latha Mitra, Sunil Kumar Mitra and Sheila Mitra - Parents and Wife of Sujit Kumar Mitra
Preface Matrix orders have fascinated mathematicians and applied scientists alike for many years. Several matrix orders have been developed during the past four decades by researchers working in Linear Algebra. Some of them are pre-orders such as the space pre-order and the Drazin pre-order, while others are partial orders like the minus order, the sharp order and the star order. These developments have grown symbiotically with advances in other areas such as Statistics and Matrix Generalized Inverses. Two closely connected concepts - parallel sums and shorted operators for non-negative definite matrices - play a major role in the study of electrical networks. Both of these share nice relationships with matrix partial orders as also between themselves. Several extensions of these concepts have been developed in the recent past by researchers working in these and related areas. There are many research articles in the areas of matrix orders and shorted operators that are scattered in various journals. This is the first full length monograph on these topics. The aim of this monograph is to present the developments in the fields of matrix orders and shorted operators for finite matrices in a unified way and illustrate them with suitable applications in Generalized Inverses, Statistics and Electrical Networks. In the process of this compilation, many new results have evolved. Virtually every chapter in the monograph contains results unpublished hitherto. In fact, Chapter 13 on partial orders of modified matrices comprises entirely of new material. We believe that dissection of matrices through matrix decompositions helps in clear understanding of the anatomy of various matrix orders in a transparent manner. We employ simultaneous decompositions such as the simultaneous normal form, the simultaneous singular value decomposition and the generalized singular value decomposition extensively in developing the properties of matrix orders. Accordingly, the reader will find new and
vii
viii
Matrix Partial Orders, Shorted Operators and Applications
intuitive proofs to several known results in this monograph. We also pose some open problems which should be of interest to researchers in these topics. There are a number of exercises at the end of virtually every chapter. These are expected to serve the dual purpose of helping in the understanding of the topics covered in the text and of introducing other related results that have not been included in the main text. This monograph is aimed at (i) graduate students and researchers in Matrix Theory and (ii) researchers in Statistics and Electrical Engineering who may use these concepts and results as tools in their work. The monograph can be used as a text for a one-semester graduate course in advanced topics in Matrix Theory and it can also serve as self-study text for those who have knowledge in basic Linear Algebra, perhaps at the level of [Rao and Bhimasankaram (2000)]. We deliberately avoided the study of majorization as there is an excellent book [Marshall and Olkin (1979)] on this topic. In this monograph, we consider matrix orders and shorted operators for finite matrices over a field. We do not consider matrices over more general algebraic structures or operators over more general spaces. There has been a good deal of work in these directions - see [Drazin (1958)], [Hartwig (1979)], [Hartwig (1980)], [Jain, Srivastava, Blackwood and Prasad (2009)], [Morley and William (1990a,b)] and [Mitch (1986)], to name a few. We have not included them here in order to keep the monograph focused and less bulky. We have also not gone into order preserving and order reversing transformations (for the minus, star, sharp and one-sided orders) and application of the sharp order in the analysis of Markov chains. These are exciting topics of research that may develop over the next few years. This monograph was conceived by the first author, S. K. Mitra who was a pioneer in the development of theory of matrix partial orders and shorted operators. He introduced the sharp order, one-sided orders and a unified theory of matrix partial orders. He developed several approaches to the shorted operators along with applications in Generalized Inverses and Statistics. The first author who is no more, has been the mentor and the main source of inspiration for the other two authors of the monograph. However, we, the second and third authors are responsible for any errors in the monograph. July, 2009
P. Bhimasankaram and Saroj B. Malik
Acknowledgements
We are highly thankful to the SQC & OR Unit, Indian Statistical Institute, Hyderabad, the Department of Mathematics and Statistics, University of Hyderabad, Hyderabad, the Center for Analytical Finance, Indian School of Business, Hyderabad and Hindu College, Delhi University, Delhi for providing excellent facilities during different stages of the preparation of this monograph. We are deeply indebted to Professor Debasis Sengupta for his constructive comments on parts of the manuscript in its different stages of preparation which led to significant improvement in the presentation. We are also thankful to Professors Probal Chaudhuri, Thomas Mathew and PSSNV Prasada Rao for useful discussions. We thank Professors T. Amarnath, R. Tandon, V. Suresh and Shri ALN Murthy for their encouragement during the preparation of the manuscript. P. Bhimasankaram records his sincere thanks and appreciation for the encouragement and inspiration received from Professor Sankar De during the preparation of the manuscript. His family members, Amrita, Chandana, Shilpa, Chandu and Vijaya had to miss his company even during the weekends for years together during the preparation of the manuscript. But for the understanding, patience and cooperation received from them, he could never have completed the task satisfactorily. He expresses his deep sense of loving appreciation to them. Saroj Malik expresses her sincere thanks to the Department of Mathematics and Statistics, University of Hyderabad for providing local hospitality during her visits to University of Hyderabad during the preparation of the manuscript (under the SAP-UGC project). She also wishes to thank the National Academy of Sciences, Delhi for partial support during one such visit. While working on the monograph, she had several useful discussions
ix
x
Matrix Partial Orders, Shorted Operators and Applications
with Professor Ajeet I. Singh, Professor Emeritus, Stat.-Math. Division, Indian Statistical Institute, Delhi. She is deeply indebted to Professor Singh for all the help and encouragement. She wishes to thank her ex-Principal Dr. Kavita Sharma for her encouragement during all these years to bring this effort to a fruitful ending. We thank Manpreet Singh and Vishal Mangla for their help in preparing the S-diagram and Naveen Reddy for his help in editing the manuscript at various stages. We thank L. F. Kwong for her co-operation at every stage during the preparation of the manuscript. We also thank D. Rajesh Babu for the technical help. Finally, we wish to thank World Scientific Publishing Co., Singapore for providing us the necessary freedom and flexibility in completing the monograph.
Glossary of Symbols and Abbreviations
aij (aij ) At A? C(A) C(At ) C(A⊥ ) N (A) tr(A) ρ(A) d(S) det(A) A−1 L A−1 R A− {A− } A− r {A− r } A− com {A− com } A† A] AD A− ` {A− ` } A− m {A− m} A− ρ
- the (i, j)th element of the matrix A - the matrix whose (i, j)th element is aij - transpose of the matrix A - conjugate transpose of the matrix A - the column space of the matrix A - the row space of the matrix A - the orthogonal complement of C(A) - the null space of the matrix A - the trace of the matrix A - the rank of the matrix A - the dimension of the subspace S - the determinant of the matrix A - a left inverse of the matrix A - a right inverse of the matrix A - a g-inverse of the matrix A - the set of g-inverses of the matrix A - a reflexive g-inverse of the matrix A - the set of reflexive g-inverses of the matrix A - a commuting g-inverse of A - the set of commuting g-inverses of A - the Moore-Penrose inverse of the matrix A - the group inverse of the matrix A - the Drazin inverse of the matrix A - a least squares g-inverse of the matrix A - the set of least squares g-inverses of the matrix A - a minimum norm g-inverse of the matrix A - the set of minimum norm g-inverses of the matrix A - a commuting g-inverse of the matrix A with the propxi
xii
Matrix Partial Orders, Shorted Operators and Applications t
{A− ρ} A− χ {A− χ} λmax (A) λmin (A) σ(A) C R F Cn m×n C Fn d(V ) I1 I1,n F m×n H Hn I Ik diag(x1 , x2 , . . . , xn ) P(A|B) S(A|B) S(A|S, T )
G(A) G(A, B) Gr (A) ] G(A) ˜ Gr (A) P(F m×n ) m-column vector n-row vector
t erty that C(A− ρ ) ⊆ C(A ) - the set of all ρ-inverses of the matrix A - a commuting g-inverse of the matrix A with the property that C(A− χ ) ⊆ C(A) - the set of all χ-inverses of the matrix A - the maximum eigen-value of A - the minimum eigen-value of A - the set of all singular values A - the field of complex numbers - the field of real numbers - arbitrary field - the vector space of complex n-tuples - the set of all m × n matrices over C - the vector space of n-tuples over F - the dimension of vector space V - the set of all matrices of index ≤ 1 - the set of all n × n matrices of index ≤ 1 - the set of all m × n matrices over F - the set of all hermitian matrices - the set of all n × n hermitian matrices - the identity matrix - the k × k identity matrix - the n × n diagonal matrix with diagonal elements xi , i = 1, 2, . . . , n - the parallel sum of matrices A, B - the shorted matrix of A with respect to the matrix B - the shorted matrix of an m×n matrix A with respect to the subspace S of F m and the subspace T of F n , also read as the shorted matrix of an m × n matrix A indexed by subspaces S and T - a subset of {A− } - the set of {B− AB− : B− ∈ G(B)} - the set G(A) ∩ {A− r } - completion of G(A) ] ∩ {A− } - the set G(A) r - set of all subset of F m×n - an m × 1 matrix - a 1 × n matrix
Contents
Preface
vii
Acknowledgements
ix
Glossary of Symbols and Abbreviations
xi
1.
Introduction
1
1.1 Matrix orders . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Parallel sum and shorted operator . . . . . . . . . . . . . 1.3 A tour through the rest of the monograph . . . . . . . . . 2.
Matrix Decompositions and Generalized Inverses 2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8
3.
Introduction . . . . . . . . . . . . . . . . . Matrix decompositions . . . . . . . . . . . Generalized inverse of a matrix . . . . . . The group inverse . . . . . . . . . . . . . Moore-Penrose inverse . . . . . . . . . . . Generalized inverses of modified matrices Simultaneous diagonalization . . . . . . . Exercises . . . . . . . . . . . . . . . . . .
. . . . . . . .
9 . . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
The Minus Order 3.1 3.2 3.3 3.4
1 3 4
9 10 17 26 36 46 55 64 67
Introduction . . . . . . . . . . . . . . . . . . . . Space pre-order . . . . . . . . . . . . . . . . . . Minus order - Some characterizations . . . . . . Matrices above/below a given matrix under the minus order . . . . . . . . . . . . . . . . . . . . xiii
. . . . . . . . . . . . . . . . . .
67 68 72
. . . . . .
81
xiv
Matrix Partial Orders, Shorted Operators and Applications
3.5 Subclass of g-inverses A− of A such and AA− = BA− when A <− B . . 3.6 Minus order for idempotent matrices 3.7 Minus order for complex matrices . . 3.8 Exercises . . . . . . . . . . . . . . . 4.
7.
Introduction . . . . . . . . . . . . . . . Sharp order - Characteristic properties Sharp order - Other properties . . . . Drazin order and an extension . . . . Exercises . . . . . . . . . . . . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
Introduction . . . . . . . . . . . . . . . . . . Star order - Characteristic properties . . . . Subclasses of g-inverses for which A
Introduction . . . . . . . . . . . . The condition AA− = BA− . . One-sided sharp order . . . . . . − Roles of A− c and Aa in one-sided One-sided star order . . . . . . . Exercises . . . . . . . . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
127 128 136 138 145 150 152 155
. . . . . . . . . . . . . . . . . . . . . sharp order . . . . . . . . . . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
Unified Theory of Matrix Partial Orders through Generalized Inverses 7.1 7.2 7.3 7.4 7.5
103 104 110 117 124 127
One-Sided Orders 6.1 6.2 6.3 6.4 6.5 6.6
84 93 95 98 103
The Star Order 5.1 5.2 5.3 5.4 5.5 5.6 5.7
6.
A− A = A− B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
The Sharp Order 4.1 4.2 4.3 4.4 4.5
5.
that . . . . . . . . . . . .
Introduction . . . . . . . . . . . . . . . . . . . . . . . . G-based order relations: Definitions and preliminaries O-based order relations and their properties . . . . . . One-sided G-based order relations . . . . . . . . . . . Properties of G-based order relations . . . . . . . . . .
155 156 160 167 171 180
183 . . . . .
. . . . .
183 184 195 200 203
Contents
xv
7.6 On G-based extensions . . . . . . . . . . . . . . . . . . . . 208 7.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . 212 8.
9.
The L¨ owner Order
215
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 8.2 Definition and basic properties . . . . . . . . . . . . . . . 8.3 L¨ owner order on powers and its relation with other partial orders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 L¨ owner order on generalized inverses . . . . . . . . . . . . 8.5 Generalizations of the L¨owner order . . . . . . . . . . . . 8.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . .
215 215
Parallel Sums
245
9.1 9.2 9.3 9.4 9.5 10.
Introduction . . . . . . . . . . . Definition and properties . . . Parallel sums and partial orders Continuity and index of parallel Exercises . . . . . . . . . . . .
. . . . . . . . . . . . sums . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
Introduction . . . . . . . . . . . . . . . . . . . . . . . Shorted operator - A motivation . . . . . . . . . . . Generalized Schur complement and shorted operator Shorted operator via parallel sums . . . . . . . . . . Generalized Schur complement and shorted operator matrix over general field . . . . . . . . . . . . . . . . 10.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . .
12.
. . . . .
. . . . .
Schur Complements and Shorted Operators 10.1 10.2 10.3 10.4 10.5
11.
. . . . .
226 230 238 243
245 246 259 264 270 273
. . . . . . . . . . . . of a . . . . . .
273 274 276 283 285 293
Shorted Operators - Other Approaches
295
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Shorted operator as the limit of parallel sums - General matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.3 Rank minimization problem and shorted operator . . . . . 11.4 Computation of shorted operator . . . . . . . . . . . . . . 11.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . .
295 296 305 310 315
Lattice Properties of Partial Orders
317
12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 317
xvi
Matrix Partial Orders, Shorted Operators and Applications
12.2 12.3 12.4 12.5 13.
Partial Orders of Modified Matrices 13.1 13.2 13.3 13.4 13.5 13.6
14.
14.6
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . Equivalence relation on g-inverses of a matrix . . . . . . . Equivalence relations on subclasses of g-inverses . . . . . . Equivalence relation on the outer inverses of a matrix . . Diagrammatic representation of the g-inverses and outer inverses . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Ladder . . . . . . . . . . . . . . . . . . . . . . . . . .
Applications 15.1 15.2 15.3 15.4 15.5 15.6
16.
Introduction . . Space pre-order Minus order . . Sharp order . . Star order . . . L¨ owner order .
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . Point estimation in a general linear model . . . . . . . . . Comparison of models when model matrices are related under matrix partial orders . . . . . . . . . . . . . . . . . Shorted operators - Applications . . . . . . . . . . . . . . Application of parallel sum and shorted operator to testing in linear models . . . . . . . . . . . . . . . . . . . . . . . . Shorted operator adjustment for modification of network or mechanism . . . . . . . . . . . . . . . . . . . . . . . . .
Some Open Problems 16.1 16.2 16.3
318 330 338 342 343
Equivalence Relations on Generalized and Outer Inverses 14.1 14.2 14.3 14.4 14.5
15.
Supremum and infimum of a pair of matrices under the minus order . . . . . . . . . . . . . . . . . . . . . . . . . . Supremum and infimum under the star order . . . . . . . Infimum under the sharp order . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . .
343 344 352 357 364 367 371 371 372 380 384 390 401 407 407 407 411 415 418 418 423
Simultaneous diagonalization . . . . . . . . . . . . . . . . 423 Matrices below a given matrix under sharp order . . . . . 424 Partial order combining the minus and sharp orders . . . 424
Contents
16.4 16.5 16.6 16.7
When is a G-based order relation a partial order? Parallel sum and g-inverses . . . . . . . . . . . . Shorted operator and a maximization problem . The ladder problem . . . . . . . . . . . . . . . .
Appendix A A.1 A.2 A.3 A.4 A.5 A.6 A.7 A.8
xvii
. . . .
. . . .
. . . .
. . . .
. . . .
Relations and Partial Orders
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . Relations . . . . . . . . . . . . . . . . . . . . . . . . . . Semi-groups and groups . . . . . . . . . . . . . . . . . . Semi-groups and partial orders . . . . . . . . . . . . . . Involution . . . . . . . . . . . . . . . . . . . . . . . . . . Compatibility of partial orders with algebraic operations Partial orders induced by convex cones . . . . . . . . . . Creating new partial orders from old partial orders . . .
425 425 426 427 429
. . . . . . . .
429 429 432 433 435 435 436 436
Bibliography
439
Index
445
Chapter 1
Introduction
1.1
Matrix orders
Last few decades have witnessed a steady growth in the area of matrix partial orders, a central theme of this monograph. These matrix orders are developed in detail in Chapters 3-8. They play an important role in the study of shorted operators, which we treat subsequently. In this section we give a simple and intuitive interpretation of some of the matrix partial orders. Let us begin by defining a pre-order and a partial order. A binary relation on a non-empty set is said to be a pre-order if it is reflexive and transitive. If it is also anti-symmetric, then it is called a partial order (see Appendix A). Let f be a linear transformation from F n → F m , F being an arbitrary field. Then there exist bases B and C of F n and F m respectively such that f is represented by the matrix diag(I, 0) with respect to these bases (normal form). Also, if F = C, the field of complex numbers, then there exist ortho-normal bases B and C of Cn and Cm respectively such that f is represented by the matrix diag(D, 0) with respect to these bases, where D is a positive definite diagonal matrix (singular value decomposition). Thus, every linear transformation can be represented by a diagonal matrix by choosing the bases appropriately. A matrix G is a generalized inverse (g-inverse) of a matrix A if AGA = A. Let A and B be matrices of the same order. Then A is said to be below B under the minus order if there exists a g-inverse G of A such that AG = BG and GA = GB. Again, let A and B be matrices of the same order. Let G be the MoorePenrose inverse of A, a matrix that satisfies the conditions AGA = A,
1
2
Matrix Partial Orders, Shorted Operators and Applications
GAG = G, AG and GA are hermitian. Then A is said to be below B under the star order if the Moore-Penrose inverse G of A satisfies the conditions AG = BG and GA = GB. If A and B are square matrices of the same order such that ρ(A) = ρ(A2 ) and ρ(B) = ρ(B2 ), then A is below B under the sharp order if AG = BG and GA = GB, the matrix G being the group inverse of A. Note that the group inverse of A is a matrix G such that AGA = A, GAG = G and AG = GA. The minus, the star and the sharp orders are partial orders. For a pair of matrices A and B, each of these orders is defined through the same type of conditions, namely: AG = BG and GA = GB, where G is required to belong to a suitable subclass of g-inverses of A. All the above partial orders can be nicely interpreted through matrix decompositions. Let us consider two diagonal matrices L = diag(D, 0, 0) and M = diag(D, E, 0), where D and E are diagonal matrices. (We say a matrix - square or rectangular - is diagonal if all the elements outside the principal diagonal are zero.) It is intuitive to say that L is below M or L is a section of M. By an extension of this notion, for suitable choices of nonsingular/unitary matrices P and Q, the matrix PLQ is also below PMQ in some sense. Several matrix partial orders can be expressed in this manner as we shall see in the later chapters. For instance, the matrix A is below the matrix B under the minus order if and only if A and B have simultaneous normal form such that A = Pdiag(I, 0, 0)Q and B = Pdiag(I, I, 0)Q, where P and Q are non-singular matrices. Let A and B be matrices representing the linear transformations f and g with respect to standard bases F n and F m of respectively. Then A is below B under minus if and only if there exist bases B and C of F n and F m respectively such that f and g are represented by diag(I, 0, 0) and diag(I, I, 0) respectively with respect to these bases. Further, A is below B under the star order if and only if A and B have simultaneous singular value decomposition such that A = Pdiag(D, 0, 0)Q and B = Pdiag(D, E, 0)Q, where D and E are positive definite diagonal matrices and P and Q are unitary matrices. Let A and B be square matrices. Then we say A is below B under the sharp order if and only if there exists a non-singular matrix P such that A = Pdiag(D, 0, 0)P−1 and B = Pdiag(D, E, 0)P−1 , where D and E are non-singular matrices. Yet another partial order is the well-known L¨owner order on non-
Introduction
3
negative definite matrices defined as follows. Let A and B be non-negative definite (nnd) matrices of the same order. Then A is below B under the L¨ owner order if B − A is nnd. Furthermore, this happens if and only if there exists a non-singular matrix P such that A = Pdiag(D, 0, 0)P? and B = Pdiag(H, E, 0)P? , where D, H and E are positive definite diagonal matrices and each diagonal element of H is at least as big as the corresponding diagonal element of D. Thus, we see that corresponding to each of the matrix orders mentioned above A and B, the matrices representing some linear transformations f and g under suitable choice of bases, are simultaneously represented by matrices L and M that have a relatively simple structure, where it is intuitively clear that L is below M. In later chapters of this book, we explore all these and other matrix orders and study their inter-relationships.
1.2
Parallel sum and shorted operator
Parallel sum and shorted operator as studied in this monograph have their origin in the study of impedance matrices of n-port electrical networks. There have been interesting extensions of these concepts to wider classes of matrices. Two matrices A and B of the same order are said to be parallel summable if the row and column spaces of A are contained respectively in the row and column spaces of A+B. When A and B are parallel summable, their parallel sum is defined as A(A + B)− B, where (A + B)− is a g-inverse of A + B. Parallel sum has some very interesting properties. For example, its column (row) space is the intersection of the column (row) spaces of A and B. In this monograph, we study the properties of the parallel sum in detail. We also study the connection between the parallel sum and the matrix orders. The shorted operator has been defined in more than one way. For example, the shorted operator of a matrix with respect to another matrix of the same order can be defined as the limit of a certain sequence of parallel sums. Yet another way: Let A be an m × n matrix, Vm and Vn be subspaces of vector spaces of m-tuples and n-tuples. The shorted operator of A indexed by subspaces Vm and Vn is a matrix that is the closest to A in the class of matrices having their column and row spaces contained in Vm and Vn respectively. These various definitions of shorted operator are based on different objectives. In this monograph, we make a comprehensive study
4
Matrix Partial Orders, Shorted Operators and Applications
of the shorted operators and show the equivalence of various definitions of the shorted operator.
1.3
A tour through the rest of the monograph
As indicated in Section 1.1, generalized inverses, matrix decompositions and simultaneous decompositions in particular are going to play an important role in the study of matrix partial orders, parallel sums and shorted operators. In Chapter 2, we develop the necessary background on matrix decompositions and generalized inverses. Chapter 3 starts with the space pre-order. We study this basic matrix order in detail as we feel that this is the stepping stone to the other matrix orders that follow. We then introduce the minus order using a generalized inverse. This is the first partial order we study in this monograph. We obtain several characterizing properties of the same. We obtain the classes of matrices that lie above and below a given matrix under the minus order. We also study the minus order for certain special classes of matrices like the projectors. In Chapter 4, we define the sharp order on the square matrices of index not exceeding 1. The sharp order is an order finer than the minus order that involves the group inverse. We make a detailed study of its characteristic and other properties. We then consider matrices which have index greater than 1. We define an order called Drazin order using the Drazin inverse (which is not a generalized inverse in the usual sense). This order turns out to be a pre-order different from the space pre-order. It does have some interesting properties. We finally extend the Drazin order to a partial order and study its properties. The star order is perhaps more extensively studied than the minus and the sharp orders. In fact, it appeared in the literature before both the other two orders. Chapter 5 is devoted to the study of this order. We study the characterizing properties of the star order and obtain the classes of matrices above and below a given matrix under the star order. We then specialize to some interesting subclasses of matrices such as range hermitian, normal, hermitian and idempotent matrices and study the star order for such matrices. We finally consider several matrices and study when each of them is below their sum under the star order. The results are very similar to the celebrated Fisher-Cochran Theorem on distribution of quadratic forms in normal variables.
Introduction
5
Let A and B be matrices of the same order. Then A is below B under each of the minus, the sharp and the star orders if there is a suitable g-inverse G of A such that AG = BG and GA = GB. In Chapter 6, we consider only one of the above two conditions and show that with an additional milder condition we can get a partial order. Such an order is called a one-sided order. A one-sided order corresponding to minus order simply coincides with the minus order. However, one-sided sharp and star orders do not coincide with the sharp and the star orders respectively and lead to an interesting study. In the process of obtaining one-sided sharp order, we develop two special classes of g-inverses, which are interesting in their own way. After a careful study of the results of Chapters 3-5, one finds several similar looking properties shared by these order relations. Is there a common thread? Is there a master characterizing property using which we can derive a number of common looking properties of these partial orders? In Chapter 7, we find an answer to these questions and give a unified theory of matrix partial orders developed via generalized inverses. Characterizations of common properties/results of all the partial orders are put under one umbrella of the unified theory. We conclude the chapter with some extensions of this unified theory. L¨ owner order, usually studied for nnd matrices, is one of the oldest known partial orders. Some material dealing with L¨owner order for hermitian matrices is also available in the literature. In Chapter 8, we bring together such results and make a comprehensive study of this order. We study the relationship of L¨ owner order with other partial orders studied earlier. We also study the ordering properties of generalized inverses and outer inverses under L¨ owner order. Finally, we consider a couple of extensions of L¨ owner order - one of them for rectangular matrices. Chapter 9 is devoted to the study of the parallel sum of matrices. Besides obtaining several interesting properties of parallel sums, we also explore the relationship of parallel sum with matrix orders, particularly, the space pre-order and the L¨ owner order. It turns out that parallel sum of two matrices A and B is the matrix closest to A and B in the sense that any matrix which is below A and B under the space pre-order is also below their parallel sum under the space pre-order. We study the shorted operators in Chapters 10 and 11. In Chapter 10, we provide motivation for shorted operator through Electrical Networks and Statistics. We first consider nnd matrices and study the shorted operator of an nnd matrix indexed by a subspace and develop several interesting
6
Matrix Partial Orders, Shorted Operators and Applications
properties. We note here that the shorted operator is a certain Schur complement. Let A be an nnd matrix of order n × n and S be a subspace of Cn . Then the shorted operator of A with respect to S turns out to be the nnd matrix closest to A, both under the L¨owner order and the minus order, in the class of all nnd matrices with column space contained in S. We also show that the shorted operator is the limit of a sequence of certain parallel sums. We then extend the concept of the shorted operator to possibly rectangular matrices over a general field. This leads us to a concept called complementability. We examine when the shorted operator indexed by two subspaces exists and obtain an explicit expression for the same when it exists. Again, it turns out to be a Schur complement. Let A be an m × n matrix over a general field and let Vm and Vn be subspaces of vector spaces of m-tuples and n-tuples respectively over this field. Let the shorted operator S(A|Vm , Vn ) of A indexed by Vm and Vn exist. We find that the shorted operator, S(A|Vm , Vn ), whenever it exists, is the closest to A under the minus order in the class of all matrices the column space and row space of which are contained in Vm and Vn respectively. Further, S(A|Vm , Vn ) turns out to be a Schur complement. In Chapter 11, we first extend the concept of shorted operator to (possibly) rectangular matrices over C, the field of complex numbers using the approach of the limit of a sequence of parallel sums. We examine when it exists and study its properties. We then give another definition of shorted operator using the approach of closeness in the sense of rank. More precisely, for a matrix A, we examine when a unique matrix B exists such that the rank of A − B is the least in the class of all matrices the column space and row space of which are contained in Vm and Vn respectively. When such a matrix B exists, we call it as the shorted operator S(A|Vm , Vn ) of A indexed by Vm and Vn . We show that all these approaches of defining the shorted operator lead to the same matrix. We finally make some remarks on the computational aspects of the shorted operator. One characterizing property of the shorted operator studied in Chapters 10 and 11 is that it is a maximal element of a certain collection of matrices under a suitable partial order. This raises the following natural question. Does a set of matrices equipped with a partial order become a lattice or at least a semi-lattice under that partial order? In Chapter 12, we study this problem in some detail for three of the major partial orders studied earlier, namely, the minus, the star and the sharp orders. Chapter 13 contain entirely new material. We make an extensive study of the matrix order relations for modified matrices. We consider here two
Introduction
7
types of modifications of matrices one: appending or deleting a row/column and the other: adding a rank 1 matrix. Let A be below B under a particular matrix order. Let A be modified as per one of the above modifications. We obtain the class of all modifications of B such that the modified matrix of A is below the modified matrix of B under the same matrix order. The matrix orders considered for this purpose are the space pre-order, the minus order, the sharp order, the star order and the L¨owner order. The proofs, in general, are highly computational and lengthy. Due to space considerations, we have omitted most of the proofs. However a reader interested in the proofs of these results may write to the authors. In Chapter 14, we give an application of the matrix partial orders in developing equivalence relations on the classes of generalized inverses and outer inverses of a matrix. This leads to the development of nice hierarchies among the various classes of inverses (both inner and outer) of a matrix. It also leads to a neat diagrammatic representation of various inverses of a matrix, which according the first author resembles a strawberry plantation. (It is often said, a picture is worth a thousand words!) In Chapter 15, we give applications of matrix orders, parallel sum and shorted operator to Statistics and Electrical Networks. We first collect some results related to inference in linear models for the benefit of a general reader. Then we give interpretation and application of matrix orders, parallel sum and shorted operators to comparison of models and inference in linear models. We also give an application of shorted operators to the recovery of inter-block information in incomplete block designs. We give an application of shorted operators of modified matrices to obtain the modified shorted operator when a new port is included in the network. We enlist a few open problems related to the material covered in this monograph which should be of interest to the researchers. Finally, Appendix A contains the basic material on the algebra of relations, semi-groups, groups, partial orders and related issues for those readers who may need a little brushing up of some of these basic concepts.
Chapter 2
Matrix Decompositions and Generalized Inverses
2.1
Introduction
This chapter aims at providing some prerequisites in Linear Algebra for smooth reading of the later chapters of the monograph. We assume that the reader has the knowledge at the level of a first course in Linear Algebra. In Section 2.2, we gather several known results on matrix decompositions. Rank factorizations of matrices have been in use for a long time. However their properties are not commonly available in elementary textbooks on matrices. Therefore, we treat them here in a fairly elaborate fashion. Sections 2.3-2.6 are devoted to an exposition of generalized inverses (g-inverses), where we try to provide motivation for a gradual development of different types of g-inverses. It is perhaps a little lengthy exposition for a monograph on matrix partial orders. The justification for this is two-fold: (i) in general, a first course in Linear Algebra does not provide a proper motivation and development of g-inverse and (ii) the later chapters in the monograph require a mature understanding of g-inverses. However, in this chapter, no attempt is made to make the exposition on g-inverses exhaustive or up to date. For a detailed study of generalized inverses the reader is referred to [Rao and Mitra (1971)], [Ben-Israel and Greville (2001)] and [Campbell and Meyer (1991)]. The main purpose of this chapter is to facilitate the reader in studying the later chapters of the book comfortably. In Section 2.7, we study simultaneous diagonalization, a concept which is very important when we deal with two or more matrices together. Apart from the standard results, we include simultaneous singular value decomposition for several matrices and generalized singular value decomposition, which will prove useful in understanding some partial orders like a one-sided star order. This chapter contains a few new results. We also give new and simpler
9
10
Matrix Partial Orders, Shorted Operators and Applications
proofs to some known results. In this chapter, by and large, we work with vectors and matrices over a general field. However, if we use vectors and matrices over the complex field, we clearly mention the same before such a use.
2.2
Matrix decompositions
In this section, we enumerate several matrix decompositions (often without proof) which will be useful later in understanding the structures of generalized inverses and matrix partial orders. Proofs of most of the results mentioned here can be found in standard text books on Linear Algebra, for example [Rao and Bhimasankaram (2000)]. Theorem 2.2.1. (Normal Form) Let A be an m × n matrix and let ρ(A) = r > 0. Then there exist non-singular matrices P and Q of order m × m and n × n respectively such that A = Pdiag(Ir , 0)Q. Theorem 2.2.2. (Rank Factorization) Let A be as in Theorem 2.2.1. Then there exist matrices R and S of orders m × r and r × n respectively such that ρ(R) = r = ρ(S) and A = RS. Conversely, if A = RS, where R and S are full rank matrices of order m × r and r × n respectively such that r ≤ min{m, n}, then ρ(A) = r. The first part of Theorem 2.2.2 follows easily from Theorem 2.2.1 by taking R as the matrix formed by taking first r columns of P and S as the matrix formed by the first r rows of Q. The second part is easy. Definition 2.2.3. If R and S are matrices as specified in Theorem 2.2.2, we say (R, S) is a rank factorization of A. Remark 2.2.4. If (R, S) is a rank factorization of a matrix A, then (i) the columns of R form a basis of C(A), the column space of A and (ii) the rows of S form a basis of C(At ), the column space of At . Remark 2.2.5. A matrix can have several rank factorizations. Let (R, S) be a rank factorization of A. Then the class of matrices −1 {(RT, T S), T non-singular} is the class of all rank factorizations of A. Remark 2.2.6. Let A be an m×n matrix of rank r over the field of complex numbers C. Then there exists a semi-unitary matrix U of order m × r (a
Matrix Decompositions and Generalized Inverses
11
matrix U such that U? U = Ir ) such that (U, V) is a rank factorization of A. Let A1 and A2 be non-null matrices of the same order. Let (Pi , Qi ) be a rank factorization of Ai , i = 1, 2. Clearly, Q1 A1 + A2 = (P1 : P2 ) ·· . Q2 Q1 When is (P1 : P2 ), ·· a rank factorization of A1 + A2 ? Before we Q2 can answer this question we give a couple of definitions. Definition 2.2.7. Two subspaces S and T of a vector space are said to be virtually disjoint if S ∩ T = {0}. Definition 2.2.8. Let A1 and A2 be matrices of the same order. Then A1 and A2 are said to be disjoint if (i) C(A1 ) ∩ C(A2 ) = {0} and (ii) C(At1 ) ∩ C(At2 ) = {0} or equivalently ρ(A1 + A2 ) = ρ(A1 ) + ρ(A2 ). In such a case the sum of A1 and A2 is denoted by A1 ⊕ A2 . We now prove the following: Theorem 2.2.9. Let A1 and A2 be non-null matrices of the same order. Let (Pi , Qi ) be a rank factorization of Ai , i = 1, 2. Then Q1 (P1 : P2 ), ·· Q2 is a rank factorization of A1 + A2 if and only if A1 and A2 are disjoint. Proof. ‘If’ part Notice that Q1 A1 + A2 = P1 Q1 + P2 Q2 = (P1 : P2 ) ·· . Q2 Let A1 and A2 be disjoint. Then ρ(A1 + A2 ) = ρ(A1 ) + ρ(A2 ).
12
Matrix Partial Orders, Shorted Operators and Applications
Hence the number of columns in (P1 : P2 ) = the number of rows in Q1 Q1 ·· = ρ(A1 + A2 ). So, (P1 : P2 ), ·· is a rank factorization Q2 Q2 of A1 + A2 . ‘Only if’ part Since (Pi , Qi ) is a rank factorization of Ai , i = 1, 2 and Q1 (P1 : P2 ), ·· is a rank factorization of A1 + A2 , therefore, Q2
ρ(A1 + A2 ) = ρ(P1 : P2 ) = number of columns in (P1 : P2 ) = ρ(A1 ) + ρ(A2 ).
Definition 2.2.10. A matrix H = [hij ] of order n × n is said to be in Hermite Canonical Form (HCF) if (i) (ii) (iii) (iv)
hij = 0 whenever i > j, i = 1, . . . , n and j = 1, . . . , n, hii = 1 or 0 for i = 1, . . . , n, if hii = 0 for some i, then hij = 0 for j = 1, . . . , n and if hii = 1 for some i, then hji = 0 for all j 6= i.
Thus, if H is in HCF, then H is an upper triangular matrix with each diagonal element either 0 or 1; if a diagonal element is 0, then the entire row containing this particular diagonal element is null and if a diagonal element is 1, then the remaining elements of the column containing the diagonal element are 0. For example 1 2 0 4 0 0 0 0 H= 0 0 1 −8 is a matrix in HCF. 0 0 0 0 Theorem 2.2.11. Let A be an n × n matrix. Then there exists a nonsingular matrix B such that H = BA is in HCF. Remark 2.2.12. A matrix in HCF is idempotent. Remark 2.2.13. Let A be an n × n matrix. Then by elementary row operations one can reduce A to a matrix in HCF.
Matrix Decompositions and Generalized Inverses
13
Remark 2.2.14. Let A, B and H be as in Theorem 2.2.11 with ρ(A) = r th and let ith 1 , . . . , ir diagonal elements of H be each equal to 1. Write R = (A?i1 : : A?ir ) and S = (Hti1 ? : : Htir ? ); th where A?ij is the ith j column of A and Hik ? is the ik row of H. Then (R, S) is a rank factorization of A.
Theorem 2.2.15. (Schur Decomposition) Let A be an n × n matrix. then there exists a non-singular matrix P such that A = PTP−1 , where T is an upper triangular matrix. Note that in Theorem 2.2.15, the eigen-values of the matrix A are precisely the diagonal elements of T. If the matrix A is over the field of complex numbers C, there exists a unitary matrix P such that A = PTP? , where T is an upper triangular matrix. Before we state our next theorem on the Jordan decomposition of a matrix, we define a Jordan block. A Jordan block of order 1 is a 1×1 matrix of the form [a]. A Jordan block J = [aij ] of order r > 1 is a matrix satisfying (i) (ii) (iii) (iv)
aik = 0, whenever i > k (upper triangular) a11 = . . . = arr (identical diagonal elements) aii+1 = 1 for i = 1, . . . , r − 1 and aik = 0 whenever k > i + 1.
For example 3 0 J= 0 0
1 3 0 0
0 1 3 0
0 0 is a Jordan block. 1 3
Theorem 2.2.16. (Jordan Decomposition) Let A be an n × n matrix over C. Then there exists a non-singular matrix P such that A = P diag (J1 , . . . , Jr ) P−1 , where each Ji i = 1, . . . , r is a Jordan block. Remark 2.2.17. Jordan decomposition is a more specialized form of Schur decomposition. Definition 2.2.18. Let A be an n × n matrix. The smallest positive integer k for which ρ(Ak ) = ρ(Ak+1 ) is called the index of A.
14
Matrix Partial Orders, Shorted Operators and Applications
Remark 2.2.19. The index of a non-singular matrix A is 0 and the index of a null matrix is 1. Remark 2.2.20. The matrices A and PAP−1 have the same index for each non-singular matrix P. Theorem 2.2.21. (Core-Nilpotent Decomposition) Let A be an n × n matrix. Then A can be written as the sum of matrices A1 and A2 i.e. A = A1 + A2 where (i) ρ(A1 ) = ρ(A21 ) i.e. A1 is of index ≤ 1 (ii) A2 is nilpotent i.e. there is a positive integer k such that Ak2 = 0 and (iii) A1 A2 = A2 A1 = 0. Here one or both of A1 and A2 can be null. A proof of Theorem 2.2.21 will be given in Section 2.4. Remark 2.2.22. Theorem 2.2.21 can be deduced from Theorem 2.2.16 for the matrices over complex field. Theorem 2.2.23. Let A be an n × n matrix. The following are equivalent: (i) (ii) (iii) (iv)
ρ(A) = ρ(A2 ) C(A) ∩ N (A) = {0} F n = C(A) ⊕ N (A) and There exists a non-singular matrix P such that A = Pdiag(T, 0)P−1 , where T is non-singular.
Remark 2.2.24. Theorem 2.2.21 can be restated as follows: Let A be an n × n matrix. Then there exists a non-singular matrix P such that A = Pdiag(T, N)P−1 , where T is non-singular and N is nilpotent. Remark 2.2.25. Let A be an n × n matrix over C. Then the algebraic multiplicity of the zero eigen-value of A is equal to its geometric multiplicity if and only if ρ(A) = ρ(A2 ). Definition 2.2.26. An n×n matrix A over C is said to be range-hermitian if C(A) = C(A? ) or equivalently if N (A) = N (A? ). An n × n matrix A over C is said to be hermitian if A = A? . Remark 2.2.27. Every hermitian matrix is range-hermitian. Every nonsingular matrix is also range-hermitian.
Matrix Decompositions and Generalized Inverses
15
Remark 2.2.28. If A is range-hermitian, then ρ(A) = ρ(A2 ). Theorem 2.2.29. Let A be an n × n matrix over C. Then A is rangehermitian if and only if there exists a unitary matrix U such that A = Udiag(T, 0)U? , where T is non-singular. Definition 2.2.30. A matrix A over C is said to be simple if all its eigenvalues are distinct. A is said to be semi-simple, if the algebraic multiplicity for each of its distinct eigen-values equals its geometric multiplicity. Theorem 2.2.31. (Spectral Decomposition of a Semi-Simple Matrix) Let A be an n×n matrix over C. Then A is semi-simple if and only if there exist matrices E1 , . . . , Es of order n × n and distinct complex numbers λ1 , . . . , λs such that (i) E2i = Ei , i = 1, . . . s (ii) Ei Ej = 0 whenever i 6= j (iii) I = E1 + . . . + Es and (iv) A = λ1 E1 + . . . + λs Es . Further, the matrices E1 , . . . , Es with these properties are unique. Remark 2.2.32. Every simple matrix is semi-simple. Remark 2.2.33. Let Ei , i = 1, . . . , s as in Theorem 2.2.31. Then n = ρ(I) = tr(I) = tr(E1 ) + . . . + tr(Es ) = ρ(E1 ) + . . . + ρ(Es ). Remark 2.2.34. Let Ei , i = 1, . . . , s be as in Theorem 2.2.31. Let (Pi , Qi ) be a rank factorization of Ei . Since E2i = Ei for all i and Ei Ej = 0 whenever i 6= j, Qi Pi = I for each i and Qi Pj = 0 whenever i 6= j. Let P = (P1 , . . . , Ps ) and Q = (Qt1 , . . . , Qts )t . Then it follows that P and Q are square matrices and QP = I, so, Q = P−1 . Thus, A = P diag (λ1 , . . . , λ1 , λ2 , . . . , λ2 , . . . , λs , . . . , λs ) P−1 where λi appear ρ(Ei ) times in the above diagonal matrix. Further, each column of P is a right eigen-vector of A. Remark 2.2.35. A is semi-simple if and only if A is similar to a diagonal matrix. Remark 2.2.36. Let A be a semi-simple matrix and let A = λ1 E1 + . . . + λs Es be the spectral decomposition of A. Then for each positive integer k, Ak = λk1 E1 + . . . + λks Es is the spectral decomposition of Ak .
16
Matrix Partial Orders, Shorted Operators and Applications
Definition 2.2.37. Let A be an n × n matrix over C. Then A is called a normal matrix if A? A = AA? . Theorem 2.2.38. (Spectral Decomposition of a Normal Matrix) Let A be an n × n matrix over C. Then A is normal if and only if there exists a unitary matrix U such that A = UΛU? , where Λ is a diagonal matrix (possibly complex). Remark 2.2.39. Every normal matrix is range-hermitian. Remark 2.2.40. The diagonal elements of Λ in Theorem 2.2.38 are eigenvalues of A and columns of U are the ortho-normal eigen-vectors of A. Remark 2.2.41. Let U = (U1 , . . . , Un ) be a unitary matrix and let Λ = diag(λ1 , . . . , λn ). Then the spectral decomposition A = UΛU? of A can also be written as n X λi Ui U?i . A= i=1
Remark 2.2.42. The above decomposition is in general not unique (unless all eigen-values are distinct). However, if µ1 , . . . , µs (s ≤ n) are distinct eigen-vectors of A, then by appropriate pooling of the eigen-values we can write s X X µi Vi Vi? , where Vi? Vi = I , Vi? Vj = 0 and I = Vi Vi? . U= i=1
This decomposition is unique in the sense that the orthogonal projectors Vi Vi? are unique. Theorem 2.2.43. (Spectral Decomposition of a Hermitian Matrix) Let A be an n×n matrix over C. Then A is hermitian if and only if there exists a unitary matrix U such that A = UΛU? , where Λ is a real diagonal matrix. Remark 2.2.44. Every hermitian matrix is normal with real eigen-values. Hence Remarks 2.2.39-2.2.42 are valid for hermitian matrices with obvious modifications. Theorem 2.2.45. (Singular Value Decomposition) Let A be an n × n matrix over C with rank r > 0. Then there exist unitary matrices U and V and a positive definite diagonal matrix ∆ of order r × r such that A = Udiag(∆, 0)V? = δ1 U1 V1? + . . . + δr Ur Vr? , where Ui and Vi are the ith columns of U and V respectively.
Matrix Decompositions and Generalized Inverses
17
Remark 2.2.46. Let A, U and V be as in Theorem 2.2.45. Then δ12 , . . . , δr2 are the non-zero eigen-values of the matrix A? A (or equivalently of the matrix AA? ) and Ui and Vi are eigen-vectors of the matrices AA? and A? A respectively corresponding to the eigen-value δi2 . Remark 2.2.47. δ1 , . . . , δr in Remark 2.2.46 are called the singular values of A and Ui and Vi , i = 1, . . . , r, are called singular vectors of A. Remark 2.2.48. Compare Theorems 2.2.1 and 2.2.45. For matrices over complex field, we can use the unitary matrices to reduce A to a diagonal form. However, we cannot, in general reduce A to diag(Ir , 0) by using unitary matrices unless each singular value of A is 1 in which case A? A and AA? are orthogonal projectors. Remark 2.2.49. In general a singular value decomposition as given in Theorem 2.2.45 is not unique. For a unique singular value decomposition, one has to pool the singular vectors corresponding to each distinct singular value. This is called the Penrose decomposition. For details see Chapter 12. See also [Rao and Bhimasankaram (2000)].
2.3
Generalized inverse of a matrix
Let A be a non-singular matrix. Then there exists a matrix G such that AG = GA = I. Such a matrix G is unique. The matrix G is called the inverse of A and is denoted by A−1 . If A is an m × n matrix (m 6= n) of rank m, there exists a matrix G such that AG = I. (However there is no matrix H such that HA = I.) Such a matrix G, denoted by A−1 R , is called a right inverse of A. Notice that A−1 is not unique unless m = n. Again, R if A is an m × n matrix of rank n (m 6= n), there exists a matrix G such that GA = I (note that there is no matrix T such that AT = I). Such a −1 matrix G is called a left inverse of A, denoted by A−1 L . Just as AR is not −1 unique, so also AL is not unique unless m = n. In all the above cases, there exists a matrix G such that AGA = A. However, if A is an m × n matrix of rank r < min{m, n}, then there is no matrix R such that AR = I or RA = I. One can ask: does a matrix G satisfying AGA = A exist? If this happens, G can be thought of as a generalization of the inverse/a right inverse/a left inverse. From the utility point of view, if A is non-singular, then Ax = b is consistent for all b and possesses unique solution given by x = Gb, where
18
Matrix Partial Orders, Shorted Operators and Applications
G is the inverse of A. Let A be an m × n matrix of rank m, m 6= n and G be a right inverse of A. Then Ax = b is consistent for all b and x = Gb is a solution. It should be noted that Ax = b does not have a unique solution in this case. On the other hand if A is an m×n matrix of rank n and G is a left inverse of A, then Ax = b is not necessarily consistent for all b; but if it is consistent then it has a unique solution x = Gb (interested readers may find examples). Now, let A be an m × n matrix of rank r < min{m, n}. Does there exist a matrix G such that, whenever Ax = b is consistent, x = Gb is a solution to Ax = b? If so, such a matrix G can be thought of as a generalization of inverse/ right inverse/ left inverse. We shall now establish the existence of a matrix G that will have most of these properties, there by answering both the questions posed in the preceding paragraphs. Let A be an m × n matrix. If A is null, then AGA = A for all matrices G of order n × m. In this case, Ax = 0 is the only consistent system and x = G0 = 0 is a solution for each matrix G of order n × m. If A is nonnull, let (P, Q) be a rank factorization of A. Then P has a left inverse P−1 L −1 −1 and Q has a right inverse Q−1 R . Write G = QR PL . It is easy to check that AGA = A. Also, if Ax = b is consistent, then b = Ac for some vector c. Now, AGb = AGAc = Ac = b, showing that Gb is a solution to Ax = b. Thus, for every matrix A, there is a matrix G having both the properties. In fact a stronger result holds: Theorem 2.3.1. Let A is an m × n matrix and G be an n × m matrix. Then the following are equivalent: (i) (ii) (iii) (iv) (v) (vi)
Gb is a solution to Ax = b for all b ∈ C(A) AGA = A AG is idempotent and ρ(AG) = ρ(A) GA is idempotent and ρ(GA) = ρ(A) ρ(I − AG) = m − ρ(A) and ρ(I − GA) = n − ρ(A).
Proof. (i) ⇔ (ii) AGb = b for all b ∈ C(A) ⇔ AGAu = Au for all u ⇔ AGA = A. (ii) ⇒ (iii) is trivial. (iii) ⇒ (ii) Since C(AG) ⊆ C(A) and ρ(AG) = ρ(A), we have C(AG) = C(A). So, A = AGD for some matrix D. Further, since AG is idempotent, AGAG = AG. So, AGAGD = AGD or AGA = A.
Matrix Decompositions and Generalized Inverses
19
(ii) ⇔ (iv) is established similarly. (iv) ⇒ (vi) Since GA is idempotent, so is I − GA. Further, I = GA + (I − GA). Therefore, n = ρ(I) = tr(I) = tr(GA) + tr(I − GA) = ρ(GA) + ρ(I − GA). Since ρ(GA) = ρ(A), the result follows. (vi) ⇒ (iv) Now, I = GA + (I − GA). So, n ≤ ρ(GA) + ρ(I − GA) ≤ ρ(A) + ρ(I − GA) = n. It follows that ρ(GA) = ρ(A). Again, ρ(I) = ρ(GA + (I − GA)) = ρ(GA) + ρ(I − GA), so, the column spaces of GA and I − GA are virtually disjoint and so are the row spaces. Moreover, GA(I − GA) = (I − GA)GA. Hence GA(I − GA) = 0. So, GA is idempotent. The proof for (iii) ⇔ (v) can be completed along the lines of proof for (iv) ⇔ (vi). Thus, we see that both the questions raised in the beginning of this section lead to the same class of solutions. Definition 2.3.2. Let A be an m × n matrix. Then a matrix G is said to be a generalized inverse of A if it satisfies any one of the six equivalent conditions in Theorem 2.3.1. A generalized inverse (or in short a g-inverse) of A is denoted by A− and the class of all generalized inverses of A is denoted by {A− }. We shall now study the use of a g-inverse in solving linear equations. Theorem 2.3.3. Let A be an m × n matrix and let G be a g-inverse of A. Then (i) Ax = b is consistent if and only if AGb = b. (ii) N (A), the null space of A (or equivalently the class of all solutions to Ax = 0) is given by C(I − GA) (or equivalently, (I − GA)ξ, where ξ is arbitrary). (iii) Let Ax = b be consistent. Then the class of all solutions to Ax = b is given by Gb + (I − GA)ξ, where ξ is arbitrary.
20
Matrix Partial Orders, Shorted Operators and Applications
Proof. (i) ‘If’ part is obvious and the ‘Only if’ part follows from Theorem 2.3.1(i). (ii) Since A(I−GA) = 0, C(I−GA) ⊆ N (A). Now, by Theorem 2.3.1(vi), ρ(I − GA) = n − ρ(A) = d(N (A)), the dimension of the null space of A, it follows that C(I − GA) = N (A). (iii) follows from (ii) and the fact that Gb is a solution to Ax = b. Thus, a g-inverse can be used to check the consistency of a given system of linear equations and also to obtain an expression for all possible solutions to a consistent system of linear equations. Remark 2.3.4. Let A be an m × n matrix and G be a g-inverse of A. Then for a matrix B, C(B) ⊆ C(A) if and only if B = AGB. Theorem 2.3.5. Let Ax = b be consistent and b 6= 0. Then the class of all solutions of Ax = b is {Gb : G ∈ {A− }}. Proof. Let G be a g-inverse of A. Clearly, AGb = b, since Ax = b is consistent. Hence, {Gb : G ∈ {A− }} is a subset of the class of all solutions to Ax = b. Now, let u be a solution to Ax = b and G be a g-inverse of A. Then by Theorem 2.3.3(iii), u = Gb + (I − GA)ξ for some ξ. Since b 6= 0, there exists an integer j such that bj 6= 0. Let wij = ξi /bj , i = 1, . . . , n and wik = 0, k = 1, . . . , n whenever i 6= j. Write W = (wpq ). It is easy to see that Wb = ξ. So, u = Gb + (I − GA)ξ = Gb + (I − GA)Wb = (G + (I − GA)W)b. Now A(G+(I−GA)W)A = AGA = A. Hence, G+(I−GA)W ∈ {A− } and that u ∈ {Gb : G ∈ {A− }}. We shall now obtain several expressions for the class of all g-inverses of a matrix. Notice that every matrix of order n × m is a g-inverse of the null matrix of order m × n. Theorem 2.3.6. Let A be an m × n matrix of rank r(> 0) and let A = Pdiag(I, 0)Q be a normal form of A (see 2.2.1). Then the class Theorem −1 Lr L −1 of all g-inverses of A is given by Q P , where L, M and N M N are arbitrary. Proof.
Straightforward.
Matrix Decompositions and Generalized Inverses
21
Remark 2.3.7. Let A be an m × n matrix. Let P and Q be non-singular matrices of order m × m and n × n respectively. Then G is a g-inverse of A ⇔ Q−1 GP−1 is a g-inverse of PAQ. Theorem 2.3.8. Let A be an m×n matrix and let G be a specific g-inverse of A. Then the class of all g-inverses of A is given by {A− } = {G + U − GAUAG, U arbitrary} or equivalently {A− } = {G + (I − GA)V + W(I − AG), V, Warbitrary}. Proof.
First notice that for all U, A(G + U − GAUAG)A = AGA + AUA − AGAUAGA = AGA = A.
So, G + U − GAUAG is a g-inverse of A for all U. Let G1 be another g-inverse of A. Choose U = G1 − G and check that G1 = G + U − GAUAG. Thus, the class of all g-inverses of A is given by {A− } = {G + U − GAUAG, U arbitrary}. Consider now H = G + (I − GA)V + W(I − AG), where V and W are arbitrary matrices of appropriate order. It is easy to see that H is a ginverse of A for all V and W. Choose and fix V and W. Let U = (I − GA)V + W(I − AG). Then AUA = 0. Therefore, G + (I − GA)V + W(I − AG) = G + U − GAUAG, so, H ∈ {A− } for all V and W. Next, choose and fix U. Then G + U − GAUAG = G + (I − GA)UAG + U(I − AG). Let V = UAG and W = U. Hence, {A− } = {G + U − GAUAG, U arbitrary} = {G + (I − GA)V + W(I − AG), V, W arbitrary}.
We now give a basic lemma which will be needed in obtaining a useful result on the invariance of BA− C under the choice of g-inverses A− of A. Lemma 2.3.9. Let a and c be m-column and n-column vectors respectively such that c 6= 0 and at Zc = 0 for all m × n matrices Z. Then a = 0.
22
Matrix Partial Orders, Shorted Operators and Applications
Proof. If possible, let a 6= 0. Then there exists an element say ai of a, which is non-null. Since c 6= 0, there is an element cj of c which is non-null. Take a matrix Z having the (i, j)th element zij = 1 and all other elements null. Then at Zc = ai cj 6= 0, which is a contradiction. So, a = 0. Remark 2.3.10. Similar result holds when a and c are matrices of suitable orders. More precisely, if A is a m × n matrix and C (non-null) is a p × q matrix such that At ZC = 0 for all n × p matrices Z, then A = 0. Theorem 2.3.11. Let A, B and C be matrices of appropriate orders such that the product BA− C is defined. Then BA− C is invariant under choices of A− and is non-null if and only if B and C are non-null, C(C) ⊆ C(A) and C(Bt ) ⊆ C(At ). Proof. ‘If’ part is trivial. ‘Only if’ part Let G be a g-inverse of A. Then G+(I−GA)V+W(I−AG) is a g-inverse of A, for arbitrary matrices V and W of appropriate orders. Now, BGC is non-null ⇒ B and C are non-null and B(G + (I − GA)V + W(I − AG))C = BGC for all V and W. So, B((I − GA)V + W(I − AG))C = 0 for all V and W. Taking W = 0, we have B(I − GA)VC = 0 for all V. As C is non-null, by Lemma 2.3.10, B(I − GA) = 0. Thus, C(Bt ) ⊆ C(At ). Similarly by taking V = 0, we have C(C) ⊂ C(A). In the later sections we often need to get a g-inverse G of A such that column space or row space (of G) is contained in a specified subspace. We shall now investigate when it is possible to get such a matrix G. We shall also obtain an explicit expression for such a g-inverse. Theorem 2.3.12. Let A be an m × n matrix. Let r and s be positive integers and P and Q be matrices of orders n×s and m×r respectively.Then there exists a g-inverse G of A such that C(G) ⊆ C(P) and C(Gt ) ⊆ C(Qt ) if and only if ρ(QAP) = ρ(A). If ρ(QAP) = ρ(A), then P(QAP)− Q is a g-inverse of A. Further, if PCQ is a g-inverse of A, then C is a g-inverse of QAP. Proof. If G is a matrix such that C(G) ⊆ C(P) and C(Gt ) ⊆ C(Qt ), then G = PCQ for some matrix C. Now, AGA = A ⇒ APCQA = A ⇒ ρ(AP) = ρ(QA) = ρ(A). Further, QAPCQA = QA. Hence ρ(QAP) = ρ(QA) and therefore, ρ(QAP) = ρ(A).
Matrix Decompositions and Generalized Inverses
23
Conversely, let ρ(QAP) = ρ(A). Then, ρ(QAP) = ρ(QA) = ρ(AP) = ρ(A). Consider T = P(QAP)− . Now, QAT = QAP(QAP)− is idempotent. Also, ρ(QAT) = ρ(QAP(QAP)− ) = ρ(QAP) = ρ(QA). Therefore, T is a g-inverse of QA, by Theorem 2.3.1(iii). Similarly, TQ = (QA)− Q is a g-inverse of A. Thus, P(QAP)− Q is a g-inverse of A. Now, let PCQ be a g-inverse of A. So, APCQA = A. Pre- and post-multiplying by Q and P respectively, we have QAPCQAP = QAP, showing C is a g-inverse of QAP. We know that, if G is a g-inverse of A, then AGA = A and therefore, ρ(G) ≥ ρ(A). Let ρ(A) = r < min{m, n}. Then by using Theorem 2.3.6, it can be shown that for each s such that r ≤ s ≤ min{m, n}, there is a g-inverse G of A with rank s. Take for example G = Q−1 diag(Is , 0)P−1 . We now specialize to the class of g-inverses G of A such that ρ(G) = ρ(A). Theorem 2.3.13. Let A be an m × n matrix. Then G is a g-inverse of A such that ρ(G) = ρ(A) if only if AGA = A and GAG = G. Proof. ‘If’ part Since AGA = A, G is a g-inverse of A and ρ(G) ≥ ρ(A). Moreover, GAG = G, so, ρ(A) ≥ ρ(G). Hence ρ(G) = ρ(A). ‘Only if’ part Since G is a g-inverse of A and ρ(G) = ρ(A), AG is idempotent and ρ(AG) = ρ(A) = ρ(G). So, by (iv) of Theorem 2.3.1, A is a g-inverse of G. Thus, GAG = G and G being a g-inverse of A, AGA = A. Definition 2.3.14. Let A be an m × n matrix. An n × m matrix G satisfying AGA = A and GAG = G is called a reflexive generalized inverse of A, denoted by A− r . Remark 2.3.15. Notice the relationship AGA = A and GAG = G is one of symmetry between A and G. It is easy to see that {GAG : G ∈ {A− }} is the class of all reflexive g-inverses of A. Remark 2.3.16. Analogous to Theorem 2.3.6, it is easy to show that if A = Pdiag(Ir , 0)Q, then the class of all reflexive g-inverses of A is given
24
Matrix Partial Orders, Shorted Operators and Applications
by A = Q−1
Ir L P−1 , M ML
where L and M are arbitrary. For the null matrix of order m × n, while every n × m matrix is a generalized inverse, there is only one reflexive g-inverse namely the null matrix. We shall give below another characterization of reflexive g-inverses of a non-null matrix using rank factorization. Theorem 2.3.17. Let A be a non-null m × n matrix. Let (P, Q) be a rank factorization of A. Then the class of all reflexive g-inverses of A is given −1 −1 −1 by Q−1 R PL , where PL and QR are arbitrary left and right inverses of P and Q respectively. −1 Proof. Clearly, G = Q−1 R PL is a reflexive g-inverse of A. Conversely, let G be reflexive g-inverse of A. Then AGA = A ⇒ PQGPQ = PQ ⇒ QGP = I, since Q has a right inverse and P has a −1 left inverse. Clearly, a choice of P−1 L is QG and a choice QR is GP. −1 −1 Now, GPQG = GAG = G. So, G = QR PL .
Let A be an m × n matrix of rank r (< min{m, n}). Let s be an integer such that r < s ≤ min{m, n}. We now obtain a characterization of all g-inverses of A with rank s. Theorem 2.3.18. Let A be a m × n matrix of rank r where 0 < r < min{m, n}. Let s be a positive integer such that r < s ≤ min{m, n}. Then G is a g-inverse of A with ρ(G) = s if and only if then exist nonsingular matrices P and Q of order m × m and n × n respectively such that A = Pdiag(Ir , 0)Q and G = Q−1 diag(Is , 0)P−1 . Proof. ‘If’ part is trivial. ‘Only if’ part Let A = R1 diag(Ir , 0)R2 be a normal form of A where R1 and R2 are non-singular. Since Gis a g-inverse of A, Ir L −1 G = R2 R−1 1 , for some matrices L, M and N. M N Ir L Further, since G is of rank s, ρ = s. Now M N Ir L Ir 0 Ir 0 Ir L = . M N M I 0 N − ML 0 I
Matrix Decompositions and Generalized Inverses
25
Hence ρ(N − ML) = s − r. Let N − ML = R3 diag(Is−r , 0)R4 be a normal form of N − ML. Write P = R1
Ir − L Ir 0 Ir 0 Ir 0 and Q = R2 . 0 I 0 R−1 0 R−1 −M I 4 3
It is easily verified that A = Pdiag(Ir , 0) Q and G = Q−1 diag(Is , 0)P−1 . Corollary 2.3.19. Consider the same setup as in Theorem 2.3.18. Then G is a g-inverse of A with rank s if and only if there exists a matrix B such that (a) ρ(A + B) = ρ(A) + ρ(B) = s and (b) G = (A + B)− r . Proof. ‘If’ part Let (P1 , Q1 ) and (P2 , Q2 ) be rank factorizations of A and B respectively. Q1 Since ρ(A + B) = ρ(A) + ρ(B), (P1 : P2 ), ·· is a rank factorQ2 ization of A + B. Since G is a reflexive g-inverse of A + B, we have S1 Q1 G = (R1 : R2 ) ·· where (R1 : R2 ) is a right inverse of ·· and S2 Q2 S1 ·· is a left inverse of (P1 : P2 ). Thus, S2 S1 I Q1 (R1 : R2 ) = (I : 0) and ·· P1 = ·· . S2 0 Now, I S1 AGA = P1 Q1 (R1 : R2 ) ·· P1 Q1 = P1 (I : 0) ·· Q1 = P1 Q1 = A. 0 S2 Thus, G is a g-inverse of A. Further, ρ(G) = ρ(A + B) = s. ‘Only if’ Part By Theorem 2.3.18, A = Pdiag(Ir , 0)Q and G = Q−1 diag(Is , 0)P−1 for some non-singular matrices P and Q. Let B = Pdiag(0r , Is−r , 0)Q. Then A + B = Pdiag(Is , 0)Q and ρ(A + B) = s = ρ(A) + ρ(B). Finally, G = Q−1 diag(Is , 0)P−1 is (A + B)− r .
26
2.4
Matrix Partial Orders, Shorted Operators and Applications
The group inverse
In the previous section, we studied one important sub-class of g-inverses of a matrix, namely, the class of reflexive g-inverses. In this section, we seek a g-inverse G of a square matrix A satisfying C(G) ⊆ C(A) or C(Gt ) ⊆ C(At ) or both. We shall see that not all matrices enjoy this property. It turns out that the same subclass of square matrices enjoys each of the properties mentioned above. We seek to identify this subclass of square matrices and for each matrix in this class, we characterize the sub-classes of all ginverses with these properties. This leads us to the concept of the group inverse in a very natural manner. We study briefly the major properties of the group inverse. We also define the Drazin inverse for a general square matrix by extending the concept of the group inverse of a matrix of index 1. The group inverse is found useful in the analysis of Markov chains and the Drazin inverse is useful in solving differential equations and difference equations. Definition 2.4.1. Let A be an n × n matrix. Then a g-inverse G of A such that C(G) ⊆ C(A) is called a χ-inverse of A and is denoted by A− χ. A g-inverse such that C(Gt ) ⊆ C(At ) is called a ρ-inverse of A and is denoted by A− ρ. A g-inverse G of A such that C(G) ⊆ C(A) and C(Gt ) ⊆ C(At ) is called a ρχ-inverse of A and is denoted by A− ρχ . − Theorem 2.4.2. Let A be an n × n matrix. Then each of A− χ , Aρ and − 2 Aρχ exists if and only if ρ(A) = ρ(A ) i.e., A is of index not greater than 1. 2 − Proof. By Theorem 2.3.12, each of A− χ and Aρ exists ⇔ ρ(A) = ρ(A ). − 3 Again by the same theorem, Aρχ exists ⇔ ρ(A) = ρ(A ). Since ρ(A) ≥ 2 ρ(A2 ) ≥ ρ(A3 ), A− ρχ exists ⇔ ρ(A) = ρ(A ).
Let A be an n × n matrix such that ρ(A) = ρ(A2 ). Then by taking Q = I and P = A and applying Theorem 2.3.12, we see that a choice of 2 − A− χ is A(A ) . Similarly, by taking Q = A, P = I, we see that a choice of − 2 − Aρ is (A ) A, where (A2 )− is any g-inverse of A2 . Theorem 2.4.3. Let A be an n × n matrix such that ρ(A) = ρ(A2 ). Then 3 − 3 − A(A3 )− A is an A− ρχ where (A ) is any g-inverse of A . Further, Aρχ is unique.
Matrix Decompositions and Generalized Inverses
27
Proof. By Theorem 2.3.12, it follows that G = A(A3 )− A is a g-inverse of A such that C(G) ⊆ C(A) and C(Gt ) ⊆ C(At ). Further, if ACA is ginverse of A, then C must be of the form (A3 )− for some g- inverse (A3 )− of A3 , again by Theorem 2.3.12. Since ρ(A) = ρ(A2 ), C(A3 ) = C(A), C((A3 )t ) = C(At ). So, by Theorem 2.3.11, G = A(A3 )− A is invariant under choices of g-inverses of A3 . Thus A− ρχ exists, is unique and is equal to A(A3 )− A. We shall now characterize the ρχ-inverse and the classes of all χ-inverses and all ρ-inverses using rank factorization. Theorem 2.4.4. Let A be an n × n matrix of rank r(> 0). Let (P, Q) be a rank factorization of A. Then (i) ρ(A) = ρ(A2 ) if and only if QP is non-singular. (ii) Let ρ(A) = ρ(A2 ). The class of all χ-inverses is given by −1 {P(QP)−1 P−1 : P is an arbitrary left inverse of P}. L L (iii) Let ρ(A) = ρ(A2 ). The class of all ρ-inverses is given by −1 {Q−1 Q : Q−1 R (QP) R is a right inverse of Q}. 2 (iv) Let ρ(A) = ρ(A ). Then the ρχ-inverse of A is P((QP)−1 )2 Q. Proof.
(i) We have ρ(A) = ρ(A2 ) ⇔ ρ(PQ) = ρ(PQPQ) = ρ(QP)
⇔ ρ(QP) = ρ(I) = r. Thus, ρ(A) = ρ(A ) ⇔ QP is non-singular. − (ii) Since ρ(A) = ρ(A2 ), A− χ exists. Clearly, ρ(Aχ ) ≥ ρ(A). On the other − − hand, C(Aχ ) ⊆ C(A), so C(Aχ ) = C(A) and ρ(A− χ ) = ρ(A). Hence, is a reflexive g-inverse of A. By Theorem 2.3.15, every reflexive A− χ −1 −1 −1 −1 g-inverse is of the type QR PL . Also, C(QR PL ) = C(Q−1 R ). Since C(A− χ ) = C(A) = C(P), Q−1 = PT for some matrix T. Further, T R must satisfy I = QQ−1 = QPT. As QP is a square matrix, we have R T = (QP)−1 . Thus, every χ-inverse must be of the form P(QP)−1 P−1 L . −1 −1 −1 −1 Conversely, A(P(QP) PL )A = PQP(QP) PL PQ = PQ = A −1 −1 and C(P(QP)−1 P−1 PL −1 L ) = C(P) = C(A), for each PL . So, P(QP) is a χ-inverse of A. Proof of (iii) is similar to proof of (ii). (iv) follows form (ii) and (iii). 2
Theorem 2.4.5. Let A be an n × n matrix such that ρ(A) = ρ(A2 ). Let A = Pdiag(C, 0) P−1 , where P and C are non-singular.
28
Matrix Partial Orders, Shorted Operators and Applications
(i) The class of all A− χ is given by −1 C L P P−1 ; where L is arbitrary. 0 0 (ii) The class of all A− ρ is given by −1 C 0 P P−1 ; where M is arbitrary. M 0 −1 (iii) G is A− , 0) P−1 ρχ if and only if G = Pdiag(C
Proof.
Straightforward.
Theorem 2.4.6. Let A be an n × n matrix such that ρ(A) = ρ(A2 ). Then G is A− ρχ if and only if (i) AGA = A (ii) GAG = G (iii) AG = GA.
and
Proof. ‘If’ part From (ii) and (iii) it follows that AG2 = G = G2 A. So, C(G) ⊆ C(A) and C(Gt ) ⊆ C(At ). This together with (i) implies that G is A− ρχ . The ‘only if’ part can be easily verified using Theorem 2.4.4(iv). We formally define a commuting g-inverse of a square matrix in the following: Definition 2.4.7. Let A be a square matrix. Then a square matrix G of the same order is called a commuting g-inverse of A if it satisfies the following: (i) AGA = A, i.e. G is a g-inverse of A and (ii) AG = GA. Remark 2.4.8. A square matrix A has a commuting g-inverse if and only if ρ(A) = ρ(A2 ). We now obtain the class of all commuting g-inverses of a square matrix. Theorem 2.4.9. (a) Let A be a square matrix. Then A has a commuting g-inverse if and only if A is of index not greater than 1. (b) Let A be of index not greater than 1 and let A = Pdiag(T, 0) P−1 , where T is non-singular. Then every commuting g-inverse of A is of the
Matrix Decompositions and Generalized Inverses
29
form G = Pdiag(T−1 , C)P−1 , where C is arbitrary. Proof. (a) ‘If’ part Let A be of index not greater than 1. Then by Theorem 2.2.23, there exists a non-singular matrix P such that A = Pdiag(T, 0)P−1 , where T is nonsingular. Let G = Pdiag(T−1 , 0)P−1 . Then G is a commuting g-inverse of A. ‘Only if’ part Let A have a commuting g-inverse G. Then, AGA = A2 G = A. This implies ρ(A) ≤ ρ(A2 ) ≤ ρ(A), or equivalently, ρ(A)2 = ρ(A), so, A is index 1. −1 (b) As noted in the proof of (a), , 0)P−1 is a commuting G = Pdiag(T H1 H2 g-inverse of A. Let H = P P−1 be a commuting g-inverse H3 H4 of A, partitioned conformably for multiplication with A. Now, by direct computation, H1 = T−1 , H2 and H3 are null and H4 is arbitrary. Remark 2.4.10. Let A be a square matrix such that ρ(A) = ρ(A2 ). Then A− ρχ is the unique commuting reflexive g-inverse of A. Remark 2.4.11. In view of symmetry of (i)-(iii) in Theorem 2.4.6, G = − A− ρχ if and only if A = Gρχ . It is obvious that the set of all non-singular matrices of order n × n form a group under matrix multiplication with identity I. The inverse for each matrix A in this group is A−1 , the usual inverse of a non-singular matrix. We now show that the matrices satisfying ρ(A) = ρ(A2 ) also have similar property. Let A be an n × n matrix of rank r such that ρ(A) = ρ(A2 ). Consider the class of matrices defined as CA = {B : C(B) = C(A) and C(Bt ) = C(At ) , ρ(B) = ρ(B2 )} . We shall show that CA is a group under matrix multiplication. Theorem 2.4.12. Let A be an n × n matrix of rank r such that ρ(A) = ρ(A2 ). Let A = Pdiag(T, 0)P−1 , where T is an r × r non-singular matrix. Then (i) CA = {Pdiag(R, 0)P−1 }, where R is an arbitrary r × r non-singular matrix. (ii) CA forms a group under matrix multiplication.
30
Matrix Partial Orders, Shorted Operators and Applications
(iii) For each B ∈ CA , B− ρχ is the inverse of B in the group CA . Proof. Notice that for B ∈ CA , since C(B) = C(A) and C(Bt ) = C(At ), we have C(PBP−1 ) = C(PAP−1 ) and C(P−1 Bt P)t = C(P−1 At P)t . Now (i) follows easily. (ii) Let each of B = Pdiag(R, 0) P−1 and C = Pdiag(S, 0) P−1 belong to CA , where R and S are non-singular matrices of order r × r. Clearly, BC ∈ CA . Thus, CA is closed under multiplication. Associativity follows since the matrix multiplication is associative. It can be easily verified that E = Pdiag(Ir , 0) P−1 is the identity element and for each B = Pdiag(R, 0)P−1 ∈ CA , its inverse is Pdaig(R−1 , 0)P−1 which belongs to CA . Thus, CA is a group. (iii) is easy to check. In view of Theorem 2.4.12, we have Theorem 2.4.13. Let A be a matrix such that ρ(A) = ρ(A2 ). Then the following statements are equivalent: (i) A square matrix G is the unique g-inverse of A with C(G) ⊆ C(A) and C(Gt ) ⊆ C(At ). (ii) G is the unique reflexive commuting g-inverse of A and (iii) G is the inverse of A in the group CA = {B : C(B) = C(A) and C(Bt ) = C(At ), ρ(B) = ρ(B2 )}. Remark 2.4.14. Henceforth, we shall refer to the matrix G satisfying any of the equivalent conditions in Theorem 2.4.13 as the group inverse of A and denote it by A# . Thus, A# is the same as A− ρχ . If A is a non-singular matrix, then Ak is non-singular and (Ak )−1 is (A ) for each (positive) integer k. We now establish a similar property for the group inverse. −1 k
Theorem 2.4.15. Let A be a matrix such that ρ(A) = ρ(A2 ) and let A# be the group inverse of A. Then ρ(Ak ) = ρ((Ak )2 ) and (Ak )# = (A# )k for each positive integer k. Proof. If ρ(A) = ρ(A2 ), it is easy to establish ρ(A) = ρ(As ) for each positive integer s. So, ρ(Ak ) = ρ((Ak )2 ) = ρ(A). The rest follows easily from Theorem 2.4.5(iii).
Matrix Decompositions and Generalized Inverses
31
For a non-singular matrix A, if λ is an eigen-value of A with algebraic multiplicity k, then 1/λ is an eigen-value of A−1 with the same algebraic multiplicity k. The following is easy to establish. Theorem 2.4.16. Let A be a matrix of index ≤ 1. If λ is a non-null eigenvalue of A with algebraic multiplicity k, then 1/λ is an eigen-value of A# with same algebraic multiplicity. Further, if zero is an eigen-value of A with algebraic multiplicity t, then zero is an eigen-value of A# with the same algebraic multiplicity t. Remark 2.4.17. A square matrix is of index not greater than 1 if and only if the algebraic and geometric multiplicities of its zero eigen-value are equal. If A is a non-singular matrix, then it is well known that A−1 is a polynomial in A. We now prove the following: Theorem 2.4.18. Let A be a square matrix. If A has a g-inverse G which is a polynomial in A, then A has index not greater than 1. Moreover, if ρ(A) = ρ(A2 ), then A# is a polynomial in A. Proof. If G is a polynomial in A, then G and A commute. So, G is a commuting g-inverse of A. By Remark 2.4.8, it follows that ρ(A) = ρ(A2 ). Let ρ(A) = ρ(A2 ). Write A = Pdiag(T, 0)P−1 , where P and T are k X −1 non-singular matrices. Since T is non-singular, T = cj Tj for some j=0
positive integer k and some scalars c0 , . . . , ck . It is easy to check that k X A# = cj Aj . j=0 The following Theorem is easy to establish. Theorem 2.4.19. Let A be an n × n matrix of index not greater than 1. Then the following hold: (i) (ii) (iii) (iv)
(A# )# = A. (A# )t = (At )# . F n = C(A) ⊕ N (A). For each non-singular matrix P, (PAP−1 )# = PA# P−1 .
32
Matrix Partial Orders, Shorted Operators and Applications
We now obtain the core-nilpotent decomposition (Theorem 2.2.21) for an n × n matrix A over a field F. Theorem 2.4.20. Let A be an n × n matrix of index k. Then A can be written as A = A1 + A2 where (i) ρ(A1 ) = ρ(A21 ) (ii) A2 is nilpotent and (iii) A1 A2 = A2 A1 = 0. Further, such a decomposition is unique. Proof. (i) If k = 0 or 1, take A1 = A and A2 = 0. Then A1 , A2 satisfy (i), (ii) and (iii). If k > 1, write A1 = Ak (A2k−1 )− Ak and A2 = A − Ak (A2k−1 )− Ak . Since ρ(Ak ) = ρ(Ak+1 ) = ρ(A2k−1 ), by Theorem 2.3.11, A1 is invariant under choices of g-inverse of A2k−1 . Now, A21 = Ak (A2k−1 )− Ak Ak (A2k−1 )− Ak = Ak (A2k−1 )− Ak+1 . Also, A21 (Ak+1 )− Ak = Ak (A2k−1 )− Ak = A1 . Thus, ρ(A1 ) = ρ(A21 ). (ii) It is easy to check that Ar2 = Ar − Ak+r−1 (A2k−1 )− Ak for all r ≥ 1. So, Ak2 = Ak − A2k−1 (A2k−1 )− Ak = Ak − Ak = 0. Thus A2 is nilpotent. (iii) is easy to verify. To prove uniqueness of the decomposition, we proceed as follows: Let A = B1 + B2 be another decomposition where B1 and B2 satisfy the conditions (i), (ii) and (iii). Then Ak = Ak1 = Bk1 . So, C(A1 ) = C(Ak1 ) = C(Bk1 ) = C(B1 ). Similarly, C(At1 ) = C(Bt1 ). Further, A1 A2 = A2 A1 = B1 B2 = B2 B1 = 0. Therefore, 0 = A1 B2 = B2 A1 = B1 A2 = A2 B1 . Thus, A1 (A1 − B1 ) = −A1 (A2 − B2 ) = 0. Now, C(A1 − B1 ) ⊆ C(A1 ), (A1 − B1 ) = A1 (A21 )− A1 (A1 − B1 ) = 0. Hence, A1 = B1 and A2 = B2 . In view of Theorem 2.2.23 and Remark 2.2.24, we can restate Theorem 2.4.20 as Theorem 2.4.21. Let A be an n × n matrix of index k and rank r. Then there exists a non-singular matrix P of order n × n such that A = Pdiag(T, N) P−1 , where T is non-singular and N is nilpotent such that Nk = 0.
Matrix Decompositions and Generalized Inverses
33
Remark 2.4.22. The matrices A1 and A2 in Theorem 2.4.20 are called the core and the nilpotent parts respectively of the matrix A. Further, ρ(A1 ) is called the core rank of A. The core rank of A is less than or equal to the rank of A. Also, k, the index of A is the least positive integer such that Ak2 = 0 (called the index of nilpotency or simply index of A2 ). Notice that ρ(A) = ρ(A1 ) + ρ(A2 ). As noted earlier, a matrix does not have a commuting g-inverse unless it is of index not greater than 1. However, for a general square matrix not necessarily of index ≤ 1, we can define a type of commuting inverse (which may not be a generalized inverse) that is a generalization of the group inverse. Definition 2.4.23. Let A be an n × n matrix. A matrix G is called the Drazin inverse of A, denoted by AD , if G satisfies the following properties: (i) GAG = G. (ii) Ak = Ak+1 G for some positive integer k and (iii) AG = GA. Remark 2.4.24. AD = A# if k ≤ 1. If A is either null or nilpotent, then AD = 0. Remark 2.4.25. Let k be the smallest positive integer such that Ak = Ak+1 G. If k ≥ 2, then G is not a g-inverse of A. We now prove the existence and uniqueness of the Drazin inverse for any square matrix. Theorem 2.4.26. Let A be an n × n matrix of index k and rank r. As in Theorem 2.4.21, let A = Pdiag(T, N)P−1 , where T and P are non-singular and N is nilpotent such that Nk = 0. Then AD = Pdiag(T−1 , 0) P−1 and AD is unique. Proof. It is easy to check that Pdiag(T−1 , 0)P−1 satisfies the conditions (i)-(iii) of Definition 2.4.23. To show that AD is unique, let G1 and G2 be any two choices for AD . Consider Gk+1 A2k+1 Gk+1 . By repeated applications of the conditions 1 2 (i)-(iii) of Definition 2.4.23, we have G1 = Gk+1 A2k+1 Gk+1 = G2 . 1 2 Remark 2.4.27. Consider the core-nilpotent decomposition as in Theorem k 2k+1 − k 2.4.20. Then AD = A# ) A . 1 = A (A
34
Matrix Partial Orders, Shorted Operators and Applications
Remark 2.4.28. Let A be a non-null matrix. Then AD = 0 ⇔ A is nilpotent. The following theorem is easy to establish. Theorem 2.4.29. Let A be an n × n matrix of index k. Then ((AD )D )D = AD . (AD )t = (At )D . (PAP−1 )D = PAD P−1 for each non-singular matrix P. Fn = C(Ak ) ⊕ N (Ak ). C(AD ) = C(Ak ), N (AD ) = N (Ak ). AAD = AD A is the projector PC(Ak ), N (Ak ) which projects vectors onto C(Ak ) along N (Ak ). (vii) (I − AAD ) = (I − AD A) = PN (Ak ), C(Ak ) . (i) (ii) (iii) (iv) (v) (vi)
We now demonstrate that the Drazin inverse AD too is a polynomial in A just as we showed that the group inverse of a matrix B when it exists, is a polynomial in B. Theorem 2.4.30. Let A be an n × n matrix. Then AD is a polynomial in A. Proof. Let k be the index of A. Then as seen in Remark 2.4.27, AD = Ak (A2k+1 )− Ak . Clearly, Ak (A2k+1 )− Ak is invariant under choice of (A2k+1 )− and ρ(A2k+1 ) = ρ((A2k+1 )2 ). So, AD = Ak (A2k+1 )# Ak . We know (A2k+1 )# is a polynomial in A2k+1 and therefore, in A. Hence, AD is a polynomial in A. We now obtain a formula for the index and Drazin inverse using rank factorization. Theorem 2.4.31. Let A be a non-null n × n matrix. Write A0 = A. Let Qi Pi if Ai 6= 0 and (Pi , Qi ) is a rank factorization of Ai Ai+1 = . 0 if Ai = 0 for i = 1, 2, . . .. Then the following hold: number of columns in Pi ifAi 6= 0 i+1 (i) ρ(A ) = 0 if A = 0 i
Matrix Decompositions and Generalized Inverses
35
(ii) There exists an integer s ≥ 0 such that As is either non-singular or null. Let k be the smallest integer ≥ 0 such that Ak is either non-singular or null. Then k if Ak is non-singular (iii) index of A = . k + 1, if A = 0 k
i+1
(iv) C(A ) = C(P0 P1 . . . Pi ) and N (Ai+1 ) = N (Qi . . . Q1 Q0 ) for i = 1, . . . k − 1 if Ak is non-singular and for i = 1, . . . , k − 2 if Ak is null. 0 if Ak = 0 . (v) AD = −(k+1) P0 P1 . . . Pk−1 Ak
Proof.
Qk−1 . . . Q0 when Ak is non-singular
(i) It is easy to see that Ai+1 = P0 P1 . . . Pi Qi . . . Q1 Q0 = P0 P1 . . . Pi−1 Ai Qi−1 . . . Q1 Q0 .
If Ai 6= 0, then Pi and Qi are full rank matrices and ρ(Ai+1 ) = ρ(Ai ) = ρ(Pi Qi ) for each i. If Ai is null, so is Ai+1 , therefore ρ(Ai+1 ) = 0 and if Ai is non-null, ρ(Ai+1 ) = ρ(Ai ) = ri = number of columns in Pi . (ii) Suppose Aj is neither null nor non-singular for each j. Let the order of Aj be rj × rj . Then Aj+1 is of order rj+1 × rj+1 , where rj+1 < rj . Thus {rj } is a strictly decreasing sequence as long as {Aj } is a sequence of neither non-singular nor null matrices. But this is not possible since rj can never be negative. This proves (ii). Suppose there exists As such that As is non-singular. Then it is of the same order as At for all t ≥ s. Also, if As = 0, then At = 0 all t ≥ s. Thus, there exists a smallest integer k ≥ 0 such that Ak is either non-singular or null. (iii) Let k be the smallest positive integer such that Ak is either non-singular or null. Then Ak+1 = P0 P1 . . . Pk−1 Pk Qk Qk−1 . . . Q1 Q0 = P0 P1 . . . Pk−1 Ak Qk−1 . . . Q1 Q0 . Let Ak be null. Then, Ak+1 = 0, so, A is nilpotent. As k is the smallest positive integer for which Ak is null, A has index k + 1. Now, let Ak be non-singular. Then ρ(Ak ) = rk , Ak+1 = Qk Pk is of order rk × rk . Therefore,
36
Matrix Partial Orders, Shorted Operators and Applications
ρ(Ak+1 ) = ρ(Ak ) = rk = ρ(Ak ) and k is the smallest positive integer for which this is true. So, index of A is k. (iv) Let i = 1, . . . , k − 1. From the proof of (ii) we have Ai+1 = P0 P1 . . . Pi−1 Pi Qi Qi−1 . . . Q1 Q0 , where Pj has a left inverse and Qj has a right inverse for j = 0, 1, . . . , i. Hence C(Ai+1 ) = C(P0 P1 , . . . Pi−1 Pi ) and N (Ai+1 ) = N (Qi . . . Q1 Q0 ). Let Ak be non-singular. Then we have Ak+1 = P0 P1 . . . Pk−1 Pk Qk Qk−1 . . . Q1 Q0 , where Pk has a left inverse and Qk has a right inverse. Hence, C(Ak+1 ) = C(P0 P1 . . . Pk−1 Pk )andN (Ak+1 ) = N (Qk . . . Q1 Q0 ). (v) If Ak = 0, then A is nilpotent. By Remark 2.4.27, AD = 0. Let Ak −(k+1) be non-singular. Let X = P0 P1 . . . Pk−1 Ak Qk−1 . . . Q1 Q0 . Then it is easy to check that XAX = X, XA = AX and XAk+1 = Ak = Ak+1 X. Thus, X is the Drazin inverse of A. Corollary 2.4.32. Let A be an n × n matrix of index ≤ 1 and (P, Q) be a rank factorization of A. Then A# = P(QP)−2 Q . 2.5
Moore-Penrose inverse
In this section we specialize to vectors and matrices over the field of complex numbers C and use the inner product (x, y) = y? x for x, y ∈ Cn . Let Ax = b be a consistent system of linear equations for A ∈ Cm×n and b ∈ Cm . In Section 2.3, we noticed that, in general, a solution to Ax = b is not unique. Since there are many solutions, we can look for solutions with some optimal property. One such is the minimum norm property as different solutions have possibly different norms. Does a solution with minimum norm exist? If so, does there exist a g-inverse G of A such that Gb is a solution to Ax = b with minimum norm, whenever the system is consistent? What if Ax = b is not consistent? Does there exist a g-inverse G of A such that kAGb − bk ≤ kAx − bk for each x? This means that we are seeking a g-inverse G of A such that the vector AGb is the closest to b.
Matrix Decompositions and Generalized Inverses
37
Here we shall attempt to find answers to the above questions. It turns out that the answers to all of these are in affirmative. We shall see that the minimum norm solution is unique but a least squares solution, in general, is not unique. The next question that naturally arises is: Does there exist a g-inverse G of A such that Gb has the smallest norm in the class of all least squares solutions to Ax = b for all column vectors b? The answer again is in affirmative. In fact, we show that such a g-inverse is unique. It is this g-inverse that Moore and Penrose arrived at from different considerations. This g-inverse, named after them, is called the Moore-Penrose inverse. In this section, we shall characterize the subclasses of g-inverses mentioned in this and previous two paragraphs and study their properties. In the previous section we studied g-inverse with specified column and row spaces. It turns out that the g-inverses studied in this section also have the column and row spaces contained in certain subspaces of interest which bear a striking similarity to those in the previous section. Thus, we notice that the g-inverses we study in this section have striking similarities to A− χ, # − − # A− and A . While A , A and A exist for a matrix A of index ≤ 1 ρ χ ρ over a general field, the g-inverses that we are about to study in this section exist for every matrix over the field of complex numbers (or a subfield of it). We first prove a lemma, which will be used frequently in this section. Lemma 2.5.1. Let A and B be m × n and m × p matrices respectively. Then kAxk ≤ kAx + Byk for all x and y if and only if A? B = 0. Proof.
We first note that
kAxk ≤ kAx + Byk ⇔ (Ax, Ax) ≤ (Ax + By, Ax + By) ⇔ y? B? Ax + x? A? By + y? B? By ≥ 0 . ‘If’ part follows trivially, since B? B is nnd and A? B = 0 implies (A? B)? = B? A = 0. ‘Only if’ part Let if possible, A? B 6= 0. Then there exists a vector y necessarily nonnull such that A? By 6= 0. So, By 6= 0 and y? B? By > 0. Let the j th component of A? By be a + ib 6= 0. Choose xj , the j th component of x equal to −((a + ib)/(a2 + b2 ))(y? B? By) and all other components of x as zero. Now, x? A? By = −((a − ib)/(a2 + b2 ))(y? B? By)(a + ib) = −y? B? By.
38
Matrix Partial Orders, Shorted Operators and Applications
Also, y? B? Ax = −(y? B? By)? = −y? B? By. So, for these choices of x and y, we have y? B? Ax + x? A? By + y? B? By = −y? B? By < 0, which is a contradiction. So, A? B = 0
Definition 2.5.2. Let Ax = b be consistent. Then a vector x0 is said to be a minimum norm solution to Ax = b if (i) Ax0 = b and (ii) kx0 k ≤ kxk for every x such that Ax = b. Theorem 2.5.3. Let A be an m × n matrix. Then there exists a g-inverse G of A such that Gb is a minimum norm solution to Ax = b whenever it is consistent. Proof. Consider G = A? (AA? )− where (AA? )− is some g-inverse of AA? . Since ρ(AA? ) = ρ(A), by Theorem 2.3.12, G is a g-inverse of A. So, Gb is a solution to Ax = b whenever it is consistent. Also, the class of all solutions to Ax = b is given by Gb + (I − GA)ξ, ξ arbitrary. We now show that kGbk ≤ kGb + (I − GA)ξk for all b ∈ C(A) and for all ξ. Now, b ∈ C(A) ⇔ b = Au for some u. Thus, we need to show kGAuk ≤ kGAu + (I − GA)ξk for all u and ξ. By Lemma 2.5.1, the above happens if and only if (GA)? (I − GA) = 0. Now, (GA)? (I − GA) = (A? (AA? )− A)? (I − A? (AA? )− A) = (A? ((AA? )− )? A)(I − A? (AA? )− A) = (A? ((AA? )− )? A) − (A? ((AA? )− )? )AA? (AA? )− A. Since C(A) = C(AA? ), A = AA? (AA? )− A. Therefore, (GA)? (I − GA) = (A? ((AA? )− )? A) − (A? ((AA? )− )? )A = 0. So, kGAu + (I − GA)ξk2 = kGAuk2 + k(I − GA)ξk2 . Hence, Gb is a minimum norm solution to Ax = b, whenever it is consistent. Definition 2.5.4. Let A be an m × n matrix. Then a matrix G of order n × m is said to be a minimum norm g-inverse of A, if Gb provides a minimum norm solution to Ax = b, whenever it is consistent. A minimum norm g-inverse of A is denoted by A− m.
Matrix Decompositions and Generalized Inverses
39
We shall now obtain some characterizations of A− m. Theorem 2.5.5. Let A be an m × n matrix. then G is a minimum norm g-inverse of A if and only if it satisfies any one of the following equivalent conditions: (i) AGA = A, (GA)? = GA (ii) GA = PA? , the orthogonal projector onto C(A? ) (iii) GAA? = A? .
and
Proof. In the proof of Theorem 2.5.3, we saw that G is a minimum norm g-inverse of A if and only if AGA = A and (GA)? (I−GA) = 0. However, (GA)? (I − GA) = 0 ⇒ (GA)? = (GA)? GA = GA, since (GA)? GA is hermitian. Conversely, if AGA = A and (GA)? = GA, then (GA)? (I − GA) = (GA)? − (GA)? (GA) = GA − (GA)(GA) = GA − GAGA = GA − GA = 0 . So, G is a minimum norm g-inverse of A if and only if (i) holds Therefore, it is enough to show (i) ⇒ (ii) ⇒ (iii) ⇒ (i). (i) ⇒ (ii) Let (i) hold. Clearly, AGA = A ⇒ (GA)2 = GA, so, GA is idempotent. Also, (GA)? = GA implies GA is hermitian. So, GA is the orthogonal projector onto C(GA). However, GA = (GA)? = A? G? and ρ(A? G? ) = ρ(GA) = ρ(A) = ρ(A? ). Therefore, C(GA) = C(A? G? ) = C(A? ), proving (ii) holds. (ii) ⇒ (iii) is trivial. (iii) ⇒ (i) GAA? = A? ⇒ GAA? G? = A? G? ⇒ GA(GA)? = (GA)? . Since GA(GA)? is hermitian, (GA)? = GA. Also, GAA? = A? ⇒ (GA)? A? = A? ⇒ AGA = A.
We now show that minimum norm solution is unique. Theorem 2.5.6. Let Ax = b be consistent. Then minimum norm solution to Ax = b is unique. Proof. Let G be one choice of A− m . Then Gb is a minimum norm solution to Ax = b. Every solution to Ax = b is of the form Gb + (I − GA)ξ
40
Matrix Partial Orders, Shorted Operators and Applications
for some vector ξ. Let, if possible, kGbk = kGb + (I − GA)ξk for some ξ. Then kGbk2 = kGb + (I − GA)ξk2 = kGbk2 + k(I − GA)ξk2 + b? G? (I − GA)ξ + ξ ? (I − GA)? Gb. Since b ∈ C(A), b = Au for some u. So, b? G? (I − GA)ξ = u? A? G? (I − GA)ξ = 0. Also, ξ ? (I − GA)? Gb = (b? G? (I − GA)ξ)? = 0. Therefore, kGbk2 = kGbk2 + k(I − GA)ξk2 , which is possible only if (I − GA)ξ = 0. Thus, minimum norm solution is unique. We now obtain the class of all minimum norm g-inverses of a matrix A. Observe that for the null matrix of order m × n, every matrix of order n × m is a minimum norm g-inverse. Theorem 2.5.7. Let A be an m × n matrix and G be a minimum norm − g-inverse of A. Then the class {A− m } of all Am is given by {G + U(I − AG), U arbitrary}. Proof.
Let U be an arbitrary matrix. Then (G + U(I − AG))AA? = GAA? = A? .
So, G + U(I − AG) is a minimum norm g-inverse of A for all U. Let G1 be any minimum norm g-inverse of A. Then it is easy to check that G1 = G + U(I − AG), where U = G1 − G. Thus, the class of all minimum norm g-inverse is given by G + U(I − AG), where U is arbitrary. (Note that GA = PA? = G1 A.) Theorem 2.5.8. Let A be an m × n matrix of rank r (> 0). Let Udiag(D, 0)V? be a singular value decomposition of A, where U and V are unitary and D is a positive definite diagonal matrix of order r×r. Then the class of all minimum norm g-inverses of A is given by −1 D L ? V U : L, N arbitrary . 0 N Proof.
Notice that the class of all g-inverses of A is given by −1 D L ? U ; L, M and N arbitrary . V M N
It is easy to check that GAA? = A? if and only if M = 0.
Matrix Decompositions and Generalized Inverses
41
Theorem 2.5.9. Let A be an m × n matrix of rank r(> 0). The class − {A− mr } of all reflexive minimum norm g-inverses Amr of A is given by {A? (AA? )− : (AA? )− is an arbitrary g-inverse of AA? }. Proof. In the proof of Theorem 2.5.3, we showed that A? (AA? )− is a minimum norm g-inverse of A. Notice that ρ(A? (AA? )− ) ≤ ρ(A? ). However, ρ(A? (AA? )− ) ≥ ρ(A) = ρ(A? ) as A? (AA? )− is a g-inverse of A. So, ρ(A? (AA? )− ) = ρ(A). Hence, A? (AA? )− is an A− mr . − Conversely, let G = A− mr . As G is an Ar and GA is hermitian, C(G) = C(GA) = C(A? G? ) = C(A? ). Thus G = A? T for some matrix T. Now, A? = GAA? = A? TAA? . Therefore, AA? = AA? TAA? , showing T is a g-inverse of AA? . Theorem 2.5.10. Let (P, Q) be a rank factorization of an m × n matrix ? ? −1 −1 A of rank r(> 0). Then G is an A− PL mr if and only if G = Q (QQ ) −1 for some left inverse PL of P. Proof. We note that C(Q? ) = C(A? ). The rest of the proof is similar to (i) of Theorem 2.4.4. Let us now turn our attention to a possibly inconsistent system of linear equations. We are given a system of linear equations Ax = b which may or may not be consistent. In case it is consistent, we want to find a solution. Otherwise the next best thing to do is to find an approximate solution x0 such that Ax0 is the closest to b among all Ax i.e. kAx0 − bk ≤ kAx − bk for all x. If such a vector exists, then for all x, (Ax0 − b)? (Ax0 − b) ≤ (Ax − b)? (Ax − b). Such an approximate solution is called a least (sum of) squares solution to Ax = b. We shall now explore the existence of a g-inverse G of A such that x = Gb is a least squares solution to Ax = b. Notice that a least squares solution should be an exact solution if Ax = b is consistent. Theorem 2.5.11. Let A be an m×n matrix. Then there exists a g-inverse G of A such that x = Gb is a least squares solution to Ax = b for all b ∈ Cm . Proof. Consider G = (A? A)− A? . By Theorem 2.3.12, G is a g-inverse of A. Now kAx − bk = kAx − AGb + AGb − bk.
42
Matrix Partial Orders, Shorted Operators and Applications
By Lemma 2.5.1, kAGb − bk ≤ kAGb − b + A(x − Gb)k for all b and for all x ⇔ A? (AG − I) = 0. Moreover, A? AG = A? A(A? A)− A? = A? , since C(A? A) = C(A? ). Thus, G = (A? A)− A? is a g-inverse of A such that Gb is a least squares solution to Ax = b for all b. Remark 2.5.12. If Gb has to be a least squares solution to Ax = b for all b, then AGb = b for all b ∈ C(A). So, G must be a g-inverse of A. Definition 2.5.13. Let A be an m × n matrix. Then a matrix G is said to be a least squares g-inverse of A, denoted by A− ` , if Gb is a least squares solution to Ax = b for all b ∈ Cm . We shall now obtain some characterizations of A− ` . Theorem 2.5.14. Let A be an m × n matrix. Then a matrix G is a least squares g-inverse of A if and only if it satisfies any one of the following equivalent conditions: (i) AGA = A, (AG)? = AG (ii) AG = PA , the orthogonal projector projecting vectors into C(A). (iii) A? AG = A? . Proof. In the proof of Theorem 2.5.11, we observed that G is a least squares g-inverse G of A ⇔ G satisfies (iii) i.e. if A? AG = A? . Thus, it is enough to prove (iii) ⇒ (i) ⇒ (ii) ⇒ (iii). Let G be a matrix satisfying (iii). Then, A? AGA = A? A. Now, C(A? A) = C(A? ), so, there exists a matrix T such that A = TA? A. Hence, A? AGA = A? A ⇒ TA? AGA = TA? A ⇒ AGA = A. Also, A? AG = A? ⇒ G? A? AG = G? A? = AG. So, AG is hermitian and therefore (i) holds. Let (i) hold. Then AG is an orthogonal projector onto C(AG). But AGA = A ⇒ C(AG) = C(A). So, AG = PA . Thus (ii) holds. (ii)⇒ (iii) is trivial. Notice the striking similarity of conditions in Theorems 2.5.5 and 2.5.14. Consequently, we have the following important duality between minimum norm g-inverse and least squares g-inverses. Theorem 2.5.15. Let A be an m × n matrix. then G is an A− m if and only if G? is an (A? )− . `
Matrix Decompositions and Generalized Inverses
43
Proof follows from Theorems 2.5.5 and 2.5.14, once we note that G is a g-inverse of A ⇒ G? is a g-inverse of A? . Consider a system of linear equations Ax = b. Then x0 is a least squares solution to Ax = b if and only if Ax0 is the orthogonal projection of b in C(A). Also, then b − Ax0 is the orthogonal projection of b in (C(A))⊥ . Hence, even if x0 may not be unique, both the vectors Ax0 and b − Ax0 are unique. Theorem 2.5.16. Let A be an m × n matrix and let G be a least squares g-inverse of A. Then the class of all least squares solutions to Ax = b is given by {Gb + (I − GA)ξ, ξ arbitrary}. Proof. Clearly, A(Gb + (I − GA)ξ) = AGb for all ξ. Further Gb is a least squares solution to Ax = b. Hence Gb + (I − GA)ξ is a least squares solution to Ax = b for all ξ. Let x0 be a least squares solution to Ax = b. By the discussion just preceding this theorem, Ax0 = AGb. So, A(x0 − Gb) = 0. Thus, x0 − Gb = (I − GA)ξ for some ξ. So, x0 = Gb + (I − GA)ξ for some ξ. The following Theorems 2.5.17-2.5.19 and Theorem 2.5.21 can be established along the lines of Theorems 2.5.7-2.5.10. Theorem 2.5.17. Let A be an m × n matrix. Let G be a least squares g-inverse of A. Then the class of all A− ` is given by {G + (I − GA)V : V arbitrary}. Theorem 2.5.18. Let A be an m × n matrix of rank r(> 0). Let Udiag(D, 0)V? be a singular value decomposition of A, where U and V are unitary and D is a positive definite diagonal matrix of order r×r. Then the class {A− ` } of all least squares g-inverse of A is given by −1 D 0 ? V U : M and N arbitrary . M N Theorem 2.5.19. Let A be an m × n matrix of rank r(> 0). The class − {A− `r } of all reflexive least squares g-inverses A`r of A is given by {(A? A)− A? : (A? A)− is an arbitrary g-inverse of A? A}.
44
Matrix Partial Orders, Shorted Operators and Applications
Remark 2.5.20. For a matrix A, G is an A− `r if and only if G is g-inverse of A such that C(G? ) = C(A). Recall that G is an A− ρ if and only if C(G? ) = C(A? ). Theorem 2.5.21. Let (P, Q) be a rank factorization of an m × n matrix −1 ? −1 ? A of rank r(> 0). Then G is an A− P `r if and only if G = QR (P P) −1 for some right inverse QR of Q. − Notice the similarity between A− χ and Amr . Are there counterparts to − and A` for index-1 matrices? Indeed there are and we shall introduce them in Chapter 6. We have seen in Theorem 2.5.16 that, in general, there are many least squares solutions to Ax = b. We shall now show that there is a g-inverse G of A such that Gb has the minimum norm among all least squares solutions to Ax = b for all b ∈ Cm .
A− m
Theorem 2.5.22. Let A be an m × n matrix. Then, there exists a ginverse G of A such that Gb has the minimum norm among all least squares solutions to Ax = b for all b ∈ Cm . Proof. Let G be a least squares g-inverse of A. Then the class of all least squares solutions to Ax = b is given by Gb + (I − GA)ξ, where ξ is arbitrary. Now, we are seeking a least squares g-inverse G such that kGbk ≤ kGb + (I − GA)ξk for all b and ξ. By Lemma 2.5.1, kGbk ≤ kGb + (I − GA)ξk ⇔ G? (I − GA) = 0 ⇔ G? GA = G? . Consider G = A? (A? AA? )− A? . Then A? AG = A? AA? (A? AA? )− A? = A? , since ρ(A? AA? ) = ρ(A? ). So G is a least square g-inverse of A. Again G? GAA? = ((A? A? AA? )− A? )? A? (A? AA? )− A? AA? = A{(A? AA? )− }? AA? , since ρ(A? AA? ) = ρ(AA? ) . As ρ(AA? ) = ρ(A), it follows that G? GA = A{(A? AA? )− }? A = G? . Thus, G = A? (A? AA? )− A? is a g-inverse with the desired properties. Remark 2.5.23. From the proof of Theorem 2.5.22, we see that Gb has the minimum norm among all least square solutions if and only if A? AG = A? − and G? GA = G? or equivalently G is an A− ` and A is a G` .
Matrix Decompositions and Generalized Inverses
45
Theorem 2.5.24. Let A be an m×n matrix. Then G is a matrix such that Gb has the minimum norm among all least squares solutions to Ax = b for all b ∈ Cm if and only if any one of the following equivalent conditions holds: (i) A? AG = A? and G? GA = G? (ii) AG = PA , GA = PG and (iii) AGA = A, GAG = G, (AG)? = AG, (GA)? = GA. Proof. (i) follows by Theorem 2.5.22. The equivalence of (i)-(iii) follows from Theorem 2.5.14. Remark 2.5.25. Let Gb have the minimum norm among all least squares solutions to Ax = b for all b ∈ Cm . Then (iii) of Theorem 2.5.24 shows that G is a minimum norm, least squares, reflexive g-inverse of A. Definition 2.5.26. Let A be an m × n matrix. Then an n × m matrix G satisfying any one of the equivalent conditions of Theorem 2.5.24 is called Moore-Penrose inverse of A and is denoted by A† . Theorem 2.5.27. Let A be an m×n matrix. Then Moore-Penrose inverse A† of A is unique. Proof. Let G1 and G2 be two choices of Moore-Penrose inverse of A. Then G1 = G1 AG1 = G1 PA = G1 AG2 = PA? G2 = G2 AG2 = G2 . The following theorem is easy to prove. Theorem 2.5.28. Let A be an m × n matrix. Then the Moore-Penrose inverse A† has the following properties: (i) (ii) (iii) (iv) (v)
A?† = A†? . (A† )† = A. (A? A)† = A† A†? . A† = (A? A)† A? . If R and S are unitary matrices such that the product RAS is defined, then (RAS)† = S? A† R? .
Remark 2.5.29. In Theorem 2.5.22, we gave an expression for A† , namely A† = A? (A? AA? )− A? . Thus, it follows that A† is the unique g-inverse G such that C(G) = C(A? ) and C(G? ) = C(A). Remark 2.5.30. Recall that A# = A(A3 )− A. Also, A# is the unique g-inverse of A such that C(G) = C(A) and C(G? ) = C(A? ).
46
Matrix Partial Orders, Shorted Operators and Applications
In view of this similarity, it is worth asking when A† and A# coincide. Theorem 2.5.31. Let A be an m × m matrix of index 1 over C. Then A† and A# coincide if and only if A is a range-hermitian. Proof follows from Remark 2.5.29 and Remark 2.5.30. The following characterizations of Moore-Penrose inverse are easy to establish. Theorem 2.5.32. Let A be an m × n matrix of rank (r > 0). Let A = Udaig(D, 0)V? be a singular value decomposition of A, where U and V are unitary and D is a positive definite diagonal matrix of order r × r. Then A† = Vdiag(D−1 , 0)U? . (Notice that we have a singular value decomposition of A† .) Theorem 2.5.33. Let A be an m × n matrix of rank r(> 0). Let (P, Q) be a rank factorization of A. Then A† = Q? (QQ? )−1 (P? P)−1 P? . We conclude this section by making a useful observation that, given an algorithm to compute a g-inverse of a matrix, we can compute various types of g-inverse of a given matrix using this algorithm. Reflexive g-inverse ρ-inverse χ-inverse Group inverse Least squares g-inverse Minimum norm g-inverse Moore-Penrose inverse
: : : : : : :
GAG, if G is a g-inverse of A A(A2 )− (A2 )− A A(A3 )− A (A? A)− A? A? (AA? )− A? (A? AA? )− A?
In each of the above cases, we apply the algorithm for computing a g-inverse to a suitable matrix (for example to A? AA? in case of MoorePenrose inverse) and make adjustment to it, be it a pre- and/or postmultiplication by a suitable matrix to get the required type of g-inverse of the given matrix.
2.6
Generalized inverses of modified matrices
In this section, we study the g-inverse of matrices that have been modified in some suitable ways. The need for a g-inverse of modified matrices arises
Matrix Decompositions and Generalized Inverses
47
in several situations. Consider a linear model Y = Xβ + η, where X is a known m × n matrix, β is an unknown but non-stochastic n-vector and η is a random m-vector such that its expected value E(η) = 0 and dispersion matrix D(η) = σ 2 I where σ 2 (> 0) is an unknown constant. Then a least ˆ = X− Y = (Xt X)− Xt Y. If pt β is estimable, squares estimator of β is β ` ˆ is its BLUE with variance V (pt β) ˆ = pt (Xt X)− p σ 2 . Suppose one then pt β more uncorrelated observation is available, namely: ym+1 = xtm+1 β+ηm+1 , where E(ηm+1 ) = 0, V (η m+1 ) = σ 2 and Cov(ηm+1 , η) = 0. Then a least squares estimator after incorporating the new observation may be obtained as ˜ = ((Xt : xt β )t )− (Yt : ym+1 )t m+1
`
= (Xt X + xm+1 xtm+1 )− (X t Y + ym+1 xm+1 ). ˜ of BLUE of pt β in the new model is Also, the variance V (pt β), t t t − p (X X + xm+1 xm+1 ) p σ 2 . Thus after incorporating the new observation, we need t t − ((Xt : xtm+1 )t )− ` and (X X + xm+1 xm+1 ) .
In order to assess the influence of an observation on the estimators of model parameters one needs to delete a row from X or subtract a rank 1 matrix from Xt X and obtain a suitable g-inverse of the modified matrix. When a new state is included in a Markov chain or a new port is introduced in an n-port network, modified matrices and their g-inverses of suitable type become important in the study of the effect of such modifications. Let A be an m × n matrix and G be a g-inverse of A. Here we will consider the following types of modifications (i) appending/deleting a row or a column, (ii) appending a row and a column, and (iii) adding a rank 1 matrix of the same order. In each case, we obtain a suitable modification of G, so that it will be a g-inverse of the modified matrix. The following lemma is useful in what follows: Lemma 2.6.1. Let A be an m × n matrix of rank r(> 0) and let u1 , u2 , . . . , uk be m-vectors such that ρ(A : u1 : . . . : uk ) = ρ(A)+k, where k ≤ m − r. Then there exists a vector c such that ct A = 0, and ct ui = 1 for i = 1, . . . , k.
48
Matrix Partial Orders, Shorted Operators and Applications
Proof. Let (P, Q) be a rank factorization of A. Then (P : u1 : . . . : uk ) is an m × (r + k) matrix of rank r + k. Let (Rt : v1 : vk )t be a left inverse of (P : u1 : . . . : uk ), where R is an r × m matrix. Then vi t P = 0 for i = 1, . . . , k and vi t uj = δij , i, j = 1, . . . , k. Let c = v1 + v2 + . . . + vk . Clearly, ct P = 0 and ct ui = 1 for i = 1, . . . , k. Corollary 2.6.2. If the matrices considered are over complex field, then there exists a vector c such that c? A = 0, c? ui = 1 for all i = 1, . . . , k. We shall start with the first type of modification. Let A be an m × n matrix and a g-inverse G (of A) of a suitable type be available. Let a be an m-column vector. Write d = Ga. We shall show in Theorems 2.6.3-2.6.10 that a corresponding type of g-inverse of (A : a) can be expressed in the form X = (Gt − cdt : c)t (2.6.1) for some suitable c. Theorem 2.6.3. Let A be an m × n matrix and G be a g-inverse of A. Let a be an m-column vector and d = Ga. If a 6∈ C(A), c is an m-column vector such that ct A = 0, ct a = 1 (such a c is guaranteed by Lemma 2.6.1), and X is as defined in (2.6.1), then (i) X is (A : a)− if G is A− . − (ii) X is (A : a)− r if G is Ar . (iii) Let all the vectors and matrices be over C. Then X is (A : a)− m if G − is a Am . Theorem 2.6.4. Let A, a, G and d be as specified in Theorem 2.6.3, but (I − AG)a with complex elements. Let a 6∈ C(A) and c = ? . Write X = a (I − AG)a ? ? ? (G − cd : c) . Then − (i) X is (A : a)− ` if G is A` . † † (ii) X is (A : a) if G is A .
Theorem 2.6.5. Let A, a, G, d be as in Theorem 2.6.3. Let a ∈ C(A). Then (i) X is (A : a)− if G is an A− and c is arbitrary. − t (ii) X is (A : a)− r if G is an Ar and c ∈ C(G ) is arbitrary. (iii) Assuming that the vectors and matrices are over C, X is (A : a)− ` if G is A− and c is arbitrary. `
Matrix Decompositions and Generalized Inverses
Theorem 2.6.6. In the setup of Theorem 2.6.4, let c = Then
49
G? Ga . 1 + a? G? Ga
− (i) X is a (A : a)− m if G is Am . † † (ii) X is (A : a) if G is A .
Suppose that a g-inverse X = (Gt : b)t of a suitable type for (A : a) is available, where G is an n×m matrix and b is an m-column vector. We shall now find a modification of X so that it is a g-inverse of A. Interestingly, G itself is a g-inverse of A when a 6∈ C(A). We shall probe this further in the following Lemma 2.6.7. Let X = (Gt : b)t be a g-inverse of (A : a). Then a 6∈ C(A) if and only if bt a = 1 and bt A = 0. Further, if a 6∈ C(A), then AGa = 0 and X1 = (Gt At Gt : b)t is a g-inverse of (A : a). Further, if the vectors and matrices are over C, then the following holds: Let X = (G? : b)? be a g-inverse of (A : a). Then a 6∈ C(A) if and only if b? a = 1 and b? A = 0. Theorem 2.6.8. In the setup of Lemma 2.6.7, let a 6∈ C(A). Then (i) G is a A− if (Gt : b)t is (A : a)− . t t − (ii) G is a A− r if (G : b) is (A : a)r . Assume that the matrices and the vectors are over C. Then ? ? − (iii) G is a A− m if (G : b) is (A : a)m . − − (iv) G is A` if (G? : b)? is (A : a)` . (v) G is A† if (G? : b)? is (A : a)† . We shall now consider the case when a ∈ C(A). t Theorem 2.6.9. In the setup of Lemma 2.6.7, let X = (Gt : b) be a gt ab . Then inverse of (A : a) and let bt a 6= 1. Write G1 = G I + 1 − bt a
(i) G1 is A− . (ii) G1 is Ar if X is (A : a)− r . Let the matrices and the vectors be over C. Let X =(G? : b)? bea ab? g-inverse of (A : a) and let b? a 6= 1. Write G1 = G I + . 1 − b? a Then − (iii) G1 is A− ` if X is (A : a)` . − (iv) G1 is Am if X is (A : a)− m. (v) G1 is A† if X is (A : a)† .
50
Matrix Partial Orders, Shorted Operators and Applications
The case that is remaining is bt a = 1 and bt A 6= 0 over general field and b? a = 1 and b? A = 0 over C. Since bt A 6= 0 (respectively b? A 6= 0 over C) there exists a positive integer j such that bt aj 6= 0 (respectively b? aj 6= 0 over C) where aj is the j th column of A. Define c = a + aj . Let E be the matrix obtained from G by replacing its j th row gjt (gj? over C) by gjt − bt (respectively gj? − b? over C). Then clearly (Et : b)t (respectively (E ? : b)? over C) is a g-inverse of (A : c). Also bt c = bt a + bt aj 6= 1 (respectively b? c = b? a + b? aj 6= 1 over C). Thus, we have Theorem 2.6.10. In the setup of Lemma 2.6.7, let X = (Gt : b)t be a gt t inverse of let b a = 1 and b A 6= 0. Write (A : a). Moreover, t (a + aj )b . Then G2 = E I − bt aj (i) G2 is A− . − (ii) G2 is A− r if X is (A : a)r . Consider the matrices and the vectors over C. Let X = (G? : b)? ? ? be a g-inverse of (A : a) and let b a = 1 and b A 6= 0. Write ? (a + aj )b G2 = E I − . Then b? aj − (iii) G2 is A− ` if X is (A : a)` . Proofs of Theorems 2.6.3-2.6.10 are computational. For detailed proofs one may consult [Mitra and Bhimasankaram (1971)]. Remark 2.6.11. Consider matrices and vectors over C. Let (G? : b)? be a minimum norm g-inverse of (A : a). Then b? a = 1 if and only if b? A = 0. This can be shown as follows: ? Since (G? : b)? is (A : a)− m , we have Ga = A b. Therefore, b? a = 1 ⇔ AGa = 0 ⇔ AA? b = 0 ⇔ A? b = 0. Thus, X = (G? : b)? can never be a minimum norm g-inverse or the MoorePenrose inverse (left out in Theorem 2.6.10) in the set up considered in Theorem 2.6.10. Remark 2.6.12. Notice that Gt is a g-inverse of At if G is a g-inverse of A. Further, G? is a minimum norm g-inverse of A? if G is a least squares g-inverse of A. With this observation one can deal with the case of g-inverse of A vs (At : b)t or (A? : b)? (in the case of vectors and matrices over C). Au We now consider g-inverse of A vs. ; where u and v are column vt a vectors and a is scalar. Suppose a particular type of g-inverse G of A is
Matrix Decompositions and Generalized Inverses
51
a available to us and we want to compute a g-inverse of the same type of Au B= . Using the formulae so far developed, we can achieve this in vt a two stages. First we compute a g-inverse H of T = (A : u). The using H, the formulae derived so far and the Remark 2.6.12, we compute a g-inverse of T Au ·· , where w = (vt : a) . B= = vt a w We givebelow another interesting way of obtaining g-inverse of M = A B in a single stage. C D A B Let A be non-singular. Then the matrix M = is non-singular C D if and only if F = D − CA−1 B is non-singular. Also, −1 −1 A B A 0 F−1 (CA−1 : −I) M−1 = + 0 0 −I −1 A + A−1 BF−1 CA−1 − A− BF−1 . = −1 −1 −1 −F CA F Remark 2.6.13. The matrix F = D − CA−1 B is known as the Schur complement of A in M. We shall now explore availability of the above form for g- inverse of the A B a partitioned matrix when a g-inverse G of A is available. C D Theorem 2.6.14. Let a g-inverse G of A be available. Let A B M= and F = D − CGB. Then C D G 0 GB R= + F− (CG : −I) 0 0 −I is a g-inverse of M if and only if C(C(I − GA)) ⊆ C(F) C((I − AG)B)t ⊆ C(Ft ) and −
(I − AG)BF C(I − GA) = 0
(2.6.2)
52
Matrix Partial Orders, Shorted Operators and Applications
or equivalently ρ(M) = ρ(A) + ρ(F).
(2.6.3)
Proof. Proof of “R is M − if and only if (2.6.2) holds” follows easily from the verification of the condition MRM = M. To prove (2.6.2) and (2.6.3) are equivalent, we proceed as follows. Write I −GAGB I 0 , P= , Q= −CG I 0 I − I − (I − AG)BF and S= 0 I I 0 . T= − −F − FF C(I − GA) I Notice that P, Q, S and T are all non-singular matrices and straightforward computation shows that SPMQT = diag(A, F) + Z, where −(I − AG)BF− C(I − GA) (I − AG)B(I − F− F) . Z= − (I − FF )C(I − GA) 0 It is easy to check that ρ(M) = ρ(SPMQT) = ρ(diag(A, F) + Z) = ρ(diag(A, F)) + ρ(Z) = ρ(A) + ρ(F) + ρ(Z). Hence, ρ(M) = ρ(A) + ρ(F) if and only if Z = 0, that is, if and only if (2.6.2) holds. Several remarks are in order. Remark 2.6.15. The matrix F = D − CGB where G is a g-inverse of A is called a Schur complement of A in M. Schur complements play an important role in the study of shorted operators. We shall study them in detail in Chapter 10. Remark 2.6.16. If G is reflexive g-inverse of A, and F− is a reflexive g-inverse of F, then M is always a g-inverse of R. Remark 2.6.17. If C(Ct ) ⊆ C(At ) and C(B) ⊆ C(A), then (i) F is invariant under choices of g-inverse of A and (ii) R is a g-inverse of M. Remark 2.6.18. The following statement is equivalent to the statement of Remark 2.6.17:
Matrix Decompositions and Generalized Inverses
53
If M = (Xt1 , Xt2 )t (Y1 , Y2 ), where ρ(X1 Y1 ) = ρ(X1 ) = ρ(Y1 ) (Here A = X1 Y1 , B = X1 Y2 , C = X2 Y1 and D = X2 Y2 ) then (i) F is invariant under the choices of g-inverses of A and (ii) R is a g-inverse of M. Theorem 2.6.19. Let M be an nnd matrix over C. Then R is a g-inverse of M. Remark 2.6.20. Let M be as in Theorem 2.6.19. − (i) Let G be an A− ` and F a least squares g-inverse of F. Then R is an − M` if and only if C(CG) ⊆ C(F). − (ii) Let G be an A− m and F a minimum norm g-inverse of F. Then R is t t an M− m if and only if C((GB) ) ⊆ C(F ). † † (iii) Let G be A and F be the Moore-Penrose inverse of F. Then R is M† if and only if C(CA† ) ⊆ C(F) and C((A† B)† ) ⊆ C(Ft ).
Obtaining a g-inverse of M in the most general case is a bit more complicated. A B Theorem 2.6.21. Let M = . For a matrix W, let us write C D Ew = I−WW− and Hw = I−W− W, where W− is some g-inverse of W. Let F = D − CA− B, L = EA B, K = CHA and N = HL (EK FHL )− EK . Then − A − A− BL− EA − HA K− CA− − HA K− FL− EA HA K− U= − L EA 0 HA K− F + A− B N(FL− EA + CA− − I) + −I is a g-inverse of M. Proof.
Check MUM = M.
We shall now consider the third type of modification. Notice that a matrix is of rank 1 if and only if it is of the form abt for some non-null vectors a and b. Theorem 2.6.22. Let A be an m × n matrix and A− be a g-inverse of A. Let a be an m-column vector and b an n-column vector. Let us write
54
Matrix Partial Orders, Shorted Operators and Applications
E = I − AA− , H = I − A− A and β = 1 + bt A− a. If a 6∈ C(A), let c be a column vector as per Lemma 2.6.1 so that ct A = 0 and ct a = 1. If b 6∈ C(At ), let d be a column vector such that dt b = 1 and Ad = 0. Then the following hold: (i) A− −A− act −dbt A− +βdct is a g-inverse of A+abt , when a 6∈ C(A) and b 6∈ C(At ). (ii) A− − β −1 A− abt A− is a g-inverse of A + abt , when β 6= 0, a ∈ C(A) or b ∈ C(At ) or both. (iii) A− − dbt A− is a g-inverse of A + abt , when β = 0, a ∈ C(A) and b 6∈ C(At ). (iv) A− − A− act is a g-inverse of A + abt , when β = 0, a 6∈ C(A) and b ∈ C(At ). (v) A− is a g-inverse of A + abt , when β = 0, a ∈ C(A) and b ∈ C(At ). Proof is straightforward. We shall now give an expression for (A+ab? )† in terms of A† in various cases. Theorem 2.6.23. Let A be an m×n matrix over C and let A† be available. Let a be an m-column vector and b an n-column vector. Write k = A† a, h = A†? b, E = I − AA† , F = I − A† A and β = 1 + b? A† a. Let u = Ea, v = F? b. Then the following hold: (i) A† − ku† − v?† h? + βv?† u† is (A + ab? )† , when a 6∈ C(A) and b 6∈ C(A? ). (ii) A† − kk† A† − v?† h? is (A + ab? )† , when β = 0, a ∈ C(A) and b 6∈ C(A? ). (iii) A† + (1/β ? )vk? A† − (β ? /σ1 )p1 q?1 is (A + ab? )† , where σ1 = kkk2 kvk2 + β 2 , p1 = ((kkk2 /β ? )v + k), and q1 = ((kβfvk/β ? ) A†? k + h) , when β 6= 0 , a ∈ C(A). (iv) A† + A† h†? h? − ku† is (A + ab? )† , when β = 0, a 6∈ C(A) and b ∈ C(A? ). (v) A† + (1/β ? )A† hu? − (β ? /σ2 )p2 q2 ? is (A + ab? )† , where p2 = (1/b? )kuk2 A† h + k, q2 = (1/β ? )khk2 u + h and σ2 = khk2 kuk2 + β 2 , when β 6= 0 and b ∈ C(A? ). (vi) A† − kk† A† − A† h†? h + k† A† h†? kh? is (A + ab? )† , when β = 0, a ∈ C(A) and b ∈ C(A? ). Proof.
See [Campbell and Meyer (1991)] for a proof.
Matrix Decompositions and Generalized Inverses
55
Group inverse of a modified matrix will be dealt with in detail in Chapter 13.
2.7
Simultaneous diagonalization
When we study matrix partial orders, parallel sums, and shorted operators in the later chapters of this monograph, we generally deal with a pair of matrices. Sometimes we study the common features of several matrices together as in Fisher-Cochran type theorems. Studying such relationships or common features is easy if the matrices involved are diagonal. In this section, we put together some results on simultaneous diagonalization of matrices by using (a) non-singular transformations, (b) contra-gradient transformations (c) unitary transformations and (d) similarity transformations. We also study the simultaneous diagonalization of a pair of (possibly) rectangular matrices using unitary transformations and simultaneous singular value decomposition of several (but finite in number) rectangular matrices. We then study the generalized singular value decomposition of a pair of matrices. Using the results mentioned above, we define generalized eigen-values and generalized singular values, which play a role in many applications. In this section, we work with matrices and vectors over the field of complex number C. Theorem 2.7.1. Let A1 , . . . Ak be hermitian matrices of the same order. Then there exists a unitary matrix U such that U? Aj U is diagonal for j = 1, . . . , k if and only if Ai and Aj commute for i, j = 1, . . . , k (or equivalently Ai Aj is hermitian for i, j = 1, . . . , k). Theorem 2.7.2. (a) Let A1 and A2 be hermitian matrices of the same order and let A1 be positive definite. Then there exists a non-singular matrix P such that P? A1 P = I and P? A2 P is a real diagonal matrix. (b) Let A1 , A2 be hermitian matrices of the same order, A1 be non-negative definite and C(A2 ) ⊆ C(A1 ). Then there exists a non-singular matrix P such that P? A1 P = diag(Ir , 0) and P? A2 P = diag(D, 0); where r = ρ(A1 ) and D is a real diagonal matrix of order r × r. Corollary 2.7.3. Let A1 and A2 be nnd matrices of the same order. Then there exists a non-singular matrix P such that P? A1 P and P? A2 P are diagonal with non-negative diagonal elements.
56
Matrix Partial Orders, Shorted Operators and Applications
For proofs of Theorems 2.7.1, 2.7.2 and Corollary 2.7.3 see [Rao and Bhimasankaram (2000)]. Theorem 2.7.4. Let A1 and A2 be hermitian matrices of the same order. Then there exists a non-singular matrix T such that the matrices T? A1 T and T−1 A2 (T−1 )? are both diagonal if and only if A1 A2 is semi- simple with real eigen-values and ρ(A1 A2 ) = ρ(A1 A2 A1 ). Corollary 2.7.5. Consider the same setup as in Theorem 2.7.4. Further, let A1 be nnd. Then there exists a non-singular matrix T such that matrices T? A1 T and T−1 A2 (T−1 )? are both diagonal if and only if ρ(A1 A2 A1 ) = ρ(A1 A2 ). To prove Corollary 2.7.5, observe that A1 is nnd ⇒ A1 = QQ? for some matrix Q. Now, the non-null eigen-values of A1 A2 are the same as those of Q? A2 Q including the algebraic and geometric multiplicities. Further, since Q? A2 Q is hermitian, it is also semi-simple and has real eigen-values. If ρ(A1 A2 ) = ρ(A1 A2 A1 ), then ρ(A1 A2 ) = ρ(A1 A2 A1 ) = ρ(A1 A2 A1 A2 ) and so, the algebraic and geometric multiplicities of zero eigen-value of A1 A2 are equal. Hence, A1 A2 is semi-simple with real eigen-values. Now, the result follows by Theorem 2.7.4. Corollary 2.7.6. Let A1 and A2 be nnd matrices of the same order. Then there exists a non-singular matrix T such that the matrices T? A1 T and T−1 A2 (T−1 )? are both diagonal. Theorem 2.7.7. Let A1 , . . . , Ak be hermitian matrices of the same order and let A1 be non-singular. Then there exists a non-singular matrix P such that the matrix P? Ai P is diagonal for each i = 1, . . . , k if and only if (i) Ai A−1 1 is semi-simple with real eigen-values for i = 1, . . . , k and (ii) Ai A−1 A = Aj A−1 j 1 1 Ai for i, j = 1, . . . , k. Corollary 2.7.8. Consider the same set up as in Theorem 2.7.7 and let A1 be positive definite. Then there exists a non-singular matrix P such that the matrix P? Ai P is diagonal for each i = 1, . . . , k if and only if −1 Ai A−1 1 Aj = Aj A1 Ai for i, j = 1, . . . , k. Corollary 2.7.9. Let A1 , . . . , Ak be hermitian matrices of the same order such that B = A1 +. . .+Ak is nnd. Also, let C(Ai ) ⊆ C(B) for i = 1, . . . , k. Then there exists a non-singular matrix P such that the matrix P? Ai P is diagonal for each i = 1, . . . , k if and only if Ai B− Aj = Aj B− Ai for i, j = 1, . . . k, where B− is any g-inverse of B.
Matrix Decompositions and Generalized Inverses
57
Corollary 2.7.10. Let A1 , . . . , Ak be hermitian matrices of the same order such that C(Ai ) ⊆ C(A1 ) for i = 1, . . . , k. Then there exists a nonsingular matrix P such that P? Ai P is diagonal for i = 1, . . . , k if and − only if (i) Ai A− 1 is semi-simple with real eigen-values and (ii) Ai A1 Aj = − − Aj A1 Ai for each i, j = 1, . . . , k, where A1 is any g-inverse of A1 . For a proof of Theorem 2.7.7, see [Bhimasankaram (1971a)]. Corollaries 2.7.8-2.7.10 easily follow from Theorem 2.7.7. Theorem 2.7.11. (Similarity Transformation) Let A1 , . . . , Ak be semisimple matrices of the same order. Then the following are equivalent: (i) A1 , . . . , Ak commute pair-wise (ii) A1 , . . . , Ak can be expressed as polynomials in a common semi-simple matrix B and (iii) There exists a non-singular matrix S such that S−1 Ai S is diagonal for i = 1, . . . , k. For a proof see [Bhimasankaram (1971a)] and [Ben-Israel and Greville (2001)]. We now study the simultaneous diagonalization of (possibly) rectangular matrices. Theorem 2.7.12. Let A1 , A2 be m × n matrices. Then there exist unitary matrices U and V of order m × m and n × n respectively such that U? Ai V is diagonal with real diagonal elements, i = 1, 2 if and only if A1 A?2 and A?1 A2 are hermitian. Proof. ‘Only if’ part is trivial. ‘If’ part From Theorem 2.2.45, there exist unitary matrices P and Q such that P? A1 Q = diag(∆ , 0), where ∆ is a positive definite diagonal matrix. B C Write P? A2 Q = , where the partitioning is conformable for adD E dition of P? A1 Q and P? A2 Q. Since A1 A?2 is hermitian, so is P? A1 A?2 P.
58
Matrix Partial Orders, Shorted Operators and Applications
Now, P? A1 A?2 P = P? A1 QQ? A?2 P ? ∆ 0 B D? = 0 0 C? E? ∆B? ∆D? = . 0 0 Thus, ∆B? is hermitian and D = 0. Let ∆ = diag(δ1 , . . . , δr ) and B = [bij ]. ∆B? = B∆ ⇒ δi b?ji = bij δj for all i and j. Similarly, ‘A?1 A2 is hermitian’ implies that δi bij = δj b?ji for all i and j and and C = 0. So, bij = b?ji for all i and j, and therefore, B is hermitian. Now, P? A2 Q = diag(B, E), where B is hermitian and B and ∆ commute. So, by Theorem 2.7.1, there exists a unitary matrix R such that R? ∆R = ∆, R? BR = Λ where Λ is a real diagonal matrix. Let E = XΓY? be a singular value decomposition of E where X and Y are unitary and Γ is diagonal matrix with non-negative diagonal elements. Write U = Pdiag(R, X) and V = Qdiag(R, Y). Clearly, U and V are unitary, U? A1 V = diag(R? , X? ) P? A1 Q diag(R, Y) = diag(∆, 0) and U? A2 V = diag(R? , X? )P? A2 Qdiag(R, Y) = diag(R? , X? )diag(B, E)diag(R, Y) = diag(Λ, Γ). Thus, U? A1 V and U? A2 V are both diagonal and U, V are unitary.
Remark 2.7.13. In the proof of Theorem 2.7.12 we have b?ji = bij (δi /δj ) = bij . Thus, δi = δj whenever bij 6= 0. Notice that all δ’s are positive. So, it is easy to see that there exists a permutation matrix T such that T? ∆T = diag(δ1 I, . . . , δs I) and T? BT = diag(B1 , . . . , Bs ) for some s where δi I and Bi are of the same order for i = 1, . . . , s and the graph of the incidence matrix corresponding to Bi is connected for i = 1, . . . , s. We now prove a theorem which enables us to diagonalize several matrices simultaneously. Before we do so, we need the following lemmas: Lemma 2.7.14. Let at = (a1 , . . . , ak ) and bt = (b1 , . . . , bk ) be complex vectors such that for i = 1, . . . , k max{|ai |, |bi |} 6= 0. Then there exists a real scalar θ such that the vector c = a + θb has no null components i.e. ci 6= 0 for all i. If vectors a and b are real, then c is also real.
Matrix Decompositions and Generalized Inverses
Proof.
59
Let α = max{|ai |}. If b = 0, take θ = 0. So c = a has no null i
components. Now let b 6= 0. Let bi1 , . . . , bir be the non-null components of α+1 b. Write β = min |bij |. Let θ = . Consider i such that 1 ≤ i ≤ k. Let j β bi = 0. Then ci = ai + θbi = ai 6= 0. Hence, ci 6= 0. Now, let bi 6= 0. Now, α+1 α+1 ci = ai + θbi = 0 ⇒ ai = −θbi = − bi . So, |ai | = |bi | ≥ α + 1, β β which is a contradiction, since |ai | ≤ α. Hence, ci 6= 0. Lemma 2.7.15. Let u1 , u2 , . . . , uk be any finite number of complex vectors such that max{|uij |} 6= 0 for all j, where uij is the j th component of ui . i
Then there exist real scalars θ1 , θ2 , . . . , θk such that no component of the vector v = θ1 u1 + θ2 u2 + . . . , θk uk is null. Proof. Apply Lemma 2.7.14 recursively, noting that there is real linear combination of vectors u1 , u2 no component of which is null. Theorem 2.7.16. Let A1 , . . . Ak be a finite number of matrices of the same order m × n. Then there exist unitary matrices U and V of order m×m and n×n respectively such that U? Ai V is diagonal with real diagonal elements if and only if A?i Aj and Ai A?j are hermitian for i, j = 1, . . . , k. Proof. ‘Only if’ part is trivial. ‘If’ part By Theorem 2.7.12, the result is true for k = 2. Let the result be true for k = s. So, there exist unitary matrices P and Q such that Di = P? Ai Q is real and diagonal for i = 1, . . . , s. Without loss of generality we can assume Di = diag(∆i , 0), i = 1, . . . , s where ∆1 . . . , ∆s are of same order (i) (i) and max{|δj |} 6= 0 for all j (here δj is the j th diagonal element of ∆i ). i
By Lemma 2.7.15, there exists a real linear combination s X
µi ∆i
i=1
which is a non-singular matrix. Write ?
P As+1 Q = where B is of Since, A?i As+1 s X B ( µi Di ) D i=1
B C D E
the same order as ∆i (all ∆i are of same order). and Ai A?s+1 are hermitian for i = 1, . . . , s, we have ? s X C B C and ( µi Di )? are also hermitian. By E D E i=1
60
Matrix Partial Orders, Shorted Operators and Applications
Theorem 2.7.12, it follows that B is hermitian, C = 0 and D = 0. Since A?i Aj and Ai A?j are hermitian for i, j = 1, . . . , s + 1, it follows that ∆1 . . . , ∆s and B commute pair-wise and are hermitian. So, there exists a unitary matrix R such that R? ∆i R are diagonal with real diagonal elements i = 1, . . . , s. Let S and T be unitary matrices such that S? ET is diagonal (singular value decomposition of E). Write U = P diag(R, S) and V = Q diag(R, T). Clearly, U and V are unitary and U? Ai V is diagonal with real diagonal elements for i = 1, . . . , s + 1. The result now follows by induction on s. Definition 2.7.17. The matrices A1 , . . . , Ak of the same order are said to have simultaneous singular value decomposition if there exist unitary matrices U and V such that U? Ai V is a diagonal matrix with non-negative diagonal elements for i = 1, . . . , k. Theorem 2.7.18. (Simultaneous Singular Value Decomposition) (i) Let A1 and A2 have simultaneous singular value decomposition. Then A?1 A2 , A1 A?2 are nnd. (ii) Let A1 and A2 be matrices of same order such that A?1 A2 and A1 A?2 are hermitian and at least one of them is nnd. Then A1 and A2 have simultaneous singular value decomposition. Proof. (i) is trivial. (ii) Since A1 A?2 and A?1 A2 are hermitian by Theorem 2.7.12, there exist unitary matrices U and V such that Ai = U∆i V? where ∆i is a diagonal matrix with real diagonal elements δji , j = 1, . . ., min(m, n), i = 1, 2 where A and B are matrices of order m × n. Let A1 A?2 be nnd. Then ∆1 ∆2 is nnd. Clearly ∆2 ∆1 is also nnd. Thus, a diagonal element of ∆1 is negative if and only if the corresponding diagonal element of ∆2 is negative. If ith diagonal element of ∆1 is negative replace ui , the ith column of U by −ui and δ1i and δ2i by −δ1i and −δ2i . Repeat the same for all negative diagonal elements of ∆1 . Call the resultant matrices obtained from U, ∆1 and ∆2 as W, Λ1 and Λ2 . Then Ai = WΛi V? when W as V are unitary, Λi is diagonal nnd, i = 1, 2. Hence Ai = WΛi V? is a singular value decomposition for i = 1, 2. Theorem 2.7.19. (Simultaneous Singular Value Decomposition for Several Matrices) Let A1 , . . . , Ak be matrices of the same order. (i) If A1 , . . . , Ak have simultaneous singular value decomposition, then A?i Aj and Ai A?j are nnd for i, j = 1, . . . , k. (ii) Let A1 , . . . , Ak be matrices such that A?i Aj and Ai A?j are hermitian
Matrix Decompositions and Generalized Inverses
61
and at least one of them is nnd for i, j = 1, . . . , k. Then A1 , . . . , Ak have simultaneous singular value decomposition. Proof. We shall prove both (i) and (ii) by induction on k. The result is true for k = 2, by Theorem 2.7.18. Let it be true for k = s. Consider A1 , . . . , As+1 such that A?i Aj and Ai A?j are hermitian and at least one of them is nnd for i, j = 1, . . . , s + 1. By induction hypothesis, A1 , . . . , As have simultaneous singular value decomposition. So, there exist unitary matrices P and Q such that P? Ai Q = Di where Di is diagonal nnd for i = 1, . . . , s. Without loss of generality, we can take Di = diag(∆i , 0), Ps where each ∆i is nnd and i ∆i is positive definite. (This is achieved by permuting the rows and the same columns of each of ∆i - the same permutation transformation for each i - such that the null rows and columns are moved to be the last rows and last columns.) Ps Since, A?i As+1 and Ai A?s+1 are hermitian for each i, ( 1 A?i )As+1 and Ps ( 1 Ai )A?s+1 are also hermitian. Moreover, X X P? A?i Q = diag ∆i , 0 . P B C Let P As+1 Q = , where B has same order as ∆i , we notice F E that B is nnd, C = 0 and F = 0. Since A?i A?s+1 are hermitian, ∆i B is hermitian, ∆i , B commute for i = 1, . . . , s. So, by Theorem 2.7.1, there exists a unitary matrix R such that R? ∆i R for i = 1, . . . , s and R? BR are each diagonal with real non-negative diagonal elements. Let E = SΓT? be singular value decomposition of E. Now construct U and V as in Theorem 2.7.16 and check that U? Ai V is diagonal with real non-negative diagonal elements for i = 1, . . . , s + 1. ?
Theorem 2.7.20. (Generalized Singular Value Decomposition) Let A and B be matrices of orders m × n and p × n respectively. Let r = ρ(At : Bt )t and max{m, p} ≥ r. Then there exist unitary matrices U and V and a non-singular matrix Z such that A = U∆1 Z and B = V∆2 Z, where ∆1 and ∆2 are diagonal matrices with non-negative diagonal elements. Before we prove the Theorem 2.7.20, we shall prove a lemma that is also of independent interest. Lemma 2.7.21. Let C and D be n × m matrices such that C? C = D? D. Then there exists a unitary matrix T such that D = TC.
62
Matrix Partial Orders, Shorted Operators and Applications
Proof. Let C? C = D? D = Pdiag(∆2 , 0)P? be a spectral decomposition, where ∆2 is a positive definite diagonal matrix and P is unitary. Then the singular value decompositions of C and D are given by C = Qdiag(∆ , 0)V? and D = Rdiag(∆ , 0) V? where Q and R are unitary matrices. Let T = RQ? . Then T is unitary and D = TC. We now prove the Theorem 2.7.20. Proof. Consider A? A and B? B. By Corollary 2.7.3, there exists a nonsingular matrix Z such that A? A = Z? Γ1 Z and B? B = Z? Γ2 Z, where Γ1 and Γ2 are diagonal matrices with non-negative diagonal elements. Since ρ(A? A + B? B) = ρ(A? : B? )? = r, Γ1 +Γ2 has exactly r diagonal elements which are positive. Let Γ1 = ∆?1 ∆1 , Γ2 = ∆?2 ∆2 where ∆1 , ∆2 are of orders m × n and p × n respectively, with all diagonal elements nonnegative and all the off-diagonal elements zero. It is now clear that A? A = Z? ∆?1 ∆1 Z and B? B = Z? ∆?2 ∆2 Z. So, by Lemma 2.7.21, there exist unitary matrices U and V such that A = U∆1 Z and B = V∆2 Z. Remark 2.7.22. Suppose r ≤ max{m, p} in Theorem 2.7.20. Let m ≥ p. Then the matrix ∆?1 ∆1 + (∆2 ? : 0)(∆2 t : 0)t , has exactly r positive elements, where (∆2 t : 0)t is of order m × n. But this is not possible if m < r. The case where m < p can be disposed of similarly. That is why we require the condition max{m, p} ≥ r = ρ(At : Bt ). We shall now present, in brief, the generalized eigen-values and generalized singular values. Definition 2.7.23. We say a non-zero scalar λ is a generalized eigen-value of A with respect to B if there exists a non-null vector x such that Ax = λBx. Such a vector x is called a generalized eigen-vector of A with respect to B corresponding to the generalized eigen-value λ. Let A and B be hermitian matrices of the same order with B positive definite. Then by Theorem 2.7.2, there exists a non-singular matrix P such that P? BP = I and P? AP = Λ, a diagonal matrix. So, AP = (P? )−1 Λ = BPΛ. Let Pi be the ith column of P. Then APi = ((P? )−1 Λ)i = λi BPi , where λi ’s are diagonal elements of Λ. Thus, all the non-null diagonal elements of Λ are the generalized eigen-values of A with respect to B and the columns Pi of P are generalized eigen-vectors. Notice that λi are the non-zero eigen values of B−1 A (and are same as non-null eigen-values of AB−1 ).
Matrix Decompositions and Generalized Inverses
63
Definition 2.7.24. Consider the generalized singular value decomposition as in Theorem 2.7.20. Consider the common non-null diagonal elements of δ1ij ∆1 and ∆2 . Let them be δ1i1 , . . . , δ1is and δ2i1 , . . . , δ2is . The scalars , δ2ij j = 1, . . . , s are called the generalized singular values of A with respect to B. − ? Notice that Z−1 ∆− 2 V is a g-inverse of B where ∆2 is any g-inverse of ∆2 . Now, construct the particular g-inverse G of ∆2 as follows: gij = 0, when1 if δ2ii 6= 0 and 0 otherwise. Let B− = Z−1 GV? . ever i 6= j and gii = δ2ii Then U∆1 GV? is a singular value decomposition of AB− , where ∆1 G is a diagonal nnd matrix. Further, the non-null diagonal elements of ∆1 G are precisely the generalized singular values of A with respect to B. This shows the similarity between the concepts of generalized eigenvalues and generalized singular values. For more details on generalized eigen-values, see [Rao and Mitra (1971)]. For more details on generalized singular values, see [Golub and Van Loan (1996)].
64
2.8
Matrix Partial Orders, Shorted Operators and Applications
Exercises
All matrices in the following exercises are over C, the field of complex numbers. (1) Let A be an idempotent matrix. Prove that ρ(A) = tr(A). However the converse is not true even if A is semi-simple. (2) Let E be any idempotent matrix. Show that E† E? = E? E† = E† . (3) Let A be a square matrix of index 1 such that A? = A† . Then prove that (i) A# = AA? A# = A# A? A and (ii) (A2 A? )† = A# AA? and (A? A2 )† = A? AA# A? A = A# AA? . (iii) Is it necessary that A# is a partial isometry? (4) Let A and B be matrices of the same order such that A ia rangehermitian and B is hermitian. Show that AB† A = A implies A is hermitian. (5) Let P and Q be two orthogonal projectors of order n × n. Then prove the following statements: (i) PQ is similar to a diagonal matrix. (ii) PQ is an orthogonal projector if and only if PQ is range hermitian. (iii) n − ρ(I − PQ) = d(C(P) ∩ C(Q)). (6) Let A be an n × n matrix. Then prove the following: (i) (ii) (iii) (iv)
A is range-hermitian ⇔ A† = A# . A is range-hermitian ⇔ AA† = A† A. A is of index ≤ 1 ⇔ C(A) and N (A) are complementary. Every range-hermitian matrix is of index 1.
(7) Let A be a partial isometry of index 1. Should A# be a partial isometry? (8) Prove or disprove: A#† = A†# . (9) Let A be an n × m matrix. Prove that A is unitarily similar to the maΣK ΣL , where KK? +LL? = Ir , Σ = diag(σ1 Ir1 , . . . , σt Irt ); trix 0 0 r1 + . . . + rt = r = ρ(A) and σ1 > . . . σt > 0. (Hint: Use Singular value decomposition.) (10) Prove or disprove: (i) A semi-simple matrix has index not greater than 1 and (ii) A tripotent matrix is semi-simple.
Matrix Decompositions and Generalized Inverses
65
(11) Let A be an m × n matrix. Then show that A is partial isometry ⇔ AA? is an orthogonal projector ⇔ A? A is an orthogonal projector. (12) Let an m × n matrix A be a partial isometry. Then prove that A is normal ⇔ A is range hermitian. (13) Prove or disprove: for a square matrix A, both the matrices A and A2 are partial isometries implies A is normal. (14) Show that an m × n matrix A is a partial isometry of rank r ⇔ A has exactly r singular values each equal to 1 and others are null. (15) Show that a hermitian partial isometry is a tripotent matrix. Show 1 0 0 the converse statement is not true. (Hint: Take A = 0 −1 1 .) 0 0 0 (16) Show that an n×n matrix A is range-hermitian if and only if A] = A† if and only if A† is a polynomial in A. All matrices in the following exercises are over F, an arbitrary field. (17) Let A and C be any two matrices with A of index 1. Then prove the following: (i) A# C = 0 ⇔ AC = 0 and (ii) CA# = 0 ⇔ CA = 0. (18) Let A be a tripotent matrix. Then prove the following: A2 is idempotent. −A is tripotent. The only eigen-values of A are −1, 0 and 1. ρ(A) = tr(A2 ). A is its own g-inverse. Conversely, if B is its own inverse then B is tripotent. (vi) For a tripotent matrix A, tr(A2 + A) = twice the number of positive eigen-values, tr(A2 − A) = twice the number of negative eigenvalues and tr(I − A2 ) = the number of null eigen-values of A.
(i) (ii) (iii) (iv) (v)
(19) Prove that A is of index not greater than 1 and A2 is idempotent ⇔ A is tripotent. (20) Let A, B be square matrices of the same order. Prove that AB(AB)− A = A if and only if ρ(AB) = ρ(A) and B(AB)− (AB) = B if and only if ρ(AB) = ρ(B). (21) Let A be a square matrix and (F, G) be a rank factorization of A. Then prove that A has a reflexive g-inverse X with C(X) = C(A) and N (X) = N (A) if and only if GF is invertible and if this is so, X = F(GF)−2 G = A] .
Chapter 3
The Minus Order
3.1
Introduction
After the first two introductory chapters, we are finally ready to explore the beautiful world of matrix partial orders. Historically these partial orders were defined through various generalized inverses. While it is true that the structure and mechanisms of these orders are more basic than the involvement of generalized inverse, we shall see that the use of generalized inverses makes the exposition elegant. Also, several properties can be expressed in a pleasant way in the process. In Section 3.2, we begin with the study of one of the most basic matrix order relations, namely, the space pre-order. This is the stepping stone for almost all the partial orders that we shall study subsequently and therefore, we give a detailed account of its properties. This pre-order can be extended to a partial order with minor additions to its definition as we shall see in Section 3.3. Since this extension does not lend to any further properties than the space pre-order itself, we leave it at that and define a very important and the most fundamental partial order (that implies space pre-order and also the extension) called the minus order. We call it fundamental because the partial orders studied in the three subsequent chapters are built on this partial order by putting additional restrictions. We obtain several of its characterizations in Section 3.3. In Section 3.4, given a matrix A, we obtain the class of all matrices B such that A is below B under the minus order (written as A <− B). Assuming that B is given, we also obtain the class of all matrices A such that A is below B under the minus order. In Section 3.5, assuming that A <− B, we characterize the class of all g-inverses of A such that A− A = A− B and AA− = BA− . One of the characterising properties for A to be below B under minus order is A = PB = BQ for some projectors P and Q. We also characterize the 67
68
Matrix Partial Orders, Shorted Operators and Applications
class of all projectors P and Q such that A = PB = BQ when A <− B. Section 3.6 is devoted to the study of special properties of the minus order for idempotent matrices. In Section 3.7, we present some properties of the minus order for hermitian and nnd (complex) matrices. We emphasize that all the matrices in this chapter are (a) possibly rectangular and (b) over an arbitrary field unless stated otherwise. 3.2
Space pre-order
A pre-order is a reflexive and transitive relation (see Appendix A). The space pre-order, which we define shortly, is the big daddy of (the most basic among) all the order relations that we study in this and subsequent chapters. We show that the space pre-order, as its name suggests, is a preorder. We also obtain some characterizations of this pre-order which will be useful in what follows. Definition 3.2.1. Let A and B be matrices (possibly rectangular) having the same order. Then A is said to be below B under the space pre-order, if C(A) ⊆ C(B) and C(At ) ⊆ C(Bt ). We denote the space pre-order by ‘<s ’ and write A <s B, whenever A is below B under ‘<s ’. It is easy to check that <s is reflexive and transitive. Hence, it is a pre-order. However, it is not a partial order, for, if A and B are distinct non-singular matrices of the same order n × n, then C(A) = C(B) = C(At ) = C(Bt ) = Fn . More specifically, take A=
1 0 0 1
and B =
1 1 . 0 1
Thus A <s B, B <s A, but A 6= B, showing ‘<s ’ is not anti-symmetric. We give some more examples of the space pre-order. Example 3.2.2. If A = diag(a1 , . . . , an ) and B = diag(b1 , . . . , bn ), then A <s B if and only if bi 6= 0 whenever ai 6= 0. Example 3.2.3. Let A and B be nnd matrices of the same order (over the field of complex numbers). Then A <s A + B. Remark 3.2.4. Let A and B be matrices of the same order. In view of Remark 2.3.4, we have A <s B ⇔ A = BB− A = AB− B = BB− AB− B
The Minus Order
69
for all B− . So, A <s B ⇔ A = BMB for some matrix M. Thus, ρ(A) ≤ ρ(B) if A <s B. Remark 3.2.5. Let A and B be matrices of the same order. Let P and Q be non-singular matrices of appropriate orders so that the products PAQ and PBQ are defined. Then it is obvious that C(A) ⊆ C(B) ⇔ C(PAQ) = C(PA) ⊆ C(PB) = C(PBQ). Similarly, C(At ) ⊆ C(Bt ) ⇔ C((PBQ)t ) ⊆ C((PBQ)t ). Thus, A <s B ⇔ PAQ <s PBQ, for all non-singular matrices P and Q of suitable orders. In other words, the space pre-order is invariant under equivalent transformations. The following theorem gives some useful characterizations of the space pre-order. Theorem 3.2.6. Let A and B be matrices of the same order. The following are equivalent: (i) A <s B, (ii) Let (P, Q) be a rank factorization of B. Then A = PTQ for some matrix T and (iii) Let (L, M) be a rank factorization of A. If ρ(A) = ρ(B), then B = LRM for some non-singular matrix R, and if ρ(A) < ρ(B), then there exist matrices E andF and the a non-singular matrix R such that M M matrices (L : E) and ·· have full rank and B = (L : E)R ·· . F F Proof. (i) ⇒ (ii) Let (L, M) be a rank factorization of A. Since A <s B, C(L) = C(A) and C(P) = C(B), we have C(L) ⊆ C(P). So, L = PT1 for some matrix T1 . Similarly, C(Mt ) ⊆ C(Qt ) implies Mt = Qt T2 for some matrix T2 . Therefore, A = LM = PT1 Tt2 Q = PTQ, where T = T1 Tt2 . (ii) ⇒ (i) Since A = PTQ, C(A) ⊆ C(P) = C(B) and C(At ) ⊆ C(Qt ) = C(Bt ). (i)⇒(iii) Let ρ(A) = ρ(B). As in the proof of (i) ⇒ (ii), it is easy to see that there exists a matrix R such that B = LRM. Let ρ(A) = a. Then R is an a × a matrix. Further, since L has a left inverse and M has a right inverse, we have a = ρ(A) = ρ(B) = ρ(LRM) = ρ(R). Hence, R is non-singular.
70
Matrix Partial Orders, Shorted Operators and Applications
Now, let ρ(B) > ρ(A). Since C(A) ⊆ C(B), there exists a full column rank matrix E such that columns of (L : E) form a basis of C(B). Similarly we M can find a full row rank matrix F such that the rows of ·· form a basis F of the row space of B. Let (G, H) be a rank factorization of B. Then G = (L : E)R1 , where R1 is non-singular. Similarly,
M H = R2 ·· , F where R2 is non-singular. Hence M B = GH = (L : E)R ·· , F where R = R1 R2 is non-singular. (iii) ⇒ (i) If ρ(A) = ρ(B), the proof is trivial. So, let ρ(B) > ρ(A). Let E and F be as in the proof of (i) ⇒ (iii). Notice that C(A) = C(L) ⊆ C(L : E) = C(B) and C(At ) = C(Mt ) ⊆ C((Mt : Ft )) = C(Bt ). Therefore, A <s B. The characterizations of the space pre-order in Theorem 3.2.6 enable us to obtain the class of matrices, which are below a given matrix B under ‘<s ’ (using (ii)) and the class of all matrices which are above a given matrix A under <s (using (iii)). Remark 3.2.7. In Theorem 3.2.6, the statements (ii) and (iii) can be equivalently written as (ii)0 Let B = Pdiag(I, 0)Q; where P and Q are non-singular. Then A = Pdiag(T, 0) Q for some matrix T and (iii)0 Let A = Sdiag(Ia , 0b−a , 0)W where S and W are non-singular. If ρ(B) = b ≥ a, then B = Sdiag(K, 0)W, where K is a non-singular matrix of order b × b respectively. Remark 3.2.8. Notice that C(At ) ⊆ C(Bt ) if and only if N (B) ⊆ N (A). Also C(A) ⊆ C(B) if and only if N (Bt ) ⊆ N (At ). Hence A <s B, if and only if N (B) ⊆ N (A) and N (Bt ) ⊆ N (At ).
The Minus Order
71
Theorem 3.2.9. Let A and B be matrices having the same order. Then A <s B if and only if AB− A is invariant under the choices of B− . Proof. If A = 0, then result follows trivially. If A 6= 0, the result follows from Theorem 2.3.11.
Let us now specialize to the case of square matrices of index 1. The following theorem can be proved along the lines of Theorem 3.2.6. Theorem 3.2.10. Let A and B be square matrices of order n × n with ρ(A) = a and ρ(B) = b. Let each of A and B have index not greater than 1. Then the following are equivalent: (i) A <s B, (ii) Let B = Pdiag(Db , 0)P−1 , where Db is a b × b non-singular matrix and P is a non-singular matrix. Then A = Pdiag(T, 0) P−1 , where T is a b × b matrix of index not greater than 1 and (iii) There exist non-singular matrices Q, D1 and D2 of orders a × a and b × b such that A = Qdiag(D1 , 0) Q−1 , and B = Qdiag(D2 , 0) Q−1 . Remark 3.2.11. Let A and B be square matrices of index ≤ 1 such that A <s B. Let A = Qdiag(Da , 0) Q−1 , where Da is non-singular. Then it is not necessarily true that B = Qdiag(E, 0) Q−1 , where E is non-singular. However, if A <s B, there exist nonsingular matrices Q, D and E of appropriate orders such that A = Q diag(D, 0)Q−1 and B = Q diag(E, 0)Q−1 . We now consider matrices over the field of complex numbers C and give one more characterization of the space pre-order using singular value decomposition. Theorem 3.2.12. Let A and B be matrices in Cm×n . The following are equivalent: (i) A <s B, (ii) Let B = Pdiag(∆, 0)Q∗ be a singular value decomposition of B, where P and Q are unitary matrices of appropriate orders and ∆ is a positive definite diagonal matrix. Then A = Pdiag(T, 0)Q∗ for some matrix T of the same order as ∆ and (iii) There exist unitary matrices U and V, a positive definite diagonal matrix D and a nonsingular matrix S of order not less than that of D such that A = U diag(D, 0)V∗ is a singular value decomposition of A and B = U diag(S, 0)V∗ . Proof can be given along the lines of Theorem 3.2.6.
72
3.3
Matrix Partial Orders, Shorted Operators and Applications
Minus order - Some characterizations
In the previous section we studied an order relation which is a pre-order, namely, the space pre-order. We also noted that it is not anti-symmetric and hence is not a partial order. It is easy to see that the following minimal modification to the definition of the space pre-order yields a partial order. Define if ρ(A) = ρ(B) A = B s+ A < B if . A <s B if ρ(A) < ρ(B) It is clear that ‘<s+ ’ is a partial order. This is the weakest partial order that implies the space pre-order. Unfortunately this partial order does not exhibit any special properties not possessed by the space pre-order. We now introduce a partial order which implies the space pre-order and also possesses interesting properties like the ones mentioned below. Let us consider A = diag(Ia , 0, 0) and B = diag(Ia , Ic , 0). It is reasonable to say A precedes B or A is below B. The same status can be accorded to matrices PAQ and PBQ, where P and Q are non-singular. Moreover, it is easy to see that A <s B and B is obtained from A by simply adding C = diag(0, Ic , 0). Notice that the column spaces of A and C are virtually disjoint and so are their row spaces. Thus, B = A ⊕ C. Such a matrix A can be thought as an independent section of B or we may say that A precedes B in some sense. In the previous section we saw that A <s B if and only if AB− A is invariant under the choices of g-inverse B− of B. Further, if AB− A = A, then {B− }, the set of the generalized inverses of B is a subset of {A− }. Thus, once again we can say A precedes B in some sense. In this section, we show that all the above requirements lead to an identical ordering between matrices of same order. In this chapter we make an in-depth study of this order. Consider the mathematically elegant relationship defined using generalized inverses as follows: Definition 3.3.1. Let A and B be matrices of the same order. Then A is said to be below B under the minus order if there exist generalized inverses G1 and G2 of A such that AG1 = BG1 and G2 A = G2 B. When A is below B under the minus order, we write A <− B. We shall show that the relation in Definition 3.3.1 is identical with the various ways of A preceding B described in the second paragraph of this
The Minus Order
73
section. We shall also show that the minus order is indeed a partial order on F m×n . The minus order was first introduced by Hartwig on the set of regular elements of a semi-group. He developed the order relation as an extension of the standard or natural partial order on the set of idempotent elements of a semi-group and in particular, of elements of F m×n . This order also extends the natural partial order of Vagner on inverse semi-groups and the star order of Drazin (which is studied in detail in Chapter 5). We first show that whenever A <− B, there is no loss of generality in taking the g-inverses G1 and G2 of A to be the same in Definition 3.3.1. Theorem 3.3.2. Let A and B be matrices having the same order. Then A <− B if and only if there exists a g-inverse A− of A such that A− A = A− B and AA− = BA− . Proof. ‘If’ part is trivial. ‘Only if’ part Let G1 and G2 to be the g-inverses of A such that AG1 = BG1 and G2 A = G2 B. Let G = G1 AG2 . Then G is a g-inverse of A and satisfies AG = BG, GA = GB. Remark 3.3.3. (i) In view of Theorem 3.3.2, we may take G1 and G2 to be the same in Definition 3.3.1. Henceforth, we do so without making a special mention. (ii) In the proof of Theorem 3.3.2, notice that G = G1 AG2 is a reflexive g-inverse of A. Hence, it is clear that if A <− B, then there exists a reflexive − − − − g-inverse A− r of A such that AAr = BAr and Ar A = Ar B. Clearly, 0 <− B for any matrix B, where matrices 0 and B have the same order. Also, if B <− 0, then B = 0. Henceforth, we shall consider A and B to be non-null in our study of the minus order. Any m×n matrix is a representation of some linear transformation from F n to F m under specified bases for F n and F m . Thus, if P and Q are nonsingular matrices of suitable orders so that the matrix PAQ is defined, then A and PAQ represent the same linear transformation. In view of Remark 3.2.5, the space pre-order is also a pre-order on the linear transformations. Our next Theorem shows the same is true for the minus order. Theorem 3.3.4. Let A and B be matrices of the same order and let P and Q be non-singular matrices of appropriate orders so that the matrix PAQ is defined. Then A <− B if and only if PAQ <− PBQ.
74
Matrix Partial Orders, Shorted Operators and Applications
Proof. ‘Only if’ part Let A <− B. Then there exists a g-inverse A− of A such that A− A = A− B and AA− = BA− . In view of Remark 2.3.7, C = Q−1 A− P−1 is a g-inverse of PAQ. Moreover, CPAQ = CPBQ and PAQC = PBQC. Thus, we have PAQ <− PBQ. ‘If’ part follows from the proof of ‘only if’ part once we observe that A = P−1 (PAQ)Q−1 and B = P−1 (PBQ)Q−1 . We now establish the assertion made earlier about the equivalence of seemingly different relationships mentioned in the beginning of this section. Theorem 3.3.5. Let A and B be non-null matrices of the order m × n. Let ρ(A) = a and ρ(B) = b. Then the following are equivalent: A <− B A <s B and AA− = BA− for some g-inverse A− of A − − A <s B and AA− r = BAr for some reflexive g-inverse Ar of A − − − t AA = BA for some g-inverse A of A and C(A ) ⊂ C(Bt ) If (P, Q) is a rank factorization of B, then A = PTQ for some idempotent matrix T (vi) There exist non-singular matrices R and S of orders m×m and n×n respectively such that
(i) (ii) (iii) (iv) (v)
A = Rdiag(Ia , 0, 0)S and B = Rdiag(Ia , Ib−a , 0)S (vii) B = A ⊕ (B − A) and (viii) {B− } ⊆ {A− }. Proof. (i)⇒(ii) and (ii)⇔(iii)⇔(iv) are trivial. (iii)⇒(v) Let (P, Q) be a rank factorization of B. Since A <s B, by Theorem 3.2.6(ii), we have B = PQ and A = PTQ for some matrix T. Further, every reflexive −1 −1 − −1 g-inverse A− r of A is of the from QR Tr PL for some right inverse QR − − − of Q, for some left inverse P−1 L of P and for some Tr . Now AAr = BAr −1 − −1 −1 − −1 − for some reflexive g-inverse Ar , so, PTQQR Tr PL = PQQR Tr PL − − or TT− r = Tr . Clearly, T = Tr T is idempotent. (v) ⇒ (vi) Since T is idempotent with ρ(T) = a, there exists a non-singular matrix −1 R1 such that T = R1 diag(Ia , 0)R−1 1 . So B = PQ = PR1 R1 Q, and −1 −1 A = PR1 diag(Ia , 0)R1 Q. Clearly, ρ(PR1 ) = ρ(R1 Q) = b. Also, −1 (PR1 , R−1 1 Q) is a rank factorization of B. Hence, PR1 and R1 Q can be extended to non-singular matrices R and S respectively. It is easy to see
The Minus Order
75
that A = Rdiag(Ia , 0, 0)S and B = Rdiag(Ia , Ib−a , 0)S. Proof for (vi) ⇒ (vii) is trivial. (vii) ⇒ (viii) Clearly, C(A) ⊆ C(B) and C(At ) ⊆ C(Bt ). So, for each B− , A = AB− B = AB− A + AB− (B − A) or A − AB− A = AB− (B − A). However, C(A − AB− A)t ⊆ C(AB− (B − A))t and C(AB− (B − A))t ⊆ C(B − A)t . So, C(A − AB− A)t ∩ C(AB− (B − A))t ⊆ C(At ) ∩ C(B − A)t = {0}. Hence, A − AB− A = 0 or equivalently A = AB− A. Thus, B− ∈ {A− }. (viii) ⇒ (i) Since A = AB− A for all B− , we have A <s B. Let B− and A− be any g-inverses of B and A respectively. Write G1 = B− AA− and G2 = A− AB− . Then it is easy to verify that G1 and G2 are g-inverse of A and AG1 = BG1 , G2 A = G2 B. Corollary 3.3.6. If A and B are non-null matrices of the same order such that A <− B and ρ(A) = ρ(B), then A = B. Remark 3.3.7. Let A and B be non-null matrices of the same order such that A is a full rank matrix and A <− B, then A = B. So, A is a maximal element of F m×n under ‘<− ’. Remark 3.3.8. The condition {B− } ⊆ {A− } in (viii) of Theorem 3.3.5 − can be replaced by {B− r } ⊆ {A }. Remark 3.3.9. Statement (vii) of Theorem 3.3.5 is equivalent to saying ρ(B) = ρ(A) + ρ(B − A). Remark 3.3.10. Let A and B be non-null matrices of same order such that A <− B. Then A and B have simultaneous normal form. Remark 3.3.11. In Theorem 3.3.5, the statements (ii), (iii) and (iv) can equivalently be stated as (ii)0 A <s B and A− A = A− B for some g-inverse A− of A − − (iii)0 A <s B and A− r A = Ar B for some reflexive g-inverse Ar of A 0 − − − (iv) A A = A B for some g-inverse A of A and C(A) ⊂ C(B).
76
Matrix Partial Orders, Shorted Operators and Applications
Remark 3.3.12. In Theorem 3.3.5, we proved that A <s B together with AA− = BA− for some g-inverse A− of A is equivalent to A <− B. It is easy to see (in view of Remark 2.3.4) that A <s B together with A = AB− A for some B− is also equivalent to A <− B. Remark 3.3.13. Let T1 and T2 be linear transformations from F n to F m . Let ρ(T1 ) = a and ρ(T2 ) = b, b ≥ a. In view of (i) ⇔ (vi) of Theorem 3.3.5, T1 <− T2 if and only if there exists bases {u1 , . . . , un } and {v1 , . . . , vm } of F n and F m respectively such that ( vi 1 ≤ i ≤ a T1 ui = 0 otherwise and T2 ui =
( vi
1≤i≤b
0
otherwise
.
We now show that the minus order is indeed a partial order. Theorem 3.3.14. The minus order is a partial order on F m×n . Proof. Reflexivity holds trivially. In view of Theorem 3.3.5 (viii), transitivity also holds. To prove anti-symmetry, observe that by Remark 3.3.9, A <− B ⇒ ρ(B) = ρ(A) + ρ(B − A). So, ρ(B) ≥ ρ(A). Similarly, we have B <− A ⇒ ρ(A) ≥ ρ(B). Combining the two inequalities, we have ρ(B) = ρ(A). Therefore, ρ(B − A) = 0 or equivalently B − A = 0. Hence, B = A. We now give some more characterizing properties of the minus order. Notice that if A <− B, then there exists a g-inverse A− of A such that AA− = BA− and A− A = A− B. Hence A = AA− A = AA− B = BA− A. Thus, there are projectors P(= AA− ) and Q(= A− A) such that A = PB = BQ. It turns out that this is a characterizing property of the minus order. However, before we exhibit this, we prove the following: Lemma 3.3.15. Let A, B be matrices such that the product AB is defined. Then ρ(AB) = ρ(B) − d(C(B) ∩ N (A)). Proof. Let ρ(B) = r and d(C(B) ∩ N (A)) = s. Let C and D be matrices of order n×s and n×(r−s) respectively such that C(C) = C(B)∩N (A) and C(C : D) = C(B). Clearly, C and D are matrices of full column rank. Since C(C) = C(B) ∩ N (A), we have AC = 0 and so, C(AD) = C(AB). Also,
The Minus Order
77
C(C)∩C(D) = {0}. Further, if ADu = 0, then Du ∈ C(B)∩N (A) = C(C). Conversely, if Du ∈ C(B) ∩ N (A) = C(C), then Du = Cv for some v. Hence ADu = ACv = 0. Now, ADu = 0 ⇔ Du ∈ C(C). Hence, Du ∈ C(C) ∩ C(D) = {0}, so, Du = 0. Trivially, Du = 0 ⇒ ADu = 0. Thus, ρ(AD) = ρ(D). As C(AD) = C(AB), ρ(AB) = ρ(D) = r−s. Hence, ρ(AB) = ρ(B) − d(C(B) ∩ N (A)). (Alternative Proof ): Consider the matrix S = B − B(AB)− (AB). Clearly, for any g-inverse B− of B, SB− S = S, SB− B = S and BB− S = S, so, S <− B, by Remark 3.3.12. It is easy to see that C(S) ⊆ C(B) ∩ N (A). Further, let x ∈ C(B) ∩ N (A). Then x = Bz and Ax = 0. So, ABz = 0. Since S <− B, we have x = Bz = Sz + (B(AB)− (AB))z = Sz and so, x ∈ C(S). Hence, C(S) = C(B) ∩ N (A). Now, ρ(B) = ρ(S) + ρ(B(AB)− (AB)) = d(C(B) ∩ N (A)) + ρ(AB), since A(B(AB)− (AB)) = AB. Thus, ρ(AB) = ρ(B) − d(C(B) ∩ N (A)). Theorem 3.3.16. Let A and B be matrices of the same order. Then the following are equivalent: (i) (ii) (iii) (iv)
A <− B, There exist projectors P and Q such that A = PB = BQ, B − A <− B and ρ(B − A) = ρ((I − AA− )B) = ρ(B(I − A− A)) for any A− .
Proof. (i) ⇒ (ii) Since A <− B, there exists a g-inverse A− of A such that AA− = BA− and A− A = A− B. Choose P = AA− and Q = A− A. It is easy to check that P and Q are projectors. (ii) ⇒ (i) For any g-inverse B− of B we have, AB− A = PBB− BQ = PBQ = AQ = BQ2 = BQ = A. Thus, AB− A = A. This gives {B− } ⊆ {A− } and therefore, A <− B. The equivalence of (i) and (iii) follows from the equivalence of (i) and (vii) of Theorem 3.3.5. (iii) ⇔ (iv) Notice that ρ((I − AA)− )B = ρ((I − AA− )(B − A)) = ρ(B − A) − d(C(B − A) ∩ N (I − AA− )) = ρ(B − A) − d(C(B − A) ∩ C(A)) = ρ(B − A) ⇔ C(B − A) ∩ C(A) = {0}.
78
Matrix Partial Orders, Shorted Operators and Applications
Similarly, ρ(B(I − A− A)) = ρ(B − A) ⇔ C((B − A)t ) ∩ C(At ) = {0}. Let A and B be matrices of the same order such that A <− B. It is easy to see that there exist g-inverses A− of A and B− of B such that A− <− B− and also B− <− A− . In fact, we can make a stronger statement as shown below. Theorem 3.3.17. Let A and B be matrices of the same order such that A <− B. Let G be a g-inverse of B, ρ(A) = a, ρ(B) = b and ρ(G) = g. Then there exists non-singular matrices P and Q such that A = Pdiag(Ia , 0, 0, 0)Q, B = Pdiag(Ia , Ib−a , 0, 0)Q and G = Q−1 diag(Ia , Ib−a , Ig−b , 0)P−1 . Proof. By Theorem 3.3.5 ((i) ⇒ (vi)), there exist non-singular matrices R, S such that A = Rdiag(Ia , 0, 0, 0)S and B = Rdiag(Ia , Ib−a , 0, 0)S. Since, G is a g-inverse of B, we have
Ia 0 L1 G = S−1 0 Ib−a L2 R−1 M1 M2 N for some matrices L1 , L2 , M1 , M2 and N of appropriate orders. Write
Ia 0 0 Ia 0 L1 T = 0 Ib−a 0 and W = 0 Ib−a L2 . M1 M2 I 0 0 I Clearly, T and W are non-singular matrices. Moreover, we have G = S−1 Tdiag(Ia , Ib−a , N − M1 L1 − M2 L2 )WR−1 and ρ(N − M1 L1 − M2 L2 ) = g − b. Let X−1 diag(Ig−b , 0)Y−1 be a normal form of the matrix N−M1 L1 −M2 L2 . Write, P = RW−1 diag(Ib : Y) and Q = diag(Ib : X)T−1 S. Clearly, A = Pdiag(Ia , 0, 0, 0)Q, B = Pdiag(Ia , Ib−a , 0, 0)Q and G = Q−1 diag(Ia , Ib−a , Ig−b , 0)P−1 .
The Minus Order
79
Theorem 3.3.18. Let A and B be matrices of the same order such that A <− B. Let G and H respectively be g-inverses of B and A such that G <− H. Then H is a g-inverse of B. Proof. In view of Theorem 3.3.17, there exist non-singular matrices P and Q such that A = Pdiag(Ia , 0, 0, 0)Q, B = Pdiag(Ia , Ib−a , 0, 0)Q and G = Q−1 diag(Ia , Ib−a , Ig , 0)P−1 , where ρ(A) = a, ρ(B) = b and ρ(G) = g. Let V be a g-inverse of G such that GV = HV and VG = VH. Then V must be of the form Ia 0 0 L1 0 Ib−a 0 L2 Q V = P 0 0 Ib−a L3 M1 M2 M3 N for some matrices L1 , L2 , L3 , M1 , M2 , M3 and N of appropriate orders. Proceeding as in Theorem 3.3.17, we can find non-singular matrices R, S such that A = Rdiag(Ia , 0, 0, 0, 0)S, B = Rdiag(Ia , Ib−a , 0, 0, 0)S and G = S−1 diag(Ia , Ib−a , Ig−a , 0, 0)R−1 and V = S−1 diag(Ia , Ib−a , Ig−b , Iv−g , 0)R−1 , where v = ρ(V). Since H is a g-inverse of Ia D12 D13 D21 D22 D23 H = P D31 D32 D33 D41 D42 D43 D51 D52 D53
A, it must be of the form D14 D15 D24 D25 D34 D35 Q. D44 D45 D54 D55
Since, GV = HV and VG = VH, D22 = Ib−a , D33 = Ig−b , D44 = Iv−g and Dij = 0 for all i, j = 1, 2, 3, 4 with i 6= j. Thus, Ia 0 0 0 D15 0 Ib−a 0 0 D25 H = P D31 D32 Ig−b 0 D35 Q. 0 0 0 Iv−g D45 0 0 0 0 D55 It is now easy to check that H is a g-inverse of B.
We have seen in Theorem 3.3.5, that for any two matrices A1 and A2 , Ai <− (A1 + A2 ) if and only if A1 and A2 are disjoint matrices. We shall now extend this result for several matrices and their sum. We shall see in
80
Matrix Partial Orders, Shorted Operators and Applications
later chapters that this is a prelude to Fisher-Cochran type theorems on distribution of quadratic forms in normal variables. Theorem 3.3.19. Let A1 , A2 , . . . , Ak be matrices of the same order. Pk Write A = i=1 Ai . Then the following are equivalent. Pk (i) ρ(A) = i=1 ρ(Ai ) (ii) There exist nonsingular matrices P and Q such that Ai = Pdiag(0 . . . , I, 0 . . . , 0)Q, where diag(0, . . . , I, 0, . . . , 0) has at least k + 1 diagonal blocks and I occurs in the ith block for i = 1, . . . , k (iii) ρ(A − Ai ) = ρ(A) − ρ(Ai ), i = 1, . . . , k and (iv) Ai <− A, i = 1, . . . , k. Proof. (i) ⇒ (ii) Let (Pi , Qi ) be a rank factorization of Ai , for each i = 1, . . . , k. Since (i) holds, the columns of (P1 : . . . : Pk ) are linearly independent and so are the columns of (Qt : Qt2 : . . . : Qtk ). Extend (P1 : . . . : Pk ) and (Qt1 : . . . : Qtk ) to nonsingular matrices P and Qt respectively. Now it is easy to see that Ai = P diag(0, . . . , I, 0 . . . , 0)Q. (ii) ⇒ (iii) is trivial. (iii) ⇒ (iv) follows from Theorem 3.3.5. (iv) ⇒ (i) Since (iv) holds, every A− ∈ {A− } is a g-inverse of Ai for each i. Hence Ai A− is idempotent and ρ(Ai A− ) = ρ(Ai ) for all i. Now k k k X X X ρ(Ai ) = ρ(Ai A− ) = tr(Ai A− ) = tr
k X Ai
i=1
i=1
i=1
i=1
!
! A
−
= tr(AA− ) = ρ(A). Remark 3.3.20. It is interesting to note that Ai and A − Ai are disjoint for all i implies that C(A1 ) + C(A2 ) + . . . + C(Ak ) and C(At1 ) + . . . + C(Atk ) are actually direct sums. Remark 3.3.21. Consider A1 , . . . , Ak and A as in Theorem 3.3.19. If Ai <− A, i = 1, . . . , k, then Ai <− Aj1 +. . .+Ajr for any sub-permutation (j1 , . . . , jr ) of (1, . . . , k) such that i ∈ (j1 , . . . , jr ).
The Minus Order
3.4
81
Matrices above/below a given matrix under the minus order
Let one of the matrices A and B be given. In this section, we characterize the class of matrices for the other such that A <− B and obtain some of its formulations that are useful in different situations. However, before we actually do so, we show that every complement of C(A) is C(I − AG) for some g-inverse G of A. Lemma 3.4.1. Let A be a matrix of order m × n. Then S is a complement of C(A) if and only if S = C(I − AG) for some g-inverse G of A. Proof. ‘If’ part We first notice that for any g-inverse G of A, C(A) ∩ C(I − AG) = {0}. Moreover, ρ(I − AG) = m − ρ(A) = m − d(C(A)). Thus, C(I − AG) is a complement of C(A). ‘Only if’ part Let (P1 , Q1 ) be a rank factorization of A. Let the columns of the matrix Q1 R form a basis of S and T be a matrix such that ·· is non-singular. T −1 Q1 Q1 I 0 and G = ·· (P1 : R)−1 is a Then A = (P1 : R) ·· 0 0 T T g-inverse of A. Hence. 00 I − AG = (P1 : R) (P1 : R)−1 and 0I 00 C(I − AG) = C (P1 : R) = C(R) = S . 0I Theorem 3.4.2. Let A be a matrix of order m × n such that 0 < ρ(A) < min{m, n}. Let (P1 , Q1 ) be a rank factorization of A. Then the class of all matrices B of order m × n such that A <− B is given by {B = A ⊕ C, C arbitrary} or equivalently by B = A or B = P1 Q1 + P2 Q2 , where P = (P1 : P2 ) and Qt = (Qt1 : Qt2 ) are arbitrary full rank extensions of P1 and Q1 respectively and involved partitions are conformable for matrix multiplication.
82
Matrix Partial Orders, Shorted Operators and Applications
Proof.
This theorem is a restatement of Theorem 3.3.5(vii).
Theorem 3.4.3. Let A and B be matrices of the same order. Then A <− B if and only if B = A + (I − AA− )W(I − A− A) for some matrix W and for some g-inverse A− of A. Proof.
Proof follows from Theorem 3.4.2 and Lemma 3.4.1.
Let C and B be matrices of the same order. We now consider a special form for the first matrix C and explore when C <− B. This result will be useful later and is also of independent interest. Theorem 3.4.4. Let A be a non-singular matrix. Then A 0 A 0 −AL − < B if and only if B = + Z(−MA I) 0 0 0 0 0 for some matrices L, M and Z of appropriate orders, where the block matrix diag(A, 0) and the matrix B have the same order. Proof. Let C = diag(A 0). In view of Theorem 3.4.3, C <− B if and only if B = C + (I − CC− )W(I − C− C) for some g-inverse C− and some matrixW. Since A is non-singular, every g-inverse of C is of the form A−1 L − C = , where L, M, N are arbitrary. Now, M N 0 − AL 0 0 I − CC− = and I − C− C = . 0 I −MA I Hence 0 − AL W11 W12 0 0 (I − CC )W(I − C C) = , 0 I W21 W22 −MA I −AL = Z(−MA I) I W11 W12 conformable for the above matrix where W is partitioned as W21 W22 multiplication and Z = W22 . −
−
Remark 3.4.5. In Theorem 3.4.4, B is non-singular if and only if Z is non-singular. We now ask the following question: Given a matrix B, what are all matrices A such that A <− B?
The Minus Order
83
The answer is actually contained in Theorem 3.3.5(v) but we restate for the sake of completeness. We note that if B = 0, and A <− B, then A = 0. So, we shall take matrix B to be non-null. Theorem 3.4.6. Let B be a non-null matrix with a rank factorization (P, Q). Then the class of all matrices A such that A <− B is given by {PTQ : T idempotent}. Theorem 3.4.7. Let A and B be matrices of the same order and B be non-null. Then A <− B if and only if A = B − BR(SBR)− r SB for some matrices S and R such that SBR is defined. Proof. ‘If’ part Let (P, Q) be a rank factorization of B. Then − A = B − BR(SBR)− r SB ⇒ A = P(I − QR(SPQR)r SP)Q . − It is easy to check that QR(SPQR)− r SP and I − QR(SPQR)r SP are − idempotent. So, by Theorem 3.4.6, A < B. ‘Only if’ part Let A <− B and (P, Q) be a rank factorization of B. By Theorem 3.4.6 A = PTQ, where T is idempotent. Write
T = I − (I − T) −1 −1 −1 − = I − QQ−1 R (I − T)((I − T)PL PQQR (I − T))r (I − T)PL P. −1 − Let R = Q−1 R (I − T) and S = (I − T)PL . Then A = B − BR(SBR)r SB.
Let B be a given matrix. We now show that each matrix A of the form B − BR(SBR)− r SB for some matrices S and R is the unique matrix below B under the minus order with column and row spaces specified in an interesting way. Theorem 3.4.8. Let B, R and S be matrices such that the product SBR is defined. Then there exists a matrix A satisfying ‘C(A) = N (S) ∩ C(B), C(At ) = N (Rt ) ∩ C(Bt ) and A <− B’ if and only if ρ(SB) = ρ(SBR) = ρ(BR). If such a matrix A exists, then it is unique and is is of the form A = B − BR(SBR)− SB. Proof. ‘If’ part Let ρ(SB) = ρ(SBR) = ρ(BR) hold and let A = B − BR(SBR)− SB. Clearly, C(A) ⊆ C(B) and C(At ) ⊆ C(Bt ). As C(SB) = C(SBR), we
84
Matrix Partial Orders, Shorted Operators and Applications
have SA = SB − SBR(SBR)− SB = 0. So, C(A) ⊆ N (S) ∩ C(B). Let x ∈ N (S) ∩ C(B). Then x = Bu and Sx = SBu = 0. Clearly, Au = Bu. Therefore, x ∈ C(AS) ⊆ C(AS). Thus, C(A) = N (S) ∩ C(B). Similarly, we can prove that C(At ) = N (Rt ) ∩ C(Bt ). Further, ρ(SB) = ρ(SBR) = ρ(BR) ⇒ BR(SBR)− SB is invariant under choices of (SBR)− . Hence by Theorem 3.4.7, A <− B. ‘Only if’ part Let x ∈ N (Rt Bt ). Then Bt x ∈ N (Rt ) ∩ C(Bt ) = C(At ). We can write Bt x = At x + (B − A)t x and also as Bt x = At u for some vector u, as Bt x ∈ C(At ). However, C(At ) ∩ C(B − A)t = {0}, since A <− B, we have (B − A)t x = 0. Therefore, N (Rt Bt ) ⊆ N (B − A)t or equivalently by Remark 3.2.8, C(B − A) ⊆ C(BR). Similarly, we can prove C(B − A)t ⊆ C(Bt St ). So, B − A = BRCSB for some matrix C. Since SA = 0, we have SB = SBRCSB or C is a ginverse of SBR, further, ρ(SB) = ρ(SBR) and ρ(SBR) = ρ(BR) . Thus, A = B − BRCSB = B − BR(SBR)− SB. Notice that BR(SBR)− SB is invariant under choices of (SBR)− , it follows A is unique. Remark 3.4.9. Consider the setup of Theorem 3.4.8. In the ‘only if’ part we have shown that if C(A) = N (S) ∩ C(B), C(At ) = N (Rt ) ∩ C(Bt ) and A <− B, then C(B−A) ⊆ C(BR) and C(B−A)t ⊆ C(Bt St ). In fact, under this hypothesis one can show C(B−A) = C(BR) and C(B−A)t = C(Bt St ).
3.5
Subclass of g-inverses A− of A such that A− A = A− B and AA− = BA− when A <− B
Let A and B be matrices of the same order such that A <− B. In this section we shall characterize the class of all g-inverses A− of A such that AA− = BA− and A− A = A− B. Let A− be one such g-inverse, so that AA− = BA− and A− A = A− B. We shall show that there exists a ginverse B− of B such that AA− = AB− and A− A = B− A. We shall also characterize the class of all such B− for which AA− = AB− and A− A = B− A holds. Let {A− }B = {G : AGA = A, AG = BG, GA = GB}. We shall obtain an explicit representation of {A− }B . Let A <− B. In view of Theorem 3.3.16, there exist projectors P and Q such that A = PB = BQ. We shall also characterize the classes of all projectors P and Q such
The Minus Order
85
that A = PB = BQ when A <− B. We begin with the following lemma, which is also of independent interest. Lemma 3.5.1. Let A <− B. Then C is a reflexive g-inverse of B − A such that AC = 0 and CA = 0 if and only if C = B− (B − A)B− for some g-inverse B− of B. Proof. ‘If’ part Let B− be an arbitrary g-inverse of B. Write C = B− (B − A)B− . Then CA = B− (B − A)B− A = B− (A − A) = 0. Similarly, AC = 0. Also, (B − A)C(B − A) = (B − A)B− (B − A)B− (B − A) = B − A, since (B − A) <− B. So, ρ(C) ≥ ρ(B − A). On the other hand ρ(C) = ρ(B− (B − A)B− ) ≤ ρ(B − A). Hence ρ(C) = ρ(B − A). Therefore, C is a reflexive g-inverse of B − A. ‘Only if’ part Since, A <− B, there exist non-singular matrices R and S such that A = Rdiag(Ia , 0, 0)S and B = Rdiag(Ia , Ib−a , 0)S, where a = ρ(A) and b = ρ(B). It is easy to check that every reflexive g-inverse C of B − A such that AC = 0 and CA = 0 is of the form 0 0 0 S−1 0 Ib−a D23 R−1 , 0 D32 D32 D23 where D23 and D32 are arbitrary. Choose and fix D23 and D32 . It is easy to check that Ia 0 0 T = S−1 0 Ib−a D23 R−1 , 0 D32 T33 is a g-inverse of B, where T33 is arbitrary and C = T(B − A)T for a fixed choice of T. Let A <− B. In the following Lemma, we obtain a class of g-inverses of A which is a subset of {A− }B . Lemma 3.5.2. Let A and B be matrices having the same order such that A <− B and B− be an arbitrary g-inverse of B. Let C = B− (B−A)B− be as in Lemma 3.5.1 and DT = B− − CTC, where T is an arbitrary matrix of suitable order such that CTC is defined. Then the following hold: (i) ADT = AB− , DT A = B− A, for each T, (ii) ADT A = ADT B = BDT A = A, for each T
and
86
Matrix Partial Orders, Shorted Operators and Applications
(iii) ADT = BDT if and only if T is a g-inverse of C. Further, if ADT = BDT , then DT A = DT B. Proof. Statement (i) follows from Lemma 3.5.1. Since A <− B, (ii) is a simple consequence of (i). (iii) ADT = BDT ⇔ A(B− − CTC) = B(B− − CTC) ⇔ AB− = B(B− − (B− (B − A)B− )TC), since AC = 0 and C = B− (B − A)B− ⇔ AB− = BB− B(B− (B − A)B− )TC = BB− − (B − A)B− TC ⇔ (B − A)B− = (B − A)B− TC ⇔ C = B− (B − A)B− = B(B − A)B− TC ⇔ C = CTC ⇔ T is a g-inverse of C. The remaining part of the statement is easy to check. Remark 3.5.3. Notice that Lemma 3.5.2(ii) and (iii) together imply that {B− − B− (B − A)B− : B ∈ {B− }} is a subset of {A− }B . In the next theorem, we show that this subset is indeed the set {A− }B . Theorem 3.5.4. Let A and B be matrices of the same order such that A <− B. Then (i) {A− }B = {B− − B− (B − A)B− : B− ∈ {B− }}. − − − − (ii) {A− r }B = {B AB : B ∈ {B }}. − − − − − (iii) {Ar }B = {Br ABr : Br ∈ {Br }}. Proof. (i) We already proved that {B− − B− (B − A)B− : B− ∈ {B− }} is a subset of {A− }B in Lemma 3.5.2. Let A− ∈ {A− }B . It is clear that BA− B = A. For any B− ∈ {B− }, we define C = B− (B − A)B− and E = A− + C. It is easy to check that E ∈ {B− }. By Lemma 3.5.1, C is a reflexive g-inverse of B−A and AC = 0, CA = 0. Moreover, A− B = A− A and BA− = AA− . Therefore, E(B − A)E = C(B − A)C = C. So, E = A− + E(B − A)E or A− = E − E(B − A)E with E ∈ {B− }. (ii) Since A <− B, we have B− AB− ∈ {A− r }. A routine check shows that B− AB− ∈ {A− } . To prove the other inclusion, we start with an B r − A− ∈ {A } and proceed along the same lines as in (i). r r B (iii) is proved in similar manner as (ii). Remark 3.5.5. Theorem 3.5.4(i) can be interpreted as follows:
The Minus Order
87
Let A <− B. Every g-inverse of A belonging to {A− }B can be obtained by first choosing an arbitrary g-inverse B− of B and subtracting from it a suitable reflexive g-inverse H of B − A, namely H = B− (B − A)B− , which also satisfies AH = 0, HA = 0. Thus, {A− }B = {B− − H , B− arbitrary, AH = 0, HA = 0}. Let A <− B. Let A− belong to {A− }B . Does there exist a g-inverse B of B such that B− A = A− A and AB− = AA− ? The answer is in the affirmative and is contained in the following. −
Theorem 3.5.6. Let A <− B and let A− be a g-inverse of A such that A− A = A− B and AA− = BA− . Then there exists a g-inverse B− of B such that B− A = A− A and AB− = AA− . Proof. Let a = ρ(A) and b = ρ(B). Since A <− B, there exist nonsingular matrices R and S such that A = Rdiag(Ia , 0, 0)S and B = Rdiag(Ia , Ib−a , 0) S, by Theorem 3.3.5(vi). It is easy to check that the class of all g-inverses A− of A such that A− A = A− B and AA− = BA− is given by I 0 C13 S−1 0 0 0 R−1 , C31 0 C33 where C13 , C31 , C33 are arbitrary. Choose I 0 C13 − −1 A =S 0 0 0 R−1 C31 0 C33 for a particular choice of C13 , C31 , C33 . Then it is easy to check that one choice of required B− is I 0 C13 B− = S−1 0 I 0 R−1 . C31 0 C33 In fact, the class of all such B− I −1 S 0 C31 where D is arbitrary.
is given by 0 C13 I 0 R−1 , 0 D
88
Matrix Partial Orders, Shorted Operators and Applications
Corollary 3.5.7. Let A and B be matrices of the same order such that A <− B and let A− be a g-inverse of A such that A− A = A− B and AA− = BA− , then for a g-inverse B− of B, (i) AA− = AB− ⇔ AA− and BB− commute. (ii) A− A = B− A ⇔ A− A and B− B commute. The following theorem deals with a question related to the one in Theorem 3.5.6 (from the point of view of a given g-inverse B− of B namely: Given A <− B and any g-inverse B− of B, does there exist a g-inverse A− of A such that A− A = A− B = B− A and AA− = BA− = AB− ?). Theorem 3.5.8. Let A and B be matrices of the same order such that A <− B. Then for any g-inverse B− of B, there exists a g-inverse A− of A such that (a) AA− = BA− , (b) A− A = A− B, (c) AA− = AB− , and (d) A− A = B− A. Proof. Since A <− B, there is a g-inverse A− of A such that A− A = A− B and AA− = BA− . Define G = B− BA− BB− . It is easy to check that G is g-inverse of A. Further, AG = AB− BA− BB− = AA− BB− = AA− AB− = AB− . Also, BG = BB− BA− BB− = BA− BB− = AA− BB− = AA− AB− = AB− . So, G satisfies (a). The other equalities can be similarly established. While proving Theorem 3.5.8, we obtained another characterization of {A }B . We state it below for completeness. −
Theorem 3.5.9. Let A and B be matrices of the same order such that A <− B and A = Rdiag(Ia , 0, 0)S and B = Rdiag(Ia , Ib−a , 0) S. Then Ia 0 D13 {A− }B = S−1 0 0 0 R−1 , D31 0 D33 where D13 , D31 and D33 are arbitrary. As observed in Theorem 3.3.16, for matrices A, B of the same order whenever A <− B, there exist projectors P and Q such that A = PB = BQ. Let A <− B. We shall now obtain explicitly the class of all projectors P and Q such that A = PB = BQ. In the process we determine the possible ranks the projectors P and Q can have and obtain explicitly the subclass of the projectors P and Q of specified rank with the property A = PB = BQ.
The Minus Order
89
Lemma 3.5.10. Let A and B be matrices of the same order and let P be a projector such that the product PB is defined. Then A = PB if and only if PA = A and P(B − A) = 0. Also, for a projector Q such that the product BQ is defined, A = BQ if and only if AQ = A and (B − A)Q = 0. Proof.
Proof is trivial.
Theorem 3.5.11. Let A and B be matrices of the same order m × n such that A <− B. Let P be a projector such that the product PB is defined and has same order as B. If A = PB, then ρ(A) ≤ ρ(P) ≤ ρ(A) + m − ρ(B) . Proof. In view of Lemma 3.5.10, PA = A and P(B − A) = 0. As PA = A, we have ρ(A) ≤ ρ(P). Also, P(B − A) = 0 ⇒ C(Pt ) ⊆ N (B − A)t ⇒ ρ(P) = ρ(Pt ) = d(C(Pt )) ≤ d(N (B − A)t ). Thus, ρ(A) ≤ ρ(P) ≤ ρ(A) + m − ρ(B).
Corollary 3.5.12. Let A, B be matrices of the same order m × n such that A <− B. Let Q be a projector such that the product BQ is defined and is of the same order as B. If A = BQ, then ρ(A) ≤ ρ(Q) ≤ ρ(A) + n − ρ(B) . Theorem 3.5.13. Let A and B be matrices of the same order m × n with ranks ‘a’ and ‘b’ respectively. Let A <− B. Let R and S be non-singular matrices such that A = Rdiag(Ia , 0, 0)S and B = Rdiag(Ia , Ib−a , 0)S. Then the class of all projectors P such that A = PB is given by Ia 0 T13 R 0 0 T23 R−1 , 0 0 T33 where T23 = UT33 , T13 = V(I − T33 ) and T33 is idempotent matrix of order (m − b) × (m − b) and matrices V and U are arbitrary. Proof. If P is an idempotent matrix of the given form, clearly, A = PB. Now, let P be projector such that A = PB. Write P = RTR−1 . Notice that T is idempotent since P is idempotent. Partition T as T11 T12 T13 T = T21 T22 T23 T31 T32 T33
90
Matrix Partial Orders, Shorted Operators and Applications
conformable for matrix multiplication with diag(Ia , 0, 0). Since, A = PB, PA = A and C(A) ⊆ C(P). Now, A = PB ⇒ T11 = Ia , T12 = 0, T21 = 0, T31 = 0, T22 = 0 and T32 = 0. As, T is idempotent, we have T13 + T13 T33 = T13 , T23 = T23 T33 and T233 = T33 . So, Ia 0 T13 T = 0 0 T23 0 0 T33 where T13 = V(I − T33 ), T23 = UT33 for some matrices U and V and T33 is idempotent. Hence the result follows. Corollary 3.5.14. Consider the setup of Theorem 3.5.13. Then the class of all projectors Q such that A = BQ is given by Ia 0 0 R 0 0 0 R−1 , T31 T32 T33 where T13 = (I − T33 )Z, T32 = T33 W and T33 is an idempotent matrix of order (m − b) × (m − b) and Z and W are arbitrary. Theorem 3.5.15. Let A and B be matrices of order m × n with ranks ‘a’ and ‘b’ respectively. Let A <− B and P be a projector of rank ‘p’ such that A = PB. Then there exist non-singular matrices X and Y such that A = Xdiag(Ia , 0, 0)Y and B = Xdiag(Ia , Ib−a , 0, 0)Y and
Ia 0 P = X 0 0
0 0 0 0 0 Ip−a 0 0
L 0 X−1 for some matrix L. 0 0
Proof. In view of Theorem 3.5.13, there exist non-singular matrices R and S such that A = Rdiag(Ia , 0, 0)S and B = Rdiag(Ia , Ib−a , 0)S and Ia 0 T13 P = R 0 0 T23 R−1 , 0 0 T33 where T23 = UT33 , T13 = V(I − T33 ) for some matrices ‘U and V’ and T33 is idempotent matrix of order (m − b) × (m − b). Since ρ(P) =
The Minus Order
91
p, ρ(T33 ) = p − a. So, there exists a non-singular matrix K of order (m − b) × (m − b) such that T33 = Kdiag(Ip−a , 0 )K−1 . Then I 0 0 L I 0 0 a I 0 0 0 0 0 0 R−1 . P = R 0 I 0 0 0 Ip−a 0 0 I 0 −1 0 0 K 0 0 K 0 0 0 0 Further, Ia 0 0 Ia 0 0 L Ia 0 0 0 I W 0 b−a 0 0 0 P = R 0 Ib−a 0 0 0 Ip−a 0 0 0 Ip−a 0 0 K 0 0 0 0 0 0 0 Ia 0 0 L 0 0 Ib−a − W 0 Ia 0 0 Ib−a 0 R−1 . × 0 0 Ip−a 0 0 0 K−1 0 0 0 0
L 0 0 0
Let Ia 0 0 Ia 0 0 0 I W b−a X = R 0 Ib−a 0 0 0 Ip−a 0 0 K 0 0 0
L 0 0 0
and Y = S. It can be checked that A = Xdiag(Ia , 0, 0, 0)Y, B = Xdiag(Ia , Ib−a , 0, 0)Y and Ia 0 0 L 0 0 0 0 −1 P = X 0 0 Ip−a 0 X . 0 0 0 0 Corollary 3.5.16. Consider the setup as in Theorem 3.5.15. Let Q be a projector of rank ‘q’ such that A = BQ. Then there exist non-singular matrices C and D such that A = Cdiag(Ia , 0, 0, 0)D, B = Cdiag(Ib−a , 0, 0)D and Ia 0 0 0 0 0 0 0 −1 Q = C 0 0 Iq−a 0 C . M0
0
0
We now obtain the subclass of all projectors P of rank ‘p’ (a ≤ p ≤ a + m − b) such that A = PB, whenever A <− B. From Theorem 3.5.13, it
92
Matrix Partial Orders, Shorted Operators and Applications
is clear that if A = Rdiag(Ia , 0, 0) S and B = Rdiag(Ia , Ib−a , 0) S, then every projector P such that A = PB is of the form Ia 0 T13 P = R 0 0 T23 R−1 , 0 0 T33 where T23 = UT33 , T13 = V(I − T33 ) for some matrices ‘U and V’ and T33 is idempotent matrix of order (m − b) × (m − b). Since P is idempotent and ρ(P) = p, ρ(P) = tr(P) = tr(Ia ) + tr(T33 ) = a + ρ(T33 ). Thus, the class of projectors P of given rank p is given by Ia 0 T13 P = R 0 0 T23 R−1 , 0 0 T33 where T23 = UT33 , T13 = V(I − T33 ), T33 is idempotent matrix of order (m − b) × (m − b), ρ(T33 ) = p − a and ‘U and V’ are arbitrary matrices. The class of all projectors P of rank ‘a’ such that A = PB deserves a special mention. It is clear that this class is given by Ia 0 T13 P = R 0 0 0 R−1 , 0 0 0 where T13 is arbitrary. Recall that in the proof of Theorem 3.5.6, the class of all g-inverse A− such that A− A = A− B, AA− = BA− is given by Ia 0 C13 P = R 0 0 0 R−1 . C31 0 C33 Hence for each A− in that class,
Ia 0 C13 AA− = R 0 0 0 R−1 , 0 0 0 which is a projector of rank ‘a’. Conversely, each projector P of rank ‘a’ such that A = PB is of the form Ia 0 T13 Ia 0 0 Ia 0 T13 P = R 0 0 0 R−1 = R 0 0 0 SS−1 0 0 0 R−1 0 0 0 0 0 0 0 0 0 = AA− , where Ia 0 T13 A− = S−1 0 0 0 R−1 satisfies A− A = A− B, AA− = BA− . 0 0 0
The Minus Order
93
Thus, we have proved Theorem 3.5.17. Let A and B be matrices of the same order m × n with ranks ‘a’ and ‘b’ respectively. Let A <− B. Then the class of all projectors P of rank ‘a’ such that A = PB is the class of all AA− satisfying A− A = A− B, and AA− = BA− . Theorem 3.5.18. Let A, B be matrices of the same order m × n with ranks ‘a’ and ‘b’ respectively. Let A <− B and P be a projector such that A = PB. Then P can be written as P = P1 + P2 where P1 is a projector of rank ‘a’ such that A = P1 B and P2 is projector such that P1 P2 = P2 P1 = 0, P2 A = 0 and P2 (B − A) = 0. Proof.
In the setup of Theorem 3.5.15, write Ia 0 0 L 0 0 0 L 0 0 0 0 −1 0 0 0 0 −1 P1 = X . 0 0 0 0 X , P2 = X 0 0 Ip−a 0 X 0 0 0 0 0 0 0 0 It is easy to check that P1 is a projector of rank ‘a’ such that A = P1 B and P2 is idempotent such that P1 P2 = P2 P= 0, P2 A = 0 and P2 (B−A) = 0.
3.6
Minus order for idempotent matrices
Idempotent matrices (being projectors too) are a very important class of matrices in various applications such as in the theory of Linear Models. In this section, we study the minus order for this class. For an idempotent matrix A, we find the class of all matrices B such that A <− B. We then obtain the class of all idempotent matrices B such that A <− B. Now, let B be an idempotent matrix. We show that if A <− B, then A is necessarily idempotent. We also obtain the class of all A such that A <− B. Notice that for an idempotent matrix E, E is a g-inverse of itself and ρ(E) = tr(E). Let (P, Q) be a rank factorization of a non-null square matrix A. Then A is idempotent if and only if QP = I. Moreover, for idempotent matrices E and F of the same order, it is trivial to check that E <s F if and only if E = FE = EF if and only if E <− F. Thus, for any idempotent matrix A, A <− I, A and I being matrices of the same order. Lemma 3.6.1. Let E and F be idempotent matrices of the same order. Then E <− F if and only if ρ(F − E) = ρ(F − EF) = ρ(F − FE).
94
Matrix Partial Orders, Shorted Operators and Applications
Proof.
Follows from Theorem 3.3.16(i) ⇔ (iv).
We now prove a lemma that is also of independent interest. Lemma 3.6.2. Let E and F be idempotent matrices of the same order. Then F − E is idempotent if and only if E = FE = EF. Proof. ‘If’ part Let E = FE = EF. Then (F − E)2 = F2 − FE − EF + E2 = F − FE − EF + E = F − E. Hence, F − E is idempotent. ‘Only if’ part Write F = E + (F − E). Since E, F and E − F are idempotent matrices, ρ(F) = tr(F) = tr(E) + tr(F − E) = ρ(E) + ρ(F − E). By Remark 3.3.9, we have E <− F, and so, E = FE = EF.
Theorem 3.6.3. Let A be an idempotent matrix with a spectral decomposition A = Pdiag(I, 0)P−1. Then the class of all matrices B such that −L A <− B is given by A + P W(−M I)P−1 where L, W and M are I arbitrary. Proof follows from Theorem 3.4.4. Theorem 3.6.4. Let A be an idempotent matrix. Then the class of all idempotent matrices B such that A <− B is given by B = A + C where C is an idempotent matrix such that AC = CA = 0. Proof. Let B = A + C, where C is an idempotent matrix such that AC = CA = 0. Then B2 = B and trivially A <− B. Now, let B be an idempotent matrix such that A <− B. Then by Theorem 3.3.5(vii), B = A ⊕ (B − A). Since B = B2 , ‘C(A) and C(B − A)’ are virtually disjoint and ‘C(At ) and C(B − A)t ’ are virtually disjoint, it follows that A(B − A) = 0, (B − A)A = 0 and B − A is idempotent. Let C = B − A. Then B = A + C where C is an idempotent matrix such that AC = CA = 0. Theorem 3.6.5. Let A and B be square matrices of the same order with B idempotent. Then A <− B if and only if A = A2 = AB = BA.
The Minus Order
95
Proof. ‘Only if’ part Since A <− B and B is idempotent, we have A = ABA = AB = BA. Therefore, A2 = (AB)(AB) = (ABA)B = AB = A. Thus, A = A2 = AB = BA. The proof of ‘If’ part is trivial. Theorem 3.6.6. Let B be an idempotent matrix. Let (P, Q) be a rank factorization of B. Then the class of all matrices A such that A <− B is given by A = PTQ where T is an arbitrary idempotent matrix of appropriate order. Proof follows from Theorem 3.4.6. Remark 3.6.7. The matrix A as in Theorem 3.6.6 is idempotent because A2 = PTQPTQ = PT2 Q = A. Remark 3.6.8. If A and B are idempotent matrices such that A <− B, then A# A = A# B and AA# = BA# . This is so, as for an idempotent matrix E, E# = E. (We shall make a detailed study of the relationship A# A = A# B and AA# = BA# for general square matrices in Chapter 4.) 3.7
Minus order for complex matrices
We shall now specialize to matrices over the field of complex numbers and study the special properties of the minus order. Let A <− B. We obtain a canonical form for A and B in term of the singular value decomposition of the matrix A. Theorem 3.7.1. Let A and B be nnd matrices of the same order. Then A <− B if and only if B − A is an nnd matrix of rank equal to ρ(B)−ρ(A). Proof. ‘If’ part Since ρ(B − A) = ρ(B) − ρ(A), we have A <− B, by Remark 3.3.9. ‘Only if’ part Clearly, ρ(B−A) = ρ(B)−ρ(A) as A <− B. We also have B−A <− B by Theorem 3.3.5. Since B is nnd, we take an nnd g-inverse B− of B. Then (B − A)B− (B − A) = B − A and hence, B − A is nnd. Corollary 3.7.2. Let A and B be matrices of the same order n × n. Let B be an nnd matrix. If A <− B, then A and B − A are nnd.
96
Matrix Partial Orders, Shorted Operators and Applications
Proof. If A <− B, then AB† A = A. Since B is nnd, B† is also nnd. Hence A is nnd. Moreover, A <− B ⇒ B − A <− B and B be an nnd matrix, so, as in proof of ‘Only if’ part of Theorem 3.7.1, B − A is nnd. By comparing Theorem 3.3.5(i) ⇔ (vii) with Theorem 3.7.1, we notice that if A and B are nnd and A <− B, B − A is nnd. We now prove a result similar to Theorem 3.3.5(i) ⇔ (vi) for complex matrices using unitary transformations. Theorem 3.7.3. Let A and B be matrices of the same order over the field of complex numbers with ρ(A) = a, ρ(B) = b, and b > a ≥ 1. Then A <− B if and only if there exist unitary matrices U and V and nonsingular matrices T1 and T2 such that A = Udiag(Da , 0, 0)V? = UT1 diag(Da , 0, 0)T2 V? and B = UT1 diag(Da , Db−a , 0)T2 V? , where Da and Db−a are diagonal positive definite matrices. Proof. ‘If’ part follows form Theorem 3.3.4(vi)⇒ (i) ‘Only if’ part Let A = U1 Da V1? be a singular value decomposition of A where U1 and V1 are semi-unitary matrices and Da is a diagonal positive definite matrix. Since, A <− B we have C(A) ⊆ C(B) and C(A? ) ⊆ C(B? ). So, there exist semi-unitary matrices U2 and V2 such that the columns of (U1 : U2 ) and (V1 : V2 ) form ortho-normal basis of C(B) and C(B? ) respectively. Extend (U1 : U2 ) and (V1 : V2 ) to unitary matrices U = (U1 : U2 : U3 ) and V = (V1 : V2 : V3 ) respectively. Then we can write A = Udiag(Da , 0b−a , 0) V? and B = Udiag(M, 0) V? , where M is a b × b non-singular matrix. (Compare with Theorem 3.3.5 − (i) ⇒ (vi).) Since, A <− B, we have diag(D a , 0, 0) < diag(M, 0) and M11 M12 diag(Da , 0) <− M. Partition M as M = , where M11 and M21 M22 M22 are matrices of orders a × a and (b − a) × (b − a) respectively. By Theorem 3.4.5, M22 is non-singular and 3.4.4andRemark Da 0 −Da L1 M = + M22 (−L2 Da , I) for some matrices L1 and 0 0 I L2 . Notice that M = Pdiag(Da , M22 ) Q and diag(Da , 0) = Pdiag(Da , 0)Q,
The Minus Order
where P =
I − Da L1 0 I
and Q =
97
I 0 are non-singular matri−L2 Da I
ces. Now, let M22 = RDb−a S be a singular value decomposition of M22 , where R and S are unitary and Db−a is a diagonal positive definite matrix, as M22 is non-singular. Then it is easy to check that A = Udiag(Da : 0 : 0) V? = UT1 diag(Da : 0 : 0) T2 V? and B = UT1 diag(Da : Db−a : 0) T2 V? , where I 0 I 0 P 0 Q 0 and T2 = 0 R . T1 = 0 R 0 I 0 I Clearly T1 and T2 are non-singular.
Corollary 3.7.4. Let A and B ∈ Cm×n with ρ(A) = a, ρ(B) = b, and b > a ≥ 1. Then A <− B if and only if there exist unitary matrices U and V, diagonal positive definite matrix Da and a b × b non-singular matrix M such that Da 0 F1 M= + M22 (F2 , I) 0 0 I such that A = Udiag(Da , 0b−a , 0) V? and B = U(M, 0)V? , where M22 is non-singular and F1 and F2 are some matrices of suitable orders. Corollary 3.7.5. Let A and B ∈ Cm×n be nnd matrices with ρ(A) = a, ρ(B) = b and b ≥ a ≥ 1. Then A <− B if and only if there exist a unitary matrix U, an a × a positive diagonal matrix Da and a definite Da 0 F b × b positive definite matrix M = + M22 (F? , I) such that 0 0 I A = Udiag(Da , 0b−a , 0) U? and B = Udiag(M, 0) U? ; where M22 is a positive definite matrix and F is some suitable matrix. Compare Theorem 3.7.3 with Theorem 3.3.5(i) ⇔ (vi). A vigilant reader might wonder: what is so special about Theorem 3.7.3 over Theorem 3.3.4(i) ⇒ (vi), since every non-singular matrix L can be written as L = UT where U is unitary and T non-singular (L = UU? L for any unitary matrix U). In connection with Theorem 3.7.3, the point to be noted is that we can retain a singular value decomposition for A, which is A = Udiag(Da , 0, 0)V? = UT1 diag(Da , 0, 0)T2 V? . The importance of this result will be further clear when we deal with the star order.
98
3.8
Matrix Partial Orders, Shorted Operators and Applications
Exercises
Unless otherwise stated matrices are over arbitrary field and may be square or rectangular. (1) Let A and B be hermitian matrices of the same order over the complex field. Show that A <s B if and only if C(A) ⊂ C(B). (2) If A and B are square matrices of the same order such that A <s B and B2 = 0, then show that A2 = 0. What happens if B2 = B? (3) Prove that 0 <− B for any matrix B. Let n > m and I be the identity matrix of order m × m. Further let B be a matrix of order n × n. Show that (I, 0) <− B if and only if B = (I, 0). State and prove the corresponding result for m > n and for m = n. (4) Let A and B be matrices of the same order such that A <− B and for a suitable matrix C, CA = I. Prove that A = B. Is the conclusion still valid if there is a suitable matrix D such that AD = I? Justify your answer. Conclude that all invertible matrices are maximal under the minus-order. (5) Let A and B be matrices of the same order such that A <− B and (P, Q) be a rank factorization of A. Prove that (i) P = BGP and (ii) Qt = Qt GB for some g-inverse G of B. Also show that A = BHB for some matrix H. (6) Prove that under the minus order every square matrix A lies below some invertible matrix. (Hint: Consider the normal form of A.) (7) Let A and B be square matrices of the same order such that A <− B and AB = BA. Show that if B is of index 1, then A is of index 1. What can be said about the index of B if index of A is 1? (8) Let A and B be square matrices of the same order over the complex field such that A <− B. Show that if B is range hermitian then A may not be range hermitian. (9) Let A and B be square matrices of the same order over C, the complex field such that A <− B. Let A be normal and B be a hermitian matrix. Show that A is hermitian. (10) Let A = AA− B = BA− A for some g-inverse A− of A. Show that there is a reflexive g-inverse G of A such that A = AGB = BGA. Let A <s B and for some reflexive g-inverse A− of A, BA− A = A. Show that A <− B. (11) Show that for matrices A and B over the field C of complex numbers (i) A <− B ⇔ A? <− B? .
The Minus Order
(12)
(13)
(14) (15)
(16) (17)
(18) (19)
(20) (21)
(22)
(23)
99
(ii) A <− B and C(A? ) ⊆ C(B? ). Then A = B. Give examples of matrices A and B over the field C such that A <− B, but A† 6<− B† . Further, show that if A <− B, then A† <− B† ⇔ A† BA† = A† . Show that the following hold: (i) A <− B ⇔ A− BA− = A− and BA− B = A for some A− ∈ {A− }. (ii) A <− B ⇔ {(A− + B− )/2} ⊆ {A− } for each A− ∈ {A} and for each B− ∈ {B− }. Show that A <− B ⇒ {B− } ⊂ {A− + (B − A)− }. Is the converse true? − − Show that A <− B ⇒ {B− r } ⊂ {Ar + (B − Ar )}. Is the converse true? Now suppose A and B are nnd matrices over the C, the complex − − − field. Show that {B− r } ⊂ {Ar + (B − A)r } ⇒ A < B. − For an idempotent matrix E, show that E < B ⇔ ρ(B − E) = ρ(B − BE) = ρ(B − EB). Let A <− B and one of C(B) ⊆ C(A) and C(Bt ) ⊆ C(At ) and holds. Show that A = B. Show the same holds if A <− B and AX = B is consistent. Show that for square matrices A and B, A <− I ⇔ A2 = A and I <− B ⇔ B = I. If for non-null matrix A, BA− B = A for some A− ∈ {A− }, then show that D = AB− A is invariant under the choice of B− . Further show that D is the unique matrix such that D <− B. For m × n matrices A and B over C, show that A <s B ⇔ AA† <− BB† and A† A <− B† B. For m × n matrices A and B over C, prove or disprove the following: (i) A <− B ⇒ AA? <− BB? (ii) A <− B ⇒ A? A <− B? B. Let A and B be m × n matrices. If B1 is an r × s matrix such that B1 0 B= , 0 0 where each 0 denotes a null matrix of appropriate order. A 0 1 Then show that 0 <− A <− B ⇔ A = ; 0 0 where A1 an r × s matrix and 0 <− A1 <− B1 . Let A and B be normal matrices of order n × n over C. Let ρ(A) = a and ρ(B) = b, 1 ≤ a < b ≤ n with m = b − a. Then show that the following are equivalent:
100
Matrix Partial Orders, Shorted Operators and Applications
(i) A <− B (ii) There exists a unitary matrix U such that U? AU = diag(D, 0) and D + RES RE 0 U? BU = ES E 0 , 0 0 0 where D and E are respectively a × a and m × m non-singular diagonal matrices, R is of order a × m and S is order m × a. (iii) There exists a unitary matrix U such that U? AU = diag(G, 0) and G + RES RF 0 U? BU = FS F 0 , 0 0 0 where D and F are a × a and m × m non-singular matrices, R is of order a × m and S is order m × a. (24) Let Q be a projector of order m × m and B be a m × n matrix. Show that the following are equivalent: (i) QB <− Q (ii) {B− } = {B− Q} + {B− (I − Q)}, {B− Q} ⊆ {(QB)− } and {B− (I − Q)} ⊆ {(I − QB)− }. (25) Let A, P and Q be m × n, p × m and n × q matrices respectively. Let matrices T and R be such that N (T) = C(AQ), N (PA) = C(R). t Show that if D = AR(TAR)− r TA, then C(D) ⊆ N (P), C(D ) ⊆ N (Qt ) and D <− A. (26) Let A, A1 and A2 be m × n matrices. Then the following are equivalent: (i) (ii) (iii) (iv)
A = A1 ⊕ A2 A1 <− A, A2 <− A A1 (A1 + A2 )− A2 = 0 − {A− } ⊆ {A− 1 } ∩ {A2 }.
(27) Let A be an m × n matrix. Let x ∈ C(A) and y ∈ C(At ). The vectors x and y are said to be separable if there exist disjoint matrices A1 and A2 such that (a) A = A1 ⊕ A2 and (b) x ∈ C(A2 ) and y ∈ C(At1 ).
The Minus Order
101
Clearly, any pair of x, y with x ∈ C(A) and y ∈ C(At ) is separable if at least one of x, y is null. Prove that for a given matrix A, x ∈ C(A) and y ∈ C(At ), the vectors x and y are separable if and only if yt A− x = 0 for all g-inverses A− of A. (28) Let A be an m × n matrix. Let x ∈ C(A) and y ∈ C(At ). Prove that A x x, y are separable if and only if ρ = ρ(A) yt 0 (29) Let A and B be m × n matrices such that A <− B. Suppose vectors x, and y are separable with reference to A. Show that they are also separable with reference to B.
Chapter 4
The Sharp Order
4.1
Introduction
We know that if A is an idempotent matrix (that is A2 = A), then it is similar to a matrix of the form diag(I, 0). It also acts as an identity on the column space C(A) of the matrix A. In the previous chapter, we saw that for two idempotent matrices A and B of the same order, A <− B if and only if A = AB = BA = ABA. Observe that an idempotent matrix is a matrix of index not exceeding 1 and is its unique commuting reflexive g-inverse. Let ‘I1 ’ denotes the set of all matrices of index ≤ 1. The set ‘I1 ’ contains a wide class of matrices. In fact, in addition to all idempotent matrices it includes the class of semi-simple matrices and the class of the range-hermitian matrices. Compare the following property of ‘I1 ’ which is analogous to the property of the set of idempotent matrices mentioned above in this paragraph: each non-null matrix A in ‘I1 ’ is similar to diag(D, 0) for some non-singular matrix D and acts as a non-singular linear operator on C(A), that is, for each non-null x ∈ C(A), Ax 6= 0. In this chapter, we define a partial order ‘<# ’ on I1 , the set of square matrices of index ≤ 1. We shall show that under this order ‘<# ’ A is below B (that is A <# B) if and only if A2 = AB = BA and therefore, ‘<# ’ coincides with the minus order when restricted to idempotent matrices. A subclass of g-inverses called the commuting g-inverses plays an important role in developing this partial order. Recall from Chapter 2 that a matrix G is said to be a commuting g-inverse of a matrix A if (i) AGA = A and (ii) AG = GA. Moreover, such a g-inverse for A exists if and only if A is of index ≤ 1. For a matrix A of index ≤ 1, the unique reflexive commuting g-inverse is called its group inverse and is denoted by A# Moreover, if A = Pdiag(D, 0)P−1 , where P and D are non-singular matrices, then
103
104
Matrix Partial Orders, Shorted Operators and Applications
every commuting g-inverse of A is of the form Pdiag(D−1 , C)P−1 where C is arbitrary and, in particular, A# = Pdiag(D−1 , 0)P−1 . In Section 4.2, we define a partial order, called the sharp order on matrices in I1,n , the set of n × n matrices of index 1 and obtain several characteristic properties of this order. In Section 4.3, we obtain the class of all matrices lying above (or below) a given matrix under the sharp order. We also specialize to some subclasses of I1,n and obtain interesting properties of the sharp order. In Section 4.4, we extend the ‘<# ’ order to the class of general square matrices. Let A = A1 + A2 and B = B1 + B2 be the core-nilpotent decompositions of A and B respectively, where A1 and B1 are core parts and A2 and B2 are nilpotent parts. We define A
4.2
Sharp order - Characteristic properties
The class of commuting g-inverses of a square matrix plays a vital role for the order relation we are about to study now. Recall from Chapter 2 that the group inverse of a square matrix is its unique commuting reflexive ginverse and that a square matrix possesses a commuting g-inverse if and only if it is of index≤ 1. We begin this section by defining the Sharp order on I1,n , the set of all n × n matrices of index ≤ 1. We also obtain several characteristic properties of this order. Definition 4.2.1. Let A, B ∈ I1,n . A is said to be below B under the sharp order if there exist commuting g-inverses G1 and G2 of A such that AG1 = BG1 and G2 A = G2 B. When A is below B under the sharp order, we write A <# B. Remark 4.2.2. It is obvious that A <# B ⇒ A <− B. Remark 4.2.3. In Definition 4.2.1, we actually do not require the matrix B to be of index ≤ 1. For example notice that the null matrix of order n × n belongs to I1,n and 0 <# B, for each n × n matrix B. We know that
The Sharp Order
105
not every matrix is of index ≤ 1. we now give a non-trivial example. Example 4.2.4. Consider 1 0 0 1 0 0 A = 0 0 0 and B = 0 0 1 . 0 0 0 0 0 0 Clearly, A2 = AB = BA, so, A <# B. Also, ρ(B) = 2, ρ(B2 ) = 1. Thus, B is not a matrix of index ≤ 1. However, in what follows, we take both A and B to be matrices of index ≤ 1. This is crucial for developing the properties of the sharp order. For example, in order to examine the transitivity of <# , we need to consider A <# B and B <# C, where the matrix B must be of index ≤ 1. In the previous chapter, we had shown that if A <− B, then we can get the same g-inverse G of A such that AG = BG and GA = GB. Is a similar thing possible for sharp order? The answer is in the affirmative and is contained in the following: Theorem 4.2.5. Let A, B ∈ I1,n . Then A <# B if and only if AA# = BA# and A# A = A# B. Proof. ‘If’ part Since A# is a commuting g-inverse of A, the proof is trivial. ‘Only if’ part Let G1 and G2 be commuting g-inverses of A such that AG1 = BG1 and G2 A = G2 B. Now, G1 AG2 is a reflexive commuting g-inverse of A. Therefore, G1 AG2 = A# by uniqueness of the group inverse and AA# = BA# and A# A = A# B. Remark 4.2.6. The sharp order was originally defined in [Mitra (1987)] using the group inverse. Remark 4.2.7. Let A, B ∈ I1,n and P be an n × n non-singular matrix. Then A <# B if and only if PAP−1 <# PBP−1 . In the following theorem we obtain several characterizations of the sharp order. Theorem 4.2.8. Let A, B ∈ I1,n . Then the following are equivalent: (i) A <# B
106
Matrix Partial Orders, Shorted Operators and Applications
(ii) A2 = AB = BA or equivalently, A(B − A) = (B − A)A = 0. (iii) Let a = ρ(A) and b = ρ(B). Then there exists a non-singular matrix Q such that A = Qdiag(Da , 0, 0) Q−1 and B = Qdiag(Da , Db−a , 0) Q−1 , where Da and Db−a are nonsingular matrices of order a × a and (b − a) × (b − a) respectively − (iv) {B− com } ⊆ {Acom } # − (v) B ∈ {Acom } (vi) There exists a projector P onto C(A) such that A = PB = BP (vii) A# A = A# B and (B − A)A = 0 (viii) AA# = BA# and A(B − A) = 0 and (ix) B − A <# B. Proof. (i) ⇒ (ii) Since A <# B, by Theorem 4.2.5, we have AA# = BA# and A# A = A# B. Also, A2 A# = A = A# A2 . So, A2 = AA# A2 = BA# A2 = BA, since AA# = BA# . Similarly, A2 = AB. (ii) ⇒ (i) follows from the equalities (A# )2 A = A# = A(A# )2 . (i) ⇒ (iii) Since A is of index ≤ 1, there exist non-singular matrices R and Da such that A = Rdiag(Da , 0, 0)R−1 equivalently R−1 AR = diag(Da , 0, 0). T11 T12 T13 Let R−1 BR = T. Partition T as T = T21 T22 T23 conformable for T31 T32 T33 matrix multiplication with the partition of R−1 AR. Since A2 = AB = BA, we have T11 = Da and T12 , T13 , T21 and T31 are null matrices. Thus, Da 0 0 T = 0 T22 T23 . 0 T32 T33 T22 T23 Since B is of index ≤ 1 and ρ(B) = b, the sub-matrix is of T32 T33 index≤ 1 with rank b − a. Hence there exists a non-singular matrix S such T22 T23 that = Sdiag(Db−a , 0) S−1 . Let Q = Rdiag(Ia , S). Then T32 T33 Q is a non-singular matrix. Further, A = Qdiag(Da , 0, 0) Q−1 and B = Qdiag(Da , Db−a , 0)Q−1 . (iii) ⇒ (iv) −1 −1 −1 Let G ∈ {B− , where C is an com }. Then G = Qdiag(Da , Db−a , C)Q
The Sharp Order
107
arbitrary matrix. It is easy to check that G ∈ {A− com }. (iv) ⇒ (v) is trivial. (v) ⇒ (i) Since B is of index ≤ 1, there exist non-singular matrices Q and T such −1 that B = Qdiag(T, 0) Q−1 . Notice that B#= Qdiag(T , 0)Q−1 . Par S11 S12 tition Q−1 AQ conformably as Q−1 AQ = . Since B# is a S21 S22 commuting g-inverse of A, we have AB# = B# A. This yields S12 = 0, S21 = 0 and S11 T−1 = T−1 S11 . Further, AB# A = A implies S22 = 0 and S11 T−1 S11 = S11 . Now, let G = B# AB# . Then G is a reflexive commuting g-inverse of A, so G = A# by uniqueness of the group inverse and GA = GB, AG = BG. (ii) ⇒ (vi) We have A = AA# A = A# A2 = A# AB. Let P = A# A. Then P is a projector and A = PB. Now, BP = BA# A = BAA# = A2 A# = A. (vi) ⇒ (vii) Notice that (vi) gives C(A) ⊆ C(P) and C(At ) ⊆ C(Pt ) and therefore, A = AP = PA. Since C(A# ) = C(A), we have A# = A# P = PA# . Now, A# A = A# PB = A# B. Again A = AP = BP2 = BP. So, (B − A)P = 0. Hence, (B − A)A = (B − A)PB = 0. (vii) ⇒ (ii) Since (B − A)A = 0, we have BA = A2 . Pre-multiplying A# A = A# B by A2 , we have A2 = A2 A# A = A2 A# B. Now, from A2 = A2 A# B we have A2 = AB. Proof for (ii) ⇒ (vi) ⇒ (viii) ⇒ (ii) is similar and proof for (ix) ⇔ (ii) follows from the equivalence of (i) and (ii). Remark 4.2.9. Let A ∈ I1,n and B be a non-singular matrix. In view of Theorem 4.2.8, A <# B if and only if AB−1 A = B−1 AB = A. Theorem 4.2.10. The sharp order is a partial order on I1,n . Proof. The result follows trivially from the representations of A and B as given by Theorem 4.2.8 (iii) and (iv), when A <# B. As noted earlier in Remark 4.2.2, the sharp order implies the minus order. The following example shows that the converse is not true. Example 4.2.11. Let 1 0 0 0 I S I S S= , T= , A= and B = ; 0 1 T 0 T −I 0 0
108
Matrix Partial Orders, Shorted Operators and Applications
where I is a 2×2 unit matrix. Then A <− B but A 6<# B, since A# = A and A(B − A) 6= 0. Notice that Example 4.2.11 shows that even on matrices of index ≤ 1 the minus order ‘<− ’ does not imply the sharp order. Thus, for the minus order to imply the sharp order some additional conditions are required. The following theorem includes some of the equivalent additional conditions towards this end. Theorem 4.2.12. Let A, B ∈ I1,n such that A <− B. Then the following are equivalent: (i) (ii) (iii) (iv) (v) (vi) (vii) (viii)
A <# B, A and B commute, A# and B commute, A and B# commute, A# and B# commute B# AB# = A# , BA# B = A and C(BA# B) ⊆ C(A), C((BA# B)t ) ⊆ C(At ).
Proof. (i) ⇒ (ii) follows from Theorem 4.2.8 (i) ⇒ (ii). (ii) ⇒ (iii) Since A and B commute and A# is a polynomial in A, it follows that A# and B commute. (iii) ⇒ (iv) Let A# and B commute. Since, (A# )# = A, A is a polynomial in A# and B# is a polynomial in B, therefore A and B# commute. (iv) ⇒ (v) Let A and B# commute. Since A# is a polynomial in A, A# and B# commute. (v) ⇒ (vi) Let B = Pdiag(T, 0) P−1 , where T is a non-singular matrix. As A <− B, we have A = Pdiag(S, 0)P−1 for some matrix S such that ST−1 S = S. Since A and B# commute, S and T−1 commute. Also, it is easy to check that S# = T−1 ST−1 and hence A# = B# AB# . (vi) ⇒ (vii) BA# B = BB# AB# B = A follows, as C(A) ⊆ C(B) and C(At ) ⊆ C(Bt ). (vii) ⇒ (viii) is obvious.
The Sharp Order
109
(viii) ⇒ (i) As A <− B, A = AB# A = AB# B = BB# A. Now, C(BA# B) ⊆ C(A) ⇒ AA# (BA# B) = BA# B ⇒ AB# (AA# (BA# B)) = AB# (BA# B) ⇒ (AB# A)(A# (BA# B)) = (AB# B)A# B ⇒ AA# (BA# B) = AA# B ⇒ BA# B = AA# B. Also, A <− B ⇒ C(A) ⊆ C(B) ⇒ A = BU for some U. Therefore, A = AA# A = AA# BU = BA# BU = BA# A. So, AA# = BA# . Similarly, A# A = A# B. Corollary 4.2.13. Let A, B ∈ I1,n . Then the following are equivalent: (i) A <# B, (ii) A# and B# commute (iii) A# <# B# .
and
It is important to note that for the equalities AA# = BA# and A A = A# B to hold simultaneously, we require only A to be of index ≤ 1, whatever be the index of B. In fact, it is easy to show that if A = Pdiag(D, 0)P−1 , AA# = BA# and A# A = A# B, then B = Pdiag(D, C)P−1 for some arbitrary matrix C. However, as mentioned earlier for the order relation ‘<# ’ to be a partial order, we do need B to be of index ≤ 1 and therefore we continue to take both A and B to be of index ≤ 1 in the rest of the chapter as well. We hasten to remark that some of the results proved above hold good even when B is not of index ≤ 1. The reverse order law for the group inverse of a product of two (or a finite number) of matrices does not hold in general. However, in presence of the sharp order A <# B on matrices A and B, we have the following: #
Theorem 4.2.14. Let A, B ∈ I1,n such that A <# B. Then (AB)# = B# A# = A# B# = (A# )2 . Proof. The proof follows trivially in view of the representation of A and B in Theorem 4.2.8(iii). Theorem 4.2.15. Let A, B ∈ I1,n such that A <# B. Then, A <− B and Ak <# Bk hold for each positive integer k. Proof. The proof follows trivially in view of the representation of A and B in Theorem 4.2.8(iii).
110
Matrix Partial Orders, Shorted Operators and Applications
Remark 4.2.16. Let A, B ∈ I1,n . The conditions Ak <− Bk for each positive integer k and Ak <# Bk for each positive integer k ≥ 2 do not imply A <# B. For example, take A and B of Example 4.2.11.
4.3
Sharp order - Other properties
In this section, we obtain the class of all g-inverses A− of A such that A− A = A# A = A# B = A− B when A <# B. We also obtain the class of all matrices B lying above a given matrix A under the sharp order. We then obtain a solution to the dual problem of obtaining the class of all matrices A lying below a given matrix B under this order. We conclude the section with an extension of Fisher-Cochran Theorem for matrices of index ≤ 1. We begin with the following theorem which is analogous to Theorem 3.5.6 but is actually much stronger. Theorem 4.3.1. Let A, B ∈ I1,n such that A <# B. Let A− com be any − − commuting g-inverse of A such that Acom A = Acom B and AA− com = − BA− . Then for every commuting g-inverse B of B, we have com com − − − A− com A = Bcom A and AAcom = ABcom . Proof. Since A <# B, by Theorem 4.2.8(iii), there exists a non-singular matrix P such that A = Pdiag(Da , 0, 0)P−1 and B = Pdiag(Da , Db−a , 0)P−1 , where Da and Db−a are non-singular matrices and a = ρ(A) and b = ρ(B). It is easy to check that the class of all commuting g-inverses A− com − − − B and AA = BA is given A = A of A such that A− com com com com −1 by Pdiag(D−1 , where C is arbitrary. Also, we know a , 0b−a , C)P that the class of all commuting g-inverses B− com of B is given by −1 −1 , where E is arbitrary. The rest of the proof is Pdiag(D−1 a , Db−a , E)P computational. Corollary 4.3.2. Let A, B ∈ I1,n such that A <# B and A− com be any − commuting g-inverse of A for which A− A = A B and AA− com com com = − − BAcom . Then for every commuting g-inverse Bcom of B, A− A and com − Bcom B commute. Corollary 4.3.3. Let A, B ∈ I1,n such that A <# B and A− com be any − commuting g-inverse of A for which A− A = A B and AA− com com com = − − BAcom . Then for any commuting g-inverse of Bcom of B we have − − − − − A− com A = Acom B = Bcom A and AAcom = BAcom = ABcom .
The Sharp Order
111
While proving Theorem 4.3.1, we have already obtained a characterization of {A− com }B = {G : AGA = A, AG = GA, AG = BG and GA = GB}, when A <# B. We shall obtain another characterization below. First we prove Lemma 4.3.4. Let A, B ∈ I1,n such that A <# B. Then − (B − A)# = B− com (B − A)Bcom ,
where B− com is any commuting g-inverse of B. Proof. Since A <# B, we have B − A <# B. By Theorem 4.2.8(ix), − − B− com is a commuting g-inverse of B − A. So, Bcom (B − A)Bcom , is − the unique reflexive g-inverse of B − A for each Bcom . Therefore, − (B − A)# = B− com (B − A)Bcom .
Theorem 4.3.5. Let A, B ∈ I1,n such that A <# B. Then − − − {A− com }B = {Bcom − Bcom (B − A)Bcom } # = {B− com − (B − A) }.
Proof.
Follows easily from the proof of Theorem 4.3.1 and Lemma 4.3.4.
Corollary 4.3.6. Let A, B ∈ I1,n such that A <# B. Then A <− B and A# = B# − (B − A)# . However, the converse of above the corollary is not true and can be shown by using the matrices A and B of Example 4.2.11. Let A <# B. In view of Theorem 4.2.8, there exists a projector P such that A = PB = BP. We now characterize the class of all projectors P such that A = PB = BP, when A <# B. Theorem 4.3.7. Let A, B ∈ I1,n such that A <# B. Then the class of all projectors P such that A = PB = BP is given by P = PA + CTD, where PA = A# A is the projector projecting vectors onto C(A) along N (A), (C, D) is a rank factorization of I − B# B and T is an arbitrary idempotent matrix of appropriate order.
112
Matrix Partial Orders, Shorted Operators and Applications
Proof. Notice that A <# B ⇒ AA# = BA# and A# A = A# B. Also I − B# B = CD ⇒ BC = 0, DB = 0. So, A = PA B = BPA . Thus, if P is a matrix of the form P = PA + CTD, then B(PA + CTD) = BPA = A and (PA + CTD)B = PA B = A for all T. Let P be a projector such that A = PB = BP. We show that P is of the given form. Let P = PA + Q. Since A = PB = BP and A = PA B = BPA , so, QB = 0, BQ = 0. Also, PA P = PPA = PA , we have PA Q = QPA = 0. Thus, Q is idempotent. Let (C, D) be a rank factorization of I − B# B. As QB = 0, BQ = 0, there exists a matrix T such that Q = CTD. Also, CTD = Q = Q2 = CTDCTD = CT2 D, since I − B# B is an idempotent. Thus, CTD = CT2 D. As C has a left inverse and D has a right inverse, we have T = T2 i.e. T is idempotent. Given a matrix A ∈ I1,n , we now characterize the class of all matrices B ∈ I1,n such that A <# B. First we prove a lemma. Lemma 4.3.8. Let A and B be matrices such that B = A ⊕ (B − A) and B2 = A2 ⊕ (B − A)2 . If any two of A, B and B − A are of index ≤ 1, the the third is of index ≤ 1. Proof.
Trivial.
Theorem 4.3.9. Let A ∈ I1,n . Then for a matrix B ∈ I1,n , A <# B if and only if B = A + (I − AA# )Z(I − A# A) for some Z such that (I − AA# )Z(I − A# A) is of index ≤ 1. Proof. Notice that B = A + (I − AA# )Z(I − A# A) ⇔ A(B − A) = (B − A)A = 0 ⇔ A <# B . Further, B2 = A2 + {((I − AA# )Z(I − A# A))2 }. In view of the fact that the sums involved are direct, the result follows by Lemma 4.3.8. Corollary 4.3.10. Let A be a non-singular matrix. Then diag(A, 0) <# B if and only if B = diag(A, C) for some matrix C of index ≤ 1. Corollary 4.3.11. Let A ∈ I1,n . The class of all matrices B ∈ I1,n such that A <# B is given by B = A ⊕ E where E is an arbitrary matrix of index ≤ 1. such that AE = EA = 0. Let B ∈ I1,n be a given matrix. We now obtain the class of all A ∈ I1,n such that A <# B.
The Sharp Order
113
Theorem 4.3.12. Let B ∈ I1,n . Then the class of all A ∈ I1,n such that A <# B is given by A = PCQ, where (P, Q) is a rank factorization of B and C is an arbitrary idempotent matrix such that CQP = QPC. Proof. Let (P, Q) is a rank factorization of B. Then QP is non-singular. Let C be an idempotent matrix such that CQP = QPC and A = PCQ. Clearly, ρ(A) = ρ(C), as P and Q are full rank matrices. Further, A2 = PCQPCQ = PC2 QPQ. Since QP is non-singular and P, Q are full rank matrices, ρ(A2 ) = ρ(C) = ρ(A). Thus, A ∈ I1,n . Also, A2 = PCQPCQ = PC2 QPQ = PCQPQ = AB. Similarly, we can show A2 = BA. Next, let A ∈ I1,n such that A <# B. Then A2 = BA = AB. Also, C(A) ⊆ C(B), and C(At ) ⊆ C(Bt ). So, A = PCQ, for some matrix C. Since A2 = BA = AB, we have PCQPCQ = PQPCQ = PCQPQ. Since P and Q are full rank matrices, this further gives CQPC = QPC = CQP. Now, CQPC = CQP and CQP = QPC, so, C2 QP = CQP. As QP is non-singular, we have C2 = C. Let A, B be square matrices of the same order over an algebraically closed field. Let B be of index ≤ 1 and B = Pdiag(J1 , . . . , Jr , 0)P−1 be the Jordan decomposition of B, where J1 , . . . , Jr are non-singular Jordan blocks and P is a non-singular matrix. Let A = Pdiag(D1 , . . . , Dr , 0) P−1 , where Dij = Jij , i = 1, . . . , s for sub-permutation (i1 , . . . , is ) of {1, . . . , r} and Dt = 0 for t ∈ {1, . . . , r} ∩ {i1 , . . . , is }c . It is easy to check that A is of index ≤ 1 and A <# B. We now prove the converse of this statement in a special case where there is exactly one Jordan block corresponding to each non-null eigen-value (or equivalently, the geometric multiplicity of each non-null eigen-value is 1). Theorem 4.3.13. Let A, B be square matrices of same order over an algebraically closed field. Let B be of index ≤ 1 and B = Pdiag(J1 , . . . , Jr , 0)P−1 be the Jordan decomposition of B, where J1 , . . . , Jr are non-singular Jordan blocks corresponding to distinct eigenvalues λ1 , . . . , λr and P is a non-singular matrix. Then A is of index ≤ 1 and A <# B if and only if either A is a null matrix or A = Pdiag(D1 , . . . , Dr , 0)P−1 , where Dij = Jij , j = 1, . . . , s for some sub-permutation (i1 , . . . , is ) of {1, . . . , r} and Dt = 0 for t ∈ {1, . . . , r} ∩ {i1 , . . . , is }c . Proof. ‘If’ part is trivial. ‘Only if’ part
114
Matrix Partial Orders, Shorted Operators and Applications
If A is null matrix, the result is trivial. So, let A be non-null. Notice that the following hold: (i) For C, D ∈ I1,n such that C <# D ⇔ RCR−1 <# RDR−1 . (ii) If J is a non-singular Jordan block, then C is an index ≤ 1 matrix such that C <# J if and only if either C is a null matrix or C = J. (iii) If J1 , J2 are non-singular corresponding to distinct Jordan blocks A11 A12 eigen-values, then A = <# diag(J1 , J2 ) if and only A21 A22 if A12 = 0, A21 = 0, ‘A11 = 0 or J1 ’ and ‘A22 = J2 or 0’. The proof follows from the above three steps and by induction on r. Conjecture: The conclusion of Theorem 4.3.13 remains valid even when some or all distinct non-null eigen-values are of geometric multiplicity exceeding 1. We now prove Theorem 4.3.14. Let A, B ∈ I1,n . Then following are equivalent: (i) A = B, # − (ii) A# ∈ {B− com } and B ∈ {A } and # − (iii) A ∈ {Bcom } and ρ(A) = ρ(B). Proof. (i) ⇒ (ii) is trivial. (ii) ⇒ (iii) # − As A# ∈ {B− com } ⇒ ρ(A ) = ρ(Bcom ) for some commuting g-inverse − # Bcom of B. So, ρ(A) = ρ(A ) ≥ ρ(B). Also, B# ∈ {A− }, therefore, ρ(B) = ρ(B# ) ≥ ρ(A). Thus (iii) follows. (iii) ⇒ (i) − # Since A# ∈ {B− = B− com }, A com for some Bcom . Since ρ(A) = ρ(B), # # ρ(A ) = ρ(B) implying A is the commuting reflexive g-inverse of B. Thus, A# = B# . Since for any matrix X of index ≤ 1, X = (X# )# , we have A = B. Let A, B ∈ I1,n such that A <# B. It is easy to see that A# <# B− com − − − − for all B− com ∈ {B com}. Since every Bcom is also an Acom , given a Bcom − # − obviously there exists an A− com such that Bcom < Acom . We now show − − that given commuting g-inverses Acom and Bcom of A and B respectively # − # − such that B− com < Acom whenever A < B, then Acom is a commuting g-inverse of B, a statement that is much stronger than the one in the previous sentence. (See also, Theorem 4.3.1.)
The Sharp Order
115
Theorem 4.3.15. Let A, B ∈ I1,n such that A <# B. Let A− com and B− com be any commuting g-inverses of A and B respectively satisfying # − − B− com < Acom . Then Acom is a commuting g-inverse of B. Proof. Since A <# B, there exist non-singular matrices P, C and D such that A = Pdiag(C, 0, 0)P−1 and B = Pdiag(C, D, 0)P−1 . Then −1 −1 A− , N)P−1 and B− , D−1 , T)P−1 com = Pdiag(C com = Pdiag(C # for some matrices N and T of appropriate order. Since B− A− com < com , −1 # −1 diag(D , T) < N, it follows that N = diag(D , S). Hence A− com is a commuting g-inverse of B. As noted at the beginning of the chapter, every idempotent matrix is of index ≤ 1. We have the following simple but important result. Theorem 4.3.16. Let E and F be idempotent matrices of the same order. Then the following are equivalent: (i) E <s F, (ii) E <− F (iii) E <# F.
and
We now examine when each summand in a sum of matrices of index 1 lies below the sum. The following theorem is an analogue of Fisher-Cochran theorem for matrices of index ≤ 1. Theorem 4.3.17. Let A1 , . . . , Ak ∈ I1,n and A =
k X
Ai . Consider the
i=1
following statements: (i) Ai Aj = 0 whenever i 6= j, (ii) There exist non-singular matrices P, D1 , . . . , Dk such that Ai = Pdiag(0, . . . , 0, Di , 0, . . . , 0)P−1 for each i = 1, . . . , k, (iii) Ai <# A for each i = 1, . . . , k, (iv) ρ(A) = ρ(A2 ) and k X (v) ρ(Ai ) = ρ(A). i=1
Then any of (i) and (ii) implies all the others from (i) to (v). (iii) and (v) together imply all the others from (i) to (iv).
116
Matrix Partial Orders, Shorted Operators and Applications
Proof. (i) ⇒ (ii) Let (Ei , Fi ) be a rank factorization of Ai for i = 1, . . . , k. Since Ai for i = 1, . . . , k is of index ≤ 1, Fi Ei is non-singular for each i = 1, . . . , k. Further, whenever i 6= j, Ai Aj = 0 implies Ei Fi Ej Fj = 0 and therefore, Fi Ej = 0 as each Ek has a left inverse and each Fi has a right inverse. Write E = (E1 : . . . : Ek ) and F = (Ft1 : . . . : Ftk )t . Let E be of orP der n × r and F of order r × n. Now, A = i Ei Fi = EF. Notice that FE is a non-singular, since Fi Ei is non-singular for each i and Fi Ej = 0 whenever i 6= j. So, r = ρ(FE) ≤ ρ(E) ≤ r. Hence ρ(E) = r. Similarly, ρ(F) = r. Thus, (E, F) is a rank factorization of A. Let Ek+1 be a matrix whose columns form a basis of the null space of F. Then Ek+1 is a matrix of order n × (n − r) of rank n − r. Now, if Ek+1 u = Ev for some vectors u and v, then 0 = FEk+1 u = FEv. As FE is non-singular, v = 0. Thus C(Ek+1 ) ∩ C(E) = {0}. Therefore, P = (E : Ek+1 ) is a non-singular matrix and FEk+1 = 0. Similarly, we can extend F to a non-singular matrix Q = (Ft : Ftk+1 )t such that Fk+1 E = 0. Now, Q, P and Fk+1 Ek+1 are non-singular. Also QP = diag(F1 E1 , . . . , Fk Ek , Fk+1 Ek+1 ) is nonsingular. Notice that P−1 = (QP)−1 Q. Now, it is easy to check that for each i = 1, . . . , k, Ai = Pdiag(0, . . . , 0, Fi Ei , 0, . . . , 0)P−1 . Let Di = Fi Ei , for each i = 1, . . . , k and recall that Fi Ei is non-singular. (ii) ⇒ (i) and (i) ⇒ (iii) are trivial. (ii) ⇒ (iv) We have A = Pdiag(D1 , . . . , Dk , 0)P−1 where D1 , . . . , Dk are nonsingular. Clearly, ρ(A) = ρ(A2 ). (ii) ⇒ (v) is trivial. (iii) and (v) ⇒ (i) k X Since ρ(A) = ρ(Ai ), the sums C(A1 ) + . . . + C(Ak ) and C(At1 ) + . . . + i=1
C(Atk ) are direct. Now, AAi = A2i implies
X
Aj Ai = 0. As the sum
i6=j
C(A1 ) + . . . + C(Ak ) is direct, so for each m 6= i, Am Ai = 0.
Corollary 4.3.18. Let A1 , . . . , Ak be square matrices of the same order k X and A = Ai . Consider the following statements: i=1
(i) Ai Aj = 0 whenever i 6= j and ρ(Ai ) = ρ(A2i ) for all i = 1, . . . , k. (ii) Ai = A2i for all i = 1, . . . , k. (iii) A = A2 .
The Sharp Order
117
P (iv) ρ(A) = i ρ(Ai ). (v) There exists a non-singular matrix P such that Ai = Pdiag(0, . . . , 0, I, 0, . . . , 0)P−1 , where I occurs in the ith diagonal block for i = 1, . . . k. Then (v) implies all the others from (i) to (iv) and any two of (i), (ii) and (iii) imply the rest. Also (iii) and (iv) imply the rest. Proof.
Exercise.
Remark 4.3.19. The above corollary is a non-hermitian version of FisherCochran theorem due to [Khatri (1968)].
4.4
Drazin order and an extension
In the last three sections, we studied the properties of the sharp order defined on the class ‘I1 ’ of matrices of index 1. In this section we consider square matrices not necessarily of index ≤ 1. As noted in Theorem 2.2.21, every square matrix A has the Core-Nilpotent decomposition A = A1 +A2 , where A1 ∈ I1 , A2 is nilpotent and A1 A2 = A2 A1 = 0. The matrix A1 is called the core part of A and A2 is called the nilpotent part of A. To start with, we define an order relation using the core part of the matrices ignoring the nilpotent part altogether. We call this order relation as the Drazin order. In the process of studying the properties of this order, we show that the Drazin order is a pre-order and is in general different from the space pre-order. We obtain some characterizations of this order relation, one of which also justifies its name: the Drazin order. We also establish some interesting properties of Drazin order. It is natural to ask is: Is there a way to extend the Drazin order so that it becomes a partial order? Indeed there is a way which involves the nilpotent part as well. We define this order and obtain a canonical form for the matrices under this order. We also show that this order is a partial order implying the minus order. Definition 4.4.1. Let A and B be square matrices of the same order. Let A = A1 + A2 and B = B1 + B2 be the core-nilpotent decompositions of A and B respectively, where A1 is core part of A, B1 is core part of B, A2 is nilpotent part of A and B is nilpotent part of B. The matrix A is said to be below the matrix B under the Drazin order if A1 <# B1 .
118
Matrix Partial Orders, Shorted Operators and Applications
When this happens we write A
The Sharp Order
119
(ii) ⇒ (iii) −1 By Remark 2.2.24, we can write A = Qdiag(C , where C1 is 1 , M1 )Q B11 B12 non-singular and M1 is nilpotent. Let B = Q Q−1 , where B21 B22 partitioning of B is conformable with the partitioning of A. Notice that AD = Qdiag(C−1 , 0)Q−1 . Since (ii) holds, we have 1 −1 −1 C1 0 C1 0 B11 B12 C1 0 = 0 M1 0 0 B21 B22 0 0 −1 C1 0 B11 B12 = . 0 0 B21 B22 −1 Or B11 C−1 = C−1 = 0 and C−1 1 1 B11 = I, B21 C1 1 B12 = 0. So, B11 = C1 , B21 = 0 and B12 = 0. Hence B = Qdiag(C1 , B22 ) Q−1 . Let B22 = Rdiag(C2 , N2 )R−1 , where C2 is non-singular and N2 is nilpotent. Write P = Qdiag(I, R). Then, B = Pdiag(C1 , C2 , N2 )P−1 and A = Pdiag(C1 , R−1 M1 R)P−1 . However, M1 is nilpotent implies R−1 M1 R is nilpotent. Let N1 = R−1 M1 R and (iii) follows. (iii) ⇒ (i) and (iii) ⇒ (iv) are trivial. (iv) ⇒ (ii) Since index of A is k, Ak = Ak+1 AD = AD Ak+1 . So, (AD )k+1 Ak B = (AD )k+1 (A)k+1 = AD A. Also, (AD )k+1 (A)k = AD . So, we have AAD = AD A = AD B. Similarly, BAk = Ak+1 yields BAD = AAD .
Theorem 4.4.4. The Drazin order is a pre-order on Fn×n . Proof. Reflexivity is trivial. To prove transitivity, let A
1 0 0 1 0 0 A = 0 0 1 B = 0 0 0 . 0 0 0 0 0 0
120
Matrix Partial Orders, Shorted Operators and Applications
Then A
0 1 0 0
2 0 0 0
0 0 . 0 0
1 0 . 0 0
and B =
Then A
and B =
Then A <s B, but A 6
The Sharp Order
121
Corollary 4.4.11. Consider the same setup as in Theorem 4.4.10. Then (i) A 1 does not imply A
1 1 0 A = 0 0 1 0 0 0
0 1 0 B = 1 0 0 . 0 0 1
Now, ρ(A) = 2 > ρ(A2 ) = ρ(A3 ) = 1. So index A = 2. The 1 1 1 Drazin inverse of A is AD = 0 0 0. Clearly, AD BAD = AD , 0 0 0 AD B2 = AD B2 = AD , but BAD 6= AAD . Thus A 6
122
Matrix Partial Orders, Shorted Operators and Applications
Theorem 4.4.15. Let A be a square matrix. Then the class of all matrices B such that A
Remark 4.4.16. Let A be a square matrix of index k. Then the class of all matrices B such that A
The Sharp Order
123
Proof. (i) ⇔ (ii) follows once we notice that AAD A is the core part of A and A
The Corollary follows from Theorem 4.4.18(iii).
We now show that the order relation (#, −) implies ‘<− ’. Theorem 4.4.20. The order relation (#, −) on Fn×n implies ‘<− ’ i.e., if A <#,− B, then A <− B. Proof. Let A <#,− B. Then by Theorem 4.4.18, there exists a non-singular matrix P such that A = Pdiag(C1 , 0, N1 )P−1 , and B = Pdiag(C1 , C2 , N2 )P−1 where P, C1 and C2 are non-singular and N1 , N2 are nilpotent satisfying N1 <− N2 . Let N− 1 be a g-inverse of N1 such that − − − −1 N− N = N N , N N = N N . Let G = P diag(C−1 , 0, N− . 1 2 1 1 2 1 1 1 1 )P Then, it is easy to check that G is a g-inverse of A for which AG = BG, GA = GB. Hence, A <− B.
124
4.5
Matrix Partial Orders, Shorted Operators and Applications
Exercises
(1) Prove or disprove the following: A <# B ⇒ AA# <# BB# . (2) Prove that the conclusion of Theorem 4.3.15 is false if A# ∈ {B− }, B# ∈ {A− }, by giving a suitable example. (3) Let A− be any commuting g-inverse of a matrix A of index ≤ 1. Then show that A− A = A# A. (4) Prove the following statements: (i) A <# B ⇔ A# <# B# (ii) A <# B ⇒ A <− B and A2 <− B2 . Does the converse of (ii) hold? (5) Let A be a matrix of index ≤ 1 and (L, R) be a rank factorization of A. Suppose that A <# B. Then show that L(RL)−1 R = L(RL)−2 RB = BL(RL)−2 R. (6) Let A and B be both of index ≤ 1 and satisfy (i) AA# BB# = AB# (ii) A# A = A# B and (iii) C(A) ⊆ C(B). Then prove that A <# B. (7) Let A# ∈ {B− }, A <s B and B# AB# = B# . Show that A = B. (8) Show that the relation B defined as A <s B and and A = AB# A is a partial order on I1,n , the n×n matrices in I1 . Show also, that this relation implies the minus order but is not the same as sharp order of I1,n . (9) Prove or disprove: A <− B ⇒ A# <− B# . (10) For A, B ∈ I1,n and for each λ 6= 0, define
B# − λ A# λ−1 G(λ) = B# − A#
if λ 6= 1
.
if λ = 1
Let A <# B. Show that A <− B and for each λ 6= 0, G(λ) = (B − λA)# , where G(λ) is as above. (11) Let A and B be square matrices of same order. Let A be of index ≤ 1. Show that A# A = A# B, AA# = BA# ⇔ A2 = AB = BA. Also, show that the relation A B defined by the equations A# A = A# B, AA# = BA# is a partial order on F n .
The Sharp Order
125
A 0 B1 B2 (12) Let A ∈ I∞ . Then <# = B ⇔ B = 0 0 B3 B4 B1 (I − AA# )S , where A <# B1 , S, T and B4 are T(I − AA# ) B4 arbitrary and partitions are conformable for matrix multiplication.
Chapter 5
The Star Order
5.1
Introduction
The present chapter is devoted to the study of yet another partial order known as the star order. This order is defined for matrices A and B (possibly rectangular) over the field C of complex numbers. We say that a matrix A is below B under the star order if AA? = BA? and A? A = A? B. If the matrices A and B are taken to be hermitian, these defining equalities read as A2 = AB = BA. Recall from the previous chapter that a matrix A is below a matrix B under the sharp order if and only if A2 = AB = BA. Thus, the star order and the sharp order coincide for the class of hermitian matrices. (Notice that a hermitioan matrix is a matrix of index ≤ 1.) In view of this, one can expect a number of results in this order to be parallel to those in the sharp order. This indeed is true as we shall see in due course. A well explored partial order, the star order was first introduced by Drazin in a star semi-group with involution. While establishing several other properties of this order, he showed that the star order was a partial order. He further showed that in the defining equalities of this relation, one could actually replace ‘?’ by ‘†’, in other words the statement ‘AA? = BA? and A? A = A? B’ is equivalent to ‘AA† = BA† and A† A = A† B’. However, M.Hestenes studied the properties of the matrices A and B satisfying the equalities AA? = BA? and A? A = A? B almost two decades earlier than Drazin. He called matrix A, a section of the matrix B whenever A and B satisfied the equalities AA? = BA? , A? A = A? B. The term star order was coined by Drazin. In Section 5.2, beginning with a formal definition of the star order, we obtain several of its characteristic properties. While studying the properties of the minus order and the sharp order, we found that the normal form
127
128
Matrix Partial Orders, Shorted Operators and Applications
and the reduced form (whichis a special case of the core-nilpotent decomposition) A = Pdiag D , 0 P−1 for any matrix A of index ≤ 1, where P and D are non-singular matrices respectively proved useful. In the case of the star order, we shall see that the key reduced form is the singular value decomposition (SVD). In Section 5.3, apart from obtaining several other properties of the star order, we compare the star order with the other partial orders studied so far. We also identify the class of all matrices that lie above/below a given matrix under the star order. In Section 5.4, we specialize to some subclasses of matrices of index ≤ 1, which are important in applications. A detailed study of star order on these subclasses that include range-hermitian, normal, hermitian and nnd matrices, is undertaken here. Section 5.5 covers the star order on idempotent matrices. Idempotent matrices play a significant role in applications and for this reason a separate section on star order for the class of idempotent matrices is included. Finally in Section 5.6, Fisher-Cochran Type theorems are established for star order. The matrices studied in this chapter are over C and the inner product of vectors x, y in Cn will be the standard inner product, (x, y) = y? x, unless otherwise stated.
5.2
Star order - Characteristic properties
In this section we define the star order and obtain several of its characteristic properties and establish that the star order is a partial order on Cm×n . We show that if A is below B under the star order, then A and B have simultaneous singular value decomposition. Definition 5.2.1. Let A and B be matrices of the same order m × n. Then A is said to be below B under the star order if AA? = BA? and A? A = A? B. When A is below B under the star order, we write A
The Star Order
129
Theorem 5.2.3. Let A and B be matrices of the same order having ranks a and b respectively. Then the following are equivalent: (i) A
130
Matrix Partial Orders, Shorted Operators and Applications
and −1 Da 0 X1 −1 ? {B− } = V U ; X , arbitrary i = 1, 2, 3 . 0 D X i b−a 2 m 0 0 X3 Remark 5.2.5. Let A
The Star Order
131
(vii) There exist orthogonal projectors P and Q such that A = PB = BQ. − † Proof. (i) ⇒ (ii) Since A− `m AA`m = A , the result follows. (ii) ⇒ (i) Since A† is a minimum norm least squares g-inverse, the result follows. (ii) ⇒ (iii) Notice that A? = A† AA? = A? AA† . So, AA† = BA† ⇒ AA† AA? = BA† AA? = B(A† A)? A? ⇒ AA? = BA? . Similarly A? A = A? B. (iii) ⇒ (iv) follows from Theorem 5.2.3 in view of Remark 5.2.4. (iv) ⇒ (v) − − − − − − Notice that for any B− `m , B`m ∈ {B` } ⊆ {A` } and B`m ∈ {Bm } ⊆ {Am }. − − − Also, {A`m } = {A` } ∩ {Am }. So, the result follows. (v) ⇒ (vi) † † † Since B† ∈ {A− `m }, we have AB A = A and AB and B A are hermitian. † † † † ? † ? † † So, A A = A AB A = (A A) (B A) = (B AA A)? = (B† A)? = B† A. Similarly, AA† = AB† . (vi) ⇒ (vii) From Remark 5.2.5, we have A = AB† B = AA† B, as AA† = AB† . Moreover, A = BB† A = BA† A, since A† A = B† A. Let P = AA† and Q = A† A. Clearly, P and Q are orthogonal projectors satisfying A = PB = BQ. (vii) ⇒ (iii) Now, A = PB ⇒ C(A) ⊆ C(P ) ⇒ PA = A ⇒ A? P = A? . Now A = PB ⇒ A? A = A? PB = A? B. Similarly, A = BQ ⇒ AA? = BA? . (iii) ⇒ (ii) Since A† = A† A†? A? = A? A†? A† , (ii) follows.
Corollary 5.2.9. Let A and B be matrices of the same order. Then the following are equivalent: (i) (ii) (iii) (iv)
A
Proof. By Theorem 5.2.3(ii) it is easy to see that (i) ⇔ (ii), (i) ⇔ (iii) and (i) ⇔ (iv).
132
Matrix Partial Orders, Shorted Operators and Applications
We now explore the additional conditions required for the minus order to yield the star order. Theorem 5.2.10. Let A and B be matrices of the same order m × n such that A <− B. Let ρ(A) = a, ρ(B) = b and k be positive (integer or fraction). (i) The following are equivalent: (a) (b) (c) (d)
A
and
(ii) The following are equivalent: (a1 ) (b1 ) (c1 ) (d1 ) (f1 ) (e1 )
A
(iii) The following are equivalent: (a2 ) (b2 ) (c2 ) (d2 ) (f2 ) (e2 )
A
Finally, all the conditions in anyone of (i), (ii) and (iii) are equivalent to all conditions in any other. Proof. (i) (a) ⇒ (b) A
The Star Order
133
Similarly, (d) ⇒ (b). (b) ⇒ (a) Since AB? and A? B are hermitian, by Theorem 2.7.12, there exits unitary matrices U, V and real diagonal matrices D1 , D2 such that A = UD1 V? and B = UD2 V? . As A <− B, it follows that D1 <− D2 . By suitable simultaneous permutations of rows and columns of D1 and D2 we have PD1 Q = diag(Da , 0b−a , 0) and PD2 Q = diag(Da , Db−a , 0), where P and Q are permutation matrices, Da and Db−a are non-singular diagonal matrices. Without loss of generality, we can take Da and Db−a as diagonal positive definite matrices. For, if some diagonal element di of Da (or Db−a ) is negative, then we can make it positive by multiplying it by −1 and multiplying the ith row of V? by −1. Thus, A = UP? diag(Da , 0b−a , 0)QV? and B = UP? diag(Da , Db−a , 0)QV? . Clearly, A
134
Matrix Partial Orders, Shorted Operators and Applications
and K2 such that L1 L1 ? = I and (K1 : K2 )? (K1 : K2 ) = I. Since A <− B, C(L1 ) ⊆ C(A? ) ⊆ C(B? ) = C(K1 : K2 ). So, there exist ma? trices F1 and F2 suchthat L1 = (K1 : K 2 )(I − ?F1 : −F2 ) . We can Da (I − F1 ) −Da F2 K1 write A = (U1 : U2 ) . Hence, B − A = 0 0 K2 ? ? Da F1 Da F2 K1 (U1 : U2 ) . As A <− B, ρ(B − A) = b − a. So, 0 0 K2 ? Da F1 = 0 and therefore, F1 = 0. Thus, L1 = K1 − K2 F2 ? . Now, I = L1 L1 ? = I + F2 ? F2 . So, F2 = 0. Thus, L1 = K1 and A = U1 Da K1 and B = (U1 : U2 )diag(D2a , D2b−a )(K1 : K2 )? . It is clear that A
A
and
Proof. (i) ⇒ (ii) By Remark 5.2.5, A <− B. By Theorem 5.2.3, there exist unitary matrices
The Star Order
135
U and V such that A = Udiag Da , 0 , 0 V? and B = Udiag Da , Db−a , 0 V? , where Da and Db−a are positive definite diagonal matrices of order a × a and (b − a) × (b − a) respectively. Clearly, (B − A)† = B† − A† . (ii) ⇒ (iii) and (ii) ⇒ (iv) are easy. (iii) ⇒ (i) Since A <− B, by Corollary 3.7.4, there exist unitary matrices U, V, a positive definite diagonal matrix Da and a b × b non-singular matrix M given as Da 0 F1 M= + K F2 , I 0 0 I such that A = Udiag(Da , 0b−a , 0)V? and B = U(M, 0)V? , where K is non-singular and F1 and F2 are some matrices of suitable orders. Also, ? † † ? B† − A† ∈ {(B − A)− ` }, so, (B − A) (B − A)(B − A ) = (B − A) . However, A† = Vdiag(Da −1 , 0b−a , 0)U? , B† = V(M−1 , 0)U? , Da −1 − Da −1 F1 −1 where M = . Hence, −F2 Da −1 K−1 ? ? F1 F2 F2 K? F?1 , I K F2 , I = K? F?1 , I . I I I ? F2 The matrix has a right inverse and K is invertible. So, we have I 0 − Da −1 F1 ? (F1 F1 + I)K F2 , I = F?1 , I . (5.2.1) −1 −1 −F2 Da K Now, (5.2.1) ⇒ (F1 F?1 + I)K(−F2 Da −1 ) = F?1 and F1 F?1 + (F1 F?1 + I)KK−1 = I. These give F1 = 0 and F2 = 0. It now follows that A
136
Matrix Partial Orders, Shorted Operators and Applications
Incidentally, Example 5.2.12 also shows that A <− B and A† <− B† together do not imply A
Subclasses of g-inverses for which A
In this section we characterize the class of all g-inverses A− of a matrix A such that A− A = A† A = A† B and AA− = AA† = BA† , when A
the set of all minimum norm least squares g-inverses of A for which A
The Star Order
137
? −1 G ∈ {A− matrix `m } if and only if G = Vdiag Da , 0b−a , T U for some −1 ? −1 T. Also, each B− is of the form Vdiag U for some D , D , T a `m b−a ? matrix S and (B − A)† = Vdiag 0 , D−1 U . The rest of the proof , T b−a is computational. We now obtain the class of all matrices that lie above a given matrix under the star order. Theorem 5.3.4. Let A and B be matrices having the same order. Then − − A
138
Matrix Partial Orders, Shorted Operators and Applications
Remark 5.3.7. The matrices U1 and V1 in Theorem 5.3.6 are in general not unique unless D has all diagonal elements distinct. Whenever U1 and V1 are not unique, we can vary them over all possible choices of U1 and V1 to obtain all A that are below B under A
Star order for special subclasses of matrices
In this section we specialize to some subclasses of matrices with index ≤ 1, namely, range-hermitian matrices, normal matrices, hermitian matrices and the nnd matrices and make a detailed study of the star order on these subclasses. We also study the relation between the minus order and the star order for several of these subclasses. Theorem 5.4.1. Let A and B be range-hermitian matrices of the same order having ranks a and b (b ≥ a) respectively. Then (i) A <− B if and only if there exist a unitary matrix U, non-singular matrices T and M, where M is of the form T 0 F1 M= + K(F2 , I), 0 0 I where K is a (b − a) × (b − a) non-singular matrix and F1 , F2 are some matrices of appropriate order such that A = Udiag(T, 0)U? and B = Udiag(M, 0)U? . (ii) A
The Star Order
(i) (ii) (iii) (iv) (v) (vi) (vii) (viii) (ix)
139
A
Proof. Equivalence of (i)-(vii) follows from Theorem 5.4.1 and equivalence of (i), (viii) and (ix) follows from Theorem 5.2.10. Theorem 5.4.3. Let A and B be matrices of the same order such that A
and
The proof follows from definition of the star order and Remark 5.2.2(i). Theorem 5.4.5. Let A and B be normal matrices of the same order n × n having ranks a and b (b ≥ a) respectively. Then (i) A <− B if and only if there exist a unitary matrix U, a non-singular matrix Da with possibly complex diagonal elements and a non-singular, Da 0 F1 normal matrix M of the form M = + K(F2 , I), where 0 K I K is a (b − a) × (b − a) non-singular matrix and F1 , F2 are some
140
Matrix Partial Orders, Shorted Operators and Applications
matrices of appropriate order such that A = Udiag(Da , 0b−a , 0)U? and B = Udiag(M, 0)U? . (ii) A
A
In view of Theorem 5.4.5(i), proof follows from Theorem 5.2.10. Remark 5.4.7. Let A and B be hermitian matrices of the same order. If 0 we replace (xii) and (xiii) in Theorem 5.4.6 with (xii) C(BA† B) ⊆ C(A) 0 † † and (xiii) C(B AB ) ⊆ C(A) respectively, then the statements (i)-(xi), 0 0 (xii) and (xiii) are equivalent. Theorem 5.4.8. Let A and B be matrices of the same order having ranks a and b respectively. Let A <− B. Then A
The Star Order
141
F1 K(F2 , I), with K a (b − a) × (b − a) I non-singular matrix and F1 , F2 some matrices of appropriate order. Now, A? B is normal ⇒ Vdiag(Da , 0b−a , 0)diag(M, 0)V? is normal Da 0 0 Da + F1 KF2 F1 K 0 ⇒ 0 0 0 KF2 K 0 is normal. 0 0 0 0 0 0 2 Da + Da F1 KF2 Da F1 K 0 (Da 2 + Da F1 KF2 )? 0 0 ⇒ 0 0 0 (Da F1 K)? 0 0 where M = diag(Da , 0) +
0 0 0 0 0 2 2 ? (Da + Da F1 KF2 ) 0 0 Da + Da F1 KF2 Da F1 K = (Da F1 K)? 0 0 0 0 0 0 0 0 0 ⇒ F1 = 0.
0 0 0 0
Similarly, AB? is normal ⇒ F2 = 0. Therefore, A = Udiag(Da , 0b−a , 0)V? , B = Udiag(Da , K, 0)V? . Clearly, A
Corollary 5.4.9. Let A and B be matrices of the same order having ranks a and b respectively. Let A <− B. Then the following are equivalent: (i) (ii) (iii) (iv) (v)
A
AB† BA† AB? BA?
are normal are normal are normal are normal.
and
Theorem 5.4.10. Let A and B be matrices of the same order having ranks a and b (b ≥ a) respectively. Let A <− B, A be range-hermitian, B be hermitian and A? B be normal. Then A
By Theorem 5.4.1, A = Udiag(T, 0)U? and B = Udiag(M, 0)U? ,
where U is unitary, T is non-singular of order a × a and M is a b × b matrix T 0 F1 matrix of the form M = + K(F2 , I), with K a (b−a)×(b−a) 0 0 I non-singular matrix and F1 , F2 are some matrices of appropriate order.
142
Matrix Partial Orders, Shorted Operators and Applications
T 0 0 T + F1 KF2 F1 K 0 Since, A? B is normal ⇒ 0 0 0 KF2 K 0 is normal. 0 0 0 0 0 0 This gives F1 = 0. As B is hermitian, F2 = 0. So, A = Udiag(T, 0b−a , 0)V? , B = Udiag(T, K, 0)V? . Clearly, A
Theorem 5.4.11. Let A and B be nnd matrices of the same order. Consider the following statements: (i) (ii) (iii) (iv) (v)
A <− B A2 <− B2 A
Then (i) and (ii) imply all the others. Any of (iii), (iv) and (v) imply all others. Proof. (i) and (ii) ⇒ (iii), (iv) and (v) By Corollary 3.7.5, there exists a unitary matrix U, an a × a positive definite diagonal matrix Da and a b × b positive definite matrix M = Da 0 F + M22 (F? , I) such that A = Udiag(Da , 0b−a , 0) U∗ and 0 0 I B = Udiag(M, 0) U? ; where M22 is a positive definite matrix and F is some suitable matrix. As A2 <− B2 , A2 (B† )2 A2 = A2 . Therefore, 2 2 † 2 2 Da 0 0 Da 0 0 Da 0 0 Da + FKF? FK 0 0 0 0 F? K K 0 0 0 0 = 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ⇒ F = 0. So,(iii), (iv) and (v) all hold. (v)⇒ (iii), (iii) ⇒ (i) and (iv)⇒ (ii) are trivial. (v)⇒ (ii) and (iv) Let (v) hold. Then there exist nnd diagonal matrices ∆1 , ∆2 and a unitary matrix V such that A = V∆1 V? and B = V∆2 V? with ∆1 ∆2 = ∆2 ∆1 = ∆21 . Hence, by a suitable permutation of rows and the corresponding columns of ∆1 , ∆2 , we have P∆1 P? = diag(D1 , 0 , 0) and P∆2 P? = diag(D1 , D2 , 0), where D1 , D2 are positive definite diagonal matrices and P is a permutation matrix. Let U = VP. Then U is unitary and A = Udiag(D1 , 0, 0)U? and B = Udiag(D1 , D2 , 0)U? .
The Star Order
143
Clearly, (ii) and (iv) hold. (iv) ⇒ (v) Let (iv) hold. Clearly, A2 and B2 are nnd and therefore, A2 B2 = B2 A2 = A4 . Thus, there exist nnd diagonal matrices D1 , D2 and a unitary matrix U such that A2 = Udiag(D1 2 , 0, 0)U? and B2 = Udiag(D1 2 , D2 2 , 0)U? . Since A and B are the unique nnd square roots of A2 and B2 , we have A = Udiag(D1 , 0, 0)U? and B2 = Udiag(D1 , D2 , 0)U? . It is now clear that (v) holds. Corollary 5.4.12. Let A and B be hermitian matrices of the same order. Consider the statements (i)-(v) in Theorem 5.4.11. Then (i) and (ii) imply all the others. Any of (iii) and (v) imply all others. Also, (iv) implies (v) if and only if (a) every negative eigen-value λ of A is an eigen-value of B and (b) the algebraic multiplicity of λ as an eigen value of A is ≤ the algebraic multiplicity of λ as an eigen value of B. Theorem 5.4.13. Let A and B be square matrices of the same order. Then the following hold: (i) If C(A? ) ⊆ C(B), then B† A† is a reflexive least squares g-inverse of AB. † (ii) If A
144
Matrix Partial Orders, Shorted Operators and Applications
1 1 2 1 and B = . The matrices A and 1 1 1 1 B satisfy all the in Theorem 5.4.13(i) and (ii). conditions 1 1 1 −1 3 3 1 1 † † † and B = . Now, (AB) = 26 , and Also, A = 4 1 1 −1 2 2 2 0 0 B† A† = 41 . Thus, (AB)† 6= B† A† . 1 1
Example 5.4.14. Let A =
Theorem 5.4.15. Let A and B be range-hermitian matrices of the same 2 order. Then A
The Star Order
5.5
145
Star order and idempotent matrices
In this section we consider idempotent matrices and orthogonal projectors and study the star order for these matrices. Recall for idempotent matrices E and F, E <− F if and only if E = EF = FE. We have the following: Theorem 5.5.1. Let E and F be any idempotent matrices of the same order. Then E <− F if and only if E
We now investigate when a given idempotent matrix E is below another under the star order. Theorem 5.5.2. Let E and F be idempotent matrices of the same order. Then the following are equivalent: (i) (ii) (iii) (iv)
E
Proof. (i) ⇒ (ii) Since E
146
Matrix Partial Orders, Shorted Operators and Applications
and FF? −EE? is nnd, we have E(FF? −EE? ) = 0, or EF? = EE? = FE? . Hence, FF? − EE? = (F − E)(F − E)? . Similarly, we can establish F? F − E? E = (F − E)? (F − E). (iv) ⇒ (i) Notice that FF? − EE? = (F − E)(F − E)? ⇒ FF? − EE? is nnd and C(E) = C(EE? ) ⊆ C(FF? ) = C(F). Similarly, F? F − E? E = (F − E)? (F − E) ⇒ F? F − E? E is nnd and C(E? ) ⊆ C(F? ). Thus, E = EFF = FFE or E = EF = FE. As in proof of (iii) ⇒ (iv), we have EF? = EE? = FE? . Similarly, E? E = E? F = F? E. So, E
The Star Order
147
as EE? = EF? = E? E = E? F = diag(0, 0, 1). However, I − F≮? I − E, 0 0 0 Since (I − F)? (I − F) = diag 0, 2, 0 and (I − F)? (I − E) = 1 1 0 . 0 0 0 Let E and F be idempotent matrices of the same order. We now investigate when both E
E − F is nnd E − F is hermitian and idempotent E − F
Proof. (i) ⇔ (ii) ⇔ (iii) is easy. (ii) ⇔ (iv) follows by Theorem 5.5.3. (ii) ⇒ (v) Since (ii) holds, E <− F. Therefore, E = EF = FE. Now, 0 =
148
Matrix Partial Orders, Shorted Operators and Applications
FE − E = FE − E2 = (F − E)E = (F − E)? E. Therefore, F? E = E? F or E? F = E? F. Similarly, by using E = EF we get EE? = FE? . Thus, E
FF? − E is nnd E = FE FF? = E + F(I − E)F? and FF? = E + (F − E)(E − F)? .
Proof. (i) ⇒ (ii) Since FF? − E = FF? − EE? is nnd, we have C(E) ⊆ C(F). Hence, FE = FFE = E. (ii) ⇒ (iii) E+F(I − E)F? = E+FF? −FEF? = E+FF? −EF? = E+FF? −E = FF? . (iii) ⇒ (iv) (F − E)(E − F)? = F(I − E)(I − E)? F? = F(I − E)F? = FF? − E. (iv) ⇒ (i) is trivial. Theorem 5.5.11. Let E and F be idempotent matrices of the same order and let E be hermitian. Then the following are equivalent: (i) (ii) (iii) (iv)
E
Proof. (i) ⇒ (ii) is trivial. (ii) ⇒ (i)
The Star Order
149
Since E <− F we have E = EF = FE. Therefore, E = E? = EF? = FE? . This further gives E = F? E = EF? . Thus, E = EE? = F? E and E = E? E = EF? and so, E
E
Proof. (i)⇒ (ii) As E
150
Matrix Partial Orders, Shorted Operators and Applications
Notice that E = FE = FE? = EF? . Now, F† E = F† FE = EF† F is hermitian. Also, EF = (EF? )F = E(F†FF) ? F = F†FEF?F = F†E. Therefore, EF is hermitian. (iv) ⇒ (i) Since EF and E are hermitian, EF − E is hermitian. So, (EF − E)? (EF − E) = (EF − E)(EF − E) = EF − E − EF + E = 0. Hence, EF = E = FE. Thus, E <− F and by Theorem 5.5.11, E
E
Proof is straightforward. 5.6
Fisher-Cochran type theorems
Recall that in Theorem 4.3.17, we had seen that if A1 , . . . , Ak are matrices Pk each of index 1 satisfying Ai Aj = 0 whenever i 6= j and A = i=1 Ai , then each Ai lies below A under the sharp order. As a consequence, we had also deduced a non-hermitian version of Fisher-Cochran theorem for the idempotent matrices. In this section we study similar relationships for the range-hermitian matrices, normal matrices and hermitian matrices. We also remark that the last one is essentially an algebraic version of the Fisher Cochran theorem on the distribution of quadratic forms in independent standard normal variables. Theorem 5.6.1. Let A1 , . . . , Ak be range-hermitian matrices of the same Pk order and let A = i=1 Ai . Consider the following statements: (i) Ai Aj = 0 for whenever i 6= j
The Star Order
151
(ii) there exist non-singular matrices D1 , . . . , Dk , and a unitary matrix U such that Ai = Udiag(0, . . . , 0, Di , 0, . . . , 0)U? , for each i = 1, . . . , k (iii) Ai
A2 = A Ai 2 = Ai for each i Ai Aj = 0 whenever i 6= j Pk ρ(A) = i=1 ρ(Ai ) and there exists a unitary matrix U such that Ai = Udiag(0, . . . , 0, Ii , 0, . . . , 0)U? , for each i = 1, . . . , k.
Then any two of (i), (ii) and (iv) imply all others from (i) to (v). Any two of (i), (ii) and (iii) imply all others from (i) to (v). (v) implies all others from (i) to (iv). Further, (i) and (iv) imply all the others.
152
5.7
Matrix Partial Orders, Shorted Operators and Applications
Exercises
(1) Let A, B be matrices of order m × n. Let the symbol
A
(6) For matrices A, B of order m × n, show the following: (i) A
The Star Order
(ii) (iii) (iv) (v)
A? , B commute A, B? commute A† , B commute A, B† commute.
153
and
(10) Let A and B be square matrices of the same order such that A is range hermitian. Prove that (i) A
Chapter 6
One-Sided Orders
6.1
Introduction
In the previous three chapters we studied the space pre-order and the minus, the sharp and the star partial orders. Recall that for matrices A and B of the same order A <− B ⇔ AA− = BA− and A− A = A− B for some A− ∈ {A− }, A <# B ⇔ AB = A2 = BA, and A
155
156
Matrix Partial Orders, Shorted Operators and Applications
(or the condition BA = A2 together with the condition C(At ) ⊆ C(Bt )), we obtain a partial order which we call the right sharp order (or the left sharp order respectively). We also obtain some conditions under which the one-sided sharp order and the sharp order coincide. In Section 6.4, we introduce two new classes of g-inverses of index 1 matrices. Their important properties are developed in the exercises at the end of this chapter. Our focus in this section is in exploring the role they play in the study of onesided sharp orders. In fact, it turns out that the order relations defined through them coincide with the one-sided sharp orders. In Section 6.5, we study the order relations by taking one of the defining conditions of the star order say, AA? = BA? along with C(A? ) ⊆ C(B? )(or the condition A? A = A? B along with C(A) ⊆ C(B)). We show that this leads us to a partial order that has properties very similar to the star order. The order relations thus obtained are known respectively as the right star and the left star orders. As seen in Chapter 5, for matrices A and B satisfying A
6.2
The condition AA− = BA−
We begin this section by studying the condition AA− = BA− or (A− A = A− B). As we shall see, this condition by itself does not even define a pre-order, yet has some interesting properties. We have the following: Theorem 6.2.1. Let A and B be matrices of the same order. Then the following are equivalent: (i) AA− = BA− for some g-inverse A− of A (ii) A = BQ for some idempotent matrix Q and (iii) C(A)t ∩ C(B − A)t = {0}. Proof. (i) ⇒ (ii) Since AA− = BA− , we have A = BA− A = BQ, where Q = A− A is idempotent.
One-Sided Orders
157
(ii) ⇒ (iii) Let A = BQ. Then AQ = BQ2 = BQ = A. Let u ∈ C(A)t ∩ C(B − A)t . Then u = At v = (B − A)t w for some column vectors v and w. So, ut = vt A = wt (B − A). This implies ut Q = vt AQ = wt (BQ − AQ). As BQ = A and AQ = A, we have ut Q = vt AQ = 0 and so, vt A = 0. It folT lows that u = 0. Thus, C(A)t C(B − A)t = {0}. (iii) ⇒ (i) Let (P, T) and (R, S) be rank factorizations of A and B − A respectively. So, B = A + (B − A) = PT + RS = (P : R)(Tt : St )t . Since C(At ) ∩ C((B − A)t ) = {0}, the matrix (Tt : St )t has full row rank. Let (L : M) be a right inverse of it. Therefore, TL = I, TM = 0; SL = 0 and SM = I. As P has full column rank, P has a left inverse say N. Then G = LN is a g-inverse of A and (B − A)G = RSLN = 0. Thus, BG = AG. Theorem 6.2.2. Let A and B be matrices of the same order. Then the following are equivalent: (i) A− A = A− B for some g-inverse A− of A (ii) A = PB for some idempotent matrix P and (iii) C(A) ∩ C(B − A) = 0. Proof is analogous to that of Theorem 6.2.1. In the following theorem we obtain some equivalent conditions under which the condition AA− = BA− yields the minus order. Theorem 6.2.3. Let A and B be matrices of the same order such that AA− = BA− . Then the following are equivalent: (i) (ii) (iii) (iv) (v)
A− A = A− B for some g-inverse A− of A A <− B C(At ) ⊆ C(Bt ) C(A) ∩ C(B − A) = 0 and A = PB.
Proof.
Use Theorems 6.2.1, 3.2.6 and 3.3.5.
We now obtain a canonical form for matrices A and B when the condition AA− = BA− holds. Theorem 6.2.4. Let A and B be matrices of order m × n. Let ρ(B) = s, ρ(A) = r and ρ(B − A) = t. Then AA− = BA− if and only if there exist
158
Matrix Partial Orders, Shorted Operators and Applications
non-singular matrices P and Q such that Ir 0 0 Ir T1 0 A = P 0 0 0 Q and B = P 0 T2 0Q, 0 0 0 0 0 0 T1 where an s × t matrix of rank t. T2 Proof. ‘If’ part Choose A− = Q−1 diag Ir , 0 , 0 P−1 . Then it is clear that AA− = BA− = P Ir , 0 , 0 P−1 . ‘Only if’ part Let (P1 , Q1 ) and (P2 , Q2 ) be rank factorizations of A and (B − A) respectively. Since AA− = BA− , C(At ) ∩ C(B − A)t = 0. Hence, C(Qt1 ) ∩ C(Q2 )t = 0. By choosing a basis of C P1 : P2 which includes the r columns of P1 , we can find a matrix R of order m × (s− r) with T1 ρ(P1 : R) = s and matrices T1 and T2 such that the matrix is of T2 I T1 order s × t and rank t and P1 , P2 = P1 , R . Since P1 , R 0 T 2 Q1 is a matrix of order m × s with rank s and is matrix of order Q2 (r + t) × n with rank r + t, we can find matrices P3 and Q3 such that Q1 P = P1 , R, P3 and Q = Q2 are non-singular. Now it is easy to see Q3 Ir 0 0 Ir T1 0 that A = P 0 0 0 Q and B = P 0 T2 0 Q. 0 0 0 0 0 0 Given a g-inverse A− of A such that AA− = BA− , we can now obtain the class of all g-inverses A− of A for which AA− = BA− . Theorem 6.2.5. Let A and matrices of the same ordersuch that B be Ir 0 0 Ir T1 0 − − AA = BA . Let A = P 0 0 0 Q and B = P 0 T2 0 Q where 0 0 0 0 0 0 T1 is of full column rank. Then for a g-inverse A− of A, AA− = BA− T2
One-Sided Orders
159
Ir L1 L2 if and only if A− is of the form A− = Q−1 0 0 0 P−1 , where N1 N2 N3 L1 , L2 , N1 , N2 , and N3 are arbitrary. Remark 6.2.6. One is tempted to believe that the relation defined by the condition AA− = BA− may be a pre-order. The belief, however, is put to rest by the following example: 1 0 2 1 3 1 Example 6.2.7. Let us take A = ,B= and C = . 0 0 0 0 1 0 1 a We can easily check that AA− = BA− for A− = , where a is −1 −a 0 0 arbitrary and BB− = CB− for B− = . However, AA− = CA− 1 1 for no g-inverse A− of A, so, the transitivity fails to hold. Our next theorem shows that the condition AA− = BA− is symmetric on the class of matrices that have same rank. Theorem 6.2.8. Let A and B be matrices of the same order. Consider the following: (i) AA− = BA− for some g-inverse A− of A. (ii) ρ(A) = ρ(B). (iii) BB− = AB− for some g-inverse B− of B. Then any two of (i), (ii), (iii) implies the third. Proof. (i) and (ii) ⇒ (iii) Let A− be a g-inverse of A such that AA− = BA− . Then, A = AA− A = BA− A. Further, ρ(A) = ρ(B), so, C(A) = C(B). So, B = AA− B = BA− B. Hence A− is a g-inverse of B and (iii) holds. (i) and (iii)⇒(ii) is obvious. Proof of (ii) and (iii) ⇒ (i) is similar to the proof of (i) and (ii) ⇒ (iii). Remark 6.2.9. In fact, we have proved a stronger statement than the one made in Theorem 6.2.8. If AA− = BA− for some g-inverse A− of A and ρ(A) = ρ(B), then A− is a g-inverse of B. Remark 6.2.10. Let C(B) = C(A) and AB− A = A for some g-inverse of B− of B. Then for any g-inverse A− of A, G = B− AA− is a g-inverse
160
Matrix Partial Orders, Shorted Operators and Applications
of A. Also, BG = AA− and BG = AG, showing that the condition AA− = BA− holds when A− = G. Similarly, let C(Bt ) = C(At ) and AB− A = A for some g-inverse of B− of B. Then GA = GB for some g-inverse G of A.
6.3
One-sided sharp order
We begin this section by defining one-sided sharp orders, namely left sharp and right sharp orders. We show each of these relations defines a partial order which implies the minus order. We also obtain some conditions under which each of the left and the right sharp order is equivalent to the sharp order. Definition 6.3.1. Let A and B be matrices of the same order and of index not greater than 1. We say A is below B under right sharp order, if A2 = BA and C(At ) ⊆ C(Bt ). We write A < # B, when this happens. Similarly, if A2 = AB and C(A) ⊆ C(B), we say A is below B under the left sharp order. We write A # < B, in case A is below B under the left sharp order. Remark 6.3.2. Notice that in Definition 6.3.2, as in the case of the sharp order, we do not require matrix B to be of index ≤ 1. However for same reasons as mentioned in the case of the sharp order, we shall henceforth take both A and B to be matrices of index ≤ 1. We shall now obtain a characterization of ‘< #’ using the matrix decompositions. Theorem 6.3.3. Let A and B be matrices of the same order and of index not greater than 1. Then A < # B if and only if there exists a non-singular matrix P such that S 0 0 S T12 0 A = P 0 0 0 P−1 and B = P 0 T22 0 P−1 , 0 0 0 0 0 0 where S and T22 are non-singular. Proof. ‘If’ part is trivial. ‘Only if’ part Since B is of index ≤ 1, we can find non-singular matrices R and T such
One-Sided Orders
161
A11 A12 R−1 , partitioned A21 A22 t in conformation for multiplication with B. Since, ) ⊆ C(Bt ), we C(A 2 A11 0 have A12 = 0 and A22 = 0. Now, A2 = R R−1 and A21 A11 0 TA11 0 BA = R R−1 . Since A2 = BA, this gives A211 = TA11 and 0 0 A21 A11 = 0. Further, ρ(A11 ) = ρ(TA11 ) = ρ(BA) = ρ(A2 ) = ρ(A), so, A21 = CA11 and CA211 = 0. Also, ρ(A2 ) = ρ(A) ⇒ C(A2 ) = C(A). Thus, A11 = A211 D for some matrix D. It follows that A21 = CA11 = CA211 D = 0. Now, index A11 ≤ 1, so, there exist non-singular −1 matrices S and Q of suitable orders such that A11 = Qdiag S 0 Q . Write T11 T12 T=Q Q−1 , where T11 and S are matrices of the same order. T21 T22 2 S 0 T11 S 0 Now, A2 = BA ⇒ A11 = TA211 ⇒ = . Therefore, 0 0 T21 S 0 T11 = S and = 0. Moreover, T is non-singular ⇒ T22 is non-singular. T21 Q 0 Let P = R and notice that A and B have the desired forms. 0 I that B = Rdiag(T, 0)R−1 . Write A = R
Theorem 6.3.4. Let A and B be matrices of the same order and index not greater than 1. Then A < # B if and only if there exists a non-singular matrix P such that S 0 S B12 A=P P−1 and B = P P−1 , 0 0 0 B22 where S is non-singular, B12 is arbitrary and B22 is arbitrary matrix of index ≤ 1. Since A is of index ≤1, we can find non-singular matrices P S 0 B11 B12 −1 −1 and S such that A = P P . Write B = P P , 0 0 B21 B22 2 partitioned inconformation with partitioning of A. Since A = BA, we 2 S 0 B11 S 0 have A2 = P P−1 and BA = P P−1 . It follows that 0 0 B21 S 0 S B12 B11 = S and B21 = 0. Hence, B = P P−1 , where B12 and B22 0 B22 are arbitrary. Since S is non-singular, it is easy to check that ρ(B) = ρ(B2 ) if and only if ρ(B22 ) = ρ(B222 ). Proof.
162
Matrix Partial Orders, Shorted Operators and Applications
Remark 6.3.5. Let A and B be matrices of the same order and index not greater than 1. If ρ(A) = ρ(B) and A < # B, then A = B. Theorem 6.3.6. Let A and B be matrices of the same order and index not greater than 1. Then A < # B if and only if there exists an A− χ such − − − that AA− = BA and A A = A B. χ χ χ χ Proof. ‘If’ part − − − − − 2 Let AA− χ = BAχ and Aχ A = Aχ B. Then AAχ = BAχ ⇒ A = 2 − 2 − − − AA− χ A = BAχ A = BA. Also, Aχ A = Aχ B ⇒ A = AAχ A = − t t AAχ B ⇒ C(A ) ⊆ C(B ). ‘Only if’ part Let A < # B. By Theorem 6.3.3, there exists a non-singular matrix S 0 0 S T12 0 P such that A = P 0 0 0 P−1 and B = P 0 T22 0 P−1 , 0 0 0 0 0 0 where S and T22 are non-singular. Notice that any A− χ is of the −1 S L M 0 0 0 P−1 for some matrices L and M. Also, form A− χ = P 0 0 0 I SL SM I SL SM AA− 0 P−1 and BA− 0 P−1 . Hence, χ = P 0 0 χ = P 0 0 0 0 0 0 0 0 I 0 0 − − − −1 and for every A− χ , AAχ = BAχ . Moreover, Aχ A = P 0 0 0 P 0 0 0 I S−1 T12 + LT22 0 A− 0 0 P−1 . χB = P 0 0 0 0 −1 − We take an A− for which L = −S T12 T−1 χ 22 . Thus, there exists an Aχ − such that A− χ A = Aχ B. Remark 6.3.7. It is clear that if T12 and L are non-null, then we cannot have AA# = BA# and A# A = A# B. In other words A cannot be below B under the sharp order. Analogous to Theorem 6.3.6, it is easy to prove that the left sharp order is same the ρ-order. Corollary 6.3.8. Let A and B be matrices of order n × n and of index not greater than 1. Then A # < B if and only if there exists an A− ρ such that
One-Sided Orders
163
− − − AA− ρ = BAρ and Aρ A = Aρ B.
Corollary 6.3.9. Let A and B be matrices of the same order and index not greater than 1. Then A < # B ⇒ A <− B and A # < B ⇒ A <− B. Corollary 6.3.10. The relation ‘< #’ (and also the relation ‘# <’) is a partial order on the set I1 of matrices of index 1. Proof. In view of Corollary 6.3.9, we only need to show that the relation ‘< #’ is transitive. Let A, B and C ∈ I1 such that A < # B and B < # C. Then by definition, A2 = BA and C(At ) ⊆ C(Bt ). and B2 = CB and C(Bt ) ⊆ C(Ct ). Clearly, C(At ) ⊆ C(Ct ). To prove A2 = CA, we use Theorem non-singular matrix P such that 6.3.4.As A < # B, there exists a S 0 S B 12 A=P P−1 and B = P P−1 , where S is non-singular 0 0 0 B22 C11 C12 and B22 is of index ≤ 1. Let C = P P−1 . Since, B2 = CB, C21 C22 we have C11 = S and C21 = 0. Clearly, CA = Pdiag S2 , 0 P−1 = A2 . Hence, A < # C. Remark 6.3.11. Notice that in Theorem 6.3.6, the matrix L is unique. − − Thus, when A < # B, there exists a unique A− χ such that AAχ = BAχ t −t − and A− χ A = Aχ B and C(Aχ ) ⊆ C(B ). Theorem 6.3.12. Let A and B be matrices of the same order and of index not greater than 1. Then A # < B if and only A# # < B# . Proof. Observe first that the group inverse of a block upper triangular matrix is also an upper triangular matrix. Now the proof follows by Theorem 6.3.3. Theorem 6.3.13. Let A and B be matrices of the same order and of index not greater than 1. Then the following are equivalent: (i) (ii) (iii) (iv) (v)
A<#B A <− B and A <− B and A <− B and A <− B and
C(BA) = C(A) C(BA# ) = C(A) C(B# A) = C(A) and C(B# A# ) = C(A).
Proof. (i) ⇒ (ii) − − − − Since (i) ⇒ there exists an A− χ such that AAχ = BAχ and Aχ A = Aχ B,
164
Matrix Partial Orders, Shorted Operators and Applications
it follows that A <− B. Also, A < # B ⇒ A2 = BA, so, C(BA) = C(A2 ) = C(A). Thus, (ii) holds. (ii) ⇒ (i) Since, A <s B and each of A and B have index ≤ 1, by Theorem 3.2.10, there exist matrices P, S and T of suitable non-singular order, such that S 0 0 T11 T12 0 A = P 0 0 0 P−1 and B = P T21 T22 0 P−1 for some non0 0 0 0 0 0 T11 T12 singular matrices P, S and T = . Since, C(BA) = C(A), it T21 T22 follows that T21 = 0. Further, since A <− B, T11 = S. So, (i) follows in view of Theorem 6.3.3. (ii) ⇔ (iii) We have C(BA# ) = C(BA(A3 )− A) ⊆ C(BA) ⊆ C(BA# A2 ) ⊆ C(BA# ). So, C(BA# ) = C(BA). It is now clear that (ii) ⇔ (iii) (i) ⇒ (iv) Since, A <− B, we have A = AB# A = AB# B = BB# A. Also, B# A = B# AA# A = B# A2 A# = B# BAA# = (B# BA)A# = AA# . Therefore, C(B# A) ⊆ C(A). Also, C(A) = C(AA# A) ⊆ C(AA# ) ⊆ C(B# A). So, C(B# A) = C(A). (iv)⇒ (i) is similar to (ii)⇒ (i). (iv)⇔ (v) Notice that C(B# A) = C(B# A# ). So, (iv)⇔ (v) holds. Corollary 6.3.14. Let A and B be matrices of the same order and of index not greater than 1. Then the following are equivalent: (i) (ii) (iii) (iv) (v)
A <# B A <− B A <− B A <− B A <− B
Proof.
and and and and
A, B commute A, B# commute A# , B commute and A# , B# commute.
Proof follows by Theorem 6.3.13 and Theorem 4.2.12.
Theorem 6.3.15. Let A and B be matrices of the same order and of index not greater than 1 such that A < # B. Then the following are equivalent: (i) A <# B (ii) C(ABt ) = C(BAt ) (iii) C(ABt ) = C(At ) and
One-Sided Orders
165
(iv) AB# = AA# . Proof.
Since A < # B, byTheorem 6.3.3, there exists a non-singular S 0 0 S T12 0 matrix P such that A = P 0 0 0 and B = P 0 T22 0 P−1 , 0 0 0 0 0 0 where S and T22 are non-singular. (i) ⇒ (ii) Let (i) hold. So, A2 = AB = BA. Therefore, 2 2 2 S 0 0 S ST12 0 S 0 0 0 0 0 = 0 0 0 = 0 0 0 . 0 0 0 0 0 0 0 0 0 It follows that T12 = 0. Clearly, C(ABt ) = C(BAt ). Thus, (ii) holds. (ii) ⇒ (iii) Next, let (ii) hold. So, ST12 = 0. Since, S is non-singular T12 = 0. Hence, C(ABt ) = C(At ) showing (iii) holds. (iii) ⇒ (i) Now, let (iii) hold. So, C(ABt ) = C(At ) = C(At )2 . Once again T12 = 0. So, A2 = BA = AB, which proves (i). Finally for (i) ⇔ (iv), use Theorem 6.3.3. Theorem 6.3.16. Let A and B be matrices of the same order and of index not greater than 1. Let A2 = BA. Then the following are equivalent: (i) (ii) (iii) (iv)
C(At ) ⊆ C(Bt ) C(A) ∩ C(B − A) = 0 A− A = A− B for some g-inverse A− of A A = PB for some projector P.
and
Proof. (i) ⇒ (ii) Since, A2 = BA and C(At ) ⊆ C(Bt ), (ii) holds by Corollary 6.3.9 and Theorem 6.2.2. (ii)⇒ (i) Let (P, Q) and (R, S) be rank factorization of A and B − A respectively. Q Then B = A + (B − A) = PQ + RS = P R . Notice that P : R S is of full column rank, since C(A) ∩ C(B − A) = 0. So, C(At ) = C(Qt ) ⊆ C(Qt : St ) = C(Bt ).
166
Matrix Partial Orders, Shorted Operators and Applications
(i)⇒ (iii) follows by Corollary 6.3.9. (iii)⇒ (iv) Since, A− A = A− B ⇒ A = AA− A = AA− B, so, A = PB, where P = AA− . (iv)⇒ (i) is trivial. Theorem 6.3.17. Let A and B be matrices of the same order and of index not greater than 1. The following are equivalent: (i) A# < B and ‘AA# and B’ commute (ii) A <# B.
and
Proof. (i) ⇒ (ii) Since A # < B, we have A2 = AB and C(A) ⊆ C(B). Now, pre- multiplying and post multiplying A2 = AB by A# and A respectively, we have A2 = A# ABA = AA# BA. Since AA# and B commute, we obtain A2 = BA. So, A2 = AB = BA proving A <# B. (ii) ⇒(i) is trivial. Remark 6.3.18. Notice that A < # B ⇒ (B − A) # < B. Further, C(B − A) ⊆ C(B), since A < # B ⇒ A <− B ⇒ B − A <− B. Theorem 6.3.19. Let A and B be matrices of the same order and of index not greater than 1. Then the following are equivalent: (i) A < # B and B − A < # B (ii) A <# B.
and
Proof is easy. Remark 6.3.20. Let A and B be matrices of the same order and of index not greater than 1. Then it is easy to see that A < # B ⇒ Ak < # Bk for each positive integer k ≥ 1 and therefore, A < # B ⇒ Ak <− Bk for each positive integer k ≥ 1. A similar statement holds for ‘# <’. We can now show that one sided sharp order coincides with the sharp order for the class of range-hermitian matrices. In the rest of this section we consider matrices over C, the field of complex numbers. Theorem 6.3.21. Let A and B be range-hermitian matrices of the same order. Then in notations of Corollary 3.7.4, A < # B if and only if A = Udiag Da , 0b−a , 0 U? and B = Udiag M, 0 U? ,
One-Sided Orders
167
where U is a unitary matrix, Da is positive definite matrix of order a ×a Da 0 and M is a non-singular matrix of order b × b of the form M = + 0 0 F1 M22 0 I , for some matrix F1 of suitable order. I Proof. ‘If’ part follows by direct verification. ‘Only if’ part Since A < # B ⇒ A <− B, by Corollary 3.7.4, A = Udiag Da , 0b−a , 0 U? and B = Udiag M, 0 U? , where U is a unitary matrix, Da is positive definite matrix and M is a non-singularmatrixof of order a × a and b × b respectively with M of the Da 0 F1 form M = + M22 F2 I , for some matrices F1 and F2 0 0 I of suitable orders. Since, A2 = BA, it follows that F2 = 0. Corollary 6.3.22. Let A and B be matrices of the same order. Let A be range-hermitian and B be normal. Then in notations of Corollary 3.7.4, A < # B if and only if A = Udiag Da , 0b−a , 0 U? and B = Udiag M, 0 U? , where U is a unitary matrix, Da is positive definite matrix and M is a non-singular matrix of oforder a × a and b × b Da 0 0 respectively with M of the form M = + M22 0 I , where 0 0 I M22 is non-singular matrix of order (b − a) × (b − a). Moreover, in this case A <# B. Corollary 6.3.23. Let A and B be range-hermitian matrices of the same order. Then A < # B if and only if A <# B or equivalently if and only if A and B are normal.
6.4
− Roles of A− c and Aa in one-sided sharp order
In this section we first introduce briefly two new classes of g-inverses, − namely: the class {A− c } and the class {Aa }. Details of their properties are given in exercises at the end of this chapter. We then show that A < # B − − − if and only if {B− c } ⊆ {Ac } and A # < B if and only if {Ba } ⊆ {Aa }. We begin with the following:
168
Matrix Partial Orders, Shorted Operators and Applications
Definition 6.4.1. Let A be a square matrix of index ≤ 1. A g-inverse G is called a c-inverse of A if C(GA) = C(A). A c-inverse of A is denoted by A− c. It is clear that every A− χ is a c-inverse of A. The following example shows that the converse is not true. However, every reflexive c-inverse is an A− χ. 1 0 Example 6.4.2. Let A = . Then it is easy to check that A− χ = 0 0 1 α 1 α and A− , where β is possibly a non-null scalar. It is now c = 0 0 0 β clear that a c-inverse may not be a χ-inverse. Definition 6.4.3. Let A be a matrix of order n × n and of index ≤ 1. Let Ax = b be a possibly inconsistent system of linear equations with b = c + d, where c ∈ C(A) and d ∈ N (A). (Since A is of index ≤ 1, we can write F n = C(A) ⊕ N (A), so each b ∈ Fn can be decomposed in the manner mentioned above.) We say x0 is a good approximate solution of Ax = b if Ax0 = c. Notice that in the setup of Definition 6.4.3, if b ∈ C(A), then x0 is a good approximate solution of Ax = b if and only if x0 is a solution of Ax = b or equivalently x0 is a solution of A2 x = Ab. Definition 6.4.4. Let A be a square matrix of index ≤ 1. A matrix G is said to be an A− a if Gb is a good approximate solution of Ax = b for each b. For a square matrix of index ≤ 1, a matrix G is an A− c if and only if G is an (At )− . a t
Definition 6.4.5. Let A and B be matrices of the same order and of index − − − − not greater than 1. We say A <− c B if AAc = BAc and Ac A = Ac B − − for some Ac ∈ {Ac }. Similarly we can define A <− a B. Theorem 6.4.6. Let A and B be matrices of the same order and of index not greater than 1. Then A < # B if and only if A <− c B. − − − Proof. Notice that for any A− c ∈ {Ac }, the matrix G = Ac AAc is a A− χ . The theorem now follows from Theorem 6.3.6.
One-Sided Orders
169
Corollary 6.4.7. Let A and B be matrices of the same order and of index not greater than 1. Then A # < B if and only if A <− a B. Theorem 6.4.8. Let A and B be matrices of the same order and of index not greater than 1. Then the following are equivalent: (i) A < # B − (ii) {B− c } ⊆ {Ac }. Proof. (i) ⇒ (ii) Since A < #B, by Theorem 6.3.3 there matrix P such exists a non-singular S 0 0 S T12 0 that A = P 0 0 0 and B = P 0 T22 0 P−1 , where S and T22 0 0 0 0 0 0 − − are non-singular. Also, A < # B ⇒ A <− B and {B− c } ⊆ {B } ⊆ {A }. − − To show that {B− } ⊆ {A− each B− c }, we show c is an Ac . Any Bc is of −1c −1 −1 S −S T12 T22 L1 2 the form P 0 0 L2 P−1 . Clearly, B− c A = A. So, 0 0 L3 − B− ∈ {A }. c c (ii) ⇒ (i) − − − Since {B− χ } ⊆ {Bc }, it follows that {Bχ } ⊆ {Ac }. Since B is of index 1, T 0 there exist non-singular matrices P and T such that B = P P−1 . 0 0 −1 T L − P−1 , where L is By Theorem 2.4.5 any Bχ is of the form P 0 0 A11 A12 arbitrary. Let A = P P−1 partitioned in conformation with A21 A22 − 2 − the partitioning of B. Since a B− χ is an Ac , so, Bχ A = A. Therefore, −1 2 T L A11 A12 A11 A12 −1 −1 P P P P = P P−1 for each L 0 0 A21 A22 A21 A22 and in particular for L = 0. Therefore, −1 2 T A11 + T−1 A12 A21 T−1 A12 A21 + T−1 A12 A22 P−1 P 0 0 A11 A12 =P P−1 A21 A22
170
Matrix Partial Orders, Shorted Operators and Applications
⇒ A12 = 0,A21 = 0, A22 = 0 and T−1 A211 = A11 . A11 0 So, A = P P−1 . As A is of index 1, A11 is of index 1. Now, 0 0 A11 is of index 1, so, there exists a non-singular matrix R and a non C 0 R 0 singular matrix C such that A11 = R R−1 . Let Q = P . 0 0 0 I C 0 0 T11 T12 0 Then A = Q 0 0 0 Q−1 and B = Q T21 T22 0 Q−1 , where 0 0 0 0 0 0 T11 T12 RTR−1 = and partitioning is in conformation with partitionT21 T22 11 12 T T −1 −1 −1 ing of A11 . Now, RTR is non-singular, let (RTR ) = . 21 T T22 11 12 T T L1 21 22 Then any B− is of the form P T T L2 P−1 . Using the fact that a χ 0 0 0 21 11 − − Bχ is an Ac , we get T = 0 and T = C−1 , so, T11 = C, T21 = 0 and T22 is non-singular. Now, it is clear that A2 = BA and C(At ) ⊆ C(Bt ). Thus, A < # B. The following theorem can be proved analogously: Theorem 6.4.9. Let A and B be matrices of the same order and of index not greater than 1. Then the following are equivalent: (i) A # < B − (ii) {B− a } ⊆ {Aa }. Theorem 6.4.10. Let A and B be matrices of the same order and index not greater than 1. Then the following are equivalent: (i) (ii) (iii) (iv)
A <# B − − − {B− c } ⊆ {Ac } and {Ba } ⊆ {Aa } − − − {B− χ } ⊆ {Ac } and {Bρ } ⊆ {Aa } # − B ∈ {Aac }.
and
Proof. We only need to prove (i) ⇔ (iv). # − − − − Since B# ∈ {A− ac } ⇔ B ∈ {Bc } ∩ {Ba } ⊆ {Ac } ∩ {Aa }, the proof of (i) ⇒ (iv) is clear. (iv) ⇒ (i)
One-Sided Orders
171
it is easy to check that the class of all commuting g-inverses of A is precisely {A− ac }. Now, (iv) ⇒ (i) follows from Theorem 4.2.8. 6.5
One-sided star order
The one-sided star orders maintain a similar relationship to one-sided sharp orders as did the star order to the sharp order. For hermitian matrices the right (left) sharp order and the right (left) star coincide. So, as expected, many theorems in this section will be similar to theorems in one-sided sharp order. However, a distinguishing feature is that there is no Fisher-Cochran type theorem for one-sided sharp orders, but for one-sided star order we show that it holds good. The important feature of this section is that: if A and B are matrices of the same order and A is below B under either the right star order or the left star order, then their reduced form (a canonical form) is generalized singular value decomposition of A with respect to B. In this section all matrices are over the field C of the complex numbers. Let us define for matrices A and B of the same order, A ≺ B if AA? = BA? (or equivalently AA† = BA† ). This relation is obviously reflexive. Moreover, if A ≺ B and B ≺ A, then (B − A)A? = 0 and (B − A)B? = 0. Therefore, (B − A)(B − A)? = 0 and so, A = B. Thus, A ≺ B is antisymmetric. However, this relation is not transitive as the following example shows: Example 6.5.1. Consider matrices A = , B = and C = 1 , −1 2 , 0 ? ? ? ? 2 , 1 . Then AA = 2 = BA and BB = 4 = BC , but AA? = 2 6= 1 = AC? . Thus, the relation ‘≺’ is not transitive and so, is not even a pre-order. Definition 6.5.2. Let A and B be matrices of same order. We define A < ? B (read as A is below B under right star order ) if AA? = BA? and C(A? ) ⊆ C(B? ) and A ? < B (read as A is below B under left star order ) if A? A = A? B and C(A) ⊆ C(B). Are the two relations defined above the same and the same as star order? Does each of them define a partial order? Or even a pre-order? We give below an example which not only shows that none of them is same as the star order and that they themselves are different order relations. 1 −1 1 −2 Example 6.5.3. Let A = and B = . Then clearly, −1 1 −1 0
172
Matrix Partial Orders, Shorted Operators and Applications
2 −2 C(A) ⊆ C(B) and A? A = A? B = . However, AA? = A? A and −2 2 3 −3 BA? = . Thus, A ? < B, A ≮ ?B and therefore, A ≮? B. −1 1 Remark 6.5.4. Notice that A < ? B ⇔ AA† = BA† and C(A? ) ⊆ C(B? ), since, A† = A? (A? AA? )− A? . Thus, A < ? B ⇒ AA† = BA† , and C(A? ) ⊆ C(B? ) and this implies A <− B, C(A? ) ⊆ C(B? ), C(A) ⊆ C(B) ? and C(A? )⊥C(B − A) . This is so because AA† = BA† ⇒ A = BA† A and therefore, AB† A = AB† BA† A = A; thus, AB† A = A and so, A <− B. Further, A(B? − A? ) = 0. Remark 6.5.5. A < ? B ⇒ A <− B and A ? < B ⇒ A <− B. We shall now first obtain a canonical form for matrices A and B when A < ? B. Theorem 6.5.6. Let A and B be matrices of the same order with ranks ‘r’ and ‘s’ respectively. Then A < ? B if and only if there exist a non-singular matrix P and a unitarymatrix V such that A = Pdiag Ir , 0 , 0 V? and B = Pdiag Ir , Ir−s , 0 V? . Proof. ‘If’ part is trivial. ‘Only if’ part Let A < ? B. Then by Remark 6.5.4, A <− B, C(A? ) ⊆ C(B? ), C(A) ⊆ ? C(B) and C(A? )⊥C(B − A) . Let (P1 , V1? ) and (P2 , V2? ) be rank factorizations of A and B − A respectively, where V1? V1 = I and V2? V2 = I. Then B = A + (B − A) = P1 V1? + P2 V2? = (P1 : P2 )(V1 : V2 )? . Since C(A) ∩ C(B − A) = {0}, (P1 : P2 ) is of full column rank ‘s’. Also, since ? C(A? )⊥C(B − A) , so,(V1 : V2 )? (V1 : V2 ) = I. Let P3 and V3? be matrices such that P = (P1 : P2 : P3 ) is non-singular and V? = (V1 : V2 : V3 )? is unitary. It is now easy to check that A and B have the desired forms. Remark 6.5.7. The canonical form of the matrices A and B in Theorem 6.5.6 is called a generalized singular value decomposition of A and B. Remark 6.5.8. Let A and B be matrices of the same order with ranks‘r’ and ‘s’ respectively. Then A ? < B if and only if A = Udiag Ir , 0 , 0 Q and B = Udiag Ir , Ir−s , 0 Q, for some non-singular matrix Q and unitary matrix U.
One-Sided Orders
173
We shall now obtain a characterization of A such that A < ? B when a singular value decomposition of B is available. Theorem 6.5.9. Let A and B be matrices of the same order. Let B = Udiag D , 0 V? , where U, V are unitary and D is a positive definite diagonal matrix. Then A < ? B if and only if A = Udiag T , 0 V? such that D−1 is a minimum norm g-inverse of T. Proof. ‘If’ part is trivial. ‘Only if’ part ? T E Let B = Udiag D , 0 V . Write A = U V? where partitioning F G is in conformation with the partitioning of B. Since C(A? ) ⊆ C(B? ), we have E = 0 and G = 0. Also, AA? = BA? . So, TT? = DT? and FF? = 0. This further gives D−1 TT? = T? and F = 0. Hence, D−1 is a minimum norm g-inverse of T, showing that A has the desired form. Theorem 6.5.10. Let A and B be matrices of the same order. Let A = Udiag ∆ , 0 V? , where U, V are unitary and ∆ is a positive definite ∆ T12 diagonal matrix. Then A < ? B if and only if B = U V? for 0 T22 some matrices T12 and T22 such that C(T12 ) ⊆ C(T22 ). Proof.
Proof is similar to Theorem 6.5.9.
Theorem 6.5.11. The relation ‘< ?’ is a partial order. Proof. The relation is trivially reflexive and as noted earlier it is also antisymmetric. So, we only need to show that it is transitive. Since A < ? B ⇒ AA? = BA? and C(A? ) ⊆ C(B? ). Also, B < ? C ⇒ BB? = CB? and C(B? ) ⊆ C(C? ). Clearly, C(A? ) ⊆ C(B? ) and C(B? ) ⊆ C(C? ) ⇒ C(A? ) ⊆ C(C? ). Moreover, AA? = BA? = AB? = AB? B†? B? = AB† BB? = AB† BC? = AC? = CA? . Therefore, A < ? C. Remark 6.5.12. The relation ‘? <’ is a partial order. Remark 6.5.13. Notice that A < ?B ⇔ A? ? < B? Theorem 6.5.14. Let A and B be matrices of the same order. Then A
174
(i) (ii) (iii) (iv) (v)
Matrix Partial Orders, Shorted Operators and Applications
A
Proof follows along the lines of Theorem 6.3.16. The following theorem follows trivially from Remark 6.5.13 and Theorem 6.5.15. Theorem 6.5.16. Let A and B be matrices of the same order such that A? A = A? B. Then the following are equivalent: (i) (ii) (iii) (iv) (v)
A?< B C(A) ⊆ C(B) C(A? ) ∩ C(B − A)? = {0} AA− = BA− for some g-inverse bf A− of A A = BQ for some projector Q.
and
− − In Theorem 5.2.8, we saw that A
{A− m}
Theorem 6.5.17. Let A and B be matrices of the same order. Then − A < ? B if and only if {B− m } ⊆ {Am }. Proof. ‘If’ part Let B = Udiag D 0 V? be a singular value decomposition of B, where U and V are unitary and D a positive definite diagonal matrix. Then is−1 D M U? , where M and N are arbitrary. every B− m is of the form V 0 N T11 T12 − Write A = U V? . Since, {B− m } ⊆ {Am }, we have T21 T22 V
? ? D−1 M T11 T12 T11 T?12 T11 T?12 U? U V? V V? = V V? . ? ? ? ? 0 N T21 T22 T21 T22 T21 T22
In particular, by taking M = 0 and N = 0 and simplifying we have
D−1 T11 T?11 + D−1 T12 T?12 0
D−1 T11 T?21 + D−1 T12 T?22 0
=
T?11 T?12 . ? ? T21 T22
So, T?12 = 0, T?22 = 0 and D−1 T11 T?11 = T?11 , and D−1 T11 T?21 = T?21 .
One-Sided Orders
175
Next, by taking M = 0 and simplifying we have NT21 T?21 = 0 for all N. So, T21 = 0. Hence, A = Udiag T11 0 V? such that D−1 is a minimum norm g-inverse of T11 . Therefore by Theorem 6.5.9, we have A < ? B. ‘Only if’ part Let A < ? B. Let B = Udiag D 0 V? be a singular value decomposition of B, where U and V are unitary and D is a positive definite diagonal matrix. Then by Theorem 6.5.9, A = Udiag T 0 V? such that D−1 TT? = T? . The class all B− m is given −1 D M ? by V U? , where M and N are arbitrary. Now, B− m AA = 0 N −1 ? ? D M T 0 T 0 T 0 ? ? ? U = V U? = A? . V U U V V 0 N 0 0 0 0 0 0 − Hence, {B− m } ⊆ {Am }. − Remark 6.5.18. In fact, we can prove that A < ? B ⇔ {B− mr } ⊆ {Am }. † The proof follows along the lines of Theorem 6.5.17. However, B ∈ {A− m} does not imply A < ? B as the following example shows. 1 0 1 0 Example 6.5.19. Let A = and B = . Then B† = B 1 0 0 0 1 1 ? }. However, AA = and BA? = and B† AA? = A? . So, B† ∈ {A− m 1 1 1 1 . So, AA? 6= BA? . Thus, A ≮ ? B. 0 0
Lemma 6.5.20. Let C be a non-singular matrix. Then C is a minimum norm g-inverse of a matrix H if and only if C−1 is a minimum norm g-inverse of H† . Proof. C is a minimum norm g-inverse of a matrix H ⇔ CHH? = H? ⇔ CHH† = H† ⇔ HH† = C−1 H† ⇔ H†? = C−1 H† H†? ⇔ C−1 ∈ {(H† )− m }. Theorem 6.5.21. Let A and B be matrices of the same order. Then the following are equivalent: (i) (ii) (iii) (iv)
AA? = BA? AA† = BA† A = BL, where L is an orthogonal projector on C(A? ) A = BL, where L is an orthogonal projector having range a subspace of C(B? ) and
176
Matrix Partial Orders, Shorted Operators and Applications
(v) A = BL, where L is an orthogonal projector. Proof. (i) ⇒ (ii) AA? = BA? ⇒ AA? A†? A† = BA? A†? A† ⇒ AA† = BA† . (ii)⇒(iii) AA† = BA† ⇒ A = AA† A = BA† A. Take L = A† A. (iii) ⇒ (iv) ⇒(v) is trivial. (v)⇒ (i) Let A = BL, where L is an orthogonal projector. Then A? = LB? . So, C(A? ) ⊆ C(L). This gives A? = LA? . Therefore, AA? = (BL)A? = BA? . Remark 6.5.22. Any one of the five equivalent conditions in Theorem 6.5.21 together with C(A? ) ⊆ C(B? ) implies A < ? B. Remark 6.5.23. None of the five equivalent 6.5.21 conditions in Theorem 1 0 1 1 ? ? implies C(A ) ⊆ C(B ). For take A = and B = . 1 0 1 1 Remark 6.5.24. Compare Theorem 6.5.15 and Theorem 6.5.21. Theorem 6.5.25. Let A and B be matrices of the same order. Then the following are equivalent: (i) (ii) (iii) (iv)
A
B B and BA? is hermitian B and BA† is hermitian B and B† A is hermitian.
and
Proof. (i) ⇒ (ii) Proof is trivial in view of Remark 6.5.5 and definition of A < ? B. (ii) ⇒ (i) and ⇒ (i) (iv) D 0 Let B = U V? be a singular value decomposition of B, where 0 0 U and V are unitary and D is a positive definite diagonal matrix. In view of Theorem 6.5.9, the proof will be complete if we show that each of T 0 (ii) and (iv) implies that A is of the form U V? , where D−1 is 0 0 T E a minimum norm g-inverse of T. Let A = U V? partitioned in F G conformity with the partition of B. Then BA? is hermitian ⇒ F = 0 and A <− B ⇒ G = 0 and D−1 is a g-inverse of T and DT is hermitian.
One-Sided Orders
177
Since, C(A? ) ⊆ C(B? ), we must have E = 0. So, D−1 is a minimum norm g-inverse of T. Similarly, when B† A is hermitian, E = 0 and using C(A) ⊆ C(B), we have F = 0 (i) ⇒ (iii) Proof is trivial in view of Remark 6.5.5 and definition of A < ? B. (iii) ⇒ (i) Let A = Udiag(∆ 0)V? bea singular value decomposition of A. Then it ∆ T12 is easy to show that B = U V? for some matrices T12 and T22 0 T22 such that C(T12 ? ) ⊆ C(T22 ? ). So, (i) follows by Theorem 6.5.10. Theorem 6.5.26. Let A and B be matrices of the same order. Then the following are equivalent: (i) (ii) (iii) (iv)
A
B B and A? B is hermitian B and A† B is hermitian B and AB† is hermitian.
and
Definition 6.5.27. Let A and B be matrices of same order. We say − − − − − − A <− ` B if AA` = BA` and A` A = A` B for some A` ∈ {A` }. − − − − − − Also, A <m B if AAm = BAm and Am A = Am B for some Am ∈ {A− m }. Theorem 6.5.28. Let A and B be matrices of the same order. Then A < ? B if and only if A <− m B. Proof. ‘If’ part − ? − − − Now, A <− m B ⇒ AAm = BAm for some Am ∈ {Am }. So, AA = − ? − ? ? − − ? AAm AA = BAm AA = BA Also, A <m B ⇒ A < B ⇒ C(A ) ⊆ C(B? ), hence, A < ? B. ‘Only if’ part We first note that A? (AA? )− is a minimum norm g-inverse of A. Since, A < ? B ⇒ AA? = BA? ⇒ AA? (AA? )− = BA? (AA? )− for any g− − ? ? − inverse (AA? )− of AA? . Therefore, AA− m = BAm for Am = A (AA ) . − † ∼ † † Also, A < ? B ⇒ A < B ⇒ A = AB A. Let A = A AB . Then A∼ AA? = A† AB† AA? = A† AA? = A∼ . Hence, A∼ is a minimum norm g-inverse of A. Now, A∼ B = A† AB† B = A† A = A∼ A. Thus, A <− m B. Theorem 6.5.29. Let A and B be range-hermitian matrices of same order. Then in notations of Theorem 5.4.1, A < ? B if and only if
178
Matrix Partial Orders, Shorted Operators and Applications
A = Udiag Da , 0b−a , 0 U? and B = Udiag M, 0 U? , where U is unitary, Da and M are non-singular matrices of order a × a and b × b Da , 0 F1 respectively with M of the form M = + M22 0 I , for 0, 0 I some matrix F1 and a non-singular matrix Mb of order (b − a) × (b − a). Proof.
Proof follows from Theorem 5.4.1(i) and Definition of A < ? B.
We now show that a one-sided star order is the star order if and only if the matrices involved are both normal. Thus, the class of the normal matrices is precisely the class of matrices for which the one-sided star order coincides with the star order. Remark 6.5.30. Let A and B be normal matrices of order. same ? Then A < ? B if and only if A = Udiag Da 0b−a 0 U and B = Udiag Mb 0 U? , where U is unitary, Da and Mb are non-singular matricesof order a × a b × b respectively with Mb of the form Mb = Da 0 0 + M22 0, I , for some non-singular matrix Mb of order 0 0 I (b − a) × (b − a). Thus, it follows that A
Ai Aj ? = 0, whenever i 6= j AAi ? = Ai Ai ? AAi † = Ai Ai † AAi ? Ai = Ai C(Ai ) ⊆ C(A) P ρ(A) = ρ(Ai ) C(Ai ) ∩ C(Aj ? ) = {0} for i 6= j
One-Sided Orders
179
(viii) AAi ? = Ai A? Then (i) ⇒ (ii) ⇒ (iii) ⇒ (iv) ⇒ (v). (i) ⇒ (vii), (iv) ⇒ (vi) and (i) ⇒ (viii). Proof.
Proof is straightforward. (Recall A† = A? (A? AA? )− A? .)
180
6.6
Matrix Partial Orders, Shorted Operators and Applications
Exercises
(1) Let A, B be range-hermitian matrices. Show that if either A2 = BA or A2 = AB and B is an orthogonal projector, then A is also an orthogonal projector. (2) Show that for square matrices A and B of the same order and of index − − − ≤ 1, A < #B < {B− χ } ⊆ {Aχ } and A# < B < {Bρ } ⊆ {Aρ }. (3) Let A and B be square matrices of the same order and of index ≤ 1 such that AA− = BA− (or A− A = A− B) for some g-inverse A− of A and AB = BA. Show that A <# B. (4) Let A and B be square matrices of the same order and of index ≤ 1 such that AA− = BA− (or A− A = A− B) for some g-inverse A− of A and A2 = AB (respectively A2 = BA). Then show that A <# B. (5) Let A and B be matrices of the same order m × n over an arbitrary field. Prove that AA− = BA− (or A− A = A− B) is invariant under choice of g-inverse A− if and only if C(Bt ) ⊆ C(At ) (or C(B) ⊆ C(B)). Show that if A <− B, the condition that each of the defining conditions is invariant under choices of A− implies A = B. (6) Let A and B be square matrices of the same order and of index ≤ 1 such that (i) A2 = BA (or A2 = AB) and (ii) BA# B = A. Then prove that A <# B. What happens if the condition (ii) is replaced by (ii)0 B# AB# = A# ? Should the same result hold if the group inverse (i) and replaced by a commuting g-inverse? (7) Let A and B be matrices of the same order m × n over the field of complex numbers such that (i) A− A = A− B and (ii) BA† B = A. Prove that A
One-Sided Orders
(i) (ii) (iii) (iv)
181
A <− B A? < B A < ?B A
(11) Let A and B be square matrices of the same order, A be a rangehermitian matrix and B a normal matrix. Show that (i) A < ?B if and only if A
and
182
Matrix Partial Orders, Shorted Operators and Applications
(iv) Gt At is a projector that projects vectors into C(At ) along N (At ). (19) Find the class of all A− a for the matrix A in question no. 15.
Chapter 7
Unified Theory of Matrix Partial Orders through Generalized Inverses
7.1
Introduction
In Chapters 3-5, we studied three major matrix partial orders namely: the minus, star and sharp orders. In each of these cases, we saw that they possess a similar characterizing property in terms of a suitable subclass of g-inverses: a matrix A is below a matrix B if AG = BG and GA = GB for some matrix G belonging to a suitable subclass of g-inverses of A. For the minus, star and sharp orders, the subclasses are {A− }, {A− `m } and {A− } respectively. A vigilant reader must have noticed that the three ac partial orders mentioned above have a number of characterizing properties which are analogous. We recall a couple of them here. If A <− B, then (i) {B− } ⊆ {A− } and (ii) there exist projectors P and Q such that A = PB = BQ. If A <# B, then − (i) {B− ac } ⊆ {Aac } and (ii) there exists a projector P onto C(A) such that A = BP = PB. Also, if A
184
Matrix Partial Orders, Shorted Operators and Applications
In the present chapter, we first develop a unified theory of matrix partial orders that are defined via g-inverses. We then attempt to obtain characterizing properties similar to the ones mentioned earlier under the unified theory. We also develop a unified theory based on subclasses of outer inverses. We finally consider some extensions of the above mentioned unifications. In Section 7.2, we define g-maps and a G-based order relation (which includes as special cases the minus, the star and the sharp orders) on matrices based on subclasses of generalized inverses. We study the conditions under which a G-based order relation is a partial order. We also show that not every partial order defined using subclasses of g-inverses is a G-based order. The order relation ‘<#,− ’ discussed in Section 4 of Chapter 4 provides a nontrivial example of the same. Section 7.3 is devoted to the study of orders based on the subclasses of outer inverses. We also study when an order relation based on a subclass of g-inverses coincides with an order relation based on a subclass of outer inverses. In Section 7.4, we develop a unified theory of one-sided orders. In Section 7.5, we study some characterizing properties of G-based partial orders similar to those mentioned in the first paragraph above. Theorem 7.5.5 provides us a true unification of almost all the partial orders via g-inverses that we studied so far. Section 7.6 deals with extensions of G-based orders to accommodate some orders that are not G-based. 7.2
G-based order relations: Definitions and preliminaries
Let ‘<’ denote any of the partial orders studied in earlier chapters. Recall that they all shared a common property: A < B ⇔ some specific subset of g-inverses of B is a subset of a suitable subset of g-inverses of A. Keeping this in view, we introduce in this section the notion of a G-map and a G-based order relation. We study the properties of this order relation and examine when it defines a partial order at least on its support to be defined shortly. We also give appropriate interpretations of the G-based order relation for most of the partial orders studied earlier. Definition 7.2.1. Let P(Fn×m ) denote the power set (class of all subsets) of Fn×m . A g-map is a map G : Fm×n −→ P(Fn×m ) such that for each A ∈ Fm×n ,
G(A) is a certain subset (possibly non-
Unified Theory of Matrix Partial Orders through Generalized Inverses
185
empty) of {A− } and the set ΩG = {A ∈ Fm×n : G(A) 6= ∅} is called the support of the g-map G. Remark 7.2.2. Notice that the set G(A) is not always non-empty. For instance, let G(A) = {A# } for each A ∈ Fn×n . If A is not of index 1, then G(A) = ∅. Definition 7.2.3. Let G : Fm×n −→ P(Fn×m ) be a g-map. For A, B ∈ Fm×n , we say A
Example 7.2.7. Consider the order relation ‘<#,− ’ of Definition 4.4.17. We show that this orderrelationis not a G-based an order relation. Let 0 0 0 0 0 0 0 0 1 1 0 0 0 1 1 0 4×4 A= . −1 − 1 − 1 0 , B = 0 1 0 0 ∈ R 1 1 1 0 0 0 1 0 Clearly, A2 = 0 and B4 = 0, hence, A and B are nilpotent. Also, ρ(A) = 1, ρ(B) = 3, and ρ(B − A) = 2. So, A <− B. Since A and B are nilpotent, it follows that A<#,− B.
186
Matrix Partial Orders, Shorted Operators and Applications
If the relation <#,− were a G-based order relation, then there exists a G ∈ G(A), such that GA = GB, AG = BG. Let T = 2B − A, then G(T − A) = 0, (T − A)G = 0. So, A<#,− T. However, T = 0 0 0 0 1 − 1 − 1 0 , ρ(T) = 3, ρ(T2 ) = 2 = ρ(T3 ). Thus, T is of 1 3 1 0 −1 − 1 1 0 index 2, so, CT , the core part is of rank 2 and NT , the nilpotent part is of rank 1. Since A<#,− T, we have 0 = CA <# CT and A = NA <− NT . It follows that A = NA = NT , which is impossible. Since T − A = CT , so, ACT = AT and CT A = TA should both be null. However, an easy computation shows that neither is a null matrix. In view of Remarks 7.2.5, it is clear that the G-based order relation is always anti-symmetric and reflexive on its support. Since our main objective here is to study partial orders on matrices, it is very natural to ask if a G-based order relation is transitive at least on its support ΩG , if not on its domain. We remark that if the support, ΩG of a g-map consists of exactly two elements, then the g-map always induces a partial order on ΩG . However, if ΩG contains three or more elements, the answer is no as the following example shows: Example 7.2.8. Let G : C3×3 −→ P(C3×3 ) be the map † A , if ρ(A) = 1 G(A) = − A , otherwise.
A
Notice that ρ(A) = 1, ρ(B) = 2 and ρ(C) = 3. Further, A
