Studnes ffiil by SydneyAfriat M.V.RamaSastry GerhardTintner
Vandenhoeck & Ruprecht
Studiesin Correlation Multivariate...
53 downloads
995 Views
4MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
Studnes ffiil by SydneyAfriat M.V.RamaSastry GerhardTintner
Vandenhoeck & Ruprecht
Studiesin Correlation Multivariate Analysisand Econometrics
by SydneyAfriat, M. V. Rama Sastry and GerhardTintner
Vandenhoeck& Ruprechtin Gcittingen
Contents
lntroduction. Sydney Afriat Regression and Projection.
Studies in the Algebra of 10
Statistical Correlation. M. V, Rama Sastry and Gerhard Tinher Multivariate
Bibtiography
Analysis.
Econometrics
and Infonnation
Theory
' ' ' ' ' 108
"
' 142
Abstract
The first part gives a geometric and algebraic background to general methods of regression and correlation,
taking into account particulary methods of
multivariate analysis. In the second part Canonical correlation, root criterion, of information
largest
principal components, weighted regression, the relation theory to econometric
estimation and to canonical correlation
and trace correlation is considered. Bayes estimation or regression models is also briefly discussed.
Introduc t ion
s c h e m et o s h o w l e a s t - s q u a r e s l i n e a r regression as resolving sanple vectols of dependent variables i-nto two orthogonal components, a regressional component It
is a familiar
in a space spanned by sample vectors of the i . n d e p e n d e n tv a r i a b l e s , a n d a r e s i d u a l c o n p o n e n t l y i n g lying
in
t h e o r t h o g o n a l c o m p l e m e n t o f t h a t s p a c e . T h i - s s c h e m ew a s i n t r o d u c e d b y K o l m o g o r o v ( 1 9 4 6 )' B u t t h e probably first step from this
to considering
the linear
transformations
these athogonal conponents of sample vectors, these being the complementary pair of symmetri-cidempotents which are the orthogonal projectors on the space and its orthogonal conplement, and then to analysis of relations
which obtain
between variables directly in terms of these projectols' is one that does not appear to have been explored' The advantage of this step apart fron its showing the way to new concepts is that it makes an enlargement methods j.n a natural formalism for handling sone
some possibly of didactic farniliar
theory vrith a direct
geonetrical
neaning, and it
di-sp1ay of
its
algebraical-
gives the framework for
simple
proofs and formulae. Proofs of some relevant algebraical propositions have already been gi-ven elsewhere (Afriat' 1957 and 1956). The view taken of regressl-onanalysis is concepts. orthogonal one that does not depend on distribution projection is presented as the fundamental princi-p1e and the least-squares principle is derived as a property of it. The association of a di-stribution with a sanple is a refinements of this view, since and it can also be considered to justify it'But this vlew can just as well be taken as primitive, in its compatibility with it the distribution nethod can be taken
method that yields
certain
The only to have a crj-tical part of its own justification. that can be given to the probability claim to priority approach ls that it is part of a more general distribution artifice
for
the interpretation
o f d a t a . W h e nr e f e r e n c e
-
is made to a variate,variance, which terns properly distribution,
all
d
-
covariance or correlation,
belong to concepts relating
that is neant is
is determined on objects
to a
a variable whose value
and certain
functj-ons of value
which, when a variable
l-s interpreted
sanple as a population
sample, can also be lnterpreted
as a variate
and a
as sanple neasures which, in the case of normality, respond in the usual way to those things Hotelling
(1935) has given an analysis of the relation
between two sets of variates theory.
cor-
in the population.
in his
c a n o n i - c a 1c o r r e l a t i o n
It
can be fornulated as an analysis of the relative of two sub-spaces of a Euclidean space, spanned b y v e c t o r s o f r n e a s u r e m e n t s ,a s h e h a s p o i n t e d o u t . T h i s position
analysis
provides a cornplete set of orthogonal invariants
which characterize the figure
formed by the spaces, in terms of a set of angles determined between them. Such a characteri-zation of a pair of spaces by angles was demonstrated by Jordan (1875), using synthetic methods; and a further
such account has been given by Somerville (j929). Algebrai-ca1 method for determination of the angles has been investigated by Schoute (1905), and more recently by Flanders (1948). Hotelling's
theory shows another
approach, equivalent to consideration of directions whose variation in the spaces leaves the angle between thern stationary,
but this
a p p r o a , c hd o e s n o t a c c o u n t f o r
the sub-space in each space which are orthogonal to the other space. A further rnethod, shownby Afriat (1956 and ,l957), j.s in accordance with the nethod of the present investigation, and proceeds entirely values and latent
by a consideration of the characteristic
vectors of the pair
orthogonal projectors
of products of the
on the spaces.
l v h i1. e a c o m p l e t e s e t o f c a n o n i c a l a n g l e s , o r t h e i r
- 9 -
cosines which are the canonical correlations, spaces, or the sets of variates,
there are coefficients
that express sumnaryaspects of this
relation.
which have such a nature,
and alienation
identical
and which can be seen to be algebraically the here-defined coefficients one space with projectors
another. Presented in terms of the orthogonal
algebraical properties properties
are immediately clear.
for
any numberof variates
coefficient
a set of several, this the correlation
itself
coeffi-cient
(1957). It it
and
being a genera)izationof
of Pearson defined between a have been derived
coefficient
h a s h a d s u b s e q u e n tc o n s i d e r a t i o n ,
though with a different has applied
of the multiple
defined between one variate
T" ^h" c n r ronerties of this
by Afriat
S o m en e w
represents a
shown. One of these coefficients
correlation
n:ir^
geornetrical rneaning and
kind are here introduced, and their
of this
generalizatlon
with
and separation of
of inclusion
on the spaces, their
coefficients
l{i1ks
(1935) have defined coefficients
(1932) and Hotelling of correlation
between the
of the relation
complete specification
glves a
derivation,
by Hooper (1959), who
to the sirnultaneous equation method in
econonetrics. A multivariate generaTization of the concept oI variance which has been studied by Wilks (1932) and Anderson (1958) enables an alternative the method of least
characterization
to be gi-ven for
squares. A formula is
shown vihich
expresses the variance of a set of variates the variances for
conplementary subsets, together with
here-defined coefficient Geometrically, this Formula for
in terms of the
of separation between them.
is a generalization
of the familiar
the area of a parallelogram, in terms of the
lengths of a pair
of edges and the sine of the angle between
them. A multidirnensional generalization
is shownalso for
the formula which gives the area as the product of base with height.
RECRESSIA ON N DP R O J E C T I O N Studies
tlie Algebra
in
and Ceometry
Correlati.on
of Statistical
by Sydney AFRIAT Part I Prediction Let U be a unlverse of objects, and 1et Y be a set of i n t h e r e a l n u n b e r f i e l d K r r ' h o s ev a l u e s c a n
variables
Then ll] K can denote
be determincd on anv ohject in U.
V determined on anv object
the value of an1'variable X a
LI.
,\ sanple of size N consists in a set of N objects taken .l , . . . , N. .,\nexperi-ment deterrnines the values of variables in V on every object in f r o r n I J , r n , hci h c a n b e d e n o t e d
the sample. is
In particular if
characteristic
there is one variable u that identified
the sample is
with the
u n i f o r m c o n di t i o n o f t h e e x p e r i m e n t , t h a t t a k e s t h e v a l u e I on everv ohject, is
The samolesnacc for the exDerintent the Iruclidean space E = K" , of dimension equal to the
samolesize. for
Tlie sample vector r"hose elenents tlre
of
\
n v uhJ i c we !t s
11] (i
).
in
[]19
a variahle
=1,
.. ., ^) d ^l l *t P- rl U .^
t h e u n i f o r n r c o . n d it i o n
A multivarialrlc its
basic
is
components.
are
defined
the
n-t'ector
l\lael
N) are the rralues of x on rr r"r . - ^ - +L ii L cu rra ra l a r tL hr r cv rsa nr . nr Pna l\ e V e c t o I I ' d t
variable
the n-vector whose elenents
x is
u is all
ll
u
equal
by an1' set
Thr.rsa p-l'ariable
= I lvhere I to
of
is
1.
variables,
can be denoted
ca11ed
-1 1 *p), so a l-variable is then just one
(x1, ..., in V.
variable
sanple natrix
A 1r-variable has associated with it I . 1 -= ( I 1 - , . . . , I l - ) o f o r d e r N * 1 , ^r
1
p conponents.
w h o s ep c o l u m n sa r e t h e l s a m p l " u " 8 t o r u f o r i t s comnonents -
n
Thesp
a
\rcctnrq
qnzn a H " .
s. r : h " - q- n a' c e"
F X
.i Il
the
be called the sample span of x.
sample space E whlch will Thus
n * , =h r r ) = l r r * r ,. . . , r ' * . , , w h er e = {Nl"c: ctKP J,
trl*
again,
NIx., ..., "1
NI*"t
= {ltl*,c, + "1
C i v e n a r n u lt i v a r i a b l e
x say with
p t q-matrix
c c Knl ,
+ Nl"-co:c1r .'f r
,..
p conponents,
.p.K}
and a
ivhich can be tlenotecl
a nul tivariable
Y
y = x c , i s d e f i n e d r v i t h q c o m p o n e n t sd e n o t e d
'Y J; eK are
where c;;
I.J
of
' lc p' Lt Co' lr l nr r i l nl r oq u, u
X C ; =
) X i C ,
J
r
I
r J
and ci
the elements,
eKP are the
colunns,
J
Its
c.
=
value
l
n r nr u
da r nr j w l
is
determined rvhen tlie value
nhi ec f
: n d r rd \ r
( ,
ir n r r
in the s,rmnle. its
the ohiects
from the sannle rnatrix of
rI ' ^d -r + L; r^\ "r r1r (^r
x
of
is
l ;cd
r
to
t
s:mnle metrix
is
determined
x hv the relation
trf*. = M*., that is, trl*.. = Il*ci' ) "
and again
l" \xf l c l i * . . . * * p . p j = '\ *. l -eI -) 'I ,"t , * a ,
'
5lnce
fl)en
g-
.I .l X
autornaticalll Lr 9t*; contained ,y
-
IL
Special
Ldll
in L^ L,g
thc
,
r- .i
that
t follows
exanlples are
e a
of
lr .r i r nr !e cer T
tl-rebasic
that
x.
1,...,Q),
y = xc then
it
tlre sanrplcslan of
is,
sample slan
^^l l d l r L lup r ]
=
+...+ Mx cni(j p,,
y
is
Any such variablc
a npnf Ln Um l , nl n ,urtgrtL
n u fI
v
t
n u rr
x-ccnrporrentsxj.
2 lhr o
i\ ' - . r^vml ln' ^l /nvar nl Lf r l L .
BLlt ncw 1et mntri ces U
^
x,
be any rnultivariables
I
l" r\r 1 -i n
t
the exnerinent.
t
rolrfinn
y.
=
v
hvnothcsis
of
a
avnarimenr
i c
r ' l r 1 t I i - . = l' rxl -C - .""--
to
with
sample
The condition lro rdmittorl
for
hv
the
t[e
But thC cOndition that this y that E--:E... Let e-, e,, denote the y r(' y x orthogonal projectors on Ex,Ey. Then, since exX = X = if and only if X Ex, this condition is or "*"y "y, couivalentlv e ll = Il y x y
holds for somec is
T H I O R I I i I] :
I I x , y b e r n u tl i v a r i a b l e s
N I - , 1 . 1 .j.n
an expcrincnt,
nrniectors
on
conclition
that
cxperimcnt
If
is
their
and if
= ev,
"*"y
e., are thc
I -
orthogonal
E-, E.,, then the
y = xc be adnissible
a relation that
ev,
qnrnq
c eu r m n lr p r r l '
r
w i t h s a n p le m a t r i c c s
or
i.n the c * 1 r 1 .=.
equiValently
s u c l r a r c l a t j o n l r o l d s , y c a n l . e s a - i d t o l - ' el r c d j c t a h l c
fronrx, and any nultivariblc
x c r t ' h o s ci d e n t j t y w i t h y i s e x p e r i n ) e n t al yl a d n i s s i b l e d c f i n e s a n x - l r c d i c t o r o f y . A multivarialrle mcnt, i f
no one of
x Iii ll its
be called
regular,
conponcnts is
in
the e.rperi-
prcJictrble
fronr
the otlrers- otlrerr.'ico flrn .^mn^nantS aIe nrrltiCOllinear. An identical
condition
of
regularity
is
that
no linear
rclation
b e t w e c l ) t l r e c o n r p o r r c n t sl , . r ee x p e r i m c n t a l l y
wlrich is
to
sa y no linear
conponent sanple sanple
matrjx is
p.
In
L . r ea t
least
the
T l l l 0 l l l ; l r 12 :
vectors,
llr,
span,
relation
equal
exists
equivalentlyr, to
the
nunber
p o{
The condition
tlre rank
dirnension of
thj.s case certainly
the
the
of
the
sanple
s a n r p J , es i z e
conutonents in
that
admissible,
betrr'een the
N must
x.
a multivariable
x witlr
s a n p l c m a t r i x l t l . ,l r-o r e o t t l a r i s t l r r t t h n m n t r i v \ r t \ l - . b c " ' X ' r x regular, in which case the orthogonal projector on the s a n r p le s p a n I l - . i s
e 'x
T'hus, for
all
t
e KP,
= \1 "x
t1"tx1" 'x1/ 1 - 1"1x "
_ 13 _ I r 1t = 0 = ) l \ 'l l ; = 0 x x x and .I ,\rx1l. \. x' -t = o J t r .l , lx' , \,txi =. o vr \ \ :, , xr .)/ r r, .l .i _t \) . =, 0 + 1, ,r . 1xt -= 0 . T h a t i s .' l t l t = 0x ( + I I l N I t x= O . w h i cr h s h o w s xt l r a t I l . . a n J I r 'l l t l * h a v e t h e x
1s regtrlar,
r-2 q e
i t
i c
sane rank, i-f and only
f o r r n uI a f o r
thc
llillx
r:.atriX cx,
tlrc orthogonaJ
is
of
i , l c n r n n t n n t
t l r r l
'x is
range,
whi ch
pro j ector
on Lx ,
It
is
consitlcrc.l is
rieJI -
s 1 ' r n n ; e t r i ca n d
=
identical
say n.
It
- x ' o' x2 a
=
rr'ith the remains
to
'x' o
ortlrogonal shoiv that
projector
olr
Il.= L*.
Tiius, if Xe R, then X = Y for since Ye " *" ) = = e*Y = X, and hence X = il* (llillx) "*\ ";Y lr'hich shornsX e Iir. Thus Rq lrx. Conversely,
l, so th:rt 1' I l x \ ,
X e L*, = extrlxt = Irl*t = X,
X = Irl*t for s ince
in
case
l S t
A l
if
some t. KP, so that "*X = c*NI* N I * , r v i ' r ci h s h o w s X e R , l h u s I ; * ! . I i ,
tr* = R,
which
in
rank p,
p.ivcn I'y thc
j-nverse exists.
definett sir)ce the
its
if
p,
rank
i.s of
re c r r l ql .
Tn thi s casc
IIence, it
so llx
then
and hence
as required.
Another
artunrent
idcmpotents,
trace
is
as fol lorts.
e* = rank e*.
lly tcncral
propclty
of
But
t r a c e e x = t r a c e N l x( I l i l l x )
- llii
= t r a c e } " i ft ' i * ( 1 1 i l l x ) 1 = trace I
p
Ilence rank e* = p. But c1earl-y R - E*, anclsince now also d j - mR = d i m l r , i t f o l l o w s t l . r a t R = l * .
y is predictable
T H E O R E 3M: I f
from x and x is regular then
of y is uniquely xc*r,where
the x-predictor
1
(MiMx)'MiMy.
.*y
' ' .y- ', w" "h 'e' r*o a- x i c B y T h e o r e r n 1 ,' e ' x. ."My. . = M
tha
nrrh4g6n3l
p r o j e c t o r o n t h e s a r n p l es p a n o f x , a n d b y T h e o r e m2 , e
x
'Ilr . x
= I'1fM'M I x' x x'
^h u- L- v^ r- sA r; r- r '^ r 1/ . , ' ,- [(\4:\r-) x L ' x x ' that
is
anv n !r
"
-xy
r h a n r { - ' . M . . c= I l . ' . M . . ,h e n c e , "y',y, """
rocrrlapair
the
by
if
this
regularity,
holds again
Regressi-on Now consider
x,
= M.,, y '
tJt.,] x y ,
\ 1 _ - c= M . . w h e r e c = c - , , . C o n v e r s e l y , x Y X Y
fny
2.
- t
of
t-t
I
x
any two nultivariables 1' -
=
residual
idenpotents
-s "o ' xc '
projectors,
orthogonal
and its
'e x t
orthogonal span of
e e X
complementarity
= O' . e
X
* 6
X
X
and assume x
sample span E* of
rnich
a pair
They are
y
are a cOmolementafy
onto the
complement i* x.
wj.th the
"
x,
of
can he called synmetric
properties
= 1.
Consider the resolution M
" Y= of the sample natrix
a
t r , t "x 'Y
of y,
the orthogonal projection samplespan of x and its
+
l
V
"x Y'
into
componentsobtained by
of sample vectors onto the orthogonal complement.
is
- 1 5 -
If
vI
"
i sY n r e d i c t a h l e e
In
fhis
case.
as
X
then
from x
My = My ' ex \ {y = 0 . been remarked there
has
c e KP
exists
Y
Such that
M.. = M-
an X-predictor
y.
of
r c '
Y
Tt
en.l then ve is
u r r u
i s a nred ictot
ll il n
tL hl l eE
^i ',i.. Sense O urf B lvrllts )EllJE
M r ' ta x LcJ' ) r ' r? y -= ,M ' ,d x c t f- = M
as the value of y in any ohject a, determinedfrom the value U3 of x on that object. The predictions are exactly x confirmed in the experiment for every object in the sample. to assume
That heing the case, there is no prohibi tion it
as universal RU r r Lf D
nr
for obiects in U.
nanarrllv SsrlqLat ) /
n ur ir vv rq l ^evn. f. lLvr l vF Y
it
Will =
-e x "ey
nOt
fg
thc
I t
i s
nossi hl e.
Q \ y-. - .
r-ese
-e x , \, {y
t. h, ,a- t. for
=
vt t y ,
i nstance,
that e*Nl,= 0, or equivalentlY e*e, = O. Since then also e.,e- = 0, this f
d e f i n e s a s y m m e t r i c a l -r e l a t i o n
between x,
^
y by which
they
are
said
considered resolution
to
For the
be uncorrelated.
in this
case
e . . N ' 1=. . 0 , X Y
e . . V , , = l t , ,. X Y Y
These are two extreme cases that have been considered. ts a set of vectors in "*ty E - x- . = .[ M - . ], t h e r e a l w a y s e x i s t s a m a t r i x c . Kql s u c h t h a t x., But generally, since always
e*\1, = M*c . If
x i.s regular then c = .*y uniquely. 0therwise, there is
an infinity
of such matrices, of the forn c + t,
^-J + ;r )a n n r t i c r r l : rr o v rn r ! er dllu L variety
dlrl
where c is
^lution of V.-t = O, the
Jw
of thern corresponding to the variety
X
of
adnissible
_ 10 _ linear relations between the conponentsof x. Noweven when y is not exactly predictable
fron x,
a
= xc can be generally introduced with the
variable f(x)
role of an x-predictor of y.
It
defines the regression
of y upon x, with c as regressionmatrix, and it
coincides
with the x-predictor of y in case y is predictable from x. rf
ic
rrninilo
*'ith
=
c
-c x- .y, -' ,
in
case
x
is
It
regular.
is
are uncorrelated. For any object a,
nu11 in case x,y
= N{3c NIl ^ r(x) c a n d e f i n e a n e x p e c t e dv a l u e l , f l o f y , g i v e n t h e v a l u e Ml of x. ,n" X
,rr""
""tr**
oy Y, = y - i t * l d e f i n e s t h e
r e g r c s s i o n r e s i d u a l , o r u n e x p e c t e dp a r t o f y r e l a t i v e t o x.
I n h a r m o n yw l t h t h i s
vatue i(*) with x.
it
is
seen that the expected
of z ts identically
zero, or z is uncorrelated
For = e NI^, . = e M = e - x "My - y " x "ex l r' ly = O . ztxJ x z (x)
Thus the consi-dered resolution N{
--
v
can be put in the form
My - y^ , txJ
tr^ lvl
ytxJ
characteri,zed by the conditions
ti n?
anrli
rre l onf
r*; l rz
i f *l Thrrs v
is
= u * M y '^ ' r - i (*) = 6*My = xc where e*N{, = lvt*c.
resolved
into
a sum of
variables
Y = i i * l+ z an expectea part i1*; unexpected residual
= XC, predictable z = y - y(x),
from x,
and an
which is uncorrelated
with x and has expected value zero relative
to x.
_ t 7 -
The rnatrix c has been characterized
in two equivalent
exMy = M*s, and also by the relation
ways, by the relation
= 0 which says that y - xc be uncorrelated "*My-*. These relations are equivalent, since
x'
= e*M*c = M*a = M*c,
"*M*a
= \{*. follows e*Mr_*. = €* (vy - tl*.) "*My = [1*.' and f rom e-M.,-.- =O follows e-M.. = X Y x c ^ Y "*M*.
= M*.-M*.=0,
so fron
4: For any variable THEOREM exists
with
y in an experiment there
x,
= M*., or equivalently "*My = cxy uniquely. x is regular then.
a matrix c such that
= o, and if "*My_*. Now two further that
c. One is
ways wi-11 be shown for from the method of
farniliar
characterizing least
squares
I i n e a r r e g r e s s i o n , a n d t h e o t h e r i s k n o w nb u t l e s s Instead of the usual methods of the calculus
fanlliar.
algebraic nethods will
be used.
x is regular,
T H E O R E 5M: I f
=
e -xy
then c = c
is defined
so ,that
ft 'M ' x ') - xr M
xy
- 1'
MrM 'x 'y
eives the absolute minimun for
the trace and determinant of Nl|_*.My_*.. Since e it
follows
+ e
X
1 , e x M x = M * , 6 * M * = 0 a n d e * l r ' l r =M * . * y ,
X
+L ^ + L I I d
L
l, I v
y
l
-
f
ii
l
l
x
^
-
-
o .
l'M x \,-,Y
V ' ' x (t "cx y
and then,
Afrtat!
s i n c e a l s o e x- - i s
Studles
M c) X-, -
cl
synnetric
+ tr - X f\'t sY *
-
M c)
-6x "My and idemPotent,
that
- t 8 -
( M . ,-
-
M-c)'(M. /
(c \"xY
=
M " X "c )/
' Xr"M x (\ "ex Y !r -/ ) r' M
-
c)+ M'e M "Y"x"Y
= (' Mx- . cx- . .y. -1 xt { - - c )' ' ( xM . x_ c -y. .M-x- _ c ) + (", which, witfr i(x)
= *.*y,
M*.*y)'(My
M*.*y)
can be stated
+ M' ^ M^ My- '--x- -c_ My--, x. .c^ = I l l 1rl ' y (x) -xc y (x) -xc y-y (x) y-y (x)
It
follows
immediately
theorems
the
matrices
(Mirsky
trace
this
identity,
and determinant
of
by the well-known
non-negati,ve
defj-nite
1955), (a'a
trace det
from
+ b'b)
(ara + b'b)
= tlace
a'a
+ b'b
= det ara + det b'b
that t r a c -e-
-
.l t' 'y[ - x' _c_ "I {y - x c
trace
^, M' yt - y
.M (x) "y-y (x) ,
and
t ti-*.ty-xc where the equalities
det Mr-i(*)My-i(*)'
hold if
and only if
= 0 M . : . ,. M^, , -' y ( x ) - x c ' Y . t x . )- x c but this is if
and only it
nti(")_*. = O.But Mi(x)_xc=
= M*.-..-*. = M*(c*r-c), and since x is regular, M*(c*y- .) xy if and only if c_,, - c = 0. Hence, the equalities hold .if
rr
--J ; t - ^ ' -= a n o ^o- n1 .r,y 1 r a cxl,
=O
a -s r- ^e^ q. .u: -i r e d .
The least squares principle
is shown in the part of the
t h e o r e m c o n c e r n i n g t h e m i n i m u mt r a c e . A p r o o f o f t h e p a r t of the theorem concerning the minimumdeterminant has been given by F. Sand (in a private
communication), by
nethods of the calculus, using the formul"
l"l-1a]"1=tracela-1aa1.
- 1 9 -
3. Experinantal
configuration
Certain relations
between variables x, y in an of the
experinent can be presented as an alalysis
nrniarrnr<
a
o, t ' y
nredi.t2hle c n n d i t -i o ' -n"
from x e ' x -ey
€--€-,= e.,. Also
is
X
Y
tnat
Ol
the
SpanS.
as has been seen the
Thus,
V
in terms of the orthogonal
be characterized
therefore
l*,
spans E*,Ey, and can
forned in E by their
configuration
is
E.,5 E-,
= e.,, or y' the
what
condition
E, be orthogonal,
for
condition and this
is
is
y
x,
to
to
stated
by the
be uncorrelated
and the condition
i s e - . e . .= O , o r e q u i v a l e n t l y e . , e - = 0 . A r e l a t i o n ^
be
by symmetry,
equivalent
for
y
for
this
of
y
inci-dence between x,y can be defined by the condition
This is the condition that there
E*, Ey properly intersect. exists mutually identifiable
c o m p o n e n t so f x a n d y ,
is variables
x a , y b s u c h t h a t N { x a= N I y b . - s i n c e t h e
intersection
of E*, E, is
relation
that
the nul1 space of
1 -
that
"*"y,
this
can be expressed by the condition
- e - - e ,1. = 0 . T h e e x p e r i m e n t a l r e l a t i o n o f n u t u a l ,l 1 x y' predictability, or a!94f-!y between x, y is expressed by the relation relation
E* = Ey, and equivalently
of orthogonal incidence is
e* =
A further "y. stated by the condition
t h a t t h e o r t h o g o n a l c o m p l e m e n t si n E , , E . , o f t h e i r
inter-
section be orthogonal; that is, E*^ EuI Eu."E*. This neans X, y are identifiable comnonto both,
together with
with a set of components
further
sets that
are mutually
- 2 0 uncorrelated. The condition for
this
that e e , eYel' x Y=
is
i n w h i C h c a S- e e - x- ." ey 'i "s t h c n r t h n o n n a l n r o i e c t o r n n E - h E . , , and e*er,6*"y
a r e t h e o r t h o g o n a1 p r o j e c t o r s o n E x ^ E y , E y ^ E * .
Sti1l a further relat'ion is that y he inclined to x, defined hw
f he
cnnd'i t i nn
l-h"s"f
" "s x N ' ^, y! "h *a v"e
case, i I y is regular, then so is its w h e r e M - - ^ = e - . \ 1 . , .T h i s i s x c x y = r a n k e .
*a "S 'M' y '.
S a m e r a- n" k"
the
In
that
regression xc on x
stated by the condition
rank e-_e-_ x y
v
These are nodel relations setisfiedhv
the
Tnoether
rel:tinns
t. h e .w
,
thcv "
implication
.
that generally will
n r e. s e / n f
harre with .
.
"
.
2|
v
n | 2 t t r a r nr
eneh /
l
n fl
nther r
g
not be
i Ln f pl r e nv n nr p r tr i n n '
nf l
v
onnnqitinn r
a n d c o n f l u e n c e a s s h o r ^ r ni n t h e f o l l o w i n g
r
'
schene
Opposites C o n fl u e n c e (-----
identical
distinct
I rl,
A I
included
excluded
/
incident
\
.r'
inclined
\ \ \ y ' c correlated
\
dlsinclined separate / \ / / \ / o n f l u e n c e \ \ // ----------) uncorrelated
Rather than the model relations
that
exact realization,
can be defined whose extreme
coefficients
values give criterla
for
the exact relations
intermediate values constitute a f + a i h m ^ n f d L L d l l l l l l g l l ( ,
^ d >
' . i W I r ll
I
nt r^ vt iF
generally
n ip oLr dr r . aa p
do not have
but whose
a measureof approximate
o1
4. Configuration coefficients x and a q-variable
Consider a p-variable
y in an experirnent, spans
and Iet e-., e.. be the orthogonal projectors on their ^
y
"x' "y' D ef i n e dxY = r a n k e e
R 2 = tt rr aacc" e e e xy "*"y'
x y
d * = t r a c e e x , i n w h i c h c a s e a l s o d x = r a n k xe ' . a n d s i m i l a r l y d e f i n e d r . T h u s d * z p , d y ' Ll 4rlu x , o r L
Also defin"
is regular if A first
d* = p, or d,
and only if
property for
nj,
is
a '
that
1 d . d o 4 R 2 1 d x' y xy xy N o w R - , . .c a n d e n o t e
xy
and can define
the
the
coeffbient
"*
of
rnrral
between x,y.
dimension of correlation dimension of both
square root of trace
positive
Uy
r
9
af 9
r
v
r
i nn l
,
and d
Clearly d
and e, E* ,
the
xy
is
xy
"*"y,
the
and the rank
o f e \ 1 a n d ey M x , x y Thc Frrrther nronerties o - -f R "xyshow how its oi vc eri teri n fnr model relations
values
between x and y. Thus,
^R - . - -= ' 0 i f xa n d o n y l y i f ^ e - e . ,t = 0 , "w h i c hx i s
t o' s a y " F
are orthogonal, or x, y are uncorrelated.
Further,
Rj.. = d-.. if and only if x y x y ^ if
e-e,, = e.,e*; that is,if y y x
E-., E.,are orthogonally incident ^ /
be identified
FY
and only
in which case x, y can
as having components in whlch possibly
are identifiable but the rest are uncorrelated. ) = Again, *i, = J* iF and only it "*ithat "*", E . ,g E - , o r ) ' i s p r e d i c t a b l e f r o n x . S i n c e O * , n= d " , d y ,
the dirnensionof correlation
some
is,
between
- 2 2 does not exceed the dimension of each, and
the variables if
d - . - .= d - _ , t h e n y i s XY Y' tf,is property
regular lf just
incl ined to x,
so in case y is
is preserved j,n its
regression on x.
x is a regular p-variable, and q = 1' that is, it
a single variable,
to the classical
Y is
is easy to see that R*,. reduces
m u 1 t i . o 1 ec o r r e l a t i - o n
coefficient.
For in that
caSe ) - 1' M ' M - 1' N l ' R' = trace M fM'M ) xy x y f' l y. ' t 'yl u)' l y x'x x'
= (Ury4 '' M X \f 'M ' X'"M X /I = Tn casc el so n y
- 1'
M'M
' M' y )' 'X"y ) f M " "y
- 1
1' t- t" h" ' i s" r e d r r c e s t o - 1' [ N l ' ] \ ' l - 1
)
^2 ^
= f' Mx ' N y l "l - xf Mx' M' I y y )' x y which is the classlcal correlation coefficient of Pearson. Tt annears that
the characteristic
squares of canonical correlation x , y . H e n c e RX_V . , i s i d e n t i f i e d the canonical correlation
r"oots of
coefficients
-ex "ey a" 'r e t h e
of flotelling
for
with the sumof the squares of
coefficients.
would not fo11ow from the established
The properties of R*, canoni.cal correlation
theory. But rather these properties can cone directly
from
properties of the product of symmetric idempotents exev, as aLso can the theory of canonical correlation, The total
multiplicity
(Afriat
of non-zero canonical roots o
and a corresponding nuinber of canonical pairs
1956, 1957)
2 ' ^ I ,, ,xy,
of conponents
is determined, with
sample vectors whi-ch are i-n the nu11 spaces 2 . z. o f 'p - 1 - e * e , , a n d o - l - e . , e * . B u t a c o n p l e t e c a n o n i c a l a n a l y s i s y y ^ requires
determination
of U* - U*, and d,
componentsfor
x and y.
The sample vectors
n r r l l JsPne aL LcJ e s o v !f
1, - e_ x e y
a n d 1 - e_ I e x .
,,uaa
- U*, further of these span the
-
L
oOa
-
Fortheregressionofyuponxthereistheresolution ty="*ty*6*I'1,
= Mi * tr-i where y = y(x),
and, since e* is
symnetric and ilempotent,
N{yMy=ti"*tr*ti6*ty = ,i " l * \i _ gtny_g. Measureof the exactness of the regression dependson a componentwith
conparison of the regressional
the residual
component. ) R*y = trace it
"*"y,
-Z a*y
trace exey
appears that = t r a c e e - . ( e , .* e , , ) = t r a c e R " l . + \O t y * x y x y y
x
This is again bY taking the trace in - l'11'- r' 'l = + (ltll,ltI.,) e*Nlr, lut.re-.lt'1.. (I,llMv)
since
trace 1 = q - 1ll;e*u, = oi, trace (M;My) -1u}"*r,{, = qi, trace (Mj.l,'ty) The coeffici,ent
o'*v = trace
"*"y/
trace ey
thus has the ProPerties, o- 1 p- x 2y < l S o P^* /. , c a n d e n o t e t h e p o s i t i v e if
and only if
square rootr
the regression of y on x is nul1, which is and P*, = 1 if
the case of x, y bei-ng uncorrelated, only 1f the regression j,s perfect. the classical
and P*, = O
correlation
and
This also generalizes
coefficient,
though it
is non-
^
just
symmetrical between x and y and applies
on the regression
of y upon x. A symmetrical coefficj.ent
between x and y
of association
can be defined by Cz xy
= (trace e e \2 / xy'
ftrace e )ftrace x"
e ). y'
This has the n r ' r" or n e r t i e s A \ , z-
C - . . -= 0 i f xy
a U x y z-
1| '
e - e . , = 0 ,' i t h a t i s , E _ , E , , a r e o r t h o g o n a l , xy )( y = y are uncorrelated, C*y 1 if and only lf e* = er;
or x,
that is,
and only if
E * = E , , , o r t h e r e g r e s s i o n s e a c h w a y h e t w e e nx a n d y ^ f
are perfect,
each is predictable from the other, or they are
experimentally equivalent. This coefficient
and the coefficient
of dissociation
defined by 1
)
. * y = t - L x r harreanalogies with the cosi-neand sine, further
generallzation.
Thus, it
) xy
and capabilities
is noted that
1
C xy
C xy
1
Then, for several multivariables x, y, z, coefficient
of dissociation
.,.,
a g e n e r a l jz e d
can be defined by 1
C xy
C xz
c
1
f
r
c
t
. . .
y L
r..ith
tL'a
nr^n6YrieS
O
<
for
zx
K - . r -I- r Z r . . , " X
zy
^ r J r p r . . .
-
;r rr
^-r
erru
Jc rnres nr rc J
^v r-r1r j./ , ;. rf Fx L t
F "Y,
--
.- * y F -
L
J
_
(u -x z
0,
cr6
m r r r r r a ' lt y
zt
be partitioned
Ex, Ey,
within
Of
OTthogonal,
= 0 if
are nutually uncorrelated, and **,r,2r.,. if
tO Say the
whiCh is
the unions of each class are identical.
and only
and such that
Nowconsider the
multivariables partj.tioned into classes, thus xl, Then the K-coefficient
*r,*2,
A v U =z n
*2,
has the property
/Y1,Y?, "'/"' = K *1,*2, ..., Y1,Y2,
Then
z,
i n t o c o m p l e m e n t a r yc l a s s e s ,
which there is mutual orthogonality,
and y1, y),
y,
X,
x1rx2,
fKx,x2, ,..,Ky1ry2
4
y2,... /
/y1,
where the value 1 is attained
if
1
there is
and only if
orthogonality between classes. Sti1l
nore complex coefficients
in an analogous way, which give
can be formed indefinitely
s u m m a r ys t a l r e m e n to n t h e p r o x i m i t y o f v a r i a b l e s t o r n o r e complex configurations defined in terms of sets,
sets of
sets. and so forth. ^ - 4^r-r 4^r1u B^/ ^ " v^f f t h e C a n d S c o e f f i c i e n t s w i t h t h e c o s i n e
nlr
a $ r rnv d
s ir nr r ev . t
J
the
reflectino
ensinp
rnd
Sine
fOrmuIaS
fOf
a
SUm
and difference of angles, is seen in the inequality C
" x y "Cx z
-
"Sx y "Sx z
= ( -
" x y "Cx z
where one or other of the equalities nnlr'
if
o
noir
nF
the
$ "xy
S "xz
is attained
-S rp a* C- e- S' - Ex -,- , p" y , F" z *e t' ' a
pair of orthogonal and the third spaces are rays, this
+
is
their
if
and
i,-lontiCal,
union.
Of
a
In case the
showshow the cosine between a pair of
- 2 6 sines and cosines with
rays is bounded when their
a thlrd
ray are given. matrix
Returning again to the decomposition of the unit associated with the regression of y on x,
rather than taking
traces, which leads to the foregoing discussion, 1et determinants be taken instead, to pursue a paral1e1 to the alternative m i n i m u m t r a c e a n d r n i n i r n u r nd e t e r n i n a n t
characterizations
for
in Theoren 4. Then comparison j.s to be
the regression matrix, nade between )
t
'
c' -x y = l l r yl ' -e x\ 1y I" 1 1y4 y' \ '4|
- 1
and
-1 ' i y = l 1 1 6 * * r l u ; u ,I (Afriat
But then, as follows from determinental identj.ties 19s7), = l r - e e 1 . 2 = l r - e e I ' x y' xy x Y'
, l xy Thesecoefficients,
defined for the spans [*,
I:.,,which can
of separati.on and inclusion,
be ca11ed coefficients
have
analogy with the sine and cosine, respectively, as appears f r o m t h e f o 1 I o w i n s D r o D e r ti e s . Thus,
.) u LA
)- ux y z-
I
r,
"c "A "c x y e q r A a n ^ f e t h e p O S i t i f ' e s . . . ,= 0 i f xy that
iL s .
h aJ v e
S-.. = xy E-,
E,,
afe
I
if
i n ' t F r s c rY - f i n n
and only
TayS,
q
"xy
i c
SqUafe fOOt.
and only i f E-,, 8.. are incident, y x' "
if rhe
nf
n. o s i t i v e
dimension.
E_, f., are orthogonal. y x' cinp
^f
tl''o
ancla
Similarly, o L , l _ - L 2x y. _" l
"s "o -c x y
can
(lenote the
rositi\rc
snUaIe
fOOt.
in case
hotr"ggn
them.
-2 7 -
.*y is;
= O if
and only if
E* is disinclined
to Er,
that
sone proper subspace of E* is orthogonal to Er, which
can be expressed by saying x is partly
with y,
uncorrelated
or the regression of part of x on y is nu11. r*y = 1 if
and only if
E* is a subspaceof Er, which
meansthe regression of x on y is perfect. while it
* ,i,
is generally not the case that cl,
= t,
i-t can be shown that .- x7y * r- xz y -1 1 and noreover, by an application
o f T h e o r e r n so f l l o l d e r a n d
It{inkowski that t ! ' l r' c ' ) i * ( s x' y)' ^i < l xy'^ where equalities
hold if
and only if
E" is either
included
in or is orthogonal to Ey. In fact, identified
with
alienation,
can be algebraically
the c and s coefficients the coefficients
of vector correlation
proposed by l{i1ks (1932) and Hotelling
and
(1935).
Introduced as here in terns of orthogonal projectors, general properties readily
evident,
are easily derived which are not otherwise
and their
geonetrical
neaning becones
transparent . W h a t e r r e rc a n b e t h e s t a t i s t i c a l such as the variety
considered here,
role of coefficients they have an intrinsic
place in the formalisrn of regression analysis, arise
in a natural
that unites
and they
way j-n terms of orthogonal projectors
algebraical
and geonetrical
meaning.
zE5. Varlance T h c s n m n lp m e t r i x
\ 4 - . = (' M X -". , X
".,
M " X DI n f c n - V a T i a b l e
with the vettor M, , ..., M" ^.1 ^p = \ ' l i \ 4 x volume V* is given by U; | | , and,
x determinesa parallelitope a s e d g e s, a n d i t s following
W i l k s ( 19 3 2 ) a n d A n d e r s o n ( 19 5 8 ) , c a n d e f i n e t h e
variance of x, ln the regression of y upon x,
t h e r e i s d e c o m p o s i t i o no f
the sample rnatrix of y in the forn = t*.
ty Four equivalent
* ty-*.' have been stated for
conditions
the
determination of c, and one is thaa Ur_*. be a minimum. Regularity for x is stated by the condjtion Vx I 0, and w i t h t h i s g i v e n t h e r e i s t h e u n i q r r ed e t e r m i n a t i o n . = . * y , -1 uhere c.... = (Ml.\'1..) Ml.V... ^
^/
A
^
y
In the case of two simple variahles x, y with joint ccnn1a
m.f
riv
. . \ {. x y
(NI-\|,,),
the
yerione
.a \./ x y
ic
niygn
ly
, . . =* . lfM,r ,
\u/ -
xY
]\'l I | 'xY xyl
l' M ' M Mx'\ 4y lI x x
l| l' |yI 't x My' M ' y I|
= lI r t l r tl.. l u :y. uy .r . 1r r x xr I
.)
(Iri\IY)'
= v ? . v ?1. 1 - c o s 2 )e x y - - v z v z si n z o x y , _ . . T h u s V _ - . .= V - . V - -s i n 0 , t h e a n g l e b e t w e e n l \ t _ _M x y x' xy Y w e l l k n o w n f o r m u l a f o r t h e a rea of a which expresses the v,'here 0 is
)
(Mir4x)(\4i\4y)
- 2 9 -
parallelogram.
But generally,
V_., can denote the volume of ^ l
a paralleiitope parallelitopes
i n p + q d i m e n s ' i o n sd e t e r m i n e d b y t w o i n s p a c e , E - , E . , o f d i m e n s i o n sp , q b e t w e e n ^
which the coefficient
f
of a separation is
sx y = 'l r - e ey 'l 1 / z x Then, as a direct
generalization
area of a parallelogram, it
of the formula for
can he shown, from
the
determinental
identities (Atriat 1957),that = V V s V x y x y x y A further
d e t e r m i n e n t a J .i d e n t i t y
(Afriat
= lI r 1 y . x' . sy .'. r 1 .'I.t1rx: .xv1 '- . 'l r
" x " .y
1957) shows that
I"
But
l u ; " . r r r l = l ( e * \ 4 y )r' e x \ 4 y ) 1 , The parallelogram wi.th edges fromed from Nt*, can be considered as having V* as a p-dimensionalbase and 6*Vy.t q-dimensional"height",
and this
corresponding
i d e n t - i t y s h o w sg e n e r a l i z a t i o n
of the rule that the area of a parallelogramis the product of base times hei.ght. S i n c e e . . t t . , = J r l . ,l w J r er e 9 = x- 'c-. x y y-y xy' i + r L
. , - - ^ - - d P P C 4 r >
+ L ^ + L l r d L
= u*t*y
'y-y which shows the relation
of the residual variance \'r_i in
the regression of y upon x,
to the variance V*. Also, as
follows from the sameidentity, v^ = v l r y x I -
ti
^
x x y
since Vi = e*r1r, - e e t\/2 x y t
- 3 0 -
remark, it
As a flnal identity
to be noted that the determinental
is
which gives the generalized parallogran
1aw, which
can be st&d T,II M N'I'N' x x x y '\ ,'1ryv
x
= l r , l r * l l u ; r . r r l-l t "*"rl
l \ 1r l \ ' l
)'y
where l ' Ix),- 1 l r^{ ,x. , e- x = l' .' {x (t N' 'x,N f n o o f h o r
w i t h
t h n
-1N,1 e ' y. = \ {' y (. r ', 'yl- ;' ,y4' ' ; y- ,.
n r n n n q i t i n n
O < l l - € * € , 1 < 1 , wherc
there
ii -S'
!e\ O , t U. d. I^r 1 L /:
+,,
^-r
d l l u
I
I
:c r l
dar n r u.
l
vn i n l r lj rr
l t ' l - ' - r 1=- -O , s h o w s t h a t x y ,\ lI ,' Vx ' "t rx, |t || |' vV' ' u l I M ' T I I {' M v l x x x y
'.if
p
p
=
O,
or equivalently
'\ 4' ry\ '{ x
'
\4r[4
y y
which is an equality of Fischer ('l908), from which follows t h e w e l l - k n o w n d e t e r m i n e n t a l i n e q t r a l i t y o f t i a d a m a r d( 1 8 9 3 ) . Another proof is thus obtained of these inequalities, the added explicit
qualification
with
shownby the factor
I' f - e x- ev. .,1' , a n d t h e p r o v i s i o n t h a t t h e e q u a l i t y i s a t t a i n e d = 0 . B e l l r n a n( 1 9 6 0 , p . 1 3 7 ) r e m a r k s t h a t o n l 'y .i f l Mt J M v .. " l l a d a m a r d ' si n e q u a l i t y i s o n e o f - t h e m o s t - p r o v e d r e s u l t s i n analysis, with well over one hundred proofs in the literature."
-
-
JI
PAP.T I I
'I
Ir'inein:l nAl ln, vw
^
I t O t
n l l r L
) 1 , d 1
nf -L
neirs
^*^
l l
d r
I
U l
d l g
nf
snrccs
^L1:^rrc
s
u u l
cornponents lr'hich are
- . ^ i - , . Lw :l ^l l 1l l .l
O
' L;
q n a c pq cJ
tL uw -v n
of
lair
redrretinn
- . (+l l1u .F^u ^r ^l ( -l ^! r
n l la l dw j /
r Y u q
inclined +L U^
hl , es
rI eq ud ur Lr e n c d u
f g
each other,
to
+( i Ll '. e S e a n d
tO
aiSO
3
and
eacll
which are mutualy inclined, are
other. The prjncipal lair,
o b t a i n c d a s t h e o r t h o g o n aI p r o j e c t i o n s o f t l l c s p a c e s i n l ai r e a c h o t h e r : a n d t h e o t h e r - t h e r o si d r r . an
are the
I
con,plernents in
orthogonal in
fhc
nrinein:l
n T n n . r t \ /t ) l , r u P l r
n u fl
projections
nair-
hu eq ir nl lob
each
lhe
ri nr r
in
the
prolositi-on
The fundamental
thc
sfaces
nrincinel
rr A a ri nl ,r rnve! eu lr sJ r L L
each otiler
of
of
nair
+ l ' ^ + L l l d (
of
either
also
+ l - ^ ' ' L l l q j /
have
- * a d l q
+ L F L l l s
space the
n v rr l hL nr cr un bn ur lr r q r
spaces.
fron
which
thj-s reductj-on
i s o b t a i n e d i s c x p r c s s e t Ji - n t h e s c h e n e : E @ k x y x y
t
x
t / ty
where the that
is
aplears in
indicated
relations
e..[,. is ^ y
i n c l inecl to
are 6 . ,, I
as orthogonaI
to both
C
x
and ty are oblique
and the R*r,
whic)r
extr and \ / , c a n b e d e f i n e d
t h T c n c n u i r ' : l p n t w a y s:
fq* o"*
xy
t h r t
the
.i q" ,
2 s
nrniectinn n rn i e e ti n n nrthnonnel
the in of
=
I
,
( e;.Ory I \E,. o"y E*
o r t h o g o n a l c o n r p l e n r e n ti n
t-_ o t r_. j n
n rn i a a t
,),, or of sinply
t-. of
E,,, or r
of
t 1 , o
^ r f l r n c n n r l
+ l - ^ Lltg
^ , + l ^ ^ ^ - ^ 1 vr Lrrvl:;urror
, y , o r o f s i n r p l y {.,, or o f the
ion of
qx in
6 y . A pnr ' I / v'
i n o " o
t lr
is
schene to
-
-
Ja
and then again with the spaces
one space with the other,
i n t e r c! rhr u:r nr a o- \ e rs ,1 . t h e r e i s o b t a i n e d t h e f u r t h e r
-
!x | o l
'l'he is
by the
l
t
)
with
for
1 -
c,c..,
)
=
con'ponents
A
ly
"* the
spaces interchanged. tlre rcLluction is
anC fi*..ts ^)
)
ortitogonaJ. projectiors
the
e*er,
the pair.
principal
the
obtair.ing
e.. [,, as tlre rangc of ,\
of
rclation
same relation
An algori thn
reduction of
ex{e)- 1r) and the
t n
.
t.="y e^-!y*
property
reciprocal
stated
'/'---ix [1' Txy | ..-l ? |
principal
the
r , ' i ' r i c hd e f i n c s
scheme
f ronr any lrases for
the
spaces,
by
the
by computing
thc nrll]-spacc having
of
beetr conrpttted, r''hich has been
fornula
given.
arrtl reflexivc
Z. Recinrocal
directions
In a pair of spaces, reciprocal directions are those w l r i c l i a r e t l ) e o r t l t o g o n a l p r o - je c t i o n s o f e a c h o t h e r 1 n t h e slaces, Thus \ f € r ' fnr
rnr
i nrnre
flra
nrfhnonnrl
I
dua
tlre
lrn oone.l
nrf
a ) " " - -
) f
u , "yn= in spaces x,
on which , ' , e v . l h e proclucts projcct iors arc cx ' x ' Ay '
the
^
'
directions )L,V
A
define
) , = G
A
- yA - x
frair of biprojectors
on the
projectors are synretric,
cna.oq
Si ne e
the drral hinroiectors
-
J
J
-
are tlie transposes of each other.
that
space of
with
identical
is
one which
is
in one space relative
direction
A reflexive
its
the
projection
orthogonal in
projection
orthogonal
to another
the
in
other'
A^ - ^ - A; - ^ 1 \, nL!ulurrt}>r/ t
L = " x" y Z c f_or a refl:exive direction
€* 1n regard to
) ,
Evi.dently a reciprocal pair
of directj.ons in the spaces are
clirection in one space in regard to the otlier.
each a reflexive
Ccnversely, any refi.exive direction gard to another forms with the other a reciprocal pair if
'
orthogonal projection
its
\ / 1, L , andU = crlt
in
in the spaces; for
of directions
)
t,c=
in one space in re-
\
\r
(A = e^ U , tlten
"*"y < l e termination of the reciprocal Thus the a pair
Ey.
between
directions
of spaces is equivalent to the deternrination of the directions
reflexive
o n e i n r e g a r c Jt o t h e o t l l e r ;
in either
anc these are the invariant
of either
directions
one of the
bipro jectors:.
3. Proper angles A proper angle betrveen the spaces is
c l e f i - n e da s a n
a n g l e m a d eb v a p a i r o f r e c i p r o c a l d i r e c t i o n s ' )-L ,1J are a pair .i.,U
of reciprocal
the directions
l
,U .o,
where Otos
Afrl8t:
the acute angle
proper angle w h i c i r t l . r e y m a k e b e t r r , e e nt j r e n d e t e r m i n e s a
between the spaces. If
,
dLrections,
Thus' if
U,\r is any pair
tl',"n 2 L,t ?/ ,
= ( L l' \ ' ) 2 / ( u ' u ) ( \ I ' \ r )
' /t \, ) J . ^lf r. ( ' o ( LL,U { 1' hitlr
Stutltes
of vectors spanning
,/Z
,
- 3 4 -
While any reciprocal pair
determines a
of directions
p r o p e r a n g l e r n a y b e r n a r l eb y unique proper angle, a given rs ' nany <listinct reciProcal Pai
I i . e c i L ' r o c a 1l t e c t o r s
4.
c l e fi - n e d a s t h o s e r v h i c i r s p a n l 'jllel J are \rcctors directions' Thus, if ll'\' are reciprocal
N o l n 'r e c i p r o c a l reciprocal i n q .^ . ,
q, then f
erLI = \'o
e*\I = Llp , for
a,o
some multipliers
appears'
whi'ch' it
o,
l
' are giYen
by a =
(tlru)-lu'v, fron
tn'hich it
follor''s -
po
(\I'\')-l\''u;
tl at t
A
coS" tl,\'
It'loreover, cre'tl = tlI
, crer\-=
latent
\'t
, dual
thc
of
vectors
t-hei'eI = po ;
so,
biprojectors
witit conrnoncharacteristic "*"),,e).ex 2 l = p" = u
whiclr js
p r o c i r . r c to f
thc
s ( l l r i r r co f
ll ,\' are
the
0' oand
nlultiplicls
r-a1ue
also
is
the
tirc cosinc ,t\
u = cosu,v of
the
angl e betrr'een thern, v;hich
tretween the If,
also
is
a p1'oper angle
sPaces.
further,
Ll,\'are tlnit vectors,
thcn
P = ll'\' = o In
this
case, o = lrq where eull =
shorving LJ, \Iq to
E =
tl
(V 6)r, ,
be a reciprocal
;
ancl then
e-(\'6)
= UP ,
palr
unit
of
vectors
h'ith
- 3 5 -
which are equal to each other,
multipliers
they are to define
with which properties
and positive;
a nornal reciprocal
pair.
the spaces are the same, and non-negative;
and they are all
the spaces are orthogonal, which is when
zero only if
biprojectors are nu11. Therefore, if ?
1et.p'denote taken that
on
values of the dual biprojectors
The characteristic
the
the spacesare oblique,
any non-zero characteristic va1ue, where it
p> O ; and let
\/Fctor of e e ^
, s o/ t h a t
U be any corresponding unit
is
latent
)
e - - e - - U = U U ' ,U r l J = ' l; x y and define V by e U = V',.
v
-
Then e*V = Uu
, and V'V = 1;
when U,V are a normal reciprocal
pair
of vectors in the
s -p a c e s^ € . - - ,6 y_ . . ^ ^ ^ ^ r , U. l ri r hr B^ r ' / l, '
n L L U I
tL nU
of the biprojectors reciprocal spaces with
pair
2nv
d r l /
n i l n u ln l - ' 4Fq rl u^
ChafaCtefiStiC \
ValUe
UZ
on the spaces, there corresponds a normal
of vectors making a proper angle between the
cosine u.
It
has already appeared that
the square
of the cosine of every proper angle between the spaces is characteristic value of the hiprojectors.
Thus it
that the characteri.stic values of the bi-projectors
is seen on the
s n a c e s a r e D r e c i s c l v t h c s n r r n r c so f t h e c o s i n e s o f t h e proper angles between them.
a
- . 1 0 -
5. Rank and mult j-plicity r T " 'a" r ' - x, 0 r o \ < o , < " / 2 ) d e n o t e t h e n u l l - s p a c e o f t h e m a t r i x ) ,1 - e X. - eV .' - , w h e r e l = c o s 2 o . T h e n e v i d e n t l y t * r o
t
t y
;
" *
and €.- I 0- just x 0'
when s is a proper angle between the
spaces, which is just rl r* a: ul rs r o
'
i n
u , hi . h
whenl is a biprojector
.aSe
i -_
fnr
the
latent
Vectors
r d
and the corresponding space E.,
have
Y t a
the samedimension ra of
reflexive
associated wi,th a proper angle o.
T h e s ^p a c e / X
r which will
be taken to define the
o as a proper angle between the given spaces.
Si-nce latent different d
Of
tha lharacterist i c value tr , which span
directions
Y
composed
X , 0
^r a 6
rank
is
characteristic
vectors
of a matrix
corresponding to
characterist ic values are independent,the spaces
a r e i_ n" -d- er -p e p r l e n t : n d t h e r e f o r e t h e d i m e n s i o n O f t h e i r
union is the sum of their
dirnensions. Thus
O r- x , c c ; and correspondingly t
e ' x i- I
r' o <'
,
r
o
where r is
the dimension of incli.nation of the spaces, being
the dimension of e^ the relation
.y, ,' w h i c h i s
t h e r a n k o f e - - ey . . . H e r e x
between spaces must be an identity
r e i a t i o n b e t w e e nd i m e n s i o n i s ,
as wiII
if
the
appear in fact to be
the case, an equality. Let the multiplicity defined as the nultiplicity
mo
of o as a proper angle be of I = .or2o
as a biprojector
- 3 7 -
Rut tltere is the gencral proposition
characteristic vaIue.
t h a t , w i t l r e - , e , , s ) - m n c t r i ca n d i d e n p o t c n t , t h e n u l l i t y l'1- c-e., is equal to the mtrltiplicity
the matrix
as a root of the equation -
'o and
that
also
the
'' r
rank
- e"erl = 0.
llt
=
x
of
f o 1 1 o r , ' st h a t
'
o
e..e.- is
of
x y
o f j - t s n o n -z e l o c h a r a c t e r i s t i c t
It
ot-
ton'o
to
eqlral
the
nultiplicity
values,
;
rvhence a o = t
t (I
Accordingly, =Q/
erty '
r v h i c h s h o r r ' st h e d ' rf f c r p n t
nr.ncr
ref lexive
of
the
h.
ortl.ocnnrl itv
Lct
a
1 f
ortliogonal possiblc
to
the pai rs
four
o,
the
r''ith the
oltitonon:l
nroicctions
other.
be trio pairs
a Jircct ion
then
other
in
this
lclatt-ons three.
are
'lhus,
four
ilotiever, shotild
directi-ons
case the
in
g,cncral indepcndent,
in
betrr'een tite pairs. reciprocal
can be
one pair
the ctlter pai r
in
four possiblc,
relatiorrs
is
directions,
of
in
f , . T l r e n a c li r e c t i o n
be f orrned of
there
ilrn
associatetl
rclations
orthogonality
irnplying
each
r v a y ' s ,t l c f i n i n g
orthogonality
spaces,
in
( a = :r,B )
lL- U ^ tl i.r
rnaking angles
qn2nnino
: n r , ll a\ Jq'
oir,'en sn2ces
,
directions,
g l l t
r
X.a
l
i-n a pair
proposition
of
that
equi-va1ent, &n)- one of in
this
case,
tltere
are
the them the
equivalences
l^ r I ancl tjre othels
U
D
c le t l u c e d f r o n
(.j l,r _L V. o * b syn'net11', by
l-rlterclianges in
ti-ie
- 3 8 and o, 0.
synbols U,lf
the four orthogonality relations
If
are taken together to define
between the menbers of the pairs orthogonality proposition
the further
then there is
as hetween the pairs,
prcper
pairs making distinct
that reciprocal
angles are orthogonal: err
( ' U o, ' 0 . r I ( U u, ? r ? ).
o * P : )
nC
T h u s , i n t h e f o 1I o u i n g s c h e m e , o n y o n e o f t h e o r t h o g o n a l i t i e s indicated
i r n n li e s
r^..-.
all
I U U I
^- r
^ 1 1I
d l l u
r
f ^,,r
are
I U u r
4 I
i mnl
.i
ed e u
hu /v
the
distinctness of 0, S
4 --.r.- 14
"l r/\ ? (I ' ,
tlf
rJ
I. i
v^
V P / .
r ) w B v r r d r a L )
Consider a pair of spaceswhjch are suclt that any direction
i-n one space is a reflexive
direction
of that
s p a c e w i t h r e g a r d t o t h e o t h e r . A n e q u i v a l e n t c o n , , l i t i o ni s t h a t t h e s n a c e s h e o f t h e s a m ed i m e n s i o n , a n d h a v e t h e proper angles
l r e t w e e nt h e m a l l
qn2eos
cal I ed
wi I I
ennrlitinn
nrnnpr
he
thetL
enolcr
is
y ^ " f . ' . . , 6 f v * ,
i snonn:l (
F
sJ Pnd-r -s ^) .
C , X ,
c n q r r
directions nfhpr
cna.a
ir cJ nv cB nv nr ra ol t
in
reciprocal
af
E , ) ,
ny ra i t r t
^-
necesstrv
d i m e n sJ ir O v rn r
" t
^ n l -
1
-
e - x c- y l' P
q n t e e s J y u r L . , t
one space projects
ir nr r f ( ov
the directions
A
and n H
[u p!
suffic
ient
jr Jc wn6ov nr r na a l r ,
uw rr Li rt rh
that ? (cos-
T r rn/
the same' Sucha pair of
d l l
^ - +L1i r, v^ S^ v^r -r -a 1L
v I
with their
= O.
d t L /
u l
- + L ^ ^ ^ - ^ 1 L I l u E U r r a l
orthogonally
-^ir y a t r
nf
onto
directions:
orthogonal projections,
pairs of directions
n : i r P o r r
r
O f
tlle tL hr reL nr
two
between the spaces are
r ,
nririno
P o r r 1 1 1 6
- 3 9 -
obtained 1{hich are Inutually orthogonal. ,\ccordingJy,by taking an orthogonal set of Jirections spanningone sface, anJ tlren takjng the ortlrogonalprojections of these clirections in the otlter, there js obtained an ' l ' h e s et w o orthogonal set of directions spanning the other. sets of di.rections are arranged in a set of reciprocal ;:airs ' in either
its
i.n the other set,
reclprocal,
is obliqtre to
one of thc ttr'oscts,
Irach Jirection,
the unique
it
naking with
proper angle betneen the spaces; but it
is orthogonal to every
oiher
the
,u lf ir rr ! er e r v t r it n t n
Thc construction of
in
directj-ons
for
their
the
to
a gencral
8.
Total- reduction
in
of
canonical
di rcc L!ons
other
in
tliat
in
angles
made ity the
canonicaL
in
angles
lvhen togetlier rti th
to
exception
reciprocal
o{
cich
pairs
of
every of
s;,aces'
at
in
oblique
the
di recti.ons .
tlrat
one in
every
the
other
nlost cne.
fhe
define
tile
tlre dccornpositicn. canonical
c li l e c t i o n s ,
be tlte ortltogonal.
si'accs, and thus 'l'he
any
to
dircctions in
resolved
are
orthogonal
oL tlre SpaceS, nust other
they
property
c r b l i c l L r ec a n o n i c a l
appears inr,erliateil,- that
projections
the
onc space is
betrieen the
such a reduction
notn'to bc exzended
be consj.clered tctal l)' redLrcetl
sfrace, ancl also
space rn'ith tlre possihle
It
to
each other
direction
forn
snaccs.
to
canonical
tvltich is
bases the
sp.Ices constitutes
isogonal
spaces is
of
rclation
into
pair
8ct.
ot|er
orthogonal
such reciprocal
deconposition,
total
A pair
of
in
i-tS Orr'nand
in
botlt
canonical
form
angles
are
- 4 0 -
the proper angles, uniquely defined;
with
then identified
deconposition,
so in every total
the canonical angles ob-
tained are the same, and given by the proper angles, the squares of the cosines of whi.ch are determined algebraically characteristic
as the biprojector A total
reduction of any oblique pair of spacesmay
now be obtained by the following principal
values.
algorithm.
reduction of the spaces, into
projections
in each other, .t x. . = o, x
Then there is
f "y
s c h e m eo f d e c o m p o s i t i o n :
where the components €
X r c
'
orthogonal
R - . x y ,. L_. y. = e _. .y 8_ .x. @- , ?_.y. x. . . .
= ? f . * , o , u y L *=
"*ty
their
take the
and the orthogonal residuals:
@
the further
First
€
tr, o,
3
Y r d
are isogonal; in particuiar €.*,o
C I
-e. r e
i dcnf
i e2l,
Yro
heing the intersection uy L ,A .r v^ = E^, .n E_ . -- e /
but all total
'
ro
o t h e r c o m p o n e n t sa r e o r t h o g o n a l . N o w t a k e t h e
d e c o r n p o s i t i o no f t h e i s o g o n a l p a i r s ,
orthogonal reciprocal
pairs of directions
into mutually all
making the
sane ang1e. Fina11y, take any orthogonal bases of directions in the residual
s -p a c e s f f^.l- . . ,n - . . . , . A 1 1 t h e d i r e c t i o n s y^
thus
constructed have togethel the property that they constitute a total
reduction of the spaces.
Itlatrices U = \1*r, or order Nxp,
5xq
where r,
\t = Vy. s are any regular square
matrices of order p, q are any bases for
the spaces
A 1 ar
-
= F I'v 1 v LFr^ J,
-
F = f-v^ ] . r h e y a r e o r t h o n o r m a l b a s e s , c o n p o s e d
q q of orthogonal sets of unit
.l U'U = ,
\/a.fnrq
i f
V'V = 1,
in which case the orthogona l p r o j e c t o r s
on the spaces have
the simpler forn
= U U ', e . , = V V t . ,
Tf
u'V
mnran\rpr ^
v
v
v
v
r
t
0 \
u,.' f : , )l
,J
( .
\
w h e r e1 > .
u,)-
pair of hasesfor the spaces.In this case (Ui, Vi)(i will
be normal reciprocal "*
...,r)
pairs of vectors on the spaces, with
Ia y -= fLr, r, . . . r r, fr Jl t 1r
o
= l,
^
F
"y.-*
-= [Ur "' l , . . . ,
v J
"fj
and
A
xy
The problem for spaces is,
constituting a total
reduction
vu ql J
of a pair
a pure matrix formulation,
of
that of
the bases, with which the spaces happen to be an equivalent canonical pair;
sinultaneously pair.
r . '
= f u L t r r * 1 , . . .' ,. -t 'rp rJ ', o^ t y x L V r * l ' " "
to give it
transforming given, into
= =
rotating
alternatively,
orthonormal bases into
to
a canonical
The analysis which has been given shows the possibility
of this,
together with an algebraic algorithm for
Alternative
the realization
computational procedures are as fo1lows.
f i I
Q i n n o
\-,
' x M" x '
l\4t
' x - y l"\ ,x1
Nlrc
e r e A n a ' i r o f s v m m e t r i cm a t r i c e s . o f w h i c h t h e f i r s t poritive
is they
a n d t h e s e c o n dn o n - n e g a t i v e d e f i n i t e ,
definite,
can be simultaneously transformed into
matrl-x, and a
the unit
n o n - n e g a t i v e d i a g o n a l m a t r i x ; t h u s , r , ' i t h s o m er e g u l a r s q u a r e natrix o, ^t\l t
\ t'
=
xt"
1'
, s. , t' \ 1xt ' ey. , 'V x. o-
I Z , u
=
n\
t
l,
o /
\ 0
w h e r ep i s a r e a l d i a g o n a l m a t r i x w i t h n o n - z e r o e l e m e n t s , of order the rank r of exey. Take LI = Mx o = (tJoul), V* = erLJ = (\': VT) , f h a
n r r f
i f i n n <
h o i n o
a t
t h p
. t h
. ^ l r r m r q '
c A
i h . f
U ' U =1 , u ; v : = r ' l ' r '=l u z ; and take Vo = t'f u
-1,
so that
U f , \ t o= u , Now 1et B., be a base for \ - T * = \ t , ,P , , 1
y
t '
V; \ro = 1.
the nu11-space of ltlre*rlr; and let
t h i s b e i n g a h a s e 1 - o rt h e i n t e r s e c t i o n
of t,. rvith the orthogonal complemeno t f Ex , €..O (., ) ^ ) which has heen seen to be orthogonal to d*. Lct \', be "r ^ "x{ an nrthnnnrmel a n. { r r i v a l e n t o f V i " ; a n d l e t \ ' = ( V o \ ' l) . ThenU,V are a canonicalpair of hases, havlng thc properties
o
u r l ; = 1 , v ' v = 1 , u , \ /= I u
(ii)
If
obtained, it
some orthonormal base lJx of
g-- has already been X
can be rotated I'y an ortlrogonal transformation
w h i c h v ; i 1 1 s i m u l t a n e o u s l y t r a n s f o r r n U x ' e . , U x +L ^V
v
Y
Take ll = U" n , n o r e s i- m l v, " ' rn' 1
\.
0 )
\0
Y
\"' = e.,ll, and proceed as in l
e. - x = l J U t .
I : ^ ^ ^ , . ^ 1 UaABUttOI
(i),
f ^ - Mlil.
where now,
-
A D tJ
-
To obtain an orthonornal base U, it a set of vectors 11,, ..., r
is
required to rcplace
n 4 -h y a n o r t h o n o r m a ls e t U ' , , . . . , P
+L.
in the .tn ,tage, tlr6[\11, ...,
in such a way that, (r = 1, ...,
tl^
Mrl = €.
P); in other words, to carry out the Gramm-Schmidt
o r t h o g o n a li z a t i o n p r o c e s s . th
stage complete, and the orthogonal
A s s u n et l r e r - 1 " "
f h n r ^ 1 6 . f
n r
-r_l
a
- ^ r F. . _ l
^ .
by taking 'Nr and
stage is completed
r r f cn n om nm p ua tr le c lT. h e r ' "
_1 Z p g = i' N = ; . l v t ' N ' t 'l ) ' " r "r-l"rt r'r'
mnrpnrrar
e
r
= e
+ lJ U' r - t" r r
9. Analysis of configurati on T w o p a i r s o f s u b s p a c e so f a E u c l i d e a n s p a c e m a y h e consideredequivalent, or the spaces in the pairs to present the sameconfiguration +l.^.. tney tL hl t eg
^ ^a- n C .i
I lm l L rd o B qe
A linear
LD ^e vn r f
-r il ^E i l C 'l tl r r tL hl l cq
vn Lt rhl Le
ynfafa,l
r
r
n n i rr P o r
i' n, !f vn
nn r !p! L v il nr !n i , l- ul aL nr e
iu lr nl ud! c r r
^-
4 l l
if
to each other,
in thej.r relation
r
a - +L1i l-u 6 u l r o r
v r
r.'itl" "..,I I
One
pall
+L rr a n S f O f m a t i O n .
transformation which preserves orthogonalLty also
p r e s e r v e s a n g l e s ; t i h e n c e t w o e q u i v a l e n t p a j -r s o f s p a c e s h a r - e total
reductions tn'ith the sane canonical angles. For, an
orthogonal transformation wlrich sends one pair of spaces into t h e o t h e r s e n d s c a n o n j - c a 1d i r e c t i o n s canonical directions it
in one pair
into
i"n the other, with angles trnchanged.
has already been settled
that different
total
reductlons of
t h e s a m ep a i r o f s p a c e s a l w a y s o b t a i n t h e s a n e c a n o n i c a l a n g l e s , oirren BrvLrr
hu w /
fhe
yn .r "n t n, e . r
:noles:
anrl
nnu
tlrc
ntreqtion
afiSeS
3S
- 4 4 -
spaces wi th
to whether different
each, are equiva) ent;
thus uniqLrely definecl for anp.les wilI
spaces.. It
is
only
f o r n , , i n t o r v li c h p a i r s
nical
:rnoles can be rotateci . pai rs
of
ae rnr oL l r n! c
spaces are
h o f r i e p n
t h e n
It
of
so that
the
of
the configuration
necessary
to
present
a cono-
spaces riith given canonical
rr'i11 then be establisl-red that
ec.tti-valent j f
and only
e r e
n Js U
t h c
Js (a r m, .n, 'Lt
tLhl c' c
i f
of
two
tlrc profier
n rr nv nl re( r I I
spaces givcs a conpletc characteri:ation
L ugLhg5ll
2 Inl ^Hl r^E- J d
their
configuration
rrnder ortlroportrl t ransforrlat ions.
invariant
Nornal
10.
anples,
analysis
a conplete
constitutc
fo:rtred by the
t h e s a r n ec a n o n i c a l
form
An orthogonal detcrn'i ne,l l-,v p o n a l s et
of
5",....
in
transfornation
a fuclidean
space being
corl'csnon\lcn('c hetr.oon nno cnnnlc'tc ortlro-
rrnit vectots
and anotlrcr. lct
t l e c o n r t li n a t c
v e c t o r s o f t h e s p a c e b c t a k e n a s o n e s u c h s e t ; a n c ll e t other be clefinedas folloris, in relation
tlie
to a given pair of
s u l . s p a c c s .T r k c t h c v c c t o r s . i n a c a n o n 'ci a l f a j - r o f l ' a s e s ; 1
ha
to thc j.ntcrscction,
t)rc duplicatcs arort thcse, beicnginl
lct
.--^.-^'1 .
a
fol
\
l :
v
l
thc
\
v
.h.l
'
arron.nnt tlo
orthogoral
con,plerent of pair
and 1et every recilrocal. ninecl by s6s o = ll '\',
of
orthoncrnal nrkcs
\/ectors base.
n:irs
nf
ll, \'naking
tlie
spaces;
an angle o,
deter-
be replacecl by an orthogonal
pair
- \' sin
o/2.
tlrus obtained
constitute
tlrc f ivcn pair
r , ' i t h c a n o n i ea l
the rrnion of
U cos d/2
'llie
Of tltOnCfn.al lraSe
L,a an
+ \'' sino/Z,
U cos a/7 The set
ren:inJer
orthoconal
of
transfornation
sl'accs tlre inaf.c of
I'rscs in thc
h:str \'ectors not
in
the
second thus
a pai r
[ol Ioriinp normal [orn: the
iltersection
defined
of
sf aces
reciprocal
have the
forn
- 4 5 n
0
'o/ d/2
cos
2
ct/)
-si:n
c'/ Z
0
0
0
0
wlii le al l
others
l.-
t h e fo rnr
\re
0
\ 0 l 0
;l l-rrerw
nair
f.-.,
o{-
crrhqn.req
li'ith bases in
rr rl
('+^+:
.
J L d L
'lhe
J U t l o l
ic
tlro
ortlrnoon-rl
irece
--
of
a lalr
such a f orn.
)
,v .( l r i a t i o n
canonicaL Cirections obtained in a total
redtrction
o f a p a i r o f s p a c c s h a v e b e e n c l i a r a ct c r i z e d b y t h e p r o p e r t y that those rvhich form oblique Fairs a r c l " e c i r r o c a l s . tlrov
arn
spaces.
tlre
orflrnoorr:l
Now there
cllaracterigation.
nrnioi'f
ions
of
each other
h,i11 be slioh'n a di fferent
in
in
tha
the
L.ut equivalent
in a pair
directions
An angle s madeby certain
of
s p a c e s w i 1 1 h e c a l l e d a s t a t j o n a r v a n g l e b e t w e e nt h e s p a c e s if
it
undergo constrained
stationary when the directions
is
appears that the stationary
in the spaces. It
varlation
the proper angles between the spaces;
angles are precisely
characterized as reciprocals
and that directions
as directions
characterized equivalently
are
which make stationary
angles. Thus let l I = h c
e n w
r r n i t
rl vn Jc
ir rnr a\
lP
t
Y.
^
, \'/
= M s
y
in the spaces, making an angle with
v c c t n f s
r^rhAre
l \ l r
s are noh'vectors of order P, Q.
t
Then rrI'l rl\1 r
-
I
s t M ' N fs =
t
v v
and
r'V|\lrs = u for
The condition variation
u to
the
under
constrained
is
), Il'\l r. r x x
fl'lrl s = x y where \r,
be stationary
N 1 . ' . 1 t 1 _=_ rI - I l . ' . N i . . s , /
^
>
l
/
\ s a r e t h e L a g r a n g i a n m uL t i p l i e r s
to the constraints on r,
s.
It
dyPvdL
)
ilvw
,
corresponding + L ^ + LllaL
trr =u = trs B u t , r v it h
l,
= I
s
one of the co straints
w h e n c e a n e c e s s a r y a n d s u f f i c i e nt stationarity
under constraint /- uN{'r{
I
x x
I
\
tynt*
Thus the stationary
l4'M
i
condition for
the
S
\
x y II
- u l r l' \ 1 ,
b e c o m e sr e d u n d a n t ;
I / l
(, r'lttlM*s = 1.
t , \ S /
values of ir are those which satisfy
- 4 7 -
f h a
a n r t a f
i n n
- u r\Iit\rx
riiNly
l\!t t\t t ^ Ilrrf
i n
of titc idcntity
\r l Clrl
, .2li t- uJ
tI - u l . l\' t rXI II i
- , ) l * q i i . i ' r r *l l r ; l r , I l u 2 r - " * " ,I
=
I1'I1 I x y i
); ' I\1 y x
I
this
- ,,r!r I! P ' , , . , , , l l
I
: u itl|Il,,
I
j
e qu a t i - o n a n c l t l r e e q u at i o n
u
2" l - e e
have the same positive The proper
angles
roots,
\!ith
sane l,Lrltiplicities.
angles
titLrs identical.
are
analysis
th'o n'u1tip1e factors
o\nerirrcr't in uhiclr tiicjr
x,
y
correlaterl
s f a n s q - . = iL fr lft r Jl .t ;e .) '
lrave canonical l.ases U = Itl*r, \'= to
the
and tlle statlonary
12. Canonical correlation Consider
t ,l = [ )
x y '
in
an
= [, -f" 1y ' . . ]
Ilrs, Tlrcy are cquivalent
factors u = xr,
t{llich I'ravell,
\'for
t r l , , ,= I 1 * ,
llrc clcnrentsoI
r r,
their
Ineasurement natrices:
= N*r
= [,I,
r',]cfinc
They have the property tlre pairs
rr', \''
(i
v = IS
=
that l,
lrt., = )lr,
= ilrs
= \' .
c a n o n i c a l c o n ' p o n c n t sf o r they r)
are
al1
of
rinit
lravepositivc
x,
y,
variance;
eorrclatiorr,
- 4 8 -
g i v e n b y n u m b e r su i
the squares of which are the characteristic
v a l u e s o f e ^- e ). , , a n d r n ' h i c h d e f i n e t h e c a n o n i c a l c o r r e l a t i o n pairs,
coefflcier)ts of x, Y) while all-other
taken fronrwithin
and fron between tlre two sets, are uncorrelated. The derivatlon
o f c o m p o n e n t sw i t h s u c h a p a t e r n o f c o r r e l a t i o n s t l . r ec a n o n i c a l a n a l y s i s o f t h e f a c t o r s .
constitutes
I l a n n a n( . l 9 6 1 ) p r o p o s e s a f o r n t o f c a n o n i c a l a n a l y s i s , a s f o 1 1 o w s , a n d w h i - c hh e t h e n
r"hich he derives essentially
therc is the ortl.rogonal
proceeds to generalize. firstly, ,-tloenmnnqitinn
+R I Iy = l r i r x x y w h e r e I I I . R= 0 , B y a w e l l - k n o w n g e n e r a l t h e o r e r n o n n r a t r l c e s , x ' = r.... o6Br, uhere o,B at'c orthoSonal nratrices and 6 a -\/ r;^^^,.-1
ur4Evrlo
-a+ri-
Tlren
llyB
= f'1*ct6 + lL B
I l' ) ' n
= rl "Xq 6
That is,
where S = RB ;
regrcssion
Ilannan, xo for
t
i' s) X e
also d l J U
the
so thar
,
1'B on l.o is
sense of
1i^^^-^1
u l d Y U l l d l
t
r*o
give
,yB
lliL
=6
= 0. Thus
diag,onal. According
t l t c c a n o n j - c aI
thcrcf ore so as to
a nornalization in
S = o'}liRP = O, since
)l*o6
n:atri.r. of
yP a rc
,
woulcl be true rt y B -
l{io
I:ioltly, = lli^
Therefore, the
sc that
+ S.
each uttit
IIotelling
if
to
c o n ' p o n c n t s, h u t Thls
variance' and only
if
^ - ' r +( rLr 6 ' 1 9 a r e n o c o r r e l a t i o n s
d l ' \ r
b e t r r ' e e nt h e x o ' s a n c l b e t w e e n t h e y B r s ;
and this
is not
g e n c r a l1 y t r u e . T l t c c a n o n i c a l a n a l y s i s d e s c r i b e d b y I l a n n a n is therefore apparcntiynot the classical one of HoteIling.
- 4 9 -
P A R TI I I
'I
.
Ohl innc
nrniect
jons
Any vector in a Luclidean space has a unique resolution into
a s u m o f c o m p o n e n t si n e a c h o f a s u p p l e n e n t a r y s e t o f
s r r h s n a c e s .t h a t i s a s e t o f s r r h s n a c e st h e s u n o f t h o s e dinensions is the dinension of thcir
union, atldwhoseunion
i s t h e r ^ h o I es p a c e ; a n d t h e s e c o n p o n e n t s ,o b t a i n e d b y I i n e a r transfornrations of the vector,
define the project-iotts of tlre
vcctor relative
t o t h a t s u p p J e n ' e n t a r sy e t . C o r r e s p o n d i n g i Y ,
the identity.is
resolted into a sun of projectors, the
i d e n r p o t e n tl i n e a r t r a n s f o r m a t i o n s w h i c h o b t a i n t l r e p r o j e c t i o n s , the range of each being one oI the spaces, rihile its j.s the union of the rest; n r n i o e t n r q
i c
n r r l lI .
nu11-space
so the product of any two of
a ^ - " ^ - - ^ l r ' LOlIVeISel.
y t
ally
i c i e n r p o t e n t ss u m ni n g t o t l l e i d e n t i t y
^ + >€L
^ f , UI
1 1 . ' IruLudrr/
these
^ - - ; L ; 1 ^ + : " ' dlrlllrrlr(rLf
are the projectors
'rb,
on tlle
s u p p l e r n e n t a r ys e t o f s u l r s p a c e s f o r n e d b y t l r e i r r a n g , e s . l i i t h a n y i r l e m p o t e n t ,i t s c o m p l e m e n t a r y ,a n d i t its
ranse narallcl
projector if
it
is
range and null-space are
iCentifieC with
tn ifs
n r r l l - ' sr n a c e . I t
thc projector
on
iS an oltltogonal
is synmetric, in which case its
range and
null-space are ortlrogonal conplcnents; and otllerrvise it
js
an oblique proj ector. Ilore generally, rclative are just
t o a n y s e t o f s u b s p a c c sw h i c l r
independent,and not necessarily supplementray'the
s u m o f w h o s e d i - m e n s i o n si s
the dimension of their
sunr,not
I t e c e s s a r i l y t h e w h o l e s f a c e , t h e r e r n a yb e t a k e n t l r e o r t h o g o t r a l projection
of any vector onto their
union, and then, with
vector obtaincd, tlte o['Iique projcctions
Afrtet:
StualleB
the
on the supp-lenicntary
- 5 0 -
set which the spaces forrn relative r* ur t2 v 2 4 !n L lv
\rtr.tnr
is
one of which is
intO
feSOlVed
In this
to thei-r union. nair
a
into
resolved further
oI
coTrnnnnntq
orthoron:]
a set of oblique corn-
ponents in the given spaces, while the other belongs to the orthogonal conplenent of their If thon
union.
the supplenentary spaces are nutually nrn'iectinn
nn
eech
snAce
relative
to
orthogonal, set
the
is
the
s a m ea s o r t h o g o n a l p r o j e c t i o n o n t h a t s p a c e , t l r a t i s t h e nrnicctinn
nr
Spl i t Thn min:tion
n f' nn
now ariSes
nv r' "nJi e C t O r s
tlroir
rrnion
terms of I f
to
in
two spaces
Ex =
-
its
nrthnonn:l
exnlicit
to
^
rvjth
-\
rosnoct f
otheli
to
comnlonrpnt!
v
| ] l l , r
t"h"t r v/
relative
x
nar:l
t].a
-*a.i6-r^t'
lel
t^ ^f
,
fof
whjch
by
a n d , e Q U i v a l e n t l y ,t
/
r tx
relaf narcl P'
irre lol
fn rn
in
spaces.
'a l l ' l " t " 1l tr I II '"
there is defined the projector, which maybe denotedbI n v n, r te
to
A fnr1pgla can be
Are senerate.
given
qn.ces.
of
al ternatively,
on the
deter-
enmnl pnrFnt2TV
2Tp
the other
the
mejr
the
e
o,
a
foI
r ' l Ey = harJ on which
are c are
to
ir:st qcnerAfo
projectors
e - e . ,1 *
formrrlee
the bases, or
r l ll.lxJ ,
criteria
It
thc
terns of
nrnicetors the
condition
to
on one parallel
the orthogonal
nrtlrnonnrl
aS
r-nsc thnw ern
in
givcn directly
lel
defined
olro n:ral1c1
more generally,
or
n:rel
pro jecl-ors
orthof onal
nupstinn
ci ther
qnAcF
thrt
their tllg
rrninn
.. rx
COmplenentary
that
,y,
SpaCe
n v'
"*lu, is giVen
b y t h e u n i o n E - - O i ^ . . . o f 8' Y .. with the orthogonal complernent Y-'x,Y r' x r y
r5 x
of "
tlreir
lniOn: '
JS vO
e L x l, y
and nrrlI-qnrce Ee;..... a e r ' \ '
t l
-i S
tho
,'nin,'a
i,la--nrcnt
uifh
renoe
eithe
-51 on the spacest
Then, in terms of the orthogonal projectors there is
the formula -
"*[y for
( ' l
-
\'
o
o
"x\,
-
c
" x e- y l,
more immediately in
or'
the determination of u*lyi
( 1
l - 1o
"x.y,
terns of bases, and in a form which directly
generalizes the
formul a 1
= V* (l\1i]4x) 'Ml
"* for
on a space in terns of a base,
the orthogonal projector
there is
the formula e-
From here it
= M' x tr N' x, 'tY' e ' - 1 u' x' 'EY ' ' ' x- 1 M
x IY
is noted, incidentally,
'Z y o' x l y
=
that
a-
".y t*
The sum of the complenentary oblique projectors union is
nei r of snaces relati-ve to their
on a
the orthogonal
nrniectnr on their union: =
* Now any vector
z =
Z in
"*,y
"yl*
"*ly
t has a unique resolution
* 6*,rt * "*r>'z "yl*Z
' i n t c r c o m n o n e n t s -i n- - t- x ' C - -r y -X - . - y. - a n d C If
C-., t.. are, ^ y
complementary,
noreo\rer,
€ -A- r. _. .I = t
e-
and
^ t f
.,
so that
= 1,
sirnply that
for which the condition now is
P + Q = N, rhen c
e " Y l x a r e c o m p l e m e n t a r yp r o j e c t o r s ' of spaces; and in this
to a complenentarypair
'a * +y |
p
' y.
=
lx
1
on and parallel case
- 5 2 -
projectors.
3 . I t ' l u 1 t i p 1 vs p l i t
p, e, r,
e -L, ' . . . b e a n y s p a c e s o f d i m e n s i o n
€.., y'
Now 1et [-., x'
which are independent, and so forrn a supple-
...
to their
nentary set relative
union ,*,r,r,,.
of dimension
Then there is determined the projective
p + q + r + resolution 7
"
- =- x | yc , r ,, . , . 7'
cf any vector Z in t andE-... L r Y r L r
+=
y l cx r z , r . , a. "
!
+
' ;x , r -t z , .2 ' ,
i n t o c o m p o n e n t si n € * , e r ,
,
,
The orthogonal projector on the union has
" '
the decomposition = * "' "rl*rzr,.,+ "*lyrzr,,, "*ryrzr.,. t h e s u r no f m u t u a l l y a n n i h i l a t i n g p r o j e c t o r s , o n e a c h
into
para11e1 to the union of the other with nl omonf
^f
fho i r
the projector
gnl6n
all
ThUS,
tOgether.
on C.. parallel
the orthogonal corn€_t ., . xlY'2
it
""
t o t h e c o m p l e m e n t a r ys p a c e
given by
t ), , @ -c , e . . . C -I XEr I -r .7 r, .-. , '; rn.l
m n r ar nv rvr a r!
L
r
t
= o' "*lyrzr,,, a n' y b a s e o b t a i n e d f o r C - . , A t J t
" '
)
Sqga1.e
rank e*rYrzr,,, = trace If \l ' "\ {x
I \' ly
M--..
x t I : "
ls
M
""'xryr.,.
t h e c o r n p o n e n t so f
ic
nhtainp.l
aS
a
fegUlaf
then matfiX,
the inverse of which, when transposed and
c o n f o r m a b l yp a r t i t i o n e d ,
define matrices N-, N.,... ;
" ' X r Yr . , . '
thus ; f' M x My . . . l -xt r Y r . . ). -' 1 = ( Nx Ny . . . Nx , Y t , , ), '' . The projectors
which have been considered can be computed thus: r Y , 2 , , . . "€ x 1
=
M
" x N" lx '
"'
5 3 4. Partial
regression
Consider three factors Xt Y, z and the regression of z on the factor
c o m p o s e do u t o f x ,
together.
It
obtains the
resolution M " z = e" x r I r ' zl + 6" x r I M Fnr
thc
rForcqsion:l
nert
here
n ' , -=
^
f,,
there
ll
''"ry
the separation
is
t*ry;t
"*ry = L" .x, - l xr l ,
into
further
parts
Y'z
+ l\{ " Yr -
Y lx , z
corresponding to the
nc
rf
i t i nn
\' 1 ' X r Y = r\ \' 4' x M . , ) ; .r an4 conformably, r
x,Yiz
=/'*lt,t\
\,r1*,,/ Now
* Yryl *t*l y x,z ,z is
the regression of z on x, y together, given as a sun
o f c o m p l e m e n t a r yp a r t i a l of z on x partially
regressions, defining the regression
with respect to y'
and on y partially
with respect to x. It
is required to have a formula for
the partial
regression
m a t r i x r ^- l l , , ,L b e J o n g i n gt o t h e r e g r e s s i o n o f z o n x p a r t i a l l y I t with respect to )'. Splitting
the orthogonal projsqtor ex,y
into oblique componcnts,corresponding to the resolution its
range t*,,
t n t o c o m P l e m e n t sC * , t
= *V, y '= , - c " *\ l yl ' t + '
e
:
' y \l x l ' z '
of
- 5 4 Substitrrting fron the formuja rvhich has been given for snlit
follows inmcdiateiy that
r r r o i e c t o r " s .i t
-
= (l!|erl'!*)
t*1y,, ,-lirneilw
- l
' l ' t i -e * l t i ,
tlrc cnrreqnnnrlino
o e n n r qr lr ir -r ia nl o E ;
u r r L L L r /
the
, formtrla
[or
a
total
regress ion. forn'tr1ait
Irrom tliis natrix
Ir L^ Lo .r la! qJ cJ ri n u rn ,
n r l t r i r
n f
-z
repress ion
r e [ r , r e s s i o t tn a t r i x .
can be cxflressed as a total
n e r f ( I i da r l l r d l
to y,
seen tl)at a part ial
is
n - l- l
A
a p p c a r s t l ) e s a m ea s t l r e t o t a l
P d l
: ^ r 1 " I ) L i d l
. ' : * h h l L r r
The rI o s n e c t
relressiotr ntatrix of
tlrc residuals in tlre reEression oI z on y on t])c rcsidlrals in tlre regressj.on of x on y.
for
these reslduals fornr the
matri-ccs * x
-
*
-
. ' ! l' i " = c "111' " - x ' 1 " z, 1 " =- eY |" 7
'
u l r i e l r r - n n s i d o r e , aj s t l t e n e a s u r e n c n t n a t r i c c s o f ,^ = 2 -
z0),
i*
= s - s(yJ, cive r*x."x ^
5 . Inversion
, L
= I_1., -, ^ t )
r L
anclDartition
z = (x,y)
Consirler a factor Its
Iactors
neasrirencntrnatrix has the
corfoscrl of
sulrfactors x,
narf
fnrn
i+inrp.l
=
"" !
.
y.
(I1.1.,,) \
J
a l r t l c o r l ' e s p c n t l i n g 1) ' , riz))z
-
/r.'x"x
I \rr;lrx 'llro
nrrncf
inn
n o n 1 s t o o L ' t . a i . nt h e r' l , 1| \ ! ) 7"7'
-1 z z
where i i = f ' \ x \ yl ' z
li'
xl\ly \ ,
,'j-"I
inverse in a sinrilar forn
- 5 5 -
confornably with Mr, and moreover, is
is partitioned
such
th at
N i M ,= 1 . It
that this
i s n o t i - m m e c l i a t e l yo b v i o u s ,
though it
can be verif ied,
s c h e m ei s o h t a i n e d b y - l . l, y r, "vy' E N . . x- -= -ey ,t r, xt [t ,r ., '1* ].ye -, 1- l/t - ) - 1, N - xr ' 1 . . ) - 1 . "y = 6 x t
It
is noted that,
f o r a n y m a t r i x t l w i t h i n d e p e n d e n tc o l u m n s , of matrices of the sameorder such that
there is a variety
- 1' = N ' N , N ' M = . 1; ( l ' '1M )
for
a special example, -,I ', N = U(Nl'\1)
exanple, as just
and there is a further ,w.a, . +L: trr . d^ r-r .j ,l
n a + +L .r i *L ;r l O n i n P d r
COlUmnS Of
the
indicated,
associated
14.
Now Nl- \': = I'l-.N-'- + I|..N.'., y y' 7 z x x which obviously
n rn jp6tnrs y r w J l u L v l J .
4--ihi
ll daL tI ir nr tcs
snl i t
nroiectors.
d t l l l l l r r
one projector
gives
e '
Tr ihl cL rnr
there
x ,l_ y
=
i s
V \ 1|
,
ri rnr
as a sum of
Vi-ew
made the
Of
tlre
two mutually
fOrmUla
fOr
identification
= M,,N,', ,
c
x ' x ' * Y l *
, )
and hence, though otherwise not obviously, e- = i'lrNi C i v e n a r e c t a n g u l a r m a t r i x \ 1 w i t h l l u { ' MI I 0 , i t according to the definition - 1'
inverse given by \1
is,
whi,Ir is
thc
The natrix
transpose oI
thc generalized
h e s t" h" "e nF r' "orn- e' -r- t- i e s - l
(fi{'M)
N defined by
= N',
NI that
of Penrose (1955), a generalized
- 1 = (M'M) 'NI' - l
has,
= N'N
, N'I'l
= 1.
inverse,
- 5 6 -
to make further
is now interesti4g
It
observation on the
fo rn o f
( l u ; l \ 1 21) = n - l N , , N l l l , = 1 , w h er e N, = (NxNy)
( ) . l x l , l y ),
Il
Thus, R l "' X1 ' l 'l \ Y = " "1 x "' Yt "x = o 'Y
are the rcsiduals in tlre regrcssions of x, 1'on eaclrotlier; and 1 |
1 |
,n . x -- , I r l x r , I ) . y - - . rf , ly
h. v .
Ivnerimnntnl
L / \ l \ r r r , ! r r L u r
TLtL
^-., dtLf
^-.-^-; sAI,sr
a fictitiorrs
r
fr h" 11 v' ^v 6r r d
1-ho
n,pn t
evncri
u r r \ r
rnen t
^ 1 , . ^ . . d r r \ d /
facto
rrrrrlrenop,lv
nnd m , , Lc n n \alucs
ru rr r nr ri vf a nl rrr nt ' i l v ; + I L
i ^ l >
s nossil'leto cntertai
t o b e C e n o tc d h 1 ' I ,
r
it
ho
rrhiclr remains
define s the unifornr factor
k i n g the valLle 1 o n a n y o b j e c t . I t s
rJeasurelncltt natr
thus
i ^ A J
i r -
t 1, = / 1 \ t l l
I ' l
tt '' ll \ r /
the vector with N o w c o n si d e r uniform factor
^ l
t
+1.e
i
+ ^ L >
g^ 1 a^ -q ^t !" t .g+l l^
r e g r e s si o n o f
I , thu
I,I
X
c -tri I
x
*
t
i
lr x
Since
,\ 1. rI l't' I it
L5
follorvs that I
N;TI\1i
equal to 1. anv
factor
x
nn
thg
o
-
D'J
that
and therefore
e r l r l *= I r l r i = n ' i ( r l w h er e Y "
1 N NIl 1 X1
' rx ' T-
=
(I')
of x in tlre cxperin'cnt, it
defines the mean value jts
^
being
c o e ff i c i e n t s o n t h e u n i f o r n f a c t o r ;
vec,o. o*n
e x p e c t e d r r a l u e o n a n y f r - r r t h e r o bj e c t ,
and i-t gives its
unif orrnlty. the residuals 6rlrl* r e g r e s s i o n m e a s u r et h e d e v i a t i o n i = x o f x f r o m i t s
s u bj e c t t o t h e f i c t i t i o u s in this
mean, tlius : -
l t
^
x-x
7/
r
l ' r r l fL r iPnr \l o
'[he of
r^ n! oA rr P! S S
r : u r
rr-'gression
x
iOnS
a factor (x,y,2,.
w of
a factor
of
composedof
dimension p+q+1+
X r yr z t . . .
l l
r
.l
subfactors
d i n r e n s i o n p r g r r r . . . h a s the form
of
( - , l r \ ^ , J r . . ' ' r v \r ^ r ) , '
^
, w
/ ; L
s
. . ; r * Y t y X r z . . .; w
r v h er e
xlY,z,
';l!'
xrY' "
Ylx,z,
Just as the total
l'rasthe determi-natlon
t {
-
"'(*ryr...)r hu rt r
n , uarn, vn rc r q v rv t l( r, n
l
nrniee
t
so the
inn
--
rt r \I X I
r
xl Y, z, . . . ll{
E
XrIr
XrIr.'
"xl y ,z
parts N1
h ave
tr1 the deternrinations
- 5 8 h u rl r
n w hu l r r iYn ur rs n
nrnioe
tinnc
lrv
, l \e tLo\ r m i n e d u
,
tlre
o b I I o,u c' J)1'o J e c t o r s
into which thc orthogonal proj ector is spli l - r a v eb c e n o b t a i n e d i n
t
But
these
the f orm -
" * fy , r , .
n t
' ' ' xl i' l t x t
.
..'
so that I lr , x \1 y,2 ,.. whence
flro snt
nf
=
t\i 'l \ x1 " \ xi ' l '
rcoressinn
nrrtiel
' r '
t
nntrices
are obtainecl
i n t h e fo rnr I
R u .
\ l r ra ltL ri n l nL , ' !n vnr fr ai cf r r r r t i n n s t , r
as varioLrs binalf
appJyinp. to factors
any Iair
of
througlr their
a pair" of
harre been considcreC,
rclations
spaces, anJ tlrereforc sf ans;
spaces attajns
to
or
or
tieparts f1.on then,
dcpartrrr(' J:ron thcr: fi\cn
coefficicnts
arc
of
l'y ccrtain
tlre spaces.
flrtl renain
for
rOIC
thc
jnvari ant
attainn'ent
cocft'icicnts,
l'lrese rclations
consic'lercclfr,rnciions of
forn'ctl b;' tlrc slaccs,
so the
l : o l . li r r p b c t r i e c n r .
ancl tt'],ich har,e criteria
togetller,
c -elf i n c ' 1 1a s f l r n c t i o n s
har,e been
i n r t l . i c l ra n y c o r I i I u r a t i c n
spflccs, b1'definiril: n-ilry rclations,
to
any fair
t l ' n e n r . fi o U f a t l O n s O f t l r l e i ' O f
sArc pi olrt lrc rlonn fol
spaces taken
to
and coeffici-ents
t le f i n e , l u i . i c i , m c r s u r c t l r t ' e \ t e r r t of
'
r , \ .
Just
of
r \: ^ ' X ' t.l' 1 r '
I
X I Y' 2 ,
attcl
configrrrations
un\lcr ortltoqona l
transfornations. The nrost obvior-rsnLrltiple for
a r r y r r u r n l r c ro f
qnpci:l r{^*^ lluIL
thrn
this
1 + l ^ l l L i l L ' , I ( r t( l l J l l
sp6qc's is iS
relation
to
be conslderecl
nutrral ortlrr:lionalit)'.
tlte lelation
+( |1L. ; ^ ^. rlr'llu(rurl ur
of
):ore
ntutual distinctneSs.
.r .j ^ Lllr)
- - 1- r :^il ld( lulr,
. n l ,i cl ,
i s
- 5 9 -
of linear
the occurrence of a coincidence, is a relation d e p e n d e n c eb e t w e e n t h e o r t h o g o n a l p r o j e c t o r s
on the spaces,
which may be terined cornposabilitv. An equivalence of the the condition that there exist
condition of composability is
unions identical
two subsets of the spaces, which have their and within
each of which the spaces are mutually orthogonal. o r t h o s" o n a l
+
t, distinct A coeffi.cient will
extremes
composable
o p p o s1 t e s ^-_-/
coincident
t
be defined which sets the configuration
formed by any three spaces in a scale between the extremes oI orthogonality and conposability. For three spaces, composability is that either a pair of them are itlentical, or a pair
are orthogonal and the third
their
uni-on. A natural
the coeffici.ent to any numberof spaces is
extension for readily
is
suggested, and partly
established.
Let the number xtY rz
determine(l for
,
a*, C*,
tr*
'
'r,
ct*
'r,
1
a n y t h r e e s p a c e s s y m m e t r i c a l l y i n t e - r m so f t h e
coefficients
of association between thej.r pai.rs, be taken
define their
coefficient
of dissociation.
lt
can be sholn
to have the property of being non-negative, and equal to if
and only if
to
the three spaces are composable. It
be seen to be at most one, and equal to one if
will
and only
t h e t h r e e s p a c e s a r e m u t u a iI v o r t h o g o n a l . F o r i t
is
zero now if
seen that
- 6 0 -
o < K " = ""
t = t(" YCz - c' ' x Yc" x 7)t z = ( 1 - " cx yz' t n- c" x 2 z) "' .j 2 = -z -2 -
ryr t
J*y)*,
L*yL*,
'ty.-
. 1, < s1,.s?.^ f
^ L
the bounds being obtained since the S and C coefficients
It
follows
equals 1 if all
between the three spaces
the K-coefficient
that
and only if
the spaces permuted.
wjth
bounded between O and 1; and sirnilarly
are
the S-coefficients betweenthe pairs
equai 1, which is if
the spacesare mutually
and only if
^-+L^^^-^1 v I LrruEvrrdr. ^ - - ^ - A ; h ^ r ' nLLVr srrrBa
jr t
o t n * , r , ,4 1 ' K-_,. _ = 0€:Xomposable X,Y,Z = 1ts;'orthogonal. In terms of orthogonal projectors,
composability is one of
the conditi-ons e _ _= e , , , o r e _ e . , = O a n d e . , + e y. . = c _ , x ^ y x z ' ) or one of the conditions obtained by permuting x, y, Now for
any numberof spaces, corresponding to x, y,
d ef i n e v
-
1
C
X t Y r Z r ' . ' c
C
xy
l
xz:
f f
C zx
zy
L
1
;
can be shown that K
XtYrZr"'
> 0 = 0
It
C
'
'
) ^
Then it
z in these.
€J
conposable
is obvious that
Kx r Y r Z , , .
= 1 Q)
elllegonal
z,
-61 -
and it
that
to conjecture
is natural
s 1
xrYrzr"'
= 1-
orthogona
a s w i 11 b e p r o v e d . Fron the identity Mt
N'tr!
N l l\
and the properti,es 1 - efi
, it
with equality induction
that
= l v ' M lNl ' N i1l "rI
li{rM
w hi c h h a v e b e e n e s t a b l i s h e d f o r t h e q u a n t i t Y
f o l l o w s that MIM
! l ' NI L
NIM
N'N
i r r c f
i f
MtN = 0.
M ' M l lN ' N l
(Fidrer, 1908),
It follows immediatelyby
if
A =l A'.'A.,, l
"
I ^ztAzz | t " " " ' is any synmetrically partitioned positive
def j.nite
matrix,
then
l A l - ([ A 1 1|] o r r l with equality
just
if
the non-diagonal conponentmatrices
all
A.. are nu11. 1J
In particular,
w i t h c o m p o n e n t sa l l
i.nequality is obtained.
of order 1, fladamard's
An advantage of this
the most general inequality
approach is
is obtained directly,
with necessaryand sufficient
the most proved results proofs in the literature.
together
c o n d i t i o n s f o r e q u a li t y .
( 1 9 6 0 , p . 1 3 7) r e m a r k s t h a t " t l a d a m a r d ' s i n e q u a l i t y
that
Bellman
is one of
in analysis, with well over one hundred "
- 6 2 An appli.cation of Iladamard'stheory (1893) on the i n m e d i ,a t e l y e s t a b l i s h e s
n'atrix
deterrninant of a positit'e
t h e c o nj e c t u r e t h a t r . v a sn a d e a b o u t t h e r n a x i m u mo f t h e o f J i s s o c i a t i o t t b e t w e c n s p a c e" c-2 x ' -eY ' coefficient X-... r ) r . . . which is that its value is 1, and that it is attained just wlicn tlre spaccs arc mutual1y orthogonal. lrlorcover, tlre t l ' L e o r e m ,. 1i s c o v e r e d b y F i s c h e r ( 1 9 0 6 ) '
nore general forrn of thjs
antl Pu, ( ,,
€- ^* , € - --). , .
K with
and
be
v "*ry,
.,,;arbr...)rrr
tlren this
cocf f icient
jrrst
if
all
i,,cr
il fI
U I L l t U E U l l d f l L j /
J U > t
the
sets . StiIt
L | 1 9 |
)
]
I ies
a set
if
it
Cefined
holds
for
an1'
Ii lK r a r b r . . .r . . / " ^ r y r . . . " I r b , bethreen O and 1; beitlg
:ero
conrllosablc,antl 1
hI Le t w e o n
nrore general
coefficients
SnACOs
talien
ffOnl
can be definecl,
s n a c c s a r C C o n r ihn e d i n t O
a n d s o f o r t h u l ) r c s t t ' i c t c tl y ,
are
def inition
^r,,-..-l^1.^1.rs
,
sets
preliottsly
d r t \ d / r l r r r u l u J
r .h n n q n l q c rf
i n r n r n i n n l r ' rt .
for
=K "Xryr,.. also
the
that
different
slraccs tal'cn togetlter arc
^-+l,a^^h^1i+,.
different
noted
t h c r - e i - s n r a c l et h e
Norv if
in
spaces
holds
conrfosaL;i1ity condition subset.
if
only
shouid
It
orthogonal.
aticlso forth,
4i\.. .. K., l ., ^ r. X r ) ' r . . . Y r D r . . . t r r I r . . , r a r l l r . r . r , . . if
equalitf
sel'era1 sets of spaces
Thus, for
leads to nlore general results.
f urther
S e t S,
l r e s c r v i n g a n a n a l o g o u ss c l l e n e
a t e v e l y s t a g , e , s h o r v i n ga n i n c x h a u s t i b l e c o n b i n a t o r i a l , qelrnnrtie
ellv
I
r! o - ll'\ r o , l r r e t i v e
nrnnFrtv
of
tl'e
rlissoci.atiOn
coefficient.
,.
l.irnits of
a s s o ci a t i o n
T h e r e i s n o t " t o b e s ] ' r o w nh o l v t l t e r e l a t i o n b e t w e e n a p a i r of sfaces, or a fair
of factors through their
by internreCiate relation
to a thircl .
spans, is limited
- 6 3 -
A l r e a d y t h e r e l . r a sb e e n s h o w n t h e i n e q u a l i t y ,2 LxyLx2J I
l.yr
irith equality i I and only i f oenor'2!
It
,liscrcn2nev
in
the
fo1lolr'sthat C.._ is
the slaces are comlosahle, tlle
cnrrr'litw
limited
y L
- S S C C "xy'xz xy'xz r^rharn anrroI i rv
anclonly if
tc
one
-Z _2 -*y)xr,
or
is
with with
r l if f e r e n r - c o f
is
f actor
associated
a third
of
is
limj.ts
is
'
attained
if
gir.,en.
are
for
the
of association and There is
cosine
of
to
be noted
a sum ancl
t h r o u g h j - t s m e a s r r r c n e n tn a t r i x
z,
a nultj-nornral di.stribution
clesirable
futtctions of
of association between a
characteristics
have been defined
to
1
( t . 1 1f r z )
itlentify
between x,
;
the
of
which
nteasurements I'ith
tIe
have been forned
)'i
ther
wit)r paranreter natrix
parameters,
characteriStics
These coefficj-ents
Ilr,
certerin coef f icients
as fr-rnctions of
the distribution
annerr 2s ,lirect
relaticn
the
coefficients
tl.ie formulae
A = and i t
"'*'Y'z'
rnclcs.
10. llistribution a
L
tlre spaces are conposable.
dissociation
Itith
hrr
4 '(-]x y C 'xz 3 ' - x z -+ 'Sr y ' S
z C = "yz
otlter
l i n r i t e c l w l ' i e nt l i e i r
an anology
oirrpn
tl-rus:
In this way the coefficient o r *a^i 'r
hpino
so that
they can
distribgtion. relative
to
a partition
and tlle measurenents have tire corre-
s n o n , l in o n : r t i t i n n lti. = L
^
[ ] ' _ l 'f 1 , , ) ,
f r o n w hj - c l . rt h e c o e f f i c i e n t s
are calcr.rlated. Correspondingly,
the parametcr matrix A, and its
inverse
- M -
r = A -1 = u ;'r, , herro nrrtitions
r = f L ^ *t - ,
o = / o * *^ *'r \
l
tl n
\
/
/
l r
)
A
\"yx"yyl
t
\
r
r
i
I
YYI
\Y*
where the diagottal subrnatrices are square, or of x, y.
t h e d j . r n e n s i o n sp , q t trly'NIx
t
r
,
"*
xy
= lt1*'l'1,
-1
. A * * = ( I : x ' 6 y l l x . ) - r , , \1x r
giving
The formulae the
coefficient
the
c li s t r i b u t i o n
-Rx2) ' or
the
--1 -
yx'**'*1'
1
)
''xY
"xI
t
'
of
between x'
s*,
R'
correlation y
and
as functions
of
are 2
.' - 1
1
= yy ' sxY lA lb I . \X' XX
x
rr /
ir nr fi n( L r er lLr re rnco l ci rr ' l l . L u .
. ^ r r ^ l a f
i n n
e n d
S i m i l : r l qwL t l ,
n f
i n e
fnr
l r r s i n n
I '- * ? y -_ lI I - r x x A x x L' a s s o c i a t i o n a n d < d i s s o cai t i o n a r e d e t e r -
of
The cocfficients
,
coefficient
r e s i c l u a1
of
the coefficients
fn
-l 't-1 = (r rt !r l ^'r 6l y l : x ) - ' ) 1 " ' e r e * l , i r ( ], \; 1 ,1 . , ""^.r.,
the
r,'i th
1
-
paralneters
sanre fornulae
enrre
,
separation
of
= trace ,
lxplicitly
nr,--lar
nj-ned by )
a*,
^
4
1
^
The canonlcal correlation
2
roots,
and their
multiplicities
together rvith their
multi-
of the equati-on , 2 lu t**A**-
. tillcrc \
^
coefficients
are determined fron the posltive p1icities,
?
Rxy/pq,tir=t-c"y
2= r -
| 1l=
0
7 L- , or tlre sane equation tritlr x, y interclranged.
- 6 5 -
Q u a d r a t i c d e c o n p o si t i o n
1 1 A I
The follorving
of the algebraical
is a reformulation
n r n n n q it i n n n n t h i c h t h e t l t e o r e n o f C o c h r a n ( 1 9 3 4 ) o n t l - r e of quadratic forn's depends, in which ortlrogonal
distribution projectors
and rvhich shows a significance
again have a roIe,
the forrn in whlch a surn-of-squares decornposition has been
for
derived fron a regression. If
are positive definite
a,b,
of rank p, q,
s y r n m e t r i cr a t r i c e s
and of order l'J, such that
a + b +
1,
then p + q + . . . = N if
and only if
there exists a deconlosition of the rrnit
matrix into mutually orthogonal projectors
'.e'
f,
that is a
and an orthogonal
+
f
+
=
1I
n F t
n v
,
,
t r a n s f o r n n a t i o n L I, s u c l - r t h a t a = l l' e U ,
L2
=
G e n e r a! i z a t i o n i n
l l ' il h e r t
b = U'fU,
snace
The analysis i+hich has been given for pairs of subspaces of a fi-nite
dimensional Euclidean space generalizes wi-th only
the slightest
m o d i f i c a t i o n s o f r n e t h o dC i r e c t l y
dimensional unitary
space; and it
can be put in a spectral
forn which can be i.nterpreted in llilbert that generalization.
A different
question of the unitary
invariants
h a s b e e n m a d e b y I l i - x m i , e r( 1 9 4 8 ) .
to a finj"te
space' and suggests
approach to the related of a paj,r of subspaces
- 6 6 -
b e a p a i - r o f s u b s p a c e so f a H i l b e r t
Let €,T and let
e, f be the orthogonal projectors
:re
nrir
the
of
haveC, fi fot
i r 1c m n o t e n t
ltcrmitian
tp^r"{
,
on them; so er f
oncrators
in
whiCh
ranges.
their
Nou efe, fef are a pair of non-negativeIlermitian o p e r a t o r s w h i c h h a v e t h e s a m es p e c t r u m I , e x c l r r d i n g0 , w h i c h i s a c o r n n a c ts e t o n t h e r e a l a x i s b e t w e e n 0 a n d 1 . Theq following the formulatjon oI thc spectral theoremfor l l e r m i t i a n o p e r a t o r s g i v e n i n I I a l m o sf 1 9 5 1 ) , t h e r c e x i s t u n i q u e s p e c t r a l n r e a s u r e sE ( o ) , F ( o ) ( " € I
),
defined on
,4
s u b s p a c e< s . Yo f I , s u c h t h a t efe =
, fef = ('^ar(^)
[rdr(.r)
J
where,
in
addition
to
the
E ( o ) E ( p )= O , rnrried
hv
the
J
autonatic
F ( o J F ( p )= 0 cnnefrel
oennrrl
there are the further
orthogonali.ties (o ao
theOfem
fOf
= 0)
,
I l e f m i t .i .a. n. .
. - lO - .n
eTatOrS.
orthogonalities
E ( o ) F ( p )= o
(oAo
which arise frorn the peculiar
= 0) ,
form of construction of this
p a i r o f I I e r m ti i a n o p e r a t o r s o u t o f a p a i r o f l l e r m i t i a n i d e m p o t e n t s .\ l o r e o v e r , t h e s p e c t r a l m e a s u r c sE ( l ) , the
DOS it
ive
i r-a ts,rvr
-S/Dt /e! c! r r u ' r '
orthogonal projections
tl-,^
ef,
F(l) of
^ vr +LL r i u l { u t r d1r -p*l ^r r Ji s^L^ L+ L1 r S
On
the
f€of F and €.on € and T ,
The matter is easily
settled
is concerned;but it
calls
so far as the point
spectrurn
for further analysis in respect to
t h e a p p r o x i m a t ep a r t o f t h e s p e c t r u m , u s i n g t h e I a c t t h a t t h c o p e r a t o r s c o n s i d e r e d c a n b e a p p r o x i r n a t e db y o p e r a t o r s w i t h nnint
qnaafra
-
oa
-
A P P E N I ] ] XI O N T H E L A T E N TV E C T O R SA N D C H A R A C T E R I S T IV CA L U E SO F P R O D U C T S O F P A i R S O F S Y N { M E T R IIC DEMPOTENTS
Let e,f ". be a pair of symmetric idempotents with real elements. Then F o r a s y m n e t r i c i d e m p o t e n tm a t r i x e , s , = e , e 2 =
the protJucts ef, = ftet
= fe;
fe are transposes of each othcr: (ef)'
s o t h e y h a v e t h e s a r n ec h a r a c t e r i s t i c
and these values and nulti-
w i t h t h e s a m em u l t i p l i c i t i e s ; plicities
characterize a symmetrical relation
THE0REM I:
values,
The characteristic
between e and f.
v a l u e s o f e f a r e a 1 . l -r e a l .
at
least zero and at most unlty. values of ef = eeff are the sane as
The characteristlc those of feef = (ef)ref, definite; all
symmetric and non-negative
hence they are real and non-negative; and they are
zero if If
whi,chis
and only if
ef = 0.
e f I U , l e t t rh e a n y n o n - z e r o c h a r a c t e r i s t i c
necessarily rea1, and 1et tJ be a real belongs to it,
latent
value,
vector which
so that efU = Ur eU = eefUl-l
Hence
Sincer I O, -1 = eftjr = U.
tJrfU = U'efU = U'U). ,
which gives I = U'fll/U'U. But the stationary values of X'fX/X'X values of f,
are the characteristic the calculus;
as appears by methods of
and these values are all
X'fX is a continuous function
zero or unity.
j-n the closed region X'X = 1, the
maxinun and minimumvalues are anong these stationary and so it
follows
values,
that 0
q-Tffi-ef
Since
<1 . +)
(1) , another proof is as f ollows: erence to Af riat r = x= le'el ""itl , l"l *-< rlrlx = t. = so that l"l* 1 . I l e n c el r l < i e r l i"l
- 6 8 For any pair of vectors U,V,
. o r 21 u , \ , 1= ( u ' v ) z/ ( u ' u )( v ' v ). T t l E O R E lI4I :
U
If
V
of ef f or a non-zero
#
V^ = ftl^ ,then
value l= o,3 and if
characteristic fi)
a l-atent vector
is
A
is a latent
vector of fe for
the characteristic
value tr; (ii) (iiil (iv)
I
= c o s 2 { U ^ ,\ ' ^ ) ;
if
IT'tl = o. then ll' V
if
al B , then U' Uu = o.
d
B
0
6
= O. tll V
Thus,efU^ = Ulr , forl=c,B feV^ = fefU^ which proves (i).
f
0
b
= fUr).
= V^ ). ,
Now = tl^ I
,
eU^ = Ur since r I 0. Therefore
= ui efU, = ui fu, ; , i U , = U ' ^f l J u = U ' t f I f u u = V i V u, ll;
a n dp , ince it
= 0 V' V^ = 0;
I O. Hence
eU^l = eefU^ = efU^ so that
d
uuu
U ' ^ U r u = U ' t r \ r u= V ' ^ V r ,
follows that
for l, u = qB. Now (iii)
follows inmediately. By interchanging
t ra n d u , w e h a v e U j U U o =U ' ' U U B , s o t h a t U J U g = 0 i f a I B , which proves (iv).
Fina11y,
. o r 2{ u ^ , v ^ )= ( u ' r \ r r2)/ ( u , x u r()v ' r v r ) = l , w h i . c hp r o v e s ( i i ) . T H E O R E IMI I :
The nullity
of ef-r1
is equal to the multiplicity
of las a characteristic value of ef. Since the caracteristic dependsonly on the cyclical characteristic
equation of a product of matrices order or the factors,
v a l r - r e st r o f e f = e ( e f f )
the
are the same as the
- 6 9 (eff)e = (fe)'fe,
characteristic values of
m^. A1so, provi.ded that I I 0, in which case
multiplicities eU = U, it
with the sane
efU = UI is
appears that
equivalent
to efeU = Utrr
so that the nul1 spaces of ef-11 and efe-l.l are identical.
l l e n c e t h e t h e o r e m i s p r o v e d w h e nI I 0 . ef and efe = ef(ef)'
observe thal X'ef I
is mr.
of the latter
B u t , b y t h e s y m m e t r yo f e f e , t h e n u l l i t y
remains now to
It
have the samenu11ity,
since
0 is equivalent, X'ef(ef)'X= O, and therefore to
X'ef(ef) ' = (,
and the nullity
of the latter
the ranges of e, f
T H E O R E IMV : I f E , T a r e
is mo.
then
€ e".}- = E aT =?ore' If
U = eV, then eU = U. Hence lJ = eV and feU = 0r is
equivalent
to U = eV and fU = fr
This gives te
e"ts = EoF.
A 1 s o , i f e U = U , t h e n l - U = C ri s e q u i v a l e n t t o f e U = d ; and €/= efU = (fe)'feU
irnplies feU = e( Hence tl = eV and
efU = O-is equivalent to U = eV and ftJ = 6, and this
gives
EeT= EarZ. T H E O R EVM: I f are all
F ,7"t"
th" t"ng"r oe e,f
the non-zero characteristic
repetitions
according to their
values of ef,
m u It i p 1 i c i t i e s - _
e x i s t u n i t v e c t o r s I 1 1 , . . . , E r a n dF 1 , . . . , -
- p
and f L
respectively,
a n d t r, , , . . . ,
I
r
with
!LI€_!_l-b-9-IS.
F. which spane7
and are such that
EiEj = 0, EiFj - o,
riFj = 0 (i I j);
ElFi =01,
2 where p i The spaces e}ancl
{€
eI which, by'lheoremTII,
are of dj.mensionequal to rank of is equa] to the total multiplicity
r of the nol-zero characteristic values of ef.
It
follows
- 7 0 -
and III
Theorems II(iv)
fron
ru rr nr irt L vv c r - t n r Js ' lFr ..
F - I
. . ,
sttch that
efE.
' - - . .
I
orthogonal
exist
= E , t r, . E a c h o f I
. l
range has
I O; and their
each ). i
in e'Isince
these lies
there
that
t h e s a m ed i m e n s i o n , a n d h e n c e j s
identical with e'f . Let
_16
vector obtained by
Fl = fEi;
Ft be the unit
and let
follows from TheoremIi (iii ) that
normalizing FT. Then it
, F, al'e orthogonal, and their
Fl,
the samedinension, and hence is
range, in f ?- , has
and (iii).
c o n c l u s i o n i s n o w o b t a i n e d f r o m T h e o r e mi I , ( i i ) TIIEOREV I 4I : T h e r e e x i s t
rnatrices E, F such that
e = E E ' ,F = F F ' , E ' E= 1 , F ' F = 1 , E ' F =
Let Ej,
Theorem\r ancl take
J
-
dlLu
,
hrr
r r . .
tlurr
r
ortbogonal
T h e o r e mT \ t , i i F j .,
uni.t
vectors
Lr+
. . , FO . . ,L'and 1+1,. I ,.
i
= 1,...,
, P and
P and j
= r+1,...r
by Theorem \I,
1
Fina11y,EE', FF' are range as e, f;
hence
These results
F ' F =
1 .
f ' F
qvmmetri c tl.nrr
cro
(r'.
. id e m n n t e n t S
irlanti
h a r . e ^d P* P- L1L :L ^d L^ a+U ir r^ , -
=
f^i
^ - r
d r l u
I i nrr
f )l
\ _ /'
0 9r
:) with
-€rl Wlth
h a d
s- , u, ^g^ ^g- e+ is^ r- .ro n , :r -n +r Ln e c a n o n i c a l c o r r e l a t i o n llnfnl
:)
= Crfor i = r + l,
q and again for
F r F =
+)
',?.
'Iei and € . . L e t E = ( E r . . . o o ) ,F = ( F 1 . . . F q ) .
s p a n n in g E e " 5 Tr 'h' a! rn r
(rt
F, be as determined in
E, and F1,
,..,
The final
with f€-.
identical
e,t
t h a i r
the -
sane
+)
n r i c i n e l
theory of
Qi
-71 -
r\PP[NDIX I I
1.
Introduct ion 'lirc
Jrc[lrcssion b1' least
lincar
orthogonal
of
projcction
projection. forn,al for
of
theory
'lhis
Its
tcrns
r v l r i c h s L r p p li c s
anrl {'corrctrjcal
made ir-r a scparatc
of
lysis,
1,n
1).sis,
of
liotell jng thcor,r'.!
tlleor-y is
to i:e
re!,ression as 3n ortllogonal
t li f f e r c n t
( . L . t f i , T r l c t, h c
f o r - r n c tl o
topi cs
in
canonicnl
c i , l rl c l b c t r a n s l a t e c l
in
c c t i - ' r ^ : i ,r l n i l a l s c t t i r a t , fl'rr,1
in
ana-
nLrltiVariate in
rvhich
rnultiVal-iate
correlation into
rn'hich
I\'ere cli-stingtrishecl'
factols topics
in
an
\\'ere distingui"shed.
analysis
a statistical
leacl to
analysis
factors
intcrcorrclaterl
cxpositiLrn of
. 1l ,
r r l tl t i l ' a r i a t e
in
a statistical
a y r p e a l . c i lt t r ; r t d i f 1 - er e n t for
altriebrai-ca1
a trnified
tlte statistical
SC\rera1 llrorrp-
interest
deVelop algebraical
to
concelts
spacc l{as fi1.st
on a liliear
appeared that
i t
n'ain soltrce of
necessarl' for
a nultiple
:irivantageous exposition
'ilicn
ancl a
paler '
Tirc expression of
grotrps of
is
obiect
forn'r.rlatjon of
anal1,sis. An exf csit jon
SeVeral
as an obliqtre
thc
lives
ofcratiorl
on a
ancl a partial
observations;
connexion bettvccn a statistical
inVestillation.
projcction
observations
of
the
c q u i ' , - a 1 c ] n tt o
s(lrlares is
can be sin'ilartl'represcntcd
algebrlticat
this
nrultiple
vectors
space spanned by vectors regrcssion
of
oferation
statisticai
the
theory
alla-
of
algebraical
t1:is aLr-el-rraical
-
fornulation, certain
it
intuitive
-
l t
was possible to introduce concepts fitting ideas rvhich, though needeJ in the subject,
were without any clear definition, a definition
for a coefficient
Thus there is c,btained
of correlation
factors eacli of which has a nultiplicity rvhich reduces to the correlation j,n the case of sin;pIe factors. concepts is
betrveen
of components,
coefficient
of Pearson
The genesis of the other
to be found in consicleration of psychological
a n d o t l ' r e r s t r c h k i n d s o f r n e a s u r e n e n t sa n c l t e s t s . idea that
a test
is
to p,ive a character
With the
to tl.re i.ndividual,
there i.s the idea that the character stancls outside the test. or that several tests rr'ith distinct
definition
may be di.rected
t o t l r e s a r n ef c a t u r e o f c h a r a c t e r , f o r c x a m p l e , i n t e l l i g e n c e , and, by this
intcntion,
thelefore have a relation of equivalence.
T h e r e s l r o u l t lh e , a c c o r d i n t l y , a t h c o r y o f t l r e e r l u i v a l e n c e of tests,
and, as a further
inclusion of tests,
example, a theory of the
by wlricll one test
can be decided as a
p r o p e r r c f i ; r c l c l ' t o f a n o t h e 4 o r a s c o n p r e l i e n d i n gc s s e n t i a l t y more f eaturcs of clraracter tlran anotlrer. Again, there is to L r e c l e c i c l e dr r ' h a t f e a t u r e s
tes ts rnayhave in conmon, if
C c n e r a 1 l 1 ' , a n u r n l , e ro f t c s t s forrninp a configrrration, ncasures for different
tog'ethcr may he regartled as
f r o r r n ' h i c hi s r l e r i v e d t l i f f e r e n t
relations
one another. In translation are various coefficients
any.
the tests may have to
to the algebraical
schene, there
defined rvhich belonc to the
confifui'ation r'lricl|r'ector subspacesof a given space fornr togcther; anrl arr account of such cocfficients this paper, rvitlr notive toriards their
is I'iven in
statistical
appJication.
- 7 3 -
)
qn.aec
\iaef^r
be the space of vectors of order n
r , e tI
f o r n e d o f t h e r e a l n u m b e r s .T l , e n , r v i t h t L e s t a n d a r d l a w s o l - cL Uoi rl .nI /nv oJ rsL ir tv ir o r n v r
n over to
the
^ r , al a L- , L
and
tlte
snan
X =
Kx.
havino
K of
ri ,r ' l , , .
of
in,lnnenrlent
, ^ l,J.
o
,
these
tltCSe VeCtOrS the
k
vcctors
voctor s of
rs
tlre
k,
which
snace
all
i.'is
lts
srrhsnace i{
k
dimgnSiOn
taken
nay be cal1ec1
oenerators.
"1,.
of
in
as
vectors tho
iS
snace
J y u L r
k vectors
of
n x
order
with
I ineaT
d
numbers. Any set
real
compose a rnatrix
@ vsL LUr
nf
rfnrc f 6 , r vV o g L L U T ) r j u
r U l
e l ements ; of.
Of
l'he naximal ltave
tlre
veCtOrS sets
sane
n u n b e r o f n e m b e r s , t . r h i c hd c f i n e s t h e d i m e n s i o n o f t h c s n a c e ; it
i s e n r r a l t o t h e r n a x i n t r mn u r n b e l n f
i r r r cl n e n , l c n t v c c t o l s
a m o n gt l r e g e n e r a t o r s , a n d t o t h e r a n k o f t h e l e n e r a t i n g matrix K. l'tlEORE1 I r.l1 : T h e r a n k o f K i s e q u a l t o t l i e r a n k o f K r K . lor
if
Kx = d
xtli'Kx = d,
, then K'Kx =di
r v l ' r i c l 'irm p l i e s k x = f ;
t h e s a n e c o l t r n : nn r r l l i t y ,
and if
K'Kx =d,
then
K, K'K itave
therefore
and, since they lrave tlte sarne
numbcrof columns, the sane rank. C O R O L L A R YT h e v e c t o r s o f K a r e i n d e l e n d e n t i f t
i
lI K ' Kr l I Let I
and only if
O. h e a s u l . s p a c e o f J - o f d i n : e n si o n p ,
of a maxinal set of independent vectors of h e r r i n o n m e r n h e r s -T h e n I vectors of I
defines a base for €
any sucll set , such that the
are thc vectors uniquely givcn in the forn X = Ex;
+ L , , -
L r r u 5.! . .,
c . . -- L / r: * ,* ,f , y i r- i e s x ' = L
(E'E)-'
exists.
vectors X of
e,
and E tlre arra)'
I
a- -ni d Il I t E l l
0, so that the matrix
T h e r e i s t h u s a c o o r d i n a t i o n b e t r v e e nt h e
the space €
o f c l i n r e n s i o np ,
ancl the vectors x
of ordcr p, the coorCination being defined relatlve base l.
The following results are immediate.
to the
- 7 + -
condition for a
T I I I I O R E1M. 2 : A n e c e s s a r ) ' a n d s t r ff i c i e n t
base I
X to belong to a space $ with
lg:j!!l
IrX = X,
that
a n d t h e n X = E x , t t h e r ex = ( L ' l ) - 1 1 . ' X ;
r r ' h e r eI , , = l t t . ' t ) - 1 | ' ,
-
I -
o r t l ) o t ' o n acl o n p l c r n c not f I
p is
is, J,\ = S, tvltcrc
=d,llrat
I,\
is
TI , . l l o r e o r . c r , i f
belongs to the
concijtion tlrat \
3nd a necessary anclsnfficient
. Il ,. . = I -
is
€,, ltlsn p =
t h e r li n e n s i o n o f
rank T, = tl'ace lt l An orthonorn'al orthogonal
t
base 1r for
conposeil of
is
so [i'l] =
r-rnit vcctors;
lr is
1 if
a sct
p
of
an orthonorn'al
b a S Cr a n c l I 1 ; = l l l J' .
l,
Lct
E ,7
f,'l
l r e s n a c c s , a n c ,l
sait'l to
be sclaratc
rihich is the is
rihich
the
is
join
the
if
i.'. t5c join
oi
conplete
spa."
the 1.ascs is
ortirollona l
, if
J
;
of
i;rclirrc,'. If in
1l Il
equivalertlv,
in
in
tlris
i,
thcn
tlicrc
the trases, '>' d c r . r t - r t c lt 'l 1 ' e @ J i r r Cj " . cl n t . ' l ' h e y
be
ancl tlrcir
sqLrarenatrix.
to
ortltogonal
1 - o I r ,l i i c h
exists
a n,atri-x a
aitclthe
is
e\:c'ry
I, I,
t l - , c ) 't r r c s a j t l
of
orclel qx I
sufficicnt
trnion
l'lrespaces are
r) r ' ) r i c l rc : r s c c i s s a i . l t o l c
anrl conve'rse1,v; ancl a necessarl'and ts
tite nu11 space,
case p + Q = n,
onc is
[rre
(frf) of
to
saitl
= Cf ; othcrt',isc
*| t r - 7J - , i l
is
tirc sprccs,
V e c t c ) r - i r r t l r c ' o t l t c - r ' , t l t c c o n r lj t i o r or
tlte bascs,
cP
tlrcl'are'scparatc
a r c g r rl a r
e \ ' 6 r r ' ) '\ ' e c t o r
t nt
the jojrr
a l r a s c ; o t h e r ' \ i i s c ' t l r c ' 1 'a r c
;lrc callccl conIlerrentnrl
rt
intersection
anclonly'i.f
anclrvith bascs
c li n e n s i o n p , q
corrcs]-ronCingll',also
if
the case,if
sy.;t1o r f also
tr" an,v spaces in"C of
= 6, to
he
ilrclrr.lc,l such that
conclition
for
I
this
IpI1, = I1., r''lrich in|lics l. = |o
rr,itird =
- 1 1 ,' 1 . . (l 'l ; '],
. \ ec o r c l i . n g J y , t h c
s1."1qg5 ale
i
C-
jtrst
iI
= Fo,
It_ = IIr;
_
-
I J
also they ale ortltogonal comllcntcntsjust
i f
i f tlre ortlrorional
l h c y e r e s a i t l t o I ' e o r t h o l o n a ll y i n c i J e n t
in eaclr of thenrarc
intersection
c o n p l e n r e n t so f t h e i r
= i.
Ill n If
i s j r r s t I I .I l i = 1 l I t . . T ) r c s p a c c € ' i s /\, s a i d t o b c c o n p .cl t c l ) ' i n c l i n e c ' l t o t h c s f i t c c J i f n o s r t b s p a c c o r t l r o l 1 o n a 1 ;a n d t h i s
is
of
totally
to 7
ortiroponal
, € o';
tlrcl'arc
inclinccl if
=D
. l r L c sl f a c e s a r e
;
conplctcly
to
saitl
irtcLinc to each
otlier. 'll.:o is
rre sairl to form a c:rnorrical pair- i f
lrascs Il,\'
oltlroncrrnal, lnil cr.cly vcctor
nost
one vector
tlr is
casc
o f t l r c ' o t h e r a n c lo l t h o g o n a l
rr'lret'cc is unil 1.
l l l L
a r c g r rl l i r
_-
1 t
,
1 ' 1 1 ' \ \
diaponal
r,'il1 e|pcal
It
= 1, Ii'\'
thc
t
at
res t ;
in
l o f'
1
r r itlr cl cl'cilts
lratrix,
t l r a t for- cvcr'1' p a i l
strclt a crtltotrical p a i r
tlrcle- is ists
3.
to
to
lo o\
t t
-1
one -is inclitretl
of
caclr
l r il S
of
bcltr't'crt
spaccs
of
C S.
J)cterr,inilnts 'l
irc 1ol1or,'i1r! lcslrlts
l - l . i l i . l ,J\ . 1
.\,
lf
I
, t r c ' . 1t , ) ' 1 ,r t r i c c ' s
I - .\lr Proof.
'iallc
o
l/
( . o l l o l , l . , \ l t Yi . 1 1 . C O I I . O l , l - , \ l lrY. i l . clraractcristic zcro
lias tl)c
riill
bc
r e c l r r i r e - c. l
o 1 - o r - t el ' l l l l / l l , l l , ' r r '
tlrcn
I - ll.,\
=
c l c t c r n r i n : r n t si n t l i c i d e n t i t l '
lt-, \ 1 , . t )
I \
on rlctcrninants
y'n 'lhc
u\=/t it ll
) I -Ali riatliccs
t'alrrcs, r,ith
3\
T I
\ /\
\ I
l
rl =
\tr
I
)ni
L l-ll,\
\0
I -liA
lli,lJA lrar.'ctlrc sanc non-zelo t _ l r cs a n ' c n r t r l t i . l L i c i t i e s ;
: , a r r r et t r t r l t j l , t j f j , l ) r : q l r
clrlrracte ristic
!n.f vaLug
l.c
- 7 6 n = n.
and onlv if
of each if
A is
L E M M A3 . 2 I f
I
'l
l
rr |
l^
square matrix,
a regular
then
- l
= lal
In-un-'vl
I U B I Proof. Take deterninants in the identity
o o \/ o u \i ' - n - l v ) , / n \ \-uo-',/(u r/to , /=(o s-un-rv/' (r
ArB are regular
C O R O L L A R YI .f
T H E O R E 3M. 3 :
A u
I
U
I
B
then
l = l o l l nI l r - u n - 1 v s -| 1
G is
If
square matrices,
the join
of bases E, F, then
l c ' c l /l a ' r lI n ' r| = l E ' . r . r lllE ' E =l l r ' . r u r l / l p ' r l = l I - r F r F l For [ E 'E
E' FI
r ul r l , l G ' c l =l ; , ; ; , ; l = l e ' E l l l r _ r r E ( E ' E ) - 1 E ' F l = l rE' '. E by Lenna 3.2.
Theprby CorolIary
3,2I
a n d b y L e r n r n a3 . 1 ,
I G ' c =l l r l l I F ' F ll r - F ' E ( E ' r ) - 1 8 ' r ( r ' r ) - 1 1 = l E ' El lF ' F l l r - r E r F I C o R O L L A R YT. h e t h r e e c o n d i t i o n s lI for
-
IEF I I
0 are equivalent,
I E'JFEI I o,
I F'JFFI I
o,
and are necessary and sufficient
the bases E, F to be separate. For G = (EF) j.s a base if
and only if
E, F are separate
bases. T T I E O R E3 M. 4 :
-t_::t
- . - l
t:i_l=
t-p J -
E I E
B y l e n m a 3 . 2 , C or o l l a r y - ^ F I F
F'E
trt
3.2I,
" - , - Il = - pI'
rl
^ E I E I r
( - o ) P * el L ' E ll F ' r - l b z r - r E r F l
and by Lenma 3.1,
2l l - p E ' El - F ' F l l r - E E E F / e
- 77 -
l l .i e e q u a t i o n s c o R O L t . A t'Y [ - l . l . L h a v e t l r e s a r . ep o s i t i v e If g,'y
4. lrojection. bases, l,
I
join
"l],t
= o - of: f "t il = . e - - f , g i o ' r - tL r tI, -| -
r o o t s , w i t h t h e s a n r cm t r l t i p l i c i t i e s . A r e c o m p l e r n c n t r a sy p a c e s , t h c n t h e i r
in a regular square rnatrix G = (lf),
and any
v e c t o r Y l r a s a u n i q u c r c s o lu t i o n
z = c , z= t r r -' l\/ v I )/ = t * * F y = x + Y , . ^-1w n e r e z = r , L , i n t o a s u r no f v e c t o r s X , Y i n fronr Z by linear
transforntations deternined just
s p a c c s . f h e c o r n p o n c n t s\ , projections
g,7
transformations which obtaj-n
of Z, and tl.relinear
I J
to E- respectiveLy.
para11e1 to'i
S r , r c ha p a i r
ba these
rcsolut j-ongive tlre
Y in this
them are the projectors,, on t
obtained
and on y' parallel on and parallel
of prcjectors,
to a pair of conplementary spaces, are conplementaryprojectors;
Their two products are nu1l,
iJcntitl',
fron r'hich it
and their
to its
nu11-space, and if
c o r n p l e m e n t s ,i n w h i c h c a s e i t it
the
follorrs that thcy are itlenpotcnts.
A n y i r i e n r p o t c r r tr c p r c s e l ) t s t h c p r o j e c t o r paral1el
sum is
on its
range
these are orthog.onal
nust be synrnetric, then
is ca1led an orthogonal projector;0thcrrvise
it
is
c a 11 e d o b 1 i q u e . --
p
r\Y
If E , f
are scfiarate,so that the join (; = {l:|) of thcir
bases L, F is a basc for their union p, [or the orthogonal conplencnt R if
let
R be a base
f, , vltjch is nu]l
of
they are colnplerrentary; then (GR) = (LI:R) is
square matrix,
just
a regular
and there 1s thc unique resolutj-on Z = C u + R v = l x + l y + P . v = l l + V
of any vector Z into
=
t h e s u r no f i t s
a n c lY , o , r € -p a r a 1 1 e l t o 7 A n
+ \ '
X + Y
oblique projections
and on !
paratlel
to fdd,
X the
- 7 8 -
its
the matrix J, = I-Ia
is
symmetric idernpotent, such that JrR = R, since IIR = 01
J f F = C / ' sj . n c e I U F = F . a n d s u c h t h a t
follows that
are separate, it
are olthogonal. Also since €,T iE'.lrEll
and of
Now,with If. = F(F'F)-rF',
orthogonal projectiort V on A
b e c a u s e7 , - R
U on 7,
the orthogonal projection
s u r no f w h i c h i s
o, Uy Theoren 2.3,
Corrollary,
so that
- 1
t h e i n v e r s et ' ' ' : i ]
=":rr:': -::'"::lr'r"= ,,JrLx (Il'JFE)
and then
-1E'Joz,
x = E(l-'JFL)-1L'J'i
_ 1' 1 , ' J 1 , . X = I I , . i F Z ,w h e r e I E , F = L ( 1 . ' J f l : )
that is,
conclusions (which are of
Ilence there are the following
importance for the theory of partial T f l E O R l l4, .t 1 : I f I g,T
regression) :
I:are bases for separate spaces8-,7,
t,
tlten
= IE,p.,where t
is the oblique projector on & e-,T to the union of !r with the ortlrogonal complement
parallel
C O R O L L A R4 Y .11. If = IE,p, wheret
a n c lI u , F = E ( l ' J f E ) - 1 1 ' J F -
(-,F
of the union of
a r e c o m p l e n e n t a r ys p a c e s , t l t e n I o . / E"1'
€,! 'i<
nnw
thc
nhlinrre
nrnieetor
on
9
L,F
para11e1 to ! Il i.s a base for
C O R O L L A R4Y, 1 2 , I f
8.,
then Ia = I' = L ( [ ' l . ) - ' lL ' .
i s t h e o r t h o g o n a lp r o j e c t o r o n € a n d i l
Il, F are separate bases with
C 0 R O L L A R4Y. 1 3 . I f then there is for N
the iclentity
Ia = IO,r * Ir,O,
join
C,
and correspondingly
spaces. O
W
T
so, since I. Io * c-
c\"f
where I"
for
E , it
cr J'
' e , 7T
-+ V , , T
+ -=
tT ErT
'
appears that
I' t It A
-=
' =R * "'
r',
T
^ - ' t J^ ir -r r: rr r^d-r t .r /'
drru
witlr
- 7 9 -
lt
and then, into
' ' F- t
g , i
I,r t
e '
'
g , u
-
t ' t p * u t ^ 1
t i , -
,
I - c f rorn t h c s e c o n r l r c I a t i o n 4 ' c -
of
by substitution
llre f i rst.
Yt
j , {
tlrat
( r - r € I , i )t 8 , 3 = I E ( 1 - I C I ' I ). l l y T h c o r e r 3n . 3 , C o r o l l a r y , a n c lh y T h c o r e n4 . 1 , 0 o r o 1 1 a r y' 1. 1 2 , t
-
l
I o,i since E, a'are separatc; thus tltc foliotr'lng
we har.e I 1-IgI'
t h e o r e m i s o bt a i n e c l : T I l f , O L E l- il . l :
(,i
If
are scparate spaces, then the obliqtte to the lrnion of I
€ -p a r a l l e l
p r o j e c t o r I , . , _o l F rJ,
is ',leternined
ortliogonal conplcnent of the union of E,? tlic ortltouonrl nroiectors
fror
t C O R O I , L A R \ "I f [ ,
(1:l:)= C is
a base,
If
tvhere
(l.ll) = )., L = JfIi(]j 'Jfli)
( C R O L L \. \ I4 . 1 1 . C \ ' = f l !
I1= F(|'F)
(C'C)-l
-
'rN,\'G
'JFIr ancl I'l = JfF(l' )
IL'+
I ' l '. l , t . h e r e I . = ( ' \ ' , I r , . =
-lJ l,
Iiare
ortitogonal,
I-, Ir are
fI
s ( ; u a i ' ( 'n i i t r . i - r r ' i t h
the
conDlementar)' olrlique
the
spans o{ S p a c c s€ i ( i they
tlsn - t'
= L, - t' .
LL',
= l.(E'E)-1, l l - t g , - Lt
- t
4.r3. COROLLAIiY
if
{t-II.Il,).
t
C O R 0 L I , A I I4\ ', 3 7 .
repular
tlrc forntt,la
theorern calL bt: r'er:if iecl directly:
folloriing
-
lI,
(I-II It)
T l i t o l l . l l l1 . 3 :
T
Iy
bases t]'ren
sepaiate
I;(ti'JfIi)-ll,'.1 = , the
on (,'I
I-
(r-rerl)lrait-rLr+).
L,T= I' are
Ig,
the
l ith
l ;,
conplernentary' jnVcrsQ \',
projectors
so that
tlren LL', on and
C ls
a
Fll' are
arallel
to
F. = l,
.
.
are n,utrral1y
'
.
t
n t ) ' l)
nr e
qonATate
qai d
rn,l
form a conplentcntary sct
l 9
union i s
thnil
spacc:
IN a
o
9 ; J
'
= (v
{ l
f
) 1 , -
=d F - t .
@ J -
i
the conpicte
- 8 0 -
T n tL hr ri rsJ tl a- Jar sr e . fi ff
Il: is
= I
F * i .\ r, i
a base for
r
t n) l ,
then
" . . . t " |
L = (' 1t . . . . 1 * ) i s a r e g u l a r s q u a r e m a t r i x . m '
fn r.
e
e n m nl nnl c nl ?r v
c af
en, l
- 1
Lct e
f
e,
e
i f
t
=
( l :1...L- ,) ,
I
-?.1
{I1...F- ) ,
j
jli
the
noi' denote
projectors
orthogonal
0
i n & o f d i n : e n s i o np , q a n d w i t h b a s e s I ,
e , f
=
F
p p r o j e c t o - IE - oi l r
-*brique ._" ; ".
tr,:,
= 1,...,n:)
w i t h b a s e sL i ( i
C O R O L L A R \ ' 4 . 3I 4f . s p a c e s € i
on spaces
F; so
e = !(L'r-)-1E', f = f 1f'F;-1p' , - e, I = f
and e = I
- { are the orthogonal project.ors on the g
s -= d .e o r t h o g o n a l c o r n p l e n r e n tS
, i
= IOT
of
= ranl' f = trace rr = ranL e = trace e, q
f.
such that R t
r,
r = ranli ef,
R = trace ef,
it
ancl
The nunbers will
of inclination
d i n e n s i o n a n d t l ' r ec o e f f i c i e n t
t ,V;
define the
b e t r v e e n€ , 7 ;
a p p e a r s i n S ( ' t h a t t l r c l ' a r e t h c s a m ei 1 -a n d o n l y i f
arc orthogonaly incirlcnt, antl zcro if
an,lonly if
€ , i
o r t h o g o l r a l ,a l s o t h a t d i s c o r p l e t e l y j n c l j n c t l t o ?
iust
r = p, anri included in f inclination
if
;ust
€,Y arc if
'1 he coefficient
I1 = p.
bctricena pair of lectors \,
of
Y o r b e t r r e e nt h e
r a y s s p a n n e d t - r yt h o s e r r e c t o r s i s L = t r a c e X ( X ' X ) - 1 x ' y i y ' y ) - l y , = ( X ' y ) 2 / ( X ' X )( y ' y ) = . o r 2 o , +r,^+ L r l ( r L
;^
r ) l
i+
l L
:^
l >
*Lc
L l r L
s e p a r a t e s t h e n r. .\" The space e.i,
qnrr:re
J r l u o r L
samn rlinronsinr
tsccn € ,?.
t( hr l e\
^^;-^
\eu J I l J ( '
^f
u l
+( l rl .L ^
^-^le
d r l t ,
O
W l fi C l f
fornredof the orthogonal projections
in & of every vector I o f t h e s p a c e]
vn rf
ofi,
gives the orthogonal projection
i n t l r e s p a c eE . civnn
lrv thn
Tl'c inclination
eY
The spacese ],
dincn5iOn
of
f t are of the
inClinatiOn
f
be-
h ' e t w e e na p a i r o f s p a c e s a p p c a r s
c q , u a lt o t h e i n c l i n a t i o n b e t u e e n t h e i r o r t h o g o n a l p r o j e c t i o n s
-81 -
on each other = R(e7, f c).
R( €.,F)
a r e c a l J e c lr e c i p r o c a l s i n € , F i f
A pair of spacerL,V ern
tlre nrthncnn:l
nrnioctions
1*= cV,
of
. in
eacli otlrer
tney
€,?;
V= r ?.r-; called rL'ciprocals if
and a pair of vcctors X, \'are
they
rays, in whiclr case
span reciprocal
= eY, Yu
xl
= fx,
lr.here y'\tr = X'eY =X'Y, YtYu = Y'fX = Y'X, s i n c e e X = X , f Y = Y , b e c a u s eN , Y b e l o n g t o € , , 7 , I! r
since
I O; and tlie nunbers ^ = ( X ' X ) - l \ ' y ,u = ( y ' y ; - 1 1 ' 1
are strchthat
:
lu 1
p'--
vihere is
a g aj - n t h e
or
the
)
positive
then
Y is
2, value p
o
''
j9.J
is
latent
nou any latent value p2,
vector
characterj'stic
ef\
of
ef
for
a
= Xp2, ancl if
eY = efX = Xo2, ancl feY = f).p2 = )'p2,
a latent
vector
reciprocal p
of
fe wi-th ciraracteristic \rectors
a
vallte p'
in
e ,T
with
Z:
Any reciprocal vectors
of
ef,
vectors fe both
X, Y in
then Y = fX
0,!
with
inclination value
t''ith clraracterj-stic
for a p::tt-tl:
X is any 11.""._5..:l_1f_ef
characteristic
= Yirl = Yp?,
fe with
so that
vectors,
f,llat
feY = fXr
ef,
of
the
l;etreen
follows
= xo2,
vectors
coefficient
'lHtORtl.1 4.4: ^r"
\
ancl X,Y are
inclj.nation
Now it
= eY, = Xlu
characteristic
notr Y = fX,
'
if
0'4. A1so,
1
= cos"o
incli-nation
span.
X, Y are latent
so that
so that
of
coef f icient
ef\
value
a
(X'Y)'/(XrI)(Y'Y)
rr'hich they
rays
p2,
is
a latent
f o r t h e s a m ec l t a r a c t e r i s t i . c v a 1 u e , a n c l\ , Y
vector
of
are reciprocal
fe
- 8 2 -
vectoY'sin
f
-il
with
incl ination
The next theoren estabishes a procedure by which the of spaces can be calculated when the
of a pair
intersection
bases are given. T H E O R E 4I 4. 5 : T h e i n t e r s e c t i o n
It
is
the nu1l-space
is
1 - df.
of the natrix Let J,
of the space (,F
fl
denote the intersection
Let X be any vector
that Ja7l.
evident
and nu11-space considered. ofT?, so that
efX = X, and let
Y = fX, so that X = eY. Then X,Y belong to
the ranges E ,7
of e,f;
and
X'X = XreY= X'Y = XrfY = YrY. Therefore, vector
by Schwartz's inequality,
X = Y, so that
X of 77belongs to both € and T,
that
any
is , to J,
and
hence ??c7.
A c c o r d i n g I y , J= n . C 0 R O L L A R YT. h e s p a c e E O F i t
the nul1-space of the matrix
1 - ef.
& o " T = 8 o T - -t e 1 8 .
T H E O R E 4M. 6 :
the orthogonal conplements in Eof u4
That is, the sane. If
U = €V, then eU = U, and hence U = eV
and feU = (
i,
equivalent
Lo"T=
eoV
A1so, if
to feU = C/; rotuover,
TIIE0REM 4.7.
tey
to U = eV and fU =g; eU = u,
is
feU = U .
to U = eV and il
equivalent
=6i
f& are reciprocals
in Q,/
and
inclined.
For the rnatrices ef, Now every vector
j-nplies
= Yet & .
The spaces e7,
they are totally
therefore
then fU = o is equivatent
|l- = eftt = (fe)'fett
Thus U = eV and efU = d
therefore
a n d .f €
efe = ef(ef)rhave
in one of
the pair
is
the sane range.
inclined
to some
are
- 8 3 -
vector of the other,
so they are totally
inclined.
Let the signs o, o and x between a pair of spaces i,ndicate inclined and orthogonal.
totally
that they are inclined,
T h e o r e n r s4 . 6 a n d 4 . 7 s l . r o wt h e f o l l o w i n g
s c h e m eo f d e c o m p o s i t i o n '
shows the procedure by which it
a n d T h e o r e m2 . 5 , C o r o l l a r y , can be effected.
T I I E 0 R E I4' .18 . A n y p a i r o f i n c l i n e d
spaces 8 ,7
have deconposition
in the scheme:
L l
d
t
|
= e ? -.. o l
.
a
eg "
1,.*
. < .t r
= re o
l
J
(
' ? Idf,
t-.._.'^
----'
5. Rank, n:ultipl r::lZ_:1.1_:fthogonali 'l'lie
Y
theory of the latent
ty
vectors and character j-stic
values of prodtrctsof orthogonal projectors is reqtrired for thc subsequent theory of the canonical deconpc ition to each other, and the con-
of a pair
of spacesrelative
struction
of bases in each which forn a canonical pair.
Since, e, f are synnetric natrices,
their products ef,
fe
are transposes of each otlter, so tltey ltave exactly the sante characteristic
v a l u e s , w i t h t h e s a n i em u l t i p l i c j . t j , e s ; a n d charactertze a syntmetric
these values and ntultiplicities relation betweenthe spaces Y,T flre
ennfiorrrrtinn
the
qnacps
or are characteristic of
nresent
felative
to
each
other. T I l E 0 R l l \ 15 . 1 . T l , e c h a r a c t e r i s t i c v a l u e s o f e f a r e a l l a t l e a st For
zero
and a t
the proof,
nost
un1ty.
see Afriat
(3)
rea1,
- 8 4 -
A
Then any r
value of ef.
characteristic
for which m^
may be ca11ed a characteristj-c value for wi,th this
Since 0< r <
nultiplicity.
as a
of any value I
Let m, denote the multiplicity
l o .--
p
the spaces L , f
1 if
0, by
m 1I
Theoren 4.1, positive acute angles o are determined by ') T h e v a l u e s I ( q I 0 ) a r e t o d e f i n e t h e canoni ca1 l= cos'o of
coefficients
between the spaces, and the
i.nclination
corresponding angles
define
If
the canonical angles.
the canonical angles between a pair
the
of spaces are all
sarne,the spaces are ca11ed isogonal. denote the nul1-spaces of
Let d , ]
Theng^I o,Tx I o just if 1 if
^ I
11 -
I o , a n dt ^ " " 7 , \ c -
O, in which case they are to define
of subspaces of (,,7
11 - fe.
ef,
f L
a canonical pair the
. The next theorern shows that
d l m e n s l o n s o l c" - l ,. T, -a.r, e e o r r a l t o m l A of the matrix
The nullitv
T L I E O R E5M. 2 .
the multiplicitv
of
as a root of the equation
C O R O L L A R5Y, 2 1 . T h e r a n k o f e f i s of its
equal to
ll1
-ef
= 0.
I
(3).
given in Afriat
Proof is
1 - ef is
non-zero characteristic
the total
nultiplicity
values.
That is, r =
r^ , where r is the dinension of iy' i n c l i n a t i o n b e t w e e nt , 3 lt - efl
T H E O R Es M .3.
= T f( r - r ) t r , l 1 - e f i = s P - r 1 1 r n ^ rl0 l
For the characteristic
first
part
1 - ef are 1 -tr
m, , and the determinant of a natrix
nultiplicity product of
values of
its of
characteristic
is
the
values; which proves the
the theoren. To prove the second part,
seen, by Theoren 3.3, that
with
it
is
- 8 5 -
l E ' r 1= l r - " f l
I
l 1 r ' r ; - ' L ' f E l= l E ' f r /
B u t , b y L e m n a3 . 1 , C o r o l I a r y 3 . 1 2 , t h e n a t r i c e s (l | |) have the
same non-zero
nrultiplicitiesi
Theoren 3.4 are the products
T I I L O I I L I5' .; 4 .
,T
If
a r r J V' . -'
required
vectors
a pair
of
the same as the of
tj,
js
V,
=
\' o' ^
, - ^ 1 , 1i t i o n s
O 3re
r
nair
of
II^ =
. r r t t i r r r It e t t f
sanle
spaces, l'rhich by latent
projectors
vectors
on the
spaces,
orthogonal i ty: of
rcciprocal
for, I =o , B I lt'
the
concltrsion follows.
reciprocal
the orthogonal
with
values,
0,
[l'
and they
\:
vcctors:.of
0, then tlie -
O,
\''
[i^
lrold atrtonratical
l v
als
U
S u p p o s et h e y a r e t r n i t v c c t o r s ,
so that
= ,,^,^'/' , flil
o ,-^^'/'
ancl
"Vr
l r J V )= l r i
Thcn
since ell, = U,,!=o = 0,
Ul
follow
hy synnetry. 0
so that
ll'
rP I
since ^ I
to LIr
6.
characterlstic
witir inclination l,
arrl,n^nn.,r ir..
I-li-l
ef
fol l owin5'propcrt i es of
Iravc thc
L
fI-,
anclnoir the
The reciprocal
of
- 1' I : r
\-g= (l if
0.
,
H e n c c l . 'l V t = 0 l s
O, ancl tlre renaining
e n r r i r r r l c n f
arrrri rrq l anccq
I'urtlrer,
r/z V'
L l z = U ' l l ^ t r"
"\
v ^ )2 = r ,
(lli
c\,
= 1"
t/z fUeB , cl)' \,
=
I. I r \' /
p
^ p
tlg
C o n f i . r ' l l a r t i o n s c ; rl c : . A set
of
ttvo or,
i n
of three or more subspaces
n r i r r e i n l e
:ts folnrjng a configtrration
of
a g i t t ' n - 1 r : t ' . i s consitleleci
of
sLrbslaccs; lurii lt colrfigrrration
is vjoncdrs attaining
_ 8 6 _
nodel types, according
to or departin5' fron sonc different t o m e a s u r e n e n ti n d i f f e r e n t a r a x i n . u mj u s t
scales, eaclt of whicli neasures
when a rnodelis perfect-ly attained and
measureszero when another mode1,which defines the opposite, i s a t t a i n c d . T h u s , f o r a c o n fi g u r a t i o n o f t r ' ' o s p a c e s , t l t e r e are tlre following ideal nodels: the spacesare orthogonal, orthogonally incident, isogonal, iCentical, incitlent, or one snace is includerl in another. AIso, for a configuration of two sfaccs, there are the fol Iov.ingnodels: the three s f a c c s a r e n u t u a l l y o r t h o F o n a l , a p a i r a r e i t .el n t i c a l , o r ' a nair are ortlroqonaland the third
is their union.
Ccefficicnts arc to he defined rihich set any confipuration i n a s c a I e , i n r e s p e c t t o e a c h n o d e l . 1 h e 1 'p r e s e n t . [ e a t u r e s n f f l r o o c r c r z I r n l n t i o n t l r e s n i t c e sh a v e t o c a c l r o t h e r w h i c h i t is possjble to translate into statj.stjcal conceptions i n t h c g e n c r a l s r r l . j e c to f n ' u l t i v a r j a t e
w h i c l ra r e f i t t i n g a n a l y si s . L n I l 1 A6 . 1 . ( i )
f or anv ratri,ces a. of
tlrp
cemp
nrr] pr
a
(t r a c e a ' a .) ' lc
^ ^ , , ^ 1 : + , , \ ( l ( r d r r L I _
hnl,ls if
€ {t r a c c aiao)
an,lonlv if
^ - - l d r r u
+ L ^ L i l s
for
s o n r es c a l a r s , l ^ ) , r n , il tc l t a r c (ii)
o
3
o
Z
l .) F (' t r a c c a . n r r C
+
+ I.a. = 0 t l
I'o t l r z e r o .
nc t
?
)
(trace aja1),
ala,) 2J h'
l'TT(tracc O
a 1
) (tracc ala,) ?-rU ^-r
d r r u
+r l Lr F
1:+"
s L [ u d l t L f
(tracc alar)' J ^.
,
Z holds
if
a n d o n l vf
which are not alI
zero.
i ft L
-I , l. i . i.
s.
-=
0 for
sone_
- 8 ? -
If
ai are of order m x n, 1et A. denote
the natrices
the vector of order nn whose elenents are the elements of position.
A, ordered by their
In the first
K = (AtAzAj). Then lilfl
K = (A1AZ),and in the secondlet holds just
and the equality rro.t^r l
in the first
if
instance let
= 0 for
KI
>0,
some non-nu11
instance of order twO and in the The lemma is
second of order three.
a restatement of this
result. L E M M A6 . 2 . dependent if (ii) if
(i)
Two symmetric idempotents are linearly
and only if
dependent
Three syrnmetric idempotents are linearly
and onlv if
a pair
either
orthogonal and the third The proof of this and it
they are identical.
L E M I ' I A6 . 3 .
or a pair
are
sun.
long but straightforward,
granted.
any symnetric
= 0,
(i
ei
idempotents
1 r . . . )
(trace
(j) where equality
(1i)
For
their
lemmais
be taken for
will
is
are identical
holds
e o ) ( t r a c e e . ,) , 6ltrace "o"j)2 if and only if e^ = er; u l z
. _ z, f f ( t r a c e e ^r e l- ) + 2 T J ( t r a c e e , e 1 ) V E { t r a c e e i ) ( t r a c e " j " t J o o 0
where equalitv
holds if
and onlv if
either
e . = e L o r e . ' ' e u= O J
and e. = e. + e, for somei.
i.
N
-
J
^
k which are a permutation of
0, 1r 2' T h i . s i s a c o r o l l a r y o f l , e m m a s5 . 1 a n d 5 . 2 . Since e, f are syrnnetric idempotents, and the trace of ^
d
*-^J,,^+ P r u u u l L
^r
v i
- ^ . - i c re\ !sJ r r l a L l
, i e n e n r l s 6 n l v/
nn v l l
t( h c
evclir-nl
of factors, R = trace ef = trace (ef)'ef
2O,
Ofdef
- 8 8 -
holds just
and the equality
ef = 0, which is the case just
if
are orthogonal; a1so, since p = trace e,
if e,T
p - R = trace e - trace ef = trace efz0, and equality
holds if
which is the case just
€ c T.
if
r,
multiplicity
R = r,
I'loreover, if
are equal to unity.
It
can be shown then
are orthogonally incident.
t h a t e , f c o m m u t e ,s o t h a t 6 , f
Accordingly the numberR, deIined for any pair T
and ca11ed the coefficient
then
values of ef, with total
the non-zero characteristic
all
ef = e,
ef = O, that is,
and only if
of spacesf-, between thern,
of inclination
has the foI I owing propert ies : T I I E O R E6M. 4 . 0 < R z p , e , r
f o r a n y s p a c e s0 , 7
p, q and dinension of inclination are orthogonal; R = r
C ,A
gonally incident; R = p if
if
r;
of dimension
R = 0 if
and only if
and only if
(.,7
s i n 2 1e , T ) + C o s (zt , T ) = p ,
ca11ed the additive
qnaces
h. . r. r f
i n c l i nr r rr qf
In
r
ion
r v r r ,
terms
of
thp
ensine
i,s such r
the
a
which
o f i n c l u s i o n o f the space€ in the space
T h e s i n e a n d t h e t a n g e n t a r e n o t s y m m e t ri c a l - r - , , - ,
t - n7 ,
cosine, sine and tangent of t
are indices of the relation
nf
l = (trace u-f)z,
= (trace ef)1, sinl€,?;
such that
the
numbert
ran(€,7) = Sin(e,T)/cos(e,T),
and
T,
are ortho-
€. is included in 3.
Thus there may be defined the positive cosl€,?;
and only if
,
rt ^t i rl hr li rr , h
:^ l >
-^
L U
> d /
functions of
also the coeffici ent
function.
canonical
angles
between
c o s l l t , T l = f r o . o r l o ,s l n l ( E , T l
f h p
=
c n a r a q
c n - T + /
r , . t '
) n
c i n - ^
- 8 9 -
If e,T
are orthogonal,
cor21Cr,fl) = cosz(9,*)
then
another space'
+ Cosz(|,&);
it
then g = e + f
for
and &
union,
by their
is
the orthogonal projector
trace gh = trace eh + trace
so that
ofLt,
fh.
For example, 1et 1r flr r be three independent rays, concurrent on the origin
the orthogonal projection of 1 on n. Then the
and 1et 1'be
Bry
that if It
of 1 to 1' ,
is the cosine of the angle between1 and 1',
and thls If
t o t i s e q u a ] , , r t toh e i n c l i n a t i o n
of i
inclination
U; and 1et "ibe the plane through m,n;
it
are the angles hetween I and m, 1 and n,
f r o m L e m r n a6 . 3
coefficient
there is defined for
that
non-negative real nunber Q, to be
any pair of spacesQ.,T^ called their
(i)
follows
= cosZ B* ,orZ \
m, n are orthogonal then .or2 o follows
sayo.
of disinclination
fhp
nf
whlch is given by Q" = pq-R' 7-o, are identical, and where where equality holds just 1t t,! ) ) Q ' = p q j u s t i f R " = 0 , w h i c h , b y T h e o r e m5 . 4 , i s t h e c a s e just
if e,T
are orthogonal. Like the coefficient
of
inclination,the coefficient of djsinclination is a symnetrical function of the two spaces, and it
has the following properties:
T I I E O R E6M. 5 . O < Q l < p q f o r a n y s p a c e s E , T ? Q' = O if
it
e,T
a n d o n l y- . .r rt bv r v4
^ L- e^ ,: -so r l t i c a l ; d
of djmensionp,q; ? Q' = pq if
and only
are orthogonal. S = lt
N o w1 e t
_ efl
;
then appears, from Theorens 5.1 and 5.3, that 0 ( Sg 1, and that S = 1 just
it €,T
a r e o r t h o g o n a l ; a n d , b y T h e o r e m4 . 5 ,
- 9 0 -
S = O if
E ,T
and only if
numberS, defined for
Accordingly the
are incident,
any pair
of spaces, which will
be
of separation between them, has the
ca11ed the coefficient following properties: T H E O R E 6M. 6 : O s S ( 1 E,T
if
for
are incident;
any spacesQ.,T;
S = 1 if
S = O if
and only i.f 6 ,f,
and onlv
are orthogonal.
The number
c = lr - erl may be ca11ed the coefficient
of inclusion of
properties now appear as a corollary
its
Q,is
is
i.ncluded in F.
in 3,
incompletely inclined
C = 1 if
to T;
and
o f T h e o r e m6 . 6 .
for any spaces^L,T; C = O if
T H E O R E6M. 7 : 0 { C < 1 if
€
and only
and only if
L
I t f o l l o w s t h a t t h e r e a r e d e f i n e d t h e n o n - n e g a t i v er e a l 1 1 2, numbers sin(€,}t= I - cr i, cosre,;t = lr l "f
t a n l € , T ) = s j n ( e , T ) / c o(st , T ) ,
and
which nay be ca11ed the multiplicative - r 1
7
tangent of L on J.
sine,
cosine
and
In terns of the canonical angles o between
t h e s p a c e s w i t h m u l t i p l l c i t i . e s m^ , i t
follows fron
'fheorem
5.3 that
sin( 6 ,3) and
= T l r i n i l o o , c o s( € , ? 1 = 6 P - r f T . o r t o o , o ( fr a = Or-P fTt".,to o. tan(L,T) a,ttlt
Their properties It
follow
is noted that,
if
from Theorens 6,6 and 6.7. (,T
a r e o f u n e q r - r a tI l i m e n s i o n , t h e n
a t l e a s t o n e o f t h e c o e ff i c i e n t s o f i n c l u s i o n . o r 2 ( € , 7 ) , Z , + r cos-[ J,*) i s z e r o ; a n d i f t h e y a r e o I e q u a l d i m e n s i o n ,t h e n \) the coefficients
of inclusion
are identical.
The coefficient
separation, however, is a synmetrical function of the two q n q a a q
of
-91 .
and disinclination is
between spacestr,6,
the orthogonal projector
then
of inclination
denote the coefficients
Let Rr, , Qil
three spaces i,
define
n - i 3j k
and call
the real
the coefficient relation
number QijU with
j , k
1 ? az j-piRir-pjRii-pr,.ij, cube thus defined the
between the three spaces. Then
of dissociation
with the coefficients
satisfies
the following
of inclination
and dis-
between the pairs:
inclination
\ ? ? ? ptat:r = ai:air - (RiRj1-RijRir)'.
THEoR6 E .M8 . For,
and
j , k having indices i,
== np ni pnj p+ k?* p2 R . r pR r i pR i
of dissociation
coefficient
e,
and p, the dinension,
on 6,
Pi = trace ei, Rij = trace eiej ^? ^2 q i j = p i p j - K i j . Now for
if
so that,
from the definition, 7 p . 0 i . , t'iYi jk
=
r' vni l -1 j
-
) ) D ' ' \ - r D ' ' i' j )/ r ' \nv i vnk - D \vi"jk "ik/
) - D - R . , ) " . "i j"ik.'
T H E O R 6E.M e. (i) o (p;aiiu < al:qlucpipipl for anythreespaces; (ii)
Qiit
= 0 if
are i.dentical.
either a pair of the spaces
or a rrair are orthogonal and the third
iCentical with their (iii)
and only if
is
union;
7
Qi,r, = p;pipr. if
and only i@
orthogonal to each other.
By Theorem 6.3, ai: t , 0, and equality holds just under the conditions stated.
By
Tr rhr as nv rr E a nr u
A . o e, u
----i*---. fu h r roq l T l S X l l T l U t O t
^5 Qijk
. RJ ,K, - - R ,. R , , ) 2 i s a t t a i n e d w h e n Q r r , Q i t a r e m a x i m u ma n d (' p ^ 1 l J 1 K ' , i s m i n i m u m ,t h a t i s , spaces are all
z e r o , and this
orthogonal.
is precisely
when the
- 9 2 -
T t l E O R E I("6l . 1 0 ) : - n R " i. j.' n' i k- t i
z D I ) j uOi k. . -Z nP. iR" j. k - " i j " i k
n
\ i j \ Or k. ,
either a pair of the
and onlv if
and an equality holds if
a
atd the_third
spaces are f_{S-$-i_-_l_:j__g_-pgff are orthogonal rvith thei.r union.
i,s iclentical I'his
for
lirnits
follorn's by
result
inclination
the
to
two spaces have high
Thus if
to
Cisinclinations to
each other.
it
is
a third,
o be the
and lotv
inclinations
t h e y n ' u s t h a v c h i p , h i n c . li n a t j o n
let
n,
fornr to
sinBsinylcoso
is
attained
sincc
then o
just
though
,
tlre fo] lowing rcsul
n nake angles
angl e they rnake togcther.
cosBcosy an,l an equal ity
rays,
are given.
a third
a lencral i zation o[,
l,et 1, ni. n be three
a n c . lg i v e s
and 5.9,
Theorem 6.10 has a sinilar
not lreciscly
throrroh m. n-
5.8
between t\{o spaces rvhen their
and disincljnations
incl inations
and I et
'l'heorens
with
B , y
1
Then
4cosB cosy+ sinBsiny, jn
w l r e n 1 I . ic s
= B + y or
tlre plane
o = B - y
7. \rolume A base L rvitl'r span & is vectors
€1r...
\cctors tlcf ines or
a-
\ thc
by
the
If
L,
=[rJ,
b y t l r ei r
p intlependent
r Lp and the p-clinensional region
= L r x , + . . . + l . ^ x _= I . x ( 0 < x , ! 1 , I r p p r paral l el otopc ['
base,
nith
= f.I ]
bases of
the parallelotope
join [ffJ
is
in
I
of
i=1,...,p)
. tc t e r r i n c t l h y t h c
vectors
volurne rlenoted by l/21 .
I: are selarate then
conposed of
of
dir:cnsion p,q
=
LfJ
dimension p + q deternrinecl
c a l l c L ] t l r e p r o t J r r tc P x
ltrO j n r )7: . 1 . 1 [ ' lz = l I ' r .I
andF
:
ol
f, ,&
.
_ 9 3 _
Proof is b,y induction on the Cinension p of the para1le1otope. The theoren i-s verified .-^r..*^ ^.
^
^:--re
lUlUlllg Ul
d
he
Fnr
nrnyerl
, - ^ 1 ; l : + , ,
r aarur
L . / fr ^u* r
1et&
= [r]
is
cflsion
p
t,. the
t:
I(
t.-t),
of
the
itS
= (F[.),
so that |
,sotlratP=A^Pr, \orv
lenoth
wil]
rvitlL ,2" = f t.l,
Thus,
1.
whereF = (tt..
a n c l l e tP r = r ' l t e r e nl r
. -
r ^-^-L
tgllELll.
L^ ^t ensron p = r on rn c Ln.y- -p^ oa Lr ^n^e: s r^ s o 1
d - .i ,r l J i u '
s .Lirr nt lnr rL -y . ir -L -)
t.ectol" is
)illFr!
clirension p = 1, since the
for
p
=&pr,
frnm
nernendierrl:r
tlte
cxtrem-
o f L , o n t o t l r e s n a c e € . c o n t a i n i n g P ; a n d t h i s p e r p e n ri lc u l a r
ity
1s given by the vector (1 - f)Er, wheref = f(f'F;-1pr ^L+^:-^.1
V | L d r l l g l l
L., U /
orthooonnl
e
r
= I
I
+( d^r 1. l. r : ' ll r
O r t h O f l O n a l.
cnmnlenent of
ft'L ) T
tliC
- 1'
T '
l'.'l'lren T
T v ,
nr^innfinn
s o tr h. .as t!
,
n v rf
l ' t V J l L ( I v l l
E'f1 - f)f T
f
pr f 1 = f- :f ( 1 . _
-f)
= lE't I
lt
T '
T
f-
On
the
E -. " I .
Let
- fe
I
I
h y 1 ' h e o r e n4 . 5 . , \ c c o r d i n g l y , h y t h e i n t lu c t i r - e h y p o t h e s i s a . '2 = L&l
I, F , and by Theoren4.3 again, J ) ) = 1 Q l =2
J l l p / 1 r ' r 1 r t ; t , r 1 1- f e r l = l l r l c o R O L t , .A RIYP , L l = I p l l . 9 - ls i n ( 0 , 9 , ) . F o r ,i f , t = h l , 3 1= f n l, t h e n l p ^ & l = l f r r=l ll I c l l ; and
l c ' c l = l r ' r ll l r l
II-"rI
o b . nu* r+ r1 n, ^o^ ^g^ o^ ln ari l -- -ar c l o n
lf
a b e s e l i , c c n p o s e do f p i n d e p e n d e n tv e c t o r s t , ' , . . . , L p
is given, then an equivalent orthonormal base U, conposed ^r
u l
P
^ - +L Lr u^ t^l ^u r- r^drr
u r
,u rl n i l l IL
rv ,gpr erL nU rr c)
lu t1
r .
..,
cen
be
COnstfucted
I '
w i t h e a c h U , i n t h e s p a n o f E . ,, . . . , qrrnn^qe the
IJ-,
eonstluction
be
can
Br (r = 1,...,p).
carried
o u- t-
f- o- r-
U - r_
For,
(r=1r...,
n - 1 ) . L c t I r = ( L t . . . L m _1 ) , \ ' = ( U t . . . U n _ 1 ) ,s o t h a t s i n c e , b y h y p o t h e si s , U
the bascs f,
= \'r\tr;and let
\
a r e e q u i v a l c n t , I F = I r n -I =
Jn_1=I-Im_1.
Then the vector
- 9 4 -
U = m
r/2 fE'J .t )n r -
,t
n
s u c h t L ^ + l' 'l
1r"....U :
J
l
m - l
vector/orthoronal to n /
a unit l
is m
' " ' ' L l rE
^ "d
d li,....,ll ' r '
have the samc
m
s p a n . T h r r st h e c o n s t r u c t i o n c a n b e c a r r i e d n s t e p s i f can be carried m - 1; thereforc, since its is evident for m = 1, it nnqsihlc
.n ,
fnr
L,,...L-
tlre orthogonal
pro icctor
follows by induction that it
I
U* = ilr
on tlre orthogonal
llris
connlenent
and
(' .I -rn: "Jn- -- l, 8- '-n)' - t / Z l ^" n -
r [ 1 = rI t . .- - p ) , "
r' l"n
are n nrtlrncnnAl rrnj1 VeCtOrS SuCItt]rat
U
1,,... r[-
is
a r e p i n d e p e n d c n tv e c t o r s , J n _ I
t l r e s D a no f F , . . . . . t
then I
possibilitl'
p.
=
T l I L O R L8I l. 1 : I f
of
it
a n J l ) 1 , . . . r 1 1 -h e v e t l r c s a n e s p a n ( n =
t l r e o r c ,r tc i. v" o s
r - , r n li c i t
fornulae
for
thc
1,...,P). Crann-Sclrnidt
o r t h o g o n a l i z a t i o n p r o c e s s.
9. Canonical deconrr_csition T I l l r O R E9I l. 1 : I f lI-fc
(I I
I-et,
tnen
€ , , F ^ a r e t h e n u 1 1 - s p a c e so f
x1-ef ,
are thc null-sfaces of 1-ef, O), an,l Lo,T o
T . C -
whcre thc spaces E- , 1
A F v ( . ) I
^}
t
=
} @
"
t
A ' V X ^
., are ident ical , the spaces 8^ , i,
( r I O ) a r e i s o p . o n a ,l a n d t h e s p a c e s E ^ , ( , r ,E; 1 , I , , ; t
-
+
J.,Fu
(\l
u ) a n d e . o , F o a r c o r t l t o g o n a r l A. l s o ITI I
-
e^ = (F € \i, r=1
llr
lIt= @ i=1
7), j
()lo),
- 9 5 -
wlrere t, e r o
rr rv i i tLl I' I
ri nr rn ! l ri rn r or ,! lu ,
li l- l. L1 li an lol fd L l U i l nl n
J
r
, J
nf
rrvs
r
Tlre sot
.
"
"
,
-
].
j L
onnnl
n r n i e c t i Ln t nu
B u r r d r
f
,
, . ' r
in /
,
ti l. L7
l l
l , r u J q L
,
^ - l d l i U
t t \ 6 t i l s
h . : - " P d r r -
F " l
r i t
, J
tlre ortllo-
ntaybe obtained by taking
,
^ vf
ae t nL v l
vn r r
sets
of
u L
an,t t\6,p the
r'jl!)'
^I
; , T \ i ( i I j ) a r eo r t h o g o n a l '
&x;i et;,Tt;i7r '',
'F,
, , ? ], \ , ) i a r e r a y s s u c h t l r a t t h e p a i r s f r' ,Y ,
, , 1 '
t Ll rl l 6v rt ,sv 6r r nu 2 l
rays t,
"trrl
st-.t of
.rr
rt sa ) v S
. and 7)
:
are
the
orthofonal projections oI each otl)er in e ,]
rl
If
C ^ . , T ^ ct e "T
0 then d
.
r
-
By Tl,coren 5. J, €.^ and
Y
J-
a r e o f d i n r e n s i o nn 1 ; a n d h y
T h e o r e n 5 . 4 , t h e p a i r s o f s p a c e sC ^ , ? , a n d 7 1 , i ,
( ^ ,L 10,I lu)
a r e o r t h o g o n a l . R y T l r e o r e n5 . 2 , C o r o l l a r y , e 3 a n d f € a r e o f c l i r n e n s i o nr = ,, 4r u ^ "r.
It
follows now that
tL = Eo\
^ T = L EI )f 0
F u r t h e r , b y T h e o r e m s3 . 5 a n d 3 . 6 ,
L= eYAeo, 7= f e@To, wherego Ls orthogonal to e Iand By Theoren 2.5 , e,
to cF and € intersection
T , anclFo is orthogonal
of
€,T
a n c tT - , , a r e e q u a l t o t h e
and therefore idcntical.
Now, cornpleting
t l r e j - n i . t i a 1 s c h e m eo f d c c o m p o s i t i o n , t h e s f a c e s e , (
tr, u I A,
f
I l u ) are orthogonal, by Theoren5.4.
Let l1 = (ti
. . .l ^,rr^ ) be an orthonornal base for ,1 a n d C , i t h e r a y s p a n n e cbl y t h e v e c t o r E l . ( i = 1 , . . . , n ) . t
I e t I r n, r.1 = ( _rAi
t
. / L- I r l . ) - t / ' f .L -^^ ,,
L
he a unjt vectorin the
, 1 ,
which is the orthogonal projection - rn. , . =/ e r 'lhen, (P- , . i n ?. } by Theoren 3.4 , r a y ?- ( .l r
efli^ = E^r and
"F.l
= ,^"/2,
, feF^tr = Frr fq
8^,
= F^^l/2'
,
of
- 9 6 -
Ei Er = 1, riF^ =1, EjF^= 1x7/2,
where --r dllu
-Lj^ Lrra)
^j.-^^ t
.l.e LIls
r so (nl rur r. l iqr ue d
\c un l nl LC l rr tl S i O n .
on Ci
Now the orthogonal projectors
f^ = F^lr,'
= Fl Ei ,
ct
, )^ur" ,
and the non-zero characteristi c values of e , rf r = E ) t i f l F i are the sane as those of E A\ FA . F A. ' L \ = l ) ,.1 .j ^L lvlLlLIr
i -
+ ^
I )
^ - , ,
Jdj/
LU
+( rl rl \ n v
n d rr e\ : l I
)
t hn
Sar ne and t
I'hcrcforc or
thcse slaccs arc
l
tO) .
Y
betwecn L^, J), are
anglcs
tlre cancnical
eOttal
e q t t a . l,
isogonal.
f =I f 9Y .11. e =Ie l, COROLLAR t r l nrthnonnrl nroieqtorS Such tl)at
f^ are
, tr']re r e e ) ,
= f 1 ; e ^ e u = C , e l f u = 0 ', ^ f . ' 1{ ,) = O ( r I u ) ".1 t r a c e e l = t r a c e f . \ = n ) . , t r a c e e . f , = lA n , ^ r l = o )
eofo = 0,
o . 1 ls p a c e s ( ,
Jn any pair
C 0 R 0 I . L A R9Y. 1 2 .
there exist
F
b a s e s E , F s r . r c ht h a t l
-
t
=
| l
t
t
-
F
=
|
t
!
t
r
-
0 ' p r
(:
0
ln
Iloeinrneal
"rd
eXtfenral
COnditiOlS
T l l l ' O R I l I1l 0 . 1 : T h e o r t h o g o n a l n l o j e c t i o n a given space is
or ra)'in
: . p g : g _r ' h l j h _ h a : _ % l I e m e fL li unl nt
tt nu
tt lt rl ge
c fi u
vt nu rl l
yggtOf
@
t h e u n i q r . r ev e c t o r o r r a y i n t h e
distance Of
:)
or coefficient ^^+:"^1"
t'av
of ^-"1
r g ) l J g L ( l t g t y .
d l t l
i.nclinat l r _e
extrente 1s a mrnlnlLln. Let
Zo
b e t l ' r e {c_i 'r^r"n" n \ r e c t n r
's_1r)-a* n, ,n, ,i n - .p, ttrh e nroiection
of
or
1et
Z
o
be a vector
T- .l 'i 'e. r. r .X. o = -I -I i -7 -i 's- t h c o r^ t.h' ' o - pr 'o- 'n a l o o , ., t l r o v p c t n r i r r t h e Fo i l - e n s l^l a c e - . l i t t n l)ase
piven
rav.
o '
-97 is a vector which spans the orthogonal projection
t^, or X^ o' o
of the given ray in the given space, and for an1'such v e c t o r X . L^ LZ - o = T ' t o "7o ' X = Eox is any vector
If
hotweon 7 -
"
'
o
s
r
r
\
'
in eo, the square of the <listance
enrl Y, and the inClinati.On bettveen the Iays !
/
!
'
s p a n n e db y t h e s e v e c t o r s ,
are given by
(. -zo - yr r ), ' r\ "7o - ,y. /) r (, -zo '. x. , ) 2,/ (. -zo -' o^^z) ( x ' x J. They are extreme just
for all
(' Zo - X ) ' 6 X = 0 , i t ^ 8 { - ' ( Zo l X ' X / ( X r X )= 0 , - )X6 o dx, where 6X = i:o6 x; and these conrlitions are f rY =
onrrirrrlnntt^
and then to n c r e n r r ir c . l X , ' o ,
:rc o r L
w w ai Ltl h l
7
40
if
-f or ' TX - 7o
'F' or - o '
=
ltl "o-o t
X = IE Zo, IXZ. = I, Zo, o o T f f o I l . o w S t' h -'a - t- X " o^ t, a n d t h e r a y S p a n n e db y
iu lr nr r i\ nt ur! rr e / l v . r o r a r n ' i n a . r
e n d tL hl l es
drlu
r1 ^4 /"
hV
the
+L rrr '-r v^u' t,<^l rr ' Lao , '
-nd
d.
Stated
i t
is
eXtremal
COnditiOnS
that
evident
tlre
extrenes are ninina. T I - I D O R EI IO{ . 2 : T h e r a y s i n a g i v e n p a i r extreme inciination nf rews in :-:---:-:J--:----:
Tliat is,
thocc
of spaces which have
to each other are the reciprocal pairs
sPaces
'
the extrenely inclined
rays are the rays which
are orthogonai projections of each other in tlie spaces; and these have been seen Lo be tlte pairs of
rays spanned
hv the rcr-inrnc:l nairs of latent vectors of the nroducts '
of the orthogonal projectors on the spaces. that if
the inclination
one space or the other,
l -
The proof is
is extreme for variations
rays in both the spaces, then it in just
_ " -
of the
is extrene for variations so it
fo11ows, by Theorern
9.1, tlrat thc rays are orthogonal projections of each other
- 9 8 -
' in
th^qF
sn2rcq
The di,rect consideration of the extrenal condition gives a n a l t e r n a t i v e f J r o c e ( l u r e o f c o m f , u t a t j o n .t h t r s , l e t X = E X , Y = Fy be unit Then for
{x'E'FY)
all
rays.
vectors spanning extrenely inclined = 0
6x, 6y of x, y under the constraints
variations
x ' E ' E x - - 1 , Y ' F ' F Y= 1 . Lquivalently,
lrFy =
F'lx =
L'lx,
F'Fy,
w h e r e ) , L a r e t h c L e g r a n g i a nn u l t i l l i c r s
c o r r e s p o n d i n gt ,o
Then
the constralnts.
= x'E'Fy=p .
I
B u t , r n ' i t h I = u , o n e o f t h e c o n s t r a i n t s b e c o m e sr e d t t t t d a n t , s o there is
the following
conslusion.
TIlE0liLl1 l 0.3. A necessary and sufficient
condition for
p = x'I-'l)t
of x, 1' uncler the
t!
!9
eflf-gn'e for
c o n s t r a i n t s x r I :t l x = 1 , I ' f ' l y
E r F\
= 1 is
/*\
=. +r'rJ (.ru
l-ro'L
\
variations
r'r.
C 0 R 0 L L A R1 Y0 . 3 1 . T h e e x t r e n e v a l l r e s flra
anrref
,
X t L t
Irx = 1 .
are the r o o t s o f
inn
-PI-'L
Ir'f -pF'f
f'L
Tt has been seen dircctly,
i n T l r e o r e n3 . 4 , t h a t t h e s e
tt r o o t s s q u a r e c ia r e t h e r o o t s o f t h e e q l r a t i o n I p
-
trIr, I = O,
r v i t h t h e s a m er n u l t i n l i c i t i e s . T l l l . O l t l i1\ i0 . 4 . l l r c n u l l i t y i.s thc
nul_tjgl i ci ty
of
of the natrix as a root
of
/ - -o; L, ;' L .. K- =i " i.. l L
tlrc cquation
L'f
- p I .F , rF
K
ol
= 0.
- 9 9 -
This can be proved indirectly
as fo11ows.
established, or directly
orthonornal
are equivalent
fron
results If
[l = Ec , V = Fc
bases, 1et
""'\ r{= o ( - ? ' u l } , y = (/ o" _oj/
\V'tr
already
o\
\o
S/
)
so that Kp, li have the same nu11ity, saY Then Y'\., = W or o D p , a n d * o l = O , I w o l = O h a v e t h e s a m er o o t s , t h u s p w i t h mrrltiplicity rno
l ' J , b y w h i c- l l
a cilaracterj.stic value of the symnetric natrix it
follows
of oas
But mo is now the nultiplicity
that np = tp
/Xn\ If|| " I denotes any solutjon correspondingto a root p , \yol j.t folIows that there exist no suclt independent solution,
and these can be chosen so as to obtain no ortliogonal vecto r S /x" t\ = 1 , . . . , m ) w h e r e x o = E * o , Y p - F y p . N o w f o r a nv tr"'1lfi R'1 Q, a , evidently sdch vectors, belonging to roots pX YJ Xo = o X'oYo oXp , XoXrg = XoYp = oY'oYo
YJ Yo = so that It
follows that
1f X;Xe = O, tlien, provided
Q , ,o I
O,
Also it appears tl"Iat xJX o = o, x;Y o = o and YlYo = 0' = p:o and X;Xe = Y;Ye i'f o =p. XJ p = O agtornatically if It
is now seen that
vectors *0,, Tft
there are found n p orthogonal unit
t n E a n c ln o o r t h o g o n a l u n i t
= i,...,m),
for eachnon-zeroroot p , such that
X u' ^r l. Yp ^r J; = O j f The final explicitly
vectors Yo,i it
olp ot i I
j
andX'-iYo,i = o' " t -
theorenr,which is easily t h e e q r - r i v a l e n c eo f
verified,
shows
the three conditions
fornul ated
between pairs of vectors in the spaces, that they have extrerne j-nc1i-nation, that
t h e y s p a n r a y s w hj - c h a r e o r t h o g o n a l
-1 0 0-
nrniFrtinns nf
the
how
nf
oech
nrodrrcts
the
otlrer. the
of
nrflrnonn:l
in
of
calculation,
in
of
solutions
pnrref i nns
2Te
ternts
t TanSfOfned
into
T t I E O R E 1I l0 . 5 : P r g v i d e d p l
lrtcnt and
of
and
nrc
nrn iectors:
t1{o processes
\-ectors,
tlrpv
anrl th2t
of
terms
the
veCtOfS it
shows
latent
extrerr"ality
eaCl'r Other.
O, the equations
L ' F y = p t ' E x , I r r E x= p F ' F I , x ' L ' L x = 1 . T ^
q
r
a d r r i r r q l e n t
\
\
\
t
u
r
!
u
4
e
r
r
l n \
r
l l r a
v
\
r
'
\
c n l t e t i n n c !
Y
v
e
v
r
v
r
r
v
e Y = X p , f X = Y p , X ' X = 1 enrl
to
the
r . o r r at i . o n s
efX = &"
7
- 1' r/ '?f X , , y = (X'fX)
XrX = l
and to thc other sets of equations obtaineC by interchanging \
=
Ir
enrl
=
Y
Iror gencral is
rnadc to
in
Afrjat,
included lltis of
Fr-
in
flroce
rccolrnts of
rr'itlt ccltain
tlre tlreory of
for
the
naner is
sal(e of
nethods for
rvorL tlonc at oripin
trlrielr nnnperod
T wi,slt to
tlrc
s u\ t lu tJi \ o c! t L.
^
projcctors, Ilalmos.
refercnce
ltlaterial
natlreratical
featurcs
tlrc Lepartnent of is
Applied
in an attcmpt to nake in
nn
invostiontiOn
Of
t h e s e a s o n a l c o r r e c t i o n o f e c o n o n j - ct i m e s e r i e s ; exnreSs
rvith
thankS
Prof. J.R. N. Stone, botli for th is
=
c o n , f r el t e n e s s .
a rlevelonnont of
tho:lonl'rn
YrY
lras lrerc been
irprover,cnts,
I c o n o n i c s , C a n r h r ' i d t . ' .I t s
end
innlv
tlrev
IlanbLrrgel and Cr j.nshaw ancl to
son:cstatjsticcl
cnneiso
;rnd
,
and q r r u
dor.nlnnnront of
al S J UO fr vOr f o l
nv
indohtedness
introduction
f9
to his work in
fL rl lrr o n r rr i nr i fL j' /' a l Ir O u nW f r ri Fn o U PnI n unr r f L u d l
t" l" r- o n rr.qcnt t'
mAterial.
A
statement
ncthod for the analysis of seasonal variation, gencral noint cf vietr of rnultivariate
r^-
r u r
of
frorn the
analysis towards
_ 1 0 1 _
which tlris naner'i s direcferl. l
Also
I
have
especial Iy
to
ideas; of
and to
of
in
L
r
is
!
in v
,
corrrseof
Adams, l'!r. Alan
Irr, Il. I'cnrose, witlt naterial,
regar(l to
our collaboration.
thc
1
,
indebtedness
di.scussions ivhich did
tlie algebraical
esfecially
\
acknorvledge ny
lr1r. Arthur
Ai tclri son - for
^
^
'
to
nrenaration. r
ny colIeagLles,
Brown and l'1r. John nucli to
trlronI
lrati tliscussion
and who rnadevaluable l e m n ' a so f
S 6,
these
elicit
suggcstions
tv)iicltare
rcsults
-LoziII APPLI,],IX
1. Orthogonal Prgjectors
tliere 1s a saInPle experirnent wl-iere bY a The clata is dc;cri.bed ' v a r i a b l e s p on of n observations e R I I and column space . -, € . n rn ,w. ,r i t h p c o l u n ' s X l , " ' ' X O t " X natrix Consicler a statistical
t = X R P
(1.1)
-
l-xt L"-
:
-- rL\ " 1t ' 1
+
'
r
tlll .
v
+
"p'p
t . l, . . ' , t p € R ]
Since
r a n l i X = P ' a n kX ' X ' i.t
fcrllorts
that d1n
which sllows indel-etltletit ' tilat
r : + ; ^ r cotltirLrvrt
tite ^J l r l l
r
lr r !n rr ^r \ e n !
to
t,
-
!f vn rr fOff
1i t r e a r l Y X . ' , . . ' , * Ot o b e
t l r e a
base for their
case
SPOI) r-'
rtL
-lx'
(1.2) i s c i cf i n e c l '
l x ' ) 'l l o '
t
I'
it
is
e = X(\'X) I = € r f r o r nw h i c h i t ider.potent.,e
ranll e = trace e' Irrtt
'X' 1
tracc
e = trace X(X'X) - t^ X ' I trace (X'X) tI'ace r-
-
y.
folloris that
-1 0 3 -
T h us c ji n e p . n = r a n k e = p I ; d i r n E . But, from (1.1) and (1.2), obviouslY elin c l, and this,
of the dinensions, inplies
rr'ith equality
elln= t.
(1.r)
T h u s e i s a n i d e n p o t e n t w i t h r a r t g eL . l r o n
(1.2) also it
is
e . B u t a n y s y r n n ' e t r i ci d e n ' p o t e n t i s i d e r r t i c a l
synnetric, c'=
rnith tl.re orthogonai pro jector
onto its
r a n g e . T h r - r se i s
the
n r r l r n o n n e ln r n i n e t c r o f L . l l r e n t l r e c o n p l e n e n t a r ' ) ' l l o j e c t o r c = t - e is
thc orthogonal projcctor .
o n t h e o r t h o g o n a l c o r n p l e n e n ti
,,n
oT t. 1n 1..
I n d e p e n c ci n c e
2.
Ye nl, p'
C o n s i c i e r a f L rr t l r e r that
lY'Y I I
coltrnn space f ,
strch
so that
o,
f is
"ith
dcfinecl, and is
= Y(v't)-lYl
the orthogonal
projcctor
on F.
t{i th
Z = f ,X. ,Y/ l _€ .R. Jr r + q , t h e c o n dj . t i o n f o r t , equivalently
dir,'lvF
is for EA F = 0,
= d i r nl ; + d i m F
lZ'Zl I 0. Rut there is the iCentity (Afriat 1957),
l : ' : , 1l /x ' x l l v ' Y li =x ' i x l z l x ' x, l
(2.3) where T
t o b e i n c l e p e n c ' l e n tt,h a t
for
(2.2) is that
I
= t
(2.4)
-
[.
I]ence thc condition
! r ' - r x Il o ,
is equivalent to
- L O 4 -
and sinilarly
t
to
i n d e p e n d e n ti f
-
l
lY'eY I I
O . A c o n s e q u e n c ei s
tl-ratF, F are
the orthogonal projection of E on
and only if
the orthogonal conplenent of F has the samedinension as L. For, since i
is
symrnetric idenrpotent, din,fl
= ranl. fX = rank X'fX,
s o t h e c o r r s i d e r c dc o n d i t i o n i s e q u i v a l e n t t o d i n I f
= p.
3 . 0 b 1i q u e p r o j e c t o r s L e t L , I r b c i n d c p e n d e n t ,c q u i v a ) e n t J y l r a v e j o i n C = I vI-'
(3.1) w l l er e
din' C = p+q
(3.2) 'll'on
Lrr
-f ^-r r, . ' 8 c ) r n f,
+ha
(3.3)
e
x
- l = X(X'fX) t\'I
i s r l e f i n e d , a n c ls i n ' i l a r l y s o i s
fx
:
Y
Srncec p.
It
t s r J e n ' p o t e n t ,i t s
r a n l . , c q r r a Jt o i t s
f o l l o w s , b y t l r e a r f l u e n r e n itn S 1 , t h a t i t s
trace, is
range is l.
l{
Silrilarly
g.
has raniic Ir.
Consider any elelrent ) C r'itli
t l r e ' o r t l r o t o n a J c o n r p J c n c n tC o f
X ( r h v ' ci u s 1 1 . e y = 0 ,
= n-p,
fol loris that
F - C is
P,rr onv i,rannnr^'tt is r ' 1 r r f l a n e r a l l n l
'ora'e1
equal
identical idcntical
to
in
the
join
the joir, C of
^ so the nu11's1,ace of
c l l n , e n s i o nq * [ n - l - q )
i + c
FVe, tlrat is
L antl f.
its
din"ension. It
l,;ith the
nu11 space of
with
F
X i- contai.ns I:\rC, of
thc nroiection X
n rur 1 l l 1 -s p a c e . l l e n c e tt ^o ii .+ tcs n e- is
fro.iector on L parallel
of
el
onto
the
to the join of F rvitlr the ortlrogonal
c o n p l e n ' c n to f t l r c j o i n o f I . a n d I i , a n t l t h e r e i s a s i n , i l a r Y
staten,ent tor
t
- 1 0 5 -
Niow
l"* = irlx'Ix1-Iv'6'
(3.41
appears as the orthogonal projector Itence fJ
the orthogonal projector
is
p r o . ie c t i o n o f t It
on the range of TX on the orthogonal
in the orthosonal complenreno t f F.
is noted that x Y Y X e - ' f" = 0 E f ' e ,
(3.s) Y
Y
s o t h a t € , f - a r e i n d e p c n d e n tp r o j e c t o r s , a n d l r e n c e t h c i r sumse
x
x
+ F- is the projector onto the join of thei r
and para11e1 to the intersection T h o f
i- c-
F = c
E ,
* ^ ' . ^ 1 1 ^ 1 lrdrarrr-a
-!ll
( F v G )n ( t v c ) = e ,
(3.7) wlricli shows tlrat
*
J
(3.6) ^A
n r - r l 1s p a c e s .
^ n t ^
t
(3.6) ^ - - t drru
of their
ffi is +
ci
!^ u^ r- ^u rr 1r ^d *r .) ,
; -
rr
+
+ l
ortllogonaL frojector
t
. X-
f
;
is
synrnetric,
wlrich
is
e L rv r ri ud\ e n t l w
x x e e = € , e g = e
i i . S l i
f
e n n p . T q
f h r f Y
(5.10) q n r l
c i m i
Y
+ ef "=
c
Y
It
e
I e r l v *
f ' + folLoivs
f e " =
f .
that }
(5,ii)
*
c-' =cf-+ f Y
an(l llcnce
-
e'
}
fe'
Y
= c + f Y
= I"'-
+ f
en:ririr'l cntlv
""'-r
(3.12)
x
x ie
g on G
fx +
o-
trtcl
the
o t h e n i i s c i n n e . li a t c J y a f p a r e n t . Sincc
ranges
+ f
= ef
+ e.
Y = f '+ fe'+ V
ef "*
e,
e
not
_ 1 0 6 _
_ x anC f both fe n r n r l r : e t i s n r : ll r r s r 4
^ r
!
f( nv
sun js the orthogonal projector on
so their
t
the joi.n of their that,
are otthogonal projectors and their
for any L, F q Rn, if
f( hr r eL
n r f l( rj nr vc! n' vnr er o' rl v r
is concluded frorn this
ranges. It
= C then the join of E
f / 7f
nl / rr nv _i e e t i o n r \ ! r r v I
n v rf
F r
i n t o
ir t \ cJ
r r r u v
is identical with tlrc join of I'to of E into
identity
n v rr t h L lov ol .nv n l uar l
c! v n, m ' P n 4 \l rer n v rer n ! t
t l r e o r t h o g o n g fp r o j e c t i o n follows that the
It
the orthogonal conplenent.
spacc in which they cojncidc contains hoth I and F, and hence their
jcrn
with j.ts trace,
C. lforeover, the rank of lex is l.
appears from this
It
s i o n o f t i r e s p a c e s i s p + q . t s r - rt h a t q n n c o ( i r i l ri c l r i t
'-- '-:-'-
that
the dinen-
i.s the dinension of the
^ - .I rl l 'L ^' n c c i t
L U l i ( d I l l > ,
, identical
d l l r l
is iJcntical r'ith C.
is establi shed that _ x _ x (3. I 3) fe + f = g = ef * €,
l'hus it
r'liich is
4.
thc
jdcntity
requirerl for
tlrc fol Ior'ing problen.
l Ghosh's Problen, G o s i . ri r ' a n t e c i t o (4.1)
fX(\'fL)
donljnates tllat qr\on
tl'o
tiitlr
forrcr
- X rilrcre lr = cf is
(4.3)
rnatrix
lllttrir
the
- 'r X ' f
qrlarlratic
But,
c.
= fc
+ f
fcrn'r'itlr
natrix
x + f
by u'hat has just
been
is
g = h + c,
(4.?)
neFative
tliat
llove
definite.
x synnctric It
i r le n ' p o t c n t , a n d t h e r e f o r e
fo1lorr,sthat,
t'g,t = trht
for
a1l
nnn-
t,
+ tret
.) ttet, sir.ce t'ht
C, and thjs
is
tlre reeuired result.
'l'lre p u r p o s e h ' a s t o s l r o r ' l' r i s f u 1 1 i n f o r n a t i o n t h r e e - s t a g c I e a s t 1 s q r r a r c s c s t i n ' a t o r ( 1 : i S S L S Jt o L e a s y n ' f t o t i c a l l y m o r e e f f i c j c n t t l i a n t h e Z e l l n e r - l ' h e i 1 3 S l S . I I t l L a r . r d2 S I - Se s t i n a t o r s .
_ 1 0 7 _
tern"s of
on a space in
the orthogonal projector
The formula e for
the elements i-n any base E was obtained in Afriat
(1957). Beside applications
l.ras
in regression tlieory, it
had use in progranning theory, in the gradient projection m e { i h o d ,p r e s e n t e d b y R o s e n ( 1 9 6 0 ) , a n d a l s o b y t h i s
rvriter
' in n n r i \ / 2 t e c n n r r n u n i c a t i o nt o L . l i l. L . B e a l e i n 1 9 5 8 . use, t{hlch has found econon'etric applications,
A further
is in the fornula trace ef,
(l!57), which provides a feneralization of the
in r\friat n , ' ru rr l tL i ,nt l, ,n L
the theory of which is developed
reornqqinn
ennf f ie ient
a n d r v l r i C l lc o i n C i d e s
with
d e a l t r v i t h b y I i o o p e r ( 1 9 5 9 ).
the statistic
A 1 1 t h e f o r m u l a e w h i " c hf e a t u r e i n t h i s
paper, but the
last which bears on Chosir's problem, are treated in the in the nicthod
paper of 1957, though there is a difference
0ther developnents are in the paper of 1969,
of argurnent
w h i c h I r r e s e n t s t h e e a r l y l a r t o f t h e u n p t r b I i s h e dp a p e r o f 'fhe l atter part of that paper sllowsthe bearing of 1960. tlrn
annroaclr
on
for
correlation
n:rti:r
I
7 ' e c f ' o s- -s i
- o - l -l
^-^1"';
d l l d r i / J l J
trvo groLrls of variables,
'^rical
t
L d r l u l l
and sore further
a n a l y s i s w h i c l ' ri n v o l r ' e s a n y n u r " b e r o f g r o u p s . A further
application
forned by a pair
is an analysis of the configtlration
of l:uclidean subspacesand deternrination of
thc canonical angles and directions. for canonical correlation
This Iias interpretation
theory and is presented in the
paper of 19-56. ine origin for
of tlris projector tlreory with interpretation
regression analysis is
i n n a r t i c r r la r o n t h e i l
in work on economic ti-meseriest
seasonal correction, donewith
P r o f e s s o r J . i l . . \ S t o n e a t t h e D e v e l o p m e n to f A p p l i e d E c o n o n i c s t z m h r rr e. l ii o F s ,
rn
t u\ 4.
BIBLIOGRAPHY the characteristic values of matrix functi-ons. Quart. J. Math. (0xford)(2) 2,81-84
AFRIAT, S.N. (1951): Boundsfor
AFRIAT, S.N. (1956): 0n the latent vectors and characteristic values of products of pairs of synmetric idenpotents. Quart. J. Math. (0xford) (2) ' 7'
76-78
A F R I A T , S . N . ( ' 1 9 5 7 ): O r t h o g o n a l a n d o b l i q u e p r o j e c t o r s the characteristics
of pairs
and
of vectors sPaces. Proc.
Canbridge Phi1. Soc. 53, 800-816 A F R I A T , S . N . ( 19 6 0 ) : S t a t i s t i c a l of orthogonal projectors
an investigation
projection:
in regressl'on analysis.
Research
Econonetric Research Progran, Princeton Univers.
Memo. 16,
Fox, K.A. and
AFRIAT, S.N. (1969): Regression and projection.
Narasimhan,G.V.L. ed. Econonic Models, Estimation and Risk Progranrning. Essays in Honor of Gerhard Tintner. Berlin,
Springer.
AITKEN,A.C. (1945): 0n least squares and linear
conbination
of observations. Proc. Royal Soc. Edinburghr SSr4? A N D E R S O NT ,. W . ( 1 9 4 8 ) : T h e a s y m p t o t i c d i s t r i b u t i o n determinantal 'l supp1. 10, 90
of certain
Analysis.
Royal Stat.
equations. Jour.
A N D E R S O NT ,. W . ( 1 9 5 8 ) : A n I n t r o d u c t i o n
of the roots
to Multivariate
Soc.
Statistical
New York. Wiley
A N D E R S O NT ,. W . a n d R U B I N , H . ( 1 9 4 9 ) : E s t i n a t i o n o f t h e p a r a m e t e r s of a single
equation in a complete systen of stochastic
equations. Ann. Math. Stat. B A R N E T TV , .
55, 650-659
(1973): Conparative Statistical
Inference.
New York.
W i1 e y B A R T L E T TM , .S. (1947): Multivariate
analysis.
Journ. Royal Stat.
S o c . s u p p 1 . 9 , 1 7 6 - 19 0 B A R T L E T TM , .S. (1948): A note on the statistical of
supply and demand functions
Econonetrica,
16 , 3 2 3
fron
deternination
time series.
_ r4 3 _
B A S M A N NR, . L .
(1957): A generalized classical
estination
linear
nethod of
in a structural
of coefficients
equation. Econonetrica, 25, 77-83 B O X , J . E . P . a n d C O X , D . R . ( 1 9 6 4 ): A n a n a l y s i s o f t r a n s f o r m a t i o n s . Journ. Royal Stat. Soc. B., 26,211-743 BOX, G.E.P. and TIA0, G. (1962): A further via
look at robustness
B a y e s t h e o r e n . B i o m a t r i k a , 4 9 , 4 19
BELLMANR , . (1960): Introduction
New York.
to lt'latrix Analysis.
C H R I S T ,C . F . ( 1 9 6 6 ) : E c o n o m e t r i c M o d e l s a n d M e t h o d s . N e w Y o r k . Wiley of quadrati-c forms in
C O C H M N ,W . G . ( 1 9 3 4 ) : T h e d i s t r i b u t i o n a nornal system, with
to analysis
applications
of variance.
P r o c , C a m b r i d g eP h i 1 . S o c . 3 O , 1 7 8 - 1 9 1. D E G R 0 0 T ,M . H . ( 1 9 7 0 ) ; 0 p t i n a l
Statistical
Decisions. NewYork.
N ' l c G r a wH i l l D H R Y M E SP, . J . D I X M I E R ,J .
(1970): Econometrics.
NewYork. Harper and Row
(1948): Position relative
de deux vari6t6s
fern6es dans un espace de Hilbert.
Rev. Sci.ent. 7, 387-399
FINETTI,B. de (1937): La pr6vision; ses lois sources subjectives.
Ann. Institut
F I S H E R , F . M . ( 1 9 6 6 ): T h e I d e n t i f i c a t i o n
lin6aires
logiques, ses
Henri Poincare 7, 1-68 Problem i-n Econornics.
New York. Ntc Graw Hill F I S K , P . R . ( 1 9 6 7 ): S t o c h a s t i c a l l y
D e p e n d e n tE q u a t i o n s . N e w Y o r k .
Hafner F L A N D E R SD, . A . ( 1 9 4 8 ) : A n g l e s b e t w e e n f l a t
subspacesof a real
n-dimensional Euclidean space. Studies and Essays Presented to R. Courant, New York F R A N K ,C . R . ( 1 9 7 1 ) : S t a t i s t i c s
and Econometrics. NewYork. Ho1t,
Rinehart and Winston G E A R Y ,R . C . ( 1 9 4 8 ) : S t u d i e s i n t h e r e l a t i o n s tine
series.
Journ. Royal Stat.
between economic
Soc. supp1. 10, 39
G E I S E R ,S . ( 1 9 6 5 ) : B a y e s i a n e s t i n a t i o n 'l 5 0 - 15 9 Ann. Math. Stat . 36,
in nultivariate
analysis.
-r4 4 -
G H O S HS,. K . ( 1 9 7 1 ) : F u t l
infornation
Waterloo Economic Series,
squares estimation. G O L D B E R G EA R.,S .
three stage least 27
( 1 9 6 4 ) : E c o n o r n e t r i c T h e o r y . N e wY o r k l t l i l e y
G O L L N I C K ,H . ( 1 9 6 8 ) : E i n f i i h r u n g i n d i e 0 k o n o n e t r i e . q r r r fr o n r t . I l 1 l n e r . ! .
e L s e ! 5 s r
G R U B E RJ, .
(1968): okonometrische Modelle des cowles connission
Typs. Hamburg. Parey H M V E L M O ,T .
(1944): The probability
approach to econornetrics.
Econometrica, suPP1. F i A D A M A RJD. ; ( 1 8 9 3 ) : R e s o l u t i o n d ' u n e q u e s t i o n 1 6 l a t i v e a u x d ' e t e r m i n a n t s . B u 1 1 . S c i . M a t h , ( 2 ) , 1 7, 2 4 0 - 2 4 6 H A R M A NH, . H . ( 1 9 6 S ) : M o d e r n F a c t o r A n a l y s i s . University of Chicago Press.
Dinensional Vector Spaces. Princeton.
H A L M O SP, . ( 1 9 4 8 ) : F i n i t e Princeton University
Press
f l A L M O SP , .R. (1951): Introduction-to theory of spectral
2nd ed. Chicago.
rnultiplicity.
Hilbert
space and the
New York
W ., E . ( 1 9 5 1 ) : L i n e a r T r a n s f o r m a t i - o n H ,. L . a n d G R I M S F L A M HAMBURGER in n-dinensional Vector Space. Canbri.dge H O C H S T A E D T EDR. , a n d U E B E , G . ( 1 9 7 0 ) : O k o n o m e t r i s c h e M e t h o d e n . Berlin.
Springer
H O L Z I N G E RK, . J . a n d H A R M A NH, . H . ( 1 9 4 1 ) : F a c t o r A n a l y s i s . Chicago. University
of Chicago Press.
S ., C . ( 1 9 5 3 ) : S t u d i e s i n E c o n o n e t r i c H O O D ,W . C . a n d K O O P M A N T Methods. New York. WileY H O O P E RJ, . W . ( 1 9 5 9 ) : S i n u l t a n e o u s e q u a t i o n s a n d c a n o n i c a l correlation theory. Econometrica, 20, 245-256. H O R S T ,P . ( 1 9 6 5 ) : F a c t o r A n a l y s i s o f D a t a M a t r i c e s . N e w Y o r k . Ho1t, Rinehart and Winston. H O E F F D I N GW, . ( 1 9 6 8 ) : W a l d M e n o r i a l L e c t u r e s . H O T E L L I N GH, . ( 1 9 3 1 ) : T h e g e n e r a l i s a t i o n A n n . M a t h . S t a t . 2 , 3 6 0 - 3 7 8.
of Studentrs ratio.
-1 4 5 -
H O T E L L I N GH, . ( 1 9 3 3 ) : A n a l y s i s o f a c o m p l e x o f s t a t i s t i c a l into
variables
principal
conponents. Journ. Educational
P s y c h o l o g y , 2 4 , 4 1 7 - 4 41 , 4 9 8 - 5 2 0 . H O T E L L I N GH, . ( 1 9 3 5 ) : R e l a t i - o n s b e t w e e n t w o s e t s o f v a r i - a t e s . B i o m e t r i k a , 2 8 , 3 2 1 - 3 7 7. J E F F R E Y SH , . (1961): Theory of Probability.
3rd ed. 0xford.
Clarendon. J O H N S T O NJ ,.
( 1 9 6 3 ) : E c o n o m e t r i c M e t h o d s . N e w Y o r k . M c G r a wI I i 1 1
J O R D A NC, . ( 1 8 7 5 , 1 8 7 7 ) : E s s a i s u r 1 a g 6 o m 6 t r i . ed n - d i m e n s i o n s . B u 1 1 . S o c . M a t h . P a r l s , 3 , 1 0 3 - 1 74 , 4 , 9 2 . JULIA, G. (1944): Sur 1es projections de lr6space Hilbertien.
C.R. Ac. Sci. 218, 892
K E N D A L LM , . G . a n d S T U A R TA , . Statistics.
( 1 9 6 6 ) : T h e A d v a n c e dT h e o r y o f
3 NewYork. Hafner
A ,. M . ( 1 9 7 2 ): M u l t i v a r i a t e KSHIRSAGAR KLEIN, L.R. Cliffs,
des systdmesorthonormaux
( 1 9 6 2 ): A n I n t r o d u c t i o n
Analysis.
NewYork. Dekker.
to Econometrics. Englewood
N.J. Prentice Ha11.
K L 0 E K , T . a n d M E N N E SL, . B . M . ( 1 9 6 0 ) : S i m u l t a n e o u s e q u a t i o n s based on principal
c o m p o n e n t so f p r e d e t e r m i n e d v a r i a b l e s .
E c o n o m e t r i c a ,2 8 , 4 5 - 6 1 . I O 4 E N T AJ, .
(1971): Elements of Econometrics. New York. lt'[acmillan
AV . N, . ( 1 9 4 6 , 1 9 4 7 ) : 0 n t h e p r o o f o f t h e r n e t h o d o f KOLMOGORO least squares. lJspekiN{atemN . auk (N.S.)(11) 1, 57-7O,8,523-533. K O O P M N ST , .C. (1937): Linear Regression Analysis in Econonic Tine Series. KOOPMANS T ,. C . e d .
lJaarlern ( 19 5 0 ) : S t a t i s t i c a l
Analysis
in Dynamic
E c o n o n i c ] t ' l o d e 1 s .N e w Y o r k . W i 1 e y . K O N I J N ,H . S . ( 1 9 6 2 ) : I d e n t i f i c a t i o n equations model with errors ?n
1O AfTlet:
70-Q7
Stutllee
and estimation in a simultaneous
in the variables.
Econonetrica,
- 1 4 6 -
K U L L B A C KS, . ( 1 9 5 9 ) : I n f o r n a t i o n
Theory and Statistics.
NewYork. Wi1ey. L E S E R ,C . E . V . ( 1 9 6 6 ) : E c o n o m e t r i . cT e c h n i q u e s a n d P r o b l e m s . New York, Hafner. L I N D L E Y ,D . V . ( 1 9 6 5 ) : I n t r o d u c t i o n t o P r o b a b i l i t y Canbridge University
and statistics.
Press.
L I N D L E Y ,D . V . a n d S M I T H , A . F . M . ( 1 9 7 2 ) : B a y e s e s t i n a t e s f o r the linear rnodel. Journal Royal Stat. Soc. 34, 1-41. (1964): Schdtzprobleme der Okonometrie. Wiirzburg.
LUEDEKED , . Physica
P ,. C . ( 1 9 3 0 ) : 0 n t e s t s a n d n e a s u r e s o f g r o u p MAHALANOBIS divergence. I. Journ. Asiatic Soc. of Bengal, 26, 541. P ,. C . ( 1 9 3 6 ) : 0 n t h e g e n e r a l l z e d d i s t a n c e i n MAHALANOBIS statistics. Proc. Nat. Institute of Scienceof India,1Z,49-55. M A L I N V A U DE, .
(1970): Statistical
Methods of Econometrics.
2nd ed. Ansterdan. North Ho11and. M A N N ,H . B . ( 1 9 6 0 ) : T h e a l g e b r a o f t h e l i n e a r ',l Ann. I4ath. Stat. 31, -15. M A R S C H A KJ,.
hypothesis.
(1955): Econonic Measurenent of policy
prediction.
and
t l o o d , W . C . a n d K o o p n a n s ,T . J . e d . S t u d i e s
in Econometric Methods. NewYork. Wi1ey. M E N G E SG , . ( 1 9 6 1 ): O k o n o m e t r i e . W i e s b a d e n . B e t r i e b s w i r t s c h a f t licher M I R S K Y ,L .
Verlag. (1955): Introduction
M O R R I S O ND, . ( 1 9 6 7 ) : M u l t i v a r i a t e
to Linear Algebra. Oxford. Statistical
Methods. NewYork.
Mc Graw Hill. M O S B A E KE, . J . a n d W O L D ,H . ( 1 9 7 0 ) : I n t e r d e p e n d e n t S y s t e m s . Arnsterdan. North Ho11and. N E Y M A N J, .
(1962): Two breakthroughs i.n the theory of statistical
decision rnaking. Rev. Inst.
of
International
Stat.
30, 11-27,
P E N R O S ER, . ( 1 9 5 5 ) ; A g e n e r a l i z e d i - n v e r s e f o r m a t r i c e s . P r o c . C a m b r i d g eP h i 1 . S o c . 5 1 , 4 0 6 - 4 13 . P I L L A I , K . C .S . ( i 9 6 0 ) : M u l t i v a r i a t e
hypotheses. The Statisti.cal
Center, Mani"1a,Universi-ty of the Phillipines
.
_r47 _
PILLAI, K.C.S. (1967): Upper percentage points of ltrgest analysis.
roots of a matrix in multivariate
characteristic
B i o m e t r i k a , 5 4 , 1 8 9 - 19 4 . P R E S S ,S . J .
( 19 7 2 ) : A p p l i e d M u l t i v a r i a t e
Analysis.
NewYork.
Ho1t, Rinehart and Winston. R A I F F A , H . a n d S C H L A I F E RR, . ( 1 9 6 1) : A p p l i e d S t a t i s t i c a l Press.
Decision Theory. Canbridge Mass. Havard University
Methods in Biological
RAO, C.R. (1952): AdvancedStatistical Research. New York. Wi.1ey.
Inference and its
R r \ O ,C . R . ( 1 9 6 5 ) : L i n e a r S t a t i s t i c a l
Application.
NewYork, Wi1ey. R A O ,C . R . ( 1 9 7 0 ) : E s t i m a t i o n o f h e t e r o s c e d a s t i c v a r i a n c e s i n l i n e a r m o d e l s . J o u r n . A r n e r i c a nS t a t . A s s o c . 6 5 , 1 6 1 - 1 7 2 , R A O , C . R . ( 19 7 2 ) : R e c e n t t r e n d s i n r e s e a r c h i n n u l t i v a r i a t e analysis. Biometrics, 28, 3-22. RAO, R. Ranga (1962): Weak and uniform convergence neasures. Ann. lvlath. Stat.
33, 659-680.
R O B B I N S ,H . ( 1 9 5 5 ) : A n e m p i r i c a l B a y e s a p p r o a c h t o s t a t i s t i c s . Proc. Third Berkeley Synp. Prob. and ltiath. Stat.1,
157-164.
R O Y ,R . ( 1 9 7 0 ) : E l e m e n t sd ' E c o n o n e t r i e . P a r i s . P r e s s e s Universitaires. R O Y , S . N . ( i 9 5 7 ) : S o m eA s p e c t s o f l r 4 u l t i v a r i a t e A n a l y s i s .
l'r-ew
York, Wl1ey. S A S T R YM , .V.R. (1969): Multicollinearity elasticities.
C a n a d i a nJ o u r n . A g r i . c . E c o n . 1 7 ' 5 0 - 6 0 .
S A S T R YM , . V . R . ( 1 9 7 0 ) : S o m el i m i t s collinearlty. S A V A G E L, . J .
and consuner dernand
in the theory of multi-
The AmericanStatistician,
39-40.
(1954): The Foundations of Statj-stics.
NewYork,
Wi1ey. S C H N E E W E IH B ., ( 1 9 7 1 ) : 0 k o n o m e t r i e . W t i r z b u r g . P h y s i c a . P ,. ( 1 9 6 9 , 1 9 7 1 ) : M e t h o d e n d e r O k o n o m e t r i e . B e r l i n . SCHONFELD Vahlen.
-1 4 8 -
S C H E F F EH, . ( 1 9 5 9 ) : A n a l y s l s o f V a r i a n c e . N e w Y o r k . W i 1 e y . S C H O U T EP, . H . ( 1 9 0 5 ) : 0 n t h e e q u a t i o n s d e t e r m i n i n g t h e a n g l e s of two polydi-nensional spaces. Proc. Sci. K. Akad. Wet. 7 , 3 4 0 - 3 4 1, S O M E R V I L L ED,. M . Y . ( 1 9 2 9 ) : A n I n t r o d u c t i o n t o t h e G e o n e t r y o f N D i m e n si o n s . L o n d o n . S T O N E ,R . ( 1 9 4 7 ) : 0 n t h e i n t e r d e p e n d e n c e o f b l o c k s o f t r a n s a c t i o n s . J o u r n . R o y . S t a t . S o c . s u p p l . 9 , 1- 3 2 . S T O N E ,R . ( 1 9 5 4 ) : T h e M e a s u r e m e n t o f C o n s u r n e r ' s E x p e n d i t u r e a n d B e h a v i o u r i n t h e U n i t e d K i n g d o n , 1 9 2 0 -1 9 3 8 . C a n b r i d g e . Cambridge University
Press.
THEIL, H. (1961): Econonic Forecasts and Policy. Ansterdan. North Ho11and. T H E I L , H . ( 1 9 6 7 ) : E c o n o m i c sa n d I n f o r n a t i o n
Theory. Chicago.
Rand McNa11y. T t l E I L , F I.
( 1 9 7 1 ): P r i n c i p l e s
o f E c o n o r n e t r i c s .A m s t e r d a m .
North Ho11and. T I N T N E R ,G . ( 1 9 4 5 ) : A n o t e o n r a n k , m u l t i c o l l i n e a r i t y nultiple
regression. Ann. Math. Stat.
T I N T N E R ,G . ( 1 9 5 0 ) : S o n e f o r m a l r e l a t i n s
and
16, 304-308.
in multivari,ate
analysis. Journ. Royal Stat. Soc. B. 12, 95-101. T I N T N E R ,G . ( 1 9 5 2 ) : E c o n o m e t r i c s . N e w Y o r k . W i 1 e y . T I N T N E R ,G . ( 1 9 6 0 ) : A p p l i c a t i o n
of the theory of information
to the problem of weighted regression. die Corrado Gini,
Studi in Onore
Rone. Universitd degli
Studi.
T I N T N E R ,G . ( 1 9 6 2 ): E i n T e s t f i i r d i e S i n g u l a r i t a t
einer Matrix.
IFO Studien, 8, 1-14. T I N T N E R ,G . ( 1 9 6 3 ) : A n o t e o n t h e r e l a t i o n
betweenMahalanobis
dj-stance and weighted regression. Rao, C.R. ed. Contributions to Statistics.
Calcutta. Statistical
Publication Society. 481-484.
T I N T N E R ,G . a n d K A D E K O D IG , .(1971): Note on the transfornation variables
in simultaneous equations systens. Journ.
Soc. Agric.
Stat. 23, 164-17?.
Indian
of
-1 4 9 T I N T N E R ,G . a n d S A S T R Y M , .V.R. (1969): Infornation the statistical Multivariate TORRENS I B- E R N , J . factorielle. W A L T E R SA, . A .
estination
Analysis,
theory and
of economicrelations.
2, New York. Acadenic
Press.
( 1 9 72 ) : M o d 6 l e s e t n 6 t h o d e s d e 1 ' a n a t y s e Paris, Dunod.
(1970): An Introduction
to Econometrics. NewYork.
Norton. W I L K S ,S . S . ( 1 9 3 2 ) : C e r t a i n g e n e r a l i s a t i o n s i n t h e a n a l y s i s o f v a r i a n c e . B i o n e t r i k a , 2 4 , 4 7 1 - 4 9 4. W I L K S ,S . S . ( 1 9 4 3 ) : M a t h e m a t i c a l S t a t i s t i c s .
Princeton.
W I L K S , S . S . ( 1 9 6 2 ): M a t h e m a t i c a l S t a t i s t i . c s . N e w Y o r k . W i 1 e y . W O L D ,H . a n d J U R E E N ,L . WOLFOWITZ J .,
( 1 9 5 3 ) : D e m a n dA n a l y s i s .
NewYork. Wi1ey.
(1954): Generalisation of the theorem of Glivenko
C a n t e 1 1 i . A n n . N { a t h ,S t a t . 2 5 , 1 3 1 - 1 3 8 . Z A C K S ,S . ( 1 9 7 1 ) : T h e T h e o r y o f S t a t i s t i c a l
Inference. NewYork.
Wi1ey. Z A R E M B K AP,. ( 1 9 6 8 ) : T h e f u n c t i o n a l
f o r m o f t h e d e m a n df o r m o n e y .
Journ. Anerican Stat. Assoc. 63, 508-511. Z E L L N E RA , .
(1971): An Introduction
Econometrics. NewYork. Wi1ey.
to Bayesian Inference in
Extract from the review by Karl A. Fox in the Journal of the American Statistical Association 71, 355 (September 1976), 769-70. … Afriat’s essay is entitled “Regression and Projection: Studies in thc Algebra and Geometry of Statistical Correlation.” Portions of it originated in his attempt to make concise the algebra which appeared in his invstigation of methods for thc seasonal correction of economic time series; this work was done during 1953-56 while Afriat was a member of the Department of Applied Economics at Cambridge University. Afriat’s first love was mathematics, and two sections of his essay first appeared in the Quarterly Journal of Mathemalics (Oxford, 1956) and the Proceedings of the Cambridge Philosophical Society (1057). He drew directly on advanced articles by pure mathematicians (Jordan, Hadamard, Halmos, Dixmeier, Flanders, Julia, Kolmogorov, Schoute, Somerville) as well as mathematical statisticians (most notably Hotelling), whom few economists in the 1950’s were equipped to read. As of 1976, most econometricians can and should read Afriat’s essay. The editors credit Kolmogorov in 1946 with the conceptualization of least-squares linear regression as resolving sample vectors of dependent variables into two orthogonal conmponents, a regressional component lying in a space spanned by sample vectors of the independent variables, and a residual component lying in the orthogonal complcment of that space. Afriat goes beyond this to consider the linear transformations which obtain among these orthogonal componcnts of sample vectors, these being the complementnry pair of symmetric idcmpotents which are the orthogonal projectors on the space and its orthogonal complement, and to ana1yze relations between variables directly in terms of these projectors. Thc view taken of regression analysis is one that does not depend on distribution concepts. Orthogonnl projection is presented as the fundamental principle and the least-squares principle is derived as a property of it. Afriat’s approach provides alternative views or derivations of Hotelling’s canonical correlntion theory and coefficients of correlation and alienation defined by Wilks and Hotelling. One of Afriat’s formulas provides a generalization of thc multiple correlartion coefficient independently derived (from a different perspective using different mathematical tools) by Hooper. The beauty of Afriat’s algebraic and geometric derivations is that so many of the different techniques used by econometricians fall out as special cases of more general formulations. …