Studies in Correlation - Multivariate Analysis and Econometrics

Studnes ffiil by SydneyAfriat M.V.RamaSastry GerhardTintner

Vandenhoeck & Ruprecht

Studiesin Correlation Multivariate Analysisand Econometrics

by SydneyAfriat, M. V. Rama Sastry and GerhardTintner

Vandenhoeck& Ruprechtin Gcittingen

Contents

lntroduction. Sydney Afriat Regression and Projection.

Studies in the Algebra of 10

Statistical Correlation. M. V, Rama Sastry and Gerhard Tinher Multivariate

Bibtiography

Analysis.

Econometrics

and Infonnation

Theory

' ' ' ' ' 108

"

' 142

Abstract

The first part gives a geometric and algebraic background to general methods of regression and correlation,

taking into account particulary methods of

multivariate analysis. In the second part Canonical correlation, root criterion, of information

largest

principal components, weighted regression, the relation theory to econometric

estimation and to canonical correlation

and trace correlation is considered. Bayes estimation or regression models is also briefly discussed.

Introduc t ion

s c h e m et o s h o w l e a s t - s q u a r e s l i n e a r regression as resolving sanple vectols of dependent variables i-nto two orthogonal components, a regressional component It

is a familiar

in a space spanned by sample vectors of the i . n d e p e n d e n tv a r i a b l e s , a n d a r e s i d u a l c o n p o n e n t l y i n g lying

in

t h e o r t h o g o n a l c o m p l e m e n t o f t h a t s p a c e . T h i - s s c h e m ew a s i n t r o d u c e d b y K o l m o g o r o v ( 1 9 4 6 )' B u t t h e probably first step from this

to considering

the linear

transformations

these athogonal conponents of sample vectors, these being the complementary pair of symmetri-cidempotents which are the orthogonal projectors on the space and its orthogonal conplement, and then to analysis of relations

which obtain

between variables directly in terms of these projectols' is one that does not appear to have been explored' The advantage of this step apart fron its showing the way to new concepts is that it makes an enlargement methods j.n a natural formalism for handling sone

some possibly of didactic farniliar

theory vrith a direct

geonetrical

neaning, and it

di-sp1ay of

its

algebraical-

gives the framework for

simple

proofs and formulae. Proofs of some relevant algebraical propositions have already been gi-ven elsewhere (Afriat' 1957 and 1956). The view taken of regressl-onanalysis is concepts. orthogonal one that does not depend on distribution projection is presented as the fundamental princi-p1e and the least-squares principle is derived as a property of it. The association of a di-stribution with a sanple is a refinements of this view, since and it can also be considered to justify it'But this vlew can just as well be taken as primitive, in its compatibility with it the distribution nethod can be taken

method that yields

certain

The only to have a crj-tical part of its own justification. that can be given to the probability claim to priority approach ls that it is part of a more general distribution artifice

for

the interpretation

o f d a t a . W h e nr e f e r e n c e

-

is made to a variate,variance, which terns properly distribution,

all

d

-

covariance or correlation,

belong to concepts relating

that is neant is

is determined on objects

to a

a variable whose value

and certain

functj-ons of value

which, when a variable

l-s interpreted

sanple as a population

sample, can also be lnterpreted

as a variate

and a

as sanple neasures which, in the case of normality, respond in the usual way to those things Hotelling

(1935) has given an analysis of the relation

between two sets of variates theory.

cor-

in the population.

in his

c a n o n i - c a 1c o r r e l a t i o n

It

can be fornulated as an analysis of the relative of two sub-spaces of a Euclidean space, spanned b y v e c t o r s o f r n e a s u r e m e n t s ,a s h e h a s p o i n t e d o u t . T h i s position

analysis

provides a cornplete set of orthogonal invariants

which characterize the figure

formed by the spaces, in terms of a set of angles determined between them. Such a characteri-zation of a pair of spaces by angles was demonstrated by Jordan (1875), using synthetic methods; and a further

such account has been given by Somerville (j929). Algebrai-ca1 method for determination of the angles has been investigated by Schoute (1905), and more recently by Flanders (1948). Hotelling's

theory shows another

approach, equivalent to consideration of directions whose variation in the spaces leaves the angle between thern stationary,

but this

a p p r o a , c hd o e s n o t a c c o u n t f o r

the sub-space in each space which are orthogonal to the other space. A further rnethod, shownby Afriat (1956 and ,l957), j.s in accordance with the nethod of the present investigation, and proceeds entirely values and latent

by a consideration of the characteristic

vectors of the pair

orthogonal projectors

of products of the

on the spaces.

l v h i1. e a c o m p l e t e s e t o f c a n o n i c a l a n g l e s , o r t h e i r

- 9 -

cosines which are the canonical correlations, spaces, or the sets of variates,

there are coefficients

that express sumnaryaspects of this

relation.

which have such a nature,

and alienation

identical

and which can be seen to be algebraically the here-defined coefficients one space with projectors

another. Presented in terms of the orthogonal

algebraical properties properties

are immediately clear.

for

any numberof variates

coefficient

a set of several, this the correlation

itself

coeffi-cient

(1957). It it

and

being a genera)izationof

of Pearson defined between a have been derived

coefficient

h a s h a d s u b s e q u e n tc o n s i d e r a t i o n ,

though with a different has applied

of the multiple

defined between one variate

T" ^h" c n r ronerties of this

by Afriat

S o m en e w

represents a

shown. One of these coefficients

correlation

n:ir^

geornetrical rneaning and

kind are here introduced, and their

of this

generalizatlon

with

and separation of

of inclusion

on the spaces, their

coefficients

l{i1ks

(1935) have defined coefficients

(1932) and Hotelling of correlation

between the

of the relation

complete specification

glves a

derivation,

by Hooper (1959), who

to the sirnultaneous equation method in

econonetrics. A multivariate generaTization of the concept oI variance which has been studied by Wilks (1932) and Anderson (1958) enables an alternative the method of least

characterization

to be gi-ven for

squares. A formula is

shown vihich

expresses the variance of a set of variates the variances for

conplementary subsets, together with

here-defined coefficient Geometrically, this Formula for

in terms of the

of separation between them.

is a generalization

of the familiar

the area of a parallelogram, in terms of the

lengths of a pair

of edges and the sine of the angle between

them. A multidirnensional generalization

is shownalso for

the formula which gives the area as the product of base with height.

RECRESSIA ON N DP R O J E C T I O N Studies

tlie Algebra

in

and Ceometry

Correlati.on

of Statistical

by Sydney AFRIAT Part I Prediction Let U be a unlverse of objects, and 1et Y be a set of i n t h e r e a l n u n b e r f i e l d K r r ' h o s ev a l u e s c a n

variables

Then ll] K can denote

be determincd on anv ohject in U.

V determined on anv object

the value of an1'variable X a

LI.

,\ sanple of size N consists in a set of N objects taken .l , . . . , N. .,\nexperi-ment deterrnines the values of variables in V on every object in f r o r n I J , r n , hci h c a n b e d e n o t e d

the sample. is

In particular if

characteristic

there is one variable u that identified

the sample is

with the

u n i f o r m c o n di t i o n o f t h e e x p e r i m e n t , t h a t t a k e s t h e v a l u e I on everv ohject, is

The samolesnacc for the exDerintent the Iruclidean space E = K" , of dimension equal to the

samolesize. for

Tlie sample vector r"hose elenents tlre

of

\

n v uhJ i c we !t s

11] (i

).

in

[]19

a variahle

=1,

.. ., ^) d ^l l *t P- rl U .^

t h e u n i f o r n r c o . n d it i o n

A multivarialrlc its

basic

is

components.

are

defined

the

n-t'ector

l\lael

N) are the rralues of x on rr r"r . - ^ - +L ii L cu rra ra l a r tL hr r cv rsa nr . nr Pna l\ e V e c t o I I ' d t

variable

the n-vector whose elenents

x is

u is all

ll

u

equal

by an1' set

Thr.rsa p-l'ariable

= I lvhere I to

of

is

1.

variables,

can be denoted

ca11ed

-1 1 *p), so a l-variable is then just one

(x1, ..., in V.

variable

sanple natrix

A 1r-variable has associated with it I . 1 -= ( I 1 - , . . . , I l - ) o f o r d e r N * 1 , ^r

1

p conponents.

w h o s ep c o l u m n sa r e t h e l s a m p l " u " 8 t o r u f o r i t s comnonents -

n

Thesp

a

\rcctnrq

qnzn a H " .

s. r : h " - q- n a' c e"

F X

.i Il

the

be called the sample span of x.

sample space E whlch will Thus

n * , =h r r ) = l r r * r ,. . . , r ' * . , , w h er e = {Nl"c: ctKP J,

trl*

again,

NIx., ..., "1

NI*"t

= {ltl*,c, + "1

C i v e n a r n u lt i v a r i a b l e

x say with

p t q-matrix

c c Knl ,

+ Nl"-co:c1r .'f r

,..

p conponents,

.p.K}

and a

ivhich can be tlenotecl

a nul tivariable

Y

y = x c , i s d e f i n e d r v i t h q c o m p o n e n t sd e n o t e d

'Y J; eK are

where c;;

I.J

of

' lc p' Lt Co' lr l nr r i l nl r oq u, u

X C ; =

) X i C ,

J

r

I

r J

and ci

the elements,

eKP are the

colunns,

J

Its

c.

=

value

l

n r nr u

da r nr j w l

is

determined rvhen tlie value

nhi ec f

: n d r rd \ r

( ,

ir n r r

in the s,rmnle. its

the ohiects

from the sannle rnatrix of

rI ' ^d -r + L; r^\ "r r1r (^r

x

of

is

l ;cd

r

to

t

s:mnle metrix

is

determined

x hv the relation

trf*. = M*., that is, trl*.. = Il*ci' ) "

and again

l" \xf l c l i * . . . * * p . p j = '\ *. l -eI -) 'I ,"t , * a ,

'

5lnce

fl)en

g-

.I .l X

autornaticalll Lr 9t*; contained ,y

-

IL

Special

Ldll

in L^ L,g

thc

,

r- .i

that

t follows

exanlples are

e a

of

lr .r i r nr !e cer T

tl-rebasic

that

x.

1,...,Q),

y = xc then

it

tlre sanrplcslan of

is,

sample slan

^^l l d l r L lup r ]

=

+...+ Mx cni(j p,,

y

is

Any such variablc

a npnf Ln Um l , nl n ,urtgrtL

n u fI

v

t

n u rr

x-ccnrporrentsxj.

2 lhr o

i\ ' - . r^vml ln' ^l /nvar nl Lf r l L .

BLlt ncw 1et mntri ces U

^

x,

be any rnultivariables

I

l" r\r 1 -i n

t

the exnerinent.

t

rolrfinn

y.

=

v

hvnothcsis

of

a

avnarimenr

i c

r ' l r 1 t I i - . = l' rxl -C - .""--

to

with

sample

The condition lro rdmittorl

for

hv

the

t[e

But thC cOndition that this y that E--:E... Let e-, e,, denote the y r(' y x orthogonal projectors on Ex,Ey. Then, since exX = X = if and only if X Ex, this condition is or "*"y "y, couivalentlv e ll = Il y x y

holds for somec is

T H I O R I I i I] :

I I x , y b e r n u tl i v a r i a b l e s

N I - , 1 . 1 .j.n

an expcrincnt,

nrniectors

on

conclition

that

cxperimcnt

If

is

their

and if

= ev,

"*"y

e., are thc

I -

orthogonal

E-, E.,, then the

y = xc be adnissible

a relation that

ev,

qnrnq

c eu r m n lr p r r l '

r

w i t h s a n p le m a t r i c c s

or

i.n the c * 1 r 1 .=.

equiValently

s u c l r a r c l a t j o n l r o l d s , y c a n l . e s a - i d t o l - ' el r c d j c t a h l c

fronrx, and any nultivariblc

x c r t ' h o s ci d e n t j t y w i t h y i s e x p e r i n ) e n t al yl a d n i s s i b l e d c f i n e s a n x - l r c d i c t o r o f y . A multivarialrle mcnt, i f

no one of

x Iii ll its

be called

regular,

conponcnts is

in

the e.rperi-

prcJictrble

fronr

the otlrers- otlrerr.'ico flrn .^mn^nantS aIe nrrltiCOllinear. An identical

condition

of

regularity

is

that

no linear

rclation

b e t w e c l ) t l r e c o n r p o r r c n t sl , . r ee x p e r i m c n t a l l y

wlrich is

to

sa y no linear

conponent sanple sanple

matrjx is

p.

In

L . r ea t

least

the

T l l l 0 l l l ; l r 12 :

vectors,

llr,

span,

relation

equal

exists

equivalentlyr, to

the

nunber

p o{

The condition

tlre rank

dirnension of

thj.s case certainly

the

the

of

the

sanple

s a n r p J , es i z e

conutonents in

that

admissible,

betrr'een the

N must

x.

a multivariable

x witlr

s a n p l c m a t r i x l t l . ,l r-o r e o t t l a r i s t l r r t t h n m n t r i v \ r t \ l - . b c " ' X ' r x regular, in which case the orthogonal projector on the s a n r p le s p a n I l - . i s

e 'x

T'hus, for

all

t

e KP,

= \1 "x

t1"tx1" 'x1/ 1 - 1"1x "

_ 13 _ I r 1t = 0 = ) l \ 'l l ; = 0 x x x and .I ,\rx1l. \. x' -t = o J t r .l , lx' , \,txi =. o vr \ \ :, , xr .)/ r r, .l .i _t \) . =, 0 + 1, ,r . 1xt -= 0 . T h a t i s .' l t l t = 0x ( + I I l N I t x= O . w h i cr h s h o w s xt l r a t I l . . a n J I r 'l l t l * h a v e t h e x

1s regtrlar,

r-2 q e

i t

i c

sane rank, i-f and only

f o r r n uI a f o r

thc

llillx

r:.atriX cx,

tlrc orthogonaJ

is

of

i , l c n r n n t n n t

t l r r l

'x is

range,

whi ch

pro j ector

on Lx ,

It

is

consitlcrc.l is

rieJI -

s 1 ' r n n ; e t r i ca n d

=

identical

say n.

It

- x ' o' x2 a

=

rr'ith the remains

to

'x' o

ortlrogonal shoiv that

projector

olr

Il.= L*.

Tiius, if Xe R, then X = Y for since Ye " *" ) = = e*Y = X, and hence X = il* (llillx) "*\ ";Y lr'hich shornsX e Iir. Thus Rq lrx. Conversely,

l, so th:rt 1' I l x \ ,

X e L*, = extrlxt = Irl*t = X,

X = Irl*t for s ince

in

case

l S t

A l

if

some t. KP, so that "*X = c*NI* N I * , r v i ' r ci h s h o w s X e R , l h u s I ; * ! . I i ,

tr* = R,

which

in

rank p,

p.ivcn I'y thc

j-nverse exists.

definett sir)ce the

its

if

p,

rank

i.s of

re c r r l ql .

Tn thi s casc

IIence, it

so llx

then

and hence

as required.

Another

artunrent

idcmpotents,

trace

is

as fol lorts.

e* = rank e*.

lly tcncral

propclty

of

But

t r a c e e x = t r a c e N l x( I l i l l x )

- llii

= t r a c e } " i ft ' i * ( 1 1 i l l x ) 1 = trace I

p

Ilence rank e* = p. But c1earl-y R - E*, anclsince now also d j - mR = d i m l r , i t f o l l o w s t l . r a t R = l * .

y is predictable

T H E O R E 3M: I f

from x and x is regular then

of y is uniquely xc*r,where

the x-predictor

1

(MiMx)'MiMy.

.*y

' ' .y- ', w" "h 'e' r*o a- x i c B y T h e o r e r n 1 ,' e ' x. ."My. . = M

tha

nrrh4g6n3l

p r o j e c t o r o n t h e s a r n p l es p a n o f x , a n d b y T h e o r e m2 , e

x

'Ilr . x

= I'1fM'M I x' x x'

^h u- L- v^ r- sA r; r- r '^ r 1/ . , ' ,- [(\4:\r-) x L ' x x ' that

is

anv n !r

"

-xy

r h a n r { - ' . M . . c= I l . ' . M . . ,h e n c e , "y',y, """

rocrrlapair

the

by

if

this

regularity,

holds again

Regressi-on Now consider

x,

= M.,, y '

tJt.,] x y ,

\ 1 _ - c= M . . w h e r e c = c - , , . C o n v e r s e l y , x Y X Y

fny

2.

- t

of

t-t

I

x

any two nultivariables 1' -

=

residual

idenpotents

-s "o ' xc '

projectors,

orthogonal

and its

'e x t

orthogonal span of

e e X

complementarity

= O' . e

X

* 6

X

X

and assume x

sample span E* of

rnich

a pair

They are

y

are a cOmolementafy

onto the

complement i* x.

wj.th the

"

x,

of

can he called synmetric

properties

= 1.

Consider the resolution M

" Y= of the sample natrix

a

t r , t "x 'Y

of y,

the orthogonal projection samplespan of x and its

+

l

V

"x Y'

into

componentsobtained by

of sample vectors onto the orthogonal complement.

is

- 1 5 -

If

vI

"

i sY n r e d i c t a h l e e

In

fhis

case.

as

X

then

from x

My = My ' ex \ {y = 0 . been remarked there

has

c e KP

exists

Y

Such that

M.. = M-

an X-predictor

y.

of

r c '

Y

Tt

en.l then ve is

u r r u

i s a nred ictot

ll il n

tL hl l eE

^i ',i.. Sense O urf B lvrllts )EllJE

M r ' ta x LcJ' ) r ' r? y -= ,M ' ,d x c t f- = M

as the value of y in any ohject a, determinedfrom the value U3 of x on that object. The predictions are exactly x confirmed in the experiment for every object in the sample. to assume

That heing the case, there is no prohibi tion it

as universal RU r r Lf D

nr

for obiects in U.

nanarrllv SsrlqLat ) /

n ur ir vv rq l ^evn. f. lLvr l vF Y

it

Will =

-e x "ey

nOt

fg

thc

I t

i s

nossi hl e.

Q \ y-. - .

r-ese

-e x , \, {y

t. h, ,a- t. for

=

vt t y ,

i nstance,

that e*Nl,= 0, or equivalentlY e*e, = O. Since then also e.,e- = 0, this f

d e f i n e s a s y m m e t r i c a l -r e l a t i o n

between x,

^

y by which

they

are

said

considered resolution

to

For the

be uncorrelated.

in this

case

e . . N ' 1=. . 0 , X Y

e . . V , , = l t , ,. X Y Y

These are two extreme cases that have been considered. ts a set of vectors in "*ty E - x- . = .[ M - . ], t h e r e a l w a y s e x i s t s a m a t r i x c . Kql s u c h t h a t x., But generally, since always

e*\1, = M*c . If

x i.s regular then c = .*y uniquely. 0therwise, there is

an infinity

of such matrices, of the forn c + t,

^-J + ;r )a n n r t i c r r l : rr o v rn r ! er dllu L variety

dlrl

where c is

^lution of V.-t = O, the

Jw

of thern corresponding to the variety

X

of

adnissible

_ 10 _ linear relations between the conponentsof x. Noweven when y is not exactly predictable

fron x,

a

= xc can be generally introduced with the

variable f(x)

role of an x-predictor of y.

It

defines the regression

of y upon x, with c as regressionmatrix, and it

coincides

with the x-predictor of y in case y is predictable from x. rf

ic

rrninilo

*'ith

=

c

-c x- .y, -' ,

in

case

x

is

It

regular.

is

are uncorrelated. For any object a,

nu11 in case x,y

= N{3c NIl ^ r(x) c a n d e f i n e a n e x p e c t e dv a l u e l , f l o f y , g i v e n t h e v a l u e Ml of x. ,n" X

,rr""

""tr**

oy Y, = y - i t * l d e f i n e s t h e

r e g r c s s i o n r e s i d u a l , o r u n e x p e c t e dp a r t o f y r e l a t i v e t o x.

I n h a r m o n yw l t h t h i s

vatue i(*) with x.

it

is

seen that the expected

of z ts identically

zero, or z is uncorrelated

For = e NI^, . = e M = e - x "My - y " x "ex l r' ly = O . ztxJ x z (x)

Thus the consi-dered resolution N{

--

v

can be put in the form

My - y^ , txJ

tr^ lvl

ytxJ

characteri,zed by the conditions

ti n?

anrli

rre l onf

r*; l rz

i f *l Thrrs v

is

= u * M y '^ ' r - i (*) = 6*My = xc where e*N{, = lvt*c.

resolved

into

a sum of

variables

Y = i i * l+ z an expectea part i1*; unexpected residual

= XC, predictable z = y - y(x),

from x,

and an

which is uncorrelated

with x and has expected value zero relative

to x.

_ t 7 -

The rnatrix c has been characterized

in two equivalent

exMy = M*s, and also by the relation

ways, by the relation

= 0 which says that y - xc be uncorrelated "*My-*. These relations are equivalent, since

x'

= e*M*c = M*a = M*c,

"*M*a

= \{*. follows e*Mr_*. = €* (vy - tl*.) "*My = [1*.' and f rom e-M.,-.- =O follows e-M.. = X Y x c ^ Y "*M*.

= M*.-M*.=0,

so fron

4: For any variable THEOREM exists

with

y in an experiment there

x,

= M*., or equivalently "*My = cxy uniquely. x is regular then.

a matrix c such that

= o, and if "*My_*. Now two further that

c. One is

ways wi-11 be shown for from the method of

farniliar

characterizing least

squares

I i n e a r r e g r e s s i o n , a n d t h e o t h e r i s k n o w nb u t l e s s Instead of the usual methods of the calculus

fanlliar.

algebraic nethods will

be used.

x is regular,

T H E O R E 5M: I f

=

e -xy

then c = c

is defined

so ,that

ft 'M ' x ') - xr M

xy

- 1'

MrM 'x 'y

eives the absolute minimun for

the trace and determinant of Nl|_*.My_*.. Since e it

follows

+ e

X

1 , e x M x = M * , 6 * M * = 0 a n d e * l r ' l r =M * . * y ,

X

+L ^ + L I I d

L

l, I v

y

l

-

f

ii

l

l

x

^

-

-

o .

l'M x \,-,Y

V ' ' x (t "cx y

and then,

Afrtat!

s i n c e a l s o e x- - i s

Studles

M c) X-, -

cl

synnetric

+ tr - X f\'t sY *

-

M c)

-6x "My and idemPotent,

that

- t 8 -

( M . ,-

-

M-c)'(M. /

(c \"xY

=

M " X "c )/

' Xr"M x (\ "ex Y !r -/ ) r' M

-

c)+ M'e M "Y"x"Y

= (' Mx- . cx- . .y. -1 xt { - - c )' ' ( xM . x_ c -y. .M-x- _ c ) + (", which, witfr i(x)

= *.*y,

M*.*y)'(My

M*.*y)

can be stated

+ M' ^ M^ My- '--x- -c_ My--, x. .c^ = I l l 1rl ' y (x) -xc y (x) -xc y-y (x) y-y (x)

It

follows

immediately

theorems

the

matrices

(Mirsky

trace

this

identity,

and determinant

of

by the well-known

non-negati,ve

defj-nite

1955), (a'a

trace det

from

+ b'b)

(ara + b'b)

= tlace

a'a

+ b'b

= det ara + det b'b

that t r a c -e-

-

.l t' 'y[ - x' _c_ "I {y - x c

trace

^, M' yt - y

.M (x) "y-y (x) ,

and

t ti-*.ty-xc where the equalities

det Mr-i(*)My-i(*)'

hold if

and only if

= 0 M . : . ,. M^, , -' y ( x ) - x c ' Y . t x . )- x c but this is if

and only it

nti(")_*. = O.But Mi(x)_xc=

= M*.-..-*. = M*(c*r-c), and since x is regular, M*(c*y- .) xy if and only if c_,, - c = 0. Hence, the equalities hold .if

rr

--J ; t - ^ ' -= a n o ^o- n1 .r,y 1 r a cxl,

=O

a -s r- ^e^ q. .u: -i r e d .

The least squares principle

is shown in the part of the

t h e o r e m c o n c e r n i n g t h e m i n i m u mt r a c e . A p r o o f o f t h e p a r t of the theorem concerning the minimumdeterminant has been given by F. Sand (in a private

communication), by

nethods of the calculus, using the formul"

l"l-1a]"1=tracela-1aa1.

- 1 9 -

3. Experinantal

configuration

Certain relations

between variables x, y in an of the

experinent can be presented as an alalysis

nrniarrnr<

a

o, t ' y

nredi.t2hle c n n d i t -i o ' -n"

from x e ' x -ey

€--€-,= e.,. Also

is

X

Y

tnat

Ol

the

SpanS.

as has been seen the

Thus,

V

in terms of the orthogonal

be characterized

therefore

l*,

spans E*,Ey, and can

forned in E by their

configuration

is

E.,5 E-,

= e.,, or y' the

what

condition

E, be orthogonal,

for

condition and this

is

is

y

x,

to

to

stated

by the

be uncorrelated

and the condition

i s e - . e . .= O , o r e q u i v a l e n t l y e . , e - = 0 . A r e l a t i o n ^

be

by symmetry,

equivalent

for

y

for

this

of

y

inci-dence between x,y can be defined by the condition

This is the condition that there

E*, Ey properly intersect. exists mutually identifiable

c o m p o n e n t so f x a n d y ,

is variables

x a , y b s u c h t h a t N { x a= N I y b . - s i n c e t h e

intersection

of E*, E, is

relation

that

the nul1 space of

1 -

that

"*"y,

this

can be expressed by the condition

- e - - e ,1. = 0 . T h e e x p e r i m e n t a l r e l a t i o n o f n u t u a l ,l 1 x y' predictability, or a!94f-!y between x, y is expressed by the relation relation

E* = Ey, and equivalently

of orthogonal incidence is

e* =

A further "y. stated by the condition

t h a t t h e o r t h o g o n a l c o m p l e m e n t si n E , , E . , o f t h e i r

inter-

section be orthogonal; that is, E*^ EuI Eu."E*. This neans X, y are identifiable comnonto both,

together with

with a set of components

further

sets that

are mutually

- 2 0 uncorrelated. The condition for

this

that e e , eYel' x Y=

is

i n w h i C h c a S- e e - x- ." ey 'i "s t h c n r t h n o n n a l n r o i e c t o r n n E - h E . , , and e*er,6*"y

a r e t h e o r t h o g o n a1 p r o j e c t o r s o n E x ^ E y , E y ^ E * .

Sti1l a further relat'ion is that y he inclined to x, defined hw

f he

cnnd'i t i nn

l-h"s"f

" "s x N ' ^, y! "h *a v"e

case, i I y is regular, then so is its w h e r e M - - ^ = e - . \ 1 . , .T h i s i s x c x y = r a n k e .

*a "S 'M' y '.

S a m e r a- n" k"

the

In

that

regression xc on x

stated by the condition

rank e-_e-_ x y

v

These are nodel relations setisfiedhv

the

Tnoether

rel:tinns

t. h e .w

,

thcv "

implication

.

that generally will

n r e. s e / n f

harre with .

.

"

.

2|

v

n | 2 t t r a r nr

eneh /

l

n fl

nther r

g

not be

i Ln f pl r e nv n nr p r tr i n n '

nf l

v

onnnqitinn r

a n d c o n f l u e n c e a s s h o r ^ r ni n t h e f o l l o w i n g

r

'

schene

Opposites C o n fl u e n c e (-----

identical

distinct

I rl,

A I

included

excluded

/

incident

\

.r'

inclined

\ \ \ y ' c correlated

\

dlsinclined separate / \ / / \ / o n f l u e n c e \ \ // ----------) uncorrelated

Rather than the model relations

that

exact realization,

can be defined whose extreme

coefficients

values give criterla

for

the exact relations

intermediate values constitute a f + a i h m ^ n f d L L d l l l l l l g l l ( ,

^ d >

' . i W I r ll

I

nt r^ vt iF

generally

n ip oLr dr r . aa p

do not have

but whose

a measureof approximate

o1

4. Configuration coefficients x and a q-variable

Consider a p-variable

y in an experirnent, spans

and Iet e-., e.. be the orthogonal projectors on their ^

y

"x' "y' D ef i n e dxY = r a n k e e

R 2 = tt rr aacc" e e e xy "*"y'

x y

d * = t r a c e e x , i n w h i c h c a s e a l s o d x = r a n k xe ' . a n d s i m i l a r l y d e f i n e d r . T h u s d * z p , d y ' Ll 4rlu x , o r L

Also defin"

is regular if A first

d* = p, or d,

and only if

property for

nj,

is

a '

that

1 d . d o 4 R 2 1 d x' y xy xy N o w R - , . .c a n d e n o t e

xy

and can define

the

the

coeffbient

"*

of

rnrral

between x,y.

dimension of correlation dimension of both

square root of trace

positive

Uy

r

9

af 9

r

v

r

i nn l

,

and d

Clearly d

and e, E* ,

the

xy

is

xy

"*"y,

the

and the rank

o f e \ 1 a n d ey M x , x y Thc Frrrther nronerties o - -f R "xyshow how its oi vc eri teri n fnr model relations

values

between x and y. Thus,

^R - . - -= ' 0 i f xa n d o n y l y i f ^ e - e . ,t = 0 , "w h i c hx i s

t o' s a y " F

are orthogonal, or x, y are uncorrelated.

Further,

Rj.. = d-.. if and only if x y x y ^ if

e-e,, = e.,e*; that is,if y y x

E-., E.,are orthogonally incident ^ /

be identified

FY

and only

in which case x, y can

as having components in whlch possibly

are identifiable but the rest are uncorrelated. ) = Again, *i, = J* iF and only it "*ithat "*", E . ,g E - , o r ) ' i s p r e d i c t a b l e f r o n x . S i n c e O * , n= d " , d y ,

the dirnensionof correlation

some

is,

between

- 2 2 does not exceed the dimension of each, and

the variables if

d - . - .= d - _ , t h e n y i s XY Y' tf,is property

regular lf just

incl ined to x,

so in case y is

is preserved j,n its

regression on x.

x is a regular p-variable, and q = 1' that is, it

a single variable,

to the classical

Y is

is easy to see that R*,. reduces

m u 1 t i . o 1 ec o r r e l a t i - o n

coefficient.

For in that

caSe ) - 1' M ' M - 1' N l ' R' = trace M fM'M ) xy x y f' l y. ' t 'yl u)' l y x'x x'

= (Ury4 '' M X \f 'M ' X'"M X /I = Tn casc el so n y

- 1'

M'M

' M' y )' 'X"y ) f M " "y

- 1

1' t- t" h" ' i s" r e d r r c e s t o - 1' [ N l ' ] \ ' l - 1

)

^2 ^

= f' Mx ' N y l "l - xf Mx' M' I y y )' x y which is the classlcal correlation coefficient of Pearson. Tt annears that

the characteristic

squares of canonical correlation x , y . H e n c e RX_V . , i s i d e n t i f i e d the canonical correlation

r"oots of

coefficients

-ex "ey a" 'r e t h e

of flotelling

for

with the sumof the squares of

coefficients.

would not fo11ow from the established

The properties of R*, canoni.cal correlation

theory. But rather these properties can cone directly

from

properties of the product of symmetric idempotents exev, as aLso can the theory of canonical correlation, The total

multiplicity

(Afriat

of non-zero canonical roots o

and a corresponding nuinber of canonical pairs

1956, 1957)

2 ' ^ I ,, ,xy,

of conponents

is determined, with

sample vectors whi-ch are i-n the nu11 spaces 2 . z. o f 'p - 1 - e * e , , a n d o - l - e . , e * . B u t a c o n p l e t e c a n o n i c a l a n a l y s i s y y ^ requires

determination

of U* - U*, and d,

componentsfor

x and y.

The sample vectors

n r r l l JsPne aL LcJ e s o v !f

1, - e_ x e y

a n d 1 - e_ I e x .

,,uaa

- U*, further of these span the

-

L

oOa

-

Fortheregressionofyuponxthereistheresolution ty="*ty*6*I'1,

= Mi * tr-i where y = y(x),

and, since e* is

symnetric and ilempotent,

N{yMy=ti"*tr*ti6*ty = ,i " l * \i _ gtny_g. Measureof the exactness of the regression dependson a componentwith

conparison of the regressional

the residual

component. ) R*y = trace it

"*"y,

-Z a*y

trace exey

appears that = t r a c e e - . ( e , .* e , , ) = t r a c e R " l . + \O t y * x y x y y

x

This is again bY taking the trace in - l'11'- r' 'l = + (ltll,ltI.,) e*Nlr, lut.re-.lt'1.. (I,llMv)

since

trace 1 = q - 1ll;e*u, = oi, trace (M;My) -1u}"*r,{, = qi, trace (Mj.l,'ty) The coeffici,ent

o'*v = trace

"*"y/

trace ey

thus has the ProPerties, o- 1 p- x 2y < l S o P^* /. , c a n d e n o t e t h e p o s i t i v e if

and only if

square rootr

the regression of y on x is nul1, which is and P*, = 1 if

the case of x, y bei-ng uncorrelated, only 1f the regression j,s perfect. the classical

and P*, = O

correlation

and

This also generalizes

coefficient,

though it

is non-

^

just

symmetrical between x and y and applies

on the regression

of y upon x. A symmetrical coefficj.ent

between x and y

of association

can be defined by Cz xy

= (trace e e \2 / xy'

ftrace e )ftrace x"

e ). y'

This has the n r ' r" or n e r t i e s A \ , z-

C - . . -= 0 i f xy

a U x y z-

1| '

e - e . , = 0 ,' i t h a t i s , E _ , E , , a r e o r t h o g o n a l , xy )( y = y are uncorrelated, C*y 1 if and only lf e* = er;

or x,

that is,

and only if

E * = E , , , o r t h e r e g r e s s i o n s e a c h w a y h e t w e e nx a n d y ^ f

are perfect,

each is predictable from the other, or they are

experimentally equivalent. This coefficient

and the coefficient

of dissociation

defined by 1

)

. * y = t - L x r harreanalogies with the cosi-neand sine, further

generallzation.

Thus, it

) xy

and capabilities

is noted that

1

C xy

C xy

1

Then, for several multivariables x, y, z, coefficient

of dissociation

.,.,

a g e n e r a l jz e d

can be defined by 1

C xy

C xz

c

1

f

r

c

t

. . .

y L

r..ith

tL'a

nr^n6YrieS

O

<

for

zx

K - . r -I- r Z r . . , " X

zy

^ r J r p r . . .

-

;r rr

^-r

erru

Jc rnres nr rc J

^v r-r1r j./ , ;. rf Fx L t

F "Y,

--

.- * y F -

L

J

_

(u -x z

0,

cr6

m r r r r r a ' lt y

zt

be partitioned

Ex, Ey,

within

Of

OTthogonal,

= 0 if

are nutually uncorrelated, and **,r,2r.,. if

tO Say the

whiCh is

the unions of each class are identical.

and only

and such that

Nowconsider the

multivariables partj.tioned into classes, thus xl, Then the K-coefficient

*r,*2,

A v U =z n

*2,

has the property

/Y1,Y?, "'/"' = K *1,*2, ..., Y1,Y2,

Then

z,

i n t o c o m p l e m e n t a r yc l a s s e s ,

which there is mutual orthogonality,

and y1, y),

y,

X,

x1rx2,

fKx,x2, ,..,Ky1ry2

4

y2,... /

/y1,

where the value 1 is attained

if

1

there is

and only if

orthogonality between classes. Sti1l

nore complex coefficients

in an analogous way, which give

can be formed indefinitely

s u m m a r ys t a l r e m e n to n t h e p r o x i m i t y o f v a r i a b l e s t o r n o r e complex configurations defined in terms of sets,

sets of

sets. and so forth. ^ - 4^r-r 4^r1u B^/ ^ " v^f f t h e C a n d S c o e f f i c i e n t s w i t h t h e c o s i n e

nlr

a $ r rnv d

s ir nr r ev . t

J

the

reflectino

ensinp

rnd

Sine

fOrmuIaS

fOf

a

SUm

and difference of angles, is seen in the inequality C

" x y "Cx z

-

"Sx y "Sx z

= ( -

" x y "Cx z

where one or other of the equalities nnlr'

if

o

noir

nF

the

$ "xy

S "xz

is attained

-S rp a* C- e- S' - Ex -,- , p" y , F" z *e t' ' a

pair of orthogonal and the third spaces are rays, this

+

is

their

if

and

i,-lontiCal,

union.

Of

a

In case the

showshow the cosine between a pair of

- 2 6 sines and cosines with

rays is bounded when their

a thlrd

ray are given. matrix

Returning again to the decomposition of the unit associated with the regression of y on x,

rather than taking

traces, which leads to the foregoing discussion, 1et determinants be taken instead, to pursue a paral1e1 to the alternative m i n i m u m t r a c e a n d r n i n i r n u r nd e t e r n i n a n t

characterizations

for

in Theoren 4. Then comparison j.s to be

the regression matrix, nade between )

t

'

c' -x y = l l r yl ' -e x\ 1y I" 1 1y4 y' \ '4|

- 1

and

-1 ' i y = l 1 1 6 * * r l u ; u ,I (Afriat

But then, as follows from determinental identj.ties 19s7), = l r - e e 1 . 2 = l r - e e I ' x y' xy x Y'

, l xy Thesecoefficients,

defined for the spans [*,

I:.,,which can

of separati.on and inclusion,

be ca11ed coefficients

have

analogy with the sine and cosine, respectively, as appears f r o m t h e f o 1 I o w i n s D r o D e r ti e s . Thus,

.) u LA

)- ux y z-

I

r,

"c "A "c x y e q r A a n ^ f e t h e p O S i t i f ' e s . . . ,= 0 i f xy that

iL s .

h aJ v e

S-.. = xy E-,

E,,

afe

I

if

i n ' t F r s c rY - f i n n

and only

TayS,

q

"xy

i c

SqUafe fOOt.

and only i f E-,, 8.. are incident, y x' "

if rhe

nf

n. o s i t i v e

dimension.

E_, f., are orthogonal. y x' cinp

^f

tl''o

ancla

Similarly, o L , l _ - L 2x y. _" l

"s "o -c x y

can

(lenote the

rositi\rc

snUaIe

fOOt.

in case

hotr"ggn

them.

-2 7 -

.*y is;

= O if

and only if

E* is disinclined

to Er,

that

sone proper subspace of E* is orthogonal to Er, which

can be expressed by saying x is partly

with y,

uncorrelated

or the regression of part of x on y is nu11. r*y = 1 if

and only if

E* is a subspaceof Er, which

meansthe regression of x on y is perfect. while it

* ,i,

is generally not the case that cl,

= t,

i-t can be shown that .- x7y * r- xz y -1 1 and noreover, by an application

o f T h e o r e r n so f l l o l d e r a n d

It{inkowski that t ! ' l r' c ' ) i * ( s x' y)' ^i < l xy'^ where equalities

hold if

and only if

E" is either

included

in or is orthogonal to Ey. In fact, identified

with

alienation,

can be algebraically

the c and s coefficients the coefficients

of vector correlation

proposed by l{i1ks (1932) and Hotelling

and

(1935).

Introduced as here in terns of orthogonal projectors, general properties readily

evident,

are easily derived which are not otherwise

and their

geonetrical

neaning becones

transparent . W h a t e r r e rc a n b e t h e s t a t i s t i c a l such as the variety

considered here,

role of coefficients they have an intrinsic

place in the formalisrn of regression analysis, arise

in a natural

that unites

and they

way j-n terms of orthogonal projectors

algebraical

and geonetrical

meaning.

zE5. Varlance T h c s n m n lp m e t r i x

\ 4 - . = (' M X -". , X

".,

M " X DI n f c n - V a T i a b l e

with the vettor M, , ..., M" ^.1 ^p = \ ' l i \ 4 x volume V* is given by U; | | , and,

x determinesa parallelitope a s e d g e s, a n d i t s following

W i l k s ( 19 3 2 ) a n d A n d e r s o n ( 19 5 8 ) , c a n d e f i n e t h e

variance of x, ln the regression of y upon x,

t h e r e i s d e c o m p o s i t i o no f

the sample rnatrix of y in the forn = t*.

ty Four equivalent

* ty-*.' have been stated for

conditions

the

determination of c, and one is thaa Ur_*. be a minimum. Regularity for x is stated by the condjtion Vx I 0, and w i t h t h i s g i v e n t h e r e i s t h e u n i q r r ed e t e r m i n a t i o n . = . * y , -1 uhere c.... = (Ml.\'1..) Ml.V... ^

^/

A

^

y

In the case of two simple variahles x, y with joint ccnn1a

m.f

riv

. . \ {. x y

(NI-\|,,),

the

yerione

.a \./ x y

ic

niygn

ly

, . . =* . lfM,r ,

\u/ -

xY

]\'l I | 'xY xyl

l' M ' M Mx'\ 4y lI x x

l| l' |yI 't x My' M ' y I|

= lI r t l r tl.. l u :y. uy .r . 1r r x xr I

.)

(Iri\IY)'

= v ? . v ?1. 1 - c o s 2 )e x y - - v z v z si n z o x y , _ . . T h u s V _ - . .= V - . V - -s i n 0 , t h e a n g l e b e t w e e n l \ t _ _M x y x' xy Y w e l l k n o w n f o r m u l a f o r t h e a rea of a which expresses the v,'here 0 is

)

(Mir4x)(\4i\4y)

- 2 9 -

parallelogram.

But generally,

V_., can denote the volume of ^ l

a paralleiitope parallelitopes

i n p + q d i m e n s ' i o n sd e t e r m i n e d b y t w o i n s p a c e , E - , E . , o f d i m e n s i o n sp , q b e t w e e n ^

which the coefficient

f

of a separation is

sx y = 'l r - e ey 'l 1 / z x Then, as a direct

generalization

area of a parallelogram, it

of the formula for

can he shown, from

the

determinental

identities (Atriat 1957),that = V V s V x y x y x y A further

d e t e r m i n e n t a J .i d e n t i t y

(Afriat

= lI r 1 y . x' . sy .'. r 1 .'I.t1rx: .xv1 '- . 'l r

" x " .y

1957) shows that

I"

But

l u ; " . r r r l = l ( e * \ 4 y )r' e x \ 4 y ) 1 , The parallelogram wi.th edges fromed from Nt*, can be considered as having V* as a p-dimensionalbase and 6*Vy.t q-dimensional"height",

and this

corresponding

i d e n t - i t y s h o w sg e n e r a l i z a t i o n

of the rule that the area of a parallelogramis the product of base times hei.ght. S i n c e e . . t t . , = J r l . ,l w J r er e 9 = x- 'c-. x y y-y xy' i + r L

. , - - ^ - - d P P C 4 r >

+ L ^ + L l r d L

= u*t*y

'y-y which shows the relation

of the residual variance \'r_i in

the regression of y upon x,

to the variance V*. Also, as

follows from the sameidentity, v^ = v l r y x I -

ti

^

x x y

since Vi = e*r1r, - e e t\/2 x y t

- 3 0 -

remark, it

As a flnal identity

to be noted that the determinental

is

which gives the generalized parallogran

1aw, which

can be st&d T,II M N'I'N' x x x y '\ ,'1ryv

x

= l r , l r * l l u ; r . r r l-l t "*"rl

l \ 1r l \ ' l

)'y

where l ' Ix),- 1 l r^{ ,x. , e- x = l' .' {x (t N' 'x,N f n o o f h o r

w i t h

t h n

-1N,1 e ' y. = \ {' y (. r ', 'yl- ;' ,y4' ' ; y- ,.

n r n n n q i t i n n

O < l l - € * € , 1 < 1 , wherc

there

ii -S'

!e\ O , t U. d. I^r 1 L /:

+,,

^-r

d l l u

I

I

:c r l

dar n r u.

l

vn i n l r lj rr

l t ' l - ' - r 1=- -O , s h o w s t h a t x y ,\ lI ,' Vx ' "t rx, |t || |' vV' ' u l I M ' T I I {' M v l x x x y

'.if

p

p

=

O,

or equivalently

'\ 4' ry\ '{ x

'

\4r[4

y y

which is an equality of Fischer ('l908), from which follows t h e w e l l - k n o w n d e t e r m i n e n t a l i n e q t r a l i t y o f t i a d a m a r d( 1 8 9 3 ) . Another proof is thus obtained of these inequalities, the added explicit

qualification

with

shownby the factor

I' f - e x- ev. .,1' , a n d t h e p r o v i s i o n t h a t t h e e q u a l i t y i s a t t a i n e d = 0 . B e l l r n a n( 1 9 6 0 , p . 1 3 7 ) r e m a r k s t h a t o n l 'y .i f l Mt J M v .. " l l a d a m a r d ' si n e q u a l i t y i s o n e o f - t h e m o s t - p r o v e d r e s u l t s i n analysis, with well over one hundred proofs in the literature."

-

-

JI

PAP.T I I

'I

Ir'inein:l nAl ln, vw

^

I t O t

n l l r L

) 1 , d 1

nf -L

neirs

^*^

l l

d r

I

U l

d l g

nf

snrccs

^L1:^rrc

s

u u l

cornponents lr'hich are

- . ^ i - , . Lw :l ^l l 1l l .l

O

' L;

q n a c pq cJ

tL uw -v n

of

lair

redrretinn

- . (+l l1u .F^u ^r ^l ( -l ^! r

n l la l dw j /

r Y u q

inclined +L U^

hl , es

rI eq ud ur Lr e n c d u

f g

each other,

to

+( i Ll '. e S e a n d

tO

aiSO

3

and

eacll

which are mutualy inclined, are

other. The prjncipal lair,

o b t a i n c d a s t h e o r t h o g o n aI p r o j e c t i o n s o f t l l c s p a c e s i n l ai r e a c h o t h e r : a n d t h e o t h e r - t h e r o si d r r . an

are the

I

con,plernents in

orthogonal in

fhc

nrinein:l

n T n n . r t \ /t ) l , r u P l r

n u fl

projections

nair-

hu eq ir nl lob

each

lhe

ri nr r

in

the

prolositi-on

The fundamental

thc

sfaces

nrincinel

rr A a ri nl ,r rnve! eu lr sJ r L L

each otiler

of

of

nair

+ l ' ^ + L l l d (

of

either

also

+ l - ^ ' ' L l l q j /

have

- * a d l q

+ L F L l l s

space the

n v rr l hL nr cr un bn ur lr r q r

spaces.

fron

which

thj-s reductj-on

i s o b t a i n e d i s c x p r c s s e t Ji - n t h e s c h e n e : E @ k x y x y

t

x

t / ty

where the that

is

aplears in

indicated

relations

e..[,. is ^ y

i n c l inecl to

are 6 . ,, I

as orthogonaI

to both

C

x

and ty are oblique

and the R*r,

whic)r

extr and \ / , c a n b e d e f i n e d

t h T c n c n u i r ' : l p n t w a y s:

fq* o"*

xy

t h r t

the

.i q" ,

2 s

nrniectinn n rn i e e ti n n nrthnonnel

the in of

=

I

,

( e;.Ory I \E,. o"y E*

o r t h o g o n a l c o n r p l e n r e n ti n

t-_ o t r_. j n

n rn i a a t

,),, or of sinply

t-. of

E,,, or r

of

t 1 , o

^ r f l r n c n n r l

+ l - ^ Lltg

^ , + l ^ ^ ^ - ^ 1 vr Lrrvl:;urror

, y , o r o f s i n r p l y {.,, or o f the

ion of

qx in

6 y . A pnr ' I / v'

i n o " o

t lr

is

schene to

-

-

Ja

and then again with the spaces

one space with the other,

i n t e r c! rhr u:r nr a o- \ e rs ,1 . t h e r e i s o b t a i n e d t h e f u r t h e r

-

!x | o l

'l'he is

by the

l

t

)

with

for

1 -

c,c..,

)

=

con'ponents

A

ly

"* the

spaces interchanged. tlre rcLluction is

anC fi*..ts ^)

)

ortitogonaJ. projectiors

the

e*er,

the pair.

principal

the

obtair.ing

e.. [,, as tlre rangc of ,\

of

rclation

same relation

An algori thn

reduction of

ex{e)- 1r) and the

t n

.

t.="y e^-!y*

property

reciprocal

stated

'/'---ix [1' Txy | ..-l ?
principal

the

r , ' i ' r i c hd e f i n c s

scheme

f ronr any lrases for

the

spaces,

by

the

by computing

thc nrll]-spacc having

of

beetr conrpttted, r''hich has been

fornula

given.

arrtl reflexivc

Z. Recinrocal

directions

In a pair of spaces, reciprocal directions are those w l r i c l i a r e t l ) e o r t l t o g o n a l p r o - je c t i o n s o f e a c h o t h e r 1 n t h e slaces, Thus \ f € r ' fnr

rnr

i nrnre

flra

nrfhnonnrl

I

dua

tlre

lrn oone.l

nrf

a ) " " - -

) f

u , "yn= in spaces x,

on which , ' , e v . l h e proclucts projcct iors arc cx ' x ' Ay '

the

^

'

directions )L,V

A

define

) , = G

A

- yA - x

frair of biprojectors

on the

projectors are synretric,

cna.oq

Si ne e

the drral hinroiectors

-

J

J

-

are tlie transposes of each other.

that

space of

with

identical

is

one which

is

in one space relative

direction

A reflexive

its

the

projection

orthogonal in

projection

orthogonal

to another

the

in

other'

A^ - ^ - A; - ^ 1 \, nL!ulurrt}>r/ t

L = " x" y Z c f_or a refl:exive direction

€* 1n regard to

) ,
Evi.dently a reciprocal pair

of directj.ons in the spaces are

clirection in one space in regard to the otlier.

each a reflexive

Ccnversely, any refi.exive direction gard to another forms with the other a reciprocal pair if

'

orthogonal projection

its

\ / 1, L , andU = crlt

in

in the spaces; for

of directions

)

t,c=

in one space in re-

\

\r

(A = e^ U , tlten

"*"y < l e termination of the reciprocal Thus the a pair

Ey.

between

directions

of spaces is equivalent to the deternrination of the directions

reflexive

o n e i n r e g a r c Jt o t h e o t l l e r ;

in either

anc these are the invariant

of either

directions

one of the

bipro jectors:.

3. Proper angles A proper angle betrveen the spaces is

c l e f i - n e da s a n

a n g l e m a d eb v a p a i r o f r e c i p r o c a l d i r e c t i o n s ' )-L ,1J are a pair .i.,U

of reciprocal

the directions

l

,U .o,

where Otos

Afrl8t:

the acute angle

proper angle w h i c i r t l . r e y m a k e b e t r r , e e nt j r e n d e t e r m i n e s a

between the spaces. If

,

dLrections,

Thus' if

U,\r is any pair

tl',"n 2 L,t ?/ ,

= ( L l' \ ' ) 2 / ( u ' u ) ( \ I ' \ r )

' /t \, ) J . ^lf r. ( ' o ( LL,U { 1' hitlr

Stutltes

of vectors spanning

,/Z

,

- 3 4 -

While any reciprocal pair

determines a

of directions

p r o p e r a n g l e r n a y b e r n a r l eb y unique proper angle, a given rs ' nany <listinct reciProcal Pai

I i . e c i L ' r o c a 1l t e c t o r s

4.

c l e fi - n e d a s t h o s e r v h i c i r s p a n l 'jllel J are \rcctors directions' Thus, if ll'\' are reciprocal

N o l n 'r e c i p r o c a l reciprocal i n q .^ . ,

q, then f

erLI = \'o

e*\I = Llp , for

a,o

some multipliers

appears'

whi'ch' it

o,

l

' are giYen

by a =

(tlru)-lu'v, fron

tn'hich it

follor''s -

po

(\I'\')-l\''u;

tl at t

A

coS" tl,\'

It'loreover, cre'tl = tlI

, crer\-=

latent

\'t

, dual

thc

of

vectors

t-hei'eI = po ;

so,

biprojectors

witit conrnoncharacteristic "*"),,e).ex 2 l = p" = u

whiclr js

p r o c i r . r c to f

thc

s ( l l r i r r co f

ll ,\' are

the

0' oand

nlultiplicls

r-a1ue

also

is

the

tirc cosinc ,t\

u = cosu,v of

the

angl e betrr'een thern, v;hich

tretween the If,

also

is

a p1'oper angle

sPaces.

further,

Ll,\'are tlnit vectors,

thcn

P = ll'\' = o In

this

case, o = lrq where eull =

shorving LJ, \Iq to

E =

tl

(V 6)r, ,

be a reciprocal

;

ancl then

e-(\'6)

= UP ,

palr

unit

of

vectors

h'ith

- 3 5 -

which are equal to each other,

multipliers

they are to define

with which properties

and positive;

a nornal reciprocal

pair.

the spaces are the same, and non-negative;

and they are all

the spaces are orthogonal, which is when

zero only if

biprojectors are nu11. Therefore, if ?

1et.p'denote taken that

on

values of the dual biprojectors

The characteristic

the

the spacesare oblique,

any non-zero characteristic va1ue, where it

p> O ; and let

\/Fctor of e e ^

, s o/ t h a t

U be any corresponding unit

is

latent

)

e - - e - - U = U U ' ,U r l J = ' l; x y and define V by e U = V',.

v

-

Then e*V = Uu

, and V'V = 1;

when U,V are a normal reciprocal

pair

of vectors in the

s -p a c e s^ € . - - ,6 y_ . . ^ ^ ^ ^ r , U. l ri r hr B^ r ' / l, '

n L L U I

tL nU

of the biprojectors reciprocal spaces with

pair

2nv

d r l /

n i l n u ln l - ' 4Fq rl u^

ChafaCtefiStiC \

ValUe

UZ

on the spaces, there corresponds a normal

of vectors making a proper angle between the

cosine u.

It

has already appeared that

the square

of the cosine of every proper angle between the spaces is characteristic value of the hiprojectors.

Thus it

that the characteri.stic values of the bi-projectors

is seen on the

s n a c e s a r e D r e c i s c l v t h c s n r r n r c so f t h e c o s i n e s o f t h e proper angles between them.

a

- . 1 0 -

5. Rank and mult j-plicity r T " 'a" r ' - x, 0 r o \ < o , < " / 2 ) d e n o t e t h e n u l l - s p a c e o f t h e m a t r i x ) ,1 - e X. - eV .' - , w h e r e l = c o s 2 o . T h e n e v i d e n t l y t * r o

t

t y

;

" *

and €.- I 0- just x 0'

when s is a proper angle between the

spaces, which is just rl r* a: ul rs r o

'

i n

u , hi . h

whenl is a biprojector

.aSe

i -_

fnr

the

latent

Vectors

r d

and the corresponding space E.,

have

Y t a

the samedimension ra of

reflexive

associated wi,th a proper angle o.

T h e s ^p a c e / X

r which will

be taken to define the

o as a proper angle between the given spaces.

Si-nce latent different d

Of

tha lharacterist i c value tr , which span

directions

Y

composed

X , 0

^r a 6

rank

is

characteristic

vectors

of a matrix

corresponding to

characterist ic values are independent,the spaces

a r e i_ n" -d- er -p e p r l e n t : n d t h e r e f o r e t h e d i m e n s i o n O f t h e i r

union is the sum of their

dirnensions. Thus

O r- x , c c ; and correspondingly t

e ' x i- I

r' o <'

,

r

o

where r is

the dimension of incli.nation of the spaces, being

the dimension of e^ the relation

.y, ,' w h i c h i s

t h e r a n k o f e - - ey . . . H e r e x

between spaces must be an identity

r e i a t i o n b e t w e e nd i m e n s i o n i s ,

as wiII

if

the

appear in fact to be

the case, an equality. Let the multiplicity defined as the nultiplicity

mo

of o as a proper angle be of I = .or2o

as a biprojector

- 3 7 -

Rut tltere is the gencral proposition

characteristic vaIue.

t h a t , w i t l r e - , e , , s ) - m n c t r i ca n d i d e n p o t c n t , t h e n u l l i t y l'1- c-e., is equal to the mtrltiplicity

the matrix

as a root of the equation -

'o and

that

also

the

'' r

rank

- e"erl = 0.

llt

=

x

of

f o 1 1 o r , ' st h a t

'

o

e..e.- is

of

x y

o f j - t s n o n -z e l o c h a r a c t e r i s t i c t

It

ot-

ton'o

to

eqlral

the

nultiplicity

values,

;

rvhence a o = t

t (I

Accordingly, =Q/

erty '

r v h i c h s h o r r ' st h e d ' rf f c r p n t

nr.ncr

ref lexive

of

the

h.

ortl.ocnnrl itv

Lct

a

1 f

ortliogonal possiblc

to

the pai rs

four

o,

the

r''ith the

oltitonon:l

nroicctions

other.

be trio pairs

a Jircct ion

then

other

in

this

lclatt-ons three.

are

'lhus,

four

ilotiever, shotild

directi-ons

case the

in

g,cncral indepcndent,

in

betrr'een tite pairs. reciprocal

can be

one pair

the ctlter pai r

in

four possiblc,

relatiorrs

is

directions,

of

in

f , . T l r e n a c li r e c t i o n

be f orrned of

there

ilrn

associatetl

rclations

orthogonality

irnplying

each

r v a y ' s ,t l c f i n i n g

orthogonality

spaces,

in

( a = :r,B )

lL- U ^ tl i.r

rnaking angles

qn2nnino

: n r , ll a\ Jq'

oir,'en sn2ces

,

directions,

g l l t

r

X.a

l

i-n a pair

proposition

of

that

equi-va1ent, &n)- one of in

this

case,

tltere

are

the them the

equivalences

l^ r I ancl tjre othels

U

D

c le t l u c e d f r o n

(.j l,r _L V. o * b syn'net11', by

l-rlterclianges in

ti-ie

- 3 8 and o, 0.

synbols U,lf

the four orthogonality relations

If

are taken together to define

between the menbers of the pairs orthogonality proposition

the further

then there is

as hetween the pairs,

prcper

pairs making distinct

that reciprocal

angles are orthogonal: err

( ' U o, ' 0 . r I ( U u, ? r ? ).

o * P : )

nC

T h u s , i n t h e f o 1I o u i n g s c h e m e , o n y o n e o f t h e o r t h o g o n a l i t i e s indicated

i r n n li e s

r^..-.

all

I U U I

^- r

^ 1 1I

d l l u

r

f ^,,r

are

I U u r

4 I

i mnl

.i

ed e u

hu /v

the

distinctness of 0, S

4 --.r.- 14

"l r/\ ? (I ' ,

tlf

rJ

I. i

v^

V P / .

r ) w B v r r d r a L )

Consider a pair of spaceswhjch are suclt that any direction

i-n one space is a reflexive

direction

of that

s p a c e w i t h r e g a r d t o t h e o t h e r . A n e q u i v a l e n t c o n , , l i t i o ni s t h a t t h e s n a c e s h e o f t h e s a m ed i m e n s i o n , a n d h a v e t h e proper angles

l r e t w e e nt h e m a l l

qn2eos

cal I ed

wi I I

ennrlitinn

nrnnpr

he

thetL

enolcr

is

y ^ " f . ' . . , 6 f v * ,

i snonn:l (

F

sJ Pnd-r -s ^) .

C , X ,

c n q r r

directions nfhpr

cna.a

ir cJ nv cB nv nr ra ol t

in

reciprocal

af

E , ) ,

ny ra i t r t

^-

necesstrv

d i m e n sJ ir O v rn r

" t

^ n l -

1

-

e - x c- y l' P

q n t e e s J y u r L . , t

one space projects

ir nr r f ( ov

the directions

A

and n H

[u p!

suffic

ient

jr Jc wn6ov nr r na a l r ,

uw rr Li rt rh

that ? (cos-

T r rn/

the same' Sucha pair of

d l l

^ - +L1i r, v^ S^ v^r -r -a 1L

v I

with their

= O.

d t L /

u l

- + L ^ ^ ^ - ^ 1 L I l u E U r r a l

orthogonally

-^ir y a t r

nf

onto

directions:

orthogonal projections,

pairs of directions

n : i r P o r r

r

O f

tlle tL hr reL nr

two

between the spaces are

r ,

nririno

P o r r 1 1 1 6

- 3 9 -

obtained 1{hich are Inutually orthogonal. ,\ccordingJy,by taking an orthogonal set of Jirections spanningone sface, anJ tlren takjng the ortlrogonalprojections of these clirections in the otlter, there js obtained an ' l ' h e s et w o orthogonal set of directions spanning the other. sets of di.rections are arranged in a set of reciprocal ;:airs ' in either

its

i.n the other set,

reclprocal,

is obliqtre to

one of thc ttr'oscts,

Irach Jirection,

the unique

it

naking with

proper angle betneen the spaces; but it

is orthogonal to every

oiher

the

,u lf ir rr ! er e r v t r it n t n

Thc construction of

in

directj-ons

for

their

the

to

a gencral

8.

Total- reduction

in

of

canonical

di rcc L!ons

other

in

tliat

in

angles

made ity the

canonicaL

in

angles

lvhen togetlier rti th

to

exception

reciprocal

o{

cich

pairs

of

every of

s;,aces'

at

in

oblique

the

di recti.ons .

tlrat

one in

every

the

other

nlost cne.

fhe

define

tile

tlre dccornpositicn. canonical

c li l e c t i o n s ,

be tlte ortltogonal.

si'accs, and thus 'l'he

any

to

dircctions in

resolved

are

orthogonal

oL tlre SpaceS, nust other

they

property

c r b l i c l L r ec a n o n i c a l

appears inr,erliateil,- that

projections

the

onc space is

betrieen the

such a reduction

notn'to bc exzended

be consj.clered tctal l)' redLrcetl

sfrace, ancl also

space rn'ith tlre possihle

It

to

each other

direction

forn

snaccs.

to

canonical

tvltich is

bases the

sp.Ices constitutes

isogonal

spaces is

of

rclation

into

pair

8ct.

ot|er

orthogonal

such reciprocal

deconposition,

total

A pair

of

in

i-tS Orr'nand

in

botlt

canonical

form

angles

are

- 4 0 -

the proper angles, uniquely defined;

with

then identified

deconposition,

so in every total

the canonical angles ob-

tained are the same, and given by the proper angles, the squares of the cosines of whi.ch are determined algebraically characteristic

as the biprojector A total

reduction of any oblique pair of spacesmay

now be obtained by the following principal

values.

algorithm.

reduction of the spaces, into

projections

in each other, .t x. . = o, x

Then there is

f "y

s c h e m eo f d e c o m p o s i t i o n :

where the components €

X r c

'

orthogonal

R - . x y ,. L_. y. = e _. .y 8_ .x. @- , ?_.y. x. . . .

= ? f . * , o , u y L *=

"*ty

their

take the

and the orthogonal residuals:

@

the further

First

€

tr, o,

3

Y r d

are isogonal; in particuiar €.*,o

C I

-e. r e

i dcnf

i e2l,

Yro

heing the intersection uy L ,A .r v^ = E^, .n E_ . -- e /

but all total

'

ro

o t h e r c o m p o n e n t sa r e o r t h o g o n a l . N o w t a k e t h e

d e c o r n p o s i t i o no f t h e i s o g o n a l p a i r s ,

orthogonal reciprocal

pairs of directions

into mutually all

making the

sane ang1e. Fina11y, take any orthogonal bases of directions in the residual

s -p a c e s f f^.l- . . ,n - . . . , . A 1 1 t h e d i r e c t i o n s y^

thus

constructed have togethel the property that they constitute a total

reduction of the spaces.

Itlatrices U = \1*r, or order Nxp,

5xq

where r,

\t = Vy. s are any regular square

matrices of order p, q are any bases for

the spaces

A 1 ar

-

= F I'v 1 v LFr^ J,

-

F = f-v^ ] . r h e y a r e o r t h o n o r m a l b a s e s , c o n p o s e d

q q of orthogonal sets of unit

.l U'U = ,

\/a.fnrq

i f

V'V = 1,

in which case the orthogona l p r o j e c t o r s

on the spaces have

the simpler forn

= U U ', e . , = V V t . ,

Tf

u'V

mnran\rpr ^

v

v

v

v

r

t

0 \

u,.' f : , )l

,J

( .

\

w h e r e1 > .

u,)-

pair of hasesfor the spaces.In this case (Ui, Vi)(i will

be normal reciprocal "*

...,r)

pairs of vectors on the spaces, with

Ia y -= fLr, r, . . . r r, fr Jl t 1r

o

= l,

^

F

"y.-*

-= [Ur "' l , . . . ,

v J

"fj

and

A

xy

The problem for spaces is,

constituting a total

reduction

vu ql J

of a pair

a pure matrix formulation,

of

that of

the bases, with which the spaces happen to be an equivalent canonical pair;

sinultaneously pair.

r . '

= f u L t r r * 1 , . . .' ,. -t 'rp rJ ', o^ t y x L V r * l ' " "

to give it

transforming given, into

= =

rotating

alternatively,

orthonormal bases into

to

a canonical

The analysis which has been given shows the possibility

of this,

together with an algebraic algorithm for

Alternative

the realization

computational procedures are as fo1lows.

f i I

Q i n n o

\-,

' x M" x '

l\4t

' x - y l"\ ,x1

Nlrc

e r e A n a ' i r o f s v m m e t r i cm a t r i c e s . o f w h i c h t h e f i r s t poritive

is they

a n d t h e s e c o n dn o n - n e g a t i v e d e f i n i t e ,

definite,

can be simultaneously transformed into

matrl-x, and a

the unit

n o n - n e g a t i v e d i a g o n a l m a t r i x ; t h u s , r , ' i t h s o m er e g u l a r s q u a r e natrix o, ^t\l t

\ t'

=

xt"

1'

, s. , t' \ 1xt ' ey. , 'V x. o-

I Z , u

=

n\

t

l,

o /

\ 0

w h e r ep i s a r e a l d i a g o n a l m a t r i x w i t h n o n - z e r o e l e m e n t s , of order the rank r of exey. Take LI = Mx o = (tJoul), V* = erLJ = (\': VT) , f h a

n r r f

i f i n n <

h o i n o

a t

t h p

. t h

. ^ l r r m r q '

c A

i h . f

U ' U =1 , u ; v : = r ' l ' r '=l u z ; and take Vo = t'f u

-1,

so that

U f , \ t o= u , Now 1et B., be a base for \ - T * = \ t , ,P , , 1

y

t '

V; \ro = 1.

the nu11-space of ltlre*rlr; and let

t h i s b e i n g a h a s e 1 - o rt h e i n t e r s e c t i o n

of t,. rvith the orthogonal complemeno t f Ex , €..O (., ) ^ ) which has heen seen to be orthogonal to d*. Lct \', be "r ^ "x{ an nrthnnnrmel a n. { r r i v a l e n t o f V i " ; a n d l e t \ ' = ( V o \ ' l) . ThenU,V are a canonicalpair of hases, havlng thc properties

o

u r l ; = 1 , v ' v = 1 , u , \ /= I u

(ii)

If

obtained, it

some orthonormal base lJx of

g-- has already been X

can be rotated I'y an ortlrogonal transformation

w h i c h v ; i 1 1 s i m u l t a n e o u s l y t r a n s f o r r n U x ' e . , U x +L ^V

v

Y

Take ll = U" n , n o r e s i- m l v, " ' rn' 1

\.

0 )

\0

Y

\"' = e.,ll, and proceed as in l

e. - x = l J U t .

I : ^ ^ ^ , . ^ 1 UaABUttOI

(i),

f ^ - Mlil.

where now,

-

A D tJ

-

To obtain an orthonornal base U, it a set of vectors 11,, ..., r

is

required to rcplace

n 4 -h y a n o r t h o n o r m a ls e t U ' , , . . . , P

+L.

in the .tn ,tage, tlr6[\11, ...,

in such a way that, (r = 1, ...,

tl^

Mrl = €.

P); in other words, to carry out the Gramm-Schmidt

o r t h o g o n a li z a t i o n p r o c e s s . th

stage complete, and the orthogonal

A s s u n et l r e r - 1 " "

f h n r ^ 1 6 . f

n r

-r_l

a

- ^ r F. . _ l

^ .

by taking 'Nr and

stage is completed

r r f cn n om nm p ua tr le c lT. h e r ' "

_1 Z p g = i' N = ; . l v t ' N ' t 'l ) ' " r "r-l"rt r'r'

mnrpnrrar

e

r

= e

+ lJ U' r - t" r r

9. Analysis of configurati on T w o p a i r s o f s u b s p a c e so f a E u c l i d e a n s p a c e m a y h e consideredequivalent, or the spaces in the pairs to present the sameconfiguration +l.^.. tney tL hl t eg

^ ^a- n C .i

I lm l L rd o B qe

A linear

LD ^e vn r f

-r il ^E i l C 'l tl r r tL hl l cq

vn Lt rhl Le

ynfafa,l

r

r

n n i rr P o r

i' n, !f vn

nn r !p! L v il nr !n i , l- ul aL nr e

iu lr nl ud! c r r

^-

4 l l

if

to each other,

in thej.r relation

r

a - +L1i l-u 6 u l r o r

v r

r.'itl" "..,I I

One

pall

+L rr a n S f O f m a t i O n .

transformation which preserves orthogonalLty also

p r e s e r v e s a n g l e s ; t i h e n c e t w o e q u i v a l e n t p a j -r s o f s p a c e s h a r - e total

reductions tn'ith the sane canonical angles. For, an

orthogonal transformation wlrich sends one pair of spaces into t h e o t h e r s e n d s c a n o n j - c a 1d i r e c t i o n s canonical directions it

in one pair

into

i"n the other, with angles trnchanged.

has already been settled

that different

total

reductlons of

t h e s a m ep a i r o f s p a c e s a l w a y s o b t a i n t h e s a n e c a n o n i c a l a n g l e s , oirren BrvLrr

hu w /

fhe

yn .r "n t n, e . r

:noles:

anrl

nnu

tlrc

ntreqtion

afiSeS

3S

- 4 4 -

spaces wi th

to whether different

each, are equiva) ent;

thus uniqLrely definecl for anp.les wilI

spaces.. It

is

only

f o r n , , i n t o r v li c h p a i r s

nical

:rnoles can be rotateci . pai rs

of

ae rnr oL l r n! c

spaces are

h o f r i e p n

t h e n

It

of

so that

the

of

the configuration

necessary

to

present

a cono-

spaces riith given canonical

rr'i11 then be establisl-red that

ec.tti-valent j f

and only

e r e

n Js U

t h c

Js (a r m, .n, 'Lt

tLhl c' c

i f

of

two

tlrc profier

n rr nv nl re( r I I

spaces givcs a conpletc characteri:ation

L ugLhg5ll

2 Inl ^Hl r^E- J d

their

configuration

rrnder ortlroportrl t ransforrlat ions.

invariant

Nornal

10.

anples,

analysis

a conplete

constitutc

fo:rtred by the

t h e s a r n ec a n o n i c a l

form

An orthogonal detcrn'i ne,l l-,v p o n a l s et

of

5",....

in

transfornation

a fuclidean

space being

corl'csnon\lcn('c hetr.oon nno cnnnlc'tc ortlro-

rrnit vectots

and anotlrcr. lct

t l e c o n r t li n a t c

v e c t o r s o f t h e s p a c e b c t a k e n a s o n e s u c h s e t ; a n c ll e t other be clefinedas folloris, in relation

tlie

to a given pair of

s u l . s p a c c s .T r k c t h c v c c t o r s . i n a c a n o n 'ci a l f a j - r o f l ' a s e s ; 1

ha

to thc j.ntcrscction,

t)rc duplicatcs arort thcse, beicnginl

lct

.--^.-^'1 .

a

fol

\

l :

v

l

thc

\

v

.h.l

'

arron.nnt tlo

orthogoral

con,plerent of pair

and 1et every recilrocal. ninecl by s6s o = ll '\',

of

orthoncrnal nrkcs

\/ectors base.

n:irs

nf

ll, \'naking

tlie

spaces;

an angle o,

deter-

be replacecl by an orthogonal

pair

- \' sin

o/2.

tlrus obtained

constitute

tlrc f ivcn pair

r , ' i t h c a n o n i ea l

the rrnion of

U cos d/2

'llie

Of tltOnCfn.al lraSe

L,a an

+ \'' sino/Z,

U cos a/7 The set

ren:inJer

orthoconal

of

transfornation

sl'accs tlre inaf.c of

I'rscs in thc

h:str \'ectors not

in

the

second thus

a pai r

[ol Ioriinp normal [orn: the

iltersection

defined

of

sf aces

reciprocal

have the

forn

- 4 5 n

0

'o/ d/2

cos

2

ct/)

-si:n

c'/ Z

0

0

0

0

wlii le al l

others

l.-

t h e fo rnr

\re

0

\ 0 l 0

;l l-rrerw

nair

f.-.,

o{-

crrhqn.req

li'ith bases in

rr rl

('+^+:

.

J L d L

'lhe

J U t l o l

ic

tlro

ortlrnoon-rl

irece

--

of

a lalr

such a f orn.

)

,v .( l r i a t i o n

canonicaL Cirections obtained in a total

redtrction

o f a p a i r o f s p a c c s h a v e b e e n c l i a r a ct c r i z e d b y t h e p r o p e r t y that those rvhich form oblique Fairs a r c l " e c i r r o c a l s . tlrov

arn

spaces.

tlre

orflrnoorr:l

Now there

cllaracterigation.

nrnioi'f

ions

of

each other

h,i11 be slioh'n a di fferent

in

in

tha

the

L.ut equivalent

in a pair

directions

An angle s madeby certain

of

s p a c e s w i 1 1 h e c a l l e d a s t a t j o n a r v a n g l e b e t w e e nt h e s p a c e s if

it

undergo constrained

stationary when the directions

is

appears that the stationary

in the spaces. It

varlation

the proper angles between the spaces;

angles are precisely

characterized as reciprocals

and that directions

as directions

characterized equivalently

are

which make stationary

angles. Thus let l I = h c

e n w

r r n i t

rl vn Jc

ir rnr a\

lP

t

Y.

^

, \'/

= M s

y

in the spaces, making an angle with

v c c t n f s

r^rhAre

l \ l r

s are noh'vectors of order P, Q.

t

Then rrI'l rl\1 r

-

I

s t M ' N fs =

t

v v

and

r'V|\lrs = u for

The condition variation

u to

the

under

constrained

is

), Il'\l r. r x x

fl'lrl s = x y where \r,

be stationary

N 1 . ' . 1 t 1 _=_ rI - I l . ' . N i . . s , /

^

>

l

/

\ s a r e t h e L a g r a n g i a n m uL t i p l i e r s

to the constraints on r,

s.

It

dyPvdL

)

ilvw

,

corresponding + L ^ + LllaL

trr =u = trs B u t , r v it h

l,

= I

s

one of the co straints

w h e n c e a n e c e s s a r y a n d s u f f i c i e nt stationarity

under constraint /- uN{'r{

I

x x

I

\

tynt*

Thus the stationary

l4'M

i

condition for

the

S

\

x y II

- u l r l' \ 1 ,

b e c o m e sr e d u n d a n t ;

I / l

(, r'lttlM*s = 1.

t , \ S /

values of ir are those which satisfy

- 4 7 -

f h a

a n r t a f

i n n

- u r\Iit\rx

riiNly

l\!t t\t t ^ Ilrrf

i n

of titc idcntity

\r l Clrl

, .2li t- uJ

tI - u l . l\' t rXI II i

- , ) l * q i i . i ' r r *l l r ; l r , I l u 2 r - " * " ,I

=

I1'I1 I x y i

); ' I\1 y x

I

this

- ,,r!r I! P ' , , . , , , l l

I

: u itl|Il,,

I

j

e qu a t i - o n a n c l t l r e e q u at i o n

u

2" l - e e

have the same positive The proper

angles

roots,

\!ith

sane l,Lrltiplicities.

angles

titLrs identical.

are

analysis

th'o n'u1tip1e factors

o\nerirrcr't in uhiclr tiicjr

x,

y

correlaterl

s f a n s q - . = iL fr lft r Jl .t ;e .) '

lrave canonical l.ases U = Itl*r, \'= to

the

and tlle statlonary

12. Canonical correlation Consider

t ,l = [ )

x y '

in

an

= [, -f" 1y ' . . ]

Ilrs, Tlrcy are cquivalent

factors u = xr,

t{llich I'ravell,

\'for

t r l , , ,= I 1 * ,

llrc clcnrentsoI

r r,

their

Ineasurement natrices:

= N*r

= [,I,

r',]cfinc

They have the property tlre pairs

rr', \''

(i

v = IS

=

that l,

lrt., = )lr,

= ilrs

= \' .

c a n o n i c a l c o n ' p o n c n t sf o r they r)

are

al1

of

rinit

lravepositivc

x,

y,

variance;

eorrclatiorr,

- 4 8 -

g i v e n b y n u m b e r su i

the squares of which are the characteristic

v a l u e s o f e ^- e ). , , a n d r n ' h i c h d e f i n e t h e c a n o n i c a l c o r r e l a t i o n pairs,

coefflcier)ts of x, Y) while all-other

taken fronrwithin

and fron between tlre two sets, are uncorrelated. The derivatlon

o f c o m p o n e n t sw i t h s u c h a p a t e r n o f c o r r e l a t i o n s t l . r ec a n o n i c a l a n a l y s i s o f t h e f a c t o r s .

constitutes

I l a n n a n( . l 9 6 1 ) p r o p o s e s a f o r n t o f c a n o n i c a l a n a l y s i s , a s f o 1 1 o w s , a n d w h i - c hh e t h e n

r"hich he derives essentially

therc is the ortl.rogonal

proceeds to generalize. firstly, ,-tloenmnnqitinn

+R I Iy = l r i r x x y w h e r e I I I . R= 0 , B y a w e l l - k n o w n g e n e r a l t h e o r e r n o n n r a t r l c e s , x ' = r.... o6Br, uhere o,B at'c orthoSonal nratrices and 6 a -\/ r;^^^,.-1

ur4Evrlo

-a+ri-

Tlren

llyB

= f'1*ct6 + lL B

I l' ) ' n

= rl "Xq 6

That is,

where S = RB ;

regrcssion

Ilannan, xo for

t

i' s) X e

also d l J U

the

so thar

,

1'B on l.o is

sense of

1i^^^-^1

u l d Y U l l d l

t

r*o

give

,yB

lliL

=6

= 0. Thus

diag,onal. According

t l t c c a n o n j - c aI

thcrcf ore so as to

a nornalization in

S = o'}liRP = O, since

)l*o6

n:atri.r. of

yP a rc

,

woulcl be true rt y B -

l{io

I:ioltly, = lli^

Therefore, the

sc that

+ S.

each uttit

IIotelling

if

to

c o n ' p o n c n t s, h u t Thls

variance' and only

if

^ - ' r +( rLr 6 ' 1 9 a r e n o c o r r e l a t i o n s

d l ' \ r

b e t r r ' e e nt h e x o ' s a n c l b e t w e e n t h e y B r s ;

and this

is not

g e n c r a l1 y t r u e . T l t c c a n o n i c a l a n a l y s i s d e s c r i b e d b y I l a n n a n is therefore apparcntiynot the classical one of HoteIling.

- 4 9 -

P A R TI I I

'I

.

Ohl innc

nrniect

jons

Any vector in a Luclidean space has a unique resolution into

a s u m o f c o m p o n e n t si n e a c h o f a s u p p l e n e n t a r y s e t o f

s r r h s n a c e s .t h a t i s a s e t o f s r r h s n a c e st h e s u n o f t h o s e dinensions is the dinension of thcir

union, atldwhoseunion

i s t h e r ^ h o I es p a c e ; a n d t h e s e c o n p o n e n t s ,o b t a i n e d b y I i n e a r transfornrations of the vector,

define the project-iotts of tlre

vcctor relative

t o t h a t s u p p J e n ' e n t a r sy e t . C o r r e s p o n d i n g i Y ,

the identity.is

resolted into a sun of projectors, the

i d e n r p o t e n tl i n e a r t r a n s f o r m a t i o n s w h i c h o b t a i n t l r e p r o j e c t i o n s , the range of each being one oI the spaces, rihile its j.s the union of the rest; n r n i o e t n r q

i c

n r r l lI .

nu11-space

so the product of any two of

a ^ - " ^ - - ^ l r ' LOlIVeISel.

y t

ally

i c i e n r p o t e n t ss u m ni n g t o t l l e i d e n t i t y

^ + >€L

^ f , UI

1 1 . ' IruLudrr/

these

^ - - ; L ; 1 ^ + : " ' dlrlllrrlr(rLf

are the projectors

'rb,

on tlle

s u p p l e r n e n t a r ys e t o f s u l r s p a c e s f o r n e d b y t l r e i r r a n g , e s . l i i t h a n y i r l e m p o t e n t ,i t s c o m p l e m e n t a r y ,a n d i t its

ranse narallcl

projector if

it

is

range and null-space are

iCentifieC with

tn ifs

n r r l l - ' sr n a c e . I t

thc projector

on

iS an oltltogonal

is synmetric, in which case its

range and

null-space are ortlrogonal conplcnents; and otllerrvise it

js

an oblique proj ector. Ilore generally, rclative are just

t o a n y s e t o f s u b s p a c c sw h i c l r

independent,and not necessarily supplementray'the

s u m o f w h o s e d i - m e n s i o n si s

the dimension of their

sunr,not

I t e c e s s a r i l y t h e w h o l e s f a c e , t h e r e r n a yb e t a k e n t l r e o r t h o g o t r a l projection

of any vector onto their

union, and then, with

vector obtaincd, tlte o['Iique projcctions

Afrtet:

StualleB

the

on the supp-lenicntary

- 5 0 -

set which the spaces forrn relative r* ur t2 v 2 4 !n L lv

\rtr.tnr

is

one of which is

intO

feSOlVed

In this

to thei-r union. nair

a

into

resolved further

oI

coTrnnnnntq

orthoron:]

a set of oblique corn-

ponents in the given spaces, while the other belongs to the orthogonal conplenent of their If thon

union.

the supplenentary spaces are nutually nrn'iectinn

nn

eech

snAce

relative

to

orthogonal, set

the

is

the

s a m ea s o r t h o g o n a l p r o j e c t i o n o n t h a t s p a c e , t l r a t i s t h e nrnicctinn

nr

Spl i t Thn min:tion

n f' nn

now ariSes

nv r' "nJi e C t O r s

tlroir

rrnion

terms of I f

to

in

two spaces

Ex =

-

its

nrthnonn:l

exnlicit

to

^

rvjth

-\

rosnoct f

otheli

to

comnlonrpnt!

v

| ] l l , r

t"h"t r v/

relative

x

nar:l

t].a

-*a.i6-r^t'

lel

t^ ^f

,

fof

whjch

by

a n d , e Q U i v a l e n t l y ,t

/

r tx

relaf narcl P'

irre lol

fn rn

in

spaces.

'a l l ' l " t " 1l tr I II '"

there is defined the projector, which maybe denotedbI n v n, r te

to

A fnr1pgla can be

Are senerate.

given

qn.ces.

of

al ternatively,

on the

deter-

enmnl pnrFnt2TV

2Tp

the other

the

mejr

the

e

o,

a

foI

r ' l Ey = harJ on which

are c are

to

ir:st qcnerAfo

projectors

e - e . ,1 *

formrrlee

the bases, or

r l ll.lxJ ,

criteria

It

thc

terns of

nrnicetors the

condition

to

on one parallel

the orthogonal

nrtlrnonnrl

aS

r-nsc thnw ern

in

givcn directly

lel

defined

olro n:ral1c1

more generally,

or

n:rel

pro jecl-ors

orthof onal

nupstinn

ci ther

qnAcF

thrt

their tllg

rrninn

.. rx

COmplenentary

that

,y,

SpaCe

n v'

"*lu, is giVen

b y t h e u n i o n E - - O i ^ . . . o f 8' Y .. with the orthogonal complernent Y-'x,Y r' x r y

r5 x

of "

tlreir

lniOn: '

JS vO

e L x l, y

and nrrlI-qnrce Ee;..... a e r ' \ '

t l

-i S

tho

,'nin,'a

i,la--nrcnt

uifh

renoe

eithe

-51 on the spacest

Then, in terms of the orthogonal projectors there is

the formula -

"*[y for

( ' l

-

\'

o

o

"x\,

-

c

" x e- y l,

more immediately in

or'

the determination of u*lyi

( 1

l - 1o

"x.y,

terns of bases, and in a form which directly

generalizes the

formul a 1

= V* (l\1i]4x) 'Ml

"* for

on a space in terns of a base,

the orthogonal projector

there is

the formula e-

From here it

= M' x tr N' x, 'tY' e ' - 1 u' x' 'EY ' ' ' x- 1 M

x IY

is noted, incidentally,

'Z y o' x l y

=

that

a-

".y t*

The sum of the complenentary oblique projectors union is

nei r of snaces relati-ve to their

on a

the orthogonal

nrniectnr on their union: =

* Now any vector

z =

Z in

"*,y

"yl*

"*ly

t has a unique resolution

* 6*,rt * "*r>'z "yl*Z

' i n t c r c o m n o n e n t s -i n- - t- x ' C - -r y -X - . - y. - a n d C If

C-., t.. are, ^ y

complementary,

noreo\rer,

€ -A- r. _. .I = t

e-

and

^ t f

.,

so that

= 1,

sirnply that

for which the condition now is

P + Q = N, rhen c

e " Y l x a r e c o m p l e m e n t a r yp r o j e c t o r s ' of spaces; and in this

to a complenentarypair

'a * +y |

p

' y.

=

lx

1

on and parallel case

- 5 2 -

projectors.

3 . I t ' l u 1 t i p 1 vs p l i t

p, e, r,

e -L, ' . . . b e a n y s p a c e s o f d i m e n s i o n

€.., y'

Now 1et [-., x'

which are independent, and so forrn a supple-

...

to their

nentary set relative

union ,*,r,r,,.

of dimension

Then there is determined the projective

p + q + r + resolution 7

"

- =- x | yc , r ,, . , . 7'

cf any vector Z in t andE-... L r Y r L r

+=

y l cx r z , r . , a. "

!

+

' ;x , r -t z , .2 ' ,

i n t o c o m p o n e n t si n € * , e r ,

,

,

The orthogonal projector on the union has

" '

the decomposition = * "' "rl*rzr,.,+ "*lyrzr,,, "*ryrzr.,. t h e s u r no f m u t u a l l y a n n i h i l a t i n g p r o j e c t o r s , o n e a c h

into

para11e1 to the union of the other with nl omonf

^f

fho i r

the projector

gnl6n

all

ThUS,

tOgether.

on C.. parallel

the orthogonal corn€_t ., . xlY'2

it

""

t o t h e c o m p l e m e n t a r ys p a c e

given by

t ), , @ -c , e . . . C -I XEr I -r .7 r, .-. , '; rn.l

m n r ar nv rvr a r!

L

r

t

= o' "*lyrzr,,, a n' y b a s e o b t a i n e d f o r C - . , A t J t

" '

)

Sqga1.e

rank e*rYrzr,,, = trace If \l ' "\ {x

I \' ly

M--..

x t I : "

ls

M

""'xryr.,.

t h e c o r n p o n e n t so f

ic

nhtainp.l

aS

a

fegUlaf

then matfiX,

the inverse of which, when transposed and

c o n f o r m a b l yp a r t i t i o n e d ,

define matrices N-, N.,... ;

" ' X r Yr . , . '

thus ; f' M x My . . . l -xt r Y r . . ). -' 1 = ( Nx Ny . . . Nx , Y t , , ), '' . The projectors

which have been considered can be computed thus: r Y , 2 , , . . "€ x 1

=

M

" x N" lx '

"'

5 3 4. Partial

regression

Consider three factors Xt Y, z and the regression of z on the factor

c o m p o s e do u t o f x ,

together.

It

obtains the

resolution M " z = e" x r I r ' zl + 6" x r I M Fnr

thc

rForcqsion:l

nert

here

n ' , -=

^

f,,

there

ll

''"ry

the separation

is

t*ry;t

"*ry = L" .x, - l xr l ,

into

further

parts

Y'z

+ l\{ " Yr -

Y lx , z

corresponding to the

nc

rf

i t i nn

\' 1 ' X r Y = r\ \' 4' x M . , ) ; .r an4 conformably, r

x,Yiz

=/'*lt,t\

\,r1*,,/ Now

* Yryl *t*l y x,z ,z is

the regression of z on x, y together, given as a sun

o f c o m p l e m e n t a r yp a r t i a l of z on x partially

regressions, defining the regression

with respect to y'

and on y partially

with respect to x. It

is required to have a formula for

the partial

regression

m a t r i x r ^- l l , , ,L b e J o n g i n gt o t h e r e g r e s s i o n o f z o n x p a r t i a l l y I t with respect to )'. Splitting

the orthogonal projsqtor ex,y

into oblique componcnts,corresponding to the resolution its

range t*,,

t n t o c o m P l e m e n t sC * , t

= *V, y '= , - c " *\ l yl ' t + '

e

:

' y \l x l ' z '

of

- 5 4 Substitrrting fron the formuja rvhich has been given for snlit

follows inmcdiateiy that

r r r o i e c t o r " s .i t

-

= (l!|erl'!*)

t*1y,, ,-lirneilw

- l

' l ' t i -e * l t i ,

tlrc cnrreqnnnrlino

o e n n r qr lr ir -r ia nl o E ;

u r r L L L r /

the

, formtrla

[or

a

total

regress ion. forn'tr1ait

Irrom tliis natrix

Ir L^ Lo .r la! qJ cJ ri n u rn ,

n r l t r i r

n f

-z

repress ion

r e [ r , r e s s i o t tn a t r i x .

can be cxflressed as a total

n e r f ( I i da r l l r d l

to y,

seen tl)at a part ial

is

n - l- l

A

a p p c a r s t l ) e s a m ea s t l r e t o t a l

P d l

: ^ r 1 " I ) L i d l

. ' : * h h l L r r

The rI o s n e c t

relressiotr ntatrix of

tlrc residuals in tlre reEression oI z on y on t])c rcsidlrals in tlre regressj.on of x on y.

for

these reslduals fornr the

matri-ccs * x

-

*

-

. ' ! l' i " = c "111' " - x ' 1 " z, 1 " =- eY |" 7

'

u l r i e l r r - n n s i d o r e , aj s t l t e n e a s u r e n c n t n a t r i c c s o f ,^ = 2 -

z0),

i*

= s - s(yJ, cive r*x."x ^

5 . Inversion

, L

= I_1., -, ^ t )

r L

anclDartition

z = (x,y)

Consirler a factor Its

Iactors

neasrirencntrnatrix has the

corfoscrl of

sulrfactors x,

narf

fnrn

i+inrp.l

=

"" !

.

y.

(I1.1.,,) \

J

a l r t l c o r l ' e s p c n t l i n g 1) ' , riz))z

-

/r.'x"x

I \rr;lrx 'llro

nrrncf

inn

n o n 1 s t o o L ' t . a i . nt h e r' l , 1| \ ! ) 7"7'

-1 z z

where i i = f ' \ x \ yl ' z

li'

xl\ly \ ,

,'j-"I

inverse in a sinrilar forn

- 5 5 -

confornably with Mr, and moreover, is

is partitioned

such

th at

N i M ,= 1 . It

that this

i s n o t i - m m e c l i a t e l yo b v i o u s ,

though it

can be verif ied,

s c h e m ei s o h t a i n e d b y - l . l, y r, "vy' E N . . x- -= -ey ,t r, xt [t ,r ., '1* ].ye -, 1- l/t - ) - 1, N - xr ' 1 . . ) - 1 . "y = 6 x t

It

is noted that,

f o r a n y m a t r i x t l w i t h i n d e p e n d e n tc o l u m n s , of matrices of the sameorder such that

there is a variety

- 1' = N ' N , N ' M = . 1; ( l ' '1M )

for

a special example, -,I ', N = U(Nl'\1)

exanple, as just

and there is a further ,w.a, . +L: trr . d^ r-r .j ,l

n a + +L .r i *L ;r l O n i n P d r

COlUmnS Of

the

indicated,

associated

14.

Now Nl- \': = I'l-.N-'- + I|..N.'., y y' 7 z x x which obviously

n rn jp6tnrs y r w J l u L v l J .

4--ihi

ll daL tI ir nr tcs

snl i t

nroiectors.

d t l l l l l r r

one projector

gives

e '

Tr ihl cL rnr

there

x ,l_ y

=

i s

V \ 1|

,

ri rnr

as a sum of

Vi-ew

made the

Of

tlre

two mutually

fOrmUla

fOr

identification

= M,,N,', ,

c

x ' x ' * Y l *

, )

and hence, though otherwise not obviously, e- = i'lrNi C i v e n a r e c t a n g u l a r m a t r i x \ 1 w i t h l l u { ' MI I 0 , i t according to the definition - 1'

inverse given by \1

is,

whi,Ir is

thc

The natrix

transpose oI

thc generalized

h e s t" h" "e nF r' "orn- e' -r- t- i e s - l

(fi{'M)

N defined by

= N',

NI that

of Penrose (1955), a generalized

- 1 = (M'M) 'NI' - l

has,

= N'N

, N'I'l

= 1.

inverse,

- 5 6 -

to make further

is now interesti4g

It

observation on the

fo rn o f

( l u ; l \ 1 21) = n - l N , , N l l l , = 1 , w h er e N, = (NxNy)

( ) . l x l , l y ),

Il

Thus, R l "' X1 ' l 'l \ Y = " "1 x "' Yt "x = o 'Y

are the rcsiduals in tlre regrcssions of x, 1'on eaclrotlier; and 1 |

1 |

,n . x -- , I r l x r , I ) . y - - . rf , ly

h. v .

Ivnerimnntnl

L / \ l \ r r r , ! r r L u r

TLtL

^-., dtLf

^-.-^-; sAI,sr

a fictitiorrs

r

fr h" 11 v' ^v 6r r d

1-ho

n,pn t

evncri

u r r \ r

rnen t

^ 1 , . ^ . . d r r \ d /

facto

rrrrrlrenop,lv

nnd m , , Lc n n \alucs

ru rr r nr ri vf a nl rrr nt ' i l v ; + I L

i ^ l >

s nossil'leto cntertai

t o b e C e n o tc d h 1 ' I ,

r

it

ho

rrhiclr remains

define s the unifornr factor

k i n g the valLle 1 o n a n y o b j e c t . I t s

rJeasurelncltt natr

thus

i ^ A J

i r -

t 1, = / 1 \ t l l

I ' l

tt '' ll \ r /

the vector with N o w c o n si d e r uniform factor

^ l

t

+1.e

i

+ ^ L >

g^ 1 a^ -q ^t !" t .g+l l^

r e g r e s si o n o f

I , thu

I,I

X

c -tri I

x

*

t

i

lr x

Since

,\ 1. rI l't' I it

L5

follorvs that I

N;TI\1i

equal to 1. anv

factor

x

nn

thg

o

-

D'J

that

and therefore

e r l r l *= I r l r i = n ' i ( r l w h er e Y "

1 N NIl 1 X1

' rx ' T-

=

(I')

of x in tlre cxperin'cnt, it

defines the mean value jts

^

being

c o e ff i c i e n t s o n t h e u n i f o r n f a c t o r ;

vec,o. o*n

e x p e c t e d r r a l u e o n a n y f r - r r t h e r o bj e c t ,

and i-t gives its

unif orrnlty. the residuals 6rlrl* r e g r e s s i o n m e a s u r et h e d e v i a t i o n i = x o f x f r o m i t s

s u bj e c t t o t h e f i c t i t i o u s in this

mean, tlius : -

l t

^

x-x

7/

r

l ' r r l fL r iPnr \l o

'[he of

r^ n! oA rr P! S S

r : u r

rr-'gression

x

iOnS

a factor (x,y,2,.

w of

a factor

of

composedof

dimension p+q+1+

X r yr z t . . .

l l

r

.l

subfactors

d i n r e n s i o n p r g r r r . . . h a s the form

of

( - , l r \ ^ , J r . . ' ' r v \r ^ r ) , '

^

, w

/ ; L

s

. . ; r * Y t y X r z . . .; w

r v h er e

xlY,z,

';l!'

xrY' "

Ylx,z,

Just as the total

l'rasthe determi-natlon

t {

-

"'(*ryr...)r hu rt r

n , uarn, vn rc r q v rv t l( r, n

l

nrniee

t

so the

inn

--

rt r \I X I

r

xl Y, z, . . . ll{

E

XrIr

XrIr.'

"xl y ,z

parts N1

h ave

tr1 the deternrinations

- 5 8 h u rl r

n w hu l r r iYn ur rs n

nrnioe

tinnc

lrv

, l \e tLo\ r m i n e d u

,

tlre

o b I I o,u c' J)1'o J e c t o r s

into which thc orthogonal proj ector is spli l - r a v eb c e n o b t a i n e d i n

t

But

these

the f orm -

" * fy , r , .

n t

' ' ' xl i' l t x t

.

..'

so that I lr , x \1 y,2 ,.. whence

flro snt

nf

=

t\i 'l \ x1 " \ xi ' l '

rcoressinn

nrrtiel

' r '

t

nntrices

are obtainecl

i n t h e fo rnr I

R u .

\ l r ra ltL ri n l nL , ' !n vnr fr ai cf r r r r t i n n s t , r

as varioLrs binalf

appJyinp. to factors

any Iair

of

througlr their

a pair" of

harre been considcreC,

rclations

spaces, anJ tlrereforc sf ans;

spaces attajns

to

or

or

tieparts f1.on then,

dcpartrrr(' J:ron thcr: fi\cn

coefficicnts

arc

of

l'y ccrtain

tlre spaces.

flrtl renain

for

rOIC

thc

jnvari ant

attainn'ent

cocft'icicnts,

l'lrese rclations

consic'lercclfr,rnciions of

forn'ctl b;' tlrc slaccs,

so the

l : o l . li r r p b c t r i e c n r .

ancl tt'],ich har,e criteria

togetller,

c -elf i n c ' 1 1a s f l r n c t i o n s

har,e been

i n r t l . i c l ra n y c o r I i I u r a t i c n

spflccs, b1'definiril: n-ilry rclations,

to

any fair

t l ' n e n r . fi o U f a t l O n s O f t l r l e i ' O f

sArc pi olrt lrc rlonn fol

spaces taken

to

and coeffici-ents

t le f i n e , l u i . i c i , m c r s u r c t l r t ' e \ t e r r t of

'

r , \ .

Just

of

r \: ^ ' X ' t.l' 1 r '

I

X I Y' 2 ,

attcl

configrrrations

un\lcr ortltoqona l

transfornations. The nrost obvior-rsnLrltiple for

a r r y r r u r n l r c ro f

qnpci:l r{^*^ lluIL

thrn

this

1 + l ^ l l L i l L ' , I ( r t( l l J l l

sp6qc's is iS

relation

to

be conslderecl

nutrral ortlrr:lionalit)'.

tlte lelation

+( |1L. ; ^ ^. rlr'llu(rurl ur

of

):ore

ntutual distinctneSs.

.r .j ^ Lllr)

- - 1- r :^il ld( lulr,

. n l ,i cl ,

i s

- 5 9 -

of linear

the occurrence of a coincidence, is a relation d e p e n d e n c eb e t w e e n t h e o r t h o g o n a l p r o j e c t o r s

on the spaces,

which may be terined cornposabilitv. An equivalence of the the condition that there exist

condition of composability is

unions identical

two subsets of the spaces, which have their and within

each of which the spaces are mutually orthogonal. o r t h o s" o n a l

+

t, distinct A coeffi.cient will

extremes

composable

o p p o s1 t e s ^-_-/

coincident

t

be defined which sets the configuration

formed by any three spaces in a scale between the extremes oI orthogonality and conposability. For three spaces, composability is that either a pair of them are itlentical, or a pair

are orthogonal and the third

their

uni-on. A natural

the coeffici.ent to any numberof spaces is

extension for readily

is

suggested, and partly

established.

Let the number xtY rz

determine(l for

,

a*, C*,

tr*

'

'r,

ct*

'r,

1

a n y t h r e e s p a c e s s y m m e t r i c a l l y i n t e - r m so f t h e

coefficients

of association between thej.r pai.rs, be taken

define their

coefficient

of dissociation.

lt

can be sholn

to have the property of being non-negative, and equal to if

and only if

to

the three spaces are composable. It

be seen to be at most one, and equal to one if

will

and only

t h e t h r e e s p a c e s a r e m u t u a iI v o r t h o g o n a l . F o r i t

is

zero now if

seen that

- 6 0 -

o < K " = ""

t = t(" YCz - c' ' x Yc" x 7)t z = ( 1 - " cx yz' t n- c" x 2 z) "' .j 2 = -z -2 -

ryr t

J*y)*,

L*yL*,

'ty.-

. 1, < s1,.s?.^ f

^ L

the bounds being obtained since the S and C coefficients

It

follows

equals 1 if all

between the three spaces

the K-coefficient

that

and only if

the spaces permuted.

wjth

bounded between O and 1; and sirnilarly

are

the S-coefficients betweenthe pairs

equai 1, which is if

the spacesare mutually

and only if

^-+L^^^-^1 v I LrruEvrrdr. ^ - - ^ - A ; h ^ r ' nLLVr srrrBa

jr t

o t n * , r , ,4 1 ' K-_,. _ = 0€:Xomposable X,Y,Z = 1ts;'orthogonal. In terms of orthogonal projectors,

composability is one of

the conditi-ons e _ _= e , , , o r e _ e . , = O a n d e . , + e y. . = c _ , x ^ y x z ' ) or one of the conditions obtained by permuting x, y, Now for

any numberof spaces, corresponding to x, y,

d ef i n e v

-

1

C

X t Y r Z r ' . ' c

C

xy

l

xz:

f f

C zx

zy

L

1

;

can be shown that K

XtYrZr"'

> 0 = 0

It

C

'

'

) ^

Then it

z in these.

€J

conposable

is obvious that

Kx r Y r Z , , .

= 1 Q)

elllegonal

z,

-61 -

and it

that

to conjecture

is natural

s 1

xrYrzr"'

= 1-

orthogona

a s w i 11 b e p r o v e d . Fron the identity Mt

N'tr!

N l l\

and the properti,es 1 - efi

, it

with equality induction

that

= l v ' M lNl ' N i1l "rI

li{rM

w hi c h h a v e b e e n e s t a b l i s h e d f o r t h e q u a n t i t Y

f o l l o w s that MIM

! l ' NI L

NIM

N'N

i r r c f

i f

MtN = 0.

M ' M l lN ' N l

(Fidrer, 1908),

It follows immediatelyby

if

A =l A'.'A.,, l

"

I ^ztAzz | t " " " ' is any synmetrically partitioned positive

def j.nite

matrix,

then

l A l - ([ A 1 1|] o r r l with equality

just

if

the non-diagonal conponentmatrices

all

A.. are nu11. 1J

In particular,

w i t h c o m p o n e n t sa l l

i.nequality is obtained.

of order 1, fladamard's

An advantage of this

the most general inequality

approach is

is obtained directly,

with necessaryand sufficient

the most proved results proofs in the literature.

together

c o n d i t i o n s f o r e q u a li t y .

( 1 9 6 0 , p . 1 3 7) r e m a r k s t h a t " t l a d a m a r d ' s i n e q u a l i t y

that

Bellman

is one of

in analysis, with well over one hundred "

- 6 2 An appli.cation of Iladamard'stheory (1893) on the i n m e d i ,a t e l y e s t a b l i s h e s

n'atrix

deterrninant of a positit'e

t h e c o nj e c t u r e t h a t r . v a sn a d e a b o u t t h e r n a x i m u mo f t h e o f J i s s o c i a t i o t t b e t w e c n s p a c e" c-2 x ' -eY ' coefficient X-... r ) r . . . which is that its value is 1, and that it is attained just wlicn tlre spaccs arc mutual1y orthogonal. lrlorcover, tlre t l ' L e o r e m ,. 1i s c o v e r e d b y F i s c h e r ( 1 9 0 6 ) '

nore general forrn of thjs

antl Pu, ( ,,

€- ^* , € - --). , .

K with

and

be

v "*ry,

.,,;arbr...)rrr

tlren this

cocf f icient

jrrst

if

all

i,,cr

il fI

U I L l t U E U l l d f l L j /

J U > t

the

sets . StiIt

L | 1 9 |

)

]

I ies

a set

if

it

Cefined

holds

for

an1'

Ii lK r a r b r . . .r . . / " ^ r y r . . . " I r b , bethreen O and 1; beitlg

:ero

conrllosablc,antl 1

hI Le t w e o n

nrore general

coefficients

SnACOs

talien

ffOnl

can be definecl,

s n a c c s a r C C o n r ihn e d i n t O

a n d s o f o r t h u l ) r c s t t ' i c t c tl y ,

are

def inition

^r,,-..-l^1.^1.rs

,

sets

preliottsly

d r t \ d / r l r r r u l u J

r .h n n q n l q c rf

i n r n r n i n n l r ' rt .

for

=K "Xryr,.. also

the

that

different

slraccs tal'cn togetlter arc

^-+l,a^^h^1i+,.

different

noted

t h c r - e i - s n r a c l et h e

Norv if

in

spaces

holds

conrfosaL;i1ity condition subset.

if

only

shouid

It

orthogonal.

aticlso forth,

4i\.. .. K., l ., ^ r. X r ) ' r . . . Y r D r . . . t r r I r . . , r a r l l r . r . r , . . if

equalitf

sel'era1 sets of spaces

Thus, for

leads to nlore general results.

f urther

S e t S,

l r e s c r v i n g a n a n a l o g o u ss c l l e n e

a t e v e l y s t a g , e , s h o r v i n ga n i n c x h a u s t i b l e c o n b i n a t o r i a l , qelrnnrtie

ellv

I

r! o - ll'\ r o , l r r e t i v e

nrnnFrtv

of

tl'e

rlissoci.atiOn

coefficient.

,.

l.irnits of

a s s o ci a t i o n

T h e r e i s n o t " t o b e s ] ' r o w nh o l v t l t e r e l a t i o n b e t w e e n a p a i r of sfaces, or a fair

of factors through their

by internreCiate relation

to a thircl .

spans, is limited

- 6 3 -

A l r e a d y t h e r e l . r a sb e e n s h o w n t h e i n e q u a l i t y ,2 LxyLx2J I

l.yr

irith equality i I and only i f oenor'2!

It

,liscrcn2nev

in

the

fo1lolr'sthat C.._ is

the slaces are comlosahle, tlle

cnrrr'litw

limited

y L

- S S C C "xy'xz xy'xz r^rharn anrroI i rv

anclonly if

tc

one

-Z _2 -*y)xr,

or

is

with with

r l if f e r e n r - c o f

is

f actor

associated

a third

of

is

limj.ts

is

'

attained

if

gir.,en.

are

for

the

of association and There is

cosine

of

to

be noted

a sum ancl

t h r o u g h j - t s m e a s r r r c n e n tn a t r i x

z,

a nultj-nornral di.stribution

clesirable

futtctions of

of association between a

characteristics

have been defined

to

1

( t . 1 1f r z )

itlentify

between x,

;

the

of

which

nteasurements I'ith

tIe

have been forned

)'i

ther

wit)r paranreter natrix

parameters,

characteriStics

These coefficj-ents

Ilr,

certerin coef f icients

as fr-rnctions of

the distribution

annerr 2s ,lirect

relaticn

the

coefficients

tl.ie formulae

A = and i t

"'*'Y'z'

rnclcs.

10. llistribution a

L

tlre spaces are conposable.

dissociation

Itith

hrr

4 '(-]x y C 'xz 3 ' - x z -+ 'Sr y ' S

z C = "yz

otlter

l i n r i t e c l w l ' i e nt l i e i r

an anology

oirrpn

tl-rus:

In this way the coefficient o r *a^i 'r

hpino

so that

they can

distribgtion. relative

to

a partition

and tlle measurenents have tire corre-

s n o n , l in o n : r t i t i n n lti. = L

^

[ ] ' _ l 'f 1 , , ) ,

f r o n w hj - c l . rt h e c o e f f i c i e n t s

are calcr.rlated. Correspondingly,

the parametcr matrix A, and its

inverse

- M -

r = A -1 = u ;'r, , herro nrrtitions

r = f L ^ *t - ,

o = / o * *^ *'r \

l

tl n

\

/

/

l r

)

A

\"yx"yyl

t

\

r

r

i

I

YYI

\Y*

where the diagottal subrnatrices are square, or of x, y.

t h e d j . r n e n s i o n sp , q t trly'NIx

t

r

,

"*

xy

= lt1*'l'1,

-1

. A * * = ( I : x ' 6 y l l x . ) - r , , \1x r

giving

The formulae the

coefficient

the

c li s t r i b u t i o n

-Rx2) ' or

the

--1 -

yx'**'*1'

1

)

''xY

"xI

t

'

of

between x'

s*,

R'

correlation y

and

as functions

of

are 2

.' - 1

1

= yy ' sxY lA lb I . \X' XX

x

rr /

ir nr fi n( L r er lLr re rnco l ci rr ' l l . L u .

. ^ r r ^ l a f

i n n

e n d

S i m i l : r l qwL t l ,

n f

i n e

fnr

l r r s i n n

I '- * ? y -_ lI I - r x x A x x L' a s s o c i a t i o n a n d < d i s s o cai t i o n a r e d e t e r -

of

The cocfficients

,

coefficient

r e s i c l u a1

of

the coefficients

fn

-l 't-1 = (r rt !r l ^'r 6l y l : x ) - ' ) 1 " ' e r e * l , i r ( ], \; 1 ,1 . , ""^.r.,

the

r,'i th

1

-

paralneters

sanre fornulae

enrre

,

separation

of

= trace ,

lxplicitly

nr,--lar

nj-ned by )

a*,

^

4

1

^

The canonlcal correlation

2

roots,

and their

multiplicities

together rvith their

multi-

of the equati-on , 2 lu t**A**-

. tillcrc \

^

coefficients

are determined fron the posltive p1icities,

?

Rxy/pq,tir=t-c"y

2= r -

| 1l=

0

7 L- , or tlre sane equation tritlr x, y interclranged.

- 6 5 -

Q u a d r a t i c d e c o n p o si t i o n

1 1 A I

The follorving

of the algebraical

is a reformulation

n r n n n q it i n n n n t h i c h t h e t l t e o r e n o f C o c h r a n ( 1 9 3 4 ) o n t l - r e of quadratic forn's depends, in which ortlrogonal

distribution projectors

and rvhich shows a significance

again have a roIe,

the forrn in whlch a surn-of-squares decornposition has been

for

derived fron a regression. If

are positive definite

a,b,

of rank p, q,

s y r n m e t r i cr a t r i c e s

and of order l'J, such that

a + b +

1,

then p + q + . . . = N if

and only if

there exists a deconlosition of the rrnit

matrix into mutually orthogonal projectors

'.e'

f,

that is a

and an orthogonal

+

f

+

=

1I

n F t

n v

,

,

t r a n s f o r n n a t i o n L I, s u c l - r t h a t a = l l' e U ,

L2

=

G e n e r a! i z a t i o n i n

l l ' il h e r t

b = U'fU,

snace

The analysis i+hich has been given for pairs of subspaces of a fi-nite

dimensional Euclidean space generalizes wi-th only

the slightest

m o d i f i c a t i o n s o f r n e t h o dC i r e c t l y

dimensional unitary

space; and it

can be put in a spectral

forn which can be i.nterpreted in llilbert that generalization.

A different

question of the unitary

invariants

h a s b e e n m a d e b y I l i - x m i , e r( 1 9 4 8 ) .

to a finj"te

space' and suggests

approach to the related of a paj,r of subspaces

- 6 6 -

b e a p a i - r o f s u b s p a c e so f a H i l b e r t

Let €,T and let

e, f be the orthogonal projectors

:re

nrir

the

of

haveC, fi fot

i r 1c m n o t e n t

ltcrmitian

tp^r"{

,

on them; so er f

oncrators

in

whiCh

ranges.

their

Nou efe, fef are a pair of non-negativeIlermitian o p e r a t o r s w h i c h h a v e t h e s a m es p e c t r u m I , e x c l r r d i n g0 , w h i c h i s a c o r n n a c ts e t o n t h e r e a l a x i s b e t w e e n 0 a n d 1 . Theq following the formulatjon oI thc spectral theoremfor l l e r m i t i a n o p e r a t o r s g i v e n i n I I a l m o sf 1 9 5 1 ) , t h e r c e x i s t u n i q u e s p e c t r a l n r e a s u r e sE ( o ) , F ( o ) ( " € I

),

defined on

,4

s u b s p a c e< s . Yo f I , s u c h t h a t efe =

, fef = ('^ar(^)

[rdr(.r)

J

where,

in

addition

to

the

E ( o ) E ( p )= O , rnrried

hv

the

J

autonatic

F ( o J F ( p )= 0 cnnefrel

oennrrl

there are the further

orthogonali.ties (o ao

theOfem

fOf

= 0)

,

I l e f m i t .i .a. n. .

. - lO - .n

eTatOrS.

orthogonalities

E ( o ) F ( p )= o

(oAo

which arise frorn the peculiar

= 0) ,

form of construction of this

p a i r o f I I e r m ti i a n o p e r a t o r s o u t o f a p a i r o f l l e r m i t i a n i d e m p o t e n t s .\ l o r e o v e r , t h e s p e c t r a l m e a s u r c sE ( l ) , the

DOS it

ive

i r-a ts,rvr

-S/Dt /e! c! r r u ' r '

orthogonal projections

tl-,^

ef,

F(l) of

^ vr +LL r i u l { u t r d1r -p*l ^r r Ji s^L^ L+ L1 r S

On

the

f€of F and €.on € and T ,

The matter is easily

settled

is concerned;but it

calls

so far as the point

spectrurn

for further analysis in respect to

t h e a p p r o x i m a t ep a r t o f t h e s p e c t r u m , u s i n g t h e I a c t t h a t t h c o p e r a t o r s c o n s i d e r e d c a n b e a p p r o x i r n a t e db y o p e r a t o r s w i t h nnint

qnaafra

-

oa

-

A P P E N I ] ] XI O N T H E L A T E N TV E C T O R SA N D C H A R A C T E R I S T IV CA L U E SO F P R O D U C T S O F P A i R S O F S Y N { M E T R IIC DEMPOTENTS

Let e,f ". be a pair of symmetric idempotents with real elements. Then F o r a s y m n e t r i c i d e m p o t e n tm a t r i x e , s , = e , e 2 =

the protJucts ef, = ftet

= fe;

fe are transposes of each othcr: (ef)'

s o t h e y h a v e t h e s a r n ec h a r a c t e r i s t i c

and these values and nulti-

w i t h t h e s a m em u l t i p l i c i t i e s ; plicities

characterize a symmetrical relation

THE0REM I:

values,

The characteristic

between e and f.

v a l u e s o f e f a r e a 1 . l -r e a l .

at

least zero and at most unlty. values of ef = eeff are the sane as

The characteristlc those of feef = (ef)ref, definite; all

symmetric and non-negative

hence they are real and non-negative; and they are

zero if If

whi,chis

and only if

ef = 0.

e f I U , l e t t rh e a n y n o n - z e r o c h a r a c t e r i s t i c

necessarily rea1, and 1et tJ be a real belongs to it,

latent

value,

vector which

so that efU = Ur eU = eefUl-l

Hence

Sincer I O, -1 = eftjr = U.

tJrfU = U'efU = U'U). ,

which gives I = U'fll/U'U. But the stationary values of X'fX/X'X values of f,

are the characteristic the calculus;

as appears by methods of

and these values are all

X'fX is a continuous function

zero or unity.

j-n the closed region X'X = 1, the

maxinun and minimumvalues are anong these stationary and so it

follows

values,

that 0
q-Tffi-ef

Since

<1 . +)

(1) , another proof is as f ollows: erence to Af riat r = x= le'el ""itl , l"l *-< rlrlx = t. = so that l"l* 1 . I l e n c el r l < i e r l i"l

- 6 8 For any pair of vectors U,V,

. o r 21 u , \ , 1= ( u ' v ) z/ ( u ' u )( v ' v ). T t l E O R E lI4I :

U

If

V

of ef f or a non-zero

#

V^ = ftl^ ,then

value l= o,3 and if

characteristic fi)

a l-atent vector

is

A

is a latent

vector of fe for

the characteristic

value tr; (ii) (iiil (iv)

I

= c o s 2 { U ^ ,\ ' ^ ) ;

if

IT'tl = o. then ll' V

if

al B , then U' Uu = o.

d

B

0

6

= O. tll V

Thus,efU^ = Ulr , forl=c,B feV^ = fefU^ which proves (i).

f

0

b

= fUr).

= V^ ). ,

Now = tl^ I

,

eU^ = Ur since r I 0. Therefore

= ui efU, = ui fu, ; , i U , = U ' ^f l J u = U ' t f I f u u = V i V u, ll;

a n dp , ince it

= 0 V' V^ = 0;

I O. Hence

eU^l = eefU^ = efU^ so that

d

uuu

U ' ^ U r u = U ' t r \ r u= V ' ^ V r ,

follows that

for l, u = qB. Now (iii)

follows inmediately. By interchanging

t ra n d u , w e h a v e U j U U o =U ' ' U U B , s o t h a t U J U g = 0 i f a I B , which proves (iv).

Fina11y,

. o r 2{ u ^ , v ^ )= ( u ' r \ r r2)/ ( u , x u r()v ' r v r ) = l , w h i . c hp r o v e s ( i i ) . T H E O R E IMI I :

The nullity

of ef-r1

is equal to the multiplicity

of las a characteristic value of ef. Since the caracteristic dependsonly on the cyclical characteristic

equation of a product of matrices order or the factors,

v a l r - r e st r o f e f = e ( e f f )

the

are the same as the

- 6 9 (eff)e = (fe)'fe,

characteristic values of

m^. A1so, provi.ded that I I 0, in which case

multiplicities eU = U, it

with the sane

efU = UI is

appears that

equivalent

to efeU = Utrr

so that the nul1 spaces of ef-11 and efe-l.l are identical.

l l e n c e t h e t h e o r e m i s p r o v e d w h e nI I 0 . ef and efe = ef(ef)'

observe thal X'ef I

is mr.

of the latter

B u t , b y t h e s y m m e t r yo f e f e , t h e n u l l i t y

remains now to

It

have the samenu11ity,

since

0 is equivalent, X'ef(ef)'X= O, and therefore to

X'ef(ef) ' = (,

and the nullity

of the latter

the ranges of e, f

T H E O R E IMV : I f E , T a r e

is mo.

then

€ e".}- = E aT =?ore' If

U = eV, then eU = U. Hence lJ = eV and feU = 0r is

equivalent

to U = eV and fU = fr

This gives te

e"ts = EoF.

A 1 s o , i f e U = U , t h e n l - U = C ri s e q u i v a l e n t t o f e U = d ; and €/= efU = (fe)'feU

irnplies feU = e( Hence tl = eV and

efU = O-is equivalent to U = eV and ftJ = 6, and this

gives

EeT= EarZ. T H E O R EVM: I f are all

F ,7"t"

th" t"ng"r oe e,f

the non-zero characteristic

repetitions

according to their

values of ef,

m u It i p 1 i c i t i e s - _

e x i s t u n i t v e c t o r s I 1 1 , . . . , E r a n dF 1 , . . . , -

- p

and f L

respectively,

a n d t r, , , . . . ,

I

r

with

!LI€_!_l-b-9-IS.

F. which spane7

and are such that

EiEj = 0, EiFj - o,

riFj = 0 (i I j);

ElFi =01,

2 where p i The spaces e}ancl

{€

eI which, by'lheoremTII,

are of dj.mensionequal to rank of is equa] to the total multiplicity

r of the nol-zero characteristic values of ef.

It

follows

- 7 0 -

and III

Theorems II(iv)

fron

ru rr nr irt L vv c r - t n r Js ' lFr ..

F - I

. . ,

sttch that

efE.

' - - . .

I

orthogonal

exist

= E , t r, . E a c h o f I

. l

range has

I O; and their

each ). i

in e'Isince

these lies

there

that

t h e s a m ed i m e n s i o n , a n d h e n c e j s

identical with e'f . Let

_16

vector obtained by

Fl = fEi;

Ft be the unit

and let

follows from TheoremIi (iii ) that

normalizing FT. Then it

, F, al'e orthogonal, and their

Fl,

the samedinension, and hence is

range, in f ?- , has

and (iii).

c o n c l u s i o n i s n o w o b t a i n e d f r o m T h e o r e mi I , ( i i ) TIIEOREV I 4I : T h e r e e x i s t

rnatrices E, F such that

e = E E ' ,F = F F ' , E ' E= 1 , F ' F = 1 , E ' F =

Let Ej,

Theorem\r ancl take

J

-

dlLu

,

hrr

r r . .

tlurr

r

ortbogonal

T h e o r e mT \ t , i i F j .,

uni.t

vectors

Lr+

. . , FO . . ,L'and 1+1,. I ,.

i

= 1,...,

, P and

P and j

= r+1,...r

by Theorem \I,

1

Fina11y,EE', FF' are range as e, f;

hence

These results

F ' F =

1 .

f ' F

qvmmetri c tl.nrr

cro

(r'.

. id e m n n t e n t S

irlanti

h a r . e ^d P* P- L1L :L ^d L^ a+U ir r^ , -

=

f^i

^ - r

d r l u

I i nrr

f )l

\ _ /'

0 9r

:) with

-€rl Wlth

h a d

s- , u, ^g^ ^g- e+ is^ r- .ro n , :r -n +r Ln e c a n o n i c a l c o r r e l a t i o n llnfnl

:)

= Crfor i = r + l,

q and again for

F r F =

+)

',?.

'Iei and € . . L e t E = ( E r . . . o o ) ,F = ( F 1 . . . F q ) .

s p a n n in g E e " 5 Tr 'h' a! rn r

(rt

F, be as determined in

E, and F1,

,..,

The final

with f€-.

identical

e,t

t h a i r

the -

sane

+)

n r i c i n e l

theory of

Qi

-71 -

r\PP[NDIX I I

1.

Introduct ion 'lirc

Jrc[lrcssion b1' least

lincar

orthogonal

of

projcction

projection. forn,al for

of

theory

'lhis

Its

tcrns

r v l r i c h s L r p p li c s

anrl {'corrctrjcal

made ir-r a scparatc

of

lysis,

1,n

1).sis,

of

liotell jng thcor,r'.!

tlleor-y is

to i:e

re!,ression as 3n ortllogonal

t li f f e r c n t

( . L . t f i , T r l c t, h c

f o r - r n c tl o

topi cs

in

canonicnl

c i , l rl c l b c t r a n s l a t e c l

in

c c t i - ' r ^ : i ,r l n i l a l s c t t i r a t , fl'rr,1

in

ana-

nLrltiVariate in

rvhich

rnultiVal-iate

correlation into

rn'hich

I\'ere cli-stingtrishecl'

factols topics

in

an

\\'ere distingui"shed.

analysis

a statistical

leacl to

analysis

factors

intcrcorrclaterl

cxpositiLrn of

. 1l ,

r r l tl t i l ' a r i a t e

in

a statistical

a y r p e a l . c i lt t r ; r t d i f 1 - er e n t for

altriebrai-ca1

a trnified

tlte statistical

SC\rera1 llrorrp-
interest

deVelop algebraical

to

concelts

spacc l{as fi1.st

on a liliear

appeared that

i t

n'ain soltrce of

necessarl' for

a nultiple

:irivantageous exposition

'ilicn

ancl a

paler '

Tirc expression of

grotrps of

is

obiect

forn'r.rlatjon of

anal1,sis. An exf csit jon

SeVeral

as an obliqtre

thc

lives

ofcratiorl

on a

ancl a partial

observations;

connexion bettvccn a statistical

inVestillation.

projcction

observations

of

the

c q u i ' , - a 1 c ] n tt o

s(lrlares is

can be sin'ilartl'represcntcd

algebrlticat

this

nrultiple

vectors

space spanned by vectors regrcssion

of

oferation

statisticai

the

theory

alla-

of

algebraical

t1:is aLr-el-rraical

-

fornulation, certain

it

intuitive

-

l t

was possible to introduce concepts fitting ideas rvhich, though needeJ in the subject,

were without any clear definition, a definition

for a coefficient

Thus there is c,btained

of correlation

factors eacli of which has a nultiplicity rvhich reduces to the correlation j,n the case of sin;pIe factors. concepts is

betrveen

of components,

coefficient

of Pearson

The genesis of the other

to be found in consicleration of psychological

a n d o t l ' r e r s t r c h k i n d s o f r n e a s u r e n e n t sa n c l t e s t s . idea that

a test

is

to p,ive a character

With the

to tl.re i.ndividual,

there i.s the idea that the character stancls outside the test. or that several tests rr'ith distinct

definition

may be di.rected

t o t l r e s a r n ef c a t u r e o f c h a r a c t e r , f o r c x a m p l e , i n t e l l i g e n c e , and, by this

intcntion,

thelefore have a relation of equivalence.

T h e r e s l r o u l t lh e , a c c o r d i n t l y , a t h c o r y o f t l r e e r l u i v a l e n c e of tests,

and, as a further

inclusion of tests,

example, a theory of the

by wlricll one test

can be decided as a

p r o p e r r c f i ; r c l c l ' t o f a n o t h e 4 o r a s c o n p r e l i e n d i n gc s s e n t i a l t y more f eaturcs of clraracter tlran anotlrer. Again, there is to L r e c l e c i c l e dr r ' h a t f e a t u r e s

tes ts rnayhave in conmon, if

C c n e r a 1 l 1 ' , a n u r n l , e ro f t c s t s forrninp a configrrration, ncasures for different

tog'ethcr may he regartled as

f r o r r n ' h i c hi s r l e r i v e d t l i f f e r e n t

relations

one another. In translation are various coefficients

any.

the tests may have to

to the algebraical

schene, there

defined rvhich belonc to the

confifui'ation r'lricl|r'ector subspacesof a given space fornr togcther; anrl arr account of such cocfficients this paper, rvitlr notive toriards their

is I'iven in

statistical

appJication.

- 7 3 -

)

qn.aec

\iaef^r

be the space of vectors of order n

r , e tI

f o r n e d o f t h e r e a l n u m b e r s .T l , e n , r v i t h t L e s t a n d a r d l a w s o l - cL Uoi rl .nI /nv oJ rsL ir tv ir o r n v r

n over to

the

^ r , al a L- , L

and

tlte

snan

X =

Kx.

havino

K of

ri ,r ' l , , .

of

in,lnnenrlent

, ^ l,J.

o

,

these

tltCSe VeCtOrS the

k

vcctors

voctor s of

rs

tlre

k,

which

snace

all

i.'is

lts

srrhsnace i{

k

dimgnSiOn

taken

nay be cal1ec1

oenerators.

"1,.

of

in

as

vectors tho

iS

snace

J y u L r

k vectors

of

n x

order

with

I ineaT

d

numbers. Any set

real

compose a rnatrix

@ vsL LUr

nf

rfnrc f 6 , r vV o g L L U T ) r j u

r U l

e l ements ; of.

Of

l'he naximal ltave

tlre

veCtOrS sets

sane

n u n b e r o f n e m b e r s , t . r h i c hd c f i n e s t h e d i m e n s i o n o f t h c s n a c e ; it

i s e n r r a l t o t h e r n a x i n t r mn u r n b e l n f

i r r r cl n e n , l c n t v c c t o l s

a m o n gt l r e g e n e r a t o r s , a n d t o t h e r a n k o f t h e l e n e r a t i n g matrix K. l'tlEORE1 I r.l1 : T h e r a n k o f K i s e q u a l t o t l i e r a n k o f K r K . lor

if

Kx = d

xtli'Kx = d,

, then K'Kx =di

r v l ' r i c l 'irm p l i e s k x = f ;

t h e s a n e c o l t r n : nn r r l l i t y ,

and if

K'Kx =d,

then

K, K'K itave

therefore

and, since they lrave tlte sarne

numbcrof columns, the sane rank. C O R O L L A R YT h e v e c t o r s o f K a r e i n d e l e n d e n t i f t

i

lI K ' Kr l I Let I

and only if

O. h e a s u l . s p a c e o f J - o f d i n : e n si o n p ,

of a maxinal set of independent vectors of h e r r i n o n m e r n h e r s -T h e n I vectors of I

defines a base for €

any sucll set , such that the

are thc vectors uniquely givcn in the forn X = Ex;

+ L , , -

L r r u 5.! . .,

c . . -- L / r: * ,* ,f , y i r- i e s x ' = L

(E'E)-'

exists.

vectors X of

e,

and E tlre arra)'

I

a- -ni d Il I t E l l

0, so that the matrix

T h e r e i s t h u s a c o o r d i n a t i o n b e t r v e e nt h e

the space €

o f c l i n r e n s i o np ,

ancl the vectors x

of ordcr p, the coorCination being defined relatlve base l.

The following results are immediate.

to the

- 7 + -

condition for a

T I I I I O R E1M. 2 : A n e c e s s a r ) ' a n d s t r ff i c i e n t

base I

X to belong to a space $ with

lg:j!!l

IrX = X,

that

a n d t h e n X = E x , t t h e r ex = ( L ' l ) - 1 1 . ' X ;

r r ' h e r eI , , = l t t . ' t ) - 1 | ' ,

-

I -

o r t l ) o t ' o n acl o n p l c r n c not f I

p is

is, J,\ = S, tvltcrc

=d,llrat

I,\

is

TI , . l l o r e o r . c r , i f

belongs to the

concijtion tlrat \

3nd a necessary anclsnfficient

. Il ,. . = I -

is

€,, ltlsn p =

t h e r li n e n s i o n o f

rank T, = tl'ace lt l An orthonorn'al orthogonal

t

base 1r for

conposeil of

is

so [i'l] =

r-rnit vcctors;

lr is

1 if

a sct

p

of

an orthonorn'al

b a S Cr a n c l I 1 ; = l l l J' .

l,

Lct

E ,7

f,'l

l r e s n a c c s , a n c ,l

sait'l to

be sclaratc

rihich is the is

rihich

the

is

join

the

if

i.'. t5c join

oi

conplete

spa."

the 1.ascs is

ortirollona l

, if

J

;

of

i;rclirrc,'. If in

1l Il

equivalertlv,

in

in

tlris

i,

thcn

tlicrc

the trases, '>' d c r . r t - r t c lt 'l 1 ' e @ J i r r Cj " . cl n t . ' l ' h e y

be

ancl tlrcir

sqLrarenatrix.

to

ortltogonal

1 - o I r ,l i i c h

exists

a n,atri-x a

aitclthe

is

e\:c'ry

I, I,

t l - , c ) 't r r c s a j t l

of

orclel qx I

sufficicnt

trnion

l'lrespaces are

r) r ' ) r i c l rc : r s c c i s s a i . l t o l c

anrl conve'rse1,v; ancl a necessarl'and ts

tite nu11 space,

case p + Q = n,

onc is

[rre

(frf) of

to

saitl

= Cf ; othcrt',isc

*| t r - 7J - , i l

is

tirc sprccs,

V e c t c ) r - i r r t l r c ' o t l t c - r ' , t l t c c o n r lj t i o r or

tlte bascs,

cP

tlrcl'are'scparatc

a r c g r rl a r

e \ ' 6 r r ' ) '\ ' e c t o r

t nt

the jojrr

a l r a s c ; o t h e r ' \ i i s c ' t l r c ' 1 'a r c

;lrc callccl conIlerrentnrl

rt

intersection

anclonly'i.f

anclrvith bascs

c li n e n s i o n p , q

corrcs]-ronCingll',also

if

the case,if

sy.;t1o r f also

tr" an,v spaces in"C of

= 6, to

he

ilrclrr.lc,l such that

conclition

for

I

this

IpI1, = I1., r''lrich in|lics l. = |o

rr,itird =

- 1 1 ,' 1 . . (l 'l ; '],

. \ ec o r c l i . n g J y , t h c

s1."1qg5 ale

i
C-

jtrst

iI

= Fo,

It_ = IIr;

_

-

I J

also they ale ortltogonal comllcntcntsjust

i f

i f tlre ortlrorional

l h c y e r e s a i t l t o I ' e o r t h o l o n a ll y i n c i J e n t

in eaclr of thenrarc

intersection

c o n p l e n r e n t so f t h e i r

= i.

Ill n If

i s j r r s t I I .I l i = 1 l I t . . T ) r c s p a c c € ' i s /\, s a i d t o b c c o n p .cl t c l ) ' i n c l i n e c ' l t o t h c s f i t c c J i f n o s r t b s p a c c o r t l r o l 1 o n a 1 ;a n d t h i s

is

of

totally

to 7

ortiroponal

, € o';

tlrcl'arc

inclinccl if

=D

. l r L c sl f a c e s a r e

;

conplctcly

to

saitl

irtcLinc to each

otlier. 'll.:o is

rre sairl to form a c:rnorrical pair- i f

lrascs Il,\'

oltlroncrrnal, lnil cr.cly vcctor

nost

one vector

tlr is

casc

o f t l r c ' o t h e r a n c lo l t h o g o n a l

rr'lret'cc is unil 1.

l l l L

a r c g r rl l i r

_-

1 t

,

1 ' 1 1 ' \ \

diaponal

r,'il1 e|pcal

It

= 1, Ii'\'

thc

t

at

res t ;

in

l o f'

1

r r itlr cl cl'cilts

lratrix,

t l r a t for- cvcr'1' p a i l

strclt a crtltotrical p a i r

tlrcle- is ists

3.

to

to

lo o\

t t

-1

one -is inclitretl

of

caclr

l r il S

of

bcltr't'crt

spaccs

of

C S.

J)cterr,inilnts 'l

irc 1ol1or,'i1r! lcslrlts

l - l . i l i . l ,J\ . 1

.\,

lf

I

, t r c ' . 1t , ) ' 1 ,r t r i c c ' s

I - .\lr Proof.

'iallc

o

l/

( . o l l o l , l . , \ l t Yi . 1 1 . C O I I . O l , l - , \ l lrY. i l . clraractcristic zcro

lias tl)c

riill

bc

r e c l r r i r e - c. l

o 1 - o r - t el ' l l l l / l l , l l , ' r r '

tlrcn

I - ll.,\

=

c l c t c r n r i n : r n t si n t l i c i d e n t i t l '

lt-, \ 1 , . t )

I \

on rlctcrninants

y'n 'lhc

u\=/t it ll

) I -Ali riatliccs

t'alrrcs, r,ith

3\

T I

\ /\

\ I

l

rl =

\tr

I

)ni

L l-ll,\

\0

I -liA

lli,lJA lrar.'ctlrc sanc non-zelo t _ l r cs a n ' c n r t r l t i . l L i c i t i e s ;

: , a r r r et t r t r l t j l , t j f j , l ) r : q l r

clrlrracte ristic

!n.f vaLug

l.c

- 7 6 n = n.

and onlv if

of each if

A is

L E M M A3 . 2 I f

I

'l

l

rr |

l^

square matrix,

a regular

then

- l

= lal

In-un-'vl

I U B I Proof. Take deterninants in the identity

o o \/ o u \i ' - n - l v ) , / n \ \-uo-',/(u r/to , /=(o s-un-rv/' (r

ArB are regular

C O R O L L A R YI .f

T H E O R E 3M. 3 :

A u

I

U

I

B

then

l = l o l l nI l r - u n - 1 v s -| 1

G is

If

square matrices,

the join

of bases E, F, then

l c ' c l /l a ' r lI n ' r| = l E ' . r . r lllE ' E =l l r ' . r u r l / l p ' r l = l I - r F r F l For [ E 'E

E' FI

r ul r l , l G ' c l =l ; , ; ; , ; l = l e ' E l l l r _ r r E ( E ' E ) - 1 E ' F l = l rE' '. E by Lenna 3.2.

Theprby CorolIary

3,2I

a n d b y L e r n r n a3 . 1 ,

I G ' c =l l r l l I F ' F ll r - F ' E ( E ' r ) - 1 8 ' r ( r ' r ) - 1 1 = l E ' El lF ' F l l r - r E r F I C o R O L L A R YT. h e t h r e e c o n d i t i o n s lI for

-

IEF I I

0 are equivalent,

I E'JFEI I o,

I F'JFFI I

o,

and are necessary and sufficient

the bases E, F to be separate. For G = (EF) j.s a base if

and only if

E, F are separate

bases. T T I E O R E3 M. 4 :

-t_::t

- . - l

t:i_l=

t-p J -

E I E

B y l e n m a 3 . 2 , C or o l l a r y - ^ F I F

F'E

trt

3.2I,

" - , - Il = - pI'

rl

^ E I E I r

( - o ) P * el L ' E ll F ' r - l b z r - r E r F l

and by Lenma 3.1,

2l l - p E ' El - F ' F l l r - E E E F / e

- 77 -

l l .i e e q u a t i o n s c o R O L t . A t'Y [ - l . l . L h a v e t l r e s a r . ep o s i t i v e If g,'y

4. lrojection. bases, l,

I

join

"l],t

= o - of: f "t il = . e - - f , g i o ' r - tL r tI, -| -

r o o t s , w i t h t h e s a n r cm t r l t i p l i c i t i e s . A r e c o m p l e r n c n t r a sy p a c e s , t h c n t h e i r

in a regular square rnatrix G = (lf),

and any

v e c t o r Y l r a s a u n i q u c r c s o lu t i o n

z = c , z= t r r -' l\/ v I )/ = t * * F y = x + Y , . ^-1w n e r e z = r , L , i n t o a s u r no f v e c t o r s X , Y i n fronr Z by linear

transforntations deternined just

s p a c c s . f h e c o r n p o n c n t s\ , projections

g,7

transformations which obtaj-n

of Z, and tl.relinear

I J

to E- respectiveLy.

para11e1 to'i

S r , r c ha p a i r

ba these

rcsolut j-ongive tlre

Y in this

them are the projectors,, on t

obtained

and on y' parallel on and parallel

of prcjectors,

to a pair of conplementary spaces, are conplementaryprojectors;

Their two products are nu1l,

iJcntitl',

fron r'hich it

and their

to its

nu11-space, and if

c o r n p l e m e n t s ,i n w h i c h c a s e i t it

the

follorrs that thcy are itlenpotcnts.

A n y i r i e n r p o t c r r tr c p r c s e l ) t s t h c p r o j e c t o r paral1el

sum is

on its

range

these are orthog.onal

nust be synrnetric, then

is ca1led an orthogonal projector;0thcrrvise

it

is

c a 11 e d o b 1 i q u e . --

p

r\Y

If E , f

are scfiarate,so that the join (; = {l:|) of thcir

bases L, F is a basc for their union p, [or the orthogonal conplencnt R if

let

R be a base

f, , vltjch is nu]l

of

they are colnplerrentary; then (GR) = (LI:R) is

square matrix,

just

a regular

and there 1s thc unique resolutj-on Z = C u + R v = l x + l y + P . v = l l + V

of any vector Z into

=

t h e s u r no f i t s

a n c lY , o , r € -p a r a 1 1 e l t o 7 A n

+ \ '

X + Y

oblique projections

and on !

paratlel

to fdd,

X the

- 7 8 -

its

the matrix J, = I-Ia

is

symmetric idernpotent, such that JrR = R, since IIR = 01

J f F = C / ' sj . n c e I U F = F . a n d s u c h t h a t

follows that

are separate, it

are olthogonal. Also since €,T iE'.lrEll

and of

Now,with If. = F(F'F)-rF',

orthogonal projectiort V on A

b e c a u s e7 , - R

U on 7,

the orthogonal projection

s u r no f w h i c h i s

o, Uy Theoren 2.3,

Corrollary,

so that

- 1

t h e i n v e r s et ' ' ' : i ]

=":rr:': -::'"::lr'r"= ,,JrLx (Il'JFE)

and then

-1E'Joz,

x = E(l-'JFL)-1L'J'i

_ 1' 1 , ' J 1 , . X = I I , . i F Z ,w h e r e I E , F = L ( 1 . ' J f l : )

that is,

conclusions (which are of

Ilence there are the following

importance for the theory of partial T f l E O R l l4, .t 1 : I f I g,T

regression) :

I:are bases for separate spaces8-,7,

t,

tlten

= IE,p.,where t

is the oblique projector on & e-,T to the union of !r with the ortlrogonal complement

parallel

C O R O L L A R4 Y .11. If = IE,p, wheret

a n c lI u , F = E ( l ' J f E ) - 1 1 ' J F -

(-,F

of the union of

a r e c o m p l e n e n t a r ys p a c e s , t l t e n I o . / E"1'

€,! 'i<

nnw

thc

nhlinrre

nrnieetor

on

9

L,F

para11e1 to ! Il i.s a base for

C O R O L L A R4Y, 1 2 , I f

8.,

then Ia = I' = L ( [ ' l . ) - ' lL ' .

i s t h e o r t h o g o n a lp r o j e c t o r o n € a n d i l

Il, F are separate bases with

C 0 R O L L A R4Y. 1 3 . I f then there is for N

the iclentity

Ia = IO,r * Ir,O,

join

C,

and correspondingly

spaces. O

W

T

so, since I. Io * c-

c\"f

where I"

for

E , it

cr J'

' e , 7T

-+ V , , T

+ -=

tT ErT

'

appears that

I' t It A

-=

' =R * "'

r',

T

^ - ' t J^ ir -r r: rr r^d-r t .r /'

drru

witlr

- 7 9 -

lt

and then, into

' ' F- t

g , i

I,r t

e '

'

g , u

-

t ' t p * u t ^ 1

t i , -

,

I - c f rorn t h c s e c o n r l r c I a t i o n 4 ' c -

of

by substitution

llre f i rst.

Yt

j , {

tlrat

( r - r € I , i )t 8 , 3 = I E ( 1 - I C I ' I ). l l y T h c o r e r 3n . 3 , C o r o l l a r y , a n c lh y T h c o r e n4 . 1 , 0 o r o 1 1 a r y' 1. 1 2 , t

-

l

I o,i since E, a'are separatc; thus tltc foliotr'lng

we har.e I 1-IgI'

t h e o r e m i s o bt a i n e c l : T I l f , O L E l- il . l :

(,i

If

are scparate spaces, then the obliqtte to the lrnion of I

€ -p a r a l l e l

p r o j e c t o r I , . , _o l F rJ,

is ',leternined

ortliogonal conplcnent of the union of E,? tlic ortltouonrl nroiectors

fror

t C O R O I , L A R \ "I f [ ,

(1:l:)= C is

a base,

If

tvhere

(l.ll) = )., L = JfIi(]j 'Jfli)

( C R O L L \. \ I4 . 1 1 . C \ ' = f l !

I1= F(|'F)

(C'C)-l

-

'rN,\'G

'JFIr ancl I'l = JfF(l' )

IL'+

I ' l '. l , t . h e r e I . = ( ' \ ' , I r , . =

-lJ l,

Iiare

ortitogonal,

I-, Ir are

fI

s ( ; u a i ' ( 'n i i t r . i - r r ' i t h

the

conDlementar)' olrlique

the

spans o{ S p a c c s€ i ( i they

tlsn - t'

= L, - t' .

LL',

= l.(E'E)-1, l l - t g , - Lt

- t

4.r3. COROLLAIiY

if

{t-II.Il,).

t

C O R 0 L I , A I I4\ ', 3 7 .

repular

tlrc forntt,la

theorern calL bt: r'er:if iecl directly:

folloriing

-

lI,

(I-II It)

T l i t o l l . l l l1 . 3 :

T

Iy

bases t]'ren

sepaiate

I;(ti'JfIi)-ll,'.1 = , the

on (,'I

I-

(r-rerl)lrait-rLr+).

L,T= I' are

Ig,

the

l ith

l ;,

conplernentary' jnVcrsQ \',

projectors

so that

tlren LL', on and

C ls

a

Fll' are

arallel

to

F. = l,

.

.

are n,utrral1y

'

.

t

n t ) ' l)

nr e

qonATate

qai d

rn,l

form a conplentcntary sct

l 9

union i s

thnil

spacc:

IN a

o

9 ; J

'

= (v

{ l

f

) 1 , -

=d F - t .

@ J -

i

the conpicte

- 8 0 -

T n tL hr ri rsJ tl a- Jar sr e . fi ff

Il: is

= I

F * i .\ r, i

a base for

r

t n) l ,

then

" . . . t " |

L = (' 1t . . . . 1 * ) i s a r e g u l a r s q u a r e m a t r i x . m '

fn r.

e

e n m nl nnl c nl ?r v

c af

en, l

- 1

Lct e

f

e,

e

i f

t

=

( l :1...L- ,) ,

I

-?.1

{I1...F- ) ,

j

jli

the

noi' denote

projectors

orthogonal

0

i n & o f d i n : e n s i o np , q a n d w i t h b a s e s I ,

e , f

=

F

p p r o j e c t o - IE - oi l r

-*brique ._" ; ".

tr,:,

= 1,...,n:)

w i t h b a s e sL i ( i

C O R O L L A R \ ' 4 . 3I 4f . s p a c e s € i

on spaces

F; so

e = !(L'r-)-1E', f = f 1f'F;-1p' , - e, I = f

and e = I

- { are the orthogonal project.ors on the g

s -= d .e o r t h o g o n a l c o r n p l e n r e n tS

, i

= IOT

of

= ranl' f = trace rr = ranL e = trace e, q

f.

such that R t

r,

r = ranli ef,

R = trace ef,

it

ancl

The nunbers will

of inclination

d i n e n s i o n a n d t l ' r ec o e f f i c i e n t

t ,V;

define the

b e t r v e e n€ , 7 ;

a p p e a r s i n S ( ' t h a t t l r c l ' a r e t h c s a m ei 1 -a n d o n l y i f

arc orthogonaly incirlcnt, antl zcro if

an,lonly if

€ , i

o r t h o g o l r a l ,a l s o t h a t d i s c o r p l e t e l y j n c l j n c t l t o ?

iust

r = p, anri included in f inclination

if

;ust

€,Y arc if

'1 he coefficient

I1 = p.

bctricena pair of lectors \,

of

Y o r b e t r r e e nt h e

r a y s s p a n n e d t - r yt h o s e r r e c t o r s i s L = t r a c e X ( X ' X ) - 1 x ' y i y ' y ) - l y , = ( X ' y ) 2 / ( X ' X )( y ' y ) = . o r 2 o , +r,^+ L r l ( r L

;^

r ) l

i+

l L

:^

l >

*Lc

L l r L

s e p a r a t e s t h e n r. .\" The space e.i,

qnrr:re

J r l u o r L

samn rlinronsinr

tsccn € ,?.

t( hr l e\

^^;-^

\eu J I l J ( '

^f

u l

+( l rl .L ^

^-^le

d r l t ,

O

W l fi C l f

fornredof the orthogonal projections

in & of every vector I o f t h e s p a c e]

vn rf

ofi,

gives the orthogonal projection

i n t l r e s p a c eE . civnn

lrv thn

Tl'c inclination

eY

The spacese ],

dincn5iOn

of

f t are of the

inClinatiOn

f

be-

h ' e t w e e na p a i r o f s p a c e s a p p c a r s

c q , u a lt o t h e i n c l i n a t i o n b e t u e e n t h e i r o r t h o g o n a l p r o j e c t i o n s

-81 -

on each other = R(e7, f c).

R( €.,F)

a r e c a l J e c lr e c i p r o c a l s i n € , F i f

A pair of spacerL,V ern

tlre nrthncnn:l

nrnioctions

1*= cV,

of

. in

eacli otlrer

tney

€,?;

V= r ?.r-; called rL'ciprocals if

and a pair of vcctors X, \'are

they

rays, in whiclr case

span reciprocal

= eY, Yu

xl

= fx,

lr.here y'\tr = X'eY =X'Y, YtYu = Y'fX = Y'X, s i n c e e X = X , f Y = Y , b e c a u s eN , Y b e l o n g t o € , , 7 , I! r

since

I O; and tlie nunbers ^ = ( X ' X ) - l \ ' y ,u = ( y ' y ; - 1 1 ' 1

are strchthat

:

lu 1

p'--

vihere is

a g aj - n t h e

or

the

)

positive

then

Y is

2, value p

o

''

j9.J

is

latent

nou any latent value p2,

vector

characterj'stic

ef\

of

ef

for

a

= Xp2, ancl if

eY = efX = Xo2, ancl feY = f).p2 = )'p2,

a latent

vector

reciprocal p

of

fe wi-th ciraracteristic \rectors

a

vallte p'

in

e ,T

with

Z:

Any reciprocal vectors

of

ef,

vectors fe both

X, Y in

then Y = fX

0,!

with

inclination value

t''ith clraracterj-stic

for a p::tt-tl:

X is any 11.""._5..:l_1f_ef

characteristic

= Yirl = Yp?,

fe with

so that

vectors,

f,llat

feY = fXr

ef,

of

the

l;etreen

follows

= xo2,

vectors

coefficient

'lHtORtl.1 4.4: ^r"

\

ancl X,Y are

inclj.nation

Now it

= eY, = Xlu

characteristic

notr Y = fX,

'

if

0'4. A1so,

1

= cos"o

incli-nation

span.

X, Y are latent

so that

so that

of

coef f icient

ef\

value

a

(X'Y)'/(XrI)(Y'Y)

rr'hich they

rays

p2,

is

a latent

f o r t h e s a m ec l t a r a c t e r i s t i . c v a 1 u e , a n c l\ , Y

vector

of

are reciprocal

fe

- 8 2 -

vectoY'sin

f

-il

with

incl ination

The next theoren estabishes a procedure by which the of spaces can be calculated when the

of a pair

intersection

bases are given. T H E O R E 4I 4. 5 : T h e i n t e r s e c t i o n

It

is

the nu1l-space

is

1 - df.

of the natrix Let J,

of the space (,F

fl

denote the intersection

Let X be any vector

that Ja7l.

evident

and nu11-space considered. ofT?, so that

efX = X, and let

Y = fX, so that X = eY. Then X,Y belong to

the ranges E ,7

of e,f;

and

X'X = XreY= X'Y = XrfY = YrY. Therefore, vector

by Schwartz's inequality,

X = Y, so that

X of 77belongs to both € and T,

that

any

is , to J,

and

hence ??c7.

A c c o r d i n g I y , J= n . C 0 R O L L A R YT. h e s p a c e E O F i t

the nul1-space of the matrix

1 - ef.

& o " T = 8 o T - -t e 1 8 .

T H E O R E 4M. 6 :

the orthogonal conplements in Eof u4

That is, the sane. If

U = €V, then eU = U, and hence U = eV

and feU = (

i,

equivalent

Lo"T=

eoV

A1so, if

to feU = C/; rotuover,

TIIE0REM 4.7.

tey

to U = eV and fU =g; eU = u,

is

feU = U .

to U = eV and il

equivalent

=6i

f& are reciprocals

in Q,/

and

inclined.

For the rnatrices ef, Now every vector

j-nplies

= Yet & .

The spaces e7,

they are totally

therefore

then fU = o is equivatent

|l- = eftt = (fe)'fett

Thus U = eV and efU = d

therefore

a n d .f €

efe = ef(ef)rhave

in one of

the pair

is

the sane range.

inclined

to some

are

- 8 3 -

vector of the other,

so they are totally

inclined.

Let the signs o, o and x between a pair of spaces i,ndicate inclined and orthogonal.

totally

that they are inclined,

T h e o r e n r s4 . 6 a n d 4 . 7 s l . r o wt h e f o l l o w i n g

s c h e m eo f d e c o m p o s i t i o n '

shows the procedure by which it

a n d T h e o r e m2 . 5 , C o r o l l a r y , can be effected.

T I I E 0 R E I4' .18 . A n y p a i r o f i n c l i n e d

spaces 8 ,7

have deconposition

in the scheme:

L l

d

t

|

= e ? -.. o l

.

a

eg "

1,.*

. < .t r

= re o

l

J

(

' ? Idf,

t-.._.'^

----'

5. Rank, n:ultipl r::lZ_:1.1_:fthogonali 'l'lie

Y

theory of the latent

ty

vectors and character j-stic

values of prodtrctsof orthogonal projectors is reqtrired for thc subsequent theory of the canonical deconpc ition to each other, and the con-

of a pair

of spacesrelative

struction

of bases in each which forn a canonical pair.

Since, e, f are synnetric natrices,

their products ef,

fe

are transposes of each otlter, so tltey ltave exactly the sante characteristic

v a l u e s , w i t h t h e s a n i em u l t i p l i c j . t j , e s ; a n d charactertze a syntmetric

these values and ntultiplicities relation betweenthe spaces Y,T flre

ennfiorrrrtinn

the

qnacps

or are characteristic of

nresent

felative

to

each

other. T I l E 0 R l l \ 15 . 1 . T l , e c h a r a c t e r i s t i c v a l u e s o f e f a r e a l l a t l e a st For

zero

and a t

the proof,

nost

un1ty.

see Afriat

(3)

rea1,

- 8 4 -

A

Then any r

value of ef.

characteristic

for which m^

may be ca11ed a characteristj-c value for wi,th this

Since 0< r <

nultiplicity.

as a

of any value I

Let m, denote the multiplicity

l o .--

p

the spaces L , f

1 if

0, by

m 1I

Theoren 4.1, positive acute angles o are determined by ') T h e v a l u e s I ( q I 0 ) a r e t o d e f i n e t h e canoni ca1 l= cos'o of

coefficients

between the spaces, and the

i.nclination

corresponding angles

define

If

the canonical angles.

the canonical angles between a pair

the

of spaces are all

sarne,the spaces are ca11ed isogonal. denote the nul1-spaces of

Let d , ]

Theng^I o,Tx I o just if 1 if

^ I

11 -

I o , a n dt ^ " " 7 , \ c -

O, in which case they are to define

of subspaces of (,,7

11 - fe.

ef,

f L

a canonical pair the

. The next theorern shows that

d l m e n s l o n s o l c" - l ,. T, -a.r, e e o r r a l t o m l A of the matrix

The nullitv

T L I E O R E5M. 2 .

the multiplicitv

of

as a root of the equation

C O R O L L A R5Y, 2 1 . T h e r a n k o f e f i s of its

equal to

ll1

-ef

= 0.

I

(3).

given in Afriat

Proof is

1 - ef is

non-zero characteristic

the total

nultiplicity

values.

That is, r =

r^ , where r is the dinension of iy' i n c l i n a t i o n b e t w e e nt , 3 lt - efl

T H E O R Es M .3.

= T f( r - r ) t r , l 1 - e f i = s P - r 1 1 r n ^ rl0 l

For the characteristic

first

part

1 - ef are 1 -tr

m, , and the determinant of a natrix

nultiplicity product of

values of

its of

characteristic

is

the

values; which proves the

the theoren. To prove the second part,

seen, by Theoren 3.3, that

with

it

is

- 8 5 -

l E ' r 1= l r - " f l

I

l 1 r ' r ; - ' L ' f E l= l E ' f r /

B u t , b y L e m n a3 . 1 , C o r o l I a r y 3 . 1 2 , t h e n a t r i c e s (l | |) have the

same non-zero

nrultiplicitiesi

Theoren 3.4 are the products

T I I L O I I L I5' .; 4 .

,T

If

a r r J V' . -'

required

vectors

a pair

of

the same as the of

tj,

js

V,

=

\' o' ^

, - ^ 1 , 1i t i o n s

O 3re

r

nair

of

II^ =

. r r t t i r r r It e t t f

sanle

spaces, l'rhich by latent

projectors

vectors

on the

spaces,

orthogonal i ty: of

rcciprocal

for, I =o , B I lt'

the

concltrsion follows.

reciprocal

the orthogonal

with

values,

0,

[l'

and they

\:

vcctors:.of

0, then tlie -

O,

\''

[i^

lrold atrtonratical

l v

als

U

S u p p o s et h e y a r e t r n i t v c c t o r s ,

so that

= ,,^,^'/' , flil

o ,-^^'/'

ancl

"Vr

l r J V )= l r i

Thcn

since ell, = U,,!=o = 0,

Ul

follow

hy synnetry. 0

so that

ll'

rP I

since ^ I

to LIr

6.

characterlstic

witir inclination l,

arrl,n^nn.,r ir..

I-li-l

ef

fol l owin5'propcrt i es of

Iravc thc

L

fI-,

anclnoir the

The reciprocal

of

- 1' I : r

\-g= (l if

0.

,

H e n c c l . 'l V t = 0 l s

O, ancl tlre renaining

e n r r i r r r l c n f

arrrri rrq l anccq

I'urtlrer,

r/z V'

L l z = U ' l l ^ t r"

"\

v ^ )2 = r ,

(lli

c\,

= 1"

t/z fUeB , cl)' \,

=

I. I r \' /

p

^ p

tlg

C o n f i . r ' l l a r t i o n s c ; rl c : . A set

of

ttvo or,

i n

of three or more subspaces

n r i r r e i n l e

:ts folnrjng a configtrration

of

a g i t t ' n - 1 r : t ' . i s consitleleci

of

sLrbslaccs; lurii lt colrfigrrration

is vjoncdrs attaining

_ 8 6 _

nodel types, according

to or departin5' fron sonc different t o m e a s u r e n e n ti n d i f f e r e n t a r a x i n . u mj u s t

scales, eaclt of whicli neasures

when a rnodelis perfect-ly attained and

measureszero when another mode1,which defines the opposite, i s a t t a i n c d . T h u s , f o r a c o n fi g u r a t i o n o f t r ' ' o s p a c e s , t l t e r e are tlre following ideal nodels: the spacesare orthogonal, orthogonally incident, isogonal, iCentical, incitlent, or one snace is includerl in another. AIso, for a configuration of two sfaccs, there are the fol Iov.ingnodels: the three s f a c c s a r e n u t u a l l y o r t h o F o n a l , a p a i r a r e i t .el n t i c a l , o r ' a nair are ortlroqonaland the third

is their union.

Ccefficicnts arc to he defined rihich set any confipuration i n a s c a I e , i n r e s p e c t t o e a c h n o d e l . 1 h e 1 'p r e s e n t . [ e a t u r e s n f f l r o o c r c r z I r n l n t i o n t l r e s n i t c e sh a v e t o c a c l r o t h e r w h i c h i t is possjble to translate into statj.stjcal conceptions i n t h c g e n c r a l s r r l . j e c to f n ' u l t i v a r j a t e

w h i c l ra r e f i t t i n g a n a l y si s . L n I l 1 A6 . 1 . ( i )

f or anv ratri,ces a. of

tlrp

cemp

nrr] pr

a

(t r a c e a ' a .) ' lc

^ ^ , , ^ 1 : + , , \ ( l ( r d r r L I _

hnl,ls if

€ {t r a c c aiao)

an,lonlv if

^ - - l d r r u

+ L ^ L i l s

for

s o n r es c a l a r s , l ^ ) , r n , il tc l t a r c (ii)

o

3

o

Z

l .) F (' t r a c c a . n r r C

+

+ I.a. = 0 t l

I'o t l r z e r o .

nc t

?

)

(trace aja1),

ala,) 2J h'

l'TT(tracc O

a 1

) (tracc ala,) ?-rU ^-r

d r r u

+r l Lr F

1:+"

s L [ u d l t L f

(tracc alar)' J ^.

,

Z holds

if

a n d o n l vf

which are not alI

zero.

i ft L

-I , l. i . i.

s.

-=

0 for

sone_

- 8 ? -

If

ai are of order m x n, 1et A. denote

the natrices

the vector of order nn whose elenents are the elements of position.

A, ordered by their

In the first

K = (AtAzAj). Then lilfl

K = (A1AZ),and in the secondlet holds just

and the equality rro.t^r l

in the first

if

instance let

= 0 for

KI

>0,

some non-nu11

instance of order twO and in the The lemma is

second of order three.

a restatement of this

result. L E M M A6 . 2 . dependent if (ii) if

(i)

Two symmetric idempotents are linearly

and only if

dependent

Three syrnmetric idempotents are linearly

and onlv if

a pair

either

orthogonal and the third The proof of this and it

they are identical.

L E M I ' I A6 . 3 .

or a pair

are

sun.

long but straightforward,

granted.

any symnetric

= 0,

(i

ei

idempotents

1 r . . . )

(trace

(j) where equality

(1i)

For

their

lemmais

be taken for

will

is

are identical

holds

e o ) ( t r a c e e . ,) , 6ltrace "o"j)2 if and only if e^ = er; u l z

. _ z, f f ( t r a c e e ^r e l- ) + 2 T J ( t r a c e e , e 1 ) V E { t r a c e e i ) ( t r a c e " j " t J o o 0

where equalitv

holds if

and onlv if

either

e . = e L o r e . ' ' e u= O J

and e. = e. + e, for somei.

i.

N

-

J

^

k which are a permutation of

0, 1r 2' T h i . s i s a c o r o l l a r y o f l , e m m a s5 . 1 a n d 5 . 2 . Since e, f are syrnnetric idempotents, and the trace of ^

d

*-^J,,^+ P r u u u l L

^r

v i

- ^ . - i c re\ !sJ r r l a L l

, i e n e n r l s 6 n l v/

nn v l l

t( h c

evclir-nl

of factors, R = trace ef = trace (ef)'ef

2O,

Ofdef

- 8 8 -

holds just

and the equality

ef = 0, which is the case just

if

are orthogonal; a1so, since p = trace e,

if e,T

p - R = trace e - trace ef = trace efz0, and equality

holds if

which is the case just

€ c T.

if

r,

multiplicity

R = r,

I'loreover, if

are equal to unity.

It

can be shown then

are orthogonally incident.

t h a t e , f c o m m u t e ,s o t h a t 6 , f

Accordingly the numberR, deIined for any pair T

and ca11ed the coefficient

then

values of ef, with total

the non-zero characteristic

all

ef = e,

ef = O, that is,

and only if

of spacesf-, between thern,

of inclination

has the foI I owing propert ies : T I I E O R E6M. 4 . 0 < R z p , e , r

f o r a n y s p a c e s0 , 7

p, q and dinension of inclination are orthogonal; R = r

C ,A

gonally incident; R = p if

if

r;

of dimension

R = 0 if

and only if

and only if

(.,7

s i n 2 1e , T ) + C o s (zt , T ) = p ,

ca11ed the additive

qnaces

h. . r. r f

i n c l i nr r rr qf

In

r

ion

r v r r ,

terms

of

thp

ensine

i,s such r

the

a

which

o f i n c l u s i o n o f the space€ in the space

T h e s i n e a n d t h e t a n g e n t a r e n o t s y m m e t ri c a l - r - , , - ,

t - n7 ,

cosine, sine and tangent of t

are indices of the relation

nf

l = (trace u-f)z,

= (trace ef)1, sinl€,?;

such that

the

numbert

ran(€,7) = Sin(e,T)/cos(e,T),

and

T,

are ortho-

€. is included in 3.

Thus there may be defined the positive cosl€,?;

and only if

,

rt ^t i rl hr li rr , h

:^ l >

-^

L U

> d /

functions of

also the coeffici ent

function.

canonical

angles

between

c o s l l t , T l = f r o . o r l o ,s l n l ( E , T l

f h p

=

c n a r a q

c n - T + /

r , . t '

) n

c i n - ^

- 8 9 -

If e,T

are orthogonal,

cor21Cr,fl) = cosz(9,*)

then

another space'

+ Cosz(|,&);

it

then g = e + f

for

and &

union,

by their

is

the orthogonal projector

trace gh = trace eh + trace

so that

ofLt,

fh.

For example, 1et 1r flr r be three independent rays, concurrent on the origin

the orthogonal projection of 1 on n. Then the

and 1et 1'be

Bry

that if It

of 1 to 1' ,

is the cosine of the angle between1 and 1',

and thls If

t o t i s e q u a ] , , r t toh e i n c l i n a t i o n

of i

inclination

U; and 1et "ibe the plane through m,n;

it

are the angles hetween I and m, 1 and n,

f r o m L e m r n a6 . 3

coefficient

there is defined for

that

non-negative real nunber Q, to be

any pair of spacesQ.,T^ called their

(i)

follows

= cosZ B* ,orZ \

m, n are orthogonal then .or2 o follows

sayo.

of disinclination

fhp


nf

whlch is given by Q" = pq-R' 7-o, are identical, and where where equality holds just 1t t,! ) ) Q ' = p q j u s t i f R " = 0 , w h i c h , b y T h e o r e m5 . 4 , i s t h e c a s e just

if e,T

are orthogonal. Like the coefficient

of

inclination,the coefficient of djsinclination is a symnetrical function of the two spaces, and it

has the following properties:

T I I E O R E6M. 5 . O < Q l < p q f o r a n y s p a c e s E , T ? Q' = O if

it

e,T

a n d o n l y- . .r rt bv r v4

^ L- e^ ,: -so r l t i c a l ; d

of djmensionp,q; ? Q' = pq if

and only

are orthogonal. S = lt

N o w1 e t

_ efl

;

then appears, from Theorens 5.1 and 5.3, that 0 ( Sg 1, and that S = 1 just

it €,T

a r e o r t h o g o n a l ; a n d , b y T h e o r e m4 . 5 ,

- 9 0 -

S = O if

E ,T

and only if

numberS, defined for

Accordingly the

are incident,

any pair

of spaces, which will

be

of separation between them, has the

ca11ed the coefficient following properties: T H E O R E 6M. 6 : O s S ( 1 E,T

if

for

are incident;

any spacesQ.,T;

S = 1 if

S = O if

and only i.f 6 ,f,

and onlv

are orthogonal.

The number

c = lr - erl may be ca11ed the coefficient

of inclusion of

properties now appear as a corollary

its

Q,is

is

i.ncluded in F.

in 3,

incompletely inclined

C = 1 if

to T;

and

o f T h e o r e m6 . 6 .

for any spaces^L,T; C = O if

T H E O R E6M. 7 : 0 { C < 1 if

€

and only

and only if

L

I t f o l l o w s t h a t t h e r e a r e d e f i n e d t h e n o n - n e g a t i v er e a l 1 1 2, numbers sin(€,}t= I - cr i, cosre,;t = lr l "f

t a n l € , T ) = s j n ( e , T ) / c o(st , T ) ,

and

which nay be ca11ed the multiplicative - r 1

7

tangent of L on J.

sine,

cosine

and

In terns of the canonical angles o between

t h e s p a c e s w i t h m u l t i p l l c i t i . e s m^ , i t

follows fron

'fheorem

5.3 that

sin( 6 ,3) and

= T l r i n i l o o , c o s( € , ? 1 = 6 P - r f T . o r t o o , o ( fr a = Or-P fTt".,to o. tan(L,T) a,ttlt

Their properties It

follow

is noted that,

if

from Theorens 6,6 and 6.7. (,T

a r e o f u n e q r - r a tI l i m e n s i o n , t h e n

a t l e a s t o n e o f t h e c o e ff i c i e n t s o f i n c l u s i o n . o r 2 ( € , 7 ) , Z , + r cos-[ J,*) i s z e r o ; a n d i f t h e y a r e o I e q u a l d i m e n s i o n ,t h e n \) the coefficients

of inclusion

are identical.

The coefficient

separation, however, is a synmetrical function of the two q n q a a q

of

-91 .

and disinclination is

between spacestr,6,

the orthogonal projector

then

of inclination

denote the coefficients

Let Rr, , Qil

three spaces i,

define

n - i 3j k

and call

the real

the coefficient relation

number QijU with

j , k

1 ? az j-piRir-pjRii-pr,.ij, cube thus defined the

between the three spaces. Then

of dissociation

with the coefficients

satisfies

the following

of inclination

and dis-

between the pairs:

inclination

\ ? ? ? ptat:r = ai:air - (RiRj1-RijRir)'.

THEoR6 E .M8 . For,

and

j , k having indices i,

== np ni pnj p+ k?* p2 R . r pR r i pR i

of dissociation

coefficient

e,

and p, the dinension,

on 6,

Pi = trace ei, Rij = trace eiej ^? ^2 q i j = p i p j - K i j . Now for

if

so that,

from the definition, 7 p . 0 i . , t'iYi jk

=

r' vni l -1 j

-

) ) D ' ' \ - r D ' ' i' j )/ r ' \nv i vnk - D \vi"jk "ik/

) - D - R . , ) " . "i j"ik.'

T H E O R 6E.M e. (i) o (p;aiiu < al:qlucpipipl for anythreespaces; (ii)

Qiit

= 0 if

are i.dentical.

either a pair of the spaces

or a rrair are orthogonal and the third

iCentical with their (iii)

and only if

is

union;

7

Qi,r, = p;pipr. if

and only i@

orthogonal to each other.

By Theorem 6.3, ai: t , 0, and equality holds just under the conditions stated.

By

Tr rhr as nv rr E a nr u

A . o e, u

----i*---. fu h r roq l T l S X l l T l U t O t

^5 Qijk

. RJ ,K, - - R ,. R , , ) 2 i s a t t a i n e d w h e n Q r r , Q i t a r e m a x i m u ma n d (' p ^ 1 l J 1 K ' , i s m i n i m u m ,t h a t i s , spaces are all

z e r o , and this

orthogonal.

is precisely

when the

- 9 2 -

T t l E O R E I("6l . 1 0 ) : - n R " i. j.' n' i k- t i

z D I ) j uOi k. . -Z nP. iR" j. k - " i j " i k

n

\ i j \ Or k. ,

either a pair of the

and onlv if

and an equality holds if

a

atd the_third

spaces are f_{S-$-i_-_l_:j__g_-pgff are orthogonal rvith thei.r union.

i,s iclentical I'his

for

lirnits

follorn's by

result

inclination

the

to

two spaces have high

Thus if

to

Cisinclinations to

each other.

it

is

a third,

o be the

and lotv

inclinations

t h e y n ' u s t h a v c h i p , h i n c . li n a t j o n

let

n,

fornr to

sinBsinylcoso

is

attained

sincc

then o

just

though

,

tlre fo] lowing rcsul

n nake angles

angl e they rnake togcther.

cosBcosy an,l an equal ity

rays,

are given.

a third

a lencral i zation o[,

l,et 1, ni. n be three

a n c . lg i v e s

and 5.9,

Theorem 6.10 has a sinilar

not lreciscly

throrroh m. n-

5.8

between t\{o spaces rvhen their

and disincljnations

incl inations

and I et

'l'heorens

with

B , y

1

Then

4cosB cosy+ sinBsiny, jn

w l r e n 1 I . ic s

= B + y or

tlre plane

o = B - y

7. \rolume A base L rvitl'r span & is vectors

€1r...

\cctors tlcf ines or

a-

\ thc

by

the

If

L,

=[rJ,

b y t l r ei r

p intlependent

r Lp and the p-clinensional region

= L r x , + . . . + l . ^ x _= I . x ( 0 < x , ! 1 , I r p p r paral l el otopc ['

base,

nith

= f.I ]

bases of

the parallelotope

join [ffJ

is

in

I

of

i=1,...,p)

. tc t e r r i n c t l h y t h c

vectors

volurne rlenoted by l/21 .

I: are selarate then

conposed of

of

dir:cnsion p,q

=

LfJ

dimension p + q deternrinecl

c a l l c L ] t l r e p r o t J r r tc P x

ltrO j n r )7: . 1 . 1 [ ' lz = l I ' r .I

andF

:

ol

f, ,&

.

_ 9 3 _

Proof is b,y induction on the Cinension p of the para1le1otope. The theoren i-s verified .-^r..*^ ^.

^

^:--re

lUlUlllg Ul

d

he

Fnr

nrnyerl

, - ^ 1 ; l : + , ,

r aarur

L . / fr ^u* r

1et&

= [r]

is

cflsion

p

t,. the

t:

I(

t.-t),

of

the

itS

= (F[.),

so that |

,sotlratP=A^Pr, \orv

lenoth

wil]

rvitlL ,2" = f t.l,

Thus,

1.

whereF = (tt..

a n c l l e tP r = r ' l t e r e nl r

. -

r ^-^-L

tgllELll.

L^ ^t ensron p = r on rn c Ln.y- -p^ oa Lr ^n^e: s r^ s o 1

d - .i ,r l J i u '

s .Lirr nt lnr rL -y . ir -L -)

t.ectol" is

)illFr!

clirension p = 1, since the

for

p

=&pr,

frnm

nernendierrl:r

tlte

cxtrem-

o f L , o n t o t l r e s n a c e € . c o n t a i n i n g P ; a n d t h i s p e r p e n ri lc u l a r

ity

1s given by the vector (1 - f)Er, wheref = f(f'F;-1pr ^L+^:-^.1

V | L d r l l g l l

L., U /

orthooonnl

e

r

= I

I

+( d^r 1. l. r : ' ll r

O r t h O f l O n a l.

cnmnlenent of

ft'L ) T

tliC

- 1'

T '

l'.'l'lren T

T v ,

nr^innfinn

s o tr h. .as t!

,

n v rf

l ' t V J l L ( I v l l

E'f1 - f)f T

f

pr f 1 = f- :f ( 1 . _

-f)

= lE't I

lt

T '

T

f-

On

the

E -. " I .

Let

- fe

I

I

h y 1 ' h e o r e n4 . 5 . , \ c c o r d i n g l y , h y t h e i n t lu c t i r - e h y p o t h e s i s a . '2 = L&l

I, F , and by Theoren4.3 again, J ) ) = 1 Q l =2

J l l p / 1 r ' r 1 r t ; t , r 1 1- f e r l = l l r l c o R O L t , .A RIYP , L l = I p l l . 9 - ls i n ( 0 , 9 , ) . F o r ,i f , t = h l , 3 1= f n l, t h e n l p ^ & l = l f r r=l ll I c l l ; and

l c ' c l = l r ' r ll l r l

II-"rI

o b . nu* r+ r1 n, ^o^ ^g^ o^ ln ari l -- -ar c l o n

lf

a b e s e l i , c c n p o s e do f p i n d e p e n d e n tv e c t o r s t , ' , . . . , L p

is given, then an equivalent orthonormal base U, conposed ^r

u l

P

^ - +L Lr u^ t^l ^u r- r^drr

u r

,u rl n i l l IL

rv ,gpr erL nU rr c)

lu t1

r .

..,

cen

be

COnstfucted

I '

w i t h e a c h U , i n t h e s p a n o f E . ,, . . . , qrrnn^qe the

IJ-,

eonstluction

be

can

Br (r = 1,...,p).

carried

o u- t-

f- o- r-

U - r_

For,

(r=1r...,

n - 1 ) . L c t I r = ( L t . . . L m _1 ) , \ ' = ( U t . . . U n _ 1 ) ,s o t h a t s i n c e , b y h y p o t h e si s , U

the bascs f,

= \'r\tr;and let

\

a r e e q u i v a l c n t , I F = I r n -I =

Jn_1=I-Im_1.

Then the vector

- 9 4 -

U = m

r/2 fE'J .t )n r -

,t

n

s u c h t L ^ + l' 'l

1r"....U :

J

l

m - l

vector/orthoronal to n /

a unit l

is m

' " ' ' L l rE

^ "d

d li,....,ll ' r '

have the samc

m

s p a n . T h r r st h e c o n s t r u c t i o n c a n b e c a r r i e d n s t e p s i f can be carried m - 1; thereforc, since its is evident for m = 1, it nnqsihlc

.n ,

fnr

L,,...L-

tlre orthogonal

pro icctor

follows by induction that it

I

U* = ilr

on tlre orthogonal

llris

connlenent

and

(' .I -rn: "Jn- -- l, 8- '-n)' - t / Z l ^" n -

r [ 1 = rI t . .- - p ) , "

r' l"n

are n nrtlrncnnAl rrnj1 VeCtOrS SuCItt]rat

U

1,,... r[-

is

a r e p i n d e p e n d c n tv e c t o r s , J n _ I

t l r e s D a no f F , . . . . . t

then I

possibilitl'

p.

=

T l I L O R L8I l. 1 : I f

of

it

a n J l ) 1 , . . . r 1 1 -h e v e t l r c s a n e s p a n ( n =

t l r e o r c ,r tc i. v" o s

r - , r n li c i t

fornulae

for

thc

1,...,P). Crann-Sclrnidt

o r t h o g o n a l i z a t i o n p r o c e s s.

9. Canonical deconrr_csition T I l l r O R E9I l. 1 : I f lI-fc

(I I

I-et,

tnen

€ , , F ^ a r e t h e n u 1 1 - s p a c e so f

x1-ef ,

are thc null-sfaces of 1-ef, O), an,l Lo,T o

T . C -

whcre thc spaces E- , 1

A F v ( . ) I

^}

t

=

} @

"

t

A ' V X ^

., are ident ical , the spaces 8^ , i,

( r I O ) a r e i s o p . o n a ,l a n d t h e s p a c e s E ^ , ( , r ,E; 1 , I , , ; t

-

+

J.,Fu

(\l

u ) a n d e . o , F o a r c o r t l t o g o n a r l A. l s o ITI I

-

e^ = (F € \i, r=1

llr

lIt= @ i=1

7), j

()lo),

- 9 5 -

wlrere t, e r o

rr rv i i tLl I' I

ri nr rn ! l ri rn r or ,! lu ,

li l- l. L1 li an lol fd L l U i l nl n

J

r

, J

nf

rrvs

r

Tlre sot

.

"

"

,

-

].

j L

onnnl

n r n i e c t i Ln t nu

B u r r d r

f

,

, . ' r

in /

,

ti l. L7

l l

l , r u J q L

,

^ - l d l i U

t t \ 6 t i l s

h . : - " P d r r -

F " l

r i t

, J

tlre ortllo-

ntaybe obtained by taking

,

^ vf

ae t nL v l

vn r r

sets

of

u L

an,t t\6,p the

r'jl!)'

^I

; , T \ i ( i I j ) a r eo r t h o g o n a l '

&x;i et;,Tt;i7r '',

'F,

, , ? ], \ , ) i a r e r a y s s u c h t l r a t t h e p a i r s f r' ,Y ,

, , 1 '

t Ll rl l 6v rt ,sv 6r r nu 2 l

rays t,

"trrl

st-.t of

.rr

rt sa ) v S

. and 7)

:

are

the

orthofonal projections oI each otl)er in e ,]

rl

If

C ^ . , T ^ ct e "T

0 then d

.

r

-

By Tl,coren 5. J, €.^ and

Y

J-

a r e o f d i n r e n s i o nn 1 ; a n d h y

T h e o r e n 5 . 4 , t h e p a i r s o f s p a c e sC ^ , ? , a n d 7 1 , i ,

( ^ ,L 10,I lu)

a r e o r t h o g o n a l . R y T l r e o r e n5 . 2 , C o r o l l a r y , e 3 a n d f € a r e o f c l i r n e n s i o nr = ,, 4r u ^ "r.

It

follows now that

tL = Eo\

^ T = L EI )f 0

F u r t h e r , b y T h e o r e m s3 . 5 a n d 3 . 6 ,

L= eYAeo, 7= f e@To, wherego Ls orthogonal to e Iand By Theoren 2.5 , e,

to cF and € intersection

T , anclFo is orthogonal

of

€,T

a n c tT - , , a r e e q u a l t o t h e

and therefore idcntical.

Now, cornpleting

t l r e j - n i . t i a 1 s c h e m eo f d c c o m p o s i t i o n , t h e s f a c e s e , (

tr, u I A,

f

I l u ) are orthogonal, by Theoren5.4.

Let l1 = (ti

. . .l ^,rr^ ) be an orthonornal base for ,1 a n d C , i t h e r a y s p a n n e cbl y t h e v e c t o r E l . ( i = 1 , . . . , n ) . t

I e t I r n, r.1 = ( _rAi

t

. / L- I r l . ) - t / ' f .L -^^ ,,

L

he a unjt vectorin the

, 1 ,

which is the orthogonal projection - rn. , . =/ e r 'lhen, (P- , . i n ?. } by Theoren 3.4 , r a y ?- ( .l r

efli^ = E^r and

"F.l

= ,^"/2,

, feF^tr = Frr fq

8^,

= F^^l/2'

,

of

- 9 6 -

Ei Er = 1, riF^ =1, EjF^= 1x7/2,

where --r dllu

-Lj^ Lrra)

^j.-^^ t

.l.e LIls

r so (nl rur r. l iqr ue d

\c un l nl LC l rr tl S i O n .

on Ci

Now the orthogonal projectors

f^ = F^lr,'

= Fl Ei ,

ct

, )^ur" ,

and the non-zero characteristi c values of e , rf r = E ) t i f l F i are the sane as those of E A\ FA . F A. ' L \ = l ) ,.1 .j ^L lvlLlLIr

i -

+ ^

I )

^ - , ,

Jdj/

LU

+( rl rl \ n v

n d rr e\ : l I

)

t hn

Sar ne and t

I'hcrcforc or

thcse slaccs arc

l

tO) .

Y

betwecn L^, J), are

anglcs

tlre cancnical

eOttal

e q t t a . l,

isogonal.

f =I f 9Y .11. e =Ie l, COROLLAR t r l nrthnonnrl nroieqtorS Such tl)at

f^ are

, tr']re r e e ) ,

= f 1 ; e ^ e u = C , e l f u = 0 ', ^ f . ' 1{ ,) = O ( r I u ) ".1 t r a c e e l = t r a c e f . \ = n ) . , t r a c e e . f , = lA n , ^ r l = o )

eofo = 0,

o . 1 ls p a c e s ( ,

Jn any pair

C 0 R 0 I . L A R9Y. 1 2 .

there exist

F

b a s e s E , F s r . r c ht h a t l

-

t

=

| l

t

t

-

F

=

|

t

!

t

r

-

0 ' p r

(:

0

ln

Iloeinrneal

"rd

eXtfenral

COnditiOlS

T l l l ' O R I l I1l 0 . 1 : T h e o r t h o g o n a l n l o j e c t i o n a given space is

or ra)'in

: . p g : g _r ' h l j h _ h a : _ % l I e m e fL li unl nt

tt nu

tt lt rl ge

c fi u

vt nu rl l

yggtOf

@

t h e u n i q r . r ev e c t o r o r r a y i n t h e

distance Of

:)

or coefficient ^^+:"^1"

t'av

of ^-"1

r g ) l J g L ( l t g t y .

d l t l

i.nclinat l r _e

extrente 1s a mrnlnlLln. Let

Zo

b e t l ' r e {c_i 'r^r"n" n \ r e c t n r

's_1r)-a* n, ,n, ,i n - .p, ttrh e nroiection

of

or

1et

Z

o

be a vector

T- .l 'i 'e. r. r .X. o = -I -I i -7 -i 's- t h c o r^ t.h' ' o - pr 'o- 'n a l o o , ., t l r o v p c t n r i r r t h e Fo i l - e n s l^l a c e - . l i t t n l)ase

piven

rav.

o '

-97 is a vector which spans the orthogonal projection

t^, or X^ o' o

of the given ray in the given space, and for an1'such v e c t o r X . L^ LZ - o = T ' t o "7o ' X = Eox is any vector

If

hotweon 7 -

"

'

o

s

r

r

\

'

in eo, the square of the <listance

enrl Y, and the inClinati.On bettveen the Iays !

/

!

'

s p a n n e db y t h e s e v e c t o r s ,

are given by

(. -zo - yr r ), ' r\ "7o - ,y. /) r (, -zo '. x. , ) 2,/ (. -zo -' o^^z) ( x ' x J. They are extreme just

for all

(' Zo - X ) ' 6 X = 0 , i t ^ 8 { - ' ( Zo l X ' X / ( X r X )= 0 , - )X6 o dx, where 6X = i:o6 x; and these conrlitions are f rY =

onrrirrrlnntt^

and then to n c r e n r r ir c . l X , ' o ,

:rc o r L

w w ai Ltl h l

7

40

if

-f or ' TX - 7o

'F' or - o '

=

ltl "o-o t

X = IE Zo, IXZ. = I, Zo, o o T f f o I l . o w S t' h -'a - t- X " o^ t, a n d t h e r a y S p a n n e db y

iu lr nr r i\ nt ur! rr e / l v . r o r a r n ' i n a . r

e n d tL hl l es

drlu

r1 ^4 /"

hV

the

+L rrr '-r v^u' t,<^l rr ' Lao , '

-nd

d.

Stated

i t

is

eXtremal

COnditiOnS

that

evident

tlre

extrenes are ninina. T I - I D O R EI IO{ . 2 : T h e r a y s i n a g i v e n p a i r extreme inciination nf rews in :-:---:-:J--:----:

Tliat is,

thocc

of spaces which have

to each other are the reciprocal pairs

sPaces

'

the extrenely inclined

rays are the rays which

are orthogonai projections of each other in tlie spaces; and these have been seen Lo be tlte pairs of

rays spanned

hv the rcr-inrnc:l nairs of latent vectors of the nroducts '

of the orthogonal projectors on the spaces. that if

the inclination

one space or the other,

l -

The proof is

is extreme for variations

rays in both the spaces, then it in just

_ " -

of the

is extrene for variations so it

fo11ows, by Theorern

9.1, tlrat thc rays are orthogonal projections of each other

- 9 8 -

' in

th^qF

sn2rcq

The di,rect consideration of the extrenal condition gives a n a l t e r n a t i v e f J r o c e ( l u r e o f c o m f , u t a t j o n .t h t r s , l e t X = E X , Y = Fy be unit Then for

{x'E'FY)

all

rays.

vectors spanning extrenely inclined = 0

6x, 6y of x, y under the constraints

variations

x ' E ' E x - - 1 , Y ' F ' F Y= 1 . Lquivalently,

lrFy =

F'lx =

L'lx,

F'Fy,

w h e r e ) , L a r e t h c L e g r a n g i a nn u l t i l l i c r s

c o r r e s p o n d i n gt ,o

Then

the constralnts.

= x'E'Fy=p .

I

B u t , r n ' i t h I = u , o n e o f t h e c o n s t r a i n t s b e c o m e sr e d t t t t d a n t , s o there is

the following

conslusion.

TIlE0liLl1 l 0.3. A necessary and sufficient

condition for

p = x'I-'l)t

of x, 1' uncler the

t!

!9

eflf-gn'e for

c o n s t r a i n t s x r I :t l x = 1 , I ' f ' l y

E r F\

= 1 is

/*\

=. +r'rJ (.ru

l-ro'L

\

variations

r'r.

C 0 R 0 L L A R1 Y0 . 3 1 . T h e e x t r e n e v a l l r e s flra

anrref

,

X t L t

Irx = 1 .

are the r o o t s o f

inn

-PI-'L

Ir'f -pF'f

f'L

Tt has been seen dircctly,

i n T l r e o r e n3 . 4 , t h a t t h e s e

tt r o o t s s q u a r e c ia r e t h e r o o t s o f t h e e q l r a t i o n I p

-

trIr, I = O,

r v i t h t h e s a m er n u l t i n l i c i t i e s . T l l l . O l t l i1\ i0 . 4 . l l r c n u l l i t y i.s thc

nul_tjgl i ci ty

of

of the natrix as a root

of

/ - -o; L, ;' L .. K- =i " i.. l L

tlrc cquation

L'f

- p I .F , rF

K

ol

= 0.

- 9 9 -

This can be proved indirectly

as fo11ows.

established, or directly

orthonornal

are equivalent

fron

results If

[l = Ec , V = Fc

bases, 1et

""'\ r{= o ( - ? ' u l } , y = (/ o" _oj/

\V'tr

already

o\

\o

S/

)

so that Kp, li have the same nu11ity, saY Then Y'\., = W or o D p , a n d * o l = O , I w o l = O h a v e t h e s a m er o o t s , t h u s p w i t h mrrltiplicity rno

l ' J , b y w h i c- l l

a cilaracterj.stic value of the symnetric natrix it

follows

of oas

But mo is now the nultiplicity

that np = tp

/Xn\ If|| " I denotes any solutjon correspondingto a root p , \yol j.t folIows that there exist no suclt independent solution,

and these can be chosen so as to obtain no ortliogonal vecto r S /x" t\ = 1 , . . . , m ) w h e r e x o = E * o , Y p - F y p . N o w f o r a nv tr"'1lfi R'1 Q, a , evidently sdch vectors, belonging to roots pX YJ Xo = o X'oYo oXp , XoXrg = XoYp = oY'oYo

YJ Yo = so that It

follows that

1f X;Xe = O, tlien, provided

Q , ,o I

O,

Also it appears tl"Iat xJX o = o, x;Y o = o and YlYo = 0' = p:o and X;Xe = Y;Ye i'f o =p. XJ p = O agtornatically if It

is now seen that

vectors *0,, Tft

there are found n p orthogonal unit

t n E a n c ln o o r t h o g o n a l u n i t

= i,...,m),

for eachnon-zeroroot p , such that

X u' ^r l. Yp ^r J; = O j f The final explicitly

vectors Yo,i it

olp ot i I

j

andX'-iYo,i = o' " t -

theorenr,which is easily t h e e q r - r i v a l e n c eo f

verified,

shows

the three conditions

fornul ated

between pairs of vectors in the spaces, that they have extrerne j-nc1i-nation, that

t h e y s p a n r a y s w hj - c h a r e o r t h o g o n a l

-1 0 0-

nrniFrtinns nf

the

how

nf

oech

nrodrrcts

the

otlrer. the

of

nrflrnonn:l

in

of

calculation,

in

of

solutions

pnrref i nns

2Te

ternts

t TanSfOfned

into

T t I E O R E 1I l0 . 5 : P r g v i d e d p l

lrtcnt and

of

and

nrc

nrn iectors:

t1{o processes

\-ectors,

tlrpv

anrl th2t

of

terms

the

veCtOfS it

shows

latent

extrerr"ality

eaCl'r Other.

O, the equations

L ' F y = p t ' E x , I r r E x= p F ' F I , x ' L ' L x = 1 . T ^

q

r

a d r r i r r q l e n t

\

\

\

t

u

r

!

u

4

e

r

r

l n \

r

l l r a

v

\

r

'

\

c n l t e t i n n c !

Y

v

e

v

r

v

r

r

v

e Y = X p , f X = Y p , X ' X = 1 enrl

to

the

r . o r r at i . o n s

efX = &"

7

- 1' r/ '?f X , , y = (X'fX)

XrX = l

and to thc other sets of equations obtaineC by interchanging \

=

Ir

enrl

=

Y

Iror gencral is

rnadc to

in

Afrjat,

included lltis of

Fr-

in

flroce

rccolrnts of

rr'itlt ccltain

tlre tlreory of

for

the

naner is

sal(e of

nethods for

rvorL tlonc at oripin

trlrielr nnnperod

T wi,slt to

tlrc

s u\ t lu tJi \ o c! t L.

^

projcctors, Ilalmos.

refercnce

ltlaterial

natlreratical

featurcs

tlrc Lepartnent of is

Applied

in an attcmpt to nake in

nn

invostiontiOn

Of

t h e s e a s o n a l c o r r e c t i o n o f e c o n o n j - ct i m e s e r i e s ; exnreSs

rvith

thankS

Prof. J.R. N. Stone, botli for th is

=

c o n , f r el t e n e s s .

a rlevelonnont of

tho:lonl'rn

YrY

lras lrerc been

irprover,cnts,

I c o n o n i c s , C a n r h r ' i d t . ' .I t s

end

innlv

tlrev

IlanbLrrgel and Cr j.nshaw ancl to

son:cstatjsticcl

cnneiso

;rnd

,

and q r r u

dor.nlnnnront of

al S J UO fr vOr f o l

nv

indohtedness

introduction

f9

to his work in

fL rl lrr o n r rr i nr i fL j' /' a l Ir O u nW f r ri Fn o U PnI n unr r f L u d l

t" l" r- o n rr.qcnt t'

mAterial.

A

statement

ncthod for the analysis of seasonal variation, gencral noint cf vietr of rnultivariate

r^-

r u r

of

frorn the

analysis towards

_ 1 0 1 _

which tlris naner'i s direcferl. l

Also

I

have

especial Iy

to

ideas; of

and to

of

in

L

r

is

!

in v

,

corrrseof

Adams, l'!r. Alan

Irr, Il. I'cnrose, witlt naterial,

regar(l to

our collaboration.

thc

1

,

indebtedness

di.scussions ivhich did

tlie algebraical

esfecially

\

acknorvledge ny

lr1r. Arthur

Ai tclri son - for

^

^

'

to

nrenaration. r

ny colIeagLles,

Brown and l'1r. John nucli to

trlronI

lrati tliscussion

and who rnadevaluable l e m n ' a so f

S 6,

these

elicit

suggcstions

tv)iicltare

rcsults

-LoziII APPLI,],IX

1. Orthogonal Prgjectors

tliere 1s a saInPle experirnent wl-iere bY a The clata is dc;cri.bed ' v a r i a b l e s p on of n observations e R I I and column space . -, € . n rn ,w. ,r i t h p c o l u n ' s X l , " ' ' X O t " X natrix Consicler a statistical

t = X R P

(1.1)

-

l-xt L"-

:

-- rL\ " 1t ' 1

+

'

r

tlll .

v

+

"p'p

t . l, . . ' , t p € R ]

Since

r a n l i X = P ' a n kX ' X ' i.t

fcrllorts

that d1n

which sllows indel-etltletit ' tilat

r : + ; ^ r cotltirLrvrt

tite ^J l r l l

r

lr r !n rr ^r \ e n !

to

t,

-

!f vn rr fOff

1i t r e a r l Y X . ' , . . ' , * Ot o b e

t l r e a

base for their

case

SPOI) r-'

rtL

-lx'

(1.2) i s c i cf i n e c l '

l x ' ) 'l l o '

t

I'

it

is

e = X(\'X) I = € r f r o r nw h i c h i t ider.potent.,e

ranll e = trace e' Irrtt

'X' 1

tracc

e = trace X(X'X) - t^ X ' I trace (X'X) tI'ace r-

-

y.

folloris that

-1 0 3 -

T h us c ji n e p . n = r a n k e = p I ; d i r n E . But, from (1.1) and (1.2), obviouslY elin c l, and this,

of the dinensions, inplies

rr'ith equality

elln= t.

(1.r)

T h u s e i s a n i d e n p o t e n t w i t h r a r t g eL . l r o n

(1.2) also it

is

e . B u t a n y s y r n n ' e t r i ci d e n ' p o t e n t i s i d e r r t i c a l

synnetric, c'=

rnith tl.re orthogonai pro jector

onto its

r a n g e . T h r - r se i s

the

n r r l r n o n n e ln r n i n e t c r o f L . l l r e n t l r e c o n p l e n e n t a r ' ) ' l l o j e c t o r c = t - e is

thc orthogonal projcctor .

o n t h e o r t h o g o n a l c o r n p l e n e n ti

,,n

oT t. 1n 1..

I n d e p e n c ci n c e

2.

Ye nl, p'

C o n s i c i e r a f L rr t l r e r that

lY'Y I I

coltrnn space f ,

strch

so that

o,

f is

"ith

dcfinecl, and is

= Y(v't)-lYl

the orthogonal

projcctor

on F.

t{i th

Z = f ,X. ,Y/ l _€ .R. Jr r + q , t h e c o n dj . t i o n f o r t , equivalently

dir,'lvF

is for EA F = 0,

= d i r nl ; + d i m F

lZ'Zl I 0. Rut there is the iCentity (Afriat 1957),

l : ' : , 1l /x ' x l l v ' Y li =x ' i x l z l x ' x, l

(2.3) where T

t o b e i n c l e p e n c ' l e n tt,h a t

for

(2.2) is that

I

= t

(2.4)

-

[.

I]ence thc condition

! r ' - r x Il o ,

is equivalent to

- L O 4 -

and sinilarly

t

to

i n d e p e n d e n ti f

-

l

lY'eY I I

O . A c o n s e q u e n c ei s

tl-ratF, F are

the orthogonal projection of E on

and only if

the orthogonal conplenent of F has the samedinension as L. For, since i

is

symrnetric idenrpotent, din,fl

= ranl. fX = rank X'fX,

s o t h e c o r r s i d e r c dc o n d i t i o n i s e q u i v a l e n t t o d i n I f

= p.

3 . 0 b 1i q u e p r o j e c t o r s L e t L , I r b c i n d c p e n d e n t ,c q u i v a ) e n t J y l r a v e j o i n C = I vI-'

(3.1) w l l er e

din' C = p+q

(3.2) 'll'on

Lrr

-f ^-r r, . ' 8 c ) r n f,

+ha

(3.3)

e

x

- l = X(X'fX) t\'I

i s r l e f i n e d , a n c ls i n ' i l a r l y s o i s

fx

:

Y

Srncec p.

It

t s r J e n ' p o t e n t ,i t s

r a n l . , c q r r a Jt o i t s

f o l l o w s , b y t l r e a r f l u e n r e n itn S 1 , t h a t i t s

trace, is

range is l.

l{

Silrilarly

g.

has raniic Ir.

Consider any elelrent ) C r'itli

t l r e ' o r t l r o t o n a J c o n r p J c n c n tC o f

X ( r h v ' ci u s 1 1 . e y = 0 ,

= n-p,

fol loris that

F - C is

P,rr onv i,rannnr^'tt is r ' 1 r r f l a n e r a l l n l

'ora'e1

equal

identical idcntical

to

in

the

join

the joir, C of

^ so the nu11's1,ace of

c l l n , e n s i o nq * [ n - l - q )

i + c

FVe, tlrat is

L antl f.

its

din"ension. It

l,;ith the

nu11 space of

with

F

X i- contai.ns I:\rC, of

thc nroiection X

n rur 1 l l 1 -s p a c e . l l e n c e tt ^o ii .+ tcs n e- is

fro.iector on L parallel

of

el

onto

the

to the join of F rvitlr the ortlrogonal

c o n p l e n ' c n to f t l r c j o i n o f I . a n d I i , a n t l t h e r e i s a s i n , i l a r Y

staten,ent tor

t

- 1 0 5 -

Niow

l"* = irlx'Ix1-Iv'6'

(3.41

appears as the orthogonal projector Itence fJ

the orthogonal projector

is

p r o . ie c t i o n o f t It

on the range of TX on the orthogonal

in the orthosonal complenreno t f F.

is noted that x Y Y X e - ' f" = 0 E f ' e ,

(3.s) Y

Y

s o t h a t € , f - a r e i n d e p c n d e n tp r o j e c t o r s , a n d l r e n c e t h c i r sumse

x

x

+ F- is the projector onto the join of thei r

and para11e1 to the intersection T h o f

i- c-

F = c

E ,

* ^ ' . ^ 1 1 ^ 1 lrdrarrr-a

-!ll

( F v G )n ( t v c ) = e ,

(3.7) wlricli shows tlrat

*

J

(3.6) ^A

n r - r l 1s p a c e s .

^ n t ^

t

(3.6) ^ - - t drru

of their

ffi is +

ci

!^ u^ r- ^u rr 1r ^d *r .) ,

; -

rr

+

+ l

ortllogonaL frojector

t

. X-

f

;

is

synrnetric,

wlrich

is

e L rv r ri ud\ e n t l w

x x e e = € , e g = e

i i . S l i

f

e n n p . T q

f h r f Y

(5.10) q n r l

c i m i

Y

+ ef "=

c

Y

It

e

I e r l v *

f ' + folLoivs

f e " =

f .

that }

(5,ii)

*

c-' =cf-+ f Y

an(l llcnce

-

e'

}

fe'

Y

= c + f Y

= I"'-

+ f

en:ririr'l cntlv

""'-r

(3.12)

x

x ie

g on G

fx +

o-

trtcl

the

o t h e n i i s c i n n e . li a t c J y a f p a r e n t . Sincc

ranges

+ f

= ef

+ e.

Y = f '+ fe'+ V

ef "*

e,

e

not

_ 1 0 6 _

_ x anC f both fe n r n r l r : e t i s n r : ll r r s r 4

^ r

!

f( nv

sun js the orthogonal projector on

so their

t

the joi.n of their that,

are otthogonal projectors and their

for any L, F q Rn, if

f( hr r eL

n r f l( rj nr vc! n' vnr er o' rl v r

is concluded frorn this

ranges. It

= C then the join of E

f / 7f

nl / rr nv _i e e t i o n r \ ! r r v I

n v rf

F r

i n t o

ir t \ cJ

r r r u v

is identical with tlrc join of I'to of E into

identity

n v rr t h L lov ol .nv n l uar l

c! v n, m ' P n 4 \l rer n v rer n ! t

t l r e o r t h o g o n g fp r o j e c t i o n follows that the

It

the orthogonal conplenent.

spacc in which they cojncidc contains hoth I and F, and hence their

jcrn

with j.ts trace,

C. lforeover, the rank of lex is l.

appears from this

It

s i o n o f t i r e s p a c e s i s p + q . t s r - rt h a t q n n c o ( i r i l ri c l r i t

'-- '-:-'-

that

the dinen-

i.s the dinension of the

^ - .I rl l 'L ^' n c c i t

L U l i ( d I l l > ,

, identical

d l l r l

is iJcntical r'ith C.

is establi shed that _ x _ x (3. I 3) fe + f = g = ef * €,

l'hus it

r'liich is

4.

thc

jdcntity

requirerl for

tlrc fol Ior'ing problen.

l Ghosh's Problen, G o s i . ri r ' a n t e c i t o (4.1)

fX(\'fL)

donljnates tllat qr\on

tl'o

tiitlr

forrcr

- X rilrcre lr = cf is

(4.3)

rnatrix

lllttrir

the

- 'r X ' f

qrlarlratic

But,

c.

= fc

+ f

fcrn'r'itlr

natrix

x + f

by u'hat has just

been

is

g = h + c,

(4.?)

neFative

tliat

llove

definite.

x synnctric It

i r le n ' p o t c n t , a n d t h e r e f o r e

fo1lorr,sthat,

t'g,t = trht

for

a1l

nnn-

t,

+ tret

.) ttet, sir.ce t'ht

C, and thjs

is

tlre reeuired result.

'l'lre p u r p o s e h ' a s t o s l r o r ' l' r i s f u 1 1 i n f o r n a t i o n t h r e e - s t a g c I e a s t 1 s q r r a r c s c s t i n ' a t o r ( 1 : i S S L S Jt o L e a s y n ' f t o t i c a l l y m o r e e f f i c j c n t t l i a n t h e Z e l l n e r - l ' h e i 1 3 S l S . I I t l L a r . r d2 S I - Se s t i n a t o r s .

_ 1 0 7 _

tern"s of

on a space in

the orthogonal projector

The formula e for

the elements i-n any base E was obtained in Afriat

(1957). Beside applications

l.ras

in regression tlieory, it

had use in progranning theory, in the gradient projection m e { i h o d ,p r e s e n t e d b y R o s e n ( 1 9 6 0 ) , a n d a l s o b y t h i s

rvriter

' in n n r i \ / 2 t e c n n r r n u n i c a t i o nt o L . l i l. L . B e a l e i n 1 9 5 8 . use, t{hlch has found econon'etric applications,

A further

is in the fornula trace ef,

(l!57), which provides a feneralization of the

in r\friat n , ' ru rr l tL i ,nt l, ,n L

the theory of which is developed

reornqqinn

ennf f ie ient

a n d r v l r i C l lc o i n C i d e s

with

d e a l t r v i t h b y I i o o p e r ( 1 9 5 9 ).

the statistic

A 1 1 t h e f o r m u l a e w h i " c hf e a t u r e i n t h i s

paper, but the

last which bears on Chosir's problem, are treated in the in the nicthod

paper of 1957, though there is a difference

0ther developnents are in the paper of 1969,

of argurnent

w h i c h I r r e s e n t s t h e e a r l y l a r t o f t h e u n p t r b I i s h e dp a p e r o f 'fhe l atter part of that paper sllowsthe bearing of 1960. tlrn

annroaclr

on

for

correlation

n:rti:r

I

7 ' e c f ' o s- -s i

- o - l -l

^-^1"';

d l l d r i / J l J

trvo groLrls of variables,

'^rical

t

L d r l u l l

and sore further

a n a l y s i s w h i c l ' ri n v o l r ' e s a n y n u r " b e r o f g r o u p s . A further

application

forned by a pair

is an analysis of the configtlration

of l:uclidean subspacesand deternrination of

thc canonical angles and directions. for canonical correlation

This Iias interpretation

theory and is presented in the

paper of 19-56. ine origin for

of tlris projector tlreory with interpretation

regression analysis is

i n n a r t i c r r la r o n t h e i l

in work on economic ti-meseriest

seasonal correction, donewith

P r o f e s s o r J . i l . . \ S t o n e a t t h e D e v e l o p m e n to f A p p l i e d E c o n o n i c s t z m h r rr e. l ii o F s ,

rn

t u\ 4.

BIBLIOGRAPHY the characteristic values of matrix functi-ons. Quart. J. Math. (0xford)(2) 2,81-84

AFRIAT, S.N. (1951): Boundsfor

AFRIAT, S.N. (1956): 0n the latent vectors and characteristic values of products of pairs of synmetric idenpotents. Quart. J. Math. (0xford) (2) ' 7'

76-78

A F R I A T , S . N . ( ' 1 9 5 7 ): O r t h o g o n a l a n d o b l i q u e p r o j e c t o r s the characteristics

of pairs

and

of vectors sPaces. Proc.

Canbridge Phi1. Soc. 53, 800-816 A F R I A T , S . N . ( 19 6 0 ) : S t a t i s t i c a l of orthogonal projectors

an investigation

projection:

in regressl'on analysis.

Research

Econonetric Research Progran, Princeton Univers.

Memo. 16,

Fox, K.A. and

AFRIAT, S.N. (1969): Regression and projection.

Narasimhan,G.V.L. ed. Econonic Models, Estimation and Risk Progranrning. Essays in Honor of Gerhard Tintner. Berlin,

Springer.

AITKEN,A.C. (1945): 0n least squares and linear

conbination

of observations. Proc. Royal Soc. Edinburghr SSr4? A N D E R S O NT ,. W . ( 1 9 4 8 ) : T h e a s y m p t o t i c d i s t r i b u t i o n determinantal 'l supp1. 10, 90

of certain

Analysis.

Royal Stat.

equations. Jour.

A N D E R S O NT ,. W . ( 1 9 5 8 ) : A n I n t r o d u c t i o n

of the roots

to Multivariate

Soc.

Statistical

New York. Wiley

A N D E R S O NT ,. W . a n d R U B I N , H . ( 1 9 4 9 ) : E s t i n a t i o n o f t h e p a r a m e t e r s of a single

equation in a complete systen of stochastic

equations. Ann. Math. Stat. B A R N E T TV , .

55, 650-659

(1973): Conparative Statistical

Inference.

New York.

W i1 e y B A R T L E T TM , .S. (1947): Multivariate

analysis.

Journ. Royal Stat.

S o c . s u p p 1 . 9 , 1 7 6 - 19 0 B A R T L E T TM , .S. (1948): A note on the statistical of

supply and demand functions

Econonetrica,

16 , 3 2 3

fron

deternination

time series.

_ r4 3 _

B A S M A N NR, . L .

(1957): A generalized classical

estination

linear

nethod of

in a structural

of coefficients

equation. Econonetrica, 25, 77-83 B O X , J . E . P . a n d C O X , D . R . ( 1 9 6 4 ): A n a n a l y s i s o f t r a n s f o r m a t i o n s . Journ. Royal Stat. Soc. B., 26,211-743 BOX, G.E.P. and TIA0, G. (1962): A further via

look at robustness

B a y e s t h e o r e n . B i o m a t r i k a , 4 9 , 4 19

BELLMANR , . (1960): Introduction

New York.

to lt'latrix Analysis.

C H R I S T ,C . F . ( 1 9 6 6 ) : E c o n o m e t r i c M o d e l s a n d M e t h o d s . N e w Y o r k . Wiley of quadrati-c forms in

C O C H M N ,W . G . ( 1 9 3 4 ) : T h e d i s t r i b u t i o n a nornal system, with

to analysis

applications

of variance.

P r o c , C a m b r i d g eP h i 1 . S o c . 3 O , 1 7 8 - 1 9 1. D E G R 0 0 T ,M . H . ( 1 9 7 0 ) ; 0 p t i n a l

Statistical

Decisions. NewYork.

N ' l c G r a wH i l l D H R Y M E SP, . J . D I X M I E R ,J .

(1970): Econometrics.

NewYork. Harper and Row

(1948): Position relative

de deux vari6t6s

fern6es dans un espace de Hilbert.

Rev. Sci.ent. 7, 387-399

FINETTI,B. de (1937): La pr6vision; ses lois sources subjectives.

Ann. Institut

F I S H E R , F . M . ( 1 9 6 6 ): T h e I d e n t i f i c a t i o n

lin6aires

logiques, ses

Henri Poincare 7, 1-68 Problem i-n Econornics.

New York. Ntc Graw Hill F I S K , P . R . ( 1 9 6 7 ): S t o c h a s t i c a l l y

D e p e n d e n tE q u a t i o n s . N e w Y o r k .

Hafner F L A N D E R SD, . A . ( 1 9 4 8 ) : A n g l e s b e t w e e n f l a t

subspacesof a real

n-dimensional Euclidean space. Studies and Essays Presented to R. Courant, New York F R A N K ,C . R . ( 1 9 7 1 ) : S t a t i s t i c s

and Econometrics. NewYork. Ho1t,

Rinehart and Winston G E A R Y ,R . C . ( 1 9 4 8 ) : S t u d i e s i n t h e r e l a t i o n s tine

series.

Journ. Royal Stat.

between economic

Soc. supp1. 10, 39

G E I S E R ,S . ( 1 9 6 5 ) : B a y e s i a n e s t i n a t i o n 'l 5 0 - 15 9 Ann. Math. Stat . 36,

in nultivariate

analysis.

-r4 4 -

G H O S HS,. K . ( 1 9 7 1 ) : F u t l

infornation

Waterloo Economic Series,

squares estimation. G O L D B E R G EA R.,S .

three stage least 27

( 1 9 6 4 ) : E c o n o r n e t r i c T h e o r y . N e wY o r k l t l i l e y

G O L L N I C K ,H . ( 1 9 6 8 ) : E i n f i i h r u n g i n d i e 0 k o n o n e t r i e . q r r r fr o n r t . I l 1 l n e r . ! .

e L s e ! 5 s r

G R U B E RJ, .

(1968): okonometrische Modelle des cowles connission

Typs. Hamburg. Parey H M V E L M O ,T .

(1944): The probability

approach to econornetrics.

Econometrica, suPP1. F i A D A M A RJD. ; ( 1 8 9 3 ) : R e s o l u t i o n d ' u n e q u e s t i o n 1 6 l a t i v e a u x d ' e t e r m i n a n t s . B u 1 1 . S c i . M a t h , ( 2 ) , 1 7, 2 4 0 - 2 4 6 H A R M A NH, . H . ( 1 9 6 S ) : M o d e r n F a c t o r A n a l y s i s . University of Chicago Press.

Dinensional Vector Spaces. Princeton.

H A L M O SP, . ( 1 9 4 8 ) : F i n i t e Princeton University

Press

f l A L M O SP , .R. (1951): Introduction-to theory of spectral

2nd ed. Chicago.

rnultiplicity.

Hilbert

space and the

New York

W ., E . ( 1 9 5 1 ) : L i n e a r T r a n s f o r m a t i - o n H ,. L . a n d G R I M S F L A M HAMBURGER in n-dinensional Vector Space. Canbri.dge H O C H S T A E D T EDR. , a n d U E B E , G . ( 1 9 7 0 ) : O k o n o m e t r i s c h e M e t h o d e n . Berlin.

Springer

H O L Z I N G E RK, . J . a n d H A R M A NH, . H . ( 1 9 4 1 ) : F a c t o r A n a l y s i s . Chicago. University

of Chicago Press.

S ., C . ( 1 9 5 3 ) : S t u d i e s i n E c o n o n e t r i c H O O D ,W . C . a n d K O O P M A N T Methods. New York. WileY H O O P E RJ, . W . ( 1 9 5 9 ) : S i n u l t a n e o u s e q u a t i o n s a n d c a n o n i c a l correlation theory. Econometrica, 20, 245-256. H O R S T ,P . ( 1 9 6 5 ) : F a c t o r A n a l y s i s o f D a t a M a t r i c e s . N e w Y o r k . Ho1t, Rinehart and Winston. H O E F F D I N GW, . ( 1 9 6 8 ) : W a l d M e n o r i a l L e c t u r e s . H O T E L L I N GH, . ( 1 9 3 1 ) : T h e g e n e r a l i s a t i o n A n n . M a t h . S t a t . 2 , 3 6 0 - 3 7 8.

of Studentrs ratio.

-1 4 5 -

H O T E L L I N GH, . ( 1 9 3 3 ) : A n a l y s i s o f a c o m p l e x o f s t a t i s t i c a l into

variables

principal

conponents. Journ. Educational

P s y c h o l o g y , 2 4 , 4 1 7 - 4 41 , 4 9 8 - 5 2 0 . H O T E L L I N GH, . ( 1 9 3 5 ) : R e l a t i - o n s b e t w e e n t w o s e t s o f v a r i - a t e s . B i o m e t r i k a , 2 8 , 3 2 1 - 3 7 7. J E F F R E Y SH , . (1961): Theory of Probability.

3rd ed. 0xford.

Clarendon. J O H N S T O NJ ,.

( 1 9 6 3 ) : E c o n o m e t r i c M e t h o d s . N e w Y o r k . M c G r a wI I i 1 1

J O R D A NC, . ( 1 8 7 5 , 1 8 7 7 ) : E s s a i s u r 1 a g 6 o m 6 t r i . ed n - d i m e n s i o n s . B u 1 1 . S o c . M a t h . P a r l s , 3 , 1 0 3 - 1 74 , 4 , 9 2 . JULIA, G. (1944): Sur 1es projections de lr6space Hilbertien.

C.R. Ac. Sci. 218, 892

K E N D A L LM , . G . a n d S T U A R TA , . Statistics.

( 1 9 6 6 ) : T h e A d v a n c e dT h e o r y o f

3 NewYork. Hafner

A ,. M . ( 1 9 7 2 ): M u l t i v a r i a t e KSHIRSAGAR KLEIN, L.R. Cliffs,

des systdmesorthonormaux

( 1 9 6 2 ): A n I n t r o d u c t i o n

Analysis.

NewYork. Dekker.

to Econometrics. Englewood

N.J. Prentice Ha11.

K L 0 E K , T . a n d M E N N E SL, . B . M . ( 1 9 6 0 ) : S i m u l t a n e o u s e q u a t i o n s based on principal

c o m p o n e n t so f p r e d e t e r m i n e d v a r i a b l e s .

E c o n o m e t r i c a ,2 8 , 4 5 - 6 1 . I O 4 E N T AJ, .

(1971): Elements of Econometrics. New York. lt'[acmillan

AV . N, . ( 1 9 4 6 , 1 9 4 7 ) : 0 n t h e p r o o f o f t h e r n e t h o d o f KOLMOGORO least squares. lJspekiN{atemN . auk (N.S.)(11) 1, 57-7O,8,523-533. K O O P M N ST , .C. (1937): Linear Regression Analysis in Econonic Tine Series. KOOPMANS T ,. C . e d .

lJaarlern ( 19 5 0 ) : S t a t i s t i c a l

Analysis

in Dynamic

E c o n o n i c ] t ' l o d e 1 s .N e w Y o r k . W i 1 e y . K O N I J N ,H . S . ( 1 9 6 2 ) : I d e n t i f i c a t i o n equations model with errors ?n

1O AfTlet:

70-Q7

Stutllee

and estimation in a simultaneous

in the variables.

Econonetrica,

- 1 4 6 -

K U L L B A C KS, . ( 1 9 5 9 ) : I n f o r n a t i o n

Theory and Statistics.

NewYork. Wi1ey. L E S E R ,C . E . V . ( 1 9 6 6 ) : E c o n o m e t r i . cT e c h n i q u e s a n d P r o b l e m s . New York, Hafner. L I N D L E Y ,D . V . ( 1 9 6 5 ) : I n t r o d u c t i o n t o P r o b a b i l i t y Canbridge University

and statistics.

Press.

L I N D L E Y ,D . V . a n d S M I T H , A . F . M . ( 1 9 7 2 ) : B a y e s e s t i n a t e s f o r the linear rnodel. Journal Royal Stat. Soc. 34, 1-41. (1964): Schdtzprobleme der Okonometrie. Wiirzburg.

LUEDEKED , . Physica

P ,. C . ( 1 9 3 0 ) : 0 n t e s t s a n d n e a s u r e s o f g r o u p MAHALANOBIS divergence. I. Journ. Asiatic Soc. of Bengal, 26, 541. P ,. C . ( 1 9 3 6 ) : 0 n t h e g e n e r a l l z e d d i s t a n c e i n MAHALANOBIS statistics. Proc. Nat. Institute of Scienceof India,1Z,49-55. M A L I N V A U DE, .

(1970): Statistical

Methods of Econometrics.

2nd ed. Ansterdan. North Ho11and. M A N N ,H . B . ( 1 9 6 0 ) : T h e a l g e b r a o f t h e l i n e a r ',l Ann. I4ath. Stat. 31, -15. M A R S C H A KJ,.

hypothesis.

(1955): Econonic Measurenent of policy

prediction.

and

t l o o d , W . C . a n d K o o p n a n s ,T . J . e d . S t u d i e s

in Econometric Methods. NewYork. Wi1ey. M E N G E SG , . ( 1 9 6 1 ): O k o n o m e t r i e . W i e s b a d e n . B e t r i e b s w i r t s c h a f t licher M I R S K Y ,L .

Verlag. (1955): Introduction

M O R R I S O ND, . ( 1 9 6 7 ) : M u l t i v a r i a t e

to Linear Algebra. Oxford. Statistical

Methods. NewYork.

Mc Graw Hill. M O S B A E KE, . J . a n d W O L D ,H . ( 1 9 7 0 ) : I n t e r d e p e n d e n t S y s t e m s . Arnsterdan. North Ho11and. N E Y M A N J, .

(1962): Two breakthroughs i.n the theory of statistical

decision rnaking. Rev. Inst.

of

International

Stat.

30, 11-27,

P E N R O S ER, . ( 1 9 5 5 ) ; A g e n e r a l i z e d i - n v e r s e f o r m a t r i c e s . P r o c . C a m b r i d g eP h i 1 . S o c . 5 1 , 4 0 6 - 4 13 . P I L L A I , K . C .S . ( i 9 6 0 ) : M u l t i v a r i a t e

hypotheses. The Statisti.cal

Center, Mani"1a,Universi-ty of the Phillipines

.

_r47 _

PILLAI, K.C.S. (1967): Upper percentage points of ltrgest analysis.

roots of a matrix in multivariate

characteristic

B i o m e t r i k a , 5 4 , 1 8 9 - 19 4 . P R E S S ,S . J .

( 19 7 2 ) : A p p l i e d M u l t i v a r i a t e

Analysis.

NewYork.

Ho1t, Rinehart and Winston. R A I F F A , H . a n d S C H L A I F E RR, . ( 1 9 6 1) : A p p l i e d S t a t i s t i c a l Press.

Decision Theory. Canbridge Mass. Havard University

Methods in Biological

RAO, C.R. (1952): AdvancedStatistical Research. New York. Wi.1ey.

Inference and its

R r \ O ,C . R . ( 1 9 6 5 ) : L i n e a r S t a t i s t i c a l

Application.

NewYork, Wi1ey. R A O ,C . R . ( 1 9 7 0 ) : E s t i m a t i o n o f h e t e r o s c e d a s t i c v a r i a n c e s i n l i n e a r m o d e l s . J o u r n . A r n e r i c a nS t a t . A s s o c . 6 5 , 1 6 1 - 1 7 2 , R A O , C . R . ( 19 7 2 ) : R e c e n t t r e n d s i n r e s e a r c h i n n u l t i v a r i a t e analysis. Biometrics, 28, 3-22. RAO, R. Ranga (1962): Weak and uniform convergence neasures. Ann. lvlath. Stat.

33, 659-680.

R O B B I N S ,H . ( 1 9 5 5 ) : A n e m p i r i c a l B a y e s a p p r o a c h t o s t a t i s t i c s . Proc. Third Berkeley Synp. Prob. and ltiath. Stat.1,

157-164.

R O Y ,R . ( 1 9 7 0 ) : E l e m e n t sd ' E c o n o n e t r i e . P a r i s . P r e s s e s Universitaires. R O Y , S . N . ( i 9 5 7 ) : S o m eA s p e c t s o f l r 4 u l t i v a r i a t e A n a l y s i s .

l'r-ew

York, Wl1ey. S A S T R YM , .V.R. (1969): Multicollinearity elasticities.

C a n a d i a nJ o u r n . A g r i . c . E c o n . 1 7 ' 5 0 - 6 0 .

S A S T R YM , . V . R . ( 1 9 7 0 ) : S o m el i m i t s collinearlty. S A V A G E L, . J .

and consuner dernand

in the theory of multi-

The AmericanStatistician,

39-40.

(1954): The Foundations of Statj-stics.

NewYork,

Wi1ey. S C H N E E W E IH B ., ( 1 9 7 1 ) : 0 k o n o m e t r i e . W t i r z b u r g . P h y s i c a . P ,. ( 1 9 6 9 , 1 9 7 1 ) : M e t h o d e n d e r O k o n o m e t r i e . B e r l i n . SCHONFELD Vahlen.

-1 4 8 -

S C H E F F EH, . ( 1 9 5 9 ) : A n a l y s l s o f V a r i a n c e . N e w Y o r k . W i 1 e y . S C H O U T EP, . H . ( 1 9 0 5 ) : 0 n t h e e q u a t i o n s d e t e r m i n i n g t h e a n g l e s of two polydi-nensional spaces. Proc. Sci. K. Akad. Wet. 7 , 3 4 0 - 3 4 1, S O M E R V I L L ED,. M . Y . ( 1 9 2 9 ) : A n I n t r o d u c t i o n t o t h e G e o n e t r y o f N D i m e n si o n s . L o n d o n . S T O N E ,R . ( 1 9 4 7 ) : 0 n t h e i n t e r d e p e n d e n c e o f b l o c k s o f t r a n s a c t i o n s . J o u r n . R o y . S t a t . S o c . s u p p l . 9 , 1- 3 2 . S T O N E ,R . ( 1 9 5 4 ) : T h e M e a s u r e m e n t o f C o n s u r n e r ' s E x p e n d i t u r e a n d B e h a v i o u r i n t h e U n i t e d K i n g d o n , 1 9 2 0 -1 9 3 8 . C a n b r i d g e . Cambridge University

Press.

THEIL, H. (1961): Econonic Forecasts and Policy. Ansterdan. North Ho11and. T H E I L , H . ( 1 9 6 7 ) : E c o n o m i c sa n d I n f o r n a t i o n

Theory. Chicago.

Rand McNa11y. T t l E I L , F I.

( 1 9 7 1 ): P r i n c i p l e s

o f E c o n o r n e t r i c s .A m s t e r d a m .

North Ho11and. T I N T N E R ,G . ( 1 9 4 5 ) : A n o t e o n r a n k , m u l t i c o l l i n e a r i t y nultiple

regression. Ann. Math. Stat.

T I N T N E R ,G . ( 1 9 5 0 ) : S o n e f o r m a l r e l a t i n s

and

16, 304-308.

in multivari,ate

analysis. Journ. Royal Stat. Soc. B. 12, 95-101. T I N T N E R ,G . ( 1 9 5 2 ) : E c o n o m e t r i c s . N e w Y o r k . W i 1 e y . T I N T N E R ,G . ( 1 9 6 0 ) : A p p l i c a t i o n

of the theory of information

to the problem of weighted regression. die Corrado Gini,

Studi in Onore

Rone. Universitd degli

Studi.

T I N T N E R ,G . ( 1 9 6 2 ): E i n T e s t f i i r d i e S i n g u l a r i t a t

einer Matrix.

IFO Studien, 8, 1-14. T I N T N E R ,G . ( 1 9 6 3 ) : A n o t e o n t h e r e l a t i o n

betweenMahalanobis

dj-stance and weighted regression. Rao, C.R. ed. Contributions to Statistics.

Calcutta. Statistical

Publication Society. 481-484.

T I N T N E R ,G . a n d K A D E K O D IG , .(1971): Note on the transfornation variables

in simultaneous equations systens. Journ.

Soc. Agric.

Stat. 23, 164-17?.

Indian

of

-1 4 9 T I N T N E R ,G . a n d S A S T R Y M , .V.R. (1969): Infornation the statistical Multivariate TORRENS I B- E R N , J . factorielle. W A L T E R SA, . A .

estination

Analysis,

theory and

of economicrelations.

2, New York. Acadenic

Press.

( 1 9 72 ) : M o d 6 l e s e t n 6 t h o d e s d e 1 ' a n a t y s e Paris, Dunod.

(1970): An Introduction

to Econometrics. NewYork.

Norton. W I L K S ,S . S . ( 1 9 3 2 ) : C e r t a i n g e n e r a l i s a t i o n s i n t h e a n a l y s i s o f v a r i a n c e . B i o n e t r i k a , 2 4 , 4 7 1 - 4 9 4. W I L K S ,S . S . ( 1 9 4 3 ) : M a t h e m a t i c a l S t a t i s t i c s .

Princeton.

W I L K S , S . S . ( 1 9 6 2 ): M a t h e m a t i c a l S t a t i s t i . c s . N e w Y o r k . W i 1 e y . W O L D ,H . a n d J U R E E N ,L . WOLFOWITZ J .,

( 1 9 5 3 ) : D e m a n dA n a l y s i s .

NewYork. Wi1ey.

(1954): Generalisation of the theorem of Glivenko

C a n t e 1 1 i . A n n . N { a t h ,S t a t . 2 5 , 1 3 1 - 1 3 8 . Z A C K S ,S . ( 1 9 7 1 ) : T h e T h e o r y o f S t a t i s t i c a l

Inference. NewYork.

Wi1ey. Z A R E M B K AP,. ( 1 9 6 8 ) : T h e f u n c t i o n a l

f o r m o f t h e d e m a n df o r m o n e y .

Journ. Anerican Stat. Assoc. 63, 508-511. Z E L L N E RA , .

(1971): An Introduction

Econometrics. NewYork. Wi1ey.

to Bayesian Inference in

Extract from the review by Karl A. Fox in the Journal of the American Statistical Association 71, 355 (September 1976), 769-70. … Afriat’s essay is entitled “Regression and Projection: Studies in thc Algebra and Geometry of Statistical Correlation.” Portions of it originated in his attempt to make concise the algebra which appeared in his invstigation of methods for thc seasonal correction of economic time series; this work was done during 1953-56 while Afriat was a member of the Department of Applied Economics at Cambridge University. Afriat’s first love was mathematics, and two sections of his essay first appeared in the Quarterly Journal of Mathemalics (Oxford, 1956) and the Proceedings of the Cambridge Philosophical Society (1057). He drew directly on advanced articles by pure mathematicians (Jordan, Hadamard, Halmos, Dixmeier, Flanders, Julia, Kolmogorov, Schoute, Somerville) as well as mathematical statisticians (most notably Hotelling), whom few economists in the 1950’s were equipped to read. As of 1976, most econometricians can and should read Afriat’s essay. The editors credit Kolmogorov in 1946 with the conceptualization of least-squares linear regression as resolving sample vectors of dependent variables into two orthogonal conmponents, a regressional component lying in a space spanned by sample vectors of the independent variables, and a residual component lying in the orthogonal complcment of that space. Afriat goes beyond this to consider the linear transformations which obtain among these orthogonal componcnts of sample vectors, these being the complementnry pair of symmetric idcmpotents which are the orthogonal projectors on the space and its orthogonal complement, and to ana1yze relations between variables directly in terms of these projectors. Thc view taken of regression analysis is one that does not depend on distribution concepts. Orthogonnl projection is presented as the fundamental principle and the least-squares principle is derived as a property of it. Afriat’s approach provides alternative views or derivations of Hotelling’s canonical correlntion theory and coefficients of correlation and alienation defined by Wilks and Hotelling. One of Afriat’s formulas provides a generalization of thc multiple correlartion coefficient independently derived (from a different perspective using different mathematical tools) by Hooper. The beauty of Afriat’s algebraic and geometric derivations is that so many of the different techniques used by econometricians fall out as special cases of more general formulations. …