Modeling Intraindividual Variability with Repeated Measures Data:
Methods and Applications
MULTIVARIATE APPLICATIONSBOOK SERIES
The multivariate Applications book series was developed to encourage the use of rigorous methodology in the study of meaningful scientific issues, and to describe the applications in easy to understand language. The series is sponsored by the Society of Multivariate Experimental Psychology and welcomes methodological applications from a variety of disciplines, such as psychology, public health, sociology, education, and business. The main goal is to provide descriptions of applications of complex statistical methods to the understanding of significant social or behavior issues. The descriptions are to be accessible to an intelligent, non-technical oriented readership (e.g., non-methodological researchers, teachers, students, government personnel, practitioners, and other professionals). Books can be single authored, multiple authored, or edited volumes. The ideal book for this series would take on one of several approaches: (1) demonstrate the application of several multivariate methods to a single, major area of research; (2) describe a multivariate procedure or framework that could be applied to a number of research areas; or (3) present a variety of perspectives on a controversial topic of interest to applied multivariate researchers. There are currently 7 books in the series: What $There Were No Significant Tests?, co-edited by L. Harlow, S. Mulaik, and J. Steiger (1977). Structural Equation Modeling With LISREL, PRELIS, and SIMPLIS: Basic Concepts, Applications, and Programming, by B. Byrne (1998). Multivariate Applications in Substance Use Research, co-edited by J. Rose, L. Chassin, C. Presson, and S. Sherman (2000). Item Response Theory for Psychologists, co-authored by S . Embretson and S. Reise (2000). Structural Equation Modeling with AMOS, by B. Byrne (2001). Conducting Meta-Analysis Using SAS, co-authored by W. Arthur, Jr., W. Bennett, Jr., and A. I. Juffcutt (2001). Modeling Intraindividual Variability with Repeated Measures Data: Methods and. Applications, co-edited by D. S. Moskowitz and S. L. Hershberger (2002). Anyone wishing to propose a book should address the following: (1) title; (2) author(s); (3) timeline including planned completion date; (4) Brief overview of focus for the book including a table of contents and a sample chapter (or more); ( 5 ) mention any competing publications in this area; (6) mention possible audiences for the proposed book. More information can be obtained from the editor, Lisa Harlow, at: Department of Psychology, University of Rhode Island, 10 Chafee Rod., Suite 8 , Kingston, RI 02881-0808; Phone: 401-874-4242; Fax: 401-874-5562; or e-mail:
[email protected]. Information can also be obtained from one of the advisory board members: Leona Aiken (Arizona State University), Gwyneth Boodoo (Educational Testing Service), Susan Embretson (University of Kansas), Michael Neale (Virginia Commonwealth University), Bill Revelle (Northwestern University), and Steve West (Arizona State University).
Modeling Intraindividual Variability with Repeated Measures Data:
Methods and Applications
Edited by
D. S . Moskowitz M cGill University
Scott L. Hershberger California State University, Long Beach
2002
LAWRENCE ERLBAUM ASSOCIATES, PUBLISHERS Mahwah, New Jersey London
The final camera copy for this work was prepared by the editors and therefore the publisher takes no responsibility for consistency or correctness of typographical style. However, this arrangement helps to make publication of this kind of scholarship possible.
Copyright @ 2002 by Lawrence Erlbaum Associates, Inc. All rights reserved. No part of the book may be reproduced in any form, by photostat, microform, retrieval system, or any other means, without the prior written permission of the publisher. Lawrence Erlbaum Associates, Inc., Publishers 10 Industrial Avenue Mahwah, New Jersey 07430
I
Cover design by Kathryn Houghtaling Lacey
I
ISBN 0-8058-3125-8
Books published by Lawrence Erlbaum Associates are printed on acid-free paper, and their bindings are chosen for strength and durability. Printed in the United States of America 10 9 8 7 6 5 4 3 2 1
List of Contributors David A. Kenny, Department of Psychology, University of Connecticut, Babbidge Road Unit 1020, Storrs, C T 06269-1020. Nial Bolger, Department of Psychology, New York University, 6 Washington Place, room 752, New York, N.Y. 10003. Deborah A. Kashy, Department of Psychology, Texas A&M University, College Station, T X 77843-4235. Stephan E. Raudenbush, School of Education, Michigan State University, 610 E. University, Ann Arbor, MI 48109-1159. Patrick J. Curran, Department of Psychology, University of North Carolina, Chapel Hill, NC 27599-3270. Andrea M. Hussong, Department of Psychology, University of North Carolina, Chapel Hill, NC 27599-3270. J.O. Ramsay, Department of Psychology, McGill University, 1205 Dr. Penfield Avenue, Montreal, Quebec, Canada, H3A 1B1. Dennis Wallace, Department of Preventive Medicine, University of Kansas Medical Centre, 4004 Robinson Hall, 3901 Rainbow Blvd., Kansas City, KS 66160. Samuel B . Green, Department of Psychology in Education, Arizona State University, 308G Payne Hall, Tempe, AZ 85287-0611. Judith D. Singer, Graduate School of Education, Harvard University, Roy E. Larsen Hall, Appian Way, Cambridge, MA 02138. Terry E. Duncan, Oregon Research Institute, 1715 Franklin, Blvd., Eugene, OR 97403-1983. Susan C. Duncan, Oregon Research Institute, 1715 Franklin, Blvd., Eugene, OR 97403-1983. Fuzhong Li, Oregon Research Institute, 1715 Franklin, Blvd., Eugene, OR 97403-1983. Lisa A. Strycker, Oregon Research Institute, 1715 Franklin, Blvd., Eugene, OR 97403-1983. Steven Hillmer, School of Business, University of Kansas, 203 Summerfield Hall, Lawrence, KS 66044-2003. John R. Nesselroade, Department of Psychology, The University of Virginia, 102 Gilmer Hall, P.O. Box 400400, Charlottesville, VA 22904-4400. John J. McArdle, Department of Psychology, The University of Virignia, 102 Gilmer Hall, P.O. Box 400400, Charlottesville, VA 22904-4400. Steven H. Aggen, Department of Psychiatry, Virginia Commonwealth University, P.O. Box 980710, Richmond, VA 23286-0440. Jonathan M. Meyers, Department of Psychology, The University of Virginia, 102 Gilmer Hall, P.O. Box 400400, Charlottesville, VA 22904-4400.
Contents ix
Preface Traditional Methods for Estimating Multilevel Models David A . Kenny, Nial Bolger, and Deborah A . Kashy
1
Alternative Covariance Structures for Polynomial Models of Individual Growth and Change Stephen W. Raudenbush
25
Structural Equation Modeling of Repeated Measures Data: Latent Curve Analysis Patrick J. Curran and Andrea M . Hussong
59
Multilevel Modeling of Longitudinal and Functional Data J . 0. Ramsay
87
Analysis of Repeated Measures Designs with Linear Mixed Models Dennis Wallace and Samuel B. Green
103
Fitting Individual Growth Models Using SAS PROC MIXED Judith D. Singer
135
Multilevel Modeling of Longitudinal and Functional Data Terry E. Duncan, Susan C. Duncan, Fuzhong Li, and Lisa A . Strycker Times Series Regressions Steven Hillmer
171
203
vii
viii 9 Dynamic Factor Analysis Models for Representing
Process in Multivariate Time-Series John R. Nesselroade, John J . McArdle, Steven H. Aggen, and Jonathan M. Meyers
233
Author Index
267
Subject Index
273
Preface This volume began as a nightmare. Once upon a time, life for social and behavioral scientists was (relatively) simple. When a research design called for repeated measures data, the data were analyzed with repeated measures analysis of variance. The BMDP 2V module was frequently the package of choice for the calculations. Life today is more complicated. There are many more choices. Does the researcher need to model behavior at the level of the individual as well as at the level of the group? Should the researcher use the familiar and well-understood least-squares criterion? Should the researcher turn to the maximum likelihood criterion for assessing the overall fit of a model? Is it possible and is it desirable to represent the repeated measures data within structural equation modeling? So the nightmare began as (shall we be dishonest and say) one night of deliberations among these choices. The thought then arose that it would be useful to have the statistical experts writing in the same volume about the possibilities and some of the dimensions that are pertinent to making these choices. Hence the origin of the present volume. The issue of the analysis of repeated measures data has commonly been examined within the context of the study of change, particularly with respect to longitudinal data (cf., Collins & Horn, 1991; Gottman, 1995). This volume contains three chapters whose primary focus is on the study of growth over several years time (Raudenbush, chapter 2; Curran & Hussong, chapter 3; Duncan, Duncan, Li, & Strycker, chapter 7). Studies of change typically imply the expectation that variation, movement in scores, is genup or generally down. Not all repeated erally unidirectional-generally measures data are concerned with change, and change is only one aspect of the variability that occurs within individuals. To illustrate, consider an example from the study of social behavior. Personality, social, and organizational psychologists are often interested in the effects of situations on behavior: to what extent are individuals’ behaviors consistent across sets of situations and to what extent does the behavior of individuals change as a function of the situation. For example, the focus might be on how people’s dominant and submissive behaviors change as a function of being in a subordinate, co-equal, or supervisory work role. There might also be interest in whether people’s responses t o these ix
X
preface
situations vary as a function of their level on personality characteristics. Some people, let’s say extraverts, may change more in their behavior than other individuals in responding to these different situations. This could be studied in the laboratory in which individuals participate in situations in which they are placed in a subordinate role, a co-equal role, and a supervisory role and their responses are recorded. This would be an example of a balanced design. All participants would participate in three situations. These data can be analyzed in the familiar technique of repeated measures analysis of variance. We might introduce the personality variable of extraversion to examine the interaction between individual differences and situation. However, there is considerable error variance in a measure based on a one-occasion assessment (Epstein, 1979; Moskowitz & Schwarz, 1982). Measurements of the individual in each situation on several occasions would improve the quality of measurement. This is possible but difficult in the laboratory, so sometimes researchers make use of naturalistic techniques for collecting this kind of data (see Kenny, Bolger, & Kashy, chapter 1; also see Moskowitz, Suh, & Desaulniers, 1994). Despite whether the researcher remains in the laboratory or whether the researcher uses a naturalistic methodology, the researcher is confronted with decisions about how to handle the data. The multiple measures for each situation could be aggregated (averaged) to provide a single measurement in each situation for each individual. If this is done within the context of the laboratory, this provides a balanced design with better measures. Unfortunately, this strategy throws away information. Some people would have less variability in their measures than other people. It may be of interest to know who has more variability in their responses to such situations as being the boss or being the supervisee. As an alternative to the laboratory context, the researcher might use a naturalistic data collection method such as event contingent recording (Moskowitz, 1986; Wheeler & Reis, 1991). In event contingent recording, participants are given standardized forms and asked to record their behavior after being in certain kinds of events, such as all interpersonal events at work. The form could request information about characteristics of the situation as well as the person’s behavior so interpersonal events can be categorized into situations with a boss, situations with a co-equal, and situations with a supervisee. This method has appeal because it provides records of behavior in real life rather than responses t o possibly artificial situations in the laboratory. However, the structure of such data presents data analytic decisions. Individuals will report differing numbers of events. Individuals will report differing numbers of events in different kinds of situations. Individuals may report events corresponding to some of the targeted situations (e.g., in the subordinate and co-equal situations) but not to other targeted situations such as having the supervisory responsibilities of the boss situation. The data structure could be simplified by aggregating across events
preface
xi
referring to the same kind of situation to obtain one measure per situation and only including people in the sample who reported events in all three kinds of situations. The simplification of the data structure would provide a balanced design, and consequently the familiar data analytic techniques of repeated measures analysis of variance and repeated measures analysis of covariance could be used. However, such simplification would also eliminate information. The simplification would (1) not take into account variability in people’s responses across events of the same type of situation; (2) throw away that portion of a sample that has “missing data”; that is, individuals whose data do not include the representation of all kinds of specified events, and (3) disregard the time ordering of events. Once one becomes involved with recording multiple assessments of individuals behavior and affect responses, the variability of people’s responses across events becomes salient and compels modeling. For example, diurnal and weekly rhythms have been demonstrated for affect and behavior (Brown & Moskowitz, 1998; Larsen, 1987; Watson, Wiese, Vaidya, & Tellegen, 1999). Behavior and affect co-occur over time in ways that cannot be identified from static assessments of these variables (Moskowitz & Cote, 1995). Similarity and dissimilarity among measures or items from occasion to occasion may be of interest (see Nesselroade, McArdle, Aggen, & Meyers, chapter 9). The shape of variation can be of considerable importance, such as the shape of change in response to stress or psychotherapy or recovery from illness (e.g., Bolger & Zuckerman, 1995; also see Ramsay, chapter 4). The time ordering of events can be used to make inferences about antecedent-consequent relations (see Hillmer, chapter 8). So the focus of this volume is the examination of how individuals behave across time and to what degree this behavior changes, fluctuates, is stable, or is not stable. We call this change in individual behavior “intraindividual variability.” Intraindividual variability can be contrasted with “interindividual variability.” The latter describes individual differences among different people; the former describes differences within a single person. Although most behavioral and social scientists believe that behavior does differ from one occasion to the next, sophisticated techniques for exploring intraindividual variability have been underutilized. Several factors have contributed to the reluctance of analysts to utilize these techniques. One factor is their newness, many of them having only been developed within the last few years. A second factor is the perceived difficulty of implementing these techniques; descriptions tend to be highly technical and inaccessible to nonmathematically trained researchers. A third factor is the unavailability of computer programs to do the analyses, a situation that has recently been much improved with the release of new computer programs. The primary goal of this volume is to make accessible to a wide audience of researchers and scholars the latest techniques used t o assess intraindividual variability. The chapters of this volume represent a group of distinguished experts who have written on a range of available techniques. The
xii
preface
emphasis is generally at an introductory level; the experts have minimized mathematical detail and provided concrete empirical examples applying the techniques. The volume opens with a chapter by David Kenny, Niall Bolger, and Deborah Kashy, who contrast several procedures for the analysis of repeated measures data. They note two problems with using traditional analysis of variance (ANOVA) procedures for analyzing many contemporary designs using repeated measures data. The first is that research participants often will not have the same number of data points. The second is that the predictor variable generally does not have the same distribution across measurement points for all research participants. They approach the analysis of intraindividual variability within the context of multilevel analyses in which research participant are independent units and the repeated observations for each individual are not assumed to be independent. They illustrate that a strength of alternative procedures to ANOVA is that they more readily permit the evaluation of random effects that reflect the extent of variability among individuals to fixed effects. They compare features of three alternative procedures for modeling the group of research participants and the variability within the group of research participants: a two-step ordinary least-squares regression procedure, a weighted least-squares variation of multiple regression, and a procedure based on a maximum likelihood criterion. Stephen Raudenbush compares advantages of the hierarchical linear model (a multilevel model), structural equation modeling, and the generalized multivariate linear model in the analysis of repeated measures data. He argues for the flexibility of the hierarchical linear model (HLM). HLM permits the inclusion of all available data, allows unequal spacing of time points across participants, can incorporate a variety of ways of characterizing change in the data such as rate of change and rate of acceleration, and can provide for the clustering of individuals within groups such as schools or organizations. He then combines ideas from the standard hierarchical linear model and the multivariate model to produce a hierarchical multivariate model that allows for different distributions within persons of randomly missing data and time-varying covariates, permits the testing of a variety of covariance structures, and examines the sensitivity of inferences about change to alternative specification of the covariance structure. The procedure discussed permits the examination of whether alternative models are consistent with key inferences about the shape of change. Patrick Curran and Andrea Hussong describe how repeated measures data can be represented in structural equation models. They discuss the advantages and disadvantages of two kinds of structural equation models for representing longitudinal change: the autoregressive crosslagged panel model and the latent curve analysis model. They emphasize the latent curve approach, an approach that first estimates growth factors underlying observed measures and then uses the growth factors in subsequent analyses. Latent curve analysis provides two key advances over autoregressive
preface
...
Xlll
crosslagged panel models. The first is the capability to model data sets with more than two time points. The second is the capability to provide estimates of the extent of variability among individuals, both the extent of variability in starting points and in rates of change. An applied example concerning the development of antisocial behavior and reading proficiency is used to illustrate the latent curve analysis model. The example illustrates that predictors of behavior at single time points (e.g., initial status) are different from the predictors of the shape of change over time. They also use the example to illustrate several options for incorporating nonlinear forms of growth in structural equation models. James Ramsay provides a commentary on issues connecting the chapters by Raudenbush, Curran and Hussong, and Kenny, Bolger, and Kashy. He makes several points relative t o the study of longitudinal data, considering the implications of missing data, the number of points necessary to define characteristics of growth curves such as level, slopes, and bumps, and the possibility that the curves for individuals are not registered such that the curves for individuals may show a similar shape but reflect different timings of events. His chapter further extends the discussion of repeated measurements to the case where there are many measurements and makes the point that such data can be represented by a sample of curves using a set of techniques referred to as functional data analysis. His chapter ends on a note of caution, reminding the reader that moving to the more complex models that are sometimes presented in this book has costs that need to be considered. For example, the maximum-likelihood procedures are sensitive to the mis-specification of the variance-covariance structure. Moreover, adding random coefficient parameters uses up degrees of freedom leading t o a loss of power and potentially unstable estimates of fixed effects. Thus, the cautious researcher who has a moderate sample size may prefer to keep the model simple such as by remaining with a least-squares-based regression procedure (cf. Kenny, Bolger, & Kashy, chapter 1). There is considerable complexity in the analysis of the models that make use of random as well as fixed effects (see chapters 1 and 2). The chapters by Judith Singer and by Dennis Wallace and Samuel Green present detailed description of how to analyze and interpret such models using a commonly available package, the PROC MIXED procedure from SAS. Dennis Wallace and Samuel Green’s chapter provides extensive information about how to estimate fixed and random effects. They provide detailed explanations of the meaning of the underlying statistics, such as maximum-likelihood and restricted maximum-likelihood methods, and an introduction to some of the structures that may be found in the variancecovariance matrices. They provide an outline of recommended steps for estimating models incorporating fixed and random effects. These steps are illustrated using an example from a longitudinal study of the effect of two treatment interventions for reducing substance abuse among homeless individuals; the illustration includes an examination of whether the effectiveness of the treatment programs vary as a function of changing levels of
xiv
preface
depression. Judith Singer’s discussion provides practical advice for all stages of the analysis including data preparation and writing computer code. She illustrates a process that is sometimes mysterious for the novice researcher in this area. Models for the representation of individuals’ variability across time are sometimes presented as single equations at multiple levels (Bryk & Raudenbush, 1992) and sometimes by single equations that specify multiple sources of variation (cf. Goldstein, 1995). She demonstrates how separate equations can be written at multiple levels and then elements can be substituted in to arrive at a representation in a single equation. The presentation is situated in the context of individual growth models; the presentation can also be extended and applied to cases with repeated measures data that are unidirectional as described in the extended example presented earlier and in the chapter by Kenny, Kashy, and Bolger. Stephen Raudenbush comments that the use of structural equation modeling has not typically been extended to the case where individuals are clustered. Terry Duncan, Susan Duncan, Fuzhong Li, and Lisa Strycker take the step of providing such an extension. They provide an introduction to representing multilevel models in structural equation models using an example from an analysis of change in adolescents’ use of alcohol. They compare the strengths and weaknesses of three approaches for modeling longitudinal data that are clustered and unbalanced. One method, a full information hierarchical linear model (HLM), is familiar from the chapter by Raudenbush. A second method, a limited information multilevel latent growth model (MLGM), is an extension of latent growth modeling that was presented in the chapter by Curran and Hussong. The third approach is based on a full information maximum likelihood (FIML) latent growth modeling using an extension of a factor of curves model which has not previously been discussed in the book. They provide examples of programming in both HLM (Bryk, Raudenbush, & Congdon, 1996) and Mplus (Muthbn & M u t h h , 1998). Steven Hillmer provides a basic introduction to using time series models to predict intraindividual change. In a time series model, data points for the same variable are arranged sequentially in time, and a basic goal is to identify a model that best represents the sequencing of these data. Hillmer reviews the differences between the main kinds of models that might be used. He contrasts two classes of models: stationary models in which the joint probability of any set of observations is unaffected by a shift backward or forward in the time series and nonstationary models in which parts of the series behave similarly although not identically to other parts of the series. He reviews the steps of building a time series model, providing extensive graphical material for understanding the issues that might arise. The chapter includes an example of an interrupted time series data in which an intervention occurs during the course of a time series and the effect of the intervention is estimated. The extended example provided is drawn from the business literature on sales. Time series analyses can also be applied
preface
xv
to the modeling of variability within a person when sufficient data points have been collected. John Nesselroade, John McArdle, Steven Aggen, and Jonathan Meyers provide an introduction t o dynamic factor analysis. Dynamic factor analysis permits the examination of the similarity and dissimilarity of data from occasion t o occasion. They introduce the topic by describing the Ptechnique factor analysis, which uses the common factor model to model the covariation of multiple variables measured across time for a single individual. They note problems with this model in the representation of process changes over time, such as the representation of effects that dissipate or strengthen over time. They present two models that allow for time-related dependencies and illustrate the application of these two dynamic factor analysis methods using reports of daily moods. The necessary LISREL code for conducting these analyses is included. The initial organization for this volume was done within the context of two symposia presented at the 1997 meeting of the American Psychological Association. We thank Lisa Harlow, the editor of the Erlbaum Multivariate Applications Series for suggesting that we prepare a volume based on these symposia. We also thank James Ramsay, Yoshio Takane, and David Zuroff for comments on drafts of these chapters. We are also grateful to Chantale Bousquet and Serge Arsenauit for their preparation of the text in I4'QjX. Preparation of this volume was partially supported by funds from the Social Sciences and Humanities Research Council of Canada. We hope that the volume provides readers with a sense of the range of reasonable options for analyzing repeated measures data and stimulates new questions and more interest in repeated measures designs that extend beyond the context of longitudinal data. Pleasant dreams.. . Debbie S. Moskowitz Scott L. Hershberger
REFERENCES Bolger, N., & Zuckerman, A. (1995). A framework for studying personality in the stress process. Journal of Personality and Social Psychology, 69, 890-902. Brown, K. W., & Moskowitz, D. S. (1998). Dynamic stability of behavior: The rhythms of our interpersonal lives. Journal of Personality, 66, 105-134. Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage.
xvi
preface
Bryk, A. S., Raudenbush, S. W., & Congdon, R. T. (1996). H L M : Hierarchical linear and nonlinear modeling with the H L M / 2 L and H L M / 3 L programs. Chicago, IL: Scientific Software International, Inc. Collins, L., & Horn, J. (Eds.). (1991). B e s t methods f o r analysis of change: Recent advances, unanswered questions, future directions. Washington, DC: American Psychological Association. Epstein, S. (1979). The stability of behavior: I. on predicting most of the people much of the time. Journal of Personality and Social Psychology, 37, 1097-1126. Goldstein, H. (1995). Multilevel statistical models (2nd ed.). New York: Halstead Press. Gottman, J. M. (1995). T h e analysis of change. Hillsdale, NJ: Erlbaum. Larsen, R. J. (1987). The stability of mood variability: A spectral analysis approach t o daily mood assessments. Journal of Personality and Social Psychology, 52, 1195-1204. Moskowitz, D. S. (1986). Comparison of self-reports, reports by knowledgeable informants and behavioral observation data. Journal of Personality, 54, 101-124. Moskowitz, D. S., & Cote, S. (1995). Do interpersonal traits predict affect: A comparison of three models. Journal of Personality and Social Psychology, 69, 915-924. Moskowitz, D. S., & Schwarz, J. C. (1982). The comparative validity of behavioral count scores and knowledgeable informants’ rating scores. Journal of Personality and Social Psychology, 42, 518-528. Moskowitz, D. S., Suh, E. J., & Desaulniers, J. (1994). Situational influences on gender differences in agency and communion. Journal of Personality and Social Psychology, 66, 753-761. Muthdn, L. K., & Muthdn, B. 0. (1998). Mplus user’s guide. Los Angeles: Muthden and Muthen. Watson, D., Wiese, D., Vaidya, J., & Tellegen, A. (1999). The two general activation systems of affect: Structural findings, evolutionary considerations, and psychobiological evidence. Journal of Personality and Social Psychology, 76, 820-838. Wheeler, L., & Reis, H. T . (1991). Self-recording of everyday life events: Origins, types, and uses. Journal of Personality, 59, 339-354.
Chapter 1
Traditional Met hods for Est irnat ing Multilevel Models David A. Kenny University of Connecticut
Nial Bolger New Yorlc University
Deborah A. Kashy Texas A &M University Researchers often collect multiple observations from many individuals. For example, in research examining the relationship between stress and mood, a research participant may complete measures of both these variables every day for several weeks, and so daily measures are grouped within participants. In relationship research, a respondent may report on characteristics of his or her interactions with a number of different friends. In developmental research, individuals may be measured at many different times as they develop. In cognition research, reaction times may be observed for multiple stimuli. These types of data structures have been analyzed using standard (ANOVA) methods for repeated measures designs. The most important limitation of the analysis of variance (ANOVA) approach is that it requires balanced data. So, in the previous examples, each person would be required to have the same number of repeated observations. For example, in the stress and mood study, everyone might have t o participate for exactly 14 days, and in the relationships study each respondent might report
I
2
Kenny, Bolger, and Kashy
on interactions with exactly four friends. It is often the case, however, that data structures generated by repeated observations are not balanced, either because of missing observations from some participants or, more fundamentally, because of the nature of the research design. If, for instance, researchers were interested in learning about naturally occurring interactions with friends, they might have individuals describe their interactions with each person whom they consider to be a friend. For individuals who have few friends, there would be very few observations, whereas for other individuals there would be many. An additional factor can make the design unbalanced even if the number of observations per person is equal. For the design to be balanced, the distribution of each predictor variable must be the same for each person. So, if the predictor variable were categorical, there would need to be the same number of observations within each category for each person. If the predictor variable were continuous, then its distribution must be exactly the same for each person. The likelihood of the distribution being the same for each person is possible, but improbable. For example, in a study of stress and mood, it is unlikely that the distribution of perceived stress over the 14 days would be the same for each person in the study. In this chapter we introduce the technique of multilevel modeling as a means of overcoming these limitations of repeated measures ANOVA. The multilevel approach, also commonly referred to as hierarchical linear modeling, provides a very general strategy for analyzing these data structures and can easily handle unbalanced designs and designs with continuous predictor variables. In introducing multilevel modeling, we focus our attention on traditional estimation procedures (ordinary least squares and weighted least squares) that, with balanced data, produce results identical to those derived from ANOVA techniques. We also introduce nontraditional estimation methods that are used more extensively in subsequent chapters. We begin by introducing a research question on how gender of interaction partner affects interaction intimacy. We follow this by presenting an artificial, balanced data set on this topic and provide a brief overview of the standard ANOVA approach to analyzing such a data set. We then introduce a real data set in which the data are not balanced, and we consider an alternative to the ANOVA model, the multilevel model. Finally, we compare the least-squares estimation approaches described in this chapter to the maximum likelihood estimation approaches discussed in other sections of this book.
STANDARD ANOVA ANALYSIS FOR BALANCED DATAS Consider a hypothetical Rochester Interaction Record (RIR; Reis & Wheeler, 1991) study of the effects of gender on levels of intimacy in social interaction. The RIR is a social interaction diary that requires persons to complete
Estimating Multilevel Models
3
a set of measures, including the interaction partner’s gender and interaction intimacy, for every interaction that he or she has over a fixed interval. In our study, each of 80 subjects (40 of each gender) interacts with six partners, three men and three women. The study permits the investigation of the degree t o which the gender of an interaction partner predicts the level of perceived intimacy in interactions with that partner. One can also test whether this relationship varies for men versus women, that is, women may have more intimate interactions with male partners, whereas men have more intimate interactions with female partners. Using conventional ANOVA t o analyze the data from this study would result in a source table similar to that presented in Table 1.1. In the table, partner gender is symbolized as X , subject gender is denoted as 2 , and S represents subjects. Listed in the table are the sources of variance, their degrees of freedom, and the error terms for the F tests (the denominator of the F ratio) that evaluate whether each effect differs significantly from zero. The multilevel modeling terms that correspond t o each effect are presented in the last column of the table. These terms are introduced later in the chapter. It is helpful t o have an understanding of the different sources of variance. The between-subject variation in Table 1.1 refers t o the variation in the 80 means derived by averaging each subject’s intimacy ratings over the six partners. This between-subject variation can be partitioned into three sources, the grand mean, subject gender (Z), and subject within gender ( S / Z ) . The mean term represents how different the grand mean is from zero, and the subject gender variation measures whether men or women report more intimacy across their interactions. The third source of variation results from differences between subjects within gender. Within the group of males and females, do some people report more or less intimacy in their interactions? The within-subject variation refers t o differences among partners for each subject: Do people differ in how intimate they see their interactions with their six partners? The partner gender effect ( X ) refers t o whether interactions with male versus female partners are more intimate. The partner gender by subject gender interaction ( X by 2)refers t o whether same or opposite gender interactions are seen as more intimate. The partner gender by subject interaction ( X by S / Z ) is the variation in the effect of gender of partner for each subject (i.e., to what degree does the mean of female partners minus the mean of male partners vary from subject t o subject). Finally, there is variation due t o partner ( P / X S / Z ) ,and the issue is how much the intimacy ratings of interactions with partners differ from one another controlling for partner gender. Each person reports about three male and three female partners, and this source of variance measures how much variation there is in intimacy across interactions with partners who are of the same gender. Because in this example participants interact with a given ‘We use subject t o refer t o the research participants so t h a t subjects ( S ) can easily be distinguished from partners ( P ) in our notation.
Kenny, Bolger, and Kashy
4
Table 1.1 ANOVA Source Table for the Hypothetical Balanced Case
Source
df
Between Subjects
80
Mean
1
Subject Gender (Z)
1
Subject (S/Z)
78
Within Subjects
400
Error Term
s/z s/z P/XS/Z
Parameter
a0
a1
,d2
Partner Gender (X)
1
X by S/Z
CO
X by Z
1
X by S/Z
C1
78
P/XS/Z
of2
320
Not tested
X by S/Z Error (P/XS/Z)
oe
partner only once, this source of variability cannot be distinguished from other, residual sources, such as measurement error in Y.We therefore call all of the remaining variance in Y error. Within this model, there are three random effects: Subject ( S / Z ) ,Subject x Partner Gender ( X by S / Z ) , and Error ( P / X S / Z ) . It is possible to use the ANOVA mean squares to derive estimates for the Subject, Subject x Partner Gender, and Error variances. The subject variance, symbolized as o d 2 for reasons that will become clear in the multilevel modeling section of this chapter, measures variation in average intimacy scores after controlling for both subject and partner gender. The Subject x Partner Gender variance] symbolized as o f 2 ,measures the degree to which the effects of Partner Gender differ from subject to subject after controlling for the subject’s gender. Denoting a as the number of levels of X ( a = 2 in this example) and b as the number of partners within one level of X ( b = 3 in this example)] then the standard ANOVA estimates of these variances are given by Subject: od2 = (MSsIZ - M S p / x s / z ) / a b
(1.1)
Subject x Gender of Partner: o f 2= (MSXbyS/Z - M S p / x s / z ) / b (1.2)
As noted, an exact estimate of the partner variance cannot be obtained because it is confounded with error variance] and so we represent the combination of partner variance and error variance as o e 2 . Finally, although not usually estimated] we could compute the covariance between Subject and Subject x Partner Gender by computing the covariance between the
Estimating Multilevel Models
5
mean intimacy of the subject and the difference between his or her intimacy with male and female partners. Such a covariance would represent the tendency of those who report greater levels of intimacy to have more intimate interactions with female (or male) partners. Although this covariance is hardly ever estimated within ANOVA, the method still allows for such a covariance. The table also presents the usual mixed model error terms for each of the sources of variance. For the fixed between-subjects sources of variance, M S s / z is the error term. To test whether there are individual differences in intimacy, M S s / Z is divided by M S p / s x / z . The error term for the fixed within-subject effects is M S x x s / z . Finally, the error term for MSx.s/z is M S p l s x l z , which itself cannot be tested.
MULTILEVEL MODELS Multilevel Data Structure The ANOVA decomposition of variance just described only applies to the case of balanced data. For unbalanced data, a multilevel modeling approach becomes necessary. A key to understanding multilevel models is to see that these data have a hierarchical, nested structure. Although researchers typically do not think of repeated measures data as being nested, it is the case that the repeated observations are nested within persons. In hierarchically nested data with two levels, there is an upper-level unit and a lower-level unit. Independence is assumed across upper-level units but not lower-level units. For example, in the repeated measures context, person is typically the upper-level unit, and there is independence from person t o person. Observation is the lower-level unit in repeated measures data, and the multiple observations derived from each person are not assumed t o be independent. Predictor variables can be measured for either or both levels, but the outcome measure must be obtained for each lower-level unit. The following example should help to clarify the data structure.
Example Data Set As an example of the basic data structure, we consider a study conducted by Kashy (1991) using the RIR. In the Kashy study, persons completed the RIR for 2 weeks. Like the previous balanced-data example, this study investigated the degree to which partner gender predicts the level of perceived intimacy in interactions with that partner and whether this relationship differs between men and women. Because persons often interacted more than once with the same partner, we computed the mean intimacy across all interactions with each partner that is, for the purposes of this example, we created a two-level data set in which subject is the upper-level unit and partner is the lower-level unit. There are 77 subjects (51 women and 26 men) and 1,437 partners in the
6
Kenny, Bolger, and Kashy
study. The number of partners with whom each person interacted over the data collection period ranged from 5 to 51. The average intimacy across all interactions with a particular partner is the outcome variable, and it is measured for every partner with whom the person interacted. Partner gender, symbolized as X , is the lower-level predictor variable. Note that X can be either categorical as in the case of partner gender ( X = -1 for male partners and X = 1 for female partners) or it can be continuous (e.g., the degree to which the person finds the partner to be attractive). Subject gender is the upper-level predictor variable and is denoted as 2. In repeated measures research, upper-level predictor variables may be experimentally manipulated conditions to which each subject is randomly assigned or person-level variables such as gender, a person’s extroversion, and so on. If Z were a variable such as person’s extroversion, it would he a continuous predictor variable, but because 2 is categorical in the example, it is a coded variable ( 2 = -1 for males and 2 = 1 for females). Finally, the outcome variable, average intimacy of interactions with the partner, is measured on a seven-point scale and is symbolized as Y . Because a second example in which the X variable is continuous is helpful, we make use of the fact that Kashy (1991) also asked subjects to evaluate how physically attractive they perceived each of their interaction partners to be. Ratings of the partner’s attractiveness were centered by subtracting the grand mean across subjects from each score. (We feel that it is generally inadvisable to center X for each subject, so-called group centering.) The second example addresses whether interactions with partners who are seen as more physically attractive tend to be more intimate. We can also use subject gender as an upper-level predictor variable, which allows us to test whether the relationship between attractiveness and intimacy differs for male and female subjects. So, in the example data set, subject is the upper-level unit, and subject gender is the upper-level predictor variable or 2 . Partner is the lower-level unit and partner gender or partner’s physical attractiveness is the lowerlevel predictor or X . Intimacy is the outcome variable or Y , and there is an average intimacy score for each partner. The intimacy variable can range from 1 to 7, with higher scores indicating greater intimacy.
MOST BASIC APPROACH TO MULTILEVEL MODELING: ORDINARY LEAST SQUARES Although it is certainly possible for multilevel modeling to be a challenging and complex data analytic approach, in its essence it is simple and straightforward. A separate analysis, relating the lower-level predictor, X , t o the outcome measure, Y , is conducted for each upper-level unit, and then the results are averaged or aggregated across the upper-level units. In this section we introduce the ordinary least squares (OLS) approach t o multilevel modeling without reference to formulas. Specific formulas describing
Estimating Multilevel Models
7
multilevel analyses follow. Using the partner’s physical attractiveness example, this would involve computing the relationship between a partner’s attractiveness and interaction intimacy with that partner separately for each subject. This could be done by conducting a regression analysis separately for each subject, treating partner as the unit of analysis. In the Kashy (1991) example, this would involve computing 77 separate regressions in which attractiveness is the predictor and intimacy is the criterion. Table 1.2 presents a sample of the regression results derived by predicting average interaction intimacy with a partner using partner attractiveness as the predictor. For example, Subject 1 had an intercept of 5.40 and a slope of 1.29. The intercept indicates that Subject 1’s intimacy rating for a partner whom he perceived to be of average attractiveness was 5.40. The slope indicates that, for this subject, interactions with more attractive partners were more intimate, that is, one could predict that, for Subject 1, interactions with a partner who was seen to be 1 unit above the mean on attractiveness would receive average intimacy ratings of 6.69. Subject 4, on the other hand, had an intercept of only 2.20 and a slope of -.37. So, not only did this subject perceive his interactions with partners of average attractiveness to be relatively low in intimacy but he also reported that interactions with more attractive partners were even lower in intimacy. Note that, at this stage of the analysis, we do not pay attention t o any of the statistical significance testing results. Thus, we do not examine whether each subject’s coefficients differ from zero. The second part of the multilevel analysis is to aggregate or average the results across the upper-level units. If the sole question is whether the lower-level predictor relates to the outcome, one could simply average the regression coefficients across the upper-level units and test whether the average differs significantly from zero using a one-sample t test. For the attractiveness example, the average regression coefficient is 0.43. The test that the average coefficient is different from zero is statistically significant [t(76) = 8 . 4 8 , ~< .001]. This coefficient indicates that there is a significant positive relationship between partner’s attractiveness and interaction intimacy such that, on average, interactions with a partner who is one unit above the mean on attractiveness were rated as 0.43 points higher in intimacy. If meaningful, it is also possible to test whether the average intimacy ratings differ significantly from zero or some other theoretical value by averaging all of the intercepts and testing the average using a one-sample t test. It is very important to note that the only significance tests used in multilevel modeling are conducted for the analyses that aggregate across upper-level units. One does not consider whether each of the individual regressions yields statistically significant coefficients. For example, it is normally of little value to tabulate the number of persons for whom the X variable has a significant effect on the outcome variable. When there is a relevant upper-level predictor variable, 2,one can ex-
8
Kenny, Bolger, and Kashy
Table 1.2 A Sample of First-Step Regression Coefficients Predicting Interaction Intimacy with Partner’s Physical Attractiveness
& J Subject Number
Intercept
Slope
1
5.40
1.29
2
3.38
.03
3
2.64
.44
4
2.20
-.37
26
4.17
.48
Mean
3.78
.38
Women Subject Number
Intercept
Slope
27
4.07
.16
28
4.10
.45
29
3.88
.98
30
5.53
.32
77
4.31
.39
Mean
4.31
.45
9
Estimating Multilevel Models
amine whether the coefficients derived from the separate lower-level regressions vary as a function of the upper-level variable. If 2 is categorical, a t test or an ANOVA in which the slopes (or intercepts) from the lower-level regressions are treated as the outcome measure could be conducted. For example, the attractiveness-intimacy slopes for men could be contrasted with those for women using an independent groups t test. The average slope for men was M = 0.38 and for women M = 0.45. The t test that the two average slopes differ is not statistically significant, t(75) = 0.70, ns. Similarly, one could test whether the intercepts (intimacy ratings for partners of average attractiveness) differ for men and women. In the example, the average intercept for men was M = 3.78 and for women M = 4.31, t(75) = 2.19, p = .03, and so women tended to rate their interactions as more intimate than men. Finally, if 2 were a continuous variable, the analysis that aggregates across the upper-level units would be a regression analysis. In fact, in most treatments of multilevel modeling, regression is the method of choice for the second step of the analysis as it can be applied to both continuous and categorical predictors.
Multilevel Model Equations In presenting the formulas that describe multilevel modeling, we return to the example that considers the effects of subject gender and partner gender on interaction intimacy. As we have noted, estimation in multilevel models can be thought of as a two-step procedure. In the first step, a separate regression equation, in which Y is treated as the criterion variable that is predicted by the set of X variables, is estimated for each person. In the formulas that follow, the term i represents the upper-level unit, and for the Kashy example i represents subject and takes on values from 1 to 77; j represents the lower-level unit, partner in the example, and may take on a different range of values for each upper-level unit because the data may be unbalanced. For the Kashy example, the first-step regression equation for person i is as follows: Kj
= boi
+ b l i X i j + eij
(1.3)
where boi represents the intercept for intimacy for person i, and bli represents the coefficient for the relationship between intimacy and partner gender for person i. Table 1.3 presents a subset of these coefficients for the example data set. Given the way partner gender, or X , has been coded (-1, l ) , the slope and the intercept are interpreted as follows:
boi: the average mean intimacy across both male and female partners b1i: the difference between mean intimacy with females and mean intimacy with males divided by two
Kenny, Bolger, and Kashy
10
Table 1.3 Predicting Interaction Intimacy with Partner’s Gender: Regression Coefficients, Number of Partners, and Variance in Partner Gender
Men Subject Number
Intercept (boi)
Slope ( b l i )
Number of Partners
ax2
1
5.35
.76
11
.87
2
3.39
-.14
8
1.14
3
2.86
.69
16
.80
4
1.94
-.34
15
.84
26
4.41
.37
14
.73
Mean
3.85
.24
Number of Partners
ux 2
Women
Subject Number
Intercept (boi)
Slope
27
4.49
-.11
35
.50
28
4.03
.03
22
.62
29
3.65
.42
15
.50
30
5.98
.47
21
.86
77
4.40
.32
19
.98
Mean
4.39
-.16
(bli)
-
Note:
Gender of partner is coded 1 = female, -1 = male.
11
Estimating Multilevel Models
Consider the values in Table 1.3 for Subject 1. The intercept, boi, indicates that across all of his partners this individual rated his interactions to be 5.35 on the intimacy measure. The slope, b l i , indicates that this person rated his interactions with female partners to be 1.52 (0.76 X 2) points higher in intimacy than his interactions with male partners. For the second-step analysis, the regression coefficients from the first step (see Equation 1.3) are assumed to be a function of a person-level predictor variable 2:
boi = a0
+ a122 + di
(1.4)
bli = co
+ ClZi + fi
(1.5)
There are two second-step regression equations, the first of which treats the first-step intercepts as a function of the 2 variable and the second of which treats the first-step regression coefficients as a function of 2. In general, if there are p variables of type X and q of type 2 , there would be p + 1 second-step regressions each with q predictors and an intercept. There are then a total of p ( q 1) second-step parameters. The parameters in Equations 1.4 and 1.5 estimate the following effects:
+
ao: the average response on
Y for persons scoring zero on both
X and 2 a l : the effect of 2 on the average response on Y
co: the effect of X on Y for persons scoring zero on 2 c1: the effect of
2 on the effect of X on Y
Table 1.4 presents the interpretation of the four parameters for the example. For the intercepts (boi, ao, and CO) to be interpretable, both X and 2 must be scaled so that either zero is meaningful or the mean of the variable is subtracted from each score (i.e., the X and 2 variables are centered). In the example used here, X and 2 (partner gender and gender of the respondent, respectively) are both effect-coded (-1, 1) categorical variables. Zero can be thought of as an “average” across males and females. The estimates of these four parameters for the Kashy example data set are presented in the OLS section of Table 1.5. As was the case in the ANOVA discussion for balanced data, there are three random effects in the multilevel models. First, there is the error component, eij, in the lower-level or first-step regressions (see Equation 1.3). This error component represents variation in responses across the lower-level units after controlling for the effects of the lower-level predictor variable, and its variance can be represented as 0:. In the example, this component represents variation in intimacy across partners who are of the same gender (it is the partner variance plus error variance that was discussed in the ANOVA section). There are also random effects in each of
Kenny, Bolger, and Kashy
12
Table 1.4 Definition of Effects and Variance Components for the Kashy Gender of Subject by Gender of Partner Example
Effect Estimate
Multilevel Parameter
Definition of Effect Typical level of intimacy across all subjects and partners
Constant
a0
Subject Gender (2)
a1
Degree to which females see their interactions as more intimate than males
Partner Gender ( X )
CO
Degree to which interactions with female partners are seen as more intimate than those with male partners
X by Z
C1
Degree to which the partnergender effect is different for male and female subjects
ad2
Individual differences in the typical intimacy of a subject's interactions, controlling for partner and subject gender
Variance Subject
X by Subject
Individual differences in the effect of partner gender, controlling for subject gender
Error
Wihin-subject variation in interaction intimacy, controlling for partner gender (includes error variance)
Table 1.5 Estimates and Tests of Coefficients and Variance Components for the Kashy Gender of Subject of Partner Example Estimation Procedure
WLS
OLS
Multilevel
t
b
t
32.99
4.105
34.14
0.249
2.00
0.270
2.24
.71
0.056
1.18
0.054
1.12
-.200
-3.72
-0.181
-3.78
-0.188
-3.94
IT2
F
g2
X2bf
Parameter
b
t
b
Constant
a0
4.120
34.08
4.097
Subject Gender (2)
a1
.269
2.23
Partner Gender ( X )
co
.038
X by Z
c1
Variances
ML
Subject ( S / Z or d )
gd
x by s/z (f)
Of2
0.026
Error (e)
ge
1.886
0.863
8.22
0.853
1.22
0.025
1.22
8.22
1.888
m. OLS, WLS, and MLS estimates were obtained using the SAS REG procedure, the SAS GLM procedure, and HLM, respectively.
14
Kenny, Bolger, and Kashy
the two second-step regression equations. In Equation 1.4, the random effect is di and it represents variation in the intercepts that is not explained by 2. Note that di in this context is parallel to M S ~ I Z within the balanced repeated measures ANOVA context, as shown in Equation 1.1. The variance in di is a combination of oi, which was previously referred to as Subject variance, and cr,". Finally, in Equation 1.5, the random effect is fi and represents variation in the gender of partner effect. Note that f i here is parallel to M S X b y S / Zwithin the repeated measures ANOVA context, as shown in Equation 1.2. The variance in f i is a combination of 0;, which was previously referred to as the Subject by Gender of P a r h e r variance, and 0,". A description of these variances for the example is given in Table 1.4. Recall that it was possible to obtain estimates of 0; and 0; for balanced designs by combining means squares. As can be seen in Equations 1.1 and 1.2, in the balanced case the formulas involve a difference in mean squares divided by a constant. In the unbalanced case (especially when there is a continuous X ) , this constant term becomes quite complicated. Although we believe a solution is possible, so far as we know none currently exists. The multilevel model, with its multistep regression approach, seems radically different from the ANOVA model. However, as we have pointed out in both the text and Table 1.1,the seven parameters of this multilevel model correspond directly to the seven mean squares of the ANOVA model for balanced data. Thus, the multilevel model provides a more general and more flexible approach to analyzing repeated measures data than that given by ANOVA, and OLS provides a straightforward way of estimating such models.
Computer Applications of Multilevel Models with OLS Estimation One of the major advantages of using the OLS approach with multilevel data is that, with some work, virtually any statistical computer package can be used to analyze the data. The simplest approach, although relatively tedious, is to compute separate regressions for each upper-level unit (each person in the case of repeated measures data). In SAS, separate regressions can be performed using a "BY" statement. If PERSON is a variable that identifies each upper-level unit, the SAS code for the first-step regressions could be:
PROC REG MODEL Y = X BY PERSON Then a new data set that contains the values for boi and bli for each upper-level unit, along with any 2 variables that are of interest, would be entered into the computer. The OLS approach is certainly easier, however, if the computer package that performs the first-step regressions can be used
Estimating Multilevel Models
15
to create automatically a data set that contains the first-step regression estimates. Although this can be done within SAS using the OUTEST = data s e t name COVOUT options for PROC REG, it can be rather challenging because SAS creates the output data set in matrix form. Regardless of how the data set is created, the coefficients in it serve as outcome measures in the second-step regressions.
Complications in Estimation with Unbalanced Data The OLS approach t o multilevel modeling allows researchers to analyze unbalanced data that cannot be handled by ANOVA. As we have noted, there are two major reasons that data are not balanced. First, persons may have different numbers of observations. This is the case in Kashy data set where the number of partners varies from 5 to 51. Second, even if the number of observations were the same, the distribution of X might vary by person. In the example, X is partner gender, and the distribution of X does indeed vary from person to person and so the variance of X differs (see Table 1.3). As noted earlier, data are unbalanced if either the number of observations per person is unequal or the distribution of the X variables differs by person. Note that a study might be designed to be balanced, but one missing observation makes the data set unbalanced.
MULTILEVEL ESTIMATION METHODS THAT WEIGHT THE SECOND-STEP REGRESSIONS The OLS approach does not take into account an important ramification of unbalanced data: The first-step regression estimates from subjects who supply many observations, or who vary more on X , are likely in principle to be more precise than those from subjects who supply relatively few observations or who vary little on X . A solution to this problem is to weight the second-step analyses that aggregate over subjects by some estimate of the precision of the first-step coefficients. How best to derive the weights that are applied to the second-step analyses is a major question in multilevel modeling, and there are two strategies that are used: weighted least squares (WLS) and maximum likelihood (ML). Because the NIL approach is treated in detail in other chapters in the volume, we focus most of our attention on the WLS solution. However, we later compare WLS, as well as OLS, with ML .
Multilevel Modeling with Weighted Least Squares Expanding the multilevel model from an OLS solution to a WLS solution is relatively straightforward. As in OLS, in the WLS approach a separate analysis is conducted for each upper-level unit. This first-step analysis is identical to that used in OLS, as given in Equation 1.3. The secondstep analysis also involves estimating Equations 1.4 and 1.5. However, in
16
Kenny, Bolger, and Kashy
the WLS solution, Equations 1.4 and 1.5 are estimated using weights that represent the precision of the first-step regression results. The key issue then is how to compute the weights. In WLS, the weights are the sums of squares for X or SSi (Kenny et al., 1998). This weight is a function of the two factors that cause data to be unbalanced: The number of lower-level units sampled (partners in the example), a,nd the variance of X (partner gender in the example).
Multilevel Modeling with Maximum Likelihood The major difference between ML and WLS solutions to multilevel modeling is how the weights are computed. The ML weights are a function of the standard errors and the variance of the term being estimated (see chapter 5 for greater detail). For example, the weight given t o a particular boi is a function of its standard error and the variance of di. ML weighting is statistically more efficient than WLS weighting, but it is computationally more intensive. There is usually no closed form solution for the estimate, that is, there is no formula that is used to estimate the parameter. Estimates are obtained by iteration and the estimates that minimize a statistical criterion are chosen. In ML estimation, the first and second-step regressions are estimated simultaneously. Several specialized stand-alone computer programs have been written that use ML to derive estimates for multilevel data: HLM/2L and HLM/3L (Bryk, Raudenbush, & Congdon, 1994), MIXREG (Hedeker, 1993), MLn (Goldstein, Rasbash, & Yang, 1994), and MLwiN (Goldstein et al., 1998). Within major statistical packages, SAS’s PROC MIXED and BMDP’s 5V are available.
ESTIMATION OF WLS USING STANDARD COMPUTER PROGRAMS The estimation of separate regression equations is awkward and computationally inefficient. Moreover, this approach does not allow the researcher to specify that the X effect is the same across the upper-level units. It is possible t o perform multilevel analyses that yield results identical to those estimated using the LLseparate regressions” WLS approach but that are more flexible and less awkward. This estimation approach treats the lower level or observation as the unit of analysis but still accounts for the random effects of the upper level. We illustrate the analysis using SAS’s GLM procedure as an example. The analysis could be accomplished within most general linear model programs. We use SAS because it does not require that the user create dummy variables, but other statistical packages could be used. The WLS analysis that we describe requires that a series of three regression models be run, and then the multilevel parameters and tests are constructed from the results of these three models. Lower-Ievel units are treated as the unit of analysis. In other words,
17
Estimating Multilevel Models
each observation is a separate data record. Each record has four variables: the lower-level predictor variable X , the upper-level predictor variable Z, the outcome variable Y , and a categorical variable, called PERSON in the example that follows, which identifies each individual or upper-level unit in the sample. In the first run or Model 1 the setup is:
PROC GLM CLASS PERSON MODEL Y = Z PERSON X
Z*X
PERSON*X
The mean square error from the model is the pooled error variance or sz. Also, the F tests (using SAS’s Type I11 Sum of Squares) for both PERSON and PERSON by X are the WLS tests of the variance of the intercepts ( s i ) and the variance of the slopes (s:), respectively. Note that this model supplies only the tests of the intercept and slope variances. The other tests are not WLS tests and should be ignored Model 2 is the same as Model 1 but the PERSON by X term is dropped:
’.
PROC GLM CLASS PERSON MODEL Y = Z PERSON X Z*X/SOLUTION This model gives the proper estimates for main effect of X (co) and the Z by X interaction (c1) (see Equation 1.5). The SOLUTION option in the MODEL statement enables these estimates to be viewed. Mean squares for these terms are tested using the PERSON by X mean square (SAS’s Type 111) from Model 1 as the error term. If there are multiple X variables, Model 2 must be re-estimated dropping each PERSON by X interaction singly. Finally, Model 3 is the same as Model 1 except the PERSON term is dropped:
PROC GLM CLASS PERSON MODEL Y = Z X Z*X PERSON*X/SOLUTION INT The term INT is added so that the intercept can be viewed. This model gives the estimates of the Z effect ( a l ) and the overall intercept (ao) from Equation 1.4. The mean squares for these terms are tested using the PERSON Mean Square (Type 111) from Model 1. If there were two X variables, X1 and X z , then Model 2 would be estimated twice. In one instance, the PERSON by X 1 term would be dropped; however, the effects of the both X1 and Xa would remain in the equation as well as the PERSON by X2 interaction. In other instance, the PERSON by X z term would be dropped; however, the effects of the both ’The reader should be warned that, in the output, the Z effect has zero degrees of freedom. This should be ignored.
18
Kenny, Bolger, and Kashy
and X 2 would remain in the equation as well as the PERSON by X 1 interaction. If there were more than one 2 variable, they could all be tested using a single Model 3. The results from the tests of the variances of Model 1 have important consequences for the subsequent tests. If there were evidence that an effect (e.g., f ) does not significantly vary across upper-level units and so s$ is not statistically significant, Model 1 should be re-estimated dropping that term. In this case, instead of using that variance as an error term for other terms in Model 2, those terms can be tested directly within Model 1 using the conventional Model 1 error term. So if s: is not included in the model, co and c1 would be tested using the sz. Rarely, if ever, is the variance of the intercepts not statistically significant. However, if there was no intercept variance, a parallel procedure would be used t o test no and n l . Table 1.5 presents the OLS, WLS, and ML results for the Kashy data set. The OLS and WLS estimates were obtained from SAS using the methods described previously. The ML estimates were obtained using the HLM program (Bryk et al., 1994). Model 1 is estimated first t o determine whether there is significant variance in the intercepts and slopes across persons. There is statistically significant evidence of variance in the intercepts [F(75,1283) = 8 . 2 2 , ~< .001]; however, there is not evidence that the slopes significantly vary [F(75,1283) = 1 . 2 2 , ~= .lo). We adopt the conservative approach and treat the slopes as if they differed. We see that the intercept is near the scale midpoint of four. Because effect coding is used, effects for respondent gender, partner gender, and their interaction must be doubled t o obtain the difference between males and females. We see from the subject gender effect that females say that their interactions are more intimate than reported by males by about half a scale point. The p a r t n e r effect indicates that interactions with females are perceived as one tenth of a point more intimate than interactions with males. Finally, the interaction coefficient indicates that opposite-gender interactions are more intimate than same-gender interactions. One feature t o note in Table 1.5 is the general similarity of the estimates. This illustrates how WLS and even OLS can be used t o approximate the more complicated ML estimates. Of course, this is one example and there must be cases in which ML is dramatically different from the least-squares estimators. We discuss this issue further in the following section. x'1
COMPARISON BETWEEN METHODS In this section we consider the limitations and advantages of OLS, WLS, and ML estimation. The topics that we consider are between and within slopes, scale invariance, estimation of variances and covariances, statistical efficiency, and generality.
19
Estimating Multilevel Models
__---
*.--.
X
Figure 1.1: Individual within (solid line), pooled within (small dashed), and between line (large dashed line).
Between and Within Slopes The coefficient b l i measures the effect of X on Y for person i. In essence, OLS and WLS average these bli values to obtain the effect of X on Y . However, there is another way to measure the effect of X on Y . We can compute the mean X and mean Y for each person, and then regress mean Y on mean X (again weighting in the statistically optimal way) treating person as the unit of analysis. So for the example, we could measure the effect having more female partners on the respondent’s overall level of intimacy. We denote this effect as bg and the average of the b l i or within-subject coefficients as bw . Figure 1.1 illustrates these two different regression coefficients. There are three persons, each with four observations denoted by the small-filled circles. We have fitted a slope for each person, designated by a solid line. We can pool these three slopes across persons to compute a common, pooled within-person slope or bw. This slope is shown in the figure as the dashed line that we fitted for each person. The figure also shows the three points through which bB is fitted (the large-filled circles). The slope bB is fitted through these points and is shown by the large dashed line. There are then two estimates of the effect of X on Y : bw and b s . In essence, bw is an average of the persons’ slopes, and bg is the slope computed from the person means. For the Kashy data set, we estimated these two slopes for the effect of partner gender on perceived intimacy. The value for bw is 0.056, indicating that interactions with female partners are seen as more intimate. However, the value for bB is negative being 0.217. This indicates that people who have relatively more female partners
20
Kenny, Bolger, and Kashy
viewed their interactions as less intimate. (The coefficient is not statistically significant .) The ML estimate, as we have described it, of the effect of X on Y is a compromise of the two slopes of bw and bB whereas the WLS and OLS estimates use only a version of bw. Note that in Table 1.5 the ML estimate for this effect ( X ) is somewhat lower than the WLS estimate because ML uses the negative between slope. In our experience, these two slopes are typically different, and, as the example shows, sometimes they even have different signs. So, it is a mistake t o assume without even testing that the two slopes are the same. The prudent course of action is to compute both slopes and evaluate empirically whether they are equal. If different, in most applications we feel that bw is the more appropriate. To estimate both slopes the following must be done: create an additional predictor variable that is the mean of the X , for each person (Bryk & Raudenbush, 1992). Thus, there are two X predictors of Y : X i j and the mean X . The slope for X i j estimates bw and the slope for mean X estimates bg. Alternatively, the X variables can be “group-centered” by removing the subject mean for each variable (for more on centering in multilevel models see Kreft, de Leeuw, & Aiken, 1995). We should note that, in the balanced case, mean X does not vary, and so bw can be estimated but bB is not identified. Perhaps, the balanced case has misled us into thinking that there is just one X slope (bw) when in fact in the unbalanced case there are almost always two (that may or may not be equal).
Scale Invariance There is a serious limitation t o WLS estimation that is not present in either ML or OLS. Second-stage estimates using WLS estimation of intercepts are not scale invariant, that is, if an X variable were transformed by adding a constant t o it, the WLS second-step solution for the intercepts cannot ordinarily be transformed back into the original solution. The reason for this lack of invariance is that the weights used in the step-two equations differ after transformation. The standard error for the intercept increases as the zero point is farther from the mean. Because of the differential weighting of the intercepts, estimates of cell “means,” using the intercepts, will not be the same. To illustrate this problem using the sample data set, we recoded the data using dummy coding (males = 0, females = 1)instead of effect coding for the both person and partner gender variables. Table 1.6 presents the estimated cell means for the four conditions. We see that there is a difference between the predicted ”means” and so the coding system matters. Because ML estimates the weights simultaneously, it does not t o have this problem. Because OLS does not weight at all, OLS does not have 3However, if the same equation were estimated twice (e.g., an X variable is present
Estimating Mu1tilevel Models
21
Table 1.6 Estimated Cell “Means” for the Four Conditions Using WLS
Person
Partner
Estimated Gel1 “mean”
Gender
Gender
Effect Coding
Dummy Coding
Female
Female
4.221
4.254
Female
Male
4.471
4.503
Male
Female
4.085
4.055
Male
Male
3.611
3.581
this problem. Thus, this serious problem applies only to WLS. One simple solution to the problem is to always center the X variables using the grand mean. It is fairly standard practice to do this anyway.
Estimation of Variances and Covariances One major advantage of ML is that it directly provides estimates of variances and covariances. A procedure for obtaining WLS estimates of variance has been developed (Kashy, 1991), but it is very complicated. We know of no appropriate method for estimating covariances within WLS. Because slopes and intercepts are typically weighted differently, it is unclear how to weight each person’s estimates t o form a covariance.
It seems logically possible that estimates of both variance and covariance could be developed within OLS. However, we know of no such estimates. If OLS were to be used more in estimation in multilevel models, it would be of value to determine these estimators. ML has the strong advantage of providing estimates of these variances and covariances. Unfortunately, we should note that all too often these terms are largely ignored in the analysis. Most of the focus is on the fixed effects. Very often the variances and covariances are as important as the fixed effects. Knowing that X has the same effect on Y for all subjects (i.e., sz is zero) can often be a very interesting result because it implies that effect of X on Y is not moderated by individual differences. in one equation and dropped in the other), ML is likely to weight the effect differently in the two equations. This differential weighting creates difficulties in the decomposition of indirect effects in mediation.
22
Kenny, Bolger, and Kashy
Statistical Efficiency If we assume that the statistical model is correct, OLS is the least efficient, WLS the next, and ML the most. The complex weighting of ML creates this advantage. We wonder, however, whether this advantage may at times be more apparent than real. Consider the Kashy study. For both ML and WLS, why should people who have more partners count more than those with fewer? Statistically, more is better, but that may not be the case in all repeated measures studies. Perhaps, if there is a disparity in the number of observations per person, the researcher might want to test if number of observations (perhaps log transformed) is a moderating variable, that is, does the effect of X on Y increase or decrease when there are more observations? Number of observations would then become a Z variable entered in the second-step equations. We estimated such a model with the Kashy data and did not find evidence for moderation, but we did find a trend that persons with more interaction partners reported lower levels of intimacy.
Generality There are several complications of the model that we might want to consider. First, the outcome variable, Y , may be discrete, not continuous. For instance, in prevention studies, the outcome might be whether the person has a deviant status or not. Second, X or Y may be latent variables. In social-interaction diary studies, there may be several outcomes (intimacy, disclosure, and satisfaction) that measure the construct of relationship quality. It may make sense to treat them as indicators of a latent variable. Third, we have assumed that after removing the effect of the X , the errors are independent. However, the error may be correlated across time, perhaps with autoregressive structure. Fourth, the distribution of errors may have some other distribution besides normal (e.g., log normal). Typically, behavioral counts are highly skewed and so are not normal. Fifth, the variance in the errors may vary by person. Some people may be inherently more predictable than others. Increasingly, ML programs allow for these and other complications. However, it would be difficult if not impossible to add these complications to a least-squares estimation solution. Thus, ML estimation is much more flexible than least-squares estimation.
SUMMARY Multilevel modeling holds a great deal of potential as a basic data analytic approach for repeated measures data. An important choice that researchers will have to make is which multilevel estimation technique to use. Although statistical considerations suggest that ML is the best estimation technique
Estimating Mu1tilevel Models
23
t o use because it provides easy estimates of variance and covariance components, is flexible, and provides estimates that are scale invariant, there are times that OLS might also be very useful. We should note that ML estimation is iterative, and sometimes there can be a failure t o converge on a solution. Moreover, ML estimation, as conventionally applied, pools the between and within slopes without evaluating their equality. Therefore, when ML is used in an unsophisticated manner, it is possible t o end up confounding what may be conceptually very different effects. OLS approaches are familiar and easy t o apply, and results generated by OLS generally agree with those produced by ML. WLS has some advantages over OLS. Its estimates are more efficient and estimates of variance components are possible. However, it suffers from the problem that the intercept estimates are not scale invariant. Notably, if the data set is balanced or very near balanced, there is only a trivial difference between the different techniques. ML estimation still has the advantage that variance components can always be estimated, but, if the design is perfectly balanced, the variance components can be estimated and tested using least squares. A major advantage of both OLS and WLS solutions is that they can be accomplished by using conventional software (although SAS’s PROC MIXED is available for ML). Thus, a researcher can use conventional software t o estimate the multilevel model. WLS and OLS may serve as a bridge in helping researchers make the transition from simple ANOVA estimation t o multilevel model estimation. It may also be a relatively easy way t o estimate multilevel models without the difficulties of convergence and iteration. Finally, and most importantly, it can provide a way for researchers who are not confident that they have successfully estimated a multilevel model using new software t o verify that they have correctly implemented their model. We have generations of researchers who are comfortable with ANOVA and who have difficulty working with multilevel regression models. These people can estimate models using a WLS approach that approximates the more appropriate ML. Regardless how the researcher estimates a multilevel model, we strongly urge the careful probing of the solution. Even the use of standard ANOVA is complicated, and errors of interpretation are all too easy t o make. Researchers need t o convince themselves that the analysis is correct by trying out alternative estimation methods (some of which may be suboptimal), plotting raw data, and creating artificial data and seeing if the analysis technique recovers the model’s structure. We worry that, in the rush t o use these exciting and extraordinarily useful methods, some researchers may not understand what they are doing and they will fail to make discoveries that they could have made using much simpler techniques.
24
Kenny, Bolger, and Kashy
ACKNOWLEDGMENTS Supported in part by grants to the first author from the National Science Foundation (DBS-9307949) and the National Institute of Mental Health (R01-MH51964). Questions to the first author can be sent by email t o
[email protected].
REFERENCES Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage. Bryk, A. S., Raudenbush, S. W., & Congdon, R. T . (1994). Hierarchical linear modeling with the H L M / 3 L programs. Chicago, IL: Scientific Software International. Goldstein, H., Rasbash, J., Plewis, I., Draper, D., Browne, W., Yang, M., Woodhouse, G., & Healy, M. (1998). A user’s guide t o M L w i N . Institute of Education, University of London. (http://www.ioe.ac.uk/multilevel/) Goldstein, H., Rasbash, J., & Yang, M. (1994). M L N : User’s guide f o r version 2.3. London: Institute of Education, University of London. Hedeker, D. (1993). M I X R E G . A F O R T R A N program for mixed-effects linear regression models. Chicago, IL: University of Illinois. Kashy, D. A. (1991). Levels of analysis of social interaction diaries: Separating t h e effects of person, partner, day, and interaction. Unpublished doctoral dissertation, University of Connecticut, Storrs, CT. Kenny, D. A., Kashy, D. A , , & Bolger, N. (1998). Data analysis in social psychology. In D. Gilbert, S. Fiske, & G. Lindsey (Eds.), T h e handbook of social psychology (Vol. 1, 4 ed., p. 233-265). Boston, MA: McGraw Hill. Kreft, I. G. G., de Leeuw, J., & Aiken, L. S. (1995). The effect of different forms of centering in hierarchical linear models. Multivariate Behavioral Research, 30, 1-21. Reis, H. T., & Wheeler, L. (1991). Studying social interaction with the rochester interaction record. In M. P. Zanna (Ed.), Advances in experimental social psychology (Vol. 24, p. 269-318). San Diego, CA: Academic Press.
Chapter 2
Alternative Covariance Structures for Polynomial Models of Individual Growth and Change Stephen VV. Raudenbush Machigan State University In studies of psychological change, researchers seek statistical models that are developmentally meaningful and provide a reasonable fit to the data. They also seek inferences that are fairly insensitive t o questionable assumptions about the random behavior of their data. This chapter compares, contrasts, and integrates two modeling approaches in light of these concerns: a hierarchical linear model and a multivariate model for incomplete data. If the complete data are multivariate normal with homogeneous covariance structure, now-standard hierarchical models are submodels of the multivariate model. This principal can be exploited to compare the fit of alternative hierarchical models with each other and with an unrestricted multivariate model. However, hierarchical models often imply heterogeneity of covariance structure and are therefore more general than the conventional multivariate models for incomplete data. Both models can readily be extended to include the clustering within groups of repeatedly observed participants. Robust standard errors for the fixed regression coefficients are available within both approaches. Taken together, these approaches allow a thorough investigation of the sensitivity of key inferences to alternative model assumptions. The two approaches are illustrated by reanalysis of data from two large-scale longitudinal studies. Hierarchical linear models (” HLM” ) have become increasingly popular
25
26
Raudenbush
in studies of growth and change. As presented by Laird and Ware (1982) and Strenio, Weisberg, and Bryk (1983), the approach is based on a nested pair of models. At the first level, an outcome varies within each person over time as function of a polynomial growth or change curve plus a within- person random error. The parameters of the individual curve are then viewed as outcomes in a second, between-person model. In this second-level model, individual differences in background and experience can be specified to account for individual differences in trajectories of change. Bryk and Raudenbush (1987) showed how this modeling framework can supply estimates of the mean trajectory, of individual variation around that mean, of the reliability of measures of change, of the correlation between true status at any time and true rate of change, and of correlates of change. The approach typically provides reasonable estimates of the individual change function even when an individual’s data are sparse. HLMs are elsewhere described as ”multilevel models” (Goldstein, 1995) or ”random coefficient models” (Gibbons, Hedeker, Waternaux, & Davis, 1988; Longford, 1993), but the essential approach in polynomial studies of change is similar (c.f., Goldstein, 1987, 1989). The models are typically estimated via maximum likelihood (ML) or restricted maximum likelihood (REML), with empirical Bayes estimation of individual growth. Bayesian estimation may be preferred to maximum likelihood when the number of participants is small (Seltzer, 1993). One major advantage of the approach is flexibility in handling timeseries data (Ware, 1985). The analyst can make use of all available data, so that any participant with one or more time point can be included in the analysis. The assumption is that the data are missing at random (Little & Shenker, 1995), although the use of all available data increases the robustness of the results to nonrandom missingness (Schafer, 1996). The approach readily handles unequal spacing of time points across participants. For example, in a study of age-related change, one might wish to assess each subject on a monthly basis or on a given birthday, but the logistics of field research may make this impossible. Then the distance between time points will vary across participants. By viewing each person’s time-series data set as nested within a person, the model readily incorporates these individual differences in the number and spacing of the time-series observations. Neither missing data nor varying timing is gracefully handled within the framework of conventional univariate or multivariate analysis of variance of repeated measures. A second important advantage of the approach is its flexibility in modeling individual change and correlates of change. The change parameters can be defined in a variety of interesting ways. For example, in a linear growth structure, it might be useful to characterize the individual trajectory simply in terms of an initial status and a constant rate of change ‘The robustness of inferences to nonrandom missingness depends strongly on the fraction of missing information, which is minimized when all data are used in the analysis.
Alternative Covariance Structures
27
(Bryk & Raudenbush, 1987). Alternatively, one might characterize growth in terms of a mean level, an average velocity, and a rate of acceleration. Time-varying covariates can readily be included in the polynomial change model (Raudenbush & Chan, 1993; Ware, 1985). Piecewise linear models may be more useful than the standard polynomial model (Bryk & Raudenbush, 1992,Chapter 6). Examples of creative first-level modeling include Francis, Fletcher, Stubing, Davidson, and Thompson’s (1991) study of recovery from head injury; Huttenlocher, Haight, Bryk, and Seltzer’s (1991) study of vocabulary growth during the first year of life; and Horney, Osgood, and Marshall’s (1995) evaluation of contextual effects on anti-social behavior. Predictors of change in the second-level model can be continuous or discrete, and the model can be tailored t o allow one set of predictors for status and other sets of predictors a given aspect of growth. The foundation of this approach to studying change is the ”level-1” model, that is, the model for individual change (Rogosa, Brand, & Zimowski, 1982). This model must be psychologically meaningful because it is the level 1 model that defines the parameters of change that will become outcomes at level 2. However, the specification of the model has strong implications for the marginal variances and covariances of the time-series data, that is, a given hierarchical model ”induces” a set of assumptions about how the outcomes vary and covary over time. A potentially important empirical test of the model is whether these induced assumptions are consistent with the manifest variation and covariation over time. To test model fit requires that a broad array of covariance structures can be estimated and compared with the covariance structure induced by the hierarchical model. Willett and Sayer (1994) showed that two-level models for change typical in a growing number of applications of HLM can also be estimated within the framework of standard structural equation modeling (SEM) software such as LISREL (Joreskog & Sorbom, 1989). The measurement model of SEM corresponds t o the level 1 model of HLM. Here SEM’s manifest outcomes are the time-series data of HLM; the latent variables of SEM are the individual change parameters of HLM; the factor loadings of SEM are the polynomial predictors of HLM and the measurement error variance of SEM coincides with the HLM’s within-person or “level 1” variance. The structural model in SEM then corresponds t o HLM’s level-:! or ”between’A general rule of thumb is that any predictor of a high-level term in the polynomial (e.g., a quadratic term) should also be included as a predictor of each lower-level term (e.g., the intercept and linear terms) in order that results will be invariant under linear transformations of the predictors. This rule can be justifiably ignored when only a specific scaling of the predictors is of interest. 3Willett and Sayer built their approach directly upon the “latent growth curve” modeling approach of McArdle (1986); see also Meredith and Tisak (1990). In contrast with Willett and Sayer’s approach, the latent grwoth curve approach does not specify the factor loadings. Rather, these are estimated from the data, allowing a flexible and parsimonious data-driven representation of the mean growth curve in addition t o allowing a family of flexible models for the variance-covariance structure of the residuals. Although both approaches have important applications, the current chapter considers only the a priori specification of the growth parameters.
28
Raudenbush
person” model. A distinct advantage of the reformulation proposed by Willett and Sayer (1994) is that, once the model is translated into the framework of SEMI the full range of covariance structures implemented in SEM software becomes available. Thus, it is easy within SEM t o allow not only randomly varying intercepts and slopes but also heterogeneous or auto-correlated within-person residuals. Using SEMI one can therefore test a wide range of covariance structures, enabling empirical tests of the covariance structure induced by any given two-level model. A distinct disadvantage of the approach is that currently most available SEM software requires ” time-structured” data that is, each participant is required t o have the same number and spacing of time points. A second disadvantage is that SEM does not admit estimation of time-varying covariates having unequal distributions across persons. Thus it appears that analysts face a forced choice between the HLM approach, which allows a wide variety of data structures and level 1 models but a limited choice of covariance structures, and the SEM approach, which allows a wide array of covariance structures but is somewhat inflexible otherwise. Ironically, psychological researchers have been largely unaware of seminal work by Jennrich and Schluchter (1986), who developed a flexible approach t o studying time-series data having a multivariate normal distribution given a set of covariates. Their approach is founded on maximum likelihood estimation of the multivariate normal variance-covariance matrix for incomplete time series data. It also allows estimation of several alternative restricted models: ”random effects” covariance structures identical t o those normally specified in HLM, autocorrelation models, models for heteroscedastic level 1variances, and models having a factor-analytic structure. This approach assumes that the design is ”time-structured” (i.e., spacing between intended time points will not vary from person t o person), but it does allow randomly missing time-series data. This approach, popular in biomedical applications for more than a decade, thus combines advantages of flexibility in level 1 modeling and missing data while allowing a broad range of covariance structures. We label this approach the ”generalized multivariate linear model” approach or ”GMLM.” Taken together, the HLM, SEM, and GMLM approaches make available t o longitudinal researchers an array of modeling approaches of remarkable flexibility for the study of continuously distributed repeated measures. Indeed, there is evidence of a convergence of approaches as those interested in hierarchical models begin to build in richer covariance structures (Goldstein, 1995) and those interested in SEM generalize their approaches t o facilitate new and interesting extensions of growth curve modeling (Meredith & Tisak, 1990; Muthkn, 1991), including, for example, the use of latent initial status t o predict subsequent growth. However, at least two consid4This work was originally embodied in BMDP’s Program 5v, which has strongly influenced more recent software development by SPSS and SAS. ’Also of intense interest currently are repeated measures models for discrete data, but this topic goes beyond the scope of this chapter.
A1t erna tive Covariance Structures
29
erations remain unaddressed in the applications cited so far. First, it will often be the case that persons who are repeatedly observed will also be clustered within groups such as families, neighborhoods, schools, HMOs, and so on. The hierarchical model can readily incorporate such nesting by adding a level to the model, as illustrated by Bryk and Raudenbush (1992; Chapter 8), who studied school differences in rates of mathematics learning using a three-level model. At level 1 was a model for individual change over time. The second-level described individual differences within schools, and the third level described differences among schools. In effect, a two-level growth model was estimated within each of many schools. Extensions of the SEM or GMLM to the clustering of persons, however, are absent. Within the hierarchical model, Thum’s (1997) approach has potential for direct application to this problem as does the multilevel multivariate model of Goldstein (1995). A second issue for applications of either HLM, SEM, or GMLM is that nearly all of the published applications so far assume the multivariate normality of the residuals. Exceptions include Thum (1997) and Seltzer (1993), who allow specification of a multivariate t distribution for the residuals. This specification enables robust estimation in the presence of ”heavytailed” data, that is, data with more extreme values than are expected under normality. Alternatively, if interest is confined to the mean growth trajectory and its correlates, and if the number of participants is reasonably large, the generalized estimating equation approach of Zeger, Liang, and Albert (1988) can be applied to obtain standard errors that are essentially nonparametric. This chapter considers methods for studying alternative covariance assumptions in studies of polynomial growth and change within the framework of the hierarchical model. It begins with a brief review of the nowstandard two-level model for change and its correlates and describes how this model induces assumptions about variances and covariances over time. It then reconceives the multivariate models as a hierarchical model in which the multiple measures constitute a level in the model, following Goldstein (1987, 1995), Kalaian and Raudenbush (1996), and Raudenbush, Rowan, and Kang (1991). It then combines the ideas from the standard hierarchical model and the multivariate model into a single, hierarchical multivariate model that allows one to: 1. test a variety of covariance structures from the simplest HLM to a completely saturated (unrestricted) model 2. allow for randomly missing data and time-varying covariates having 6Raudenbush and Chan (1993) and Rasbash and Goldstein (1994) provide estimation theory and computational approaches for cross-classified random effects model. Such models allow repeated measures on persons who migrate across social contextual boundaries during the course of a longitudinal study. 71nference for hierarchical models based on bootstrap standard errors appears in Goldstein (1996) and Raudenbush and Willms (1995).
Raudenbush
30 different distributions within persons
3. examine sensitivity of inferences about change to alternative specification of the covariance structure 4. examine robustness of inferences about fixed effects to the assumption of multivariate normality 5. extend application to incorporate nesting of persons within social settings These analytic approaches are illustrated by re-analysis of longitudinal data from the National Youth Survey and the Sustaining Effects Study of Compensatory Education.
HIERARCHICAL MODEL AND ITS IMPLICATIONS FOR VARIATION AND COVARIATION OVER TIME Data To illustrate what has become a standard application of HLM in studies of change, we reanalyze data from the first cohort of the National Youth Survey (Elliott, Huizinga, & Menard, 1989). These data, summarized in Table 2.1, were analyzed by Raudenbush and Chan (1993). Members of the first cohort were sampled in 1976 at age 11 and interviewed annually until 1980 when they were 15. The outcome, described in detail in Raudenbush and Chan (1993), is a measure of attitudes toward deviant behavior] with higher values indicating greater tolerance of pro-deviant activities such as lying, cheating, stealing, vandalism, and drug use. We shall refer to this outcome as "tolerance." The table appears to indicate an increase in tolerance as a function of age during these early adolescent years. However, the means at each age are based on different sample sizes because of missing data. In fact, 168 persons had a full complement of five time-series measurements, whereas 45 had only four, 14 had three, and 5 had two, and 7 had one. To illustrate the SEM approach to the study of change, Willett and Sayer (1994) analyzed the subset of 168 participants with complete data. Our analysis, in contrasts, makes use of all available data from the 239 participants.
Simple Model The general theory of crime of Gottfredson and Hirschi (1990) predicts a near-linear increase in antisocial behavior during these early adolescent years, and it may be that tolerance to deviant thinking is similarly linear. Thus we might formulate the simple Iinear model for each person:
A1t ernative Covariance Structures
31
Table 2.1 Prescription of NYS Sample
Tolerance of Deviant Attitudes Number of Observations
Frequency
Age
n
m
sd
11
237
.217
.197
168
12
232
.241
.212
45
13
230
.332
.270
14
14
220
.410
.290
5
15
219
,444
.301
7
where Yij is the tolerance score for person j at occasion i; aij is the age minus 13 of that person at that occasion, so that 7roj represents the expected tolerance level for participant j at age 13; and 7r1j represents the annual rate of increase in tolerance between the ages of 11 and 15; and rij is the within-person residual, assumed independently normally distributed with mean 0 and constant variance 02. In sum, j indexes persons ( j = 1,...239) and i indexes occasions (i = 1,...,n j ) where nj is the number of interviews for person j with a maximum 5 in these data. The change trajectory for each person j consists of two parameters: 7roj = status age 13 and 7r1j = annual rate of increase. This pair of person-specific change parameters become outcomes in a level-:! model for variation between persons. The simplest level-2 model enables us to estimate the mean trajectory and the extent of variation around the mean: noj = Po0 Tlj
= PlO
+ uoj + u1j
Thus, Po0 is the population mean status a t age 13 and Plo is the population mean annual rate of increase from age 11 to 15. The person-specific random effects are uoj, the deviation of person j ’ s status at 13 from the population mean; and u l j , the deviation of person j ’ s rate of increase from the population mean rate. These random effects are assumed bivariate normally distributed, that is
so that TOO is the variance in status at age 13, 711 is the variance of the rates of change, and 701 = 710 the covariance between status at age 13 and rate of change.
Raudenbusli
32
Results We first consider the results for the fixed effects (Table 2.2, under "random linear slope"). Mean tolerance at age 13 is estimated to be &^= 0.327, se = 0.013. The mean rate of increase is significantly positive, Plo = 0.065, se = 0.0049, t = 13.15. In terms of the standard deviation of the outcome (Table 2.2), this is equivalent to an increase of roughly 20 to 25% of a standard deviation per year. The variance-covariance estimates give information about the degree of individual variation in status and change. For example, the variance of the rates of change is estimated at .ioo = .0025, equivalent to a standard deviation of about ,050. This implies that a p$rticipant who is one standard deviation below the mean in rate of change Po0 = .065 would have a rate of .065 - .050 = .015, quite near to zero, while a participant with a rate one standard deviation above the mean would have a rate .065 .050 = .115, quite a rapid rate of increase (at least a third of a standard deviation per year).
+
Implied Assumptions Concerning Variation and Covariation over Time If we combine the level-] model (Equation 2.1) and the level-2 model (Equation 2.2), we have the combined model
or
where
which has a mean of zero and a variance VUT(Eij)
= roo
2 + 2aij.ro1 + aij711 + cT2
(2.7)
Thus, under the linear model, the variance of an observation at a particular occasion is a quadratic function of aij = age - 13 for person j at time i. By taking the first derivative with respect to age, we also see that the variance will change as a linear function of age:
Thus, the rate of change in the variance has an intercept proportional of 701 and a slope proportional to 711. These are strong assumptions, and it is natural to ask whether the variances across the five time points behave in the way implied by the model.
A1t erna tive Covariance Structures
33
The model also has strong implications for the covariance between two outcomes Yij and Yi/j for person j , that is, outcomes observed, at occasion i and occasion i' for person j: C O V ( € i j ,Q j )
+
= TOO (aij
+
Ui/j)T01 $. U i j U i / j T 1 1
(2.9)
Again the question is whether the covariances between pairs of time points implied by Equation 2.9 accurately capture the "true" covariances. If a study is designed to collect T time points per participant, and if each person has the same variance-covariance matrix, there will be T variances and T ( T - l ) / 2 covariances for a total of T ( T 1)/2 variancecovariance parameters overall. In the current example, with T = 5, there will be 5 variances and 10 covariances for a total of 15 variance covariance parameters. Yet our simple linear model of Equations 2.1 and 2.2 implies that these 15 parameters are effectively linear functions of four underlying parameters: T O O , 701, 711, and c2 (see Equations 2.7 and 2.9). It is possible that four parameters are insufficient to adequately represent the 15 variances and covariances that might be estimated. In this case, our model, which is based on randomly varying linear change functions across the population of persons, is too simple to adequately represent the variation and covariation over time. We might then elaborate the model, for example, by formulating a quadratic model, which would have three random effects per person. The level-1 model might be
+
In this model, 7roj remains the status of person j at age 13; 7r1j becomes the "average velocity," that is, the average rate of increase in tolerance; 7 r 2 j becomes "acceleration." According to past research, tolerance of prodeviant thinking, although increasing during adolescence, will reach a peak and then decline in early adulthood. The quadratic model enables us t o assess whether this diminishing rate of increase has begun to occur as early as 15. If so, values of 7 r 2 j will tend to be negative. We might decide to keep the structure of the level-1 variance simple here so that the level-1 residuals are independent and homoscedastic. However, the variance-covariance structure is now elaborated at level 2: TOj Tlj
7i-2j
= Po0 =PlO = Pzo
+ uoj + Ulj + u3j
(2.11)
where we assume 'This is the homogeneity of dispersion assumption common in multivariate repeated measures. It provides a reasonable starting point for a multivariate analysis, aithough the modeling framework t o be presented is not limited to the homogeneity assumption.
Rauden b ush
34
Note that the level-2 model has six unique parameters: three variances and three covariances. Together with the level-1 variance, then, the model uses 7 parameters to represent the 15 marginal variance-covariance parameters of the five time points. It is of interest to assess whether this model provides a significantly better fit to the data than does the linear change model, which, as we have seen, generated 4 parameters to account for the 15 marginal variances and covariances. Alternatively, it might be that an even simpler between-person model might fit the 15 variances and covariances as well as those given by Equations 2.7 and 2.9. Suppose, for example, that in the linear model the variance of the linear rates of change were null, that is n11 = no1 = 0. Then the expression for the variance in Equation 2.7 would simplify t o Var(cij) equals ~~0 o2 and the expression for the covariance in Equation 2.9 would simplify to C O V ( E~~i~, j, = ) TOO. This model, which constrains the linear rates of change of all persons t o be the same but allows the intercept to vary, would then generate the compound symmetry model commonly assumed in univariate repeated measures analysis of variance. According to this model, variances are constant across time as are the covariances, and the 15 possible variance-covariance parameters associated with the five time points would be effectively reproduced by two parameters. These possibilities are explored in Table 2.2 The fits of the alternative models (linear mean change with compound symmetry, linear mean change with varying linear functions, quadratic mean change with varying quadratic functions) are compared by comparing model deviance statistics. Models can be compared by computing the differences between deviances, which are asymptotically distributed as x2 variates under the null hypothesis that the simpler model adequately fits the data as well as the more complex data does. The degrees of freedom for the test is the difference between the numbers of parameters estimated in the two models. The total number of parameters is the number of variance-covariance parameter plus the number of fixed effects. The results indicate clearly that the compound symmetry model with fixed linear slopes provides a poorer fit than does the model with randomly varying linear slopes. We reach this conclusion by computing the difference between the deviance based on the compound symmetry model and the deviance based on the model with randomly varying linear slopes, obtaining - 229.00 - (-338.07) = 109.07, comparable to the percentiles of the x2 distribution with df = 2, the difference in the number of parameters estimated (the compound symmetry
+
'If the model is estimated via restricted maximum likelihood, the number of parameters is just the number of covariance parameters. See Bryk & Raudenbush, 1992, Chapters 3 and 10.
36
Rauden bush
model has 4 parameters and the randomly varying linear model has 6), p = 0.000. In comparing the model with randomly varying linear func-
tions with the model with randomly varying quadratic functions, there is marginally significant evidence that the quadratic model fits better. Here the difference in deviances is -338.07 - (-348.23) = 10.16, df = 4, p = 0.037. Note also that the standard error estimated for is considerably smaller under the compound symmetry assumptions than under the other two models. Given that the compound symmetry model provides a poorer fit to the data than does either of the other two models, we must conclude that this smaller standard error is unjustified and that inferences about the fixed effects are sensitive to incorrectly assuming that compound symmetry holds. This illustrates a key point about inference for these models. The question is not only whether the data support the variance-covariance assumptions but whether inferences about fixed effects are sensitive to misspecification of the variance-covariance structure. Note, however, that none of the three models reported in Table 2.3 is compared with a model that estimates all 15 parameters associated with the 5 time points nor have we considered alternative covariance structures, including autocorrelated or heteoscedastic level-1 errors. We now turn to that problem.
THE GENERALIZED MULTIVARIATE LINEAR MODEL AS A HIERARCHICAL MODEL Following Jennrich and Schluchter (1986), we now seek to estimate multivariate regression model, where each person, in principal, has five time points, so that it will be possible to estimate the 5 by 5 variance-covariance matrix by pooling data across persons. This would not be difficult if each person had complete data and our results would reproduce the results of a conventional mutivariate analysis of variance (Bock, 1975) or of unrestricted SEM model based on a single population. When the data are unbalanced, however, the task is more challenging. The problem has been solved, of course, not only by Jennrich and Schluchter (1986) but in various algorithms for the imputation of missing data, following the work of Little and Rubin (1987).
Reformulation as a Two-Level Model To address this problem in a way that will readily generalize to a variety of two- and three-level hierarchical models, we adopt the approach of Goldstein, 1995; (see also Raudenbush, 1995; Kalaian & Raudenbush, 1996), who construct a level-1 model that relates the observed to the complete data:
A1t ernative Covariance Structures
37
T
y ~ = j
C
mtijYt;
(2.13)
t=l
where Yij is again the outcome for person j associated with occasion i. Here Ytj.is the value that person j would have displayed if that person had been observed at time t , and mtij is an indicator variable taking on a value of unity of y Z j is observed at time t and zero otherwise. Thus q:, t = 1,... T , represents the complete data for person j whereas y Z j , i = 1,..., nj represents the observed data, and the five indicator variables mtij tell us the pattern of missing data for person j . To make this clear, consider a person in the National Youth Survey who was interviewed at ages 11, 12, and 14 but not at ages 13 or 15; that is, this person would have data at times 1, 2, and 4 but not at times 3 and 5. Then Equation 2.13, in matrix notation, would be
( ;:; ) = ( i Y3j
0 0 0 1 0 0 0 0 1
(2.14)
Yj = MjY;
(2.15)
or
This model says simply that the three available data points for person j were observed at times 1, 2, and 4 so that data were missing at times
3 and 5. Although these data were missing, they do exist, in principle. Thus, every participant has a full 5 by 1 vector of "complete data" Yj* even though the nj by 1 vector of observed data, y j , will vary in length across persons. The level-2 model describes how the complete data change over time. We have, in the case of a mean structure that is linear
y,; = Po0 + P l O U t j + E t j
(2.16)
where utj is age-13 at time t for person j. Thus Po0 and retain their earlier definitions. If we allow the residuals E t j to have arbitrary variances and covariances as a function of time, we might write (2.17)
With these definitions in mind, we may generally write the level-2 model, in matrix notation. as
yj' = xjp + E j ,
Ej
N
N ( 0 , C)
(2.18)
38
Rauden bush
where Y; is again the T by 1 vector of complete data for person j , X j is the T by f matrix of predictors associated with the f ”fixed effects” regression coefficients contained in p, and ~j is a T by 1 vector of residuals assumed T-variate normal, each with mean zero and a T by T covariance matrix A having variances 6; and covariances In the present case, T = 5 and f = 2. In sum, our two-level formulation poses a level-1 model (Equation 2.14) that relates the observed data Y to the ”complete data” Y * ,that is, the data that would have been observed if the researchers were successful in obtaining outcome data a t every time point. Our level-2 model (Equation 2.18) is a standard multivariate normal regression model for the complete data. Algebraically substituting the level-2 expression for Y * into the level-2 model yields the combined model (2.19) Our strategy for estimating this model is described in the technical appendix.
Placing Restrictions on the Model To replicate the linear change model of Equations 2.1 and 2.2 (see also Equations 2.7 and 2.9), we might constrain these variances and covariances in A to be quadratic functions of four parameters:
Thus, the ”standard” hierarchical models for growth and change can be viewed as special cases of the multivariate model for incomplete data. In fact, using the algorithm described in the technical appendix, the results of Table 2.2, produced with ”standard” HLM software, were exactly reproduced by constraining the variances and covariances of the multivariate model as in Equation 2.20. This principle holds when the design calls for time-structured data (the same time-series design for each participant), although the time series data are incomplete as a result of data missing a t random. l o The idea of representing well-known models for change as special cases of an unrestricted multivaraite model with missing data is at the heart of Jennrich and Schlucter’s (1986) approach. The fit of the model in each case can be compared with that provided by the unrestricted model (Equation 2.17). For illustration, Table 2.3 presents results of estimating a series of models of increasing complexity. All models are of the form ‘OThe assumption of missing at random is not as restrictive as it may sound (Little & Rubin, 1987). The assumption is that the probability of a time-series outcome, Yttj being missing is conditionally independent of Ytj given the observed data.
Alternative Covariance Structures
39
Y,f = Do0 + P l o a t j + h o a g + c t j
(2.21)
What varies are the assumptions about the variances and covariances of the residuals c t j . Of interest are the fit of the alternative models for the covariance structure and the robustness of the inferences about the fixed effects, ,8, to misspecification of the variance-covariance matrix. All model comparison tests are reported in Table 2.4. Starting with the simplest, the following models are estimated. 1. The Compound symmetrycompound symmetry model. We have ctj
where uoj and implying
etj
=U
+
(2.22)
O ~ etj
are mutually independent and
etj,
is independent of
ettj
(2.23)
Thus, the compound symmetry model represents the 15 variances and covariance as functions of 2 parameters. The results are similar to those in Table 2.3 because &,is estimated to be very close t o zero. 2. A first-order autoregressive "AR(1)" model, which has the same form as Equation 2.23 but where e t j and e t l j are dependent: ctj
= (1 - p ) e t j
+
pet-lj
(2.24)
Thus
(2.25) (2.26) Note that the autocorrelation parameter is significantly different from zero, a result that can be deduced by referring to the model comparisoii test (model 1 versus model 2), x2 = 65.30, df = 1, = p = .OOO. The estimate is j? = .397, se = .054. 3. A model in which the linear rates of change vary randomly while the quadratic rates are held constant: ctj
= uoj
+ u l j a t j + etj
(2.27)
Here we have a standard HLM with a 2 by 2 covariance matrix at level 2 (Equation 2.3) and independent, homogenous level-1 variance. Thus, the 15 marginal variances and covariances are represented as functions of 4
Table 2.3: Some Alternative Covariance Structures for Autoregressive (1) at level 1 Fixed Effects Intercept, Linear,
Coeff
SE
POO 0.3276 0.0153
PIO
Quadratic,
0.0614 0.0048
PZO 0.0002 0.0034
Log linear at level 1
+
= poo ploaij
+ ,Bzoa21j+ eij
Separate level 1 variances f o r each
Unrestricted
Ratio
Coeff
SE
Ratio
Coeff
SE
Ratio
21.46
0.3281
0.0152
21.56
0.3276
0.0152
21.48
0.3202 0.0150
21.37
12.70
0.0620
0.0048
13.01
0.0608
0.0047
12.85
0.0593 0.0047
12.60
0.06
-0.0002 0.0032
-0.06
-0.0005 0.0032
-0.17
0.0003 0.0031
0.10
0.0403 0 0077 -0 0021
Level 2
yZj
+ 0.0243
i
0.0035 -0.0000 0.0008
]
0.0407 0.0074 -0.0024
+=[
0.0038 -0.0003 0.0011
]
Coeff
SE
Ratio
0.035 0.057 0.019 0.215 0.025 0.045 0.028 0.025 0.027
A =
0.073 0.053 0.048 0.086 0.066 0.090-
Level 1
m 2 = 0.0416
6 = ,3968
a-0'
= -.3503
62'
" 1 '
= -0.063
c-s3 = 0 0269
0'13 = -0.221
= 0 0260
m2'
= 0 0260
m-5'
= 0.0028
Model Fit Deviance
Df
-294.32
-360.99
-363.21
-378.27
ic?
6
12
14
18
CD
Q 5
U
A1ternative Covariance Structures
41
Table 2.4 Summary of Model Comparisons
(a) Sumarry of Fit 1.
2.
3.
4.
5. 6.
7. (b)
Random intercept model (homogeneous level 1 variance) Random intercept model [AR (I) at level 11 Random linear slope (homogeneous level 1 variance) Random quadratic model (homogeneous variance a t level 1) Random quadratic model (log-linear at level 1) Random quadratic model (separate level 1 variance for each time point) Unstructured Comparison of Nested Models
Model 1 versus 2 Model 1 versus 3 Model 1 versus 4 Model 1 versus 5 Model 1 versus 6 Model 1 versus 7 Model 2 versus 7 Model 3 versus 4 Model 3 versus 5 Model 3 versus 6 Model 3 versus 7 Model 4 versus 5 Model 4 versus 6 Model 4 versus 7 Model 5 versus 6 Model 5 versus 7 Model 6 versus 7
Deviance -229.02
df 5
-294.32
6
-338.07
7
-384.23
10
-360.99
12
-363.21
14
-378.27 Difference between Deviances 65.30 109.02 119.21 131.97 134.19 199.25 83.95 10.16 22.92 25.14 40.20 12.76 14.98 30.04 2.22 17.28 15.06
18 df
1 2 5 7 9 13 13 3 5 7 11 2
4 8 2 6 4
p
0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.017 0.001 0.001 0.000 0.002 0.005 0.000 0.330 0.009 0.005
42
Rauden bush
underlying parameters. This model also fits better than does the compound symmetry model x2 = 109.02, df = 2. l1 4. The "standard" HLM quadratic model (where both the linear and quadratic rates of change vary randomly): (2.28) This model corresponds to the HLM with a 3 by 3 level-:! covariance matrix of Equation 2.12 and homogeneous, independent level-1 variance, cz = o2 (results are in Table 2.2) . Now the quadratic as well as the linear growth rates are viewed as varying at random. This model also fits better than does the compound symmetry model x2 = 119.21, df = 5; it fits marginally better than does model 3, x2 = 10.16, df = 3, = 0.17, providing some evidence that participants do vary in rates of "acceleration" (that is, quadratic components of change). 5. A log-linear model for heteroscedastic level-1 variances (nine covariance parameters). In this case, we estimated the 3 by 3 level-2 covariance matrix of Equation 2.12. However, the level-1 errors, although independent, were heteroscadastic and modeled according to a log-linear model (2.29) This model fits better than does model 4, x2 = 12.76, df = 2, p = .002. In particular there is evidence of a quadratic relationship between age and the log of the level-1 variance, 6 2 = -.221, se = .077. 6. A model with time-varying level- 1 variances (11covariance parameters). Again, the level-2 covariance matrix is 3 by 3 as in Equation 2.12 and the level-1 errors are independent, but now a separate 0: is estimated for each t . This model does not fit better than the log-linear model, x2 = 2.22, df = 2, p = .33. 7. The unrestricted model, like the others, represents mean change with three fixed effects but estimates 15 variance-covariance parameters, that is, five variances, S t z , t = 1, ..., 5, and 10 covariances St,!. The results suggest that none of the previous models fits the data as well as does this unrestricted model. The random quadratic model with log-linear level-1 variance structure comes close, x2 = 17.28, df = 6 , p = .009. The unrestricted model appears to facilitate no theoretically interesting interpretation regarding variation in individual change. However, as we shall see, this unrestricted model can be viewed as a special HLM for individual change. Once seen in this light, we can assess sensitivity of inferences regarding variation in individual change to alternative models. I'Note that this model is not directly comparable with the AR(1) model. They are not nested models (one cannot be obtained by simplifying the other). It was not possible simultaneously to obtain a non-negative estimate of variance of the linear growth rates and an estimate of the autocorrelation parameter. It appears that models with random coefficients and autocorrelated level-1 variance will be useful only when the number of time points, T, is relatively large.
Alternative Covariance Structures
43
The Unrestricted Model as a ”Standard” HLM Model For time-structured data, there is always a ”standard” HLM that will duplicate the results of the unrestricted model. Specifically, if the level-1 model is a polynomial of degree T - 2 with all coefficients random, and if the level-1 variance is allowed to vary with time (one variance per time point), the two-level HLM will include T(T+1)/2 covariance parameters, exactly reproducing the marginal variance and covariance estimates of the unrestricted model. However, as T increases, this model will become theoretically uninteresting. In our example, with T = 5, a random cubic model with heterogeneous variances will reproduce the results of the unrestricted model. However, little theory is available to make sense of the cubic term in the change function.
Sensitivity of Inferences t o Alternative Covariance Specificat ions Researchers ought to evaluate models not only by their fit to the data but according to their interpretability. Suppose that interest were to focus primarily on the mean and variance in the ”average velocity,” that is, the average rate of increase in tolerance between the ages of 11 and 15. In that light, higher-order effects might be viewed as incidental to this primary focus. An important question would then be the sensitivity of key inferences to alternative specifications of the variance-covariance structure. Table 2.5 summarizes the results depending on the order of polynomial specified for mean change and for variation in change as well as on the assumptions about heterogeneity at level 1. The results across six models are generally similar. However, there are some differences. In particular, models d and e, which have quadratic mean change, varying quadratic functions across persons and heterogeneous level-1 variances, indicate somewhat greater between person-variation in linear rates than do models a, b, and c, which fit the data less well. However, the best fitting model (model f ) has a cubic mean structure, varying cubic effects, and heterogeneous level-1 variances. This model, in fact, corresponds to an unrestricted variance- covariance structure with cubic mean structure. Inferences about the mean and variation in linear rates for this model are more similar to those based on simpler models (a, b, and c) than to those based on moderately more complex models (d and e). It will generally be sensible to assess sensitivity of key inferences to alternative plausible covariance specifications.
On the ”Generality” of the Unrestricted Model With time-structured data, it is tempting to view the unrestricted model as the most general model that can be estimated within a family of multivariate normal models. However, the restricted model and all submodels presented in Table 2.3 share the assumption of homogeneity of dispersion
Raudenbush
44
Table 2.5 Sensitivity of Inferences Regarding Mean and Variation in Linear Rate of Increasea
Fixed Effects
Level 2
Level 1
Mean Rate
Variance
Deviance
df
a) Linear
Linear
homogeneous
,0647 (.00492)
,00251
-338.07
6
b) Quadratic
Linear
homogeneous
,0647 (.00492)
,00251
-338.07
7
c) Quadratic
Quad.
homogeneous
.0647 (.00493)
.00277
-384.23
10
d) Quadratic
Quad.
log-quadratic
,0620 (.00477)
,00346
-360.99
12
e) Quadratic
Quad.
heterogeneous
,0609 (.00474)
.00382
-363.21
14
f ) Cubic
Cubic
heterogeneous
,0639 (.00489)
.00297
-389.23
19
"Standard errors are in parentheses.
that is, the complete data of each participant are assumed to have a common marginal variance-covariance matrix, A. In fact, many interesting models will not meet that assumption. For example, in our data, we have at each occasion a measure of the exposure of the participant to deviant peers, that is, a measure of the extent t o which that person's peers are tolerant of deviant behavior (see Raudenbush & Chan, 1993, for details). This "exposure" variable will have a different distribution over time for different participants. Suppose we specify "exposure" as a predictor (a "time-varying covariate") in the level-1 model of HLM. Then the marginal variance-covariance matrix will be heterogeneous if either of two conditions hold: exposure has a random coefficient, or exposure is related t o the level1 variance. These models can readily be estimated within the framework of HLM. However, these models are not special cases of the generalized multivariate linear model nor can they be estimated within the framework of SEM. To illustrate, we estimate a "standard" two-level model, where a t level-1 we have (2.30) The level-1 variance is allowed to remain homogeneous. At level-2, all coefficients are random: (2.31)
Alternative Covariance Structures
45
Thus, the level-2 model has 10 parameters, with 11 covariance parameters overall. We compare this with the "unrestricted model"
which involves 15 variance covariance parameters. Results are displayed in Table 2.6. Note that inferences about the fixed effects are essentially identical. However, the deviance associated with the HLM model based on only 15 parameters is actually smaller than the deviance associated with the unrestricted model, which has 19 parameters. The models are not nested because the HLM model, which has fewer parameters, is not a submodel of the unrestricted model. The HLM model induces heterogeneous variancecovariance matrices across participants as a function of participant variation in exposure, whereas the unrestricted model is general only within the class of models assuming homogenous variance-covariance matrices. l 2
Robust Standard Errors All of the results discussed so far assume that the random effects in the model are multivariate normal in distribution. All but the last HLM model assume homogeneous covariance matrices, and that model implies a definite structure to the heterogeneity (it must depend on exposure). If interest is confined t o the fixed effects of the model (the P s ) and their standard errors, one can compute Huber-corrected robust standard errors that do not assume normality nor any particular covariance structure (cf. Zeger et al., 1988). The consistency of these standard errors depends on the number of participants. (See Appendix for technical details.) Table 2.7 displays results of standard error estimation for the model having a mean function that is quadratic. The model-based standard errors are founded on the assumption of random variation in quadratic change and homogeneous variance. The "robust" standard errors require no assumptions about the specific structure of the variances and covariances and do not require normality of the residuals. Yet the two sets of standard errors are identical t o three significant digits, implying that the results are not sensitive t o model assumptions in this case. This happy result will certainly not always arise, and assessing the robustness of standard errors is a useful strategy, especially when the number of participants is reasonably large.
INCORPORATING CLUSTERING OF PERSONS WITHIN SOCIAL SETTINGS Two-level models for repeated measures on persons can readily be adapted t o incorporate the clustering of persons within social settings. Bryk and "Note also that exposure is essentially continuous. Thus, the HLM results cannot be duplicated by construction of a finite set of covariance matrices.
Table 2.6: A Comparison between an Unrestricted Model (Homogeneous Dispersion) and a Model with Heterogeneous Dispersion Model 1 : Complete Data have Unstructured but Homogeneous Dispersion Faxed Effects
Coeff
Intercept, BOO
0.3252
Linear,
PIO
SE
*
Model 2: Dispersion Depending on Exposure to Deviant Peers
SE
Ratio
Coeff
Ratio
0.0127
25.67
0.3251
0.0125
25.86
0.0487
0.0045
10.74
0.0466
0.0047
10.00
Quadratic,
,I320
-0.0006
0.0030
-0.21
0.0006
0.0030
0.21
Exposure,
p30
0.3186
0.0244
13.07
0.3430
0.0295
11.62
Variance-covariance Component 0.035
A=
I
0.011
0.014
0.015
0.014
0.035
0.018
0.016
0.016
0.054
0.034
0.028
0.062
0.042 0.062
0.0236
0.0034 0.0021
-0.0016 0.0000
Df
-0.0029
T =
0.0038
0.0000
0.0457
-
2
= 0.0210
Model Fit Deviance
0.0072
-517.26
-520.63
19
15
ic?
E-cr E-
A1t erna tive Covariance Structures
47
Table 2.7 Robustness of Model-Based Standard Errors for Y z j = POO P10aij P z o ~ : ~E i j
+
+
+
HLM with + u1jaij + EZj homogeneous level-1 variance
Eij
Parameter Mean,
Po0
Linear, Pl0 Quadratic,
= uoj
t
Robust Standard Errors
t
Coeff
SE
0.3272
0.0153
21.38
0.0153
23.38
0.0647
0.0049
13.14
0.0249
13.14
PZO 0.0002 0.0032
0.05
0.0032
0.05
SE
Raudenbush (1988) studied school differences in children’s growth by adding a third level to the standard two-level model for individual change. The first level thus represented individual change over time, the second level represented individual differences in change within schools, and the third level represented variation between schools. Given time-structured data, the three-level HLMs are submodels of a two-level general multivariate linear model (cf. Thum, 1997). Thus, by adding a level to Jennrich and Schlucter’s (1986) multivariate model, we can estimate a range of covariance structures within a model that incorporates clustering. We shall reanalyze the Sustaining Effects Data earlier analyzed by (Bryk & Raudenbush, 1988). Mathematics achievement, measured on a vertically equated scale to reflect growth, is the outcome. Children were observed at spring of kindergarten and twice annually during first and second grades. Although the aim was to obtain five repeated measures, some children were absent at various testing times. Thus, the data were time-structured in design but incomplete in practice. The 618 participants were nested within 86 schools.
Three-Level HLM Model Bryk and Raudenbush (1988) formulated a three-level model, where, at level 1, the mathematics outcome for student j in school Ic was represented as depending linearly on time plus the effect of a time-varying covariate:
48
Raudenbush
Here time = 0,1,2,3,4 at times 1,2,3,4,5. Thus, 7rojk represents the initial status of student j in school k . “summer” takes on a value of 1 if the previous time was summer and 0 if not. It can readily be shown, then, that 7 r 1 j k is the calendar year growth rate for student J’ in school k ; T l j k T2jk is the summer growth rate and nljk - 7 r 2 j k is the academic year growth rate. Thus, three 7r’s capture the growth of each student as a function of an initial status, a calendar year growth rate and a summer effect. At level-2, the 7r’s become outcome variables in a model that explains variation between students within schools. For example, we might have
+
npjk
= PpOk f
P p l k ( c h i l d p0v)jk
+upjk
(2.35)
Here (child pov)jk is an indicator taking on a value of 1 if child j k is in poverty (as indicated by eligibility for a free lunch) and 0 if not. The variances and covariances among the u p j k are collected in matrix T. Combining these two models, we have
The model implies, then, that, within a school, math achievement at a given time depends upon the “time,” “summer,” and “child poverty,” two-way interactions of child poverty with “time” and “summer,” plus a random error:
The ”standard” three-level HLM can test alternative covariance structures by setting one or two of the u’s in Equation 2.35 to zero. The level-3 model accounts for variation between schools. For example, we might simply estimate
PlOk
=
+ WOOk 7100 + WlOk
P20k
=
7200 +W20k
POPk
=
YOp0,P
book
=
Yo00
(2.38)
>0
Here the variances and covariances of the
up0k
are collected in a matrix
R. In subsequent analyses, the random effect of summer is dropped from Equations 2.33 and 2.36.
A1t ernative Covariance Structures
49
Reformulation as a Hierarchical Multivariate Model with Incomplete Data Level-1 Model The first level of the model again represents the relationship between the observed and complete data. Schools are incorporated simply by adding a subscript to Equation 2.13: T=5
(2.39) Thus x j k is the math achievement score on the ith occasion (i = 1,...,njk) for student j in school k and q;kis the score that would have been observed if student j within school k had been present a t time t (t = 1,...5 ) . The indicator rntijk takes on a value of 1 if occasion i corresponds to time t.
Level-2 Model The level-2 model is a multivariate, within-school model for the complete data:
q*jk
=
POOk
+ PlOk(time)tjk + P20k(summer)tjk
+POlk(childpov)jk +Pzlk(child p 0 v ) j k
+ Pllk(childpov)jk * (time)ijk
(2.40)
* (SUmmer)ijk -k E t j k
This level-2 model has the same form as the level-2 model of HLM, but now the residuals may have an arbitrary variance-covariance matrix, A , composed of elements
Alternatively, the covariance matrix A may be structured as in the twolevel case. The level-3 model has the same form as Equation 2.38.
Results Table 2.8 compares results from three models. The first two models are three- level hierarchical models with randomly varying intercepts and annual growth rates at level 2 (between children within schools) and level 3 (between schools). They differ in that the first model assumes homogenous level-1 variance whereas the second model allows a separate level-1 variance for each time point. This second model fits better than the first, x2 = 29998.90 - 29966.80 = 32.10, df = 4, p = ,000. However, neither fits as well as the model with unrestricted variances and covariances at level 2. Despite the poorer fit of the simpler models, inferences about the fixed effects
50
Rauden bush
are remarkably similar. Of most importance in the original analysis was the extent of variation between schools in the annual rates of growth. The estimate of 14.78 based on the model with unrestricted level-:! covariance structure is similar to that based on the hierarchical model with heterogeneous variances, 14.33, and a bit smaller than the estimate based on the hierarchical model with homogenous level-1 variance. As in the two-level case, it is possible and often useful to estimate key parameters under a variety of plausible alternative specifications as a sensitivity analysis. It will also be useful and straightforward to compare standard errors for fixed effects based on robust estimation, especially when the number of level-3 units is reasonably large.
DISCUSSION In applying hierarchical models to repeated measures data, the level-1 or "within-person" model is often a polynomial function of age or time. Its parameters might include status, rate of change, and acceleration at a given age, as illustrated by reanalyses of data from the National Youth Survey (NYS). As we have seen, the level-1 model might include other time-varying covariates, such as exposure to deviant peers in the NYS data or the summer effect on learning in the Sustaining Effects data. Any parameter of the level-1 model might be viewed as varying randomly over persons. Such modeling decisions induce assumptions about the covariance structure of the time-series data. This chapter has considered how to compare the fit of alternative models and how to assess the sensitivity of key findings to alternative assumptions about the covariance structure of the time-series data. In comparing these alternative models, it is essential to make three fundamental distinctions: (a) between models that assume homogeneity of dispersion of the "complete data" and models that do not, (b) between models that assume multivariate normality and those that do not, and (c) between models that do or do not incorporate nesting of participants within social settings. These distinctions will have definite implications for choosing approaches to estimation and computational algorithms. Moreover, an awareness of these distinctions is essential in considering sensitivity of findings to alternative assumptions.
Homogeneity Versus Heterogeneity of Dispersion This chapter has contrasted what have become conventional hierarchical linear models (HLMs) with multivariate models for incomplete data (general multivariate models or GMLMs). The GMLM can be viewed as a special case of a hierarchical model where level-1 represents the relationship between the observed and complete data and level-:! represents the model for the complete data. If the complete data have homogeneous covariance
Table 2.8: Model Comparison for Repeated Measures on Children within School Fixed Effects Intercept, yo0 Child Poverty, yolo Time out, 7100 Child Poverty x Time out, yllo Summer Drop, yzoo Child Poverty x Summer Drop, 7 2 1 0 Variance-Covariance Parameters
Coeff
SE
Ratio
Coeff
SE
Ratao
Coeff
SE
Ratio
403.40 0.49 27.74 0.37 -27.27 -0.12
3.70 1.87 0.83 0.42 2.09 1.06
109.03 0.26 33.56 0.89 -13.03 -0.49
403.20 0.60 27.74 0.29 -27.27 -0.51
3.67 1.85 0.83 0.42 2.05 1.03
109.86 0.33 33.47 0.70 -13.33 -0.49
402.98 0.66 27.58 0.35 -27.46 -0.43
3.64 1.84 0.82 0.42 1.91 0.96
110.97 0.32 33.46 0.85 -14.38 -0.45
R=
116.36
-0.09 17.36
R=
126.71
-6.36 14.33
R=
140.07
1707
u 2 = 610.70
8.1= " 543.99 8; = 555.92 8; = 683.53 8: = 468.05 6-52 = 844.03
29998.90 13
29966.80 17
-2.80 14.78 1119 1143 1255 1212 2080 1463 1853
Model Fat VI r
Deviance Df
29941.26 24
1 1128 1260 1367 1346 2201
52
Raudenbusli
structure (each participant within a given population has the same timeseries covariance structure), alternative ”conventional” HLMs can, in turn, be viewed as submodels of the GMLM. This was illustrated in our reanalysis of the NYS data. We showed that by imposing certain specific restrictions on a full T by T covariance matrix, we could replicate the results of the now-standard HLM analysis. Within that framework, it is straightforward to compare alternative HLMs with each other, with the unrestricted model, and with models that have not been widely applied within the HLM framework, models including autocorrelated level-1 errors and log-linear models for heterogeneity of level-1 variance over time (see Tables 2.2, 2.3, and 2.4 and associated discussion). The conventional HLMs are therefore submodels of a GMLM, but only within the class of models that assume the complete data t o have homogeneity of dispersion within subpopulations. A key feature of such ”homogeneity” models is that level-1 predictors having random effects must have the same distribution within the complete data of every participant in a given subpopulation. For example, in the NYS data, age has the same distribution within every participant’s complete-data record: the planners of that study aimed to collect data annually, at ages 11, 12, 13, 14, and 15. However, exposure to deviant peers will have a different distribution for different participants. Thus, a model with randomly varying effects of exposure falls outside the class of GMLMs but not outside the class of HLMs. In that sense the GMLM is a special case of the HLM when the withinpopulation covariance structure is constant over participants (see Table 2.6 and associated discussion). Another example arises when the level-1 variance depends on a continuous level-1 predictor that varies randomly over participants. For example, if the level-1 variance were a function of exposure, the marginal variance-covariance structure would be different for every participant. In many cases, the theoretical focus will be on a small number of key parameters where other parameters are of more incidental interest. One can and generally should assess the sensitivity of key inferences to plausible alternative covariance specifications (see Table 2.5 and associated discussion).
Normal Versus Non-normal Models Both standard HLM and the GMLM require the assumption that the residual vector for each participant has a multivariate normal distribution. Seltzer (1993) and Thum (1997) relax this assumption by allowing the random effects at level 2 to have a multivariate t distribution, an approach that is particularly appropriate when the number of participants is small and one seeks robustness with respect to outlying participants. If the number of participants is large and attention is restricted to fixed effects, Huber-type robust standard errors are available that are essentially nonparametric. One can compare inferences based on these robust standard
A1t erna tive Covariance Structures
53
errors with inferences based on a given model for the covariance structure as a way of assessing sensitivity of inferences to assumptions about the covariance structure (see Table 2.8 and the associated discussion). An important but separate set of models is available for discrete outcomes, including binary data, ordinal data, and counts. The level-1 models are generalized linearized linear models (McCullogh & Nelder, 1989; Diggle, Liang, & Zeger, 1994), defined by a sampling distribution (e.g., binomial, multinomial, or Poisson), and a nonlinear link function (e.g., logit or log link). The coefficients of the level-1 model then vary randomly over the participants in a level-2 model. The level-2 random effects have most often been assumed multivariate normal (e.g., Goldstein, 1995; Longford, 1993). However, robust standard errors are available (e.g., Bryk et al., 1996; Zeger et al., 1988; Chapters 5 and 6).
Clustering of Participants Having reformulated the GMLM as a hierarchical model, it is straightforward to add levels to incorporate the clustering of repeatedly observed persons within social settings such as schools or treatment centers. The level-1 model relates the observed to the complete data for each participant; the level-2 model is a multivariate normal regression model for variation within clusters, the regression coefficients of which vary randomly over clusters in a level-3 model. Thus, it is possible to compare and contrast alternative level-1 covariance structures just as in the case of two-level models (see Table 2.8 and associated discussion). Again, robust standard errors are available for the fixed effects, and these are most applicable when the number of clusters is relatively large.
FINAL REMARKS In approaching the study of individual growth and change, this chapter emphasized certain key principles: (a) the formulation of level-1 models that are developmentally meaningful, (b) the specification of level-:! models that link to key hypotheses about individual differences in development, and (c) the examination of the sensitivity of key inferences to alternative models of the covariance structure in light of the adequacy of their fit to the data, and (d) examination of the sensitivity of inferences about fixed effects to parametric assumptions. A topic of vital interest not considered here is the robustness of inferences to non-ignorable missingness. An explicit model for the complete data and its relationship to the observed data, key to the approach we have adopted, is a foundation for inquiry into this problem (Schafer, 1996). As an increasing variety of new algorithms and packages become increasingly accessible, it is essential to keep the fundamental issues of model specification and statistical conclusion validity in the foreground. An as-
Raudenbush
54
sessment of the robustness of key inferences within a study appears crucial in building durable knowledge in behavioral science.
ACKNOWLEDGEMENTS Research reported here was supported by the Project on Human Development in Chicago Neighborhoods with funding from the John D. And Catherine T . MacArthur Foundation, the National Institute of Justice, and the National Institute of Mental Health.
TECHNICAL APPENDIX We can estimate the complete-data model of Equation 2.18 by applying maximum likelihood to the observed-data model of Equation 2.19. Setting X j = M j Xj*and ~j = Mj ~ j ,* we have
T=Xjp+Ej,
€j"(O,V,)
(2.42)
where V, = Mj AM?. Then, give V,,p can be estimated via generalized least squares:
Iterative re-estimation of Equation 2.43 based on an updated value of Vj gives the Fisher-scoring algorithm for estimation of p. To estimate 5, we need to estimate the unknown elements of A. Let 6 = the vector of unique elements of A and Ej = partial of V, with respect to S; let e j = yj - Xjp. Then, given the current estimates of Vj and p, A can be estimated by generalized least squares as
Iterative recomputation of Equation 2.44 based on updates of p from Equation 2.43 gives the Fisher scoring algorithm for maximizing the likelihood (Raudenbush, 1994). All complete-data models estimated in this chapter used this approach. The various applications require specifications of Ej and V,. Of course, algebraic manipulation is required to render these equations computationally feasible in each case. Details are available upon request to the author. Robust standard errors for /3 are computed at convergence of the algorithm as the square roots of the diagonal elements of
Alternative Covariance Structures
55
(2.45) (2.46)
REFERENCES Bock, R. (1975). Multivariate statistical methods in behavioral research. New York: McGraw-Hill. Bryk, A. S., & Raudenbush, S. W. (1987). Application of hierarchical linear models to assessing change. Psychological Bulletin, 101, 147-158. Bryk, A. S., & Raudenbush, S. W. (1988). Toward a more appropriate conceptualization of research on school effects: A three-level hierarchical linear model. A m e r i c a n Journal of Education, 97, 65-108. Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage. Bryk, A. S., Raudenbush, S. W., & Congdon, R. T. (1996). H L M : Hierarchical linear and nonlinear modeling with t h e H L M / 2 L and H L M / 3 L programs. Chicago, IL: Scientific Software International, Inc. Diggle, P. J., Liang, K.-Y., & Zeger, S. L. (1994). Analysis of longitudinal data. Oxford: Clarendon Press. Elliott, D., Huizinga, D., & Menard, S. (1989). Multiple problem youth: Delinquency, substance use, and mental health problems. New York: Springer-Verlag. Francis, D., Fletcher, J., Stubing, K., Davidson, K., & Thompson, N. (1991). Analysis of change: Modeling individual growth. Journal of Consulting and Clinical Psychology, 39, 27-37. Gibbons, R., Hedeker, D., Waternaux, C., & Davis, J. (1988). Random regression models: A comprehensive approach to the analysis of longitudinal psychiatric data. Psychopharmacology Bulletin, 24, 438-443. Goldstein, H. (1987). Multilevel models in educational and social research. New York: Oxford University Press. Goldstein, H. (1989). Models for multilevel response variables with a n application t o growth curves. New York: Academic Press. Goldstein, H. (1995). Multilevel statistical models (2nd ed.). New York: Halstead Press.
56
Raudenbusli
Goldstein, H. (1996). Consistent estimators for multilevel generalized linear models using an iterated bootstrap. Multilevel Modeling Newsletter, 8, 3-6. Gottfredson, M., & Hirschi, T. (1990). A general theory of crime. Stanford, CA: Stanford University Press. Horney, J., Osgood, D., & Marshall, I. (1995). Criminal careers in the short-term: Intra-individual variability in crime and its relation to local life circumstances. American Sociological Review, 60, 805-820. Huttenlocher, J . E., Haight, W., Bryk, A. S., & Seltzer, M. (1991). Early vocabulary growth: Relation to language input and gender. Developmental Psychology, 27, 236-248. Jennrich, R., & Schluchter, M. (1986). Unbalanced repeated-measures model with structured covariance matrices. Biometrics, 42, 809-820. Joreskog, K., & Sorbom, D. Mooresville, IN.
(1989). Lisrel 7 user's reference guide.
Kalaian, H., & Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis. Psychological Methods, 1, 227-235. Laird, N. M., & Ware, J. H. (1982). Random-effects models for longitudinal data. Biometrics, 38, 963-974. Little, R., & Shenker, N. (1995). Missing data. In G. Arminger, C. Clogg, & M. Sobel (Eds.), Handbook of statistical modeling for the social and behavioral sciences (p. 39-75). New York: Plenum Press. Little, R. J. A., & Rubin, D. B. (1987). Statistical analysis with missing data. New York: John Wiley and Sons. Longford, N. T. (1993). R a n d o m coeficients models. Oxford: Clarendon Press. McArdle, J . J. (1986). Latent growth within behavior genetic models. Behavioral Genetics, 16, 163-200. McCullogh, P., & Nelder, J. A. (1989). Generalized linear models. New York: Chapman and Hall. Meredith, W., & Tisak, J. (1990). Latent curve analysis. Psychometrika, 55. 107-122. Muthkn, B. 0. (1991). Analysis of longitudinal data using latent variable models with varying parameters. In L. C. Collins & J . L. Horn (Eds.), B e s t methods for the analysis of change (p. 1-17). Washington, DC: American Psychological Association.
Alternative Covariance Structures
57
Rasbash, J., & Goldstein, H. (1994). Efficient analysis of mixed hierarchical and cross classified random structures using a multilevel model. Journal of Educational and Behavioral Statistics, 19, 337-350. Raudenbush, S. W. (1994). Equivalence of fisher scoring t o iterative generalized least squares in the normal case with application t o hierarchical linear models. (Unpublished manuscript) Raudenbush, S. W. (1995). Hierarchical linear models t o study the effects of social context on development. In J. Gottman (Ed.), T h e analysis of change (p. 165-201). Hillsdale, NJ: Lawrence Erlbaum. Raudenbush, S. W., & Chan, W. (1993). Application of hierarchical linear model to the study of adolescent deviance in an overlapping cohort design. Journal of Clinical and Consulting Psychology, 61, 941-951. Raudenbush, S. W., Rowan, B., & Kang, S. (1991). A multilevel, multivariate model for studying school climate in secondary schools with estimation via the EM algorithm. Journal of Educational Statistics, 16, 295-330. Raudenbush, S. W., & Willms, J. D. (1995). The estimation of school effects. Journal of Educational and Behavioral Statistics, 20, 307-335. Rogosa, D. R., Brand, D., & Zimowski, M. (1982). A growth curve approach to the measurement of change. Psychological Bulletin, 90, 726-748. Schafer, J. (1996). Analysis of incomplete multivariate data. London: Chapman & Hall. Seltzer, M. H. (1993). Sensitivity analysis for fixed effects in the hierarchical model: A Gibbs sampling approach. Journal of Educational Statistics, 18, 207-235. Strenio, J., Weisberg, H., & Bryk, A. S. (1983). Empirical bayes estimation of individual growth curves parameters and their relationships to covariates. Biometrics, 39, 71-86. Thurn, Y. (1997). Hierarchical linear models for multivariate outcomes. Journal of Educational and Behavioral Statistics, 22, 77-108. Ware, J. H. (1985). Linear models for the analysis of longitudinal data. The American Statistician, 39, 95-101. Willett, J. B., & Sayer, A. G. (1994). Using covariance structure analysis to detect correlates and predictors of individual change over time. Psychological Bulletin, 116, 363-381. Zeger, S., Liang, K.-Y., & Albert, P. (1988). Models for longitudinal data: A likelihood approach. Biometrics, 44, 1049-1060.
Chapter 3
Structural Equation Modeling of Repeated Measures Data: Latent Curve Analysis Patrick J. Curran & Andrea M. Hussong University of North Carolina The statistical analysis of repeated measures data over time can be a remarkably challenging task that, if successful, has the potential for allowing significant insight into many important theoretical questions of interest. Over the years, a wide variety of longitudinal statistical models have been proposed t o address this challenge, including repeated measures t-tests, analysis of variance (ANOVA), analysis of covariance (ANCOVA), multivariate analysis of variance (MANOVA), multiple regression, and path analysis. Advances in structural equation modeling (SEM) over the past 25 years have provided many additional statistical methods for analyzing longitudinal data. One SEM method that has had a long and important history within a wide variety of social science research settings is the autoregressive crosslagged (ARCL) panel model. However, because of several limitations associated with this modeling approach when applied under certain conditions (e.g., Rogosa, 1995), the past decade has witnessed the rise of an alternative SEM-based analytic approach t o modeling longitudinal data, the latent curve model. Although latent curve analysis overcomes a number of limitations associated with the ARCL model, it is not without its own limitations. Applied researchers must be able to weigh the advantages and disadvantages of each of these analytic approaches so that an informed decision can be made about the optimal analytic strategy for evaluating
59
68
Curran and Hussoiig
the particular research question at hand (Curran & Bollen, in press). The goal of this chapter is to explicate the advantages and disadvantages of using the SEM-based latent curve model in applied longitudinal research. This will be accomplished both through a discussion of the basic concepts and equations underlying latent curve analysis and through an applied example concerning the development of antisocial behavior in children. We begin the chapter with a description of the theoretical framework, specific hypotheses, empirical sample, and measures that will be used in the applied example. We then briefly review the ARCL model and discuss the potential advantages and disadvantages of this analytic strategy for evaluating longitudinal research hypotheses. We follow this with an introduction to latent curve analysis and a detailed application of these models to a set of theoretically derived research questions. Our primary intent is for this chapter to address the needs of applied researchers by providing a detailed pedagogical introduction to the latent curve model that describes the analytic technique and highlights its advantages, limitations, and potential future directions.
T H E DEVELOPMENT OF ANTISOCIAL BEHAVIOR IN CHILDREN The onset and escalation of antisocial behavior during early childhood can place a child at increased risk for a variety of negative developmental outcomes in adolescence and adulthood, affecting academic attainment, mental health, substance abuse, social adjustment, criminality, and employment success (Caspi, Bem, & Elder, 1989; Loeber & Dishion, 1983; Reid, 1993). Relations between early childhood behavioral problems and later adjustment difficulties unfold in a developmental process (Patterson, Reid, & Dishion, 1992) such that more severe forms of adolescent and young adult conduct problems are likely to be initiated early in childhood and, without intervention, become increasingly difficult to modify over time (Coie & Jacobs, 1993). The seemingly intractable nature of antisocial behavior indicates that early prevention efforts targeting high-risk children within this developmental process are likely to be our most successful mode of intervention [Conduct Problems Prevention Research Group (CPPRG), 1992; Kazdin, 1993; Reid, 19931. However, many previous attempts to prevent and treat childhood antisocial behaviors have been ineffective (CPPRG, 1992; Kazdin, 1985, 1987, 1993), and the lack of developmental theory in conceptualizing these interventions has been at least partially implicated as underlying such intervention failures (Cicchetti, 1984; CPPRG, 1992; Dodge, 1986, 1993). The development of antisocial behavior in children is embedded within a series of complex reciprocal relationships among parents, children, and teachers set across the contexts of the home, school, and peer group (see, e.g., CPPRG, 1992; Patterson et al., 1992). For example, previous re-
Repeated Measures Data
61
search has shown that parents may contribute t o their children’s academic readiness for school entry by providing both emotional support and a cognitively stimulating home environment for their children (CPPRG, 1992). Children who show lack of academic readiness at school entry often experience greater impediments to learning at school, especially if combined with pre-existing inattentiveness, antisociality, and hyperactivity (Moffitt, 1990). As the child progresses through school, continued aggressive and antisocial behavior decreases the time children spend on school-related tasks, further delaying the development of academic skills (Patterson, 1982; Patterson et al., 1992; Wilson & Herrnstein, 1985). As a result, conductdisordered children are more likely t o display a number of academic deficiencies, particularly in the development of age-appropriate reading skills (I
62
Curran and Hussong
t o the initial levels or rates of change in antisocial behavior? Finally, are earlier levels of antisocial behavior related to later development in reading recognition, and are earlier levels of reading recognition related to later development in antisocial behavior? A series of latent curve models are applied to the empirical data both to better understand these research questions and to provide a detailed example of this analytic strategy.
SAMPLE AND MEASURES Subjects for the present study were drawn from the National Longitudinal Survey of Youth (NLSY) of Labor Market Experience in Youth, a study that was initiated in 1979 by the U S . Department of Labor to examine the transition of young people into the labor force. The original 1979 panel included a total of 12,686 respondents, 6,283 of whom were women. Beginning in 1986, an extensive set of assessment instruments was administered to the children of the 6,283 female respondents of the original NLS Youth sample. A total of 4,971 children was assessed in 1986, which represented approximately 95% of those children eligible for interview. These child assessments were again administered every other year following the 1986 interview: 6,266 children were interviewed in 1988, 5,803 in 1990, and 6,509 in 1992. The mothers of the children continued to participate in the NLS Youth annual interviews during this time. As of 1992, at least 1 interview was obtained on 9,360 biological children of the original 6,283 women first interviewed for the NLSY in 1979. Although a very large number of children of the original NLS Youth mothers was interviewed at least one time to date ( N = 9360 by 1992), a much smaller number of mother-child pairs was considered for the present study. Three key criteria were required for inclusion in the sample. First, children must have been between the ages of 6 and 8 years at the first wave of measurement. This choice of age range insured that all children were eligible t o be assessed on the measures of interest to the present study. Second, children must have reported complete data on all measures of interest to the present study at the first wave of measurement. Subjects were included in the present sample if they were missing data on some or all measures after the first wave of measurement. Finally, only one biological child was considered from each mother. Based upon these three criteria, a total sample of N = 405 children was considered for the present analyses. All N = 405 children and mothers were interviewed at Time 1, N = 374 were interviewed at Time 2, N = 297 were interviewed at Time 3, N = 294 were interviewed at Time 4, and N = 221 were interviewed at all four assessments. Of the total sample of N = 405, 49.9% were female and the average child age at Time f was 6.9 years (sd = .64). Data for the NLSY Child survey were primarily collected using personal home interviews administered by trained interviewers. Measures used in the present study were a subset drawn from the much larger complete battery
Repeated Measures Data
G3
of assessments administered to the NLSY mothers and children. Antisocial behaviorAntisocia1 behavior was measured using the Behavior Problems Index (BPI) antisocial behavior subtest, one of six subtests of the BPI developed by Zill and Peterson (Baker, Keck, Mott, & Quinlan, 1993). The antisocial behavior subscale consisted of the mother’s report on six items that assessed the child’s antisocial behavior having occurred over the previous 3 months. The three possible response options were not true (scored = 0 ) , sometimes true (scored = 1) or often true (scored = 2). These six items were summed to compute an overall measure of antisocial behavior, and scores could range in value from 0 to 12. The child’s reading recognition skill was measured using the 84-item Peabody Individual Achievement Test (PIAT) Reading Recognition subtest, one of five subtests of the PIAT. The reading recognition subtest measures word recognition and pronunciation ability, components considered essential to reading achievement. The reading recognition measure was computed by summing the total number of correct items for the 84-item subtest, and scores could range in value from 0 to 84. The final reading recognition scores were divided by 10 to better equate these variances with the other variables under consideration for later analyses. Antisocial behavior and reading recognition were assessed every other year for four assessments. Four explanatory variables were of interest to the current paper. Emotional support and cognitive stimulation provided to the child were assessed using the Home Observation for Measurement of the Environment-Short Form (HOME-SF; based on Baker et al., 1993). Emotional support was computed as a summation of 13 dichotomously scored items as reported by the mother and as observed by the interviewer, and scores could range in value from 0 to 13. Cognitive stimulation was computed as a summation of 14 dichotomously scored items as reported by the mother, and scores could range from 0 to 14. Only Time 1 measures of emotional support and cognitive stimulation were considered. Finally, child age was measured in years at Time 1, and child gender was dichotomously scored such that female was coded as zero and male was coded as one. The means, standard deviations, and correlations for all measured variables are presented in Table 3.1. Given that there was missing data evident at Times 2, 3, & 4, these descriptive statistics are based upon all available cases at that particular time point. A standard statistical analysis of this data would use a listwise deletion procedure in which only cases that had measures on all time points would be included. If this approach were applied here, 45% of the cases (184 of 405) would be deleted. However, recent developments in both statistical theory and available software have provided extremely important options to avoid this significant loss of data. For all models presented here, the newly developed program Mplus (Muthkn & M u t h h , 1998) was used, which allows for the utilization of a missing data estimator. Drawing on statistical theory described by Little and Rubin (1987), maximum likelihood estimation is used incorporating the EM algorithm that provides proper parameter estimates, standard errors,
Table 3.1: Means Standard Deviations, and Correlations for all Measured Variables (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) 1.00 (405) (2) T 2 Anti .45 1.00 (374) (374) .56 1.00 .42 (3) T3 Anti (297) (280) (297) .41 .58 .58 1.00 (4) T 4 Anti (294) (275) (277) (294) (5) T1 Read - .07 - .08 - .02 - .10 1.00 (405) (374) (297) (294) (405) - .13 - .12 - .13 - .10 .66 1.0; (6) T2 Read (375) (362) (283) (279) (375) (375) (7) T3 Read - .14 - .10 - .15 - .08 .54 .78 1.00 (275) (263) (271) (255) (275) (264) (275) (8) T 4 Read - .22 - .18 - .19 - .14 .45 .76 .80 1.00 (270) (260) (256) (264) (270) (261) (239) (270) .22 .24 .21 .18 - .05 - .08 - .04 - .04 1.00 (9) Gender (405) (374) (297) (294) (405) (375) (275) (270) (405) (10) Age .09 .01 .06 - .07 .60 .31 .19 .12 - .04 1.00 (405) (374) (297) (294) (405) -(375) (275) (270) (405) (405) - .12 - .21 - .14 - .22 .12 .19 .20 .20 - .03 .03 1.00 (11) Cognitive (405) (374) (297) (294) (405) (375) (275) (270) (405) (405) (405) (12) Emotional - .24 - .18 - .23 - .22 .05 .14 .18 .22 .04 - .03 .36 (405) (374) (297) (294) (405) (375) (275) (270) (405) (405) (405) 8.89 6.93 5.77 0.50 4.08 5.01 2.06 2.52 1.83 Means 1.66 2.03 (405) (374) (297) (294) (405) (375) (275) (270) (405) (405) (405) 2.58 0.64 1.25 0.50 1.08 1.16 2.15 0.92 1.90 2.04 Standard Deviations 1.66 Note: Italicized numbers in parenthesis indicate the available sample size for each pair-wise correlation. Numbers on the horizontal axis of the table refer to variables numbered on the vertical axis of the table. The mean and standard deviation fox each variable are reported in the last two lines of the table.
(12)
(1) T1 Anti
1.00 (405) 9.20 (405) 2.31
65
Repeated Measures Data
and an omnibus likelihood ratio (or x 2 ) test statistic. A full discussion of this method of estimation is beyond the scope of this chapter, but the end result of this approach is the utilization of all available cases at each time point. See Little and Rubin (1987) and Muthkn and Muthkn (1998) for further details. Prior to applying the latent curve models to the empirical data, we will first present a brief review of the structural equation model followed by an introduction to the ARCL modeling approach.
THE STRUCTURAL EQUATION MODELING FRAMEWORK Structural equation modeling, or SEM, is a term used to define a broad class of multivariate statistical models that are widely used in the social sciences. The SEM approach simultaneously estimates relations between observed variables and the corresponding underlying latent constructs and between the latent constructs themselves (Bentler, 1980, 1983; Joreskog, 1971a, 1971b; Joreskog & Sorbom, 1978). From this perspective, the factor analytic model relates the observed variables y to the underlying latent construct q such that y =w
+ Aq +
E
(3.1)
where w is a vector of measurement intercepts, A is a matrix of factor loadings (or measurement slopes), and E is a vector of measurement residuals. Conceptually, this reflects that the observed measures on y are used to define the unobserved latent construct, or factor, believed to have given rise to the measures. Further, the observed variance in y is partitioned into two parts: (a) that which is explained by the underlying factors and (b) the remainder, which is considered error. Thus, the variance of the underlying factors are theoretically “error free.” Once these factors are estimated, then relations among the factors can be examined such that 7 = 01
+BqfC
(34
where a is a vector of structural intercepts, B is a matrix of structural slopes, and C is a vector of structural residuals and V ( [ )= 9 represents the covariance structure among the latent factors. Thus, Equation 3.1 relates the observed measures to the underlying latent factors, and Equation 3.2 relates the latent factors to one another. The SEM framework allows for the estimation of a wide array of powerful and flexible statistical models that can be applied to test research hypotheses in many areas of social science research. Although a comprehensive review of all such techniques is beyond the scope of this chapter, more detailed discussions of these models can be found in Dwyer (1983) and Windle (1997). However, two longitudinal SEMs that have received significant recent attention are the autoregressive crosslagged (ARCL) model and
Curran and Hussong
66
the latent curve model. We next provide a brief review of the ARCL model followed by a more detailed discussion of the latent curve model.
T H E AUTOREGRESSIVE CROSSLAGGED MODEL Our analytic goal is to empirically evaluate the four research questions described earlier. One technique available to accomplish this is the ARCL modeling strategy. The ARCL approach is based on the classic simplex model developed by Guttman (1954) and extended by Anderson (1960), Humphreys (1960) , Heise (1969), and Joreskog (1979). This modeling strategy incorporates two main components. First, later measures of a construct are predicted by earlier measures of the same construct, thus giving rise to the term “autoregressive.” For example,
ANTISOCIALit = pt
+ ptANTISOCIALi,t-l + &it
(3.3)
indicating that the measure of antisocial behavior for individual i a t time point t is an additive combination of a time specific intercept ( p t ) , a weighted contribution of the prior measure of antisocial behavior ( p t ) , and an individual and time specific random error ( & i t ) . Larger positive values of the regression parameter are usually interpreted as indicating greater stability of the construct over time (but see Rogosa, 1995, for concerns about this interpretation), that is, on average, scores above the mean at time t tend also to be above the mean at time t 1. Whereas the model described in Equation 3.3 is uniuariate (given that the repeated measures of only a single construct is considered), this can be extended to a multivariate model in which two or more constructs are examined simultaneously. Here, not only are the later measures of one construct regressed upon earlier measures of the same construct but the later measures of one construct are also regressed on earlier measures of other constructs as well. For example, the measures of reading recognition could be incorporated such that
+
indicating that later measures of antisocial behavior are a function of an intercept, the weighted contribution of the prior measure of antisocial behavior, the weighted contribution of the prior measure of reading recognition, and a random error term. The model in Equation 3.4 thus has both the autoregressive component as before, but now also has the crosslagged prediction such that earlier measures of reading predict later measures of antisocial behavior. This is sometimes referred to as a residualized change model given that earlier measures of reading recognition predict later measures of antisocial behavior above and beyond the effects of earlier antisocial
Repeated Measures Data
67
behavior. This model can be extended to examine bidirectional relations such that earlier measures of antisocial behavior predict later measures of reading recognition as well. Despite the widespread use of the ARCL modeling approach in many areas of social science research, this analytic technique has been subjected to a great deal of criticism. Critiques address both statistical and theoretical aspects of the model, and many of these are articulated in Rogosa (1995), Rogosa and Willett (1985), and Willett (1988). Briefly, there are three primary concerns within ARCL modeling when addressing hypotheses similar to our four research questions. First, the ARCL is a fixed effects model, meaning that a single value is estimated for the regression parameters (eg., the p’s) that holds for all subjects in the sample. As such, the magnitude of these influences does not vary across individuals, often an unrealistic condition to impose upon the model. Second, the observed mean structure is usually omitted from the ARCL model, thus ignoring potentially important information about mean changes over time. Although means can be introduced into the ARCL model (as was shown in Equations 3.3 and 3.4), it is difficult to explicitly model the structure of the means as a function of time. Finally, changes in the construct between two time points is independent of the influence of both earlier changes and later changes in the same construct. So, change between Times 2 and 3 does not consider change between Times 1 and 2 nor change between Times 3 and 4. This approach thus tends to “chop up’’ multiple repeated measures into a series of two-time point comparisons, which is often not consistent either with substantive theory nor with the structure of the observed empirical data.
LATENT CURVE ANALYSIS Given these limitations, there has been a call for the development of alternative statistical models for analyzing repeated measures over time. One particularly promising technique is often referred to as latent curve analysis. Although the historical roots underlying the latent curve model can be traced back to the seminal work of Tucker (1958) and Rao (1958), the SEMbased latent curve model was first proposed by Meredith and Tisak (1984) and formalized by Meredith and Tisak (1990). This was further developed and expanded upon in a variety of important ways by McArdle (1986, 1988, 1989, 1991), hlcArdle and Epstein (1987), Muthkn (1991, 1993, in press), and Muthkn and Curran (1997), among many others. Examples of recent substantive applications using latent curve analysis includes Curran and Bollen (in press), Curran and Muth6n (1999), Curran, Stice, and Chassiri (1997), Duncan, Duncan, and Hops (1996, 1998), and Stoolmiller (1994). Latent curve analysis draws on the many strengths of the structural modeling framework. One of the basic concepts in structural modeling is that, although we have a set of observed measures of a theoretical construct of interest (e.g., “antisocial behavior”), we are not inherently interested in
68
Curran and Hussong
this set of observed measures. Instead, we are interested in the unobserved latent factor that is thought to have given rise to the set of observed measures. Similarly, within the latent curve framework we are not inherently interested in the observed repeated measures of the construct over time. Instead, we are interested in the unobserved factors that are hypothesized to underlie these repeated measures. However, unlike in the standard SEM factor model where we would like to estimate the latent factor of say “antisocial behavior”, in the latent curve model we would like to estimate latent factors that represent the growth trajectories thought to have given rise to the repeated measures over time. McArdle and Epstein (1987) nicely described these as chronometric factors. So, instead of evaluating the repeated measures from the ARCL framework in which the relations among the observed variables are examined between Time 1 and Time 2, and then between Time 2 and Time 3, and so on, the latent curve model attempts to smooth over the observed measures to estimate the continuous trajectory that gave rise to these time specific observed measures. This conceptualization highlights the elegance of the SEM framework in the estimation of these unobserved growth factors that generated the observed repeated measures. Instead of focusing our explicit interest on the observed repeated measures themselves, we instead use these observed measures to estimate an unobserved latent component thought to underlie our set of measures. Once estimated, the key focus of the analyses is then on these unobserved components of growth. To better understand the basics of the latent curve model, we now progressively explore the research questions about the relation between antisocial behavior and reading recognition described previously. We first present the equations that describe the various latent curve models and then apply these models to the empirical data and interpret the resulting findings. We start by examining the characteristics of developmental trajectories in antisocial behavior over an 8-year period. We then extend this model to include our four explanatory variables of interest and explore several options available for including the influences of reading recognition. Developmental trajectories in reading will then be examined, and this model will be combined with the antisocial behavior latent curve model to evaluate a comprehensive developmental model of the relations between these two constructs. Finally, we discuss limitations of this modeling approach along with interesting directions for future research.
LATENT CURVE MODELING OF T H E PROPOSED RESEARCH QUESTIONS To summarize thus far, the ARCL model approaches the analysis of the repeated measures data in a series of two-time point comparisons. Regardless of how many assessments are made, the relations among the constructs are examined between time t and time t 1. The latent curve model ap-
+
69
Repeated Measures Data
proaches the analysis of the repeated measures data in a rather different way. Instead of focusing on specific time adjacent comparisons, the latent curve model uses the SEM analytic framework to estimate the growth factors believed to underlie the observed measures. It is these growth factors that then become of primary interest for later analysis. To accomplish this, the observed repeated measures of antisocial behavior are expressed as
ANTISOCIALit = qa;
+ AtVP; + &it
(3.5)
where At = 0, 1, 2, 3 for T = 4 assessments. In other words, Equation 3.5 reflects that the observed measure of antisocial behavior for each person i at time point t is expressed as an additive combination of the individual’s own intercept ( Q ~ , )their , own slope ( ~ p , multiplied ) by the coefficient of time ( A t ) , and an individual and time specific random error (&it). Note the subscripting used on these latent factor q’s. First, both the intercept and slope are subscripted with an i indicating that these values are allowed to vary across individuals. This helps overcome one of the limitations of the ARCL model in which o n e parameter estimate was used to represent the stability of the construct for all individuals. Further, there is not a subscript of t on either the intercept or slope indicating that, although these influences are individually specific, they are independent of time. This means that the observed repeated measures taken a t each time point t have been used to estimate the underlying growth trajectories that are independent of t. What this modeling strategy thus provides is a way of expressing the individual intercept and slope components of growth as a function of group and individual influences. These are expressed as
indicating that an individual’s own intercept and slope can be expressed as an additive function of an overall mean intercept ( p a ) and mean slope ( P O ) for the entire sample plus the individual’s own deviation from each of these mean values. The mean values are sometimes referred to as fixed effects and the deviation values as r a n d o m effects. This formulation accomplishes two important things. First, the observed repeated measures of antisocial behavior have now been smoothed over to provide an estimate of the trajectories thought to underlie the repeated measures. Second, the variance of the deviation terms in Equations 3.6 and 3.7 is a direct estimate of the degree of individual variability in the intercepts and slopes within the sample; the greater the variance, the greater the individual differences in starting point and rates of change over time. This model is often referred to as an unconditional model given that there are no predictor variables
70
Curran and Hussong
Figure 3.1: Unconditional univariate latent curve model of antisocial behavior. on the right hand side of Equations 3.6 and 3.7, that is, variance in the intercept and slope factors are not modeled as a function of other explanatory variables. We will see in a moment that these equations can easily be extended to incorporate predictors of the individual differences in these developmental trajectories. As a starting point for the analysis, the unconditional univariate latent curve model presented in Equations 3.5, 3.6, and 3.7 was fit to the sample data (see Figure 3.1). This model will provide initial insight into the first research question relating to the characteristics of developmental trajectories in antisocial behavior prior to introducing additional explanatory measures of interest. For this and all subsequent analyses, model fit was evaluated using the following criteria: the relative magnitude of the omnibus x2 test statistic and the associated p-value where small x2 values indicate better model fit; the root mean squared error of approximation (RMSEA; Browne & Cudeck, 1993) accompanied with a 90% confidence interval (C190) where values falling below about .05 to .08 indicate better model fit; examination of modification indices that are 1 df tests of improvement in model fit associated with the freeing of a particular parameter; and examination of covariance and mean structure residuals as indication of potential local misspecification (see Curran, 2000, for further discussion of assessing model fit). Other traditional incremental goodness-of-fit indices are not available due to the use of the missing data estimator. Using these fit criteria, we concluded that the unconditional latent curve model for antisocial behavior fit the data moderately well [x2(5) = 14.9, p = .01; RMSEA = .07, CIg, = .03, . l l , p(RMSEA < .05) = .18]. There was a significant mean estimate for both the intercept (fia = 1.7) and slope (fro = .15) factors, indicating that, on average, the group was reporting
71
Repeated Measures Data
significant initial antisocial behavior of 1.7 units and significant linear increases of .15 units per time point. Further, there was a sjgnificant variance estimate for both the intercept = 1.29) and slope (Go = .133) factors, indicating that there was meaningful individual variability around both of these group level estimates. Thus, some children were reporting high levels of initial antisocial behavior whereas others were reporting lower levels, and some children were reporting steep increases in antisocial behavior over time, whereas others were reporting no increases at all. Finally, there was a marginally significant correlation between the intercept and slope factors = .36) suggesting that children who reported higher initial levels of antisocial behavior also tended to report steeper increases in antisocial behavior over time. These modeling results indicate that developmental trajectories thought to underlie the four repeated measures of antisocial behavior are characterized by both significant fixed and random effects. Because we found significant individual variability in the intercept and slope factors, we may now introduce our four explanatory variables to try to predict this observed variability. To accomplish this, Equation 3.5 remains the same, but Equations 3.6 and 3.7 are extended to include the effects of our predictor variables of interest such that:
(Ga
7o.i =
+ y5AGEi + 7sGENi + y7COGi + ysEMOTEi + coi
(3.9)
This model is often referred to as a conditional growth model because individual differences observed in the initial starting point and rate of change over time in antisocial behavior are being modeled as a function of the child’s age, gender, and cognitive and emotional support in the home. Note that the explanatory variables are denoted with a subscript i to clarify that these values vary across individuals, but the variables are n o t denoted with a t to denote time. This is because all of the variables were measured only at the first time point. Variables such as these are often referred to as time-invariant couariates to denote that they do not vary as a function of time (Bryk & Raudenbush, 1992). Note also that the regression parameters linking the explanatory variables to the growth factors (e.g., the eight y parameters) are n o t subscripted by i, indicating that these values are estimated pooling over all individuals. The path diagram corresponding to Equations 3.8 and 3.9 is presented in Figure 3.2. This model was estimated and found to fit the data moderately well [x2(13) = 25.2, p = .02; RMSEA = .05, CI90 = .G2, .08, p(RMSEA < .05) = .50]. Results indicated that gender, age, and emotional support all significantly predicted the intercept factor such that boys, older children, and children with lower emotional support at home reported higher
72
Curran and Hussong
Figure 3.2: Conditional univariate latent curve model of antisocial behavior with four exogenous variables. All paths between exogenous variables and growth factors were estimated, but only significant ( p < .05) paths are displayed. Values associated with each path are standardized regression coefficients. initial levels of antisocial behavior. Further, age and cognitive support significantly predicted the slope factor, indicating that younger children and children with lower cognitive stimulation at home reported steeper increases in antisocial behavior over time. Taken together, the set of four explanatory variables accounted for 27% of the observed variance in the intercept factor and 11%of the observed variance in the slope factor. We have found thus far that there are significant initial levels of antisocial behavior, there are significant linear increases in antisocial behavior over time, there is significant individual variability in both starting point and rate of change, and this variability can be modeled as a function of our four explanatory variables. In addition to these important questions, recall that one of our key interests is to better understand the relation between antisocial behavior and reading recognition over time. One option would be to include the Time 1 measure of reading recognition in Equations 3.8 and 3.9 and thus be able to draw the same type of conclusions that were drawn for the other four predictors. However, this is not ideal given that we would be ignoring the latter three measures of reading recognition, thus disposing of a great deal of important information. A better option would be to treat the repeated measures of reading as a time varying covariate. Here, instead of considering the effects of Time 1 reading on the underlying growth trajectories of antisocial behavior (as would be accomplished by including Time 1 reading in Equations 3.8 and 3.9), we instead can introduce the effects of reading on antisocial behavior into Equation 3.5. This is given as
Repeated Measures Data
73
indicating that the observed measure of antisocial behavior for person
i at time t is a function of the underlying intercept factor, the underlying slope factor, and the time-specific reading recognition score for person i at time t. The model in Equation 3.10 is nearly equivalent to the timevarying covariate model within the HLM framework (Bryk & Raudenbush, 1992, Equation 6.21). The gamma (y) parameter estimates in Equation 3.10 would indicate the unique effects of reading directly upon the timespecific measures of antisocial behavior above and beyond the effects of the underlying developmental trajectory of antisociality. Although Equation 3.10 limits a cross-sectional relation between reading and antisocial behavior, Curran, Muthkn, and Harford (1998) describe extensions to the latent curve model that allows for the longitudinal prediction from the time varying covariates as well. The model described in Equation 3.10 (extended to allow for the longitudinal effects of reading on antisocial behavior) is presented in Figure 3.3. This model was estimated and found to fit the data well [x2(19) = 29.1, p = .06; RMSEA = .05, Clgo = 0, .06, p(RMSEA < .05) = 301. Prior to interpretation of the final results, equality constraints were imposed across the regression parameters relating each reading measure to all later antisocial measures (e.g., the regression of all four measures of antisocial behavior on Time 1 reading were set to be equal). These constraints were imposed to evaluate if the regression parameters were of the same magnitude across all time points, and these constraints were not associated with a significant decrement in model fit and were thus retained. The final model fit the data well [x2(25)= 36.9, p = .06; RMSEA = .06, CIgo = 0, .06, p(RR/ISEA < .05) = 371. All the results relating the growth factors to the four explanatory variables remained nearly identical to those found for the previous model. Interestingly, the Time 1 measure of reading recognition was significantly and negatively associated with all four measures of antisocial behavior, although no significant relations were found for the Time 2, 3, or 4 reading recognition assessments. This indicates that higher levels of Time 1 reading recognition were associated with lower levels of antisocial behavior above and begond the effects of the developmental trajectory underlying antisocial behavior. However, the results suggest that, once the effects of Time 1 reading recognition are considered, later measures of reading exerted no unique effect on antisocial behavior. Although this model provides initial evidence that there is indeed some longitudinal relation between reading recognition and antisocial behavior over time, two key limitations remain. First, both developmental theory and previous research would strongly suggest that reading recognition itself is characterized by an underlying developmental process that is clearly not being modeled here. Given that significant intercept and slope factors were found to underlie the four measures of antisocial behavior, it is important that a similar process be examined for reading recognition. Second, although earlier reading recognition was found to predict later antisocial behavior, this model does not evaluate the effects of earlier antisocial bc-
Curran and Hussong
74
Figure 3.3: Conditional univariate latent curve model of antisocial behavior with effects of reading as a time varying covariate. All paths between exogenous variables and growth factors were estimated, but only significant ( p < .05) paths are displayed. All paths between time varying covariates and repeated measures of antisocial behavior were estimated, and non-significant paths are denoted with a value of zero. All other values associated with each path are standardized regression coefficients. havior on later reading recognition. The model must be further extended to allow for the introduction of both of these important characteristics. Prior to estimating this more complicated latent curve model, we must first take a step back and estimate a univariate model to examine the developmental trajectories in reading recognition. This is because we must first accurately understand the unconditional growth process of reading recognition prior to examining the relation of this process with other influences of interest (see, e.g., MacCallum, Kim, Malarkey, & Kielcolt-Glaser, 1997). We thus repeat the unconditional latent curve model presented in Equation 3.5 with the simple modification that instead of modeling the four repeated measures of antisocial behavior, we consider the four repeated measures of reading recognition. This is given as
+
READit = ?la; XtVoi
+ &it
(3.11)
It is extremely important to appreciate the differences between Equations 3.10 and 3.11. Placement of reading recognition on the right side of Equation 3.10 indicates that these measures are treated as independent variables and are used to predict the time specific measures of antisocial behavior. However, reading recognition is placed on the left side of Equation 3.11 indicating that these measures are now dependent variables such that the observed scores depend upon the influences of the underlying growth trajectories . This model was estimated and was found to fit the observed data ex-
Repeated Measures Data
75
tremely poorly [x2(5) = 175.04, p < .001; RMSEA = .29, CIgo = .25, .33, p(RMSEA < .05) < .001].Although significant parameter estimates were found for both the fixed and random effects of the intercept and slope factors, these cannot be interpreted given the remarkably poor fit of the overall model. Further information is needed to better understand this lack of fit. First, an examination of the observed means and variances of the four reading recognition scores indicates that, although there appears to be a monotonic increase in these measures as a function of time, the increment of increase is not equal across time. For example, there is a 1.5-unit increase in the means from Times 1 to 2, a 1.0-unit increase from Times 2 to 3, and a .70-unit increase from Times 3 to 4. Thus, it appears that the pattern of mean change over time is curvilinear, whereas the model estimated in Equation 3.11 forces the functional form of the developmental trajectory to be linear (e.g., At = 0 , 1, 2, 3). Large and significant modification indices (1 df tests that evaluate the degree of improvement in model fit if some of the fixed factor loadings were to be freed) provided further support that the source of model misfit stems from imposing a linear growth function on data that demonstrate a strong curvilinear pattern of change. There are three general options for dealing with this nonlinear form of growth. The first is to simply ignore the issue and move ahead with the analyses. However, we cannot stress strongly enough how ill advised this choice can be. Given that the latent curve model is incorrectly specifying the pattern of growth that is evident in the observed data, no inferences can be made about any aspect of this model given the likely bias that exists throughout many (if not all) of the parameter estimates. Thus, if there is a significant curvilinear trend in the data, it is imperative that this be incorporated in some way into the model. The second choice is to retain the intercept and linear growth factor, but t o add a third factor to account for the nonlinear component in the trajectory (see, e.g., Willett & Sayer, 1994). To accomplish this, Equation 3.11 is extended to add this third factor such that
+
READit = 770; AtqL;
+ A F ~ Q+ &it~
(3.12)
where At = 0, 1, 2, 3 as before, and A: = 0, 1, 4, 9. Now qai represents the intercept, q~~ represents the linear component of the trajectory, and q~~ represents the curvilinear component of the trajectory. Then, just as was done in Equations 3.8 and 3.9, the intercept, linear, and curvilinear components of growth can be regressed upon our four explanatory variables. The model in Equation 3.12 was fit to the observed data and, although the inclusion of the quadratic factor led to a drastic improvement in fit over the linear-only model, the overall model still did not adequately reproduce the observed data [x2(1)= 10.65, p < .001; RMSEA = .P5, CIgo = .08, .24, p(RMSEA < .05) < .01].Examination of the fixed effects indicated a significant positive intercept, a significant positive linear component, and a significant negative curvilinear component. These parameters are consis-
76
Curran and Hussomg
tent with what we would have expected given the pattern of the observed means. However, although there was a significant random component for the intercept and linear factors, there was a near zero random component for the quadratic factor. This suggests that, although there is a curvilinear component of growth for the overall group, there is not significant individual variability in these curvilinear rates of change. Given the nonsignificant variability in the curvilinear component of growth, we re-estimated this model fixing the variance of the quadratic factor t o zero as well as the covariances of the quadratic factor with the other two growth factors (as described by Bryk & Raudenbush, 1992). This resulted in a n equally poor fit to the observed data [x2(4) = 28.0, p < .001; RMSEA = .12, CIgo = .08, .17, p(RMSEA < .05) < .002]. It appears, then, that the quadratic model does not adequately represent the four repeated measures of reading recognition. The third option that we consider is to return to the intercept and linear growth factor model presented in Equation 3.11, but instead of fixing the factor loadings to 0, 1, 2, & 3 as we did before, we fix the first two loadings to 0 and 1 (to set the metric of the slope factor), but we will freely estimate the second two loadings from the data. This approach was described by McArdle (1989), who proposed this completely latent curve model t o allow for "stretching" and "shrinking" of the time measure t o account for nonlinearity in the observed data. We estimated this model and found it to fit the observed data quite well [x2(3)= 4.4, p = .22; RMSEA = .03, CIgo = 0, .lo, p(RMSEA < .05) = .58]. The first two factor loadings were fixed to 0 and 1, and the third and fourth loadings were estimated to be 1.6 and 2.1, respectively. There were significant and positive fixed and random effects for both the intercept and slope factors. The interpretation of the significant fixed effect for the slope factor is that there is a large positive increase in reading recognition over the four time points, but the magnitude of increase diminishes as a function of time. Further, the significant random effects indicate that there is substantial individual variability in both the initial starting point and rate of change over time. We thus conclude that this final model best characterizes the observed mean and covariance structure in the four repeated measures of reading recognition. Now that we better understand the characteristics of the developmental process underlying reading recognition, we are ready to combine this univariate latent curve model with the latent curve model estimated for antisocial behavior described earlier. Given that we are also interested in the influences of the four explanatory variables, these will be included as well. We thus estimated a single multivariate latent curve model that included the four repeated measures of antisocial behavior and the underlying growth factors, the four repeated measures of reading recognition and the underlying growth factors, and the four explanatory variables. The intercept and slope factors were allowed to covary within construct, the intercept factors were covaried across construct, the slope factors were co-
Repeated Measures Data
77
varied across construct, each slope factor was regressed upon the intercept factor across construct, and the residual variances of the repeated measures were covaried within each time point. Finally, all four growth factors were regressed upon the four explanatory variables of gender, age, cognitive stimulation and emotional support. We estimated this model and found it t o fit the data very well [x2(35) = 49.8, p = .05; RMSEA = .03, CI,, = 0, .05, p(RMSEA < .05) = .93]. This model is presented in Figure 3.4 in which only significant ( p < .05) regression parameters are shown for sake of clarity. Of initial interest are the relations among the latent growth factors. There was a marginally significant ( p < . l o ) negative correlation between the residual variance of the intercept factor for antisocial behavior and the residual variance of the intercept factor for reading recognition. This suggests that, after estimating the influences of the exogenous variables, children reporting higher levels of initial antisocial behavior also tended t o report lower levels of initial reading recognition. Further, the antisocial behavior intercept factor negatively predicted the reading recognition slope factor, indicating that children reporting higher initial levels of antisociality also tended to report less steep increases in reading over time. However, a similar relation was not found between earlier reading recognition predicting later changes in antisocial behavior. This latter finding is quite interesting given the results of the time-varying covariate model (e.g., Equation 3.10) in which Time 1 reading was found to modestly predict later antisociality. This discrepancy in findings highlights the importance of articulating precisely what type of change is of most interest. The timevarying covariate model indicated that Time 1 reading negatively predicted later time-specific levels of antisociality after controlling for the underlying developmental trajectories of antisociality. In contrast, the final latent curve model indicates that earlier reading recognition is not reliably associated with later developmental trajectories of antisociality. So, the former is examining the relation while controlling for antisocial trajectories, and the latter is examining the relation with the antisocial trajectories. These are not necessarily contradictory findings, but instead are different insights into different questions. Also of interest is the relation between the four explanatory variables and the four growth factors. To summarize, boys were found t o report higher initial levels of antisocial behavior, but not differences in any of the other growth factors. Older children were found t o report higher initial levels and less steep increases in both antisocial behavior and reading recognition. Higher levels of cognitive stimulation were associated with less steep increases in antisocial behavior and, although higher levels of cognitive stimulation were associated with higher levels of initial reading recognition, cognitive stimulation was not associated with rates of change in reading recognition. Finally, higher levels of emotional support were associated with lower initial levels of antisocial behavior and were thus indirectly associated with later changes in reading recognition as mediated
78
Curran and Hussong
Figure 3.4: Final multivariate conditional trajectory model with exogenous variables. All paths between exogenous variables and growth factors were estimated, but only significant ( p < .05) paths are displayed. Values associated with each path are standardized regression coefficients.
Repeated Measures Data
79
through the antisocial intercept. The results of this final model thus allow for a number of interesting insights into the fourth theoretically derived research question described earlier.
CONCLUSIONS The goal of this chapter was to review the basics of latent curve analysis and to present an extended example demonstrating the analysis of individual differences in rates of change over time. The motivating substantive example for these analyses was an examination of the relation between developmental trajectories of antisocial behavior and reading recognition and the relation between these trajectories and child age, gender, emotional support, and cognitive stimulation. Drawing on theories of developmental psychopathology, four research questions of key interest were identified. These questions related to individual and group characteristics of developmental trajectories in antisocial behavior, in reading recognition, between trajectories of antisocial behavior and reading recognition, and between these developmental trajectories and measures of age, gender, emotional support, and cognitive stimulation. We estimated a series of latent curve models that allowed for inferences to be drawn about all four questions of interest. Specifically, both antisocial behavior and reading recognition were found to be characterized by underlying intercept and slope factors, and there were significant group and individual components of growth. Initial starting points of antisocial behavior and reading recognition were negatively related such that children reporting higher initial levels of antisocial behavior also tended to report lower levels of initial reading recognition. Further, although higher initial levels of antisocial behavior were associated with less steep increases in reading recognition, initial reading recognition was not reliably associated with later increases in antisocial behavior. Finally, interesting relations were found between developmental trajectories of both antisocial behavior and reading recognition and the set of four explanatory variables. Taken together, these results provide important information about the characteristics and predictors of developmental trajectories of antisocial behavior and reading recognition over an 8-year period. We believe that the latent curve modeling framework provides a number of potential advantages to the empirical study of repeated measures data over time. First, this approach is much more consistent with many types of theoretical questions of development and change compared to the ARCL model. This is because many developmental questions are inherently interested in the continuous underlying trajectories that vary across individual, not necessarily the time-specific levels of a particular measure. However, the ARCL approach can be advantageous in some settings, and recent work has moved toward integrating elements of both the ARCL and latent curve model into a single modeling framework (e.g., Bollen & Curran, 1999; Curran & Bollen, in press). Second, the latent curve model can easily
80
Curran and Hussong
be extended in a variety of interesting ways using the strength of the SEM framework. For example, multiple indicator latent factors could be used to model measurement error within each time point (e.g., Sayer, in press). Additional variables could be examined as potential mediators between the explanatory variables and the growth factors or between the growth factors themselves (e.g., Muthkn & Curran, 1997). Multiple group analysis could be used to examine the potential moderating influences of discrete measures such as gender or ethnicity (see, e.g., Curran et al., 1997; McArdle, 1989). Indeed, most any advantage offered by the general structural modeling framework is applicable to the latent curve model. Having said that, there are, of course, a number of limitations of the latent curve modeling approach as well, and these should be closely considered prior to adopting this technique in practice. First, as with all growth modeling approaches, latent curve analysis requires a minimum of three time points to estimate individual variability in growth trajectories. These models cannot be used for two time point comparisons. Second, although recent advances in software have allowed for the inclusion of partially missing data, the maximum likelihood estimator used here still assumes multivariate normal distributions, both for the repeated measures data as well as the distributions of the latent growth factors. This poses a problem in many areas of research such as child drug use or psychiatric symptomatology where distributions are expected to be significantly skewed. Third, when predicting the growth factors in latent curve analysis, the intercept and slope components are considered separately from one another, thus making interpretation sometimes difficult. For example, the models presented here discussed predictions of the slope factor, but this did not simultaneously consider the corresponding intercept level. Considering both the height and the rate of change of the trajectory simultaneously may be very important information to consider in many research applications. Finally, it remains difficult to evaluate the latent curve models in multilevel settings. For example, in the models presented here we assume that all of the children are independent from one another. However, there are many situations in which children might be nested within a higher level such as classroom or treatment condition, and this can pose a challenge for latent curve analysis (but see Muthkn & hluthkn, 1998, for new developments in this area). In conclusion, latent curve analysis is one example of a new generation of statistical models that is well suited for evaluating research questions about repeated measures data over time. The latent curve framework is extremely flexible and can be applied across a variety of applied research settings. Recall that all of the previous equations distinguished both i for individual and t for time. Although "individual" referred to children here, this could equivalently denote mouse, treatment group, school, or city. Further, although the increment of "time" referred to 24-month periods here, this could equivalently denote minute, week, or decade. There are thus a remarkably wide number of research settings in which latent curve
Repeated h!Ieasures Data
81
analysis can be applied to examine questions about individual differences in rates of change over time.
REFERENCES Anderson, T. W. (1960). Some stochastic process models for intelligence test scores. In K. J. Arrow, K. Karlin, & P. Suppes (Eds.), Mathematical methods in the social sciences, i 9 5 9 . Stanford, CA: Stanford University Press. Baker, P. C., Keck, C. K., Mott, F. L., & Quinlan, S. V. (1993). Nlsy child handbook: A guide t o the ig86-1990 national longitudinal survey of youth child data. Columbus, OH: Center for Human Resource Research. Bentler, P. M. (1980). Multivariate analysis with latent variables: Causal modeling. A n n u a l Review of Psychology, 31, 419-456. Bentler, P. M. (1983). Some contributions to efficient statistics for structural models: Specification and estimation of moment structure. Psychometrika, 48, 493-517. Bollen, K. A., & Curran, P. J. (1999). An autoregressive latent trajectory (alt) model: A synthesis of two traditions. (Paper presented at the 1999 meeting of the Methodology Section of the American Sociological Association) Browne, M. W., & Cudeck, R. (1993). Testing structural equation models. In K. Bollen & K. Long (Eds.), (p. 136-162). Newbury Park: Sage. Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage. Caspi, A., Bem, D. J., & Elder, G. H. (1989). Continuities and consequences of interactional styles across the life course. Journal of Personality, 57, 375-406. Cicchetti, D. (1984). The emergence of developmental psychopathology. Child Development, 55, 1-7. Coie, J. D., & Jacobs, M. R. (1993). The role of social context in the prevention of conduct disorder. Development and Psychopathology, 5, 263-275. Curran, P. J . (2000). Multivariate applications in substance use research. In J. Rose, L. Chassin, C. Presson, & J. Sherman (Eds.), (p. 1-42). Hillsdale, NJ: Erlbaum.
82
Curran and Hussong
Curran, P. J., & Bollen, K. A. (in press). The best of both worlds: Combining autoregressive and latent curve models. In A. Sayer & L. Collins (Eds.), N e w methods for the analysis of change. Washington, DC: APA. Curran, P. J., & M u t h h , B. 0. (1999). The application of latent curve analysis to testing developmental theories in intervention research. American Journal of C o m m u n i t y Psychology, 27, 567-595. Curran, P. J., M u t h h , B. O., tk Harford, T. C. (1998). The influence of changes in marital status on developmental trajectories of alcohol use in young adults. Journal of Studies o n Alcohol, 59, 647-658. Curran, P. J., Stice, E., & Chassin, L. (1997). The relation between adolescent and peer alcohol use: A longitudinal random coefficients model. Journal of Consulting and Clinical Psychology, 65, 130-140. Dodge, K. A. (1986). A social information processing model of social competence in children. In Minnesota symposium in child psychology (p. 7-125). Hillsdale, NJ: Erlbaum. Dodge, K. A. (1993). The future of research on the treatment of conduct disorder. Development and Psychopathology, 5, 311-319. Duncan, S. C., Duncan, T . E., & Hops, H. (1996). Analysis of longitudinal data within accelerated longitudinal designs. Psychological Methods, 1, 236-248. Duncan, T . E., Duncan, S. C., & Hops, H. (1998). Latent variable modeling of longitudinal and multilevel alcohol use data. Journal of Studies o n Alcohol, 59, 399-408. Dwyer, J . H. (1983). Statistical models for the social and behavioral sciences. New York: Oxford. Elliott, D. S., Huizinga, D., & Ageton, S. S. (1985). Explaining delinquency and drug use. Beverly Hills, CA: Sage. Guttman, L. A. (1954). A new approach to factor analysis. the radex. In P. F. Lazarsfeld (Ed.), Mathematical thinking in the social sciences (p. 258-348). New York: Columbia University Press. Heise, D. R. (1969). Separating reliability and stability in test-retest correlation. American Sociological Review, 34, 93-101. Humphreys, L. G. (1960). Investigations of the simplex. Psychometrika, 25. 313-323. Joreskog, K. G. (1971a). Simultaneous factor analysis in several populations. Psychometrika, 36, 409-426.
Repeated Measures Data
83
Joreskog, K. G. (1971b). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109-133. Joreskog, K. G. (1979). Statistical estimation of structural models in longitudinal developmental investigations. In J . R. Nesselroade & P. B. Baltes (Eds.), Longitudinal research in the s t u d y of behavior and development. New York: Academic Press. Joreskog, K. G., & Sorbom, D. (1978). Advances in f a c t o r analysis and structural equation models. Cambridge, MA: Abt Books. Kazdin, A. E. (1985). Treatment of antisocial behavior in children and adolescents. Homewood, IL: Dorsey. Kazdin, A. E. (1987). Treatment of antisocial behavior in children: Current status and future directions. Psychological Bulletin, 102, 187-203. Kazdin, A. E. (1993). Treatment of conduct disorder: Progress and directions in psychotherapy research. Development and Psychopathology, 5, 277-310. Little, R. J. A., & Rubin, D. B. (1987). Statistical analysis with missing data. New York: John Wiley and Sons. Loeber, R., & Dishion, T. J. (1983). Early predictors of male delinquency: A review. Psychological Bulletin, 94, 68-99. MacCallum, R. C., Kim, C., Malarkey, W., & Kielcolt-Glaser, J. (1997). Studying multivariate change using multilevel models and latent curve models. Multivariate Behavior Research, 32, 215-253. McArdle, J. J. (1986). Latent growth within behavior genetic models. Behavioral Genetics, 16, 163-200. McArdle, J. J. (1988). Dynamic but structural equation modeling of repeated measures data. In J . R. Nesselroade & R. B. Cattell (Eds.), Handbook of multivariate experimental psychology (2nd ed.). New York: Plenum Press. McArdle, J. J. (1989). Structural modeling experiments using multiple growth functions. In P. Ackerman, R. Kanfer, & R. Cudeck (Eds.), Learning and individual differences: Abilities, motivation and methodology (p. 71-117). Hillsdale, N J : Lawrence Erlbaum Associates. McArdle, J. J. (1991). Structural models of developmental theory in psychology. In P. Van Geert & L. P. Mos (Eds.), A n n a l s of theoretical psychology (Vol. VII, p. 139-160). New York: Plenum Press. McArdle, J . J., & Epstein, D. (1987). Latent growth curves within developmental structural equation models. Child Development, 58, 110-133.
84
Curran and Hussong
Meredith, W., & Tisak, J. (1984). ”tuckerizing” curves. (Paper presented at the aiinual meeting of the Psychometric Society) Meredith, W., & Tisak, J. (1990). Latent curve analysis. P s y c h o m e t r i k a , 55, 107-122. Moffitt, T. E. (1990). Juvenile delinquency and attention deficit disorder: Boy’s developmental trajectories from age 3 to age 15. Child D e v e l o p m e n t , 55, 107-122. Muthen, B. (1993). Latent variable modeling of growth with missing data and multilevel data. In C. M. Cuadras & C. R. Rao (Eds.), Multivariate analysis: Future directions 2 (p. 199-210). Amsterdam: North Holland. M u t h h , B. (in press). Second-generation structural equation modeling with a combination of categorical and continuous latent variables: new opportunities for latent class/latent growth modeling. In A. Sayer & L. Collins (Eds.), N e w mothods f o r t h e analysis of change. Washington, DC: APA. Muthkn, B. 0. (1991). Analysis of longitudinal data using latent variable models with varying parameters. In L. C. Collins & J. L. Horn (Eds.), B e s t methods f o r t h e analysis of change (p. 1-17). Washington, DC: American Psychological Association. h I u t h h , B. O., & Curran, P. J. (1997). General longitudinal modeling of individual differences in experimental designs: A latent variable framework for analysis and power estimation. Psychological Methods, 2, 371-402. Muthkn, L. K., & Muthkn, B. 0. (1998). M p l u s user’s guide. Los Angeles: Muthken and M u t h h . Patterson, G. R. (1982). Coercive f a m i l y process. Eugene, OR: Castalia. Patterson, G. R. (1986). Performance models for antisocial boys. A m e r i c a n Psychologist, 41, 432-444. Patterson, G. R., Reid, J. B., & Dishion, T. J. (1992). Antisocial boys. Eugene, OR: Castalia.
Rao, C. R. (1958). Some statistical methods for comparison of growth curves. B i o m e t r i k a , 51, 83-90. Reid, J. B. (1993). Prevention of conduct disorder before and after school entry: Relating interventions to developmental findings. D e v e l o p m e n t and Psychopathology, 5, 243-262.
Repeated Measures D a t a
85
Rogosa, D. R. (1995). Myths and methods: ”myths about longitudinal research” plus supplemental questions. In J. Gottman (Ed.), T h e analysis of change (p. 2-65). Hillsdale, NJ: Lawrence Erlbaum Associates. Rogosa, D. R., & Willett, J. B. (1985). Satisfying simplex structure is simpler than it should be. Journal of Educational Statistics, 10, 99107. Sayer, A. (in press). Multiple indicator latent variables in latent curve analysis. In A. Sayer & L. Collins (Eds.), N e w methods f o r t h e analysis of change. Washington, DC: APA. Stoolmiller, M. (1994). Antisocial behavior, delinquent peer association, and unsupervised wandering for boys: Growth and change from childhood to early adolescence. Multivariate Behavioral Research, 29, 263288. Tucker, L. R. (1958). Determination of parameters of a functional relation by factor analysis. Psychometrika, 23, 19-23. Willett, J. B. (1988). Measuring change: The difference score and beyond. In H. J. Walberg & G. D. Haertel (Eds.), T h e international encyclopedia of educational evaluation. Oxford, England: Pergamon Press. Willett, J. B., & Sayer, A. G. (1994). Using covariance structure analysis to detect correlates and predictors of individual change over time. Psychological Bulletin, 116, 363-381. Wilson, J. Q., & Herrnstein, R. J. (1985). C r i m e and h u m a n nature. New York: Simon and Schuster. Windle, M. (1997). Alternative latent-variable approaches t o modeling change in adolescent alcohol involvement. In K. J. Rryand & M. Windle (Eds.), T h e science of prevention: Methodological advances f r o m alcohol and substance abuse research (p. 43-78). Washington, DC: American Psychological Association.
Chapter 4
Multilevel Modeling of Longitudinal and Functional Data J. 0. Ramsay McGill University I signed on t o write this commentary because I was confused. Thirty-two years of teaching multivariate statistics t o psychology graduate students, and of being a consultant on research projects in a range of disciplines, had not left me with a clear sense of what multilevel analysis was all about. Seemingly related terms in statistics journals, such as hierarchical linear model (HLM), empirical Bayes estimation, random coefficient model, variance component, repeated measures, longitudinal data analysis, latent curve analysis, and even the term multilevel itself seem t o have similar but not completely consistent implications. Moreover, my own research preoccupation is with the modeling of functional data, that is, observations that are curves or images, and I wished that I had the links with these areas of statistics better arranged in my mind. So a read through these three fine papers convinced me t o organize what I thought I knew, learn some new things, and try t o share some of this with readers of this valuable volume. All three papers are solidly data oriented, all three sets of authors were already on my admiration list, and all propose some rather up-market statistical technology for a rather challenging data context. If I can add a little something, either to a reader’s appreciation, or t o future research efforts, it will be a great pleasure. I begin by exploring the term multilevel, which, it appears, means rather different things if applied t o data and research designs than it means if applied t o models and parameter spaces. Because for me data always have precedence over models and mathematics, I’m happy t o agree with the 87
88
Ramsay
authors in their use of the term. Moreover, more of my focus in what follows will be on data rather than model characteristics, hence, first two multilevel data sections, one on the general situation, and one focused on longitudinal data. There follows two multilevel model sections, again offering a general multilevel framework, followed by specifics for the data of focus. Then we return to consider features of longitudinal data, an important special type of multilevel data. The chapter concludes with a review of some modeling strategies along with an assessment of what these three papers have achieved. The references that have been especially helpful to me are annotated at the end. To save space, I hope that neither the readers nor the authors will object if I refer to Curran and Hussong; Kenny, Bolger, and Kashy; and Raudenbush as CHI KBK, and R, respectively.
MULTILEVEL DATA An observation, Yij,is organized on two levels. The lower or most basic level is associated with subscript j = 1,... , n i l and corresponds to the most elementary unit of data that we wish to describe. For CH and R, who consider longitudinal data, this is the value of a variable at the year indexed by j . For KBK, on the other hand, this basic unit is the partner with whom a person interacts. On the other hand, lower level units may, themselves, be collections of variable values. For example, we may refer to an individual’s entire longitudinal record as vector yij, so that the individual is the lowerlevel unit. However, observations are also organized by a higher level index i = 1,. . . , m, and this refers to one of m categories within which lower units fall, or within which they are nested. For the National Longitudinal Survey of Youth analyzed by CH, the National Youth Survey analyzed by R, and most longitudinal data, the upper level unit is the individual who is measured at ni time points. The upper level for KBK corresponds to the m = 77 persons, each of whom interacts with a number of partners. The number of lower basic units ni can vary over upper level index i. Upper level units are considered either random or fixed, with random units being categories that will normally change if the data collection process is replicated (e.g., subjects or schools), whereas fixed units are those such as treatment groups that will be reused over replications. Third and higher-level types of classification of units are often needed, and R, for example, considers schools to be a third level for the Sustaining Effects Data, where students are second- level units, and time points are at the lowest level. This necessitates another subscript, so that the observation in this case would be Yijk. Covariates are often a part of multilevel data designs, and these may be indicated by the vectors xi and zij, having p and qi elements, respectively, and with the number and nature of subscripts indicating whether these
Longitudinal and Functional Data
89
covariates are associated with upper or lower levels of data organization. We shall use xi to refer to covariates for upper-level units and zij for lowerlevel covariates. For example, R’s models for the National Youth Survey data use as covariates a constant level, coded by 1, and age, aij (he exchanges the roles of i and j in his notation). This means that the covariate vector zij = (1,a i j ) t . In fact, as we see, the same covariate vector is used for the upperlevel units (individuals), so that a set of covariates may appear twice in the same model, once for modeling lower-level effects and once for modeling upper-level effects. Similarly, CH propose the same multilevel model in their Equations 4.5 to 4.7, with the covariates being, in their notation, 1 and t , (their t corresponding to my j ) . KBH use the gender and physical attractiveness of the partner as lower-level covariates and subject gender as the upper level covariate.
SPECIAL FEATURES OF LONGITUDINAL DATA Repeated measures, longitudinal data, and times series are all terms with a long history in statistics referring to measurements taken over time within a sampling unit. Add to these the term functional data, used by Ramsay and Dalzell (1991) to describe data consisting of samples of curves. Each curve or functional datum is defined by a set of discrete observations of a smooth underlying function, where smooth usually means being differentiable a specified number of times. A comparison of Analysis of Longitudinal Data by Diggle et al. (1994) with the more recent Functional Data Analysis by Ramsay and Silverman (1997) highlights the strong emphasis that the latter places on using derivative information in various ways. The more classic repeated measures data are typically very short time series as compared with longitudinal or functional data, and time series data are usually much longer. Most time series analysis is directed at the analysis of a single curve and is based on the further assumptions that the times at which the process is observed are equally spaced and that the covariance structure for the series has the property of stationarity, or time invariance.
The Missing Data or Attrition Problem Longitudinal or functional data are expensive and difficult to collect if the time scale is months or years and the sampling units are people. Attrition due to moving away, noncompliance, illness, death, and other events can quickly reduce a large initial sample down to comparatively modest proportions. Much more information is often available about early portions of the process, and the amount of data defining correlations between early and late times can be much reduced. Although there are techniques for plugging holes in data by estimating missing values, a few of which are mentioned by CH and R, these methods
90
Ramsay
implicitly assume that the causes of missing data are not related to the size or location of the missing data. This is clearly not applicable to measurements missing by attrition in longitudinal data. Indeed, the probability of a datum being missing can be often related to the features of the curve; one can imagine that a subject with a rapidly increasing antisocial score in CH’s data over early measurements is more likely t o have missing later measurements. I am proposing, therefore, to use the term missing data to refer to data missing for unrelated reasons and the term attrition to indicate data missing because of time- or curve-related factors. The data analyst is therefore faced with two choices, both problematical to some extent. If only complete data records are used, then the sample size may be too limited to support sophisticated analyses such as multilevel analysis. Moreover, subjects with complete data are, like people who respond to questionnaires, often not typical of the larger population being sampled. Thus, an analysis of complete data, or a balanced design, must reconcile itself, like studies of university undergraduates, with being only suggestive by a sort of extrapolation of what holds in a more diverse sampling context. Alternatively, one can use methods that use all of the available data. The computational overhead can be much greater for unbalanced designs, and in fact the easily accessible software packages tend to work only with balanced data. In any case, as already noted, results for later times will tend to both have larger sampling variance, due to lack of data, and be biased if attrition processes are related to curve characteristics. Thus, although using ”missing-at-random” hole-filling algorithms, such as developed by Jennrich and Schluchter (1986) and Little and Rubin (1987), may make the design balanced in terms of the requirements of software tools in packages such as SAS and BMDP, the results of these analyses may have substantial biases, especially regarding later portions of the curves, and this practice is risky. I must say that my own tendency would be, lacking appropriate full data analysis facilities, to opt to live with the bias problems of the complete data subsample rather than those that attrition and data substitution are all too likely to bring.
Regist rat ion Problem Figure 4.1 displays curves showing the acceleration in growth for ten boys, and we see there that there are two types of variation happening. Phase variation occurs when a curve feature, such as the pubertal growth spurt, occurs a t different times for different individuals, whereas the more familiar amplitude variation occurs when two units show differences in the characteristics of a feature even though their timings may be similar. We can see that averaging these curves results in a mean curve that does not resemble any of the actual curves; this is because the presence of substantial phase variation causes the average to be smoother than actual curves. In effect, phase variation arises because there are two systems of time
91
n
10-
-15;
8
10
12
14
16
18
Years
Figure 4.1: Acceleration curves for the growth of 10 boys showing a mixture of phase and amplitude variation. The heavy dashed line is the mean curve, and it is a poor summary of curve shapes.
involved here. Aside from clock time, there is, for each child or subject, a biological, maturational or system time. In clock time, different children hit puberty at different times, but, in system time, puberty is an event that defines a distinct point in every child's growth process, and two children entering puberty can reasonably be considered at equivalent or identical system times, even though their clock ages may be rather different. From this perspective, we can imagine a possibly nonlinear transformation of clock time t for subject i, which we can denote as hi(t),such that, if two children arrive at puberty at ages tl and t2, then hl(t1) = hz(t2). These transformations must, of course, be strictly increasing. They can be referred t o as time warping functions. If we can estimate for each growth curve a suitable warping function h such that salient curve features or event are coincident with respect t o warped time, then we say that the curves have been registered. In effect, phase variation has been removed from the registered curves, and what remains is purely amplitude variation. The first panel of Figure 4.2 displays the warping functions hi(t) that register the 10 acceleration curves in Figure 4.1. The second panel of Figure 4.2 shows the registered acceleration curves in Figure 4.1. The multileveling modeling methods considered by these three authors, and in most applications, concern themselves solely with amplitude variation. However, it seems clear that children evolve in almost any measurable respect at different rates and at rates that vary from time t o time within individuals. The potential for registration methods t o clarify and enhance multilevel analyses seems important. Recent references on curve registration are Ramsay and Li (1998) and Ramsay and Dalzell (1995).
Ramsay
92 18
15
ir^
2
2 16 (d $14
10
$0 5 v
-g
-
12
.5
0
2
!!
10
a, a,
0 -5
8 -10 a
t - 8
-
66
8
10 12 14 16 18 Years
156
8 10 12 14 16 18 Years
Figure 4.2: The left panel contains the warping functions for registering the growth acceleration curves in Figure 5.1, and the right panel contains the registered curves.
Serial Correlation Problem Successive longitudinal observations usually display some amount of correlation, even after model effects are removed. Typically, this correlation between successive residuals decays fairly rapidly as the time values become more widely separated. This serial correlation is obvious in CH’s Table 3.1, for example, and we know from studies of human growth that heights measured 6 months apart have correlations between residuals of about minus 0.4. This may be due t o ”catch-up” processes that ensure that slow growth is followed by more intense growth episodes. Negative serial correlation causes actual observations t o tend to oscillate rapidly around a smooth curve that captures the longer-scale effects, whereas positive serial correlation results in slow smooth oscillations. Modeling these serial effects requires introducing some structure in the covariances among successive errors eij. Diggle et al. (1994) have an excellent discussion of these effects, as well as modeling strategies, and all three papers mention the problem. We return t o this issue in the modeling sections.
Resolution of Longitudinal Data The amount of information about a curve that is available in a set of ni measurements is not well captured by ni itself. Let us consider other ways of thinking about this. An event in a curve is a feature or characteristic that is defined by one or more measurable characteristics. 1. Levels are heights of curves and are defined by a minimum of one observation. The antisocial score of a child at time 1 in CH’s data is a level.
Longitudinal and Functional Data
93
2. Slopes are rates of change and require at least two observations to define. Both CH and R are concerned with slope estimation. 3. B u m p s are features having (a) the amplitude of a maximum or minimum, (b) the width of the bump, and (c) its location; they therefore are three-parameter features. We may define the resolution of a curve as the smallest size on the time scale of the features that we wish to consider. Discrete observations are usually subject to a certain amount of noise or error variation. Although three error-free observations are sufficient, if spaced appropriately, to define a bump, even the small error level present in height measurements, which have a signal-to-noise ratio of about 150, means that five observations are required to accurately assess a bump, corresponding to twice- yearly measurements of height. More noise than this would require seven measurements, and the error levels typical in many social science variables will imply the need for even more. This is why R says that slope inferences cannot be made with only two observations. We may, therefore, define the resolution of the data to be the resolution of the curve that is well-identified by the data. The five observations per individual considered by CH are barely sufficient to define quadratic trend, that is, a bump, and confining inferences to slopes for these data is probably safer, especially given their high attrition level. However, the data resolution is not always so low, and many data collections methods under development in psychology and other disciplines permit the measurement of subjective states over time scales of hours and days rather than weeks and months. Brown and Moskowitz (1998) offer an interesting example of higher-resolution emotional state data. Equipment for monitoring physio!ogical or physical data are now readily available with time scales of seconds or milliseconds and with impressive signal-to-noise ratios. The optical motion tracking equipment that is used in studies such as Ramsay, Heckman, and Silverman (1997) has sampling rates of up to 1,200 Hz and records positions accurate to within a half a millimeter.
MULTILEVEL MODELS Notation Here we use notation that has become fairly standard in statistics texts and journals, such as Diggle et al. (1994) and Searle, Casella, and McCulloch (1992) as well as for multilevel analysis in other sciences. We shall use and ui to indicate regression coefficient vectors to be applied to covariate vectors xi and zij, respectively, so that
Ramsay
94
This model can be expressed more cleanly in matrix notation, so we will need to use the vector notation y to indicate a vector containing all of the lower-level observations. The length of this vector is N = n n i . Within this vector, the lower level index j will vary inside the upper index i. At the same time, we need to gather the covariate vectors xi and zij into matrices X and Z , respectively. These two matrices will both have N rows. Here is what these matrices look like for R’s rather typical longitudinal data model specified in Equations 2.1 and 2 . 2 , which, when these equations are combined, and we exchange the roles of i and j , is
Kj = Po0 + Ploaij + U O +~ uliaij + rij First, let matrix Ti have ni rows and 2 columns; and in column 1 place l’s, and in column 2 place the centered age values a i j . Then the matrix X has 2 columns, and contains
and matrix Z contains 2 m columns and is 0
Ti 0
T2
...
.. .
I
0
10
I
I
... ... ... ...
0 0 I
. ..
IT,
Corresponding to matrix Z, we also define vector u as containing, one after another, the individual covariate vectors ui = (uoi,U I ~ ) ~ .
Upper Model Level The behavior of observations yZj given fixed values of regression coefficients of the residual or error term e i j . In the most commonly used multilevel model, this error distribution is defined to be normal with mean 0. In some applications, it is assumed that these errors are independently distributed for different pairs of subscripts, and that they have variance c2.However, for longitudinal data where the lower level of data is an observation of an outcome variable at a specific time, this may be too simple. In this setting, therefore, we specify the covariances among errors by gathering the errors eij into a large vector e, just as we did for the observations yZj , and declare in the upper level model that e is normally distributed with mean vector 0 and with variance-covariance matrix of R order N . The upper level model may also be expressed as, conditional on p and u,
P and ui is determined by the behavior
95
Longitudinal and Functional Data
or, in other words, given fixed known values of p and u, y has a normal distribution with mean Xp Zu and variance-covariance matrix R. Thus, the upper level model for statisticians corresponds to the lowest level of the data. Perhaps this indicates that they see themselves as reporting up to mathematicians rather than down t o we scientists on the ground.
+
Lower Model Level One of the paradigm shifts in statistics has been the gradual acceptance by statisticians of a Bayesian perspective. The process has been fraught with controversy, and there is no lack of statisticians who would not like t o be thought of as Bayesians, but the concept that parameters, too, have random distributions has caught on. According to the Bayesian view, we must now specify the random behavior of our regression coefficients /3 and U.
Our lower model level assumptions are that (i) p and u are independently distributed, (ii) p is normally distributed with mean PO and variancecovariance matrix B , and (iii) u is normally distributed with mean 0 and variance-covariance matrix D. These assumptions imply that the observation vector y is unconditionally normally distributed with mean XP and variance-covariance matrix
v a r ( y ) = XBXt
+ ZDZt + R
Upper level (in the model sense) parameters p and 2~ are often called effects, lower model level matrices B, D, and R are called variance components, and coefficient vector PO can be called a hyperparameter.
Variance Components Notice that R is N by N , and therefore too large to estimate from the data without strong restrictions on its structure. It is often assumed to be diagonal and of the form a21. However, special alternative structures may be required for longitudinal data, and we consider these in the next section. However, D may also be large, and is of order 2m in the models used by CH and R. Because m may be large, special structural assumptions are required to reduce the number of variances and covariances to be estimated from the data. If we can assume that between-subject covariances are 0, then D has a block-diagonal structure, with the same variance-covariance submatrix, denoted here by C , in each diagonal block, that is, D has the structure
Ramsay
96
This simplifies things considerably because we now only have to estimate q(q 1 ) / 2 variance components. For CH’s example, q = 2, and C is specified in their Equation 3.1. However, sometimes this independence between upper data level units is not sensible; schools as an upper level, for example, which are geographically close, will share many features in common, and their effects should be considered to be correlated. Parameter independence assumption i is especially important and is perhaps the central structural characteristic of the multilevel model. It asserts that lower level variation in the data sense is unrelated to higher level variation. There are certainly situations where this might be challenged. For example, we might suppose that where lower level units are students and upper level units are schools, certain types of students will respond differently to being placed in certain types of schools, such as parochial, than they will in other types, such as public schools. In fact the assumption is critical to methods for fitting the model. Assumptions ii and iii indicate, by their specifications of the mean vectors for p and u, that the general or upper-level mean structure is determined by hyperparameters 0 , and that lower-level unit means are centered on this mean structure.
+
Effects as Fixed or Determined by Further Covariates A specialization of this model is often assumed; the upper data level parameter is declared to be a fixed parameter, which corresponds to the assumption that B-1 -+ 0. This case is referred to as the mixed multilevel model. In this case, the variance-covariance matrix for y simplifies to UUT(Y)
= ZDZ’
+ RandP =
On the other hand, because we can think of ,/3 as a random variable, we can propose a linear model for its variation. This is what CH do in their Equations 3.8 and 3.9. Consequently, let W be a p by r matrix of covariates for modeling p, and let a be a vector of T regression coefficients for the model /3 = W a e * . Then we have
+
wur(y) = XWBWtXt
+ ZDZt + R
where B is now the variance-covariance matrix for a. Actually, this does not really require a modification of our multilevel model notation because the net effect is simply to replace X by XW, and /3 by a , and this is therefore formally the same model.
Multilevel Modeling Objectives The multilevel model does three things. First, it specifies a relatively specific covariance structure for observation vector y. The smaller the numbers of columns for X and Z, and/or the more specialized their structure, the
Longitudinal and Functional Data
97
more restrictive this model is. The papers of CH and R offer examples of how a simple initial covariance structure specification can be incrementally extended, with a test after each extension indicating whether the fit to the observed variance covariance matrix has significantly improved. Moreover, as we shall see in discussing longitudinal data, there is the potential in most multilevel software to further specialize the structure of the two variance-covariance matrices D and R. Secondly the multilevel model is especially powerful for estimating lowerlevel data effects defined by the q / s . It is often the case that there are few measurements within upper-level units to define these effects, or in extreme cases insufficient data to define them uniquely, but the model permits each unit t o borrow information from other units to supplement sparse or missing information. The result is much like increasing the sample size for each unit. This borrowing of strength depends, first, on the data within a unit being relatively weak, either because of the limited number of observations available or because of the high noise level in measurements, and second on the degree of uniformity of variation across upper level units that is, the extent to which variance component matrix D is small relative to error component R. Finally, upper model level effects captured in /3 are often the main interest and in this multilevel analysis shares goals with ordinary multiple regression. Indeed, /3 may be captured by averaging across equivalent lower level effects within the ui1.s. This is what KBK aim at in their use of ordinary and weighted least squares methods, and they are quite right to point out that, relative to this task, there may be little achieved by going to the more demanding multilevel technology. How important is it to get the variance-covariance structure right, if all we need are upper-level fixed effect estimates? Diggle et al. (1994) suggest that in many situations a simple variance components model, such as that in classical repeated measures ANOVA, may be preferable from the perspective of effect estimation, even when the data confirm a more complex model. I tend to agree; when the sample size is modest, it’s more important to conserve degrees of freedom for error by economizing on the model than pulling out all the stops to make bias small. However, in social science applications, one has the impression that the primary focus is on estimating variance components, so that the multilevel model belongs squarely in the long tradition of such models, which includes principal components, factor, and canonical correlation analysis as well as structural equation modeling. Both CH and R model how well the fit to the data improves when both effects and variance components are added to an existing model.
98
Ranisay
SPECIAL FEATURES OF LONGITUDINAL MODELS AND FUNCTIONAL DATA Let us now have a look at modeling longitudinal data, with the possibility that the number of observations ni may be rather larger than for the data sets considered by CH and R. These remarks are selected from material in Ramsay and Silverman (1997).
Within-Subject or Curve Modeling There are two general approaches t o modeling curves f
1. Parametric: A family of nonlinear curves defined by a specific model is chosen. For example, test theorists are fond of the three-parameter logistic model, P ( 0 ) = c + (1- c)/{l + ezp[-a(0 - b ) ] } , for modeling the relationship between the probability of getting a test item correct as a function of ability level. 2 . Nonparametric: A set of K standard fixed functions & , called basis functions, are chosen, and the curve is expressed in the form
The parametric approach is fine if one has a good reason for proposing a curve model of a specific form, but this classic approach obviously lacks the flexibility that is often called for when good data are available. The so-called nonparametric approach can be much more flexible given the right choice of basis functions and a sufficient number of them, although it is, of course, also parametric in the sense that the coefficients k must be estimated from the data. However, t,he specific functional form of the basis functions tends not to change from one application t o another. However, this approach is linear in the parameters and therefore perfect for multilevel analysis. Here are some common choices of basis functions: 1. Polynomials: Q,(t) = t”’. These well-known basis functions (or equivalent bases such as centered monomials, orthogonal polynomials, and so on) are great for curves with only one or two events, as defined previously. A quadratic function, for example, with K = 3, is fine for modeling a one-bump curve with 3 to 7 or more data points. These bases are not much good for describing more complex curves, though, and have been more or less superseded by the B-spline basis described subsequently. 2 . Polygons: 0, = 1 if j = k, and 0 otherwise. For this simple basis, coefficient a k is simply the height of the curve at time t k , and there is a basis function for each time at which a measurement is taken. Thus, there is no data compression; and there are as many parameters to estimate as observations.
Longitudinal and Functional Data
99
3. B-splines: This is now considered the basis of choice for nonperiodic data with sufficient resolution to define several events. A B-spline consists of polynomials of a specified degree joined together at junction points called knots, and is required to join smoothly in the sense that a specified set of derivatives are required to match. The B-spline basis from which these curves are constructed has the great advantage of being local, that is, nonzero only over a small number of adjacent intervals. This brings important advantages at both the modeling and computational levels. 4. Fourier series: No list of bases can omit these classics. However, they are more appropriate for periodic data, where the period is known, than for unconstrained curves of the kind considered by CH and R. They are, too, in a sense, local, but in the frequency domain rather in the time domain.
5 . Wavelets: These basis functions, which are the subject of a great deal of research in statistics, engineering and pure mathematics, are local simultaneously in both the time and frequency domain and are especially valuable if the curves have sharp local features at unpredictable points in time. Which basis system to choose depends on (a) whether the process is periodic or not and (b) how many events or features the curve is assumed to have that are resolvable by the data at hand. The trick is to keep the total number K of basis functions as small as possible while still being able to fit events of interest. Basis functions can be chosen so that they separate into two orthogonal groups, one group measuring the low-frequency or smooth part of the curve and the other part measuring the high-frequency and presumably more variable component of within-curve variation. This opens up the possibility of a within-curve separation of levels of model. The large literature on smoothing splines follows this direction, and essentially uses multilevel analysis, but refers to this as using regularization.
Variance Components We have here the three variance components: (a) R containing the withincurve variances and covariances, (b) D containing the second level variances and covariances, and (c) B containing the variances and covariances among the whole-sample parameters a. The behavioral science applications that I’ve encountered tend to regard components in R as of little direct interest. For nonlongitudinal data, it is usual to use R = ~ ’ 1 and, , as Diggle et al. (1994) suggest, this may even be a wise choice for longitudinal data because least squares estimation is known to be insensitive to mis-specification of the data’s variance-covariance structure. However, in any case, the counsel tends to be to keep this part of
100
ltamsay
the model as simple as possible, perhaps using a single first-order autocorrelation model if this seems needed. More elaborate serial correlation models can burn up precious degrees of freedom very quickly and are poorly specified unless there are large numbers of observations per curve. It is the structure of D that is often the focus of modeling strategies. If, as is usual, between-subject covariances are assumed to be 0, then we noted previously that D is block-diagonal and the m submatrices C in the diagonal will be of order K and all equal. Both CH and R consider saturated models in which the entire diagonal submatrix is estimated from the data. Of course, as the number of basis functions K increases, the number of variances and covariances, namely K ( K 1)/2, increases rapidly. The great virtue of local basis systems, such as B-splines, Fourier series, and wavelets, is that all covariances sufficiently far away from the diagonal of this submatrix can be taken as 0, so that the submatrix is band-structured. The PROC MIXED program in the SAS (1995) enables a number of specialized variance-covariance structures. The contents of B will be of interest only if the group parameters in p are considered random. This seems not often to be the case for behavioral science applications.
+
CONCLUSIONS Multilevel analysis requires a substantial investment in statistical technology. Programs such as SAS PROC MIXED can be difficult to use because of unfriendly documentation, poor software design, and options not well adapted to the structure of the data at hand. Although this is not the place to comment on algorithms used to fit multilevel models, these are far from being bulletproof, and failures to converge, extremely long computation times, and even solutions that are in some respect absurd are real hazards. Indeed, the same may be said for most variance components estimation problems, and there is a large and venerable literature on computational strategies in this field. The best known variance components model for behavioral scientists is probably factor analysis, where computational problems and the sample sizes required for stable estimates of unique variances have led most users with moderate sample sizes to opt for the simpler and more reliable principal components analysis. The emphasis that CII and R place on testing the fit of the various extensions of their basic models might persuade one that fitting the data better is always a good thing. It is, of course, if what is added t o the model to improve the fit is of scientific or practical interest. In the case of multilevel models, what are added are the random components and whatever parameters are used to define the variance-covariance structures in D and
R. On the other hand, adding fitting elements may not be wise if the effects of interest can be reasonably well estimated with simpler methods and fewer
Longitudinal and Functional Data
101
parameters. For example, if there is little variability in the curve shapes across individuals, then adding random coefficients burns up precious degrees of freedom for error. The resulting instability of fixed effect parameter estimates, and loss of power in hypothesis tests for these effects, will more than offset the modest decrease in bias achieved relative to that for a simple fixed effects model. As for variance components, is worth saying again along with Diggle et al. (1994) and others that modeling the covariance structure in D and R will usually not result in any big improvement in the estimation and testing of the fixed effect vector /3. KBK have added an important message of caution in this regard. If you are specifically interested in the structure of D and/or R, and have the sample size t o support the investigation, then multilevel analysis is definitely for you. If you want to augment sparse or noisy data for a single individual by borrowing information from other individuals, so that you can make better predictions for that person, then this technique has much to offer. However, if it is the fixed effects in /3 that are your focus, then I wish we knew more about when multilevel analysis really pays off. If you have longitudinal data, it might be worth giving some consideration t o the new analyses such as curve registration and differential models emerging in functional data analysis. To sum up, which method t o use very much depends on your objectives.
REFERENCES Brown, K. W., & Moskowitz, D. S. (1998). Dynamic stability of behavior: The rhythms of our interpersonal lives. Journal of Personality, 66, 105-134. Diggle, P. J., Liang, K.-Y., & Zeger, S. L. (1994). Analysis of longitudinal data. Oxford: Clarendon Press. Jennrich, R., & Schluchter, M. (1986). Unbalanced repeated-measures model with structured covariance matrices. Biornetrics, 42, 809-820. Little, R. J. A., & Rubin, D. B. (1987). Statistical analysis with missing data. New York: John Wiley and Sons. Ramsay, J. O., & Dalzell, C. (1991). Some tools for functional data analysis (with discussion). Journal of the Royal Statistical Society, Series B, 53, 539-572. Ramsay, J. O., & Dalzell, C. (1995). Incorporating parametric effects into functional principal components analysis. Journal of the Royal Statistical Society, Series B, 57, 673-689. Ramsay, J. O., Heckman, N., & Silverman, B. (1997). Spline smoothing with model-based penalties. Behavior Research Methods, 29(1), 99106.
102
Ramsay
Ramsay, J. O., & Li, X. (1998). Curve registration. Journal of the Royal Statistical Society, Series B, 60, 351-363. Ramsay, J. O., & Silverman, B. W. (1997). Functional data analysis. New York: Springer.
SAS Institute. (1995). Introduction t o the M I X E D procedure course notes (Tech. Rep.). Cary, NC: SAS Institute Inc. Searle, S.R., Casella, G., & McCulloch, C. E. (1992). Variance components. New York: Wiley.
Chapter 5
Analysis of Repeated Measures Designs with Linear Mixed Models Dennis Wallace University of Kansas Medical Center
Samuel B. Green Arizona State University Repeated measures or longitudinal studies are broadly defined as studies for which a response variable is observed on a t least two occasions for each experimental unit. These studies generate serial measurements indexed by some naturally scaled characteristic such as age of experimental unit, length of time in study, calendar date, trial number, student’s school grade, individual’s height as a measure of physical growth, or cumulative exposure of a person to a risk factor. The experimental units may be people, white rats, classrooms, school districts, or any other entity appropriate to the question that a researcher wishes to study. To simplify language, we will use the term ”subject” t o reference the experimental units under study and the term ”time” to reference the longitudinal indexing characteristic. In most general terms, the objectives of repeated measures studies are to characterize patterns of the subjects’ response over time and to define relationships between these patterns and covariates (Ware, 1985). Repeated measures studies may focus simply on change in a response variable for subjects over time. Examples include examining changes in satisfaction with health as a function of time since coronary bypass surgery, assessing the pattern of change in self concept for adolescents as a function of height, and characterizing the growth of mathematical ability in elemen-
103
104
Wallace and Green
tary school children as a function of grade level. Studies may incorporate simple, non-time-varying covariates that differentiate subjects into two or more subpopulations. The primary emphasis in these studies is on characterizing change over time for each subpopulation and assessing whether the patterns of change differ across subpopulations. For example, an objective of a study might be to assess whether the growth rate of emotional maturity across grade levels differs between male and female students. Besides having an interest in differences between average growth rates for subpopulations, we could also be interested in the differences between variability in growth rates for these subpopulations. The non-time-varying covariate could also be a continuous variable, such as family income at beginning of study, and the focus could be on growth rate as a function of income. Alternatively, a repeated measures design might include a covariate that varies over time. For example, a study examining weekly ratings by kneesurgery patients of their health satisfaction might include weekly ratings by physicians of the inflamation level of the patients’ knees. Regardless of whether a study includes non-time-varying covariates and/or time-varying covariates, we may choose not to include time as a variable in our analysis but to have time act simply as an index for defining different observations. One example might be a study in which we explore the relationship between body image and sexual self-confidence over time. We could ignore time as a predictor in the analysis and investigate the hypothesis that the slope coefficients in predicting sexual self-confidence from body image vary more greatly for obese men than for normal weight men. As suggested by the previous research questions, we may want to use repeated measures designs to answer questions about variability in behavior as well as average behavior. The linear mixed model allows us to model both means and variances so that we can investigate questions about both of these components. In addition, because the linear mixed model allows accurate depiction of the within-subject correlation inherently associated with the repeated measurements, we can make better statistical infcrences about the mean performances of subjects. This chapter introduces the linear mixed model as a tool for analyzing repeated measures designs. Basic terminology for repeated measures studies (Helms, 1992) is introduced to simplify the subsequent discussion. A repeated measures study has a regularly timed schedule if measurements are scheduled at uniform time intervals on each subject; it has regularly timed data if measurements are actually obtained at uniform time intervals on each subject. Note that the uniform intervals can be unique for each subject, but the time intervals between observations are constant for any given subject. A repeated measures study has a consistently timed schedule if all subjects are scheduled to be measured at the same times (regardless of whether the intervals between times are constant) and has consistently timed data if the measurements for all subjects are actually made at the same times. If studies are designed with a regular and consistent schedule and data are actually collected regularly and consistently, analyses can be accomplished relatively easily with
Mixed Models
105
general linear models techniques. Although many experimental studies are designed t o have regularly and consistently timed data, they often yield mistimed data or data with missing observations for some subjects. In nonexperimental, observational studies, data are rarely designed and collected in this manner. Consequently, we need flexible, analytic methods for analyzing data that are not regularly and consistently timed, and the linear mixed model meets this need. Although many articles have been published throughout the history of statistics discussing methods for the analysis of repeated measures data, the literature has expanded rapidly in the past 15 years, as evidenced by the publications of Goldstein (1987), Lindsey (1993), Diggle et al. (1994), Bryk and Raudenbush (1992), and Littell, Milliken, Stroup, and Wolfinger (1996). Some authors have focused on the analysis of repeated measures data from studies with rigorous experimental designs and discuss analysis of variance ( ANOVA) techniques using univariate and multivariate procedures. Procedures using an ANOVA approach are described in detail by Lindquist (1953), Winer (1971), Kirk (1982a, 1982b), and Milliken and Johnson (1992). Although these methods work well for some experimental studies, they encounter difficulties if studies do not have regularly and consistently timed data. Consequently, researchers faced with observational data have approached repeated measures data through the use of regression models, including hierarchical linear models (HLM) and random coefficients models. The discussions of HLM by Bryk and Raudenbush (1992) and of random coefficients models by Longford (1993) and Diggle et al. (1994) take this approach. As we illustrate in this chapter, linear mixed models provide a general approach t o modeling repeated measures data that encompasses both the ANOVA and HLM/random-coefficients approaches. The remaining sections of the chapter outline how the mixed model can be used to analyze data from repeated measures studies. We first introduce the linear mixed model and discuss it in the context of a simple example. Next we describe methods for estimating model parameters and conducting statistical inference in the mixed model. We also briefly introduce how t o implement these methods with the SAS procedure MIXED. Then we outline some of the issues that must be considered in constructing models arid discuss strategies for building models. In the last section, we present an example analysis illustrating the use of the MIXED procedure and the strategies for building models.
THE MIXED MODEL We describe the linear mixed model in the context of most repeated measures applications and ignore some of its most general features. With the linear mixed model, we predict y from the fixed component Xp and the random components Z u and error e :
Wallace and Green
106
y = Xp
+ Zu+ e
y is an N x 1 vector containing response variable scores y i j i at time j:
Y‘ = [YII 3/12
-.. 3/iivl
3/21 3/22
.- -
YZN~
---
3/11
---
(5.1) for subject
YIN~]
(5.2)
where the number of times that subject i is observed is Ni, the number of subjects is I , and the total number of observations is, therefore, N = g = I Ni. The first term on the right side of the equation, XP, is defined in the same way as it is in the general linear model. X is an N x q design matrix for the fixed component, containing the N scores for the q predictors assumed to be measured without error, whereas p is the q x 1 vector of unknown population parameters, P o , ...pp-l,. The ps are fixed-effects parameters for predicting subjects’ responses in general or, in other words, parameters that describe population mean behavior. The population parameters in p may inaccurately depict the relationship between the response variable scores, yij, and the predictors for some subjects. The random-effects component Zu allows us to define different relationships among subjects. Zi is an Ni x r design matrix for an individual subject and includes r predictors. For most models, the r predictors are a subset of the q predictors in the X design matrix, and the subset of predictors is the same for all subjects. Z is a block diagonal N x I r design matrix for the random component, with the matrices Z I , Z Z , ..., ZI down its diagonal. The vector ui, where ui[uoiuli...u(,-l)i] is an r x 1 vector of unknowns associated with Z i . The unknowns in ui are subject-specific random effects and represent the differences in coefficients for the T predictors for subject i and the s for the same predictors. The vectors u1, u ~ , ..., UI are stacked within u, an I r x 1 vector containing the subject-specific random effects for all subjects. Finally, the random-effects component e is a N x 1 vector containing the within-subject random errors, e i j :
...
e’ = [e11e12 e1N1e21e22...ezN 2...ell...elN1]
(5.3) The distributional assumptions for the model are that the residual coefficients in u are normally distributed with a mean of 0 and variance of G. G is an I r x I r block diagonal matrix, with each block, G;, containing the between-subject variances and covariances among the residual coefficients. As characterized by the block diagonality of G, vectors for any two subjects ui and ui, are assumed to be independent. Also the errors in e are independent of the residuals in u and are normally and independently distributed with a mean of 0 and covariance matrix R. R is N x N block diagonal matrix, with each block, Ri, containing the variances and covariances of the within-subject errors. Typically, we assume that Gi and Ri are homogeneous across subjects; however, the mixed model is flexible and permits
Mixed Models
107
Table 5.1 Data for a Simple Longitudinal Study
Time 1
Time 2
Time 3
Time 4
(0 months)
(3 months)
(6 months)
(9 months)
Subject 1
2
3
7
9
Subject 2
5
10
18
20
Subject 3
3
4
4
5
different covariance matrices for the between-subjects random effects (Gi,) or for the within-subject errors (Ri,) for different treatment groups. The variances and covariances among yij around their expected value Xp is V, where
V = ZGZr
+R
(5.4)
The parameters of the model and, consequently, the parameters that must be estimated are the fixed effects ( p ) , the random-effect variances and covariances ( G ) , and the error variances and covariances (R). The random effects (u) are not model parameters and often are not of interest to researchers because they are specific t o particular subjects (although the random effects can be predicted and the predictions could be included in hypotheses and tested).
A Simple Example To illustrate the mixed model, we consider a very simple example in which infants are assessed in their perceptual skills at 0, 3, 6, and 9 months of age, and we are interested in their rate of growth over this time period. The data are presented in Table 5.1. In developing a model, we assume that perceptual skills develop linearly across time, but that this relationship might vary from person t o person. First we substitute into the prediction equation for the mixed model, y = Xp Z u e , the appropriate values:
+
+
108
Wallace and Green Prediction Line
251
j01 Hxed -Effecls
..el 4-E,,
3
0
9
6
Tmein Months
Figure 5.1: Graph for simple example illustrating fixed and random effects.
-
-
2 3 7 9 5 10 18 20 3 4 4 5 d
-1
-
1 1 1 1 1 1 1 1 1 1 1
-
0 3 6 9 0 3 6 9 0 3 6 9-
-
[$;I+
1 0 0 0 0 0 1 3 0 0 0 0 1 6 0 0 0 0
-
-
el 1 el2 el3
0 0 1 3 0 0 0 0 1 6 0 0 0 0 1 9 0 0
:5.5) ez 4
e31
0 0 0 0 1 3 0 0 0 0 1 6 -0 0 0 0 1 9-
-
e32 e33 e34
-
The design matrix X and the fixed-effects coefficients ,Ll are defined in exactly the same manner as they are in the general linear model, and /30 and P1 have comparable meaning. According to a mixed model analysis, PO = 3.300 and P1 = .933; the intercept, PO, represents the mean perceptual skill for all children at 0 months of age, whereas the slope, P I , represents the mean perceptual skill for each month of age. The fixed-effects (or population-mean) prediction line is shown in Figure 5.1. Next we examine the random-effects component, Zu. Along the diagonal of the random-effects design matrix Z are the design matrices for the individual subjects, and these design matrices are the same for all subjects:
The random coefficients in u,uoi and uli, represent residual intercepts and slopes, respectively. For this model, the first two observations for subject 1 can be written algebraically as:
109
Mixed Models 'Subjectl: .SubjectZ: Subject3.
E ( y i j I U O ~ , U I I ) = .69xlj+ 2.40 E ( p j I~0l,u1?)=1.79~?,+5.11 E ( y 3 j I u o ~ , u o ~ ) = - 3 2 ~ 3 j +2.39..,. Subject2 -...-**
........
,..-.".....Fixed Effects:
3
0
6
Time in Months
E(J,,) = . 9 3 ~+ j 3.30
9
Figure 5.2: Graph for simple example illustrating the variability of the prediction lines for individual subjects around the line for the population.
and y12
= Po f 3pi 4-2101
+ 32111 f e12
(5.8)
As shown in Figure 5.1, the prediction line for subject 2 has an intercept of ?!a, 2102 and a slope of 1 + u12,whereas the prediction line for the population has an intercept of PO and a slope of 81. Accordingly, the residual intercept, 2102, is the difference between the intercept for subject 2 and the intercept for the population prediction line. Similarly, the residual slope for subject 2, u12, is the difference between the slope for subject 2 and the slope for the population prediction line. Although typically we would not estimate the random coefficients, we did so for illustrative purposes. The residual intercepts are -.900, 1.806, and -.907 for subjects 1, 2, and 3, respectively, whereas the residual slopes are -.241, .852, and -.612 for the same three subjects. Given the fixed-effects intercept and slope are 3.300 and .933, respectively, the intercepts and slopes are 2.400 (3.300 .900) and .692 (.933 - .241) for subject 1, 5.106 (3.300 1.806) and 1.785 (.933 .852) for subject 2, and 4.393 (3.300 - ,907) and .321 (.933 - .612), for subject 3, respectively. The differences in the intercepts and slopes for subjects are graphically displayed in Figure 5.2. The model parameters in G are the variances and covariances among elements of the subject-specific vectors imbedded in the vector u. In this example, we included not only the variances for the residual intercepts and residual slopes but also the covariances between the residual intercepts and slopes. Given that no distinctions were made among subjects, we
+
+
+
110
Wallace and Green
assumed homogeneity of variances and covariance of the random effects across subjects. Under these conditions, the covariance matrix G; is the same for all subjects,
Gi =
[
ff;
ff12
(712
u;
]
(5.9)
where a: is the variance in intercepts between subjects, 0: is the variance in slopes between subjects, and u: is the covariance between the intercepts and slopes. The structure of G is:
r
0;
O12
012
a;
0 0 0 0
0 0 0 0-
G=I
0 0
0 0
OT
(712
ff12
0 0
u2”
0 0 0 0
O 0 0 0
0 0
ff;
ff12
012
u;
l
I
(5.10)
For our example, a?,a;, and 0 1 2 were estimated to be 2.707, .591, and 1.130, respectively. The magnitudes of these variances and covariance are consistent with the values for the estimated residual intercepts and slopes. For example, the estimated residual intercepts and slopes appear strongly positively related, and the correlation between them based on variances and covariances in G is .89(= 1.130/42.707 * .591). As illustrated in Figure 5.1, a within-subject error, e i j , represents the deviation of an observation at time j for subject i from that subject’s individual prediction line. These deviations may represent departures of the subject’s growth curve from linearity, measurement errors, or a combination of both. The variance of these errors is a model parameter and was estimated in our example to be 1.333. Given we do not hypothesize any covariance among the within-subject errors and assume homogeneity of these errors across all subjects, R is a 12 x 12 matrix with 1.333 along its diagonal. The covariance matrix among the dependent variable scores is a function of the random-effects design matrix, the covariance matrix among the random effects, and the covariance matrix among the within-subject errors, that is, V = Z G Z l + R. In our example, we estimate V as
- 4.04
6.10 9.49 6.10 16.15 23.52 9.49 23.52 38.88 12.88 32.22 51.57 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 - 0 0 0
0 12.88 0 0 0 32.22 0 0 0 51.57 0 0 0 72.25 0 0 0 4.04 6.10 9.49 0 6.10 16.15 23.52 0 9.49 23.52 38.88 0 12.88 32.22 51.57 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 12.88 32.22 51.57 72.25 0 0 0 0
0 0 0 0 0 0 0 0 4.04 6.10 9.49 12.88
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 6.10 9.49 16.15 23.52 23.52 38.88 32.22 51.57
0 0 0
0 0
0 0 0 12.88 32.22 51.57 72.25
(5.11)
111
Mixed Models
V indicates that variances increase with time and correlations are larger if they are between later observations. These results are consistent with the data as displayed in Figures 5.1 and 5.2 that show a greater divergence in the Y scores from the line for the fixed effects at later observations.
ESTIMATION AND STATISTICAL INFERENCE Estimation is more complicated in the mixed model than in the general linear model in that, not only are the fixed-effects parameters in unknown, but also the variance and covariance parameters in G and R must be estimated. Unlike the general linear model, they generally cannot be estimated independently of the fixed-effects parameters. As noted by Littell et al. (1996), the appropriate method for estimating the components of ,b is no longer ordinary least squares but rather generalized least squares using the inverse of V as the weight matrix. However, the elements of V are seldom known. Consequently, estimation is generally based on likelihoodbased methods as described in detail by Laird and Ware (1982), Harville (1977), and Jennrich and Schluchter (1986) among others, assuming that u and e are normally distributed. Likelihood-based methods yield distinct estimating equations for the fixed-effects parameters and variance components, with the fixed-effects estimating equations requiring estimates of the variance components and estimating equations for the variance components requiring estimates of the fixed effects. Consequently, the overall estimation procedures require iteration between the fixed-effects equations and variance components equations t o reach a final solution. First, we consider methods t o obtain variance component estimating equations followed by a discussion of the fixed-effects estimating equations. The two principal methods for estimating variance components are maximum likelihood (ML) and restricted (or residual) maximum likelihood (REML). Under the assumption of normality for both the random effects and the error terms, the likelihood function for the mixed model for all subjects combined has the form (Hocking, 1985):
N 1 1 l o g ( 2 ~) -log IVI - -(y - X p ) ’ V - l ( y - XP) (5.12) 2 2 2 1 1 = - -log IVI - -(y - xp)’v-’(y- xp) 2 2 in which the variables in the model are defined previously. Standard maximization procedures can be used t o derive a set of nonlinear estimating equations for the components of V, but these equations generally involve the elements of p. Patterson and Thompson (1971) proposed REML as an alternative method for estimating covariance parameters. This procedure relies on -
,
--
c
112
Wallace and Green
using the error space of the fixed-effects design matrix t o estimate the covariance parameters. The justification for the REML suggested by Patterson and Thompson (1971) relies on modifying the ML procedure via factorization of the likelihood function. Conceptually, this process involves partitioning the data space into two orthogonal components: the estimation space, defined as the column space of X , and the orthogonal complement of the estimation space, the error space. Mathematically, one can project the data vector onto the error space and estimate the variance components from this error space independently of the fixed effects. This procedure accounts for the loss of degrees of freedom needed to estimate the fixed effects and is analogous t o using the mean square error t o estimate o2 in classic ANOVA. Harville (1974) offers a justification of this procedure from a Bayesian framework and presents a convenient form of a likelihood function that can be used t o obtain REML estimates:
1 1, = C - - log IX'V-lXJ 2
-
1 2
1 2
- log IVI - - ( y - XP)'V-'(y - X b )
(5.13)
Estimates for the components of V can be obtained by maximizing this function, again to obtain a set of nonlinear estimating equations dependent on the elements of p. The SAS procedure MIXED can be used t o develop either ML or REML estimates, with REML estimates generally preferred unless the data sets are quite large. Conditional upon V being known, either ML methods or general leastsquares methods can be used t o derive the following set of estimating equations for the fixed effects parameters:
p = [x'v-'x]-1x'v-'y
(5.14)
To obtain estimates for p, the estimates of the variance components from the NIL or REML estimation procedure are used in the previous estimating equation. Either the ML or REML estimation can be implemented with the SAS MIXED procedure. Statistical inference in the mixed model is generally concerned with tests concerning the fixed effects that can be formulated in terms of sccondary parameters that are linear combinations of the fixed-effects parametcrs in p. More precisely, c secondary parameters are in a column vector 8, with 0 = C1p - 80,where C1 is c x q matrix that contains the coefficients for tlic c linear combinations of fixed-effects parameters and 00 is a c x 1 vector consisting of constants, in practice almost always zeros. The hypotheses of interest are of the form Ho : 8 = 0 versus Ha : 19 # 0. Three types of test statistics have been proposed for testing hypotheses of this type: likelihood ratio tests, Wald tests, and approximate F tests (Littell et al., 1996). Simulation studies indicate that Wald tests tend t o be overly liberal, possibly by as much as a factor of 2 for mixed models (Woolson & Leeper, 1980), and that likelihood ratio tests also tend t o be overly liberal (Andrade
113
Mixed Models
& Helms, 1986). 11s contrast, simulation studies by McCarroll and Helms (1987) suggested that an approximate F-test provides reasonable Type I error protection. Consequently, we recommend that the approximate F test, as described by Littell et al. be used. Using hypothesis tests as defined previously, the approximate F-test has the form:
where V = $I' . The numerator degrees of freedom for the test are generally accepted t o be the rank of C1, but as discussed in the section on modeling issues, no general consensus has been reached as to the best choice of denominator degrees of freedom for the test. The test statistic is implemented in the SAS MIXED procedure.
ISSUES IN APPLYING MIXED MODELS TO LONGITUDINAL DATA Statistical modeling provides best results if researchers have a clear understanding of the characteristics and structure of their data and implement a well-formulated analysis strategy in response t o explicitly defined research questions. Issues such as how measurements are made, how data are collected, bias in subject selection, and distributional properties of outcomes that are important in constructing a statistical model using classic regression or ANOVA techniques are also important for analysis of repeated measures data with mixed models. Investigators must address these issues t o obtain reliable results from mixed model analyses; however, we will focus on issues that are unique t o mixed models and repeated measures data. Longitudinal data with multiple observations on the same subject are inherently more complicated statistically than independent observations from different subjects. Two interrelated factors that contribute t o this complexity are that observations on the same subject are correlated and that the data contain both between-subject and within-subject information. The linear mixed model provides great flexibility for handling these added complexities, but the flexibility comes with a price, as investigators must decide among diverse options in selecting a model structure. All statistical models involve distributional assumptions about the mean and variance structures of the data, but modelers tend t o focus on the mean or expected value component given the limited choices among the variance components for most statistical procedures. In contrast, application of linear mixed models t o longitudinal data requires the modeler to address both the expected value (fixed effects) and the variance (or random effects) components explicitly. Recall that the mathematical formulation of the mixed model is: y =X p
+ Zu + e
(5.16)
114
Wallace and Green
with ,b representing the fixed effects and both u and e representing random effects, although the actual parameters of the random effects are associated with the covariance matrices G and R of u and e, respectively. Hence, in developing the model, the researcher must specify the parameters that are included in the vector p , generally through the specification of the design matrix X, and the structure of G and R. Specifying the structure of the individual parameters in the fixed effects vector ,b is a bit more complicated in the mixed model with longitudinal data than in the classic linear model for independent data in that the investigator must determine how to handle between-subject and within-subject information. The paragraphs that follow first discuss some of the options available for characterizing the variance components of the model. Then, issues related to handling withinsubject and between-subject information in the fixed-effects component of the model are discussed.
Random-Effects Components of the Mixed Model The variance-covariance components in the random-effects components of the linear mixed model can be partitioned into two types-those associated with u, the between-subject variance-covariance components, and those associated with e , the within-subject variance-covariance components. The ways in which the variability in the outcome can be partitioned between these two components and the structures that can be used for G and R are quite diverse and are limited only by the capabilities of the software and the insight of the researcher. Because the number of structures that one might encounter is so wide ranging, we describe only some of the more widely used ones in this chapter.
Between-Subjects Random Effects (G Matrix). One of the simpler models that falls within the framework of the mixed model is the classic univariate approach to repeated measures analysis of variance. In this model, the variance is partitioned into two components, a between-subject variance component ( 0 % )and a within-subject variance component (0;). We start with this relatively simple model and then generalize to models with more complex random-effects components. Consider a simple repeated measures experiment with 2 groups representing 2 subpopulations, 3 subjects per group, and 3 replications, for a total of 18 observations. The fixed-effects component of the model can be modeled in the usual way and is not of concern here. To obtain the appropriate structure for the variance components, which is a compound symmetric V matrix under the usual assumptions of repeated measures ANOVA, we construct a block diagonal Z matrix with 18 rows and 6 columns as shown in Figure 5.3. The G matrix is simply 0: * I6 (where IN is an N x N identity matrix), and the R matrix is 0; * 118. Under these conditions, V is compound symmetric, as can be shown by substituting into the following
Mixed Models
115
1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0
Z=
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0