Multi-Level Issues in Strategy and Methods (Research in Multi-Level Issues) (Research in Multi-Level Issues)

MULTI-LEVEL ISSUES IN STRATEGY AND METHODS i RESEARCH IN MULTI-LEVEL ISSUES Series Editors: Francis J. Yammarino and...

Author: Fred Dansereau; Francis Yammarino

46 downloads 1412 Views 5MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

MULTI-LEVEL ISSUES IN STRATEGY AND METHODS

i

RESEARCH IN MULTI-LEVEL ISSUES Series Editors: Francis J. Yammarino and Fred Dansereau Volume 1: The Many Faces of Multi-Level Issues Volume 2: Multi-Level Issues in Organizational Behavior and Strategy Volume 3: Multi-Level Issues in Organizational Behavior and Processes

ii

RESEARCH IN MULTI-LEVEL ISSUES VOLUME 4

MULTI-LEVEL ISSUES IN STRATEGY AND METHODS EDITED BY

FRED DANSEREAU State University of New York at Buffalo, USA

FRANCIS J. YAMMARINO State University of New York at Binghamton, USA

2005

Amsterdam – Boston – Heidelberg – London – New York – Oxford Paris – San Diego – San Francisco – Singapore – Sydney – Tokyo iii

ELSEVIER B.V. Radarweg 29 P.O. Box 211 1000 AE Amsterdam The Netherlands

ELSEVIER Inc. 525 B Street, Suite 1900 San Diego CA 92101-4495 USA

ELSEVIER Ltd The Boulevard, Langford Lane, Kidlington Oxford OX5 1GB UK

ELSEVIER Ltd 84 Theobalds Road London WC1X 8RR UK

r 2005 Elsevier Ltd. All rights reserved. This work is protected under copyright by Elsevier Ltd, and the following terms and conditions apply to its use: Photocopying Single photocopies of single chapters may be made for personal use as allowed by national copyright laws. Permission of the Publisher and payment of a fee is required for all other photocopying, including multiple or systematic copying, copying for advertising or promotional purposes, resale, and all forms of document delivery. Special rates are available for educational institutions that wish to make photocopies for non-proﬁt educational classroom use. Permissions may be sought directly from Elsevier’s Rights Department in Oxford, UK: phone (+44) 1865 843830, fax (+44) 1865 853333, e-mail: [email protected]. Requests may also be completed on-line via the Elsevier homepage (http://www.elsevier.com/locate/permissions). In the USA, users may clear permissions and make payments through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; phone: (+1) (978) 7508400, fax: (+1) (978) 7504744, and in the UK through the Copyright Licensing Agency Rapid Clearance Service (CLARCS), 90 Tottenham Court Road, London W1P 0LP, UK; phone: (+44) 20 7631 5555; fax: (+44) 20 7631 5500. Other countries may have a local reprographic rights agency for payments. Derivative Works Tables of contents may be reproduced for internal circulation, but permission of the Publisher is required for external resale or distribution of such material. Permission of the Publisher is required for all other derivative works, including compilations and translations. Electronic Storage or Usage Permission of the Publisher is required to store or use electronically any material contained in this work, including any chapter or part of a chapter. Except as outlined above, no part of this work may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without prior written permission of the Publisher. Address permissions requests to: Elsevier’s Rights Department, at the fax and e-mail addresses noted above. Notice No responsibility is assumed by the Publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein. Because of rapid advances in the medical sciences, in particular, independent veriﬁcation of diagnoses and drug dosages should be made. First edition 2005 British Library Cataloguing in Publication Data A catalogue record is available from the British Library. ISBN: 0-7623-1184-3 ISSN: 1475-9144 (Series)

8 The paper used in this publication meets the requirements of ANSI/NISO Z39.48-1992 (Permanence of Paper). Printed in The Netherlands.

Working together to grow libraries in developing countries www.elsevier.com | www.bookaid.org | www.sabre.org

iv

CONTENTS ABOUT THE EDITORS

ix

LIST OF CONTRIBUTORS

xi

OVERVIEW: MULTI-LEVEL ISSUES IN STRATEGY AND METHODS Francis J. Yammarino and Fred Dansereau

1

PART I: INNOVATION INNOVATION IN ORGANIZATIONS: A MULTI-LEVEL PERSPECTIVE ON CREATIVITY Michael D. Mumford and Samuel T. Hunter

11

RESOLVING SOME PARADOXES OF INNOVATION: A CROSS-LEVEL INTERPRETATION Jane M. Howell and Kathleen Boies

75

‘‘WE WANT CREATIVITY! NO, WE DON’T!’’ Robert J. Sternberg

93

THE CREATIVITY PARADOX: SOURCES, RESOLUTIONS, AND DIRECTIONS Michael D. Mumford and Samuel T. Hunter

105

v

vi

CONTENTS

PART II: STRATEGIC PERFORMANCE MULTI-LEVEL ISSUES FOR STRATEGIC MANAGEMENT RESEARCH: IMPLICATIONS FOR CREATING VALUE AND COMPETITIVE ADVANTAGE Paul Drnevich and Mark Shanley

117

DEVELOPING MULTI-LEVEL THEORY IN STRATEGIC MANAGEMENT: THE CASE OF MANAGERIAL TALENT AND COMPETITIVE ADVANTAGE Alison Mackey and Jay B. Barney

163

A RESOURCE-BASED LENS ON VALUE CREATION, COMPETITIVE ADVANTAGE, AND MULTI-LEVEL ISSUES IN STRATEGIC MANAGEMENT RESEARCH Margaret Peteraf

177

MULTI-LEVEL ISSUES FOR STRATEGIC MANAGEMENT RESEARCH: FURTHER REFLECTIONS Paul Drnevich and Mark Shanley

189

PART III: UPPER ECHELONS A MULTI-LEVEL ANALYSIS OF THE UPPERECHELONS MODEL Albert A. Cannella, Jr. and Tim R. Holcomb

197

Contents

vii

MOVING (FINALLY) TOWARD A MULTI-LEVEL MODEL OF THE UPPER ECHELONS Mason A. Carpenter

239

UPPER ECHELONS PERSPECTIVE AND MULTILEVEL ANALYSIS: A CASE OF THE CART BEFORE THE HORSE? Dan R. Dalton and Catherine M. Dalton

249

A MULTI-LEVEL ANALYSIS OF THE UPPERECHELONS MODEL: PLANTING SEEDS FOR FUTURE RESEARCH Albert A. Cannella Jr. and Tim R. Holcomb

263

PART IV: LATENT GROWTH MODELING MULTIVARIATE LATENT GROWTH MODELS: READING THE COVARIANCE MATRIX FOR MULTI-LEVEL INTERPRETATIONS Kai S. Cortina, Hans Anand Pant and Joanne Smith-Darden

277

MULTIVARIATE LATENT GROWTH MODELING: ISSUES ON PRELIMINARY DATA ANALYSES David Chan

319

A NOTE ON THE COMPUTER GENERATION OF MEAN AND COVARIANCE EXPECTATIONS IN LATENT GROWTH CURVE ANALYSIS Kevin J. Grimm and John J. McArdle

335

THE VALUE OF HEURISTICS: CLARIFYING THE PURPOSE OF THREE-STEP-APPROACH TO ANALYZE MULTIVARIATE LATENT GROWTH MODELS Kai S. Cortina, Hans Anand Pant and Joanne Smith-Darden

365

viii

CONTENTS

PART V: INTRA-CLASS CORRELATION SIGNIFICANCE TESTS FOR DIFFERENCES BETWEEN DEPENDENT INTRACLASS CORRELATION COEFFICIENTS (ICCs) Ayala Cohen and Etti Doveh

375

INTERPRETING CHANGES IN ICCS: TO AGREE OR NOT TO AGREE, THAT IS THE QUESTION Paul J. Hanges and Julie S. Lyon

421

A MODEL SELECTION APPROACH TO TESTING DEPENDENT ICCs Wolfgang Viechtbauer and David Budescu

433

MORE ON THE COMPARISON OF INTRA-CLASS CORRELATION COEFFICIENTS (ICCs) AS MEASURES OF HOMOGENEITY Ayala Cohen and Etti Doveh

455

PART VI: ABOUT THE AUTHORS ABOUT THE AUTHORS

463

ABOUT THE EDITORS Fred Dansereau is a Professor of Organization and Human Resources in the School of Management at the State University of New York at Buffalo. He received his Ph.D. from the Labor and Industrial Relations Institute at the University of Illinois with a specialization in Organizational Behavior. Dr. Dansereau has extensive research experience in the areas of leadership and managing at the individual, dyad, group, and collective levels of analysis. Along with others, he has developed a theoretical and empirical approach to theorizing and testing at multiple levels of analysis. He has served on the editorial review boards of the Academy of Management Review, Group and Organization Management, and Leadership Quarterly. Dr. Dansereau is a Fellow of the American Psychological Association and the American Psychological Society. He has authored eight books and over 70 articles and is a consultant to numerous organizations, including the Bank of Chicago, Occidental, St. Joe Corp., Sears, TRW, the United States Army and Navy, Worthington Industries, and various educational institutions. Francis J. Yammarino is SUNY Distinguished Professor of Management and Director and Fellow of the Center for Leadership Studies at the State University of New York at Binghamton. He received his Ph.D. in Organizational Behavior (Management) from the State University of New York at Buffalo. Dr. Yammarino has extensive research experience in the areas of superior–subordinate relationships, leadership, self–other agreement processes, and multiple levels of analysis issues. He serves on the editorial review boards of seven scholarly journals, including the Academy of Management Journal, Journal of Applied Psychology, and the Leadership Quarterly. Dr. Yammarino is a Fellow of the American Psychological Society and the Society for Industrial and Organizational Psychology. He is the author of nine books and has published about 100 journal articles and book chapters. Dr. Yammarino has served as a consultant to numerous organizations, including IBM, Textron, TRW, Lockheed Martin, Medtronic, United Way, and the U.S. Army, Navy, Air Force, and Department of Education. ix

This page intentionally left blank

x

LIST OF CONTRIBUTORS Jay B. Barney

Ohio State University, USA

Kathleen Boies

Concordia University, Canada

David Budescu

University of Illinois at UrbanaChampaign, USA

Albert A. Cannella, Jr.

Arizona Sate University, USA

Mason A. Carpenter

University of Wisconsin, USA

David Chan

National University of Singapore, Singapore

Ayala Cohen

Technion – Israel Institute of Technology, Israel

Kai S. Cortina

University of Michigan, USA

Catherine M. Dalton

Indiana University, USA

Dan R. Dalton

Indiana University, USA

Fred Dansereau

State University of New York at Buffalo, USA

Etti Doveh

Technion – Israel Institute of Technology, Israel

Paul Drnevich

Purdue University, USA

Kevin J. Grimm

University of Virginia, USA

Paul J. Hanges

University of Maryland, USA

Tim R. Holcomb

Texas A & M University, USA

Jane M. Howell

University of Western Ontario, Canada

Samuel T. Hunter

University of Oklahoma, USA

Julie S. Lyon

University of Maryland, USA xi

xii

LIST OF CONTRIBUTORS

Alison Mackey

Ohio State University, USA

John J. McArdle

University of Virginia, USA

Michael D. Mumford

University of Oklahoma, USA

Hans Anand Pant

University of Michigan, USA

Margaret Peteraf

Dartmouth College, USA

Mark Shanley

Purdue University, USA

Joanne Smith-Darden

University of Michigan, USA

Robert J. Sternberg

Yale University, USA

Wolfgang Viechtbauer

University of Maastricht, The Netherlands

Francis J. Yammarino

State University of New York at Binghamton, USA

OVERVIEW: MULTI-LEVEL ISSUES IN STRATEGY AND METHODS Francis J. Yammarino and Fred Dansereau INTRODUCTION ‘‘Multi-Level Issues in Strategy and Methods’’ is Volume 4 of Research in Multi-Level Issues, an annual series that provides an outlet for the discussion of multi-level problems and solutions across a variety of ﬁelds of study. Using a scientific debate format of a key scholarly essay followed by two commentaries and a rebuttal, we present, in this series, theoretical work, significant empirical studies, methodological developments, analytical techniques, and philosophical treatments to advance the ﬁeld of multi-level studies, regardless of disciplinary perspective. Similar to Volumes 1 (Yammarino & Dansereau, 2002), 2 (Dansereau & Yammarino, 2003), and 3 (Yammarino & Dansereau, 2004), this volume, Volume 4, contains ﬁve major essays with commentaries and rebuttals that cover a range of topics, but in the realms of strategy and analytical methods. In particular, the ﬁve ‘‘critical essays’’ offer extensive literature reviews, new model developments, methodological advancements, and some empirical data for the study of innovation, strategic performance, upper echelons, latent growth modeling, and intra-class correlations. While each of the major essays, and associated commentaries and rebuttals, is unique in orientation, they show a common bond in raising and addressing multi-level

Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 1–8 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04023-3

1

2

FRANCIS J. YAMMARINO AND FRED DANSEREAU

issues or discussing problems and solutions that involve multiple levels of analysis.

INNOVATION In the ﬁrst essay, Mumford and Hunter focus on the impact of innovation on organizational performance, a research area for many disciplines. In their extensive review of the literature, they discover a number of key variables that make innovation possible (e.g., creativity) and a complex set of relationships among these variables, innovation per se, and performance at multiple levels. To begin to sort out these complexities and apparently contradictory ﬁndings from prior work, Mumford and Hunter develop the notion of cross-level differences in the requirements for innovation and discover the existence of complex interactions operating at a single (focal) level of analysis. They employ this multi-level perspective for understanding how innovation and creativity occur in organizational settings. A key outcome of this effort is the generation of nine testable propositions at the individual level of analysis, eight at the group level of analysis, eight at the organizational level of analysis, six at the environmental level of analysis, and seven primary or critical cross-level interactions that are asserted for further research and exploration. In their commentary, Howell and Boies describe both the contribution and limitations of the work of Mumford and Hunter. In particular, Howell and Boies highlight the distinctions between innovation and creativity, and propose ﬁve multi-level variables – trust, social identity, mental models, networks, and time – to resolve some of the Mumford and Hunter paradoxes and to formulate 10 additional multi-level propositions. They also indicate several future research directions for multi-level research on innovation and creativity. In his commentary, Sternberg complements the work of Mumford and Hunter by indicating how different types of creativity are rewarded differentially. He discusses different kinds of creative contributions and how these can accept existing paradigms, propose new paradigms, or integrate various paradigms. Sternberg’s comments follow from the creativity paradox: organizations say they value creativity, but when they get creativity, they do not uniformly value it. In their reply to the commentators, Mumford and Hunter address a critical question raised by Howell and Boies as well as Steinberg; i.e., why is creativity so difﬁcult even though organizations say they want creativity?

Overview: Multi-Level Issues in Strategy and Methods

3

Mumford and Hunter identify sources of, resolutions regarding, and future research directions for, this creativity paradox. Their points are both multivariate in nature and address cross-level interactions.

STRATEGIC PERFORMANCE In the second essay, Drnevich and Shanley focus on problem-oriented multilevel issues for strategic management research in general, for strategic performance in particular, and even more specifically, for the core concepts of competitive advantage and value creation. They note that most prior work in these areas is from a single perspective, at a single level of analysis (e.g., top-management team, ﬁrm level, or network level), without considering alternative, integrative, or multi-level perspectives. As a result, Drnevich and Shanley assert that what actually constitutes ‘‘good’’ performance is how performance is measured effectively, and how these measures align with different perspectives remains problematic. To begin to address these issues, they examine the multi-level nature of strategic management phenomena and develop three multi-level approaches – transaction, management, and atmosphere – to resolve conﬂicts in the literature regarding the core multilevel constructs of competitive advantage, value creation, and their linkages. Drnevich and Shanley conclude their work by emphasizing the inherent multi-level nature of strategy research on value redistribution versus creation; competitive benchmarking, advantage, and distinctive competency; and strategic pricing. In their commentary, Mackey and Barney discuss the case of managerial talent and competitive advantage for developing multi-level theory in strategic management. They do so using arguments presented by Drnevich and Shanley on strategic leadership and by examining interactions among three levels of analysis – individual, industry, and market – for the relationship between managerial capabilities and ﬁrm performance. In particular, Mackey and Barney discuss conditions in which leadership can be a competitive advantage, when managerial talent is imperfectly allocated across ﬁrms via labor markets, and when managers can appropriate ‘‘rents’’ that their specific talents generate. In her commentary, Peteraf employs a resource-based lens on value creation, competitive advantage, and multi-level issues in strategic management research. In particular, she illustrates the multi-level nature of value creation and competitive advantage and shows how the resource-based view is linked in a multi-level way to the external market environment. Peterof also notes

4

FRANCIS J. YAMMARINO AND FRED DANSEREAU

that a resource-based approach provides a way to link the organizational level of analysis, content and process connections, and relationships across time. In their reply to the commentators, Drnevich and Shanley applaud the applications and extensions of their ideas by Mackey and Barney, and Peteraf. After noting general agreement with the commentaries, Drnevich and Shanley reﬂect on three common themes across all these writings: the dual, complex role of management as a resource/capability and integrator; the importance of strategic decision processes and ﬁrm structures for linking levels both in theory and practice; and the need for dynamic multi-level approaches to strategic management.

UPPER ECHELONS In the third essay, Cannella and Holcomb focus on a multi-level analysis of the upper-echelons model, a key and well-established foundation of strategic management research. This model, which is presumed to be multi-level in nature, explains how experiences, backgrounds, preferences, biases, and values of senior executives (top management team (TMT) members) can impact the decisions they make in organizations. Cannella and Holcomb point out, however, that the upper-echelons model has never been subjected to rigorous multi-level analysis in terms of both theory (conceptualization) and method (empiricism). In their critical review of the model, Cannella and Holcomb juxtapose levels concepts and theories on the upper-echelons model to highlight its strengths and weaknesses. They conclude that the model is inherently individual-level in focus and suffers from several important limitations that must be addressed before the upper-echelons model provides a complete explanation of team-level decision-making – a muchtouted advantage of the model. In particular, Cannella and Holcomb discuss the assumptions of individuality, homogeneity, and heterogeneity among TMT members, including the CEO; framing and the TMT decisionmaking process; and TMT heterogeneity and temporal dynamics from a multi-level perspective. In his commentary, Carpenter appreciates that the work of Cannella and Holcomb is ﬁnally moving the ﬁeld to a multi-level model of upper echelons. After highlighting ﬁve key contributions in their work, Carpenter suggests three supplementary directions for future work: i.e., viewing the upperechelons perspective as both a theory and a method, determining who is on the TMT, and extending the definition of upper echelon to include boards of

Overview: Multi-Level Issues in Strategy and Methods

5

directors and external consultants. He also notes the importance of encompassing both strategy formulation and strategy implementation, given their interdependence, to generate a fully multi-level upper-echelons model. In their commentary, Dalton and Dalton agree with Cannella and Holcomb that TMTs are not necessarily the appropriate level of analysis for upper-echelons research. They question whether currently employed research constructs and variables are sufﬁciently developed to adequately test an upper-echelon view or to sensibly warrant a multi-level approach. In particular, Dalton and Dalton discuss the implications of the fragility of agency theory, TMT turnover and homogeneity/heterogeneity, and the lack of inclusion of boards of directors for the conceptualization of TMTs and a multi-level upper-echelons perspective. In their reply to the commentators, Cannella and Holcomb ﬁrst thank Carpenter and Dalton and Dalton for their insights on the original chapter on the upper-echelons model. Cannella and Holcomb continue to believe that TMTs are an important level of analysis for strategic management research, but also that the original upper-echelons model cannot be applied at the team level. They then highlight their views on several joint as well as some specific concerns of Carpenter and Dalton and Dalton about upperechelons research and the levels of analysis issues that are involved.

LATENT GROWTH MODELING In the fourth essay, Cortina, Pant, and Smith-Darden focus on multivariate latent growth models (MLGM) and multi-level interpretations that can result from latent growth modeling (LGM) analyses. While technical in nature, they also describe their work and results conceptually and in a form that is useful for less methodologically sophisticated readers. LGM, via Hierarchical Linear Models (HLM) or Structural Equation Models (SEM), is an approach for the assessment and analysis of change over time. Because data of this type are ‘‘nested’’, the models are inherently multi-level in nature. From a theory of change perspective, Cortina et al. present and illustrate a three-step approach to prescreen the covariance matrix in repeated measurement that permits the identiﬁcation of major trends in the data prior to conducting MLGM. They examine several different cases – independent growth, intercept correlation, within-variable intercept-slope correlation, and combinations – to develop a three-step approach for ‘‘reading’’ the covariance matrix prior to statistical analysis for multi-level data.

6

FRANCIS J. YAMMARINO AND FRED DANSEREAU

In his commentary, Chan agrees with Cortina et al. that MLGM provides a ﬂexible data analytic framework for representing and assessing betweenconstructs relationships in intra-individual changes over time and for multiple levels of analysis. He then points out important preliminary data analysis and interpretation issues prior to conducing MLGM analyses. A key point for Chan, given MLGM’s infancy of development, is that it is critical to address fundamental questions adequately and ground practical recommendations in sound conceptual, measurement, and statistical logic. He argues all of this should occur prior to any decision heuristic such as the three-step approach. In their commentary, Grimm and McArdle focus on computing structural expectations in LGM analysis. In particular, in their more general extension, they develop and demonstrate an approach for computer generation of mean and covariance expectations, which are the source of parameter estimates, ﬁt statistics, and substantive interpretations for latent growth curves. As such, using a computer program like Maple, they extend the work of Cortina et al. from linear change growth models to more complex models that can include non-linear processes and multiple longitudinal measurements. In their reply to the commentators, Cortina, Pant, and Smith-Darden remind readers that their three-step approach is a heuristic device for preliminary analysis of a more complete and complex multivariate latent growth analysis. In that sense, they note some disagreements with the comments of Chan who focuses on the more sophisticated, complete analytic approach. Cortina et al. also acknowledge the more general extension of their work by Grimm and McArdle to a variety of latent growth-modeling cases. All the authors recognize the multi-level nature of LGM in their writings.

INTRA-CLASS CORRELATION In the ﬁfth essay, Cohen and Doveh focus on developing significance tests for the differences between dependent intra-class correlation coefﬁcients (ICCs). Their work is technical in nature but Cohen and Doveh also present their ideas conceptually and illustrate them well via several examples to help less methodologically sophisticated readers appreciate the value and usefulness of the tests. In particular, viewing various changes within levels of analysis over time, they present a statistical method to assess whether the degree of interdependency within entities (e.g., groups) has changed over

Overview: Multi-Level Issues in Strategy and Methods

7

time, using ICCs as an indicator of the degree of homogeneity within entities. Essentially, Cohen and Doveh provide a new test procedure for comparing dependent ICCs when those ICCs are obtained from the same entities at multiple points in time. While the procedures focus on ICC(1), they can be extended or generalized to other forms of ICCs. This new test is critical for multi-level research as it permits researchers, for example, to explore the question of whether groups form or dissolve over time, and whether the degree of interdependence among individuals changes (increases or decreases) longitudinally. In their commentary, Hanges and Lyon commend the development of the test of dependent ICCs by Cohen and Doveh. They note that Cohen and Doveh use ICC as an index of homogeneity and that their test permits inferences about changes in homogeneity over time. Hanges and Lyon then discuss the applicability and extension of the work of Cohen and Doveh beyond the group level to the individual, organizational, and societal levels of analysis. They close with a caveat regarding the interpretation of ICCs as an index of within-group homogeneity. In their commentary, Viechtbauer and Budescu extend the work of Cohen and Doveh using a model-selection approach to testing dependent ICCs. In particular, in this more general approach, they suggest ﬁnding subsets of homogeneous ICCs across time and experimental tasks by specifying all possible models, using a top-down selection strategy, and employing the full data set when ﬁtting each model. Viechtbauer and Budescu believe this approach is powerful and efﬁcient, minimizes the chances of inconsistencies in inferences across tests, and provides a more complete understanding of the overall results. In their reply to the commentators, Cohen and Doveh note the contribution to statistical methodology of the work of Viechtbauer and Budescu as well as the extensions of their (Cohen and Doveh’s) original work to other levels of analysis as articulated by Hanges and Lyon. In addition to concurring with many of the commentators’ points, Cohen and Doveh provide other thoughts on the interpretation of ICC values as indices of homogeneity.

CONCLUSION The essays, commentaries, and replies in this book illustrate the kind of issues that arise in dealing with multiple levels of analysis. The definitions of concepts (albeit, innovation, strategic performance, or upper echelons)

8

FRANCIS J. YAMMARINO AND FRED DANSEREAU

change depending on what combination of levels of analysis are involved and added to them. The nuances of analytical methods (albeit, latent growth modeling or intra-class correlations) change when one moves from one to multiple levels of analysis. Moreover, although different paradigms may guide different scholars’ theories and research methods and techniques, levels of analysis issues must be resolved to have a viable paradigm. While offered within the context of strategy and analytic methods, we believe that much of what has been said provides insights, applications, and potential advances to other areas of scholarly investigation. The authors in this volume have challenged theorists, researchers, and methodologists to raise and address multilevel issues in all their disciplinary and interdisciplinary work. If you would like to be a part of contributing ideas to this scholarly endeavor, please contact us directly or visit our website at: www.levelsofanalysis.com.

ACKNOWLEDGMENTS The publication of the Research in Multi-Level Issues annual series and this volume have been greatly facilitated by Mary Malin and Julie Walker as well as Michelle Lewsey, Zoe Youd at Elsevier in the United Kingdom and the staff at Macmillan India Ltd. Closer to home, we thank our Schools of Management, the Center for Leadership Studies at Binghamton, the Jacobs Management Center at Buffalo, our secretaries, Wendy Clark, Marie Iobst, and Cheryl Tubisz, and our copy-editor, Jill Hobbs, for their help in preparing this book for publication. Finally and perhaps most importantly, we offer our sincere thanks to our contributors. The authors of the essays, commentaries, and rebuttals in this volume have provided new ideas and insights for unraveling the challenges of dealing with multiple levels of analysis and multi-level issues. Thank you all.

REFERENCES Dansereau, F., & Yammarino, F. J. (Eds) (2003). Multi-level issues in organizational behavior and strategy: Research in multi-level issues (Vol. 2). Oxford, UK: Elsevier. Yammarino, F. J., & Dansereau, F. (Eds) (2002). The many faces of multi-level issues: Research in multi-level issues (Vol. 1). Oxford, UK: Elsevier. Yammarino, F. J., & Dansereau, F. (Eds) (2004). Multi-level issues in organizational behavior and processes: Research in multi-level issues (Vol. 3). Oxford, UK: Elsevier.

PART I: INNOVATION

9

This page intentionally left blank

10

INNOVATION IN ORGANIZATIONS: A MULTI-LEVEL PERSPECTIVE ON CREATIVITY Michael D. Mumford and Samuel T. Hunter ABSTRACT Recognizing the impact of innovation on organizational performance, scholars from a number of disciplines have sought to identify the conditions that make innovation possible. Although these studies have served to identify a number of key variables, the relationship between these variables and innovation is complex. In this chapter, we argue that the apparent complexity of these relationships may be attributed to crosslevel differences in the requirements for innovation and the existence of complex interactions among the phenomena operating at a given level of analysis. The implications of this multi-level perspective for understanding how innovation occurs in organizational settings are discussed.

INTRODUCTION Ongoing changes in technology, markets, and competitive pressures have conspired to place a new premium on innovation (Dess & Pickens, 2000; Mumford, Scott, Gaddis, & Strange, 2002b). The available evidence indicates, Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 11–73 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04001-4

11

12

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

in fact, that innovation may be one of the more powerful inﬂuences on organizational performance. For example, Eisenhardt and Tabrizi (1995) have provided evidence indicating that the rate at which new products ﬂow to market is a crucial inﬂuence on the performance of high-technology ﬁrms. Using a broad set of 721 ﬁrms in the United Kingdom, Geroski, Machin, and Van Reenen (1993) found that the rate of innovation was related to profitability, with innovation exerting both direct and indirect effects on ﬁrm performance. Studies examining these indirect effects indicate that innovation allows ﬁrms to grasp or understand the implications of new technologies (Cohen & Levinthal, 1990), cope more effectively with radical environmental change (Tushman & O’Reilly, 1997), and formulate stronger business plans (Dean & Sharfman, 1996). These examples sufﬁce to make our basic point. Innovation, the development and implementation of new ideas, and creativity, the initial generation of these new ideas (Ghiselin, 1963; Mumford & Gustafson, 1988), make a difference. This conclusion, which has emerged rather slowly over the course of the last 20 years, has given rise to a large, and growing, body of research on organizational innovation. As valuable as this new wave of research has been with regard to understanding the factors that shape innovation, a careful reading of this literature can leave one feeling not only dazed, but also confused – a confusion that arises from the apparently contradictory ﬁndings that seem endemic to studies of innovation (Pelz, 1967). One illustration of this point may be found in a comparison of Oldham’s (2003) observations with those of Abbey and Dickson (1983) and Cardinal (2001). Oldham’s (2003) argument, which is based on sound empirical studies, states that goals and extrinsic rewards tend to inhibit creativity. At ﬁrst blush, it cannot be easily reconciled with Abbey and Dickson’s (1983) and Cardinal’s (2001) ﬁndings indicating that goals, output processes, and extrinsic rewards are all positively related to organizational innovation. Another illustration of these apparent contradictions may be obtained by comparing Keller’s (1986) observations concerning the need for cohesiveness in successful research and development teams with the oft-noted conclusion that creative work requires a high degree of autonomy (Amabile & Conti, 1999; Greenberg, 1992; Pelz & Andrews, 1966). Yet another illustration of this point may be found by comparing the observations of Quinn (1989) concerning the negative effects of organizational obstacles and bureaucracy with the observations of Chandy and Tellis (2000) and Gopalakrishnan and Damanpour (2000), which indicate that large bureaucratic organizations are more likely than smaller organizations to produce and/or adopt radical innovations.

Innovation in Organizations: A Multi-Level Perspective on Creativity

13

Of course, many explanations might be proposed for these apparently contradictory ﬁndings. Depending on the particular phenomenon at hand, these effects might be explained by differences in the requirements for various forms of innovations (e.g., technical process, product, market), relevant boundary conditions, multivariate interactions, and/or nonlinear relationships where positive effects diminish at the extreme (Damanpour, 1991; Baer, Oldham, & Cummings, 2003; Sternberg, 1988; Mumford & Moertl, 2003). An alternative explanation for these apparent contradictions, however, might be found in multi-level theory (Woodman, Sawyer, & Grifﬁn, 1993). Specifically, different levels of analysis (e.g., organizational, group, individual) may impose different requirements for innovation, with the interactions of variables across levels giving rise to the kind of contradictory ﬁndings noted above. In fact, the available research does indicate that multi-level interactions may be critical in understanding innovation in organizational settings. In one study along these lines, Bain, Mann, and Pirola-Merlo (2001) obtained evaluations of individual innovation and team innovation for 193 scientists working in 38 teams. Ratings of individual and team innovation were correlated with a climate measure examining participative safety, support for innovation, objectives, and task orientation. It was found that participative safety and support were particularly important inﬂuences on team innovation, whereas objectives and task orientation were particularly important inﬂuences on individual innovation. In another study along these lines, Taggar (2002) examined 94 groups, containing 480 business undergraduates, working on 13 business-relevant creative problem-solving exercises. Measures were obtained of individual creative processes (e.g., preparation, synthesis) and team-relevant creative process (e.g., communication, involvement, conﬂict management). It was found that individual and team processes interacted in determining creative performance on the various problem-solving exercises under consideration. The ﬁndings obtained by Taggar (2002) and Bain et al. (2001) indicate the need for a multi-level examination of innovation in organizational settings. Accordingly, our intent in this chapter is threefold. First, we will identify the critical variables inﬂuencing innovation at four levels of analysis: (1) individual, (2) group, (3) organizational, and (4) environmental. Second, we will examine potential interactions among the variables operating within a given level of analysis. Third, we will examine notable cross-level interactions. Implicit in these objectives is an argument that studies of innovation have been plagued by a failure to integrate ﬁndings across levels. Although this problem is not uncommon, it is a particularly pernicious problem in studies

14

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

of innovation because innovation is clearly affected by variables operating at all of these levels of analysis. Without a careful multi-level examination of relevant variables and their interactions, it is difﬁcult to see how we will ever attain a truly comprehensive understanding of this phenomenon. Unlike other phenomena in the social sciences, the variables operating across levels to shape innovation do not appear to be well integrated. In fact, we will argue, as we examine the ﬁndings obtained in various studies of innovation, that the requirements for innovation at one level of analysis (e.g., the individual level) often contradict the requirements for innovation at other levels of analysis (e.g., group and organizational levels). The existence of these contradictory requirements, while unusual, takes on signal import because an understanding of these contradictions is essential both for theory development and for the management of innovation.

INDIVIDUAL INFLUENCES Ultimately it is people – individuals – who generate new ideas and develop these ideas into useful products. Studies of creativity and innovation have, as a result, stressed individual-level studies (Mumford, 2003). Although the ﬁndings obtained in these studies are complex, they do point to four key sets of variables that inﬂuence individuals’ ability to generate new ideas and new products: (1) knowledge, (2) creative processing activities, (3) dispositional characteristics, and (4) motivation.

Knowledge Recognizing that it is impossible to create something from nothing, most students of creativity have come to stress the fundamental importance of knowledge (Ericsson & Charness, 1994; Weisberg, 1999). Knowledge, however, is a complex construct involving two critical attributes: (1) information and (2) a framework for interpreting, organizing, gathering, and acting on this information. The frameworks used by individuals in interpreting, organizing, gathering, and acting on information are commonly subsumed under the rubric of expertise. Experience, or expertise, provides people with the cognitive structures, more extensive and better organized knowledge bases, and relevant experiential cases that allow them to work effectively with available information in solving the kind of complex, ill-deﬁned, novel problems that call for creative thought (Chi, Bassock, Lewis, Reitman, & Glaser,

Innovation in Organizations: A Multi-Level Perspective on Creativity

15

1989; Hershey, Walsh, Read, & Chulef, 1990; Mumford, Blair, & Marcy, in press; Reeves & Weisburg, 1999). The available evidence indicates that expertise is, in fact, a critical determinant of innovation in organizational settings. In one study, Thamhain (2003) obtained appraisals of individual expertise for 938 professionals working in 74 research and development teams. When these evaluations of expertise were correlated with senior managers’ appraisals of innovation, based on considerations such as new product ﬁelding, revenue, customer satisfaction, and patents, correlations in the high 0.30s were obtained. In another study, Dewar and Dutton (1986) assessed expertise in an organizational-level study through the number of technical specialists available. They found that the availability of requisite expertise produced correlations in the 0.40s with adoption of incremental and radical innovations by the 40 ﬁrms under consideration. Although expertise provides a foundation for innovation, when considering the application of expertise in organizational settings, three points should be noted. First, multiple types of expertise (e.g., technical, administrative, marketing) will be required in most real-world creative efforts, bringing the following question to the fore: What types of expertise should be brought to better bear on a problem at what time? Accordingly, Olson, Walker, Ruekert, and Bonner (2001) in a study of product development teams found that different mixes of expertise were required for project performance as projects moved through different stages of development, with marketing expertise proving important early on, and production expertise proving important at a later stage. Second, the development of different types of expertise in organizations depends on differentiation and specialization – with both of these variables being positively related to ﬁrm innovation (Damanpour, 1991). Differentiation and specialization, however, will also make the integration of individuals’ efforts more difﬁcult. Third, relevant expertise changes over time as a function of markets, competitors, and technology – changes that may require shifts in expertise particularly when radical, as opposed to incremental, innovations are being pursued (Mooreman & Miner, 1997). Of course, expertise is unlikely to prove of much value if experts lack necessary information. This point is aptly illustrated in a study by Monge, Cozzens, and Contractor (1992). They examined product improvement suggestions – a marker of innovation – provided by the employees of ﬁve companies. Measures of information availability and information exchange were obtained through self-report measures. In this study, information availability and information exchange produced correlations in the mid 0.40s with suggestions. Other works by Allen and Cohen (1969), Brown and

16

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Eisenhardt (1995), Cooper and Kleinschmidt (2000), and Troy, Szymanski, and Rajan (2001) also point to the need for information in creative efforts. What should be noted here, however, is that simply providing people with more information is not necessarily desirable due to the debilitating effects of overload (Meyer, 1998). Creative thought, moreover, requires the right information. In a study of information-gathering strategies, Mumford, Baughman, Supinski, and Maher (1996) found that higher-quality and more original solutions to novel marketing and policy problems were obtained when people searched for key facts and anomalies rather than a wide range of information. Similar results were obtained by Souitaris (2001), who, in a study of 105 ﬁrms, found that product innovation was positively related to a search for specific, problem-relevant information rather than for general information. Although the availability of specific, targeted information appears critical to creativity and innovation, it should also be recognized that information gathering is a costly and time-consuming activity. As a result, the depth and breadth of the information search will be limited when people lack capacity due to time pressure, production demands, extraneous commitments, and stress (Amabile, Hadley, & Kramer, 2002; Andrews, 1968; Kobe & ReiterPalmon, 2000). As a consequence, demand management may be essential for promoting creativity and innovation in organizational settings. One way that people cope with these pressures is by applying informationgathering strategies that reduce demand. Andersen, Glassman, McAfee, and Pinelli (2001) and Culnan (1983), in studies of the use of information sources by research and development personnel, found that people relied on locally available verbal information rather than less readily accessible information (e.g., technical journals, conferences). One way that organizations compensate for restricted information gathering is by constructing teams that include people with different backgrounds, thereby increasing the range of information likely to be applied. In fact, the use of multifunctional teams and/or teams where members posses relevant but distinct expertise has been found to contribute to creativity, idea generation, and the success of new product introductions (Cooper & Kleinschmidt, 2000; Dunbar, 1995; Gassman & van Zedwitz, 2003). Another way of ensuring information availability is by assigning gatekeepers to teams. Gatekeepers are individuals who actively seek out technical, organizational, and product information external to the group. The available evidence indicates that the presence of gatekeepers is positively related to the success of new product development efforts (Allen & Cohen, 1969; Keller & Holland, 1983). However, gatekeepers, who are often high-status, recognized

Innovation in Organizations: A Multi-Level Perspective on Creativity

17

experts, serve not only to disseminate but also to ﬁlter information – ﬁltering that is often based on extant expertise and dominant models. This ﬁltering may, of course, prove problematic if it results in the discounting of certain information. In keeping with this observation, Perry-Smith and Shalley (2003), in an examination of the inﬂuence of network structures on innovation, concluded that moderate centrality and moderate diversity in connections were most likely to enhance innovation. As alluded to in our foregoing observations, organizational-level variables may also condition information availability. Notably, centralization and balkanization, due to strong departmental boundaries, can limit information ﬂow and thereby inhibit innovation (Troy et al., 2001). In a study of organizational blockages to information ﬂow, Adams and Day (1998) interviewed 137 people in 15 ﬁrms involved in one or more new product development efforts. The resulting interview data were content-analyzed to identify organizational blockages to information ﬂow. Three key obstacles were identiﬁed: (1) a focus on easily understood information, (2) use of the same information as in the past, and (3) compartmentalization or a tendency to focus only on information relevant to one’s immediate work role. These ﬁndings suggest that organizations – whether because of history, experience, extant processes, and infrastructure – may channel their information searches along selected avenues that, while reducing load and increasing efﬁciency, may inhibit creativity and innovation. This observation, in turn, suggests that when radical (but not necessarily incremental interventions) are required, extended information searches and the application of nonnormative information-gathering procedures may be required.

Processes Of course, if we had only existing knowledge and information to work with, it would be impossible to create something new. For this reason, studies of creativity have sought to identify the cognitive processing activities that allow people to generate new ideas using available information and extant knowledge (e.g., Brophy, 1998; Finke, Ward, & Smith, 1992; Lubart, 2001; Merriﬁeld, Guilford, Christensen, & Frick, 1962; Parnes & Noller, 1972; Sternberg, 1988). In reviews of this literature, Mumford and his colleagues (Mumford, Mobley, Uhlman, Reiter-Palmon, & Doares, 1991; Mumford, Peterson, & Childs, 1999) identiﬁed eight core processes commonly involved in creative thought:

18

1. 2. 3. 4. 5. 6. 7. 8.

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Problem identiﬁcation Information gathering Concept selection Conceptual combination Idea generation Idea evaluation and revision Implementation planning Monitoring

The available evidence indicates that effective execution of these processing activities is related to the production of higher-quality and more original solutions as people work on creative problem-solving tasks (Brophy, 1998; Mumford, Supinski, Baughman, Costanza, & Threlfall, 1997; Okuda, Runco, & Berger, 1991; Rostan, 1994). The inﬂuence of process execution on creative efforts in organizational settings has received less attention than expertise and information. Nonetheless, the available evidence does underscore the importance of effective cognitive processing activities. In one study along these lines, Vincent, Decker, and Mumford (1992) presented 1,818 army ofﬁcers with a military problem-solving scenario where they were asked to provide written answers to questions intended to elicit key processes (e.g., what do you see as the problem facing your army – problem identiﬁcation). Judges’ evaluations of the effectiveness of the heuristic application were obtained along with measures of leader problem solving, critical incident performance, and achievement. In a series of structural equation modeling efforts, it was found that process execution mediated the effects of intelligence, divergent thinking, and expertise on performance, with process execution yielding multiple correlations in the 0.50s with the various performance criteria under consideration. The apparent impact of effective process execution on creative thought brings up a new question: What are the conditions required for effective process execution? The available evidence indicates that execution of these processes is difﬁcult and demanding, calling for a substantial investment of cognitive resources (Getzels & Csikszentmihalyi, 1976). As a result, focus and attention appear critical to effective process execution. Moreover, people need time. In a study of leader inﬂuences on creative problem solving, for example, Redmond, Mumford, and Teach (1993) found that the production of original, high-quality advertising campaigns improved when people were given instructions that encouraged them to devote time to problem definition.

Innovation in Organizations: A Multi-Level Perspective on Creativity

19

Although our stereotype of the creative act assumes processing activities should be unconstrained, the available evidence indicates that effective execution of these processes requires relevant, high-quality material. This observation, in turn, implies that critical thought will play a role in most creative efforts (Baer, 2003; Halpern, 2003; Mumford, Baughman, & Sager, 2003). Thus, actions taken that encourage critical analysis of both material and the approach being taken in problem solving can be expected to contribute to creative thought. This observation is consistent with the observations of Cosier and Rose (1977) and Schwenk and Cosier (1980) concerning the merits of analyzing conﬂicting positions in managerial decision making. Moreover, actions that provide people with parameters or requirements seem beneﬁcial in part because they help structure this kind of analysis (Finke et al., 1992). By the same token, it should be recognized that when critical analysis is overbearing, or conﬂict laden and personally evaluative, and when parameters and restrictions are considered ﬁxed and are tightly speciﬁed, creative thought will suffer. These points are of some importance because they suggest that tight control and intense personalized political conﬂict in organizational settings will tend to inhibit creative thought. Within this process model, conceptual combination – that is, linking things in new ways – is held to give rise to the new understandings that provide a basis for idea generation (Mobley, Doares, & Mumford, 1992). Initially, these new understandings tend to be poorly articulated. As a consequence, exploration of new concepts, and the alternative ideas emerging from these concepts, is required for creative thought (Estes & Ward, 2002; Ward, Smith, & Finke, 1999). Organizational actions that support exploration and reﬁnement such as incremental experimentation and application search (Weintroff, 1992) can, therefore, prove useful. This need for exploration and experimentation further implies that evaluation and revision are important aspects of creative thought. This observation is consistent with the ﬁndings of Runco and his colleagues (Basadur, Runco, & Vega, 2000; Runco & Chand, 1994), indicating that idea generation and idea evaluation skills evidence strong positive relationships. Furthermore, the need for evaluation and revision following exploration indicates that the standards applied in evaluating ideas will have a marked impact on creative performance (Galbraith, 1982; Lei, Slocum, & Pitts, 1999; Lonergan, Scott, & Mumford, 2004). The impact of evaluation standards on the adoption of new technologies is illustrated in a study by Kitchell (1995). He found that the use of ‘‘innovative’’ standards (e.g., learning, capability enhancement, and market

20

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

growth) as opposed to ‘‘operating efﬁciency’’ standards (e.g., cost savings, ﬁnancing, and disruption) led to more rapid adoption of new technologies in manufacturing ﬁrms. The apparent effect of standards on idea acceptance and idea revision is noteworthy because it points to a critical cross-level effect on innovation. More specifically, organizational experience, like professionalization, instills in individuals standards for evaluating ideas. These standards will, in turn, shape both the occurrence of creative efforts and the kind of creativity observed (Csikszentmihaly, 1999). Thus, in shaping evaluation standards through socialization, culture, and role modeling, organizations may exert a profound inﬂuence on individuals’ creative efforts and subsequent organizational innovation – albeit an inﬂuence that may prove problematic when organizational standards are inconsistent with professional standards. Note, however, that the ambiguity that surrounds new ideas will make application of completely objective standards difﬁcult. As a result, attributes of the person – for example, his or her track record, interpersonal skills, perceived creativity, and style – may all act to inﬂuence evaluations of an idea (Elsbach & Kramer, 2003; Kasof, 1995). Moreover, evaluation cannot be absolute given the need for revision and reﬁnement. This point indicates (1) a need to avoid premature evaluation in appraising ideas and (2) a need for a staged approach to evaluation that moves from appraisals of potential to more concrete evaluations of business considerations such as proﬁtability, competitors, market size, and manufacturing requirements (Cooper & Kleinschmidt, 1991; Hitt, Hoskisson, Johnson, & Moesel, 1996; Scott, 1995).

Dispositions In addition to process and knowledge, creativity is inﬂuenced by a select set of dispositional attributes. Reviews of studies examining the relationship between dispositional characteristics and creativity by Barron and Harrington (1981), Feist and Gorman (1998), and Mumford and Gustafson (1988) paint a clear picture of this relationship. Creative people tend to be open and ﬂexible, evidencing substantial autonomy and a high degree of achievement motivation. Moreover, they tend to be somewhat competitive, domineering, and critical. In addition to these core characteristics, conscientiousness appears to be linked to creativity in technical professions, while rebelliousness and nonconformity are often linked to creativity in artistic professions (Feist, 1999).

Innovation in Organizations: A Multi-Level Perspective on Creativity

21

The significance of these dispositional characteristics is due, in part, to their implications for performance and motivation (Mumford & Gustafson, 1988). Clearly, characteristics such as openness and ﬂexibility will engender the kind of exploration required for creative thought, just as conscientiousness and criticality will engender the ability to perform careful analysis. Openness will lead people to tackle the novel, ill-deﬁned problems that call for creative thought, whereas achievement motivation and dominance will encourage people to persist when they begin work on these problems. The effects of dispositional characteristics on creativity and innovation may, however, be somewhat more far-reaching. Specifically, it appears that creative people are attracted to and likely to perform better in work environments consistent with their broader pattern of dispositional characteristics. For example, given their desire for autonomy and individual achievement, one might expect creative people to perform better in settings characterized by high autonomy. Studies by Amabile, Conti, Coon, Lazenby, and Herron (1996), Enson, Cottam, and Band (2001), and Pelz and Andrews (1966, 1976) support this proposition in ﬁelds ranging from research and development to marketing. Additional support for the existence of these dispositional/contextual interactions has been provided by Oldham and Cummings (1996). In a study of contributions to a suggestion program by 171 manufacturing employees, they found that contributions for creative individuals placed in a setting of low complexity, low support, and high supervisory control were relatively sparse. Creative individuals’ contributions were higher – and much higher vis-a`-vis noncreative people – when they were placed in a setting of high complexity, high support, and low supervisory control. The nature of this interaction was such that it indicated high environmental reactivity among creative people. The apparent reactivity of creative people to environmental characteristics relevant to their dispositional characteristics has two noteworthy implications with regard to person–organization interactions. First, one can expect that (1) creative people will be strongly attracted to organizations that provide, or create the image of providing, a work context conducive to innovation, and (2) they will be more likely to stay in jobs where the work setting, in fact, proves conducive to innovation (Shalley, Gilson, & Blum, 2000). This strong attraction–survival effect, especially given the tendency of creative people to restructure environments so as to make them more conducive to creativity, opens up the possibility of a virtuous cycle effect, whereby creative organizations become more creative over time (Bunce & West, 1995). A similar phenomenon can be observed with respect to industries (Wise, 1992).

22

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Second, organizational actions, policies, and procedures that inhibit expression of these dispositional characteristics can be expected to block creativity and innovation. This point is illustrated in a study by George and Zhou (2001), who measured conscientiousness and openness in a sample of 159 ofﬁce workers whose creative performance was assessed using Scott and Bruce’s (1994) behavioral self-report scale. A measure of close monitoring, or closeness of supervision, was also obtained. It was found that overly close supervision interacted with conscientiousness to inhibit creative behavior. These ﬁndings are of special interest because they suggest that the balance among multiple dispositional characteristics must be attended to in deﬁning an appropriate work context. Thus, extreme achievement and production pressures might inhibit the expression of openness, whereas ﬂexibility might be lost in the face of strong competitive demands. As noteworthy as these effects of dispositional characteristics may be, it is important not to lose sight of a more basic – albeit often discounted – implication of this pattern of dispositional characteristics. Specifically, domineering, critical, competitive people who are highly autonomous are not easily socialized, nor can they be expected to work easily with others. This point is of some importance because most real-world creative efforts require collaboration (Abra, 1994) and the involvement of multiple parties (Thamhain, 2003). This inherent conﬂict, in turn, implies that a premium will be placed on the management of interactional processes in virtually all creative efforts (Bain et al., 2001).

Motivation Earlier, we pointed out that these dispositional characteristics are of interest in part because they motivate creative efforts. In fact, the effects of dispositional characteristics on motivation may be rather subtle. Criticality, conscientiousness, competitiveness, and achievement motivation represent a syndrome of characteristics that will lead people to set high standards. In turn, these standards, vis-a`-vis observed discrepancies, will give rise to dissatisfaction; dissatisfaction and its associated negative affect may spur on creative efforts (Rinaldi, Cordone, & Casagrandi, 2000). Some evidence supporting this proposition has been provided by Zhou and George (2001). They assessed creative behavior using Scott and Bruce’s (1994) behavioral self-report measure in a sample of 149 ofﬁce workers. These researchers found that dissatisfaction was positively related to creativity, at least under

Innovation in Organizations: A Multi-Level Perspective on Creativity

23

conditions where the work context was consistent with the needs of creative people and creative people were committed to their jobs. The effects of standards and dissatisfaction on the initiation of creative efforts are linked to another key motivational characteristic of creative people. Studies by Feldman (1999) and Heinzen, Mills, and Cameron (1993) indicate that intense curiosity about some phenomenon is one of the key markers of creative potential in virtually all ﬁelds of endeavor. Arvey and Neel (1975) have provided evidence indicating that job curiosity is likewise related to creativity in the workplace. This intensive focus on, or curiosity about, a phenomenon not only serves to ensure the development of requisite expertise, but, given the dispositional characteristics of creative people, will also serve to engender motivating dissatisfaction with the current state of affairs in their particular area of interest. This intensity of interest, and its role in motivating creative work, suggests that intrinsic rather than extrinsic factors tend to inspire creativity. In fact, an extensive series of studies by Amabile and her colleagues (Amabile, 1985, 1997; Amabile, Hennessey, & Grossman, 1986; Collins & Amabile, 1999) have provided evidence indicating that intrinsic motivation is critical to creativity and innovation. Given the autonomy of creative people, these ﬁndings suggest that motivating creativity and innovation in organizational settings will, to a large extent, depend on structuring the work in such a way as to bring about personal engagement (Mumford et al., 2002b) through the use of strategies such as (1) emphasizing the purposefulness or meaningfulness of the work (Organ & Greene, 1974), (2) allowing self-selection of projects (Mumford, 2000), and (3) articulating the relevance of the problem at hand to the individuals’ areas of interest (Mumford et al., 2002b). While the available evidence underscores the importance of intrinsic motivation, a motivation that in part depends on freedom and autonomy, the role of goals, and rewards for goal attainment should not be discounted (Eisenberger & Cameron, 1996). Indeed, as noted earlier, studies by Abbey and Dickson (1983) and Cardinal (2001) indicated that the existence of goals and rewards is positively related to innovation at the organizational level. Other individual-level survey studies by Chalupsky (1953) and Shapiro (1953) examining the reasons scientists are attracted to certain jobs indicate that extrinsic rewards such as pay, promotion, and professional recognition are potential motivating factors. Experimental support for these observations has been provided by Maier and Hoffman (1964). These apparently contradictory ﬁndings with regard to intrinsic and extrinsic motivation may, however, reﬂect a rather complex set of effects. First, it should be recognized that extrinsic rewards validate, in social terms, the meaningfulness and worth of individuals’ creative efforts. Second, extrinsic

24

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

rewards provide the recognition sought by competitive, somewhat dominant, achievement-oriented individuals. Third, goals, extrinsic rewards, and goal feedback provide directive information about work progress, and this information may be essential if creative people are to direct their own work (George & Zhou, 2001). Fourth, when creativity goals are at hand, providing goals with the expectation of feedback, and thus presumably rewards, can result in enhanced creativity (Shalley, 1995). This ﬁnal point is noteworthy because it suggests that organizations must bring goals and rewards into line with creative efforts specifying generative goals and rewarding successful creative ventures, even if they do not immediately contribute to profitability. Given the relevance of both intrinsic and extrinsic motivation to creative work, one might ask what contextual conditions determine the relative emphasis that should be placed on intrinsic and extrinsic rewards. This issue has been examined by Baer et al. (2003). In a study of the employees of two manufacturing ﬁrms, they found that when complexity was low and the concern at hand was adaptive, or incremental, innovation extrinsic rewards were particularly useful. In contrast, when the complexity was high and the concern at hand required more radical innovation, intrinsic rewards were particularly useful. Thus, as creative efforts increase in structure and move toward implementation, goal speciﬁcity and extrinsic rewards may prove more useful. Of course, goals and rewards will have little value if people do not believe they can be attained. This observation would lead one to expect that efﬁcacy beliefs will play a role in motivating creativity. Tierney and Farmer (2002) developed a measure of employees’ beliefs about their capability for creative work. This measure was applied to 158 employees of a technology ﬁrm and 589 employees of a manufacturing ﬁrm, along with a measure of job self-efﬁcacy. Managerial ratings of creative performance were also obtained. The researchers found not only that creative self-efﬁcacy could be distinguished from job self-efﬁcacy but also that creative self-efﬁcacy was positively related ðr ¼ 0:17Þ to supervisors’ ratings of employee creativity. These ﬁndings suggest that organizational actions that lead people to believe in their capability for creative work (e.g., articulating a history of prior success, calling attention to unique capabilities, presenting problems as challenges, and providing requisite resources) will serve to motivate creative efforts.

Summary Taken as a whole, the available literature indicates that four key individuallevel variables – knowledge, processes, dispositional characteristics, and

Innovation in Organizations: A Multi-Level Perspective on Creativity

25

motivation – inﬂuence individuals’ willingness to engage in, and capability for, idea production and innovation. What should be recognized with regard to these variables, however, is that their expression is shaped by a number of factors operating in the organizational environment. In other words, the expression of individual-level creativity is context dependent. Indeed, creative people appear to be particularly sensitive to these contextual inﬂuences. This section examined a variety of contextual variables that are likely to shape the nature and success of individuals’ creative efforts. Some of the key conclusions ﬂowing from this review are presented in Table 1. These conclusions serve to remind us of three more general ﬁndings. First, stereotypes related to the ‘‘creative person’’ are not always accurate. Second, organizations may not only support individual-level creative efforts through time and resources, but may also shape the nature of creative thought vis-a`-vis the information made available, the problems provided, and the standards Table 1. Proposition 1

Proposition 2

Proposition 3 Proposition 4 Proposition 5

Proposition 6

Proposition 7

Proposition 8

Proposition 9

Summary of Propositions Flowing from Individual-Level Studies. Creative thought depends on expertise and the availability of high-quality, highly relevant information for experts to work out Organizational actions that reduce the resources available for gathering and working with information will inhibit creativity and innovation Creative thought involves both exploration and critical evaluation of information, ideas, and approaches Creative processing activities have a more direct inﬂuence on creative thought than basic abilities do Effective execution of requisite processing activities may be inﬂuenced by both environmental opportunities and situational demands Creative people display a pattern of dispositional characteristics consistent with the demands of creative thought but not necessarily with the demands made by creative work in organizational settings Creative people are usually sensitive to the congruence, or ﬁt, of the work environment to their pattern of differential characteristics Creative work is motivated by a mix of intrinsic and extrinsic rewards, with the balance of this mix varying as a function of the nature of the creative effort The motivation for creative work appears contingent on appraisals of the self and the work environment

26

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

applied in appraising problem solutions. Third, the motivation for creative work is complex, with both organizational-level and group-level variables operating to condition whether more basic motives will be expressed.

GROUPS Creative efforts, and especially the production of innovative new products, rarely result from the work of a single individual. Instead, ideas and innovative products emerge from individuals’ interactions with their work environment – a distinctly social environment requiring collaboration and sustained group effort. Four key attributes of the work group’s environment appear to represent noteworthy inﬂuences on creativity and innovation: (1) climate, (2) leadership, (3) process, and (4) group structure.

Climate The fact that creative people tend to react – sometimes strongly – to their immediate work environment has led students of creativity to stress the role of climate (i.e., perceptions of the work/social environment) as a noteworthy inﬂuence on creativity and innovation (Andrews & Gordon, 1970; Ellison, James, & Carron, 1970; Tesluk, Farr, & Klein, 1997). The intent of these studies has been to identify those attributes of the work environment that make creativity and innovation possible. Four distinct approaches have been used to identify these climate variables: the team, performance, context, and psychological approaches. Measurement has occurred through self-report descriptions of the individuals’ perceived work environment. The team-focused approach is illustrated in the work of West and his colleagues (Anderson & West, 1998; Bunce & West, 1995; Burningham & West, 1995; West et al., 2003). Within this approach, climate dimensions are speciﬁed based on the requirements for effective interactions among groups of people working on projects calling for change or innovation. Four dimensions are held to be critical for effective interaction: (1) participant safety, (2) support for innovation, (3) clarity of objectives, and (4) task orientation. In other words, a creative climate is one in which people are focused on well-deﬁned creative objectives, the work is supported, and people feel safe to express new ideas. The available evidence provides some rather compelling support for these propositions. Indeed, measures of these four team climate variables have produced correlations in the 0.30–0.50

Innovation in Organizations: A Multi-Level Perspective on Creativity

27

range in studies of creative performance among research and development teams and teams initiating changes in work group procedures – ﬁndings indicating that team climate is related to both process and product innovation. The performance approach differs from the team approach in that a broader framework – the job as a whole – is used to identify climate dimensions. Specifically, the environment, or work context, that differentiates more or less creative groups is used as a basis for specifying relevant climate dimensions. One illustration of this approach may be found in the work of McGourty, Tarshis, and Dominick (1996), who interviewed research and development personnel working in 14 Fortune 500 companies recognized for sustained innovation. The interview data were then content-analyzed to identify the four work context variables contributing to success: (1) inquisitiveness (e.g., searching for new ideas and technologies), (2) advocating new ideas (e.g., encouraging the idea generation efforts of others, (3) collaboration (e.g., facilitating informal relationships), and (4) goal directedness (e.g., working toward specific technological goals and objectives). In a quantitative study applying this approach, Baer and Frese (2003) developed a climate inventory to measure initiative (e.g., people in our company quickly size up opportunities in an effort to attain goals) and psychological safety (e.g., no one in the company would undermine other people’s efforts). In a study of senior managers in 37 German ﬁrms adopting process innovations, they found not only that these climate dimensions interacted with organizational and economic variables in conditioning adoption but also that climate and adoption were related to ﬁrm profitability. In another set of studies applying the performance approach, Oldham and his colleagues (Baer et al., 2003; Oldham & Cummings, 1996) examined work environments with respect to job complexity, noncontrolling supervision, and supervisory support. They found that job complexity, noncontrolling supervision, and supervisory support were all related ðr ¼ 0:30Þ to contributions to a suggestion system, whereas job complexity was related ðr ¼ 0:27Þ to patent awards. While the performance approach stresses behavior and work conditions, the context approach focuses on broader environmental attributes that support (or inhibit) creative efforts. Perhaps the best illustration of this approach may be found in the work of Amabile and her colleagues (Amabile & Conti, 1999; Amabile et al., 1996; Amabile & Gryskiewicz, 1989). Their climate inventory provides measures of (1) organizational encouragement, (2) supervisory encouragement, (3) workgroup supports, (4) sufﬁcient resources, (5) challenging work, (6) organizational impediments, (7) freedom, and (8) workload pressure. Scores on scales measuring these variables have been shown to distinguish among more and less successful project

28

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

development efforts (Amabile et al., 1996). Moreover, changes in broader patterns of organizational operations – for example, downsizing – have been shown to be linked to changes in scores on these climate dimensions (Amabile & Conti, 1999). The fourth approach used in climate studies appraises the work environment in terms of the dispositional characteristics of creative people. This approach is illustrated in the work of Ekvall (1996) and Isaksen, Lauer, and Ekvall (1999). The latter group developed a climate inventory measuring (1) challenge, (2) freedom, (3) idea support, (4) trust/openness, (5) dynamism/ liveliness, (6) playfulness/humor, (7) debates, (8) low conﬂict, (9) risk taking, and (10) idea time. Scores on a self-report measure intended to provide measures of these variables have been shown to (1) correlate with productivity in university departments, (2) differentiate more innovative ﬁrms from less innovative ﬁrms, and (3) differentiate more and less innovative divisions within a ﬁrm (Ekvall, 1996; Ekvall & Ryhammer, 1999). Apparently, all four aspects of climate – team, performance, context, and psychological factors – inﬂuence creativity and innovation, typically producing multiple correlations in the 0.30–0.50 range. This conclusion, of course, begs a question: Is one aspect of climate more important than another? Although we lack evidence bearing on this question, it seems reasonable to argue that all four aspects of climate should work in synchrony if one is to maximize creativity and innovation. Indeed, incongruence among these aspects of climate may prove disruptive. A related question broached by these ﬁndings is this: What dimensions of climate are most important to creativity and innovation? Bain et al. (2001) found, in a study of 38 research and development teams, that task orientation and team support were critical inﬂuences on innovation. These observations are consistent with the ﬁndings of Amabile et al. (1996) concerning the importance of challenging work, workgroup support, and freedom. Thamhain’s (2003) study of product development efforts also points to the importance of stimulating work, accomplishment, and workgroup support. Thus, it appears that a setting where people are undertaking complex, challenging work in a context characterized by interpersonal support and collaboration is essential for creativity and innovation. Given the curiosity, achievement motivation, and professional orientation of creative people, it is not surprising that challenge is a necessary aspect of their work environment. What is surprising, however, is the strong effects exerted by workgroup support – an unexpected ﬁnding given the autonomy and competitiveness of creative people. This ﬁnding cannot be accounted for simply on the basis of the particular methodological approach applied in climate studies. For

Innovation in Organizations: A Multi-Level Perspective on Creativity

29

example, in a study of turnover among research and development personnel, Farris (1972) found that embeddedness in a network of focused, highly interactive colleagues was related to performance, satisfaction, and lower turnover among scientists and engineers working in industrial settings. Working along similar lines, Ayres, Dahlstrom, and Skinner (1997), in a study of new product introductions, found that relational norms – specifically, solidarity (treatment of problems as matters of mutual concern), conﬂict harmonization (the ability to resolve disputes internally), and ﬂexibility (mutual adjustment of behavior) – were related to the successes of new product introductions. The apparent inﬂuence of workgroup support on innovation might be accounted for in any one of the four ways. First, other workgroup members provide requisite expertise and fresh approaches to problems. Second, workgroups are the most likely source of collaborators. Third, a supportive workgroup provides the critical feedback needed for creative thought under conditions where threat is low and participative safety is high. Fourth, workgroup support ameliorates the stress and burdens associated with most forms of creative work. These observations about workgroup support and challenge imply three noteworthy interactive effects. First, organizations must deﬁne and support challenging, potentially innovative projects, despite the risk attached to highchallenge efforts (Nystrom, 1990). Second, the value of, and necessity for, collaboration in meeting mission challenges must be emphasized. Third, organizations should try to use the shared values and norms that characterize professions as well as the core values and norms of the organization as a basis for encouraging collaboration. Of course, deﬁning objectives and establishing interactional norms is typically the province of people occupying leadership roles. In one study examining leader–climate interactions, Gonzalez-Roma, Peiro, and Tordera (2002) found that leaders’ informing activities interacted with three key climate variables – namely, support, goal orientation, and innovation – in determining satisfaction and commitment. Pirola-Merlo, Haertel, and Mann (2002) found that leader behavior inﬂuenced climate, which in turn inﬂuenced project performance in a sample of 54 research and development teams.

Leadership In studies of creativity and innovation, the role of leaders has traditionally been viewed as passive and supportive (Sessa, 1998). The available evidence, however, indicates that leaders may play a more direct, and a more important, role in shaping the success of creative efforts than has been typically

30

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

assumed. Creative efforts, by virtue of their novelty and complexity, tend to be ill deﬁned or poorly structured. These efforts, moreover, often involve multiple parties interacting in a rather chaotic environment. Under these conditions, there is a need for leaders who can provide the guidance that helps people structure their creative problem-solving activities (Trevelyan, 2001). Some support for this proposition may be found in Drazin, Glynn, and Kazanjian (1999) and Kazanjian, Drazin, and Glynn (2000). In a qualitative study of new aircraft development projects, they found that technical complexity and interaction among multiple teams gave rise to crises – crises that, in fact, appear endemic to most creative efforts. Project success depends on crisis resolution, with effective resolution requiring sense-making activities on the part of leaders whereby the origins and significance of events are articulated to team members. Leader sense-making has also been found to inﬂuence performance in other studies examining technical (e.g., Kidder, 1981) and artistic (e.g., Dunham & Freeman, 2000) work. Sense-making activities of the sort described above require two forms of expertise. First, the leaders of creative efforts will need a broad understanding of the organization, its strategy, business practices, and sociopolitical structure (Mumford, 2000). This point is of some importance because it suggests that isolating the leaders of creative efforts can inhibit innovation (Cohen & Levinthal, 1990). Second, because crises are often sociotechnical in nature, the leaders of creative efforts will need technical expertise. The need for technical expertise has been demonstrated in a study by Andrews and Farris (1967). They had 94 scientists working in 21 teams assess their leaders with respect to technical skills, critical evaluation, motivation of others, and autonomy granted. Leader technical skills ðr ¼ 0:53Þ were found to be the best predictor of team performance. Barnowe (1975), in a study of 963 chemists working in 51 research and development teams, found that leader technical skills ðr ¼ 0:40Þ were better predictors of team creativity and innovation than support, participation, closeness of supervision, and task emphasis. Leader technical expertise may exert indirect effects on team performance – for example, by providing the leader with a basis for appraising follower capabilities, creating an awareness of professional expectations, and providing a basis for the effective exercise of power (Mumford et al., 2002b). This expertise, however, appears to exert a more direct effect on innovation. In a study examining when research and development personnel communicated with team leaders, Farris (1972) found that followers sought leader input during initial definition of the problem and after the ﬁrst round of

Innovation in Organizations: A Multi-Level Perspective on Creativity

31

work was completed. These ﬁndings, along with those of Mumford, Connelly, and Gaddis (2003), suggest that effective leaders must be able to shape follower-generated ideas if they are to avoid the problems associated with overly close supervision and to appraise ideas in such a way as to maximize their utility to the organization – requirements that also impose a need for creative thinking skills (Tierney, Farmer, & Graen, 1999). Leaders must not merely direct, but also motivate. Vision is the key mechanism held to be used by outstanding leaders (e.g., charismatic and transformational leaders) in motivating followers. Unlike leadership in other domains of endeavor, however, visioning does not appear to motivate creative people (Jaussi & Dionne, 2003; Kahi, Sosik, & Avolio, 2003), although charismatic and transformational leaders appear more likely to accept and support innovation (Jung, Chow, & Wa, 2003). These effects appear to arise from the restrictions that a leader’s vision places on followers’ pursuit of their own unique, creative visions. Although vision may not motivate creative people, it does appear that the definition of missions – that is, broad, challenging, technical/professional projects – is a viable motivational technique (Mumford et al., 2002b). Definition and articulation of meaningful missions by leaders may, moreover, be a noteworthy inﬂuence on development of a collaborative, supportive workgroup. Leaders also inﬂuence climate and effective team interaction through role modeling. In a rare study examining the effects of leader role modeling on group creative problem solving, Jaussi and Dionne (2003) found that groups produced more creative products when confederate leaders modeled unconventional behavior. Presumably, leader role modeling of other requisite behaviors, such as the need for interaction with other groups, collaboration with colleagues, and service to the group enterprise, will exert similar effects (Mouly & Sankaran, 1999). By the same token, leaders might ﬁnd it difﬁcult to model these kinds of behaviors if they are inconsistent with the broader organizational culture, especially normative expectations for managerial behavior. When these inconsistencies arise, the successful leadership of creative ventures may require people who can ﬂexibly adapt to, and smoothly manage transitions between, multiple roles (Hunt, Stelluto, & Hooijberg, 2004). How leaders interact with and manage the group on a day-to-day basis also inﬂuences innovation. A series of studies by Maier and his colleagues (Hoffman, Hamburg, & Maier, 1962; Maier, 1950, 1953; Maier & Hoffman, 1960; Maier & Solem, 1962) has examined the strategies that should be used by leaders to manage groups working on business-based creative problemsolving tasks. The ﬁndings obtained in these studies indicate that leaders

32

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

should (1) request a creative or innovative solution; (2) frame the problem in terms of broader issues rather than specify a particular outcome being sought; (3) encourage group members to search for and consider a range of information; (4) avoid premature evaluation; (5) extend discussion to allow for the generation of multiple ideas; and (6) use disagreements to frame integrative solutions. Studies of innovation among research and development teams by Arvey, Dewhurst, and Boling (1976), Elkins and Keller (2003), and Mossholder and Dewhurst (1980) indicate that these kinds of tactics, which induce intellectual stimulation and participation, contribute to satisfaction, involvement, and performance.

Process The preceding observations, of course, also indicate that leaders will have a significant effect on group processes. Although process studies have been relatively rare, the available evidence does underscore the relevance of the group process to understanding creativity and innovation. In one such study, Taggar (2002) examined performance on group creative problemsolving tasks with respect to seven process variables: (1) team citizenship, (2) performance management, (3) effective communication, (4) involving others, (5) providing feedback, (6) reaction to conﬂict, and (7) averting conﬂict. These process variables were found to make a unique contribution to predicting creative problem solving even when individual-level inﬂuences were taken into account. While undue conﬂict within a group is likely to inhibit creativity and innovation, drawing resources away from idea generation and idea implementation (a type of conﬂict noted earlier) may not always be undesirable. Conﬂicting analyses that challenge assumptions and approaches can stimulate creative thought, at least when the debate centers on ideas and approaches rather than on people and their performance (Schwenk & Thomas, 1973). In keeping with this proposition, Frankwick, Walker, and Ward (1994), in a study examining the development of new marketing plans, found that articulation of differences in assumptions among marketing and research development managers promoted the synthesis needed for the generation of creative ideas. Debate about assumptions and approaches is unlikely to prove productive unless the parties involved share a common mission or general understanding of the issues at hand. This point was illustrated in a study by Mumford, Feldman, Hein, and Nago (2001). They had undergraduates

Innovation in Organizations: A Multi-Level Perspective on Creativity

33

work on a group creative problem-solving task where some groups were provided with task-relevant training, some with task-irrelevant training, and some with no training. Training was intended to induce shared mental models among group members. It was found that training, including taskirrelevant training, contributed to idea generation, presumably because these shared models facilitated group processes such as communication. Effective within-group communication has been found to contribute to innovation in a variety of creative efforts, ranging from jazz performances to new product introductions (Bastien & Hostager, 1988; Cooper & Kleinschmidt, 2000; Ebaldi & Utterback, 1984; Thamhain, 2003). In organizational settings, however, communication must also extend beyond group boundaries. In one study along these lines, Andrews and Smith (1996) asked marketing managers to describe the behaviors observed among team members working on an advertising campaign. Both the consumers’ and project managers’ evaluations of campaign creativity were found to be positively related ðr ¼ 0:30Þ to external communication. Similar ﬁndings were obtained by Anacona and Caldwell (1992) in a study of 47 product development teams. Specifically, they found that external communication by team members contributed to ratings of team performance, budget and schedule performance, and the creativity of the product produced. These ﬁndings are of some importance because they suggest that if the group becomes so highly cohesive that external communication ceases, creativity and innovation will suffer (Nystrom, 1979; Scott & Bruce, 1994). While extremely high levels of cohesiveness may inhibit creativity, in part due to lack of communication, and in part due to discounting new ideas, especially ideas from outside the group (Gerstenberger & Allen, 1968), cohesiveness in general appears to contribute to creativity (Keller, 1989). Cohesiveness and spending time in the group make group processes more effective, inducing shared models and an awareness of other members’ needs and skills. Given these effects, it is not surprising that Cooper and Kleinschmidt (2000), in their study of new product introductions, found that project performance improved when team members devoted a substantial proportion of their time to the project and were involved in the project over its full duration. Two effects of cohesiveness are more rapid decision making and product development (Kessler & Chakrabarti, 1996). In addition, cohesiveness builds trust and liking among group members. Trust not only makes it possible to express untried ideas, thereby promoting creativity, but also ensures that the social support needed for the development and ﬁelding of these ideas will be available (Bouty, 2000; Madjar, Oldham, & Pratt, 2002).

34

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Because creative efforts are difﬁcult, demanding undertakings, their success will depend on commitment – not merely to the mission but to the group as well (Sethi & Nicholson, 2001). Recognition of this point led James, Clark, and Cropanzano (1999) to argue that organizational commitment and citizenship may be necessary for creativity and innovation (Keller, 1997). This need for cohesion and team citizenship is inconsistent with the autonomy, competitiveness, and individual achievement concerns that characterize creative people. This point suggests that cohesion and citizenship may prove particularly difﬁcult to develop in teams working on creative efforts. One strategy for addressing this issue was suggested by Pelz (1968). He argued that contact around the work will prove useful particularly in building cohesion, especially when resources and recognition are distributed in an equitable fashion that recognizes contributions to the group (Abbey & Dickson, 1983; James et al., 1999; McFadzean, 2002). Of course, as Albrecht and Roop (1984) point out, all contact – including social contact – may have some value in this regard. Finally, by articulating the interdependency of roles with regard to a mission, leaders can build a sense of shared interest and thus cohesion and commitment (Dunham & Freeman, 2000). Structure Team processes, of course, depend to some extent on team structure. The ﬁrst and perhaps most straightforward question that arises in this regard concerns optimal team size. Unlike in organizations, where size, if the organization is appropriately structured, may enhance innovation (Arad, Hanson, & Schneider, 1997; Chandy & Tellis, 1998), larger team size is negatively related to creativity and innovation. Curral, Forrester, Dawson, and West (2001) obtained managerial ratings of team creative performance for 87 teams ranging in size from 2 to 18 individuals. On the basis of these ratings, teams were classiﬁed as more or less innovative. Additionally, the members of these groups were asked to complete a measure of team climate examining participant safety, support for innovation, clarity of objectives, and emphasis on quality. The researchers found that larger teams were less likely to be innovative, a phenomenon attributable to the tendency of larger teams to evidence a poorer climate when working under pressure. As Steck and Sundermann (1978) point out, however, in small teams consisting of 2 or 3 individuals, creativity will suffer due to a lack of requisite expertise. Thus, a team size ranging from 4 to 7 individuals appears optimal, although larger teams may still prove effective when cohesion is high, substantial incentives for cooperation exist, and the

Innovation in Organizations: A Multi-Level Perspective on Creativity

35

group (or the organization) is characterized by a strong climate stressing creativity. Larger teams are often required to ensure the availability of requisite expertise. As a result, the question arises as to how multiple teams working on complex projects should be structured. Gassman and van Zedwitz (2003) interviewed research and development directors working in 37 large technology-driven companies concerning team structure. They found that four team structures were commonly used in these companies, with the structures varying with respect to their degree of centralization: (1) decentralized selforganizing teams, (2) teams with a system integration coordinator, (3) a core team structure where an architectural team directs other teams working on some aspect of the larger project, and (4) centralized venture teams. Moreover, analysis of these interview data in relation to team performance revealed some noteworthy ﬁndings. Ad hoc teams proved effective for incremental innovations when complexity and resource requirements were low. As complexity and resource requirements increased, both group size and centralization increased. Under these conditions, successful efforts were characterized by a core design team or venture group directing the efforts of a number of contributing teams. These core teams, due to the need for multiple forms of expertise, were often structured as multifunctional teams. In fact, virtually all studies examining the factors that contribute to the success of product development teams stress the need for multifunctional structure (Brown & Eisenhardt, 1995; Cooper & Kleinschmidt, 2000; Grifﬁn, 1997; Lovelace, Shapiro, & Weingart, 2001; Thamhain, 2003). While the available evidence indicates that multifunctional structure contributes to product development efforts with respect to criteria such as speed, quality, and innovation, two points should be borne in mind. First, if multifunctional teams are to prove successful, an investment in both climate enhancement and process development will be required. This point is illustrated in a study by Lovelace et al. (2001). In an investigation focusing on 43 teams involved in technology development efforts, they found that intrateam disagreements (i.e., disagreements arising from differences in functional perspective) led to lower performance, particularly when doubts could not be expressed and addressed collaboratively due to lack of trust, balkanization, and politics. Second, it appears that leaders of multifunctional teams must not only build a sense of trust, but also manage stress and workload. This conclusion ﬁnds some support in a study by Keller (2001). He examined the performance of 93 research and development teams with respect to technical quality, budget performance, and schedule performance. Functional diversity was found to result

36

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

in reduced cohesiveness, and this loss of cohesiveness was in turn associated with increased stress and poor internal communication. These negative effects on process, moreover, suggest planning may be at premium when multifunctional structures are employed (Mumford, Schultz, & Osburn, 2002a). These observations indicating that process loss is associated with the use of multifunctional structures suggest that multifunctional teams should not be employed unless diversity in expertise is required vis-a`-vis project needs. This point is nicely illustrated in a study by Cardinal and Hatﬁeld (2000). They contrasted drug enhancements and new drug introductions across 110 pharmaceutical ﬁrms. Their study found that separation of research facilities from corporate headquarters, and thus cross-functional exposure, was related to the number of new drug introductions. In contrast, drug enhancements appeared to beneﬁt from closer ties to corporate headquarters. The effects of process loss are associated with a ﬁnal phenomenon relevant to the structure of creative groups. At ﬁrst glance, one would assume that team creativity and innovation would increase as the number of creative people in the group increases. In fact, as Taggar (2001) has shown, groups composed primarily of highly creative people often exhibit poorer performance on creative problem-solving tasks than groups that include just a few creative members. These effects arise from the loss of focus associated with having a large number of diverse ideas. Moreover, they suggest that structuring, planning, and evaluation will often be at a premium in groups composed of a number of highly creative people – a point attested to in even the most cursory review of the history of the atom bomb.

Summary Our examination of group-level inﬂuences on creativity and innovation indicates that climate, leadership, group process, and group structure all represent potentially noteworthy inﬂuences on performance. In fact, grouplevel variables seem to be as powerful an inﬂuence on performance as individual-level variables do. Given the fact that real-world creative work is often collaborative work, the effects of group-level variables on creativity and innovation should not be especially surprising. By the same token, the inﬂuence of group-level variables on creativity and innovation appears rather complex. This point is illustrated in Table 2, which summarizes some of the key propositions ﬂowing from our review of group-level inﬂuences. Climate variables and leadership tactics that ensure guidance and direction, both of which are required as multiple parties work

Innovation in Organizations: A Multi-Level Perspective on Creativity

Table 2. Proposition 1 Proposition 2 Proposition 3

Proposition 4 Proposition 5 Proposition 6

Proposition 7 Proposition 8

37

Summary of Propositions Flowing from Group-Level Studies. The creative climate has multiple distinct aspects that should work in synchrony if creativity and innovation are to be maximized Task focus on a strong, clear mission along with peer support are particularly powerful climate inﬂuences operating at the group level Leaders of creative groups should seek to provide a clear technical mission and engage in sense-making activities that serve to provide guidance and manage crises Leader expertise is an important inﬂuence on the performance of creative groups Creativity and innovation will increase in cohesive groups where trust, collaboration, and a sense of shared mission are evident Creativity and innovation will increase in groups, where there is a high level of communication and information sharing across functional areas The use of structures that allow for integration of multiple small teams will prove useful in encouraging collaboration and communication Intellectual stimulation arising from debate in a supportive environment will prove beneﬁcial while non-task-based conﬂict will prove detrimental to creativity and innovation

on ill-deﬁned tasks, clearly represent noteworthy inﬂuences on innovation. Likewise, creativity and innovation appear to ﬂower in small cohesive teams where support, trust, and a shared sense of mission are evident and multiple forms of expertise are available and shared. One caveat should be recognized: Many of the requirements for innovation applying at the group level are not necessarily consistent with those observed at the individual level. For example, while creative efforts require cohesion and collaboration at the group level, autonomous, competitive, critical individuals may ﬁnd it difﬁcult to work with others. Along similar lines, although many ideas may be valuable at the individual level, a few good ideas appear more valuable at the group level.

ORGANIZATIONS Our earlier observations with respect to the need to manage idea generation would lead one to expect that organizational-level variables would exert some noteworthy inﬂuences in terms of creativity and innovation. In fact, numerous studies have examined organizational-level inﬂuences on creativity and innovation (e.g., Damanpour, 1991; Pierce & Delbecq, 1977).

38

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Broadly speaking, the results obtained in these studies indicate that creativity and innovation are inﬂuenced by variables that can be subsumed under four rubrics: (1) control, (2) resources, (3) advocacy, and (4) structure.

Control Creative ideas represent new untried ideas, many – indeed most – of which can be expected to fail (Huber, 1998; Sharma, 1999). To develop ideas, organizations must commit often substantial time and resources to the effort. The costs and risks associated with development of a new idea are not, however, simply a matter of investment in the idea. The development of new ideas may prove disruptive with respect to current organizational operations (Germain, 1996), thereby posing the problem of efﬁciency losses. As a result of these considerations, organizations cannot simply pursue ideas – even if those ideas are truly valuable. Instead, controls must be imposed to determine ideas will be pursued, when they will be pursued, and how they will be pursued. One of the most powerful and persuasive controls imposed by organizations lies in the strategy being pursued. Hadjimanolis (2000) examined innovation across 140 small and mid-size ﬁrms where innovation at the ﬁrm level was operationalized in terms of the more rapid introduction of new products. He found that a corporate strategy stressing innovation was positively related ðr ¼ 0:32Þ to ﬁrm innovation. Ettlie (1983) examined strategy in terms of the specific areas that ﬁrms emphasized. He found that ﬁrms stressing innovation with respect to customers were not especially likely to prove innovative. In contrast, ﬁrms that had policies stressing technological and market innovation were more likely to innovate in terms of both process and product ðr ¼ 0:2020:40Þ It should be noted that a strategy that recognizes the value of innovation does not necessarily ensure that the organization itself will prove innovative. Capon, Farley, Lehman, and Hulbert (1992) assessed 113 Fortune 500 companies with respect to strategy, environment, organization, and innovation. Subsequent clustering of ﬁrms into organizational types based on these variables yielded two types that proved to be successful in their attempts to profit from innovation. One type followed an internal development strategy, whereas the other type followed an acquisition strategy. As might be expected, ﬁrms following an internal development strategy invested substantially more in research and development than did ﬁrms following an acquisition strategy.

Innovation in Organizations: A Multi-Level Perspective on Creativity

39

Strategy not only inﬂuences investments, but also shapes the kind of controls applied by an organization. The significance of these controls, or decision standards, with respect to innovation was illustrated in a study by Hitt et al. (1996). They found that ﬁrms employing ﬁnancial controls were less likely to produce innovative products than ﬁrms employing strategic, or business development, controls. Because creative ideas are untried and typically require expensive development, application of ﬁnancial controls will lead to the rejection of new ideas as part of the pursuit of efﬁciency and profits. The effects of applying ﬁnancial controls, however, may be more subtle and pervasive: The burden of proof placed on ideas may become excessive, processes applied in workforce management may inhibit communication and collaboration, and the exploration of ideas and their implications may be restricted. One must also remember that several types of strategic controls might be applied. Strategic evaluation criteria might stress market growth, customer satisfaction, technological advantage, or other measures. These differences in strategic control emphasis may give rise to differences in the types of innovation that emerge as a result of both direct mechanisms (e.g., evaluations of and investments in certain kinds of new ideas) and indirect mechanisms (e.g., kind of expertise available and the backgrounds of senior management). Two further points should be kept in mind when considering the inﬂuence of strategic controls on innovation. First, controls (either strategic or ﬁnancial) shape the goals set by the organization and the rewards provided for attainment of those goals. As suggested by Abbey and Dickson (1983), organizational goals and rewards can be a powerful inﬂuence on innovation at the organizational level. Second, the standards applied in evaluation exert a noteworthy inﬂuence over culture vis-a`-vis modeling and feedback, and culture will condition not only the climate’s ability to promote creativity but also reactions to creative ideas (Galbraith, 1982). Of course, in most organizations, the evaluation of ideas will eventually involve rather concrete pragmatic considerations. In the case of ‘‘forprofit’’ ﬁrms, profitability, market size, customer needs, service/production costs, and synergies have been found to inﬂuence the launch of a new product (Grifﬁn, 1997). What must be remembered is that these objective standards can be applied only relatively late in the idea development cycle due to the complexity and ambiguity inherent in any new idea. This point led Cooper and Kleinschmidt (1991) to argue that multiple evaluations should occur in determining whether an organization should pursue an idea, with these evaluations moving from initial feasibility

40

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

assessments to more specific business and market assessments as development of the idea proceeds. The ambiguity inherent in these early evaluations inspires a new question: What characteristics of new ideas lead organizations to be willing to invest in their development? Chandy and Tellis (1998) examined this question through a phenomenon they refer to as cannibalization – giving up market share on an existing product for the sake of a new product. They found that willingness to cannibalize was related to ﬁrms’ production of radical innovations. Cannibalization, in turn, was strongly inﬂuenced by future market focus. Along similar lines, Avlonitis, Kouremenous, and Tzokas (1994) have provided evidence indicating that future orientation with respect to technology, materials, and production methods is related to ﬁrm innovation. Given how a future orientation may affect the willingness of ﬁrms to pursue new ideas, it is not surprising that environmental scanning is related to innovation (Kim, Song, & Lee, 1993; Porac & Thomas, 1990). However, interpretation of the information obtained through scanning will require the ready availability of people who can both interpret the implications of this information and inﬂuence organizational policy (Cohen & Levinthal, 1990). In a qualitative study of eight cases of radical product innovations, O’Connor (1998) found that the development of these products was contingent on the availability and involvement of senior managers who understood and could articulate the downstream impact of an idea on markets, capabilities, and consumers – that is, creative executives who can recognize emergent opportunities (Kickul & Gundry, 2001). In addition to forecasting, at least two other variables inﬂuence the willingness of ﬁrms to pursue new ideas: (1) ﬁt and (2) the conditions confronting senior managers. The effects of ﬁt on appraisals of ideas were illustrated in a study by Dougherty and Heller (1994). They conducted interviews with people involved in one or more new product development efforts. Their interview data were then used to identify organizational practices that inhibited innovation. These researchers found that poor ﬁt of an idea with respect to structure, strategy, and climate was a common reason for negative evaluation. In a quantitative study examining go/no-go decisions for 262 product development efforts, Danneels and Kleinschmidt (2001) found that market and technology ﬁt and market and technology familiarity were positively related ðr ¼ 0:2020:40Þ to decisions to pursue an idea. Given the fact that innovation and idea development depend on ﬁrm-specific knowledge and the fact that development and introduction will depend on the ﬁrm’s core capabilities and available support structures (Hargadon & Douglas, 2001), it is not surprising

Innovation in Organizations: A Multi-Level Perspective on Creativity

41

that ﬁt is an important consideration in appraising ideas (Brown & Eisenhardt, 1995). While ﬁt assessments are undoubtedly necessary, they are not without problems. Particularly when radical, highly ambiguous, long-term development efforts are under consideration, such assessments will involve substantial ambiguity. Ambiguity allows subjective appraisals to impinge on ﬁt assessments, including political positioning, apparent credibility of the people involved, and track record, among other things – all extraneous sources of information that might be used to reduce ambiguity. Moreover, under conditions of ambiguity and complexity, articulation of how an idea ﬁts will prove difﬁcult. As a consequence, communication skills will be at a premium and conditions that make communication difﬁcult, such as isolation of a research and development group, will block idea acceptance. Finally, because ﬁt assessments depend on a comparison to the ‘‘as is’’ state of the organization, when confronted with turbulence or radical innovation, the ﬁrm’s ﬁt assessments may prove ineffective unless future conditions are explicitly considered as part of the assessment. These observations would lead one to expect that the conditions confronting senior managers at the time when an idea is presented will also inﬂuence their decisions with regard to creative ideas. In a study of 51 senior executives, Ford and Gioia (2000) asked participants to identify more and less creative decisions and more and less valuable decisions. These executives were also asked to complete a self-report inventory describing the content surrounding these decisions. The researchers found that trust and the importance of the issue contributed to high-value decisions, whereas a common perspective, negative feedback, and feasibility contributed to creative decisions. Apparently, when executives have decision-making latitude, perceive a solution to an immediate problem, and feel supported, they are more likely to make creative decisions and, by extrapolation, to support the development of creative ideas.

Resources Resources provide executives with ﬂexibility. Not only do resources provide the ﬂexibility needed to pursue promising ideas, but without resources it is unlikely that even the most promising idea can be pursued. The development of new ideas is a time-intensive, expensive undertaking in all organizations. As a result, it can be expected that resource availability will exert a powerful inﬂuence on innovation. Some support for this proposition may be found in Klein, Conn, and Sorra (2001). In a study examining adoption of

42

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

computerized manufacturing technologies, they found that successful adoption depends not only on management support but also on available ﬁnancial resources. Dougherty and Hardy (1996) conducted interviews with personnel involved in 40 new product development efforts. In keeping with the observations of Klein et al. (2001), they found that sustained resource availability – that is, resource availability over the course of the product development effort – was critical for project success. Along similar lines, Henderson (1993), in a quantitative study of innovation in the photo-lithographic industry, found that the failure of established ﬁrms to exploit new technologies was partly due to a pattern of underinvesting in new technology. If resources are critical to innovation, then one would expect that ﬁrm size, by virtue of the resources available to a larger ﬁrm, would be related to innovation – in particular, to resource-intensive radical innovations (Meyer & Goes, 1988). Some rather compelling support for this proposition has been provided by Chandy and Tellis (2000). In a historical study of 63 radical innovations in the 20th century (e.g., the transistor), they found that radical innovations were more likely to emerge from large, rather than small, ﬁrms. In a comparison of large and small banks, Gopalakrishnan and Damanpour (2000) found that large banks were more likely to adopt radical innovations but moved less quickly to exploit incremental innovations due to their greater structural rigidity. Although the bulk of the available evidence indicates that resources are needed for innovation, especially complex radical innovations, not all studies have observed a positive relationship between resources and innovation (Scott & Bruce, 1994). One possible explanation for these negative effects may be found in the relatively low cost of some incremental innovations. Another putative explanation may be found in the nature of the relationship between resources and innovation. Graves and Lanowitz (1993) examined research and development expenditures in 16 pharmaceutical companies over a 19-year period. They found that past a certain point, increased research and development expenditures did not necessarily result in the production of more new products. Nohria and Gulati (1996) obtained similar results, demonstrating the existence of a curvilinear relationship between research and development expenditures and innovation, in a study of 264 departments within two large multinational ﬁrms. One explanation for this curvilinear relationship is that excessive resources lead organizations to pursue marginal ideas. The pursuit of marginal ideas may, in turn, lead to a loss of focus, resulting in inappropriate allocation of talent and failure to address the development issues necessary to ensure success.

Innovation in Organizations: A Multi-Level Perspective on Creativity

43

These observations underscore the importance of effective distribution and management of available resources. These endeavors will, however, be complicated by another aspect of creative efforts. Creative ideas are often exciting ideas. Moreover, prior to the initial proposal of an idea, people and groups will often have made a substantial investment in the idea. As a result of excitement, investment, and accompanying involvement, optimistic biases are likely to arise (Mumford, Blair, Dailey, Leritz, & Osburn, 2003), which may lead people to underestimate time requirements, resource requirements, and the difﬁculties likely to be encountered in idea development (Buehler, Grifﬁn, & Ross, 1994; Langholtz, Gettys, & Foote, 1995; Mumford et al., 2002a). The existence of these estimation biases has four noteworthy implications for resource management by organizations. First, some degree of skepticism will be required in evaluating proposals, and checks will need to be put in place at the organizational level to ensure that realistic estimates are considered in idea evaluation. Second, plans for developing ideas must include some slack with respect to time and budget (Judge, Fryxell, & Dooley, 1997). Third, caution must be applied when schedule performance is used as a basis for evaluating project performance, with evaluations being made based on reasonable progress rather than the attainment of a priori milestones. Fourth, ongoing monitoring to track reasonable progress with respect to business objectives appears necessary for the successful direction of innovative efforts and effective allocation of resources (Lewis, Welsh, Dehler, & Green, 2002).

Advocacy As might be expected given the need for resources, the success of creative efforts in organizations depends on top management support and the willingness of senior managers to work as advocates for the effort (Jelinek & Schoonhoven, 1990). In one study examining top management support, Meyer and Goes (1988) focused on the adoption of 12 medical innovations by 25 hospitals. They found that CEO advocacy was positively related ðr ¼ 0:28Þ to adoption of the innovation and its routine use by hospital staff. Not only is top management support relevant to the adoption of innovations, but it also appears critical to the generation of innovative new products. In one study related to this issue, Maidique and Zirger (1984) surveyed technical, Professional, and marketing managers involved in 158 new product introductions in the electronics industry. They found that top management

44

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

support (i.e., support sustained throughout the course of the project) was necessary for the successful development and introduction of new products. Top management support is not simply a matter of being willing to approve, and provide resources for, creative efforts. Instead, the success of creative ventures appears to depend on the active engagement and involvement of top managers in the project. Ong, Wan, and Chang (2003) examined leadership support for innovation through a self-report inventory stressing engagement and contact. They found that both the production of ideas and the implementation of these ideas was positively related to top management contact and engagement. In their qualitative study of new product introductions, Dougherty and Hardy (1996) found that the active engagement of top management was necessary, with management engagement being essential to remove organizational blockages. Furthermore, Ramus (2001) has provided evidence indicating that visible support by top management not only serves to remove obstacles but also signals to the rest of the organization that the effort has both value and importance. Given the apparent impact of top management support on innovation, the question arises as to the conditions under which top management will actively support creative efforts. Of course, strategic considerations and ﬁt can be expected to inﬂuence the willingness of top managers to become actively involved in, and to actively support, a project. In addition, a number of other, more complex considerations appear to inﬂuence top management support. One of these inﬂuences is the ability of the top management team to understand the idea and its implications for the business. Daellenbach, McCarthy, and Shoenecker (1999), in a study of commitment to technical innovation, found that support for innovation, as operationalized in terms of research and development investments, was positively related to the technical background of the top management teams. Of course, in many cases, top management will not necessarily have the background needed to fully understand the implications of a new idea. Moreover, the risk attached to creative ideas may lead to its premature rejection, even given an understanding of the idea, especially when ﬁt is ambiguous and cost high. Under these conditions, the active championing of the idea by other members of the organization may prove critical to building support and acquiring requisite resources. This support is often obtained through the efforts of product champions. Product champions are people outside the group working on the project (typically senior managers), who recognize the significance of the idea and actively build social and ﬁscal support for it (Howell & Higgins, 1988; Markham & Aiman-Smith, 2001). In keeping with the need for support, studies of new

Innovation in Organizations: A Multi-Level Perspective on Creativity

45

product introductions indicate that successful introductions typically have one or more champions who have taken an active role in building support for the effort (Markham, Green, & Basu, 1991; Markham & Grifﬁn, 1998). Most commonly, these product champions are politically skilled and politically wellpositioned visionary leaders who emerge from groups that have a vested interest in the innovation (Markham & Aiman-Smith, 2001). The role of product champions, however, is complex and demanding. Howell and Boies (2004) contrasted 19 matched pairs of champions and nonchampions involved in one of 28 technological innovations. Interviews with these individuals were content-analyzed with regard to organizational involvement, organizational support, and idea promotion. The researchers found, in a partial least-squares analysis, that contextual and normative organizational knowledge (particularly contextual knowledge) was needed for idea promotion – specifically, idea packaging. Flexible role orientation contributed to idea-generation activities, involvement, and support, which in turn contributed to idea promotion as couched in terms of project sales activities. Thus champions must both package and sell ideas. A similar point has been made by Hauschildt and Kirchmann (2001). They argue that championing involves three distinct roles: technology promotion (providing technical expertise), process promotion (providing organizational know-how), and power promotion (providing sponsorship in the organizational hierarchy). In a study of 133 innovations in German construction and engineering ﬁrms, these researchers found that when all three roles were ﬁlled, greater ﬁnancial and technical success was observed vis-a`-vis projects that had either no champion or only one champion. What should be recognized here is that the emergence of champion teams – or, for that matter, a single champion who attempts to combine these roles – is by no means assured in an organizational setting. For example, if external communication and contact (i.e., outside the project team) is discouraged, then it is unlikely that champions will be recruited. Moreover, champion involvement may require a top management culture that supports entrepreneurial efforts along with a business strategy that recognizes the value of innovation. Finally, structure, by determining interaction patterns among functional units, can be expected to inﬂuence champion emergence.

Structure In the literature on organizational innovation, few topics have received as much attention as requisite structure. One of the key propositions underlying

46

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

this research is the concept that a ﬂexible organic structure will contribute to innovation (Burns & Stalker, 1961). In a study of 44 small ﬁrms, Keller (1978) found that use of organic, less mechanistic or less bureaucratic, structure was related to the ﬁelding of innovative new products. Certainly, organic structures will promote the championing, communication, contact, and multifunctional efforts considered necessary for innovation. By the same token, the relationship between structure and innovation may be more complex than one would assume based on this organic/mechanistic distinction. Damanpour (1991) conducted a meta-analysis of the relationship between structural variables and innovation. Broadly speaking, his results suggest that structures that support the acquisition and application of specialized expertise – specifically, specialization, functional differentiation, professionalization, and technical knowledge resources – enjoy strong positive relationships ðr ¼ 0:35Þ with innovation. Given the role of expertise in innovation, it is not surprising that structures built around expertise would facilitate innovation. Nevertheless, it is important to recognize that innovation may suffer in expertise-based structures if the organizational structure does not allow for the integration of different forms of expertise through the use of mechanisms such as multifunctional teams. The more surprising ﬁnding emerging from Damanpour’s (1991) metaanalysis concerned the relationship of structural control variables (i.e., formalization, centralization, vertical differentiation, and administrative intensity) with innovation. Following the proposition that organic structures contribute to innovation, one might expect that all of these variables would evidence strong negative relationships with innovation. In fact, Damanpour’s (1991) ﬁndings do not reveal a strong consistent relationship, in any direction, between these structural controls and innovation. Further analysis of these data by Damanpour and Gopalakrishnan (2001) examining variation in the relationships of formalization, centralization, and vertical differentiation as a function of type of innovation (administrative versus technical innovations), magnitude of innovation (radical versus incremental), and stage of innovation (initiation versus implementation) did not reveal any marked shifts in the patterns of these relationships. Centralization did exert a consistent weak negative effect ðr ¼ 0:15Þ; suggesting that centralization, by virtue of decreased participation and entrepreneurialism, may inhibit innovation. Although these ﬁndings may, at ﬁrst glance, seem surprising, one potential explanation may be found in our earlier observations about control: By ensuring effective evaluation of creative efforts in relation to organizational strategy and capabilities, structural controls may contribute to innovation at the organizational level even as they inhibit innovation at the individual or

Innovation in Organizations: A Multi-Level Perspective on Creativity

47

group level. Additionally, more effective allocation of resources across creative efforts, accompanied by the resources available to large bureaucratic organizations, may make innovation more likely even though these organizations may experience difﬁculty in rapidly deploying innovations (Gopalakrishnan & Damanpour, 2000). These trade-offs between organizational-level and individual- and grouplevel effects, however, have a number of implications for innovation in nonorganic organizations. First, actions taken to buffer either creative people or teams from undue control may prove useful (Cardinal & Hatﬁeld, 2000; Mumford, 2000). Second, efforts should be made to allow creative groups to have some degree of local autonomy (Dougherty & Hardy, 1996). Third, actions taken to minimize frustration on the part of creative people concerning organizational responsiveness, or the lack thereof, may prove critical. Indeed, ensuring responsiveness may be a key role of product champions and group leaders (Mumford et al., 2002b; Stringer, 2000). In addition to expertise and control, at least two other topics have received significant attention in studies examining the inﬂuence of structure on innovation. First, it has been argued that diversiﬁcation may inhibit innovation. Hitt, Hoskisson, and Kim (1997) examined the effects of product diversiﬁcation and international diversiﬁcation on 295 ﬁrms where intensity of research and development investments was used as a proxy for innovation. They found that international diversiﬁcation was positively related to innovation but that product diversiﬁcation was negatively related to innovation. These negative effects of product diversiﬁcation appeared to be attributable both to a loss of focus and to the application of ﬁnancial controls. Second, it has been argued that the development of structures that support internal markets or product competition within the organization will contribute to innovation. In contrast to product diversiﬁcation, internal competition does appear to contribute to innovation (Kidder, 1981). For example, in the Chandy and Tellis (1998) cannibalization study, the researchers found that the availability of internal markets was positively related to both cannibalization and radical product innovation. Although internal competition may sometimes prove useful in stimulating innovation, structures promoting internal competition may limit the resources that can be devoted to any given effort while making it difﬁcult for top management to provide highly visible support for one particular project. As a result, this kind of structural manipulation may prove useful only when large, resource-rich ﬁrms are confronting turbulent environments and it is unclear exactly how to evaluate the likely success of any given approach.

48

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Summary The evidence accrued via organizational-level studies indicates that strategic control, structure, resources, and advocacy all represent potentially noteworthy inﬂuences on innovation at the organizational level. It is less clear exactly how these organizational-level variables inﬂuence creativity and innovation at the group and individual levels. Nonetheless, the pattern of the results obtained in these studies does suggest that organizational-level variables affect creativity and innovation. Table 3 summarizes some of the key conclusions ﬂowing from our review of organizational-level inﬂuences. Broadly speaking, these conclusions underscore the operation of interactions among various organizational-level variables. For example, although evaluation of ideas in relation to corporate strategy appears necessary for innovation, these ﬁt assessments may prove problematic under the conditions of uncertainty brought about by radical innovation. Continuing along similar lines, size and centralization may not necessarily inhibit creativity and innovation, but only if relevant organizational structures bring about focused application of expertise and requisite resources.

Table 3. Proposition 1

Proposition 2

Proposition 3

Proposition 4 Proposition 5

Proposition 6 Proposition 7 Proposition 8

Summary of Propositions Flowing from Organizational-Level Studies. Organizations and top management will carefully evaluate new ideas or new product proposals in relation to strategy and ﬁt with current business practices Evaluations based on ﬁt with current business practices will become less effective in turbulent environments, leading to the use of alternative evaluation strategies such as internal competition Future orientation and understanding of the business implications of an idea or product will be an important inﬂuence on the willingness of organizations to pursue new ideas A substantial and sustained investment of resources will be necessary to support the development of new ideas Organizational-level variables, such as size and profitability, associated with the availability of requisite resources will contribute to the willingness to pursue new ideas The availability of multiple champions willing to support an effort will contribute to the successful development of new ideas Organizational structures built around expertise, and the integration of expertise, will contribute to idea generation and innovation Structure and strategy that support a clear focus on future markets will contribute to idea generation and innovation

Innovation in Organizations: A Multi-Level Perspective on Creativity

49

These observations about organizational-level inﬂuences reiterate a point made earlier: The requirements for innovation at the organizational level are not necessarily consistent with the requirements that apply at the individual level and the group level. One illustration of this point may be found in the need for evaluation – and caution – at the organizational level as opposed to exploration and support at the individual and group levels. In fact, advocacy and product champions may be necessary as mechanisms for overcoming organizational caution. Another illustration of this point may be found in the number of projects pursued. While multiple projects may be desirable for stimulating creativity and innovation at the group and individual levels, organizations, by virtue of their complexity and the many difﬁculties inherent in their new product development efforts, may be able to pursue only a small number of ideas, resulting in a situation in which innovative efforts are rarely recognized and rewarded.

ENVIRONMENT The structure and strategies applied by organizations, at least in part, represent a response to the environment confronting the organization. Environments may exert a rather complex set of effects on innovation, inﬂuencing not only the feasibility of innovation but also the likelihood that an innovative idea or product will be adopted (Pool, 1997; Rodgers & Adhikurya, 1978). With regard to innovation in organizational settings, two environmental inﬂuences appear especially significant: turbulence and networking.

Turbulence Turbulence refers to the amount and rate of change in the organization’s operating environment with respect to variables such as technology, competition, and markets (Mumford et al., 2002a). One illustration of the impact of turbulence on innovation may be found in work carried out by Greve and Taylor (2000). They examined the inﬂuence of format changes on innovation among radio broadcasters. Their results revealed that the introduction of change by one broadcaster led to changes in the activities of other broadcasters. These changes were not necessarily imitations, but rather represented unique, innovative, format changes in their own right. Thus innovation, and the turbulence it induces, appears to stimulate innovation. In a related study examining the ﬁelding of innovative products among

50

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

manufacturing ﬁrms, Hadjimanolis (2000) found that environmental change was positively related to innovation ðr ¼ 0:32Þ across 140 small and mid-size ﬁrms. Apparently, external pressures (i.e., pressures associated with change) tend to encourage organizational innovation. As noted earlier, change comes in many forms – for example, technology, processes, markets, and competition. Thus a question arises: Which forms of change are most likely to stimulate innovation on the part of organizations? Clearly, the development of new technologies represents a potentially powerful form of turbulence with organizational innovation being particularly high during the period in which the technology is sufﬁciently well developed to permit its widespread application. This point is illustrated in the studies of innovation in the electrical industry by Hughes (1989) and Wise (1992). This pattern of effects begs still another question: Is the stimulus for innovation technology (push) or potential product demand (pull)? Although the uncertainty induced by the development of new technology may stimulate innovation by turning attention to change and encouraging environmental scanning (Greve & Taylor, 2000), product demand appears to be a more powerful inﬂuence on organizational innovation. In a qualitative study of urban living innovations (e.g., energy conservation, recycling), Pelz (1983) found that demand was a more important inﬂuence on organizational innovation than simple technological capability. In another study examining innovation in the banking industry, Buzzacchi, Colombo, and Mariotti (1995) found that demand pull variables were particularly powerful inﬂuences on innovation. Of course, these observations suggest that markets and market needs represent noteworthy inﬂuences on innovation. The impact of market forces on organizational innovation, of course, points to the potential impact of competitive pressure on innovation. Hadjimanolis’s (2000) study of innovation in manufacturing ﬁrms indicates that simple competitive intensity is not necessarily related to innovation – a point attested to by General Motors, Chrysler, and Ford during the 1950s and 1960s. Instead, innovation appears to be triggered by competitor introduction of products that inﬂuence markets and market potential (Majumdar & Venkataraman, 1993). Debruyne et al. (2002) examined competitor reactions to 509 new product launches. Competitors reacted to new product introductions when the introduction was relevant to, and represented a threat to, the market for extant products. In contrast, if a radical innovation was introduced (no threat to extant products) and a niche strategy was employed in its introduction, then competitors tended not to react to a new product introduction.

Innovation in Organizations: A Multi-Level Perspective on Creativity

51

The observations of Debruyne et al. (2002) suggest that the strategy being pursued by an organization represents a critical mediator of responses to turbulence brought about by competitor actions, markets, and the development of new technology. Some support for this proposition may be found in work conducted by Li and Atuahene-Gima (2001). These researchers examined environmental variables inﬂuencing innovation in Chinese hightechnology ﬁrms. Measures of product innovation strategy, dysfunctional competition, environmental turbulence, strategic alliances, and political networking were obtained from senior managers using a behavioral selfreport inventory. Assessments were also obtained of the success of the venture. The results showed that use of a strategy emphasizing innovation was strongly related to venture performance. Although turbulence and competition were not strongly related to venture success, they were positively related to use of a product innovation strategy. Ettlie (1983), in a study of 54 ﬁrms in the food processing industry, also found that strategy moderates the effects of turbulence and uncertainty on ﬁrm innovation. In other words, the effects of turbulence on innovation will depend on whether the organization decides to use innovation as a means of coping with change. When organizations use innovation as a means of coping with change, two points should be kept in mind. First, change is an inherently temporal phenomenon, so timing and planning will be a critical aspect of any successful strategy (Mumford et al., 2002a). Accordingly, Sharma (1999), in a qualitative study of nine ﬁrms initiating new business ventures, found that successful strategy took into account two key temporal considerations: (1) the strategic envelope or the readiness of the technology and markets for developments in an area, and (2) the strategic pacing or the timing of product delivery to allow for adequate development prior to ﬁelding. Similar ﬁndings were reported by Lynn, Lipp, Sharon, Ali, & Cortez (2002), who, in a study of 110 small manufacturing ﬁrms, found that either underdevelopment or overdevelopment of a product can lead to failure. In the overdevelopment case, failure was the result of demanding too much too early, which slowed product ﬁelding and the establishment of market control. In the underdevelopment case, failure came from providing too little too late, resulting in the ﬁelding of products with little or no tangible advantages relative to those already available. Second, turbulence can be expected to place greater demands on organizations with respect to the evaluation of new ideas. More specifically, the ﬁt appraisals commonly used to evaluate new ideas may prove less useful under conditions of turbulence. Under such conditions, ﬁt appraisals should ideally focus on future markets, technology, and competition. Moreover,

52

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

effective appraisal of ﬁt may require substantial scanning and ongoing active analysis of the environment considering information obtained from multiple sources – sources both internal and external to the organization. Networking A major source of information concerning change and its implications may be found in the actions of other organizations. In fact, in their study of innovation among Chinese high-technology ﬁrms, Li and Atuahene-Gima (2001) found that both political networking and the formation of strategic alliances were positively related ðr ¼ 0:35Þ to use of an innovation strategy for coping with change. Alliances and political networking were also found to be positively related ðr ¼ 0:18Þ to the success of the venture. Souitaris (2001) examined the inﬂuence of 32 external information sources on the rate of innovation observed in 105 manufacturing ﬁrms. Managerial interviews were used to assess the use and importance of these different sources of information. His ﬁndings indicated that the use of external information sources such as customer contact and feedback, supplier contact and market research, monitoring competitors, joint ventures, and international contacts contributed to innovation. Adams and Day (1998) and Nellore and Balachandra (2001), in studies of new product development efforts, likewise found that information provided by customers and suppliers was essential for innovation, not only serving as a stimulus for innovation but also providing key information needed by creative individuals for successful product development efforts (Nonaka, 1990; Olson et al., 2001). Not only does maintaining a strong network of relationships with customers and suppliers contribute to innovation, but contacts with other organizations that provide requisite information, new ideas, and new approaches can also prove beneﬁcial. In a study of new product development efforts, Alam (2003) found that contact with consulting ﬁrms – specifically, engineering consulting ﬁrms in the study at hand – could provide organizations with ideas likely to trigger innovation. Consulting ﬁrms, moreover, may contribute to innovation by serving as a mechanism for disseminating information and innovations across ﬁrms (Hargadon & Sutton, 1997). Relationships with consulting ﬁrms, of course, represent one form of alliance. Alliances among ﬁrms have been found to contribute to innovation through two mechanisms. First, alliances reduce the risk associated with innovation by distributing requisite investments and potential losses across multiple ﬁrms (Lengnick-Hall, 1992). Second, alliances provide information while serving to build certain new capabilities needed to make innovation possible,

Innovation in Organizations: A Multi-Level Perspective on Creativity

53

especially radical innovations not closely tied to the ﬁrm’s current core capabilities (Gemunden, Heydebreck, & Herden, 1992; McDermott & O’Connor, 2002). In the Gemunden et al. (1992) study, in fact, both market success and technological success were found to be positively related to the formation of alliances. Although some evidence indicates that alliances can contribute to creativity and innovation, one must remember that a substantial proportion of the alliances formed by organizations eventually fail. Sivadas and Dwyer (2000) conducted a study of 68 semiconductor manufacturing ﬁrms to identify the variables inﬂuencing the formation of successful alliances. Descriptions of the nature of these alliances using survey instruments indicated that successful new product development was related to ﬁve factors: (1) cooperative competence, (2) complementary skills across ﬁrms, (3) clarity with regard to goals, (4) shared control, and (5) a lack of history of competition. These observations about the conditions needed for successful alliance formation point to some noteworthy – albeit often overlooked – implications of group-level phenomena. Alliances are unlikely to prove effective whenthe group processes and climates within the participating organizations are such that they do not emphasize cooperation and communication. Moreover, certain group-level effects, such as the use of multifunctional teams, may provide the organizational experience that lays the groundwork for interﬁrm cooperation. Thus, just as environmental- and organizationalinﬂuences interact in shaping innovation, we can expect some noteworthy interactions to emerge among environmental-level and group-level inﬂuences.

Summary Clearly, some support is available for the inﬂuence of turbulence and networking on creativity and innovation. Like organizational-level inﬂuences, however, the effects of these two environmental-level inﬂuences appear rather complex. Table 4 summarizes the key conclusions emerging from our review of the effects of turbulence and networking on creativity and innovation. Turbulence may, at least sometimes, serve to stimulate innovation. However, its most noteworthy effects emerge when change opens new markets and/or when change threatens markets for extant products. Moreover, the impact of markets on responses to change suggests that strategic planning represents an important inﬂuence on whether innovation is used as a strategy for adapting to change. Networking, like strategy and planning, serves as a mechanism by

54

Table 4. Proposition 1 Proposition 2

Proposition 3 Proposition 4 Proposition 5 Proposition 6

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Summary of Propositions Flowing from Environmental-Level Studies. Rapid changes in technology, competitors, and markets will induce innovation Changes in markets and competitive threat with respect to current markets can be expected to be a particularly powerful inﬂuence on innovation Turbulence, by making ﬁt assessments more difﬁcult, may hinder efforts to direct creative and innovative efforts Active monitoring of competitors and ongoing environmental scanning will contribute to innovation Organizations will use alliances both as a way of monitoring change and as a vehicle for developing creative responses to change The ability of an organization to use alliances as a vehicle for innovation will depend on the organization already possessing a climate and culture that support innovation

which organizations cope with change. Not only do the alliances derived from networking ameliorate risk, but they also provide information and capacities that allow organizations to anticipate and respond effectively to change. At the same time, complex, apparently conﬂicting, cross-level interactions may emerge. Thus, while turbulence and market change may serve to stimulate innovation, turbulence and market change may disrupt standard idea appraisal techniques being applied at the organizational level.

MULTI-LEVEL CONSIDERATIONS Cross-Level Effects In examining inﬂuences on creativity and innovation at the individual, group, organizational, and environmental levels, a noteworthy conclusion emerged. It appears that the requirements for creativity and innovation observed at one level of analysis are not necessarily consistent with those observed at other levels of analysis. In fact, the requirements for creativity and innovation at one level sometimes seem to contradict the requirements for creativity and innovation at other levels of analysis. These contradictory cross-level effects are important for both substantive and practical reasons. It is difﬁcult to see how we can begin to form a truly comprehensive theory of creativity and innovation in organizations without

Innovation in Organizations: A Multi-Level Perspective on Creativity

55

understanding the nature and form of these interactions (Sharma, 1999). In addition, it seems likely that optimizing innovation in organizational settings will require strategies or approaches that allow organizations to balance these countervailing forces. In this chapter, we identiﬁed seven key contradictory cross-level interactions – interactions we believe must be a focus of future research. These interactions are summarized in Table 5. First, at an organizational level, controlled allocation of resources and the critical evaluation of ideas in relation to markets, competitors, and broader strategy is essential. The need for control at the organizational level, however, will tend to inhibit the open exploration of ideas needed for creativity and innovation at the individual level. This observation, in turn, suggests that there is a need for research examining when organizations are willing to encourage exploration and the mechanisms that organizations use to channel exploration along avenues likely to prove useful. Of course, missions and rewards may represent key control mechanisms in this regard (Abbey & Dickson, 1983; Mumford et al., 2002b). Nevertheless, the question remains as to when and how these controls should be applied if one is to maintain the motivation of creative individuals. Second, at the group level, creativity and innovation depend on active, open, collaborative exchanges and cooperation. Collaboration, cohesiveness, and open exchange, however, are not interactions that come naturally to autonomous, competitive people oriented toward individual achievement. The obvious implication of this statement is that we need to know far more about the conditions that bring about collaboration among creative people (Abra, 1994). One can expect group leaders and these leaders’ articulation of Table 5. Interaction 1 Interaction 2 Interaction 3 Interaction 4 Interaction 5 Interaction 6 Interaction 7

Summary of Critical Cross-Level Interactions. Control at the organizational level versus autonomous exploration at the individual level Collaboration and cohesion at the group level versus personal achievement at the individual level Differentiation of expertise at the group and organizational levels versus integration of expertise at group level External focus at the organizational level versus internal focus at the individual level Resource control at the organizational level versus support at the individual level Turbulence at the environmental level versus ﬁt with current operating procedures at the organizational level Direction at the organizational and group levels versus personal control at the individual level

56

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

a meaningful mission to prove of some importance in this regard. At the same time, other, more complex considerations, such as perceptions of procedural and distributive justice, norms with regard to interpersonal interaction, and resource competition, may all play a significant role in shaping the conditions that permit effective collaboration. Third, at the group level, the integration of expertise appears critical for innovation. Indeed, this requirement has been demonstrated in studies examining the inﬂuence of multifunctional teams on creativity and innovation (Thamhain, 2003). For innovation to occur at either the individual level or the organizational level, distinct types of expertise must be acquired. Thus, creativity and innovation apparently require a balance of differentiation and integration. The question that arises at this juncture, however, is exactly how organizations should manage this balance. One way this might be accomplished is to seek integration through projects but not structure. Another way that organizations might balance these competing demands for differentiation and integration is by stressing professional development through project contributions. Fourth, at the organizational level, the focus of the organization is on markets and competitors – attributes of the external environment that inﬂuence the need for innovation. At the individual level, however, the creative ideas that provide a basis for innovation arise from individuals evidencing intrinsic motives – motives that are tied to the solution of significant technical problems. Thus, to manage innovation, organizations must ﬁnd a way to engage professionals in the business without inducing a loss of focus on the work at hand (Cardinal, 2001). On the basis of the ﬁndings reviewed in this chapter, one plausible solution to these countervailing pressures might be to encourage creative people to seek and deﬁne technical problems in terms of key aspects of the business environment – specifically, the technical requirements imposed by market change, customer needs, and competitor capabilities. While this approach appears feasible, one is left with a new question: When and how should these requirements be imposed? Also, how does one engage creative people in generating responses to the challenges framed by these requirements? Fifth, at the organizational level, innovation requires resources – ﬁscal resources, information resources, and human capital resources. Of course, acquisition of these resources reduces processing time for the problem at hand, while placing creative individuals and groups in a situation where they may be confronted with undue external pressures resulting from the imposition of ﬁnancial controls and schedule controls. Apparently, organizations use two strategies to balance these pressures: the recruitment of project

Innovation in Organizations: A Multi-Level Perspective on Creativity

57

champions and the recruitment of support by top management (Howell & Boies, 2004). Other strategies might also be applied, such as the allocation of discretionary budget pools, the designation of chief technical ofﬁcers, and the use of project review teams (perhaps teams including external advisors). Sixth, at the environmental level, creativity and innovation appear linked to turbulence or change. Turbulence and change make it far more difﬁcult to apply the standards and ﬁt assessments commonly used in appraising new ideas. In fact, ﬁt assessments may, under conditions of high turbulence, effectively block creative efforts. Given the observations of O’Connor (1998), one might argue that future-oriented ﬁt assessments, especially those focused on future markets and future technical capabilities, may be necessary. This observation, in turn, suggests that environmental scanning and forecasting warrant more attention in studies of innovation than has been the case to date. In efforts along these lines, one must consider who, how, and when such scanning and forecasting should occur, as effective scanning and forecasting is likely to require a unique mix of technical and business expertise. Seventh, innovation at the group and organizational levels requires planning and direction. The planning and direction occurring at these levels may, in turn, inhibit the autonomous exploration required at the individual level. One obvious approach to resolving this dilemma is to involve creative people in the planning process (Cohen & Levinthal, 1990). By itself, participation is unlikely to fully resolve this dilemma. Instead, what may be required is the use of a more indirect approach – one in which plans and missions are used to frame the kind of problems of interest, and evaluation is used as a generative mechanism to enhance the potential contribution of a piece of work to a broader mission (Mumford et al., 2003).

GENERAL CONCLUSIONS Before turning to the broader implications suggested by our argument, certain limitations should be mentioned. To begin, we have not sought to provide a description of all of the various phenomena that might inﬂuence creativity in organizational settings. For example, although the layout of physical workspace is related to creativity and innovation (McCoy & Evans, 2002), a detailed examination of this topic is beyond the scope of this chapter. Affect inﬂuences creativity and innovation, and various organizational variables will act to shape the kind of affect people express in their work (Zhou & George, 2003). However, affective inﬂuences on creative thought received only scant attention here.

58

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Given the complex nature of creativity and innovation in organizational settings, it is impossible to examine all relevant inﬂuences in this short chapter. Instead, our intent here is to identify the variables operating at the individual, group, organizational, and environmental levels that are known to make a large difference with respect to creativity and innovation. In this chapter, we have, therefore, provided a set of core variables operating at each level that seem critical to creativity and innovation. Variable identiﬁcation is, of course, the ﬁrst step in theory development (Fleishman & Quaintance, 1984). Hence, given the availability of a sound list of key variables operating at the individual, group, organizational, and environmental levels, one might reasonably ask why we have not proposed a formal theory of creativity and innovation in organizational settings. We defer to other future efforts in the regard for three reasons. First, a host of criteria – new product introductions, managerial ratings, suggestions, and so on – have been applied in studies of creativity and innovation. Formal theory development must wait until a sufﬁcient body of evidence has been built up examining the nature of these criteria. Second, studies of creativity and innovation have not clearly deﬁned relevant boundary conditions; without identiﬁcation of such conditions, theory development will suffer (Mumford & Van Doorn, 2001). Third, with a few notable exceptions (e.g., Taggar, 2002), studies of cross-level inﬂuences on creativity and innovation have been rare. Instead, different investigators have focused on phenomena operating at the different levels of analysis. Lacking a strong cross-level research tradition, proposal of a formal multi-level theory is at best premature and at worst misguided, especially given the kind of contradictory cross-level interactions that appear to characterize creativity and innovation in organizational settings. Our goals in this chapter were substantially more limited. We sought to lay a foundation for subsequent multi-level theory development by departing from the tradition of single-phenomenon, single-level-of-analysis studies, and examining the literature through a new lens, one that considers the interaction of various phenomena within and across levels. Perhaps the most clear-cut conclusion that can be drawn from the present effort is that creativity and innovation are inherently multi-level phenomena. The studies examined in the course of our review indicate that creativity and innovation are strongly inﬂuenced by phenomena operating at the individual, group, organizational, and environmental levels. At the individual level, knowledge, processes, dispositions, and motivation are critical inﬂuences. At the group level, climate, leadership, process, and structure represent critical inﬂuences. At the organizational level, control, resources,

Innovation in Organizations: A Multi-Level Perspective on Creativity

59

advocacy, and structure represent critical inﬂuences. At the environmental level, turbulence in technology, markets, and competition, along with networking, represent critical inﬂuences. Moreover, the phenomena observed at each of these levels of analysis exhibit noteworthy effects on creativity and innovation, not uncommonly producing relationships with criteria applicable at a given level of analysis in the 0.30–0.50 range. In examining the key phenomena operating at a given level of analysis, it is important to bear in mind the complexity of those phenomena. For example, creativity and innovation do not involve one form of climate, but rather multiple aspects of climate – team, performance, contextual, and individual. Knowledge is not simply a matter of information, but also involves having the ‘‘right’’ information as well as frameworks for interpreting this information. Strategy may be a useful control, but it is unlikely to contribute much to innovation unless it is framed in terms of the future. To complicate matters further, our stereotypical images of how a given phenomenon operates within a level are not always borne out by the available data. For example, although we often assume that autonomy is the key aspect of the creative climate, in fact challenge and workgroup support appear more important (Bain et al., 2001). We tend to assume that a vision of the future is essential to lead creative people, but in reality definition of a challenging mission and technical direction appear more important (Mumford et al., 2002b). We tend to assume that small size and a lack of formality contribute to creativity and innovation, yet radical organizations come from large bureaucratic organizations that can provide the resources and buffering needed to support creative work (Chandy & Tellis, 2000; Cardinal, 2001). The complex nature of the phenomena that inﬂuence creativity and innovation imply that interactions can be expected among the phenomena operating at a given level of analysis. One relatively straightforward example of these within-level interactions may be found in the inﬂuence of group leaders on climate (Pirola-Merlo et al., 2002). At other times, it appears that the interactions observed among the phenomena operating within a given level of analysis are quite complex. As a consequence, use of multifunctional teams enhances creativity and innovation with respect to multiple criteria but, vis-a`vis loss of cohesion, induces stress and lower workgroup support, thereby placing more pressure on the leader to manage workload pressure and build team citizenship (Keller, 2001). In the face of a rather complex pattern of within-level interactions, it appears that these variables, within a given level, typically operate in an internally consistent and coherent fashion. A leader’s definition of a demanding technical mission will create a sense of challenge in a group of

60

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

professionals. Development of a viable strategy with regard to innovation will clearly contribute to the appropriate allocation of resources to various projects and development of a well-rounded portfolio of projects. Networking and alliances provide the kind of multisource feedback that organizations need to guide innovation in turbulent environments. The intense curiosity characterizing creative people allows them to acquire not only the requisite information but also the expertise needed to work with this information. In some cases, the interactions observed across levels display this kind of coherence. On the basis of the ﬁndings of Sivadas and Dwyer (2000), it seems plausible that the use of multifunctional teams, vis-a`-vis cooperative competence, would contribute to networking and alliance formation. The need for a professional structure is, of course, consistent with the role of expertise in innovation. A challenging climate accompanied by important missions is undoubtedly necessary to bring about engagement among autonomous, workfocused, achievement-oriented individuals. The difﬁculty in understanding and managing creativity in organizational settings arises from the fact that the requirements for creativity and innovation at one level of analysis sometimes directly contradict the requirements of creative and innovation at another level of analysis. In the course of this chapter, seven key conﬂicting multi-level interactions emerged – interactions that must be managed if one is to ensure that creativity and innovation are possible in organizations. These contradictions in the requirements for creativity and innovation across levels of analysis are noteworthy in part because they suggest that sustained innovation in organizational settings will require mechanisms, such as buffering or envisioning, that permit management of these countervailing forces. Given the potential impact of these kinds of cross-level interactions on innovation, there would seem to be a need for multi-level studies of creativity and innovation – far more studies than have been conducted to date. In fact, the existence and substantive significance of these interactions make it difﬁcult to see how we can ever hope to attain a true understanding of creativity and innovation in organizations without application of a multi-level approach. Ideally, the present effort will lay a foundation for future research along these lines.

ACKNOWLEDGMENTS We thank Ginamarie Scott, Blaine Gaddis, and Jill Strange for their contributions to this chapter.

Innovation in Organizations: A Multi-Level Perspective on Creativity

61

REFERENCES Abbey, A., & Dickson, J. (1983). R&D work climate and innovation in semiconductors. Academy of Management Journal, 25, 362–368. Abra, J. (1994). Collaboration in creative work: An initiative for investigation. Creativity Research Journal, 8, 205–218. Adams, J. E., & Day, G. S. (1998). Enhancing new product development performance: An organizational learning perspective. Journal of Product Innovation Management, 15, 403–422. Alam, I. (2003). Commercial innovations from consulting engineering ﬁrms: An empirical exploration of a novel source of new product ideas. Journal of Product Innovation Management, 20, 300–313. Albrecht, T. L., & Roop, V. A. (1984). Communicating about innovation in networks of three U.S. organizations. Journal of Communication, 34, 78–91. Allen, T. J., & Cohen, S. I. (1969). Information ﬂow in research and development laboratories. Administrative Science Quarterly, 14, 12–19. Amabile, T. M. (1985). Motivation and creativity: Effects of motivational orientation on creative writers. Journal of Personality and Social Psychology, 48, 393–399. Amabile, T. M. (1997). Entrepreneurial creativity through motivational synergy. Journal of Creative Behavior, 31, 18–26. Amabile, T. M., & Conti, R. (1999). Changes in the work environment for creativity during downsizing. Academy of Management Journal, 42, 630–641. Amabile, T. M., Conti, R., Coon, H., Lazenby, J., & Herron, M. (1996). Assessing the work environment for creativity. Academy of Management Journal, 39, 1154–1184. Amabile, T. M., & Gryskiewicz, N. D. (1989). The creative environment scales: Work environment inventory. Creativity Research Journal, 2, 231–253. Amabile, T. M., Hadley, C. N., & Kramer, S. J. (2002). Creativity under the gun. Harvard Business Review, 80, 52–61. Amabile, T. M., Hennessey, B. A., & Grossman, B. S. (1986). Social inﬂuences on creativity: The effects of contracted for reward. Journal of Personality and Social Psychology, 50, 14–23. Anacona, D., & Caldwell, D. (1992). Demography and design: Predictors of new product team performance. Organization Science, 3, 321–341. Anderson, C. J., Glassman, M., McAfee, R. B., & Pinelli, T. (2001). An investigation of factors affecting how engineers and scientists seek information. Journal of Engineering Technology Management, 18, 131–155. Anderson, N. R., & West, M. A. (1998). Measuring climate for work group innovation: Development and validation of the team climate inventory. Journal of Organizational Behavior, 19, 235–258. Andrews, F. M. (1968). Scientific performance as related to time spent on technical work, teaching, or administration. Administrative Science Quarterly, 13, 183–193. Andrews, F. M., & Farris, G. F. (1967). Supervisory practices and innovation in scientific teams. Personnel Psychology, 20, 497–515. Andrews, F. M., & Gordon, G. (1970). Social and organizational factors affecting innovation research. Proceedings for the American Psychological Association, 78, 570–589. Andrews, J., & Smith, D. C. (1996). In search of marketing imagination: Factors affecting the creativity of marketing programs for the mature products. Journal of Marketing Research, 33, 174–187.

62

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Arad, S., Hanson, M. A., & Schneider, R. J. (1997). A framework for the study of relationships between organizational characteristics and organizational innovation. Journal of Creative Behavior, 31, 42–58. Arvey, R. D., Dewhurst, H. D., & Boling, J. C. (1976). Relationships between goal clarity participation in goal setting, and personality characteristics on job satisfaction in a scientific organization. Journal of Applied Psychology, 61, 103–105. Arvey, R. D., & Neel, C. W. (1975). Motivation and obsolescence in engineers. Industrial Gerontology, 18, 113–120. Avlonitis, G., Kouremenous, A., & Tzokas, N. (1994). Assessing the innovativeness of organizations and its antecedents: Project innovstrat. European Journal of Marketing, 28, 5–28. Ayres, D., Dahlstrom, R., & Skinner, S. J. (1997). An exploratory investigation of organizational antecedents to new product success. Journal of Marketing Research, 34, 107–116. Baer, J. (2003). Evaluative thinking, creativity, and task speciﬁcity: Separating wheat from chaff is not the same as ﬁnding needle in haystacks. In: M. A. Runco (Ed.), Critical creative processes (pp. 129–152). Cresskill, NJ: Hampton. Baer, M., & Frese, M. (2003). Innovation is not enough: Climates for initiative and psychological safety, process innovations, and ﬁrm performance. Journal of Organizational Behavior, 24, 45–68. Baer, M., Oldham, G. R., & Cummings, A. (2003). Rewarding creativity: When does it really matter? Leadership Quarterly, 14, 478–486. Bain, P. G., Mann, L., & Pirola-Merlo, A. (2001). The innovation imperative: The relationships between team climate, innovation, and performance in research and development teams. Small Group Research, 32, 55–73. Barnowe, J. T. (1975). Leadership and performance outcomes in research organizations. Organizational Behavior and Human Performance, 14, 264–280. Barron, F., & Harrington, D. M. (1981). Creativity, intelligence, and personality. Annual Review of Psychology, 32, 439–476. Basadur, M., Runco, M. A., & Vega, L. A. (2000). Understanding how creative thinking skills, attitudes, and behaviors work together: A causal process model. Journal of Creative Behavior, 34, 77–100. Bastien, D. T., & Hostager, T. J. (1988). Jazz as a process or organizational innovation. Communication Research, 15, 582–602. Bouty, I. (2000). Interpersonal and integration inﬂuences on informal resource exchanges between R&D researchers across organizational boundaries. Academy of Management Journal, 43, 50–65. Brophy, D. R. (1998). Understanding, measuring, and enhancing individual creative problemsolving efforts. Creativity Research Journal, 11, 123–150. Brown, S., & Eisenhardt, K. (1995). Product development: Past research, present ﬁndings, and future directions. Academy of Management Review, 20, 343–378. Buehler, R., Grifﬁn, D., & Ross, M. (1994). Exploring the planning fallacy: Why people underestimate their task completion times. Journal of Personality and Social Psychology, 67, 366–381. Bunce, D., & West, M. A. (1995). Self perceptions and perceptions of group climate as predictors of individual innovation at work. Applied Psychology: An International Review, 44, 199–215.

Innovation in Organizations: A Multi-Level Perspective on Creativity

63

Burningham, C., & West, M. A. (1995). Individual, climate, and group interaction processes as predictors of work team innovation. Small Group Research, 26, 106–117. Burns, T., & Stalker, C. (1961). The management of innovation. London, UK: Tavistock. Buzzacchi, L., Colombo, M. G., & Mariotti, S. (1995). Technological regimes and innovation in services: The case of the Italian banking industry. Research Policy, 24, 151–168. Capon, N., Farley, J. U., Lehman, D. R., & Hulbert, J. M. (1992). Proﬁles of product innovators among large U.S. manufacturers. Management Science, 38, 157–169. Cardinal, L. B. (2001). Technological innovation in the pharmaceutical industry: The use of organizational control in managing research and development. Organization Science, 12, 19–37. Cardinal, L. B., & Hatﬁeld, D. E. (2000). Internal knowledge generation: The research laboratory and innovative productivity in the pharmaceutical industry. Journal of Engineering Technology Management, 17, 247–271. Chalupsky, A. B. (1953). Incentive practices as viewed by scientists and managers of pharmaceutical laboratories. Personnel Psychology, 6, 385–401. Chandy, R. K., & Tellis, G. J. (1998). Organizing for radical product innovation: The overlooked role of willingness to cannibalize. Journal of Marketing Research, 35, 474–487. Chandy, R. K., & Tellis, G. J. (2000). The incumbent’s curse? Incumbency size, and radical innovation. Journal of Marketing, 64, 1–17. Chi, M. T., Bassock, M., Lewis, M. W., Reitman, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13, 145–182. Cohen, W. M., & Levinthal, D. A. (1990). Absorptive capacity: A new perspective on learning and innovation. Administrative Science Quarterly, 35, 128–152. Collins, M. A., & Amabile, T. M. (1999). Motivation and creativity. In: R. J. Sternberg (Ed.), Handbook of creativity (pp. 297–312). Cambridge, UK: Cambridge University Press. Cooper, R. G., & Kleinschmidt, E. J. (1991). New product processes at leading industrial ﬁrms. Industrial Marketing Management, 20, 137–148. Cooper, R. G., & Kleinschmidt, E. J. (2000). New product performance: What distinguishes the star products. Australian Journal of Management, 25, 17–46. Cosier, R. A., & Rose, G. L. (1977). Cognitive conﬂict and goal conﬂict effects on task performance. Organizational Behavior and Human Performance, 18, 378–391. Csikszentmihaly, M. (1999). Implications of a systems perspective for the study of creativity. In: R. J. Sternberg (Ed.), Handbook of creativity (pp. 312–338). Cambridge, UK: Cambridge University Press. Culnan, M. J. (1983). Environmental scanning: The effects of task complexity and source access ability on information gathering behavior. Decision Sciences, 14, 194–207. Curral, L. A., Forrester, R. H., Dawson, J. F., & West, M. A. (2001). It’s what you do and the way you do it: Team task, team size, and innovation-related group processes. European Journal of Work and Organizational Psychology, 10, 187–204. Daellenbach, U. S., McCarthy, A. M., & Shoenecker, T. S. (1999). Commitment to innovation: The impact of top management team characteristics. R&D Management, 29, 199–208. Damanpour, F. (1991). Organizational innovation: A meta-analysis of effects of determinants and moderators. Academy of Management Journal, 34, 555–590. Damanpour, F., & Gopalakrishnan, S. (2001). The dynamics of the adoption of product and process innovations in organizations. Journal of Management Studies, 38, 45–66.

64

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Danneels, E., & Kleinschmidt, E. J. (2001). Product innovativeness from the ﬁrm’s perspective: Its dimensions and their relation with project selection and performance. Journal of Product Innovation Management, 18, 357–363. Dean, J. W., & Sharfman, M. P. (1996). Does decision process matter: A study of strategic decision making effectiveness. Academy of Management Journal, 39, 368–396. Debruyne, M., Moenaert, R., Grifﬁn, A., Hart, S., Hultink, E. J., & Robben, H. (2002). The impact of new product launch strategies on competitive reaction to industrial markets. Journal of Product Innovation Management, 19, 159–170. Dess, G. G., & Pickens, J. C. (2000). Changing roles: Leadership in the 21st century. Organizational Dynamics, 28, 18–34. Dewar, R. D., & Dutton, J. E. (1986). The adoption of radical and incremental innovations: An empirical analysis. Management Science, 32, 1422–1433. Dougherty, D., & Hardy, B. F. (1996). Sustained innovation production in large mature organizations: Overcoming organization problems. Academy of Management Journal, 39, 826–851. Dougherty, D., & Heller, T. (1994). The illegitimacy of successful product innovation in established ﬁrms. Organization Science, 5, 200–281. Drazin, R., Glynn, M. A., & Kazanjian, R. K. (1999). Multilevel theorizing about creativity in organizations: A sense-making perspective. Academy of Management Review, 24, 286–329. Dunbar, K. (1995). How do scientists really reason: Scientific reasoning in real-world laboratories. In: R. J. Sternberg & J. E. Davidson (Eds), The nature of insight (pp. 365–396). Cambridge, MA: MIT Press. Dunham, L., & Freeman, R. E. (2000). There is business like show business: Leadership lessons from the theater. Organizational Dynamics, 29, 108–122. Ebaldi, Y. M., & Utterback, J. M. (1984). The effects of communication on technological innovation. Management Science, 30, 572–585. Eisenberger, R., & Cameron, J. (1996). Detrimental effects of reward: Reality or myth? American Psychologist, 51, 1153–1166. Eisenhardt, K. M., & Tabrizi, B. (1995). Accelerating adaptive processes: Product innovation in the global computer industry. Administrative Science Quarterly, 40, 617–636. Ekvall, G. (1996). Organizational climate for creativity and innovation. European Journal of Work and Organizational Psychology, 5, 105–124. Ekvall, G., & Ryhammer, L. (1999). The creative climate: Its determinants and effects at a Swedish university. Creativity Research Journal, 12, 303–310. Elkins, T., & Keller, R. T. (2003). Leadership in research and development organizations: A literature review and conceptual framework. Leadership Quarterly, 14, 486–505. Ellison, R. L., James, L. R., & Carron, T. (1970). Prediction of R&D performance criteria with biographical information. Journal of Industrial Psychology, 5, 37–57. Elsbach, K. D., & Kramer, R. M. (2003). Assessing creativity in Hollywood pitch meetings: Evidence for a dual-process model of creativity judgments. Academy of Management Journal, 46, 283–302. Enson, J., Cottam, A., & Band, C. (2001). Fostering knowledge management through the creative work environment: A portable model from the advertising industry. Journal of Information Science, 27, 147–155. Ericsson, K. A., & Charness, W. (1994). Expert performance: Its structure and acquisition. American Psychologist, 49, 725–747.

Innovation in Organizations: A Multi-Level Perspective on Creativity

65

Estes, Z., & Ward, T. B. (2002). The emergence of novel attributes in concept modiﬁcation. Creativity Research Journal, 14, 144–156. Ettlie, J. E. (1983). Organizational policy and innovation among suppliers to the food-processing sector. Academy of Management Journal, 26, 27–44. Farris, G. F. (1972). The effect of individual role on performance in innovative groups. R&D Management, 3, 23–28. Feist, G. J. (1999). The inﬂuence of personality on artistic and scientific creativity. In: R. J. Sternberg (Ed.), Handbook of creativity (pp. 273–296). Cambridge, UK: Cambridge University Press. Feist, G. J., & Gorman, M. E. (1998). The psychology of science: Review and integration of a nascent discipline. Review of General Psychology, 2, 3–47. Feldman, D. H. (1999). The development of creativity. In: R. J. Sternberg (Ed.), Handbook of creativity (pp. 169–188). Cambridge, UK: Cambridge University Press. Finke, R. A., Ward, T. B., & Smith, S. M. (1992). Creative cognition: Theory, research, and applications. Cambridge, MA: MIT Press. Fleishman, E. A., & Quaintance, M. K. (1984). Taxonomies of human performance: The description of human tasks. Orlando, FL: Academic Press. Ford, C. M., & Gioia, D. A. (2000). Factors inﬂuencing creativity in the domain of managerial decision making. Journal of Management, 26, 705–732. Frankwick, G. L., Walker, B. A., & Ward, J. C. (1994). Belief structures in conﬂict: Mapping a strategic marketing decision. Journal of Business Research, 31, 183–195. Galbraith, J. R. (1982). Designing the innovating organization. Organizational Dynamics, 10, 5–25. Gassman, O., & van Zedwitz, M. (2003). Trends and determinants of managing virtual R&D teams. R&D Management, 33, 243–263. Gemunden, H. G., Heydebreck, P., & Herden, R. (1992). Technological interweavement: A means of achieving innovation success. R&D Management, 22, 359–376. George, J. M., & Zhou, J. (2001). When openness to experience and conscientiousness are related to creative behavior: An interactional approach. Journal of Applied Psychology, 86, 51–524. Germain, R. (1996). The role of context and structure in radical and incremental logistics innovation adoption. Journal of Business Research, 35, 117–127. Geroski, P., Machin, S., & Van Reenen, J. (1993). The profitability of innovating ﬁrms. RAND Journal of Economics, 24, 198–212. Gerstenberger, P. C., & Allen, T. J. (1968). Criteria used by research and development engineers in the selection of an information source. Journal of Applied Psychology, 52, 272–279. Getzels, J. S., & Csikszentmihalyi, M. (1976). The creative vision: A longitudinal study of problem finding in art. New York: Wiley. Ghiselin, B. (1963). Ultimate criteria for two levels of creativity. In: C. W. Taylor & F. Barron (Eds), Scientific creativity: Its recognition and development (pp. 30–43). New York: Wiley. Gonzalez-Roma, V., Peiro, J. M., & Tordera, N. (2002). An examination of the antecedents and moderator inﬂuences of climate strength. Journal of Applied Psychology, 87, 465–473. Gopalakrishnan, S., & Damanpour, F. (2000). The impact of organizational context on innovation adoption in commercial banks. IEEE Transactions on Engineering Management, 47, 14–25. Graves, S. B., & Lanowitz, N. S. (1993). Innovative productivity and returns to scale in the pharmaceutical industry. Strategic Management Journal, 14, 593–606.

66

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Greenberg, E. (1992). Creativity, autonomy, and the evaluation of creative work: Artistic workers in organizations. Journal of Creative Behavior, 26, 75–80. Greve, H. R., & Taylor, A. (2000). Innovations as catalysts for organizational change: Shifts in organizational cognition and search. Administrative Science Quarterly, 45, 54–80. Grifﬁn, A. (1997). PDMA research on new product development practices: Updating trends and benchmarking best practices. Journal of Product Innovation Management, 14, 429–458. Hadjimanolis, A. (2000). An investigation of innovation antecedents in small ﬁrms in the context of a small developing country. R&D Management, 30, 235–245. Halpern, D. F. (2003). Thinking critically about creative thinking. In: M. A. Runco (Ed.), Critical creative processes (pp. 189–208). Cresskill, NJ: Hampton. Hargadon, A. B., & Douglas, Y. (2001). When innovations meet institutions: Edison and the design of the electric light. Administrative Science Quarterly, 46, 476–501. Hargadon, A., & Sutton, R. I. (1997). Technology brokering and innovation in a product development ﬁrm. Administrative Science Quarterly, 42, 716–749. Hauschildt, J., & Kirchmann, E. (2001). Teamwork for innovation—the ‘‘troika’’ of promoters. R&D Management, 31, 41–49. Heinzen, J. E., Mills, C., & Cameron, P. (1993). Scientific innovation potential. Creativity Research Journal, 6, 261–270. Henderson, R. (1993). Underinvestment and incompetence as responses to radical innovation: Evidence from the photolithographic alignment equipment industry. RAND Journal of Economics, 24, 248–270. Hershey, D. A., Walsh, D. A., Read, S. J., & Chulef, A. S. (1990). The effects of expertise on ﬁnancial problem-solving: Evidence for goal-directed problem-solving scripts. Organizational Behavior and Human Decision Processes, 46, 77–101. Hitt, M. A., Hoskisson, R. E., Johnson, R. A., & Moesel, D. D. (1996). The market for corporate control and ﬁrm innovation. Academy of Management Journal, 39, 1084–1196. Hitt, M. H., Hoskisson, R. E., & Kim, H. (1997). International diversiﬁcation effects on innovation and ﬁrm performance in product diversiﬁed ﬁrms. Academy of Management Journal, 30, 767–798. Hoffman, L. R., Hamburg, E., & Maier, N. (1962). Differences and disagreements as factors in creative problem solving. Journal of Abnormal and Social Psychology, 64, 20–214. Howell, J. M., & Boies, K. (2004). Champions of technological innovation: The inﬂuence of contextual knowledge, role orientation, idea generation, and idea promotion on champion emergence. Leadership Quarterly, 15, 130–149. Howell, J. M., & Higgins, C. A. (1988). Champions of technological innovation. Administrative Science Quarterly, 35, 317–341. Huber, J. C. (1998). Invention and inventivity as a special kind of creativity with implications for general creativity. Journal of Creative Behavior, 32, 58–72. Hughes, T. P. (1989). American genesis: A history of the American genius for invention. New York: Penguin. Hunt, J. G., Stelluto, G. E., & Hooijberg, R. (2004). Toward new-wave organizational creativity: Beyond romance and analogy in the relationship between orchestra conductor leadership and musician creativity. Leadership Quarterly, 15, 565–583. Isaksen, S. G., Lauer, K. J., & Ekvall, G. (1999). Situational outlook questionnaire: A measure of the climate for creativity and change. Psychological Reports, 85, 665–674.

Innovation in Organizations: A Multi-Level Perspective on Creativity

67

James, K., Clark, K., & Cropanzano, R. (1999). Positive and negative creativity in groups, institutions, and organizations: A model and theoretical extension. Creativity Research Journal, 12, 211–226. Jaussi, K. S., & Dionne, S. D. (2003). Leading for creativity: The role of unconventional behavior. Leadership Quarterly, 14, 351–368. Jelinek, M., & Schoonhoven, C. B. (1990). The innovation marathon: Lessons learned from high technology firms. Oxford, UK: Blackwell. Judge, W. Q., Fryxell, G. E., & Dooley, R. S. (1997). The new task of R&D management: Creating goal-directed communities for innovation. California Management Review, 39, 72–85. Jung, D. I., Chow, C., & Wa, A. (2003). The role of transformational leadership in enhancing organizational innovation: Hypotheses and some preliminary ﬁndings. Leadership Quarterly, 14, 386–411. Kahi, S. S., Sosik, J. J., & Avolio, B. J. (2003). Effects of leadership style, anonymity, and rewards on creativity-relevant processes and outcomes in an electronic meeting system context. Leadership Quarterly, 14, 369–376. Kasof, J. (1995). Explaining creativity: The attribution perspective. Creativity Research Journal, 8, 311–366. Kazanjian, R. K., Drazin, R., & Glynn, M. A. (2000). Creativity and technological learning: The roles of organization, architecture, and crisis in large-scale projects. Journal of Engineering Technology Management, 17, 273–298. Keller, R. T. (1978). Dimensions of management systems and performance in continuous process organizations. Human Relations, 31, 119–129. Keller, R. T. (1986). Predictors of the performance of project groups in R&D organizations. Academy of Management Journal, 29, 715–726. Keller, R. T. (1989). A test of the path–goal theory of leadership with need for clarity as a moderator in research and development organizations. Journal of Applied Psychology, 74, 208–212. Keller, R. T. (1997). Job involvement and organizational commitment as longitudinal predictors of job performance: A study of scientists and engineers. Journal of Applied Psychology, 82, 539–545. Keller, R. T. (2001). Cross-functional project groups in research and new product development: Diversity, communications, job stress, and outcomes. Academy of Management Journal, 44, 547–553. Keller, R. T., & Holland, W. E. (1983). Communicators and motivators in research and development organizations. Academy of Management Journal, 4, 742–749. Kessler, E., & Chakrabarti, A. K. (1996). Innovation speed: A conceptual model of context, antecedents, and outcomes. Academy of Management Review, 21, 1143–1191. Kickul, J., & Gundry, L. K. (2001). Breaking through boundaries for organizational innovation: New managerial roles and practices in e-commerce ﬁrms. Journal of Management, 27, 347–361. Kidder, T. (1981). The soul of a new machine. New York: Avon. Kim, Y., Song, K., & Lee, J. (1993). Determinants of technological innovation in the small ﬁrms of Korea. R&D Management, 23, 215–226. Kitchell, S. (1995). Corporate culture, environmental adaptation, and innovation adoption: A quantitative/qualitative approach. Journal of the Academy of Marketing Science, 23, 195–205.

68

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Klein, K. J., Conn, A. B., & Sorra, J. S. (2001). Implementing computerized technology: An organizational analysis. Journal of Applied Psychology, 86, 811–824. Kobe, L.M., & Reiter-Palmon, R. (2000). The effect of stress on creative problem solving. Paper presented at the meetings of the society for industrial and organizational psychology, New Orleans, LA. Langholtz, H., Gettys, C., & Foote, B. (1995). Are resource ﬂuctuations anticipated in resource allocation tasks? Organizational Behavior and Human Decision Processes, 64, 274–282. Lei, D., Slocum, J. W., & Pitts, R. A. (1999). Designing organizations for competitive advantage: The power of learning and unlearning. Organizational Dynamics, 27, 24–38. Lengnick-Hall, C. A. (1992). Innovation and competitive advantage: What we know and what we need to learn. Journal of Management, 18, 399–429. Lewis, M., Welsh, M. A., Dehler, G. E., & Green, S. G. (2002). Product development tensions: Exploring contrasting styles of project management. Academy of Management Journal, 45, 546–564. Li, H., & Atuahene-Gima, K. (2001). Product innovation strategy and the performance of technology ventures in China. Academy of Management Journal, 44, 1123–1134. Lonergan, D., Scott, G. M., & Mumford, M. D. (2004). Evaluative aspects of creative thought: Effects of appraisal and revision standards. Creativity Research Journal, 16, 231–246. Lovelace, K., Shapiro, D. L., & Weingart, L. R. (2001). Maximizing cross-functional new product teams’ innovativeness and constraint adherence: A conﬂict communications perspective. Academy of Management Journal, 44, 779–793. Lubart, T. I. (2001). Models of the creative process: Past, present, and future. Creativity Research Journal, 13, 295–308. Lynn, G., Lipp, S., Sharon, M., Ali, A., & Cortez, J. (2002). Factors impacting the adoption and effectiveness of the world wide web in marketing. Industrial Marketing Management, 31, 35–50. Madjar, N., Oldham, G. R., & Pratt, M. G. (2002). There’s no place like home? The contributions of work and nonwork creativity support to employees’ creative performance. Academy of Management Journal, 45, 757–787. Maidique, M., & Zirger, B. J. (1984). A study of success and failure in product innovation: The case of the U.S. electronics industry. IEEE Transactions in Engineering Management, 31, 192–203. Maier, N. R. (1950). The quality of group discussions as inﬂuenced by the discussion leader. Human Relations, 3, 155–174. Maier, N. R. (1953). An experimental test of the effect of training on discussion leadership. Human Relations, 6, 161–173. Maier, N. R., & Hoffman, L. R. (1960). Quality of ﬁrst and second solutions in group problem solving. Journal of Applied Psychology, 44, 278–283. Maier, N. R. F., & Hoffman, L. R. (1964). Financial incentives and group decision in motivational change. Journal of Social Psychology, 64, 369–378. Maier, N. R. F., & Solem, A. R. (1962). Improving solutions by turning choice situations and problems. Personnel Psychology, 15, 151–157. Majumdar, S. K., & Venkataraman, S. (1993). New technology adoption in US telecommunications: The role of competitive pressures and ﬁrm-level inducements. Research Policy, 22, 521–536.

Innovation in Organizations: A Multi-Level Perspective on Creativity

69

Markham, S. K., & Aiman-Smith, L. (2001). Product champions: Truths, myths, and management. Research Technology Management, 44, 44–50. Markham, S. K., Green, S., & Basu, R. (1991). Champions and antagonists: Relationships with R&D project characteristics and management. Journal of Engineering and Technology, 8, 217–242. Markham, S. K., & Grifﬁn, A. (1998). The breakfast of champions: Associations between champions and product development, environments, practices, and performance. Journal of Product Innovation Management, 15, 436–454. McCoy, J. M., & Evans, G. W. (2002). The potential role of the physical environment in fostering creativity. Creativity Research Journal, 14, 209–426. McDermott, C. M., & O’Connor, G. C. (2002). Managing radical innovation: An overview of emergent strategy issues. Journal of Product Innovation Management, 19, 424–439. McFadzean, E. (2002). Developing and supporting creative problem-solving teams: Part 1—a conceptual model. Management Decision, 40, 463–476. McGourty, J., Tarshis, L. A., & Dominick, P. (1996). Managing innovation: Lessons from world class organizations. International Journal of Technology Management, 11, 354–368. Merriﬁeld, P. R., Guilford, J. P., Christensen, P. R., & Frick, J. W. (1962). The role of intellectual factors in problem solving. Psychological Monographs, 76, 1–21. Meyer, A. D., & Goes, J. B. (1988). Organizational assimilation of innovations. Academy of Management Journal, 31, 897–923. Meyer, J. A. (1998). Information overload in marketing management. Marketing Intelligence and Planning, 16, 200–209. Mobley, M. I., Doares, L., & Mumford, M. D. (1992). Process analytic models of creative capacities evidence for the combination and reorganization process. Creativity Research Journal, 5, 125–156. Monge, P. R., Cozzens, M. D., & Contractor, N. S. (1992). Communication and motivational predictors of the dynamics of organizational innovation. Organization Science, 3, 250–274. Mooreman, C., & Miner, A. S. (1997). The impact of organizational memory of new product performance and creativity. Journal of Marketing Research, 34, 91–107. Mossholder, K. W., & Dewhurst, H. D. (1980). The appropriateness of management by objectives for development and research personnel. Journal of Management, 6, 145–156. Mouly, V. S., & Sankaran, J. K. (1999). The ‘‘permanent’’ acting leader: Insights from a dying Indian R&D organization. Leadership Quarterly, 10, 637–652. Mumford, M. D. (2000). Managing creative people: Strategies and tactics for innovation. Human Resource Management Review, 10, 1–29. Mumford, M. D. (2003). Where have we been, where are we going? taking stock in creativity research. Creativity Research Journal, 14, 107–120. Mumford, M. D., Baughman, W. A., & Sager, C. E. (2003). Picking the right material: Cognitive processing skills and their role in creative thought. In: M. A. Runco (Ed.), Critical creative processes (pp. 19–68). Cresskill, NJ: Hampton. Mumford, M. D., Baughman, W. A., Supinski, E. P., & Maher, M. A. (1996). Process-based measures of creative problem-solving skills: II. Information encoding. Creativity Research Journal, 9, 77–88. Mumford, M.D., Blair, C., & Marcy, R.J. Alternative knowledge structure in creative thought: Schema, associations, and cases. In: J. Kaufman & J. Baer (Eds), The relationship between creativity knowledge and reason. Cambridge, UK: Cambridge University Press. (in press).

70

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Mumford, M. D., Blair, L., Dailey, L., Leritz, L. E., & Osburn, H. K. (2003). Errors in creative thought? cognitive biases in a complex processing activity. Norman, OK: University of Oklahoma. Mumford, M. D., Connelly, M. S., & Gaddis, B. (2003). How creative leaders think: Experimental ﬁndings and cases. Leadership Quarterly, 14, 336–358. Mumford, M. D., Feldman, J. M., Hein, M. B., & Nago, D. J. (2001). Tradeoffs between ideas and structure: Individual versus group performance in creative problem-solving. Journal of Creative Behavior, 35, 1–23. Mumford, M. D., & Gustafson, S. B. (1988). Creativity syndrome: Integration, application, and innovation. Psychological Bulletin, 103, 27–43. Mumford, M. D., Mobley, M. I., Uhlman, C. E., Reiter-Palmon, R., & Doares, L. (1991). Process analytic models of creative capacities. Creativity Research Journal, 4, 91–122. Mumford, M. D., & Moertl, P. (2003). Cases of social innovation: Lessons from two innovations in the 20th century. Creativity Research Journal, 14, 261–266. Mumford, M. D., Peterson, N. G., & Childs, R. A. (1999). Basic and cross-functional skills: Taxonomies, measures, and ﬁndings in assessing job skill requirements. In: N. G. Peterson, M. D. Mumford, W. C. Borman, P. R. Jeanneret & E. A. Fleishman (Eds), An occupational information system for the 21st century: The development of O*Net (pp. 49– 76). Washington, DC: American Psychological Association. Mumford, M. D., Schultz, R. A., & Osburn, H. K. (2002a). Planning in organizations: Performance as a multi-level phenomenon. In: F. J. Yammarino & F. Dansereau (Eds), Research in multi-level issues: The many faces of multi-level issues, (Vol. 1, pp. 3–36). Oxford, UK: Elsevier. Mumford, M. D., Scott, G. M., Gaddis, B., & Strange, J. M. (2002b). Leading creative people: Orchestrating expertise and relationships. Leadership Quarterly, 13, 705–750. Mumford, M. D., Supinski, E. P., Baughman, W. A., Costanza, D. P., & Threlfall, K. V. (1997). Process-based measures of creative problem-solving skills: I. Overall prediction. Creativity Research Journal, 10, 77–85. Mumford, M. D., & Van Doorn, J. R. (2001). The leadership of pragmatism: Reconsidering Franklin in the age of charisma. Leadership Quarterly, 12, 279–309. Nellore, R., & Balachandra, R. (2001). Factors inﬂuencing success in integrated product development (IPD) projects. IEEE Transactions on Engineering Management, 48, 164–173. Nohria, K., & Gulati, S. (1996). Is slack good or bad for innovation? Academy of Management Journal, 39, 799–825. Nonaka, I. (1990). Redundant, overlapping organization: A Japanese approach to managing the innovation process. California Management Review, 32, 27–38. Nystrom, H. (1979). Creativity and innovation. New York: Wiley. Nystrom, H. (1990). Organizational innovation. In: M. S. West & J. L. Farr (Eds), Innovation and creativity at work: Psychological and organizational strategies (pp. 143–162). New York: Wiley. O’Connor, G. C. (1998). Market learning and radical innovation: A cross case comparison of eight radical innovation projects. Journal of Product Innovation Management, 15, 151–166. Okuda, S. M., Runco, M. A., & Berger, D. E. (1991). Creativity and the ﬁnding and solving of real world problems. Journal of Psychoeducational Assessment, 9, 145–153.

Innovation in Organizations: A Multi-Level Perspective on Creativity

71

Oldham, G. R. (2003). Stimulating and supporting creativity in organizations. In: S. E. Jackson, M. A. Hitt & A. S. DeNisi (Eds), Managing knowledge for sustained competitive advantage (pp. 243–273). San Francisco, CA: Jossey-Bass. Oldham, G. R., & Cummings, A. (1996). Employee creativity: Personal and contextual factors at work. Academy of Management Journal, 39, 607–634. Olson, E. M., Walker, O. C., Ruekert, R. W., & Bonner, J. M. (2001). Patterns of cooperation during new product development among marketing, operations, and R&D. Journal of Product Development Management, 18, 258–271. Ong, C. H., Wan, D., & Chang, S. H. (2003). Factors affecting individual innovation: An examination within a Japanese subsidiary in Singapore. Technovation, 23, 617–633. Organ, D. W., & Greene, L. N. (1974). The perceived purposefulness of job behavior: Antecedents and consequences. Academy of Management Journal, 17, 69–77. Parnes, S. J., & Noller, R. B. (1972). Applied creativity: The creative studies project: Part results of a two year study. Journal of Creative Behavior, 6, 164–186. Pelz, D. C. (1967). Creative tensions in the research and development climate. Science, 157, 160–165. Pelz, D. C. (1968). Some social factors related to performance in a research organization. Administrative Science Quarterly, 13, 311–325. Pelz, D. C. (1983). Quantitative case histories of urban innovation: Are there innovating stages? IEEE Transactions on Engineering Management, 30, 60–67. Pelz, D. C., & Andrews, F. M. (1966). Autonomy, coordination, and simulation in relation to scientific achievement. Behavioral Science, 12, 89–97. Pelz, D. C., & Andrews, F. M. (1976). Scientists in organizations. Ann Arbor, MI: Institute for Social Research. Perry-Smith, J. E., & Shalley, C. E. (2003). The social side of creativity: A static and dynamic social network perspective. Academy of Management Review, 28, 89–106. Pierce, J., & Delbecq, L. A. (1977). Organizational structure, individual attitudes, and innovation. Academy of Management Review, 2, 27–37. Pirola-Merlo, A., Haertel, C., & Mann, L. (2002). How leaders inﬂuence the impact of affective events on team climate and performance in R&D teams. Leadership Quarterly, 13, 561–581. Pool, R. (1997). Beyond engineering: How society shapes technology. New York: Oxford University Press. Porac, J., & Thomas, H. (1990). Taxonomic mental model in competitor analysis. Academy of Management Review, 15, 224–243. Quinn, J. B. (1989). Technological innovation, entrepreneurship, and strategy. In: M. L. Tushman, C. O’Reilly & D. A. Adler (Eds), The management of organizations (pp. 549–581). New York: Harper & Row. Ramus, C. A. (2001). Organizational support for employees: Encouraging creative ideas for environmental sustainability. California Management Review, 43, 85–105. Redmond, M. R., Mumford, M. D., & Teach, R. J. (1993). Putting creativity to work: Leader inﬂuences on subordinate creativity. Organizational Behavior and Human Decision Processes, 55, 120–151. Reeves, L. M., & Weisburg, R. W. (1999). The role of content and abstract information in analogical transfer. Psychological Bulletin, 115, 381–400. Rinaldi, S., Cordone, R., & Casgrandi, R. (2000). Instabilities in creative professions: A minimal model. Nonlinear Dynamics, Psychology and Life Sciences, 4, 255–272.

72

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Rostan, S. M. (1994). Problem ﬁnding, problem solving, and cognitive controls: An empirical investigation of criticality acclaimed productivity. Creativity Research Journal, 7, 97–110. Runco, M. A., & Chand, I. (1994). Problem ﬁnding, evaluative thinking, and creativity. In: M. A. Runco (Ed.), Problem finding, problem solving, and creativity (pp. 40–76). Norwood, NJ: Ablex. Schwenk, C. R., & Cosier, R. A. (1980). Effects of the expert, devil’s advocate, and dialectical inquiry methods on prediction performance. Organizational Behavior and Human Decision Processes, 26, 409–424. Schwenk, C. R., & Thomas, H. (1973). Effects of conﬂicting analysis of managerial decision making: A laboratory experiment. Decision Sciences, 17, 467–482. Scott, R. K. (1995). Creative employees: A challenge to manage. Journal of Creative Behavior, 29, 64–71. Scott, S. G., & Bruce, R. A. (1994). Determinants of innovative behavior: A path model of individual innovation in the workplace. Academy of Management Journal, 37, 580–607. Sessa, V. I. (1998). E ¼ GR2P: A model for managing research and development teams. In: D. J. Sessa & J. L. Willett (Eds), Paradigm for the successful utilization of renewable resources (pp. 17–29). Champaign, IL: AOCS Press. Sethi, R., & Nicholson, C. (2001). Structural and contextual correlates of charged behavior in product development teams. Journal of Product Innovation Management, 18, 154–168. Shalley, C. A., Gilson, L. L., & Blum, T. C. (2000). Matching creativity requirements and the work environment: Effects on satisfaction and intentions to leave. Academy of Management Journal, 43, 215–223. Shalley, C. E. (1995). Effects of coaction, expected evaluation, and goal-setting on creativity and productivity. Academy of Management Journal, 38, 483–503. Shapiro, T. R. (1953). What scientists look for in their jobs. Scientific Monthly, 16, 335–340. Sharma, A. (1999). Central dilemmas of managing innovation in large ﬁrms. California Management Review, 41, 65–85. Sivadas, E., & Dwyer, F. R. (2000). An examination of organizational factors inﬂuencing new product success in internal and alliance-based processes. Journal of Marketing, 64, 31–49. Souitaris, V. (2001). External communication determinants of innovation in the context of a newly industrialized country: A comparison of objective and perceptual results from Greece. Technovation, 21, 25–34. Steck, R., & Sundermann, J. (1978). The effects of group size and cooperation on the success of interdisciplinary groups in R& D. R&D Management, 8, 59–64. Sternberg, R. J. (1988). A three facet model of creativity. In: R. J. Sternberg (Ed.), The nature of creativity (pp. 125–148). Cambridge, UK: Cambridge University Press. Stringer, R. (2000). How to manage radical innovation. California Management Review, 42, 70–88. Taggar, S. (2001). Group composition, creative synergy, and group performance. Journal of Creative Behavior, 35, 261–286. Taggar, S. (2002). Individual creativity and group ability to utilize individual creative resources: A multilevel model. Academy of Management Journal, 45, 315–330. Tesluk, P. E., Farr, J. L., & Klein, S. R. (1997). Inﬂuences of organizational culture and climate on individual creativity. Journal of Creative Behavior, 31, 27–41. Thamhain, H. J. (2003). Managing innovative R&D teams. R&D Management, 44, 297–322.

Innovation in Organizations: A Multi-Level Perspective on Creativity

73

Tierney, P., & Farmer, S. M. (2002). Creative self-efﬁcacy: Its potential antecedents and relationship to creative performance. Academy of Management Journal, 45, 1137–1148. Tierney, P., Farmer, S. M., & Graen, G. B. (1999). An examination of leadership and employee creativity: The relevance of traits and relationships. Personnel Psychology, 52, 591–620. Trevelyan, R. (2001). The paradox of autonomy: A case of academic research scientists. Human Relations, 54, 495–525. Troy, L. C., Szymanski, D. M., & Rajan, V. P. (2001). Generating new product ideas: An initial investigation of the role of market information and organizational characteristics. Journal of Academy of Marketing Science, 29, 89–101. Tushman, M. L., & O’Reilly, C. A. (1997). Winning through innovation. Cambridge, MA: Harvard Business School Press. Ward, T. B., Smith, S. M., & Finke, R. A. (1999). Creative cognition. In: R. J. Sternberg (Ed.), Handbook of creativity (pp. 189–213). Cambridge, UK: Cambridge University Press. Weintroff, R. H. (1992). The synthesis of diamonds. In: R. J. Weber & D. N. Perkins (Eds), Inventive minds: Creativity in technology (pp. 154–165). New York: Oxford University Press. Weisberg, R. W. (1999). Creativity and knowledge: A challenge to theories. In: R. J. Sternberg (Ed.), Handbook of creativity (pp. 226–259). Cambridge, UK: Cambridge University Press. West, M. A., Borill, C. S., Dawson, J. F., Brodbeck, F., Shapiro, D. A., & Haward, B. (2003). Leadership clarity and team innovation. Leadership Quarterly, 14, 246–278. Wise, G. (1992). Inventions and corporations in the maturing electrical industry. In: R. J. Weber & D. N. Perkins (Eds), Inventive minds: Creativity in technology (pp. 291–310). New York: Oxford University Press. Woodman, R. W., Sawyer, J. E., & Grifﬁn, R. W. (1993). Toward a theory of organizational creativity. Academy of Management Review, 18, 293–321. Zhou, J., & George, J. M. (2001). When job dissatisfaction leads to creativity: Encouraging the expression of voice. Academy of Management Journal, 44, 682–696. Zhou, J., & George, J. M. (2003). Awakening employees creativity: The role of leader emotional intelligence. Leadership Quarterly, 14, 461–478.

This page intentionally left blank

74

RESOLVING SOME PARADOXES OF INNOVATION: A CROSS-LEVEL INTERPRETATION Jane M. Howell and Kathleen Boies ABSTRACT This chapter on Mumford and Hunter’s chapter ‘‘Innovation in Organizations: A Multi-Level Perspective on Creativity’’ (this volume) describes both its contributions and limitations to the development of a cross-level theory of innovation. To resolve some of the cross-level paradoxes highlighted by Mumford and Hunter, we propose five variables that operate at multiple levels including trust, social identity, mental models, networks, and time, and formulate some new multi-level propositions. Future directions for innovation theory development and research are also discussed.

INTRODUCTION Reﬂecting on how innovation in organizations occurs and operates at multiple levels of analysis can significantly advance theory and research on this topic. By explicitly considering different levels of analysis, theory can be reﬁned, propositions can be more accurately formulated, and constructs can Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 75–92 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04002-6

75

76

JANE M. HOWELL AND KATHLEEN BOIES

be more precisely measured. This is of particular importance in the area of innovation where distinctions between levels of analysis are often fuzzy. With few exceptions (e.g., Tsai & Ghoshal, 1997; Van de Ven, Polley, Garud, & Venkataraman, 1999), differentiating between individuals, dyads, groups or teams, organizations, and environments is seldom discussed in detail. This chapter begins by noting the contributions of Mumford and Hunter’s chapter (this volume) to a cross-level innovation theory, as well as its limitations. We then attempt to resolve some of the cross-level inconsistencies noted by them. To do so, we propose variables that operate at multiple levels of analysis and then formulate new cross-level propositions using these variables in an effort to integrate effects at different levels. Specifically, we explore ﬁve variables that may reconcile the contradictions identiﬁed in cross-level interactions: trust, social identity, mental models, networks, and time. We conclude by suggesting some questions and directions for future theory development and research on innovation.

CONTRIBUTIONS AND LIMITATIONS OF MUMFORD AND HUNTER’S CHAPTER Mumford and Hunter’s chapter ‘‘Innovation in Organizations: A MultiLevel Perspective on Creativity’’ presents a comprehensive, indeed exhaustive, review and integration of the literature on creativity and innovation at four levels of analysis. They challenge our assumptions about creativity and innovation by synthesizing the extant literature and reaching counter-intuitive conclusions that cause us to rethink and reexamine our models in use. Mumford and Hunter’s chapter lays the foundation for the development of a multi-level theory of innovation by examining variables within and across levels of analysis. The review is well organized, highlighting themes, comparing and contrasting empirical results, ﬁnding discrepancies and commonalities among the results, and attempting to reconcile these discrepancies by proposing cross-level effects and/or other intervening variables. The chapter makes a compelling argument that cross-level interactions may account for the seemingly inconsistent ﬁndings in the innovation literature. Resolving these inconsistencies is critical for determining whether certain variables really matter in impeding or enhancing innovation or creative performance. Turning to the chapter’s limitations, Mumford and Hunter clearly differentiate between innovation, ‘‘the development and implementation of new ideas,’’ and creativity, ‘‘the initial generation of these new ideas,’’ yet

Resolving Some Paradoxes of Innovation

77

the chapter sometimes lumps the two together in the review of the literature and development of hypotheses. While Mumford and Hunter claim that confusion ‘‘arises from the apparently contradictory ﬁndings that seem endemic to studies of innovation,’’ the interchangeable use of the terms creativity and innovation may lead to some of the inconsistencies highlighted. For example, Mumford and Hunter cite a contradiction based on Oldham’s (2003) empirical ﬁndings that goals and extrinsic rewards inhibit creativity and Abbey and Dickson’s (1983) and Cardinal’s (2001) ﬁndings that goals, output processes, and extrinsic rewards are positively related to organizational innovation. While Mumford and Hunter attribute this contradictory ﬁnding to different levels of analysis (individual versus organization), it is likely that the apparent inconsistency is also due to different outcome variables (creativity versus innovation). During idea generation, a contracted-for reward can undermine individual creativity (e.g., Amabile, 1988; Amabile, Hennessey, & Grossman, 1986), yet, the implementation of innovation requires goal setting and evaluation standards as Mumford and Hunter have noted. Although Mumford and Hunter attribute some of the contradictory ﬁndings to different levels of analysis, the apparent inconsistencies may also be explained by certain variables operating at multiple levels of analysis (e.g., trust, social networks, etc.), different outcome variables (e.g., resource exchange, incremental innovation, or radical change), and/or to the life cycle of the innovation process. Indeed, the impact of a critical variable, time, and the dynamic nature of the innovation process are not addressed directly by Mumford and Hunter; rather a static picture of the messy, chaotic nature of innovation is presented. However, these factors may account for some of the inconsistent ﬁndings in the literature, as well as the differences in requirements for innovation at one level of analysis as compared to another level of analysis. Building on Mumford and Hunter’s solid foundation of a cross-level theory of innovation in organizations, we propose ﬁve variables in an attempt to resolve some of the contradictions between levels of analysis and formulate new multi-level propositions. In the following sections, we discuss the roles of trust, social identity, mental models, networks, and time in resolving these cross-level contradictions.

TRUST AND SOCIAL IDENTITY Mumford and Hunter point out the inconsistency between the need for collaboration and cohesion at the group level versus the need for

78

JANE M. HOWELL AND KATHLEEN BOIES

achievement at the individual level (cross-level interaction two). Two constructs, trust and social identity, may reconcile the contradiction between individual achievement, and group collaboration and cooperation. Trust is deﬁned as a willingness to be vulnerable to someone’s actions based on the belief that these will not be harmful, and despite the lack of ability to control or monitor these actions (Mayer, Davis, & Schoorman, 1995). It has been suggested that trust forms the basis of collaboration (Bromiley & Cummings, 1995; Jones & George, 1998; Williams, 2001) and therefore may play an important role in the innovation process. A few studies have examined the role of trust in the context of innovation. Tsai and Ghoshal (1997) deﬁned trust as a dimension of social capital and found that it was positively related to collaborative exchange (e.g., information, human resources, and so on). In turn, collaborative exchange was positively related to value creation for the organization through radical product innovation (see also Nahapiet & Ghoshal, 1998). In a study of managerial decision-making, Ford and Gioia (2000) reported that trust was related to creative (novel and valuable) solutions. Deﬁning trust in the specific context of innovation, Clegg, Unsworth, Epitropaki, and Parker (2002) suggested the existence of two dimensions: a belief that top management would (1) pay attention to employees’ suggestions, and (2) have their employees’ best interests at heart. These two dimensions had differential relations to idea generation and implementation. Specifically, when design engineers believed that top management was attentive to their suggestions, they were more likely to expend effort in order to implement the idea. On the other hand, when these employees believed that top management had their best interests in mind, they were more likely to contribute by making suggestions and generating ideas. These ﬁndings are particularly useful in resolving the apparent contradiction between the need for collaboration and cohesion at the group level versus the need for achievement at the individual level (cross-level interaction two). Given employees’ trust in top management, myopic self-interest may be relegated to a lower priority, whereas making a unique contribution to the organization and its purpose, and engaging in activities that are meaningful and genuinely needed, take precedence. These ﬁndings indicate that trust may promote collaboration (i.e., the free ﬂow of information among individuals, teams, and organizations). This in turn will facilitate creativity and innovation by creating conditions conducive to these activities (McEvily, Perrone, & Zaheer, 2003). Extending Clegg et al.’s (2002) arguments, trust may not only have a direct role in the innovation process, but will likely operate at multiple

Resolving Some Paradoxes of Innovation

79

levels. Trust is an inherently multi-level phenomenon: it occurs between a trustor, the party engaging in the action, and a trustee, the party in which trust is placed. Indeed, trust has been operationalized at the individual level as a dispositional tendency (Rotter, 1967), at the dyadic level (e.g., Podsakoff, MacKenzie, & Bommer, 1996), at the group level (e.g., Dirks, 1999), and at the organizational, or interﬁrm, level (e.g., Carson, Madhok, Varman, & John, 2003). In addition, trust can be conceptualized at the societal level. More specifically, in an extensive review of the trust literature at multiple levels (individual, dyadic, and societal), Reeves-Ellington (2004) highlighted the role of culture (using Bulgaria, Japan, and Indonesia as illustrative cases) in the creation of climates conducive to trust or distrust. This societal level conceptualization of trust can be linked to Mumford and Hunter’s environmental level of analysis. Trust also operates across these different levels. It is conceivable, for example, to think of teams that would trust or distrust their manager, other teams at the same level, or their organization. Trust in different referents also appears to have different antecedents and outcomes (e.g., Tan & Tan, 2000), thus highlighting the need for a multi-level conceptualization of trust. Given the above arguments, new multi-level propositions are introduced: Proposition 1a. Trust in referents at different levels of analysis (dyad, group, organization, environment) will be positively related to collaboration. Proposition 1b. Collaboration at different levels of analysis (dyad, group, organization, environment) will be positively related to idea generation and innovation. Beyond trust, social identity theory (Ashforth & Mael, 1989; Tajfel & Turner, 1985) may help to reconcile the inherent contradiction of personal achievement at the individual level and cohesion and collaboration at the group level posited in Mumford and Hunter’s cross-level interaction two. This theory states that individuals’ self-evaluations are attributable, in part, to their group membership. Through a relational, comparative categorization process, individuals segment their social environment based on categories such as education, profession, or group membership, and then determine their place within this environment. Social identiﬁcation with the group or organization is also affected by individuals’ judgments of the prestige of the group, distinctiveness, and group formation factors such as similarity, liking, proximity, or shared goals (Ashforth & Mael, 1989). Since identiﬁcation inﬂuences commitment to and internalization of group values

80

JANE M. HOWELL AND KATHLEEN BOIES

and norms, we expect that when individual identiﬁcation with the group is high, cohesiveness and collaboration will be higher at group and organizational levels. Proposition 2. Individual social identiﬁcation with a higher level unit (group or organization) will be positively related to cohesion and collaboration at different levels of analysis (group or organization).

MENTAL MODELS Another paradox of innovation highlighted by Mumford and Hunter is the differentiation of expertise at the group and organizational levels versus the integration of expertise at the group level (cross-level interaction three). Expertise provides an organizing framework for creative individuals. It is a mental model that helps them organize their knowledge and hence understand and predict events (Mathieu, Heffner, Goodwin, Salas, & CannonBowers, 2000; Rouse & Morris, 1986). Rentsch and Hall (1994) reported that individuals with more experience had more organized and sophisticated mental models which they could generalize more easily across situations. Given the complexity and ambiguity of creative undertakings, individuals alone will not possess all the knowledge and expertise necessary to bring a creative idea to fruition. Therefore, as Mumford and Hunter rightly point out, groups and organizations need to possess adequate coverage of knowledge and expertise (differentiation), and this expertise needs to be integrated. Shared mental models, which are knowledge structures that are similar among team members (Cannon-Bowers, Salas, & Converse, 1993), could act to integrate expertise and specialization. The same way that individual-level cognitive processes serve as an organizing framework for creative individuals, shared mental models represent a common interpretive context that can help teams interpret, organize, gather, and act on information. They allow individuals to form accurate expectations about their teammates and therefore to coordinate more effectively. Shared mental models may contain information about different knowledge domains. Research in the area however has focused on two dimensions that are most relevant to the current discussion: teamwork and taskwork mental models (Klimoski & Mohammed, 1994; Mathieu et al., 2000). Teamwork mental models contain information about each team member’s roles and responsibilities, the patterns of interactions among team members (e.g., communication channels), sources of information, and each team

Resolving Some Paradoxes of Innovation

81

member’s knowledge, skills, and abilities. Taskwork mental models contain information related to the task definition, the procedures and strategies involved, the problems that may be encountered, the environmental conditions, and the available equipment (Mathieu et al., 2000; Rentsch & Hall, 1994). Thus, both teamwork and taskwork mental models relate to basic facts and procedures necessary for effective team functioning. Shared mental models are expected to increase team performance by fostering more effective team processes. More specifically, shared mental models have been related to more effective communication, coordination, interpersonal relations, and cooperation in teams (Marks, Zaccaro, & Mathieu, 2000; Mathieu et al., 2000). This in turn leads to greater team performance. Extending these ﬁndings to the context of innovation, individuals with diverse expertise but more similar mental models may be able to coordinate more effectively and make decisions more rapidly, which may lead to greater speed of innovation development. Thus a shared interpretive context could help the team integrate otherwise disparate expertise by providing a common frame of reference. A shared interpretive context, however, can only help to the extent that it is functional and aligned with the goals of the group and organization. In other words, sharing dysfunctional beliefs or cognitions could be detrimental to the innovation process. Campbell (2000), for example, underscores the unanticipated consequences that can result when individuals exercise initiative, independent judgment, and creativity. To eliminate these unintended effects while at the same time encouraging individual idea generation, he recommends that managers create shared perspectives and common frameworks by identifying and communicating the organization’s core values explicitly, and sharing information. Thus, shared mental models in groups may support the innovation process. Furthermore, according to Cannon-Bowers and Salas (2001), certain elements of knowledge should be shared (or overlapping), certain elements should be identical, and other elements should be distributed. Although further research is called for to determine what must be shared, we can speculate that a certain level of knowledge or expertise needs to be distributed (and complementary), and that innovation team members need to agree on basic facts and procedures related to team functioning (i.e., taskwork and teamwork mental models). Austin (2003) studied multiple dimensions of shared cognitions (transactive memory), including specialization and agreement, as well as different knowledge content. He found that the different dimensions of shared cognitions all contributed to team performance. In sum, innovation team members need to agree on basic facts and procedures (to facilitate

82

JANE M. HOWELL AND KATHLEEN BOIES

integration), and possess diverse skills and abilities and potentially different ways to approach the problems at hand (differentiation). The basic agreement for group process needs to encourage constructive dissent and respect ﬂexibility in problem solving. Although shared mental models have been conceptualized at the team level of analysis, the notion of shared cognitions can generalize to higher levels of analysis, such as the organization, where communication of an overarching vision and core values can serve to align individual, team, and organizational interests and create coherent action. Shared mental models at the organizational level could serve to integrate the expertise of different teams working on a common product innovation. Proposition 3a. Shared mental models (at the team and organizational levels) will be positively related to coordination and integration of expertise. Proposition 3b. Coordination and integration of expertise (in groups or organizations) will be positively related to the speed of idea development and implementation.

NETWORKS If groups or organizations do not have access to all of the expertise necessary for them to innovate successfully, then they need to draw on external resources. It is therefore helpful for group members to have access to networks outside of their unit. Austin (2003) reported that teams that had knowledge of what resources other team members had access to functioned more effectively. In other words, networks may be an important component of creativity and innovation. Mumford and Hunter acknowledge the multi-level role of networks in innovation. They briefly discuss the role of networks at the individual level as a way to acquire knowledge. At the group level, they mention the importance of networks for facilitating creativity and innovation by providing support for creative people. Although individual and group social networks both internal and external to the organization are required for creativity and innovation, Mumford and Hunter only specify networks as a variable operating at the environmental level. This conceptualization may underplay the important role of networks in facilitating the innovation process at multiple levels.

Resolving Some Paradoxes of Innovation

83

An actor’s position in a network, be it an individual, a team, or an organization, may help to explain the relationship between social networks and creativity or innovation. Position is often described in terms of centrality, or an actor’s position relative to all other actors in the network. Less central positions (peripheral) are presumably linked to other networks, and composed of a number of weak contacts. Actors in peripheral positions would presumably have access to other networks, and to more information and a greater exposure to new ideas, both key factors in idea generation. This links to individuals’ and teams’ capacity to scan their environment for information, which is related to innovation (e.g., Ancona & Caldwell, 1992; Howell & Shea, 2001). For example, prior research has demonstrated that champions of innovation scan their environment to identify new opportunities by relying on their personal interactions with customers, suppliers, competitors or colleagues in other organizations (Howell & Shea, 2001). Similarly, Ancona and Caldwell (1992) reported that new-product development teams scanned for ideas and information about the competition, the market or the technology. These ﬁndings complement the social network literature which suggests that the existence or strength of relationships is pivotal for gaining access to information and identifying opportunities for new ventures (Burt, 1992; Granovetter, 1973). Collectively, these studies help to reconcile Mumford and Hunter’s cross-level interaction four regarding the countervailing forces of external focus at the organizational level versus internal focus at the individual level. Thus, actors’ position in a network may determine their ability to scan their environment for new ideas while allowing these ideas to be linked to external trends that will inﬂuence the need for innovation at the organization level. Peripheral positions are not as constraining as central positions in a network. In peripheral positions, actors are under less obligation to conform to established social norms; this, in turn, increases their autonomy from their network, thus enhancing their creative potential. Actors in peripheral positions may also connect two separate networks or, in Burt’s (1992) terms, bridge structural holes. This may represent entrepreneurial or innovative opportunities not afforded to actors who are central in their network. According to Perry-Smith and Shalley (2003), a moderate degree of centrality would be optimal for enhancing creativity. Actors in moderately central positions would beneﬁt from the advantages of information access mentioned above while having ties strong enough in their own network to take moderately informed risks. Thus, network centrality may be an important determinant of creativity.

84

JANE M. HOWELL AND KATHLEEN BOIES

Perry-Smith and Shalley (2003) go further in proposing a mutual causal relationship between position centrality and creativity that takes the form of a spiral. In a moderately central position, actors have enough information to allow them to take informed risks and create. However, as actors become successful in generating ideas, they serve as a source of information and support for others, and slowly move more toward the center of the network. While this ‘‘extreme’’ centrality can be detrimental to the creative process, it also means that actors now have strong contacts on which they can rely in order to receive approval for implementing creative ideas. Thus, while a certain degree of centrality may be important for gathering information and generating new ideas, more centrality represents a greater opportunity to inﬂuence implementation and adoption of an innovation. Proposition 4a. Network centrality of a unit (individual, group, or organization) will be negatively related to environmental scanning and idea generation. Proposition 4b. Network centrality of a unit (individual, group, or organization) will be positively related to innovation implementation. In addition, networks may also help to clarify a single-level proposition. At the group level, Mumford and Hunter suggest that structures that allow the integration of multiple small teams should encourage collaboration and communication (group-level proposition seven). This proposition, however, clearly focuses on the formal structure of the team. Although we agree with this statement, a more complete picture of the integration of teams can be drawn by the addition of social networks, the informal structure. Thus, while the structural integration of multiple small teams may indeed beneﬁt the innovation process by giving access to a greater degree of expertise and information, the position of these teams within their network might be of even greater value. We expect that the integration of multiple small teams who form a closed network with few boundary-spanning ties with other networks will not be as effective in promoting idea generation as a network with multiple small teams who have access to multiple networks. This idea can also extend to other levels of analysis. Individuals’ and organizations’ position in their respective network and the number of their boundaryspanning ties may also promote idea generation. Proposition 5. The number of boundary-spanning ties of a unit (individual, group, or organization) in a network will be positively related to idea generation.

Resolving Some Paradoxes of Innovation

85

In summary, trust, social identity, mental models, and networks may help to explain some of the cross-level contradictions highlighted by Mumford and Hunter. These variables are proposed to operate at multiple levels. For higher-level processes to occur, however, interactions, and therefore time, are needed (Kozlowski & Klein, 2000). The notion of time is indeed critical for all of these variables. For example, the relation between creativity and network centrality will take the form of a spiral over time (Perry-Smith & Shalley, 2003). In addition, the notion of time is implicit in shared mental models (e.g., Gibson, 2001). Finally, the development of trust clearly occurs over time, and the cycle of trust has been mentioned by several authors (e.g., Jones & George, 1998). We now turn to a discussion of time in the innovation process.

TIME Levels of analysis and time are intimately related (Dansereau, Yammarino, & Kohles, 1999). Over time, levels of analysis can change up (e.g., from individual, to dyads, to groups, and so on), change down, or remain stable. Although the mechanisms by which these changes occur are complex, it is essential to examine the interaction between levels of analysis and time in order to understand the innovation process. Some studies have highlighted the inﬂuence of time in the innovation process. For example, researchers have reported different predictors of idea generation and implementation (Axtell et al., 2000; Clegg et al., 2002). To illustrate, in a study of incremental innovation by shop ﬂoor employees, Axtell et al. (2000) found that lower-level variables such as role breadth selfefﬁcacy (a belief in one’s ability to take on more proactive tasks) predicted idea generation, whereas group and organizational variables (e.g., team support for innovation and participation in decision-making) played a greater role in idea implementation. Although Axtell et al. (2000) did not adopt a longitudinal design, their results suggest that the relative importance of predictors at different levels of analysis depends on the stage of the innovation process, which implies the notion of time. Mumford and Hunter point out the need for control at the organizational level versus autonomous exploration at the individual level (cross-level interaction one). Although the intervention of multiple levels is clearly a factor in the resolution of this interaction, examining the progression of events in the innovation development process may also be useful.

86

JANE M. HOWELL AND KATHLEEN BOIES

In their longitudinal ﬁeld studies of 14 innovations Van de Ven et al. (1999) identiﬁed three temporal periods during the innovation process: initiation, developmental, and implementation/termination. They reported that during the initiation period that often spanned 3 years or more, a conﬂuence of events triggered the recognition of a feasible new program or business idea. Multiple coincidental events, activities, and interactions led focal players to recognize and access potential opportunities for their organization. During this period, many new innovative ideas were generated. At the juncture of the end of the initiation period and the outset of the developmental period, the creation and submission of business plans, timetables, and budgets to senior managers or venture capitalists is needed to obtain resources to launch an innovation. Focal players are then held accountable for meeting the targets and schedules speciﬁed in the business plans to evaluate innovation progress. Thus, early in the innovation process autonomous exploration may spawn new ideas for development, whereas in the developmental period that follows, control is imposed to track innovation progress. To resolve the contradiction highlighted in cross-level interaction one, we propose: Proposition 6. Autonomous exploration at the individual level will be positively related to the initiation period. Proposition 7. Control at the organizational level will be positively related to the developmental period.

FUTURE DIRECTIONS This article has focused on resolving some of the seeming inconsistencies in the cross-level propositions advanced by Mumford and Hunter by considering different variables that operate at multiple levels of analysis: trust, social identity, mental models, networks, and time. We then outlined testable propositions to guide future research on innovation in organizations across levels of analysis. Other questions remain. For example, as Mumford and Hunter point out, the inﬂuence of affect on creativity and innovation requires further exploration. Drawing on Angle and Van de Ven’s (1989) work, the emotions of innovation participants change during the initiation, developmental, and implementation/termination periods. During the start-up period, participants are enthusiastic and conﬁdent in the success of the innovation. As problems come to light and the stark reality of the challenge and complexity

Resolving Some Paradoxes of Innovation

87

of the innovation effort become apparent in the developmental period, enthusiasm declines and erosion of trust and conﬁdence in peers and leadership occurs. In the implementation/termination period, if the innovation was successful, then the innovation team celebrates the ‘‘win’’ and the talent, commitment, and exceptional efforts of the team are heralded. Conversely, if the innovation is judged as a failure, then team members point to uncontrollable external factors that undermined their success, or rationalize why their efforts were not wasted. These emotions of innovation teams may or may not be shared by individual participants since their involvement in the innovation ebbs and ﬂows as the innovation develops over time (Van de Ven et al., 1999). The role of emotions in creativity has also been highlighted by Zhou and her colleagues. For example, George and Zhou (2002) found that dissatisfaction, or negative emotions, fueled creative ideas. Specifically, negative mood led to higher creativity when employees were aware of their own feelings and had clear creativity goals. Positive mood displayed the opposite pattern, whereby it was negatively related to creativity when both feeling awareness and clarity of creative goals were high. These results highlight the importance of context in determining the roles of positive and negative emotions. These different streams of research prompt several questions related to the role of affect in organizational innovation. For example, how does affect inﬂuence communication, cooperation, and resource exchange? Does affect at different levels of analysis lead to different outcomes of innovation? What is the role of affect at different points in the innovation life cycle? What is the relative importance of positive and negative emotions and how does their role change over time? In their group-level propositions, Mumford and Hunter mention the need for a ‘‘clear’’ or ‘‘shared’’ mission (Propositions 2, 3, and 5). However, the need to articulate an overarching purpose that is common to all innovation team members and which they internalize is not clearly reﬂected in the crosslevel interactions, despite its importance to the innovation process. For example, in Ford and Gioia’s (2000) study of creative managerial decisionmaking, the existence of a common perspective was identiﬁed as an important factor. Therefore, an issue to be explored in future research is whether the communication of an overarching goal will align personal goals with those of the team and the organization, and thereby increase collaboration and information exchange within the innovation team. Another area for future inquiry is the possibility of efﬁcacy-performance spirals in individuals, groups, and organizations pursuing innovation.

88

JANE M. HOWELL AND KATHLEEN BOIES

Drawing on Bandura’s (1986) work on self-efﬁcacy, the belief in one’s ability to master a task, Lindsley, Brass, and Thomas (1995, p. 650) use the term spiral to describe ‘‘a pattern of consecutive increases (or decreases) in both perceived efﬁcacy and performance over a minimum of three task attempts.’’ Three patterns are suggested: an upward spiral, a downward spiral, and a self-correcting cycle. An upward or downward spiral exists when changes in efﬁcacy and performance, either consistently positively (resulting in overconﬁdence or complacency) or consistently negatively (resulting in lack of conﬁdence), build on each other. A self-correcting cycle occurs ‘‘when a decrease in performance and self-efﬁcacy is followed by an increase in performance or self-efﬁcacy (or vice versa)’’ (Lindsley et al., 1995, p. 650). Lindsley et al. (1995) assume that self-correcting cycles are preferable to either upward or downward spirals since it spurs active experimentation necessary for learning and innovation. The notion of spirals can likely extend to the context of innovation. Tierney and Farmer (2002) advanced the concept of creative self-efﬁcacy, or the belief in one’s ability to generate novel and useful ideas. Furthermore, Lindsley et al. (1995) proposed several variables that are related to the innovation process to be either negatively (e.g., the availability of accurate, timely, and specific performance feedback) or positively related (e.g., task complexity, and uncertainty and emotional arousal) to the occurrence of spirals. For example, with increasing task uncertainty and complexity, individuals rely on group performance, and groups rely on organizational performance to reach efﬁcacy judgments. Given that the innovation process is inherently uncertain, dynamic, and complex (Van de Ven et al., 1999), it seems likely that individuals’ belief in their ability to innovate will be inﬂuenced by the innovation team’s performance, and the innovation team’s performance will be affected by the performance of the organization. Several research questions related to efﬁcacy and innovation come to mind. How do individual efﬁcacy judgments relate to innovation inﬂuence choices of subsequent actions and performance? How do shifting and conﬂicting innovation performance criteria used by resource controllers and innovation managers (Van de Ven et al., 1999) affect an individual’s interpretation of innovation progress? How do upward efﬁcacy-performance spirals contribute to innovation learning disabilities and performance failures, such as escalating commitment to a failing course of action? How do efﬁcacy-performance spirals change as the innovation moves from the initiation, to the developmental, and ﬁnally the implementation/termination period? Mumford and Hunter’s chapter focuses on the facilitating and constraining conditions for idea generation and innovation. Taking a different

Resolving Some Paradoxes of Innovation

89

perspective, what are the dynamics of withdrawal from an innovation? Royer (2001) has discussed the idea of ‘‘stopping champions’’ of failing innovation projects. Stopping champions use various sociopolitical tactics and their networks to rally supporters to halt a failing project, pose strategic and ﬁnancial arguments for project withdrawal, and involve innovation team members in establishing decision criteria to evaluate whether to continue or exit a project. Exploring how the interactions evolve between key players that can facilitate effective decisions about initial and continuing investment in innovation projects would be a worthwhile direction for future research. In conclusion, adopting a multi-level perspective recognizes that individuals, groups, and organizations do not operate in isolation, but are parts of a whole, each inﬂuencing and being inﬂuenced by the other. As Rousseau (1985) points out, failure to account for multiple levels can lead to misspeciﬁed theoretical models. Moreover, management interventions at one level may have unintended effects at a different level. Given the ambiguity, complexity, and ﬂuidity inherent in the innovation process, it is imperative to consider cross-level effects. In our article we proposed ﬁve constructs that may form the basis of a parsimonious explanation for some of the crosslevel interactions posited by Mumford and Hunter. Although the role of these constructs needs to be discussed at greater length, we hope that it may shed some light on the innovation process and stimulate further research in the area.

REFERENCES Abbey, A., & Dickson, J. (1983). R&D work climate and innovation in semiconductors. Academy of Management Journal, 25, 362–368. Amabile, T. M. (1988). A model of creativity and innovation in organizations. In: B. M. Staw & L. L. Cummings (Eds), Research in organizational behavior, (Vol. 10, pp. 123–167). Greenwich, CT: JAI Press. Amabile, T. M., Hennessey, B. A., & Grossman, B. S. (1986). Social inﬂuences on creativity: The effects of contracted-for reward. Journal of Personality and Social Psychology, 50, 950–967. Ancona, D. G., & Caldwell, D. F. (1992). Bridging the boundary: External activity and performance in organizational teams. Administrative Science Quarterly, 37, 634–665. Angle, H. L., & Van de Ven, A. H. (1989). Suggestions for managing the innovation journey. In: A. H. Van de Ven, H. L. Angle & M. S. Poole (Eds), Research on the management of innovation: The Minnesota studies (pp. 663–698). New York: Ballinger/Harper & Row. Ashforth, B. E., & Mael, F. (1989). Social identity theory. Academy of Management Review, 14, 20–39.

90

JANE M. HOWELL AND KATHLEEN BOIES

Austin, J. R. (2003). Transactive memory in organizational groups: The effects of content, consensus, specialization, and accuracy on group performance. Journal of Applied Psychology, 88(5), 866–878. Axtell, C. M., Holman, D. J., Unsworth, K. L., Wall, T. D., Waterson, P. E., & Harrington, E. (2000). Shopﬂoor innovation: Facilitating the suggestion and implementation of ideas. Journal of Occupational and Organizational Psychology, 73, 265–284. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall. Bromiley, P., & Cummings, L. L. (1995). Transaction costs in organizations with trust. In: R. J. Bies, B. Sheppard & R. J. Lewicki (Eds), Research on negotiations in organizations, (Vol. 5, pp. 219–247). Greenwich, CT: JAI. Burt, R. S. (1992). Structural holes: The social structure of competition. Cambridge, MA: Harvard University Press. Campbell, D. J. (2000). Empowerment: Rejuvenating a potent idea. Academy of Management Executive, 14(3), 67–80. Cannon-Bowers, J. A., & Salas, E. (2001). Reﬂections on shared cognition. Journal of Organizational Behavior, 22, 195–202. Cannon-Bowers, J. A., Salas, E., & Converse, S. (1993). In: N. J. Castellan Jr. (Ed.), Individual and group decision making: Current issues. Hillsdale, NJ: Lawrence Erlbaum. Cardinal, L. B. (2001). Technological innovation in the pharmaceutical industry: The use of organizational control in managing research and development. Organization Science, 12, 19–37. Carson, S. J., Madhok, A., Varman, R., & John, G. (2003). Information processing moderators of the effectiveness of trust-based governance in interﬁrm R&D collaboration. Organization Science, 14(1), 45–56. Clegg, C., Unsworth, K., Epitropaki, O., & Parker, G. (2002). Implicating trust in the innovation process. Journal of Occupational and Organizational Psychology, 75, 409–422. Dansereau, F., Yammarino, F. J., & Kohles, J. C. (1999). Multiple levels of analysis from a longitudinal perspective: Some implications for theory building. Academy of Management Review, 24(2), 346–357. Dirks, K. T. (1999). The effects of interpersonal trust on work group performance. Journal of Applied Psychology, 84(3), 445–455. Ford, C. M., & Gioia, D. A. (2000). Factors inﬂuencing creativity in the domain of managerial decision making. Journal of Management, 26(4), 705–732. George, J. M., & Zhou, J. (2002). Understanding when bad moods foster creativity and good ones don’t: The role of context and clarity of feelings. Journal of Applied Psychology, 87, 687–697. Gibson, C. B. (2001). From knowledge accumulation to accommodation: Cycles of collective cognition in work groups. Journal of Organizational Behavior, 22, 121–134. Granovetter, M. S. (1973). The strength of weak ties. American Journal of Sociology, 78, 1360–1380. Howell, J. M., & Shea, C. M. (2001). Individual differences, environmental scanning, innovation framing, and champion behavior: Key predictors of project performance. Journal of Product Innovation Management, 18, 15–27. Jones, G. R., & George, J. M. (1998). The experience and evolution of trust: Implications for cooperation and teamwork. Academy of Management Review, 23(3), 531–546.

Resolving Some Paradoxes of Innovation

91

Klimoski, R., & Mohammed, S. (1994). Team mental model: Construct or metaphor? Journal of Management, 20(2), 403–437. Kozlowski, S. W. J., & Klein, K. J. (2000). A multi-level approach to theory and research in organizations: Contextual, temporal, and emergent processes. In: K. J. Klein & S. W. J. Kozlowski (Eds), Multi-level theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 3–90). San Francisco: Jossey-Bass. Lindsley, D. H., Brass, D. J., & Thomas, J. B. (1995). Efﬁcacy-performance spirals: A multilevel perspective. Academy of Management Review, 20, 645–678. Marks, M. A., Zaccaro, S. J., & Mathieu, J. E. (2000). Performance implication of leader briefings and team-interaction training for team adaptation to novel environments. Journal of Applied Psychology, 85(6), 971–986. Mathieu, J. E., Heffner, T. S., Goodwin, G. F., Salas, E., & Cannon-Bowers, J. A. (2000). The inﬂuence of shared mental models on team process and performance. Journal of Applied Psychology, 85(2), 273–283. Mayer, R., Davis, J., & Schoorman, F. (1995). An integrative model of organizational trust. Academy of Management Review, 20(3), 709–734. McEvily, B., Perrone, V., & Zaheer, A. (2003). Trust as an organizing principle. Organization Science, 14(1), 91–103. Nahapiet, J., & Ghoshal, S. (1998). Social capital, intellectual capital, and the organizational advantage. Academy of Management Review, 23(2), 242–266. Oldham, G. R. (2003). Stimulating and supporting creativity in organizations. In: S. E. Jackson, M. A. Hitt & A. S. DeNisi (Eds), Managing knowledge for sustained competitive advantage (pp. 243–273). San Francisco, CA: Jossey-Bass. Perry-Smith, J. E., & Shalley, C. E. (2003). The social side of creativity: A static and dynamic social network perspective. Academy of Management Review, 28(1), 89–106. Podsakoff, P. M., MacKenzie, S. B., & Bommer, W. H. (1996). Transformational leader behaviors and substitutes for leadership as determinants of employee satisfaction, commitment, trust, and organizational citizenship behaviors. Journal of Management, 22(2), 259–298. Reeves-Ellington, R. (2004). Trustscapes and distrustscapes: A multi-level approach for understanding stability and change. In: F. J. Yammarino & F. Dansereau (Eds), Research in multi-level issues, (Vol. 3, pp. 91–150). Amsterdam: JAI Press. Rentsch, J. R., & Hall, R. J. (1994). Members of great teams think alike: A model of team effectiveness and schema similarity among team members. In: M. Beyerlein (Ed.), Advances in interdisciplinary studies of work teams, (Vol. 1, pp. 223–261). Greenwich, CT: JAI Press. Rotter, J. B. (1967). A new scale for the measurement of interpersonal trust. Journal of Personality, 35(4), 651–665. Rouse, W. B., & Morris, N. M. (1986). On looking into the black box: Prospects and limits in the search for mental models. Psychological Bulletin, 100, 349–363. Rousseau, D. M. (1985). Issues of level in organizational research: Multi-level and cross-level perspectives. In: L. L. Cummings & B. M. Staw (Eds), Research in organizational behavior, (Vol. 7, pp. 1–37). Greenwich, CT: JAI Press. Royer, I. (2001). Stopping-champions of failing projects. Paper presented at the Academy of Management Conference, Washington, DC, August.

92

JANE M. HOWELL AND KATHLEEN BOIES

Tajfel, H., & Turner, J. C. (1985). The social identity theory of intergroup behavior. In: S. Worchel & W. G. Austin (Eds), Psychology of intergroup relations, (2nd ed.) (pp. 7–24). Chicago: Nelson-Hall. Tan, H. H., & Tan, C. S. F. (2000). Toward the differentiation of trust in supervisor and trust in organization. Genetic Social and General Psychology Monograph, 126(2), 241–260. Tierney, P., & Farmer, S. M. (2002). Creative self-efﬁcacy: Its potential antecedents and relationship to creative performance. Academy of Management Journal, 45, 1137–1148. Tsai, W., & Ghoshal, S. (1997). Social capital and value creation: The role of intraﬁrm networks. Academy of Management Journal, 41(4), 464–476. Van de Ven, A. H., Polley, D. E., Garud, R., & Venkataranman, S. (1999). The innovation journey. New York: Oxford University Press. Williams, M. (2001). In whom we trust: Group membership as an affective context for trust development. Academy of Management Review, 26(3), 377–396.

‘‘WE WANT CREATIVITY! NO, WE DON’T!’’ Robert J. Sternberg ABSTRACT This chapter complements that of Mumford and Hunter (this volume) by pointing out ways in which creativity can be more or less rewarded, depending on its type. Whereas Mumford and Hunter (this volume) discuss levels of organization, I discuss here different kinds of creative contributions. These contributions can either accept existing paradigms, propose new ones, or integrate new ones with old ones.

Mumford and Hunter (this volume) have written a tour de force regarding the prickly subject of creativity in organizational settings. They show how creativity can have different effects at different levels of an organization. What is valued at one level may not be valued at another.

THE CREATIVITY PARADOX Mumford and Hunter’s discussion is relevant to what might be called the creativity paradox: On the one hand, it would be hard to ﬁnd an organization that does not say it values creativity; on the other hand, it Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 93–103 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04003-8

93

94

ROBERT J. STERNBERG

would be hard to ﬁnd an organization that, when it gets creativity, uniformly values it. Mumford and Hunter (this volume) suggest that the creativity paradox can be resolved by analyzing organizational creativity at different levels. Creativity may be viewed differently at different levels of an organization. For example, a novel idea that may seem perfectly reasonable at a lower level may be seen, at a higher level, as counter to the organizational culture of the organization. My goal in this brief chapter is simply to offer some personal reﬂections on the point made by Mumford and Hunter (this volume). My article will not be critical, in that I fully agree with the main point the authors make. Rather, I will simply provide a modest elaboration of their main point, suggesting a complementary and, I believe, compatible way of viewing a possible resolution to the creativity paradox. Our ﬁrst introduction to what I referred to as the ‘‘creativity paradox’’ was in a study in which we found that management experts, unlike experts in other ﬁelds we studied (physics, philosophy, and art), perceived an inverse relationship between creativity and wisdom. That is, they viewed wise managers as less creative, and more creative managers as less wise (Sternberg, 1985). This result seemed odd. Why would it be unwise to be creative?

THE PROPULSION THEORY OF CREATIVE CONTRIBUTIONS Some years later, my colleagues and I proposed a theory that we believed helped to resolve the creativity paradox. Mumford and Hunter’s solution is to look at different levels of the organization that may differentially value creativity. Certainly this is true. I suggest an additional mechanism – that there are different kinds of creativity (Sternberg, 1999; Sternberg, Kaufman, & Pretz, 2002, 2003). Others, of course, have also suggested different kinds of creativity. Csikszentmihalyi (1996) has discussed the notion that creativity in a domain (e.g., coming up with ideas in physics) might be different from creativity in a ﬁeld (e.g., coming up with fundable ideas in physics). Gardner (1993) has suggested that creativity takes somewhat different forms in different domains (e.g., linguistics versus logical-mathematics). Our view of different kinds of creativity is more similar to Kuhn’s (1970), in emphasizing how people differentially perceive current paradigms. Some people accept current paradigms, some attempt to change current

‘‘We Want Creativity! No, We Don’t!’’

95

paradigms, and some attempt to meld together different paradigms (Sternberg et al., 2003). On the present view, all represent different kinds of ‘‘propulsions’’ – creative leaders try to move people (and ideas) from where they are to a particular place, which may or may not be a new place.

TYPES OF CREATIVITY THAT ACCEPT CURRENT PARADIGMS Replication This type of leadership is an attempt to show that a ﬁeld or organization is in the right place at the right time. The leader therefore attempts to maintain it in that place. Replication is thus, in a sense, the limiting minimal case of creativity. The propulsion keeps the organization where it is rather than moving it. The view of the leader is that the organization is where it needs to be. The leader’s role is to keep it there. This type of creativity is represented by stationary motion, as of a wheel that is moving but staying in place. The replicative leader metaphorically pedals in place, as with a stationary bicycle. Replicative leadership can be seen in various types of innovations. Examples of innovations that represent replications are the Mercury version of a Ford car, Rocky III (and similar sequels), Harlequin romances (all of which are small variations on each other), and IBM clones. Replicative leaders tend to be chosen when an organization is succeeding and the goal of those seeking the new leader is to maintain the perceived status, and perhaps, preeminence of the organization. The greatest threat to the organization is likely to be perceived to be loss of current status, not failure to gain new status. The organization is seen as one that does not need to change or appear to change. Organizations with highly successful product or service lines may seek replicative leaders who will maintain the standing of these lines. Organizations that have had a highly successful and possibly charismatic leader for some time may be happy to seek a leader who can, to the extent possible, replicate the success of the previous leader. Replicative leadership is likely to be most successful during time periods of relative stability, both in terms of consumer demands and in terms of competitive threats. In times of ﬂux, the kind of leader that worked before may not work again, and the organization may lose preeminence by selecting a leader like the last one.

96

ROBERT J. STERNBERG

Part of the creativity paradox is that organizations that most need to change may be most afraid to take the risks that they need to take to change. If the bottom line is suffering, the organization may be afraid to risk what little is left of the bottom line. But their aversion to risking what is left may lead them to lose even that small margin. Redefinition This type of leadership is an attempt to show that a ﬁeld or organization is in the right place, but not for the reason(s) that others, including previous leaders, think it is. The current status of the organization thus is seen from a different point of view. The propulsion in this kind of creativity leads to circular motion, such that the creative leadership directs back to where the organization is, but as viewed in a different way. Metaphorically, this type of leadership is like riding a bicycle in a circle, so that one returns to where one is but sees it from a different vantage point. There are many examples of leading products that have represented redefinitions in their marketplaces. That is, they lead the market by functionally redeﬁning what is already there. One example is the four-wheel drive ‘‘off-road’’ utility vehicle. Very few people who drive such vehicles actually go off-road, ever. Rather, they buy the vehicles for their cachet, snow-handling ability, roominess, or any other reason but the purpose for which they were originally intended, namely, to go off-road. A second and just as profitable redefinition is the use of aspirin to prevent heart attacks. Aspirin probably now is more widely used for this purpose than for its original purpose of pain relief. It has become the leading product for the purpose, simply by redeﬁning itself. Many drugs, of course, are redeﬁned in similar ways, such as Ritalin, a stimulant that is used to calm down hyperactive children! Wellbutrin, originally marketed as an anti-anxiety pill, is now also used for weight-loss purposes. A third example is the current use of computers, which originally were used almost exclusively as number crunchers, word processors, chess experts, medical diagnosers, and so forth. A ﬁnal example is the use of academic tests to measure the quality of schools, not just of the individuals taking the tests. Historically, achievement tests were designed to measure student progress. They are now being used as much to measure school progress as they are being used to measure the progress of individual students. Redefinitions can save product lines. For example, aspirin today is probably selling much better as a preventative of heart attacks than as a

‘‘We Want Creativity! No, We Don’t!’’

97

headache remedy. But redefinitions can also serve as excuses for continuing to do what one has done before – for failing to renew or replenish ideas or product lines. One therefore has to ask whether the redefinition is indeed what is best for the organization at a given time.

Forward Incrementation This type of leadership is an attempt to lead a ﬁeld or an organization forward in the direction it is already going. The propulsion leads to forward motion. It is like riding a bicycle forward at a normal speed. Most leadership is probably forward incrementation. In such leadership, one takes on the helm with the idea of advancing the leadership program of whomever one has succeeded. The promise is of progress through continuity. Creativity through forward incrementation is probably the kind that is most easily recognized and appreciated as creativity. Because it extends existing practices, it is seen as creative. Because it does not threaten the assumptions of such notions, it is not rejected as useless or even harmful. Most products on the market represent forward incrementations. New improved versions of detergents, new models of cars, new breakfast cereals – almost all are small incremental variants, and sometimes, improvements on what came before. For example, two breakfast cereals introduced in 2003 are Cheerios with mixed berries and Cheerios with strawberries. They are fairly typical new forward-incremental products. They take an existing product, Cheerios, and add some ingredients to them, hoping to capitalize on the success of other cereals (such as Special K) that have introduced versions with berries. Another example is found in ketchup. Heinz ketchup, in order to better appeal to children, has introduced green and purple ketchup. There is absolutely no difference in the taste of ingredients of these ketchups except for food coloring – yet children enjoy eating the oddly colored ketchups more than the ‘‘regular’’ ketchup. Forward incrementations tend to be successful when times are changing in relatively predictable and incremental ways. The times thus match the leadership strategy, whether in terms of leadership of people or leadership of products. When times change unpredictably, leaders may ﬁnd that their strategy no longer works. Forward incrementation is probably, on average, the most popular form of leadership, because it moves things forward, but in a way that is generally non-threatening. One has the feeling of progress, but not the feeling that an old way of life or value system is seriously threatened.

98

ROBERT J. STERNBERG

Advance Forward Incrementation This type of leadership is an attempt to move an organization forward in the direction it is already going, but by moving beyond where others are ready for it to go. The propulsion leads to forward motion that is accelerated beyond the expected rate of forward progression. It is like riding a bicycle forward but at a very fast rate of speed. Advance forward incrementations are often not successful at the time they are attempted, because followers in ﬁelds and organizations may not be ready to go where the leader wants to lead. Or significant portions of them may not wish to go to that point, in which case they form an organized and sometimes successful source of resistance. Products that are advance forward incrementations generally do not succeed at the time they are ﬁrst introduced. Some are introduced only in concept. For example, many of Leonardo DaVinci’s inventions, such as the ﬂying machine, were so far ahead of their time that they could not be engineered at the time of their conception. But the same concept applies today. When fax machines were ﬁrst introduced, they were slow to catch on. Today, they are a routine part of most ofﬁces and even many hotel rooms. Sometimes, the product for which a market is not ready is conceptual. Countries that are suddenly introduced to democratic institutions often have trouble maintaining them, as we have seen in a number of the Asian ‘‘republics’’ that were formerly part of the Soviet Union, as well as in a number of African and Latin American countries. The countries are, for whatever reason, not ready for democracy, and fail to establish the institutions (some kind of free-market system, a free press, respect for individual rights) that are important to maintaining democracy. Probably, the best way to gain acceptance for advance forward incrementations is to sell them as forward incrementations – that is, to play down what is new about them and play up how they carry forward ideas that have been accepted before. If the ideas are perceived as radical, they will be hard to sell, almost without regard to how good they might be.

TYPES OF CREATIVITY THAT REJECT CURRENT PARADIGMS AND ATTEMPT TO REPLACE THEM These types of creativity reject current ways of doing things and propose new assumptions or paradigms. Thus, they are the crowd-defying types of creativity (Sternberg & Lubart, 1995).

‘‘We Want Creativity! No, We Don’t!’’

99

Redirection This type of leadership is an attempt to redirect an organization, ﬁeld, or product line from where it is headed toward a different direction. The propulsion thus leads to motion in a direction that diverges from the way the organization is currently moving. It is like riding a bicycle and then rather suddenly changing directions. Examples of products that represent redirections are binary computers in comparison with calculators, electric cars, and electric razors. All do what was done before, but in a new way. The basic starting point is not different. For example, electric cars often cannot be distinguished from gasolinepowered cars by appearance. Electric razors can be distinguished in appearance from manual razors, but the basic cutting mechanism, the blade, is the same. Nevertheless, the basic product is taken in a different direction to form a new product. Redirections are difﬁcult for most organizations to accept because they involve very substantial change. Jobs and, more generally, an organizational culture that has grown used to a certain way of doing things, may be threatened. Leaders cannot expect people to accept changes in direction comfortably. Indeed, they may try to sabotage the changes so that they can maintain their customary way of doing things. Reconstruction/Redirection This kind of creative leadership is an attempt to move a ﬁeld or an organization or a product line back to where it once was (a reconstruction of the past) so that it may move onward from that point, but in a direction different from the one it took from that point onward. The propulsion thus leads to motion that is backward and then redirective from an earlier point in time. It is like riding one’s bicycle back to where one was in the past, and then taking off in a new direction from there. Examples of products and services that represent reconstruction/redirection are not hard to ﬁnd. Indeed, one can ﬁnd whole stores that sell modern versions of old nostalgia products, such as advertisements for long-gone products such as Bromo-Seltzer or Brill Cream. Watches with mechanical movements are made by many of the most prestigious manufacturers, such as Rolex, and also represent a modern twist on an old and, in many respects, dated idea (in that battery-powered watches save one the bother of having to reset the watch). American Airlines adding legroom in coach seating areas is actually another example of a return, and in this case, a welcome one.

100

ROBERT J. STERNBERG

Reconstruction/redirections may be seen as ways of saving an organization that has come to be viewed as corrupted or as moving in the wrong direction. People think back to an often imaginary past, and remember how much better things were then. They then try to recapture this reconstructed past, which no longer exists and, in most instances, probably never existed. Reinitiation This kind of leadership is an attempt to move a ﬁeld, organization, or product line to a different as yet unreached starting point and then to move from that point. The propulsion is thus from a new starting point in a direction that is different from that the ﬁeld, organization, or product line previously has pursued. It is like moving one’s bicycle to a new location, and starting off in a different direction from the new locale. A number of different types of products can be viewed as forms of reinitiations. Examples are electric and gas washers in comparison with hand washboards, the ﬁrst airplanes in comparison with ground transportation, the ﬁrst space ships in comparison with airplanes, and the ﬁrst use of ﬁre for cooking. What these reinitiations have in common is that they do not just build on mechanisms of their predecessors. Rather, they use wholly different mechanisms. They conceive of a way of providing something people need through a means that is essentially totally different from what came before. Reinitiation is probably the hardest kind of creativity to sell to an organization. It means that one essentially takes the name and whatever good reputation one has built up, and then essentially starts over. Most people in an organization will not see a need for such a radical solution, and will be threatened by the uncertainty it will bring into their lives.

A TYPE OF CREATIVITY THAT INTEGRATES EXISTING PARADIGMS TO CREATE A NEW ONE Synthesis In this type of creative leadership, the creator integrates two ideas that previously were seen as unrelated or even as opposed. What formerly were viewed as distinct ideas now are viewed as related and capable of being uniﬁed. Integration is a key means by which progress is attained in the sciences. It represents neither an acceptance nor a rejection of existing

‘‘We Want Creativity! No, We Don’t!’’

101

paradigms, but rather, a merger of them. It is like combining one’s own bicycle route with someone else’s to achieve a route that incorporates elements of both original routes. Examples of products that are syntheses are seaplanes, which combine features of planes and boats; ofﬁce suite software, which combines multiple largely independent products into one integrated package; E-books, which display what formerly were printed books through software; and handheld scheduling devices, which combine aspects of computers with aspects of traditional calendars.

THE RESOLUTION The resolution of the creativity paradox, according to this analysis, is that organizations appreciate creativity more to the extent that it is of a less radical kind. For example, replicative creative leadership minimally shakes things up, and hence is likely to be appreciated. Redefinition allows things to stay as they are with a different rationale. Similarly, forward incrementation moves things forward without threatening existing structures and assumptions. These three kinds of creativity are least threatening to an organization. The other kinds of creativity all represent greater threats, and hence will be less well received. Advance forward incrementation may be seen as moving things too fast. Reconstruction/redirection may be seen as anachronistic and risky in that it requires quite a bit of change, even if it is change back to a way things once were. Redirection and reinitiation are both quite radical and will be perceived, on average, as extremely risky. Synthesis is likely to be accepted to the extent that it minimizes the new and maximizes the old.

AN EXAMPLE I would like to give an example of the principles I discussed above, drawing on my own personal experience in a product domain I know well – textbooks. In 1995, I published the ﬁrst edition of an introductory-psychology textbook (Sternberg, 1995). The book was quite different from many extant texts, for example, in its use of art and in its emphasis on creative and practical thinking as well as on memory and analytical thinking. The book sold well for a ﬁrst edition. But, of course, sales could have been better. How? The publisher, like any manufacturer, was interested in feedback from reviewers and users, in general as to how to enhance sales

102

ROBERT J. STERNBERG

appeal. The feedback the publisher got was diverse, but as one would expect, every new feature of the book was criticized by someone. The more radical the feature was, the more the market was hesitant to adopt it, as it departed from what users were used to doing. So the second edition (Sternberg, 1998) was somewhat less radical. It still sold well. But sales did not increase, and the publisher was worried. So the third edition (Sternberg, 2001) essentially jettisoned almost every feature that distinguished the book from other books. In this way, the publisher hoped, there would be nothing that would offend anyone. Sales decreased noticeably. Why? The book now had virtually no features to distinguish it from competitors. In refusing to take any risk, the publisher also lost the opportunity to gain new markets and lost the market that did, like the unusual features of the book. The fourth edition (Sternberg, 2004) is back to taking some risks, with sales yet to be determined. So organizations may not want to take risks because they put income in jeopardy, but without risks, it is difﬁcult to increase and, often, to maintain income. It is thus little wonder that organizations are reluctant to take risks (Sternberg & Lubart, 1995). But if they do not, ultimately, they will not survive.

CONCLUSION In evaluating the creativity paradox, Mumford and Hunter have suggested the need to look at multiple levels of an organization. I would further suggest that one would want to look at how these multiple levels might interact with different kinds of creativity. It may be, for example, that some kinds of creativity may be more well accepted at various levels than at others. Sometimes, when top management wants to take a big risk, its desires will be thwarted by managers at lower levels. This may be because the lower levels do not believe in the sincerity of the upper level’s desires, or because the lower levels view creativity as risky to their own jobs or success in those jobs. A few years ago, the government of Singapore attempted to introduce a national campaign to encourage creativity in its citizens. When I visited Singapore at the time, I was initially impressed by the government’s efforts. No one there was. Their view was, ﬁrst, that the government meant only creativity that did not threaten it or what it did. (One can understand why. Mikhail Gorbachev was creative in this thinking about government, and took the former Soviet Union down with him. Most governments do not want the same to happen.) Second, they believed, the society functioned best the way it was, and could not become something it was not.

‘‘We Want Creativity! No, We Don’t!’’

103

In sum, in evaluating the creativity paradox, we perhaps have to look at both levels and kinds of creativity, and study them in interaction. I hope this essay was helpful in suggesting ways in which such study might be done.

ACKNOWLEDGMENTS I am grateful to Todd Lubart, James Kaufman, and Jean Pretz, three of my collaborators in creativity research, for their many inﬂuences on my thinking and on this chapter. Preparation of this chapter was supported by Grant REC-9979843 from the National Science Foundation and by a government grant under the Javits Act Program (Grant No. R206R000001) as administered by the Ofﬁce of Educational Research and Improvement, U.S. Department of Education. Grantees undertaking such projects are encouraged to express freely their professional judgment. This chapter, therefore, does not necessarily represent the positions or the policies of the U.S. government, and no ofﬁcial endorsement should be inferred.

REFERENCES Csikszentmihalyi, M. (1996). Creativity. New York: Harper Collins. Gardner, H. (1993). Creating minds. New York: Basic Books. Kuhn, T. S. (1998). The structure of scientific revolutions (2nd ed.). Chicago: University of Chicago Press. Sternberg, R. J. (1985). Implicit theories of intelligence, creativity, and wisdom. Journal of Personality and Social Psychology, 49(3), 607–627. Sternberg, R. J. (1995). In search of the human mind. Orlando, FL: Harcourt Brace College Publishers. Sternberg, R. J. (1998). In search of the human mind (2nd ed.). Orlando, FL: Harcourt Brace College Publishers. Sternberg, R. J. (1999). A propulsion model of types of creative contributions. Review of General Psychology, 3, 83–100. Sternberg, R. J. (2001). Psychology: In search of the human mind (3rd ed.). Ft. Worth, TX: Harcourt College Publishers. Sternberg, R. J. (2004). Psychology (4th ed.). Belmont, CA: Wadsworth. Sternberg, R. J., Kaufman, J. C., & Pretz, J. E. (2002). The creativity conundrum: A propulsion model of kinds of creative contributions. New York: Psychology Press. Sternberg, R. J., Kaufman, J. C., & Pretz, J. E. (2003). A propulsion model of creative leadership. Leadership Quarterly, 14, 455–473. Sternberg, R. J., & Lubart, T. I. (1995). Defying the crowd: Cultivating creativity in a culture of conformity. New York: Free Press.

This page intentionally left blank

104

THE CREATIVITY PARADOX: SOURCES, RESOLUTIONS, AND DIRECTIONS Michael D. Mumford and Samuel T. Hunter ABSTRACT In their articles on ‘‘Innovation in Organizations: A Multi-Level Perspective on Creativity,’’ Robert Sternberg, along with Jane Howell and Kathleen Boies, broach a critical question bearing on the nature of creativity in organizational settings. Why is creativity in organizations so difficult even though organizations say they want creativity? In the present chapter, we examine some likely sources of this paradox and the ways one might go about resolving this paradox. Subsequently, we discuss directions for future research.

INTRODUCTION In ‘‘Innovation in Organizations: A Multi-Level Perspective on Creativity’’ (Mumford & Hunter, this volume), we apparently provided a reasonably comprehensive review of the literature on creativity in organizational settings – creativity which in our view is reﬂected in the development and

Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 105–114 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04004-X

105

106

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

ﬁelding of new products and services (Mumford & Gustafson, 1988). Nonetheless, in the best dramatic tradition, we concluded with what is aptly described as a ‘‘third act cliffhanger.’’ In our conclusions we pointed out a fundamental paradox that emerged from the literature. What seems to be required for creativity and innovation at one level of analysis is not necessarily consistent with the requirements for creativity and innovation at other levels of analysis. Both Sternberg (this volume) and Howell and Boies (this volume) noted that this rather paradoxical ﬁnding represents the critical issue confronting those of us interested in understanding creativity and innovation in organizational settings. Although Sternberg and Howell and Boies both stress the need to come to grips with this paradox, they take two different, albeit complimentary, approaches to this problem it poses. Sternberg asks the question what, ultimately, is the source of this paradox – a question of fundamental theoretical importance. Howell and Boies, ask the question what strategies might be used to resolve this paradox – a question of fundamental practical importance. In this chapter, we will, in turn, attempt to address each of these questions vis-a`-vis the articles provided by Sternberg and Howell and Boies. We will then consider the implications of our observations in this regard with respect to requisite directions for future research.

ORIGINS Sternberg (this volume), in his discussion of the origins of the creativity paradox, puts forth a straightforward and compelling argument. In his view, the ultimate source of this paradox is leadership, more specifically, the strategies top management teams decide to pursue with respect to creativity and innovation. In other words, the kind of creativity encouraged, and the nature of the innovations pursued, is held to depend on the strategic approach selected from seven alternatives: (1) replication, (2) redefinition, (3) forward incrementation, (4) advance forward incrementation, (5) redirection, (6) reinitiation, and (7) synthesis. Sternberg argued that these different creative strategies will give rise to qualitatively different forms of creativity with creativity being rare, especially more radical forms of creativity such as advanced forward incrementation and reinitiation, because organizations prefer less risky, albeit less powerful, strategies such as replication, redefinition, and forward incrementation.

The Creativity Paradox: Sources, Resolutions, and Directions

107

Sternberg (this volume), in his observations with regard to strategy, assumed that organizations have a coherent, well thought out, strategy with regard to creativity and innovation. We suspect this strategy, more often than not, is not expressly deﬁned, however, desirable that may be, but rather exists as a set of implicit assumptions about goals, markets, and the nature of successful performance (O’Connor, 1998). Even bearing this caveat in mind, however, the available evidence does indicate that corporate strategy can have a marked impact on creativity and innovation (Capon, Farley, Lehman, & Hulbert, 1992; Ettlie, 1983; Hajimanolis, 2000; Hitt, Hoskisson, Johnson, & Moesel, 1996). This observation, of course, broaches a new question. When should ﬁrms pursue a certain type of creative strategy? It is difﬁcult to answer this question, because, with a few notable exceptions (e.g., Tushman & O’Reilly, 1997; Sharma, 1999), creativity and innovation have received less attention in the strategy literature than economic outcomes such as return on investment, rents, and capital intensity. By the same token, however, current models of corporate strategy do provide some clues about when ﬁrms will be willing to pursue certain creative strategies. Short, Palmer, and Ketchen (2003) describe the two general models of strategy: (1) a resource-based model where the ﬁrm’s goal is to maximize unique resources and return on resource investment, and (2) a strategic groups model where the ﬁrm’s goal is to exploit certain market niches. This thumbnail description of current strategy models, however, does suggest some apparently viable hypotheses with regard to creativity and innovation. For example, in a resource model, it seems plausible to argue that ﬁrms will encourage forward incrementation when it provides unique new resources that the ﬁrm can control and exploit. In a strategic groups model, replication can be expected in ﬁrms pursuing a low cost, mass market niche while forward incrementation is likely in ﬁrms that pursue high-end markets where innovation serves as a vehicle for competitor differentiation. Although other examples along these lines might be cited, the foregoing examples seem sufﬁcient to make our basic point. It is possible to extend current models of corporate strategy to take into account creativity and innovation. Moreover, it is possible that extensions along these lines will provide one explanation for the origins of the creativity paradox. In our view, however, it is only one explanation. And, at least ﬁve other explanations exist for the origins of this paradox that warrant some attention. First, risks and rewards are not constant across levels in organizational systems (Williams & Yang, 1999). These cross-level differences in salient outcomes may impose different standards with respect to idea generation and idea evaluation (Lonergan, Scott, & Mumford, 2004; Mumford,

108

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Connelly, & Gaddis, 2003). Differences in applicable standards may, in turn, induce the kind of cross-level conﬂicts noted by Mumford and Hunter (this volume). Second, organizations are subject to two ongoing and apparently contradictory pressures. On the one hand, organizations must produce a reliable, high-quality product in an efﬁcient fashion. On the other hand organizations must engage in adaptive, innovative behavior to keep abreast of change. Differences across level in the demand for, and value placed on, quality and originality (Marion, Erez, & Naveh, 2004) may also give rise to the conﬂicts observed with respect to the requirements for creativity and innovation – a point illustrated by noting that while solution quality and originality are highly correlated at the individual level, they are less highly correlated at the group and organizational levels. Third, as one moves across levels in an organization, system complexity and time frame increase (Jacques, 1976). The need to manage complex dynamic interactions among multiple components of a system, however, may induce different demands across levels with respect to the requirements for creativity and innovation. For example, as an idea at the individual level, hydrogen powered cars represent an attractive solution to resource and pollution problems. At an organizational, or environmental level, the attractiveness of this idea decreases due to the need for production retooling and the need for new infrastructure (e.g., fuel stations). Fourth, we should not overlook the possibility that the origins of this paradox may lie, in part, in role requirements and socialization mechanisms (Basadur & Hausdorf, 1996). We train managers to run and maintain extant systems, not to generate ideas – a value of those ever disruptive, overly passionate techies and those inherently irresponsible entrepreneurs. These differences in roles and role socialization, given the fact that the multiple roles are entailed in most creative efforts, may in turn give rise to paradoxical cross-level relationships especially when politics, control, and identity issues begin to enter the picture. Fifth, and ﬁnally, as Howell and Boies (this volume) reminded us, product development cycles may induce differences in the need for, and value placed on, idea generation. Given recent ﬁndings indicating that late cycle activities (e.g., idea evaluation, planning) not only call for substantial creativity but may serve as a stimulus to creative thought (Lonergan et al., 2004; Osburn & Mumford, 2004), this argument, at least as presented by Howell and Boies, is not especially compelling. Nonetheless, it is possible that differences in standards, requirements, and work demands as one moves through the product development cycle may represent one source of the creativity paradox.

The Creativity Paradox: Sources, Resolutions, and Directions

109

RESOLUTION However, it may be useful to understand the origins of this paradox, from a pragmatic perspective, the critical question at hand is how can organizations resolve these conﬂicting cross-level requirements so as to facilitate creativity and innovation? Howell and Boies’ (this volume) chapter represents an important initial attempt to answer this question. Drawing from the observation that creative people react, and react strongly, to their social environment, Howell and Boies argued that this paradox might be resolved by formulating organizational practices so as to optimize ﬁve variables: (1) trust, (2) identity, (3) shared mental models, (4) networks, and (5) time. Although, at ﬁrst glance, Howell and Boies’ (this volume) argument appears plausible, it is, in our view, open to question whether optimization of these ﬁve variables will, in fact, enhance creativity and innovation across settings. The problem here arises from the fact that creativity and innovation represent unusually complex phenomena. As a result, multiple interactions may, at times, obviate the value of interventions intended to optimize these variables. A case in point may be found in Howell and Boies’ (this volume) argument that shared mental models will consistently contribute to creativity and innovation. Over the past few years, my colleagues and I (Mumford, Feldman, Hein, & Nago, 2001; Marta, Leritz, & Mumford, 2005) have conducted two studies examining the inﬂuence of shared mental models on group creative problem-solving. In the ﬁrst study (Mumford et al., 2001), we found consistent with Howell and Boies, that induction of shared mental models through a training manifestation resulted in improved performance. In the second study (Marta et al., 2005), however, we induced a cross-level interaction through a charge manifestation. It was found that shared mental models resulted in the production of higher quality and more original problem-solutions under stable conditions. When change was induced in the conditions of task performance, however, groups that lacked shared mental models produced better solutions – apparently because they had available a wider range of models that might be used to adapt to change. The implication here, of course, is that in turbulent settings it may not be desirable to use shared mental models as a vehicle for enhancing creativity and innovation. Another illustration of this point may be found in the effects of identity on creativity and innovation. Howell and Boies (this volume) argued that identiﬁcation with a higher-level unit (e.g., group or organizations) will be positively related to cohesion and collaboration, and, by virtue of its effects

110

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

on cohesion and collaboration, identiﬁcation can be expected to contribute to creativity and innovation. The problem with this argument is that it ignores a critical feature of creativity at the individual level. More specifically, creative peoples’ source of identity is rarely others, or institutions, but rather the work they are doing and the ﬁeld, or profession, which provides the basis for their work (Mumford, Scott, Gaddis, & Strange, 2002). Accordingly, when group or organizational identity is inconsistent with professional work identity it is unlikely that creativity and innovation will be observed. Indeed, at times identiﬁcation with the group or organization may inhibit creativity and innovation. A ﬁnal illustration of this point may be found in Howell and Boies’ (this volume) proposition that network position will inﬂuence creativity and innovation. More specifically, Howell and Boies, following Perry-Smith and Shalley (2003), argued that network centrality will be positively related to innovation, or idea implementation, but negatively related to creativity, or idea generation. One problem with this proposition is that people have multiple networks and, given the ﬁndings of Allen and Cohen (1969), it seems reasonable to expect that centrality in professional networks will be positively related to creativity and innovation. The other problem with this proposition is that network commitments can, at an organizational level, inhibit implementation of network inconsistent innovations. Thus IBM, to develop its ﬁrst personal computer, found that innovation and idea development, required reducing network centrality by moving the group to Boca Raton, Florida. On the basis of the observations of the sort sketched out above, we do not believe the integrative constructs approach proposed by Howell and Boies (this volume) will prove of much value in resolving the creativity paradox. The question broached by this statement, of course, is what would prove to be a viable alternative? We are not sure, in a general sense, that this paradox can be resolved. We do believe, however, that a staged multi-level strategy might provide a local resolution applicable in specific organizational settings. Implementation of this staged multi-level approach would begin by addressing the broader strategic issues giving rise to this paradox. In other words, organizations must decide whether they want creativity and innovation and they must manage creativity and innovation in terms of a broader strategy (Sternberg, this volume). Ideally, this strategy should consider issues such as risk, quality versus originality requirements, and complexity. However, this strategy must ultimately be formulated in such a way that it provides answers to two key questions: (1) what type of creativity do

The Creativity Paradox: Sources, Resolutions, and Directions

111

we want? and (2) what domains do we want creativity in? Definition of the domain, or domains, of interest is critical for two reasons. First, creativity and innovation are not uniformly distributed, nor are they uniform in value, across all functions of an organization – in fact this domain localization may be what makes creativity and innovation tolerable to, and feasible in, organizations. Second, the variables that represent critical inﬂuences on creativity and innovation tend to be domain specific (Kaufman & Baer, 2004; Sternberg, this volume). Once the domain of interest has been deﬁned, it is necessary to identify the critical inﬂuences on creativity and innovation in this domain or work setting. Along with identiﬁcation of critical inﬂuences, crucial cross-level contingencies must be speciﬁed. Thus, scientists, need tangible research support and time. And, tangible research support and time require profitability and a well functioning bureaucracy (Chandy & Tellis, 2000; Mumford et al., 2002). Having identiﬁed these variables, the variables that can be, and must be, manipulated in the setting at hand to promote creativity and innovation over time need to be identiﬁed and used to construct a set of localized conditions likely to facilitate creativity and innovation. Our review of the literature (Mumford & Hunter, this volume), in fact, suggests that it might be possible to apply this approach. For example, scientific creativity requires relevant expertise, adequate resources, a relevant significant professional mission consistent with organizational core capabilities, skilled knowledgeable management, and top management support and involvement. By deﬁning and committing to a strategy with regard to this domain (scientific creativity), and then carefully building the expertise, resources, and leadership cadre needed to implement this strategy, the chances of scientific creativity and innovative achievement increase remarkably. A model of the successful implementation of this approach may be found in Hounshell’s (1992) description of industrial research laboratories.

FUTURE RESEARCH This kind of staged, multi-level approach, of course, requires an adequate research infrastructure – a body of research that would permit us to identify the key variables inﬂuencing creativity and innovation at different levels with respect to different domains of effort. Thus, Howell and Boies’ (this volume) call for further research is not merely the traditional cant of the academic enterprise but rather a critical step forward in resolving the creativity paradox.

112

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Although we would not dispute Howell and Boies’ (this volume) call for studies of affect, mission communication, and self-efﬁcacy, we believe that effective implementation of this staged multivariate approach will require a broader, more ambitious research agenda that moves beyond the kind of social–psychological studies one currently ﬁnds in the literature. For example, there is a need for studies examining how organizations should go about planning for innovation. We need studies examining the standards that shape ﬁt assessments. Studies examining the role of demonstration projects in building organizational capabilities with respect to creativity and innovation might also prove valuable. Although other examples of this sort might be cited, the forgoing seem sufﬁcient to make our point – successful management of creativity and innovation will require a broad, multifaceted program of research examining phenomena operating at multiple different levels of analysis. Moreover, in building the kind of infrastructure needed to resolve the paradox of creativity in organizational settings it may be necessary to consider new types of studies. It is not enough to know that a variable effects creativity and innovation, we must also know (a) the relative importance of this variable vis-a`-vis other variables operating at a given level of analysis, (b) we must know how these variables operate in different domains, and (c) we must know what kind of cross-level interactions will be generated by these variables. Given the kind of cross-level interactions that are observed in studies of creativity and innovation (Mumford & Hunter, this volume) even these steps will not prove sufﬁcient to resolve the creativity paradox. In addition we will need studies that identify moderator and mediator variables that act to offset inhibitory cross-level interactions. For example, in one recent study along these lines, Dailey and Mumford (2004) showed that the estimation errors that plague idea evaluation could be offset by the induction of implementation intentions. In fact, Howell and Boies (this volume), in their discussion of trust, provide an excellent illustration of the potential value of research along these lines in resolving one aspect of the creativity paradox – the conﬂict between the need for individual achievement and need for collaboration.

CONCLUSIONS In ‘‘Innovation in Organizations: A Multi-Level Perspective On Creativity’’ (Mumford & Hunter, this volume), we attempted to summarize what we

The Creativity Paradox: Sources, Resolutions, and Directions

113

know about creativity and innovation from a multilevel perspective and point out the paradoxical nature of the cross-level interactions that condition creativity and innovation in organizational settings. In the sense that we were able to articulate the nature and significance of this paradox in such a way that it stimulated scholars such as Sternberg (this volume) and Howell and Boies (this volume) to think about the origins of this paradox, and the ways it might be resolved, it would seem that our initial effort had some value. The question now is, how do we use our understanding of these paradoxical cross-level interactions as a point of departure for a new wave of research that allows us to provide viable strategies for enhancing creativity and innovation in organizational settings.

ACKNOWLEDGMENTS We thank Robert Sternberg, Jane Howell, and Kathleen Boies for a series of insightful articles that provided the basis for the present effort.

REFERENCES Allen, T. J., & Cohen, S. I. (1969). Information ﬂow in research and development laboratories. Administrative Science Quarterly, 14, 12–19. Basadur, M., & Hausdorf, P. A. (1996). Measuring divergent thinking attitudes related to creative problem-solving and innovation management. Creativity Research Journal, 9, 21–32. Capon, N., Farley, J. C., Lehman, D. R., & Hulbert, J. M. (1992). Proﬁles of product innovations among large U.S. manufacturers. Management Science, 35, 157–169. Chandy, R. K., & Tellis, G. J. (2000). The incumbant’s curse? Incumbancy, size, and radical project innovation. Journal of Marketing, 64, 1–18. Dailey, L., & Mumford, M. D. (2004). Evaluative aspects of creative thought: Appraising the implications of new ideas. Norman, OK: University of Oklahoma. Ettlie, J. E. (1983). Organizational policy and innovation among suppliers to the food processing sector. Academy of Management Journal, 26, 27–44. Hajimanolis, A. (2000). An investigation of innovation antecedents in small ﬁrms in the context of a small developing country. R&D Management, 30, 235–245. Hitt, M. H., Hoskisson, R. E., Johnson, R. A., & Moesel, D. D. (1996). The market for corporate control and ﬁrm innovation. Academy of Management Journal, 39, 1084–1196. Hounshell, E. A. (1992). Invention in the industrial research laboratory: Individual or collective process. In: R. J. Weber & D. N. Perkins (Eds), Incentive minds: Creativity in technology (pp. 273–291). New York: Oxford University Press. Jacques, E. (1976). A general theory of bureaucracy. London, England: Heinemann.

114

MICHAEL D. MUMFORD AND SAMUEL T. HUNTER

Kaufman, J. C., & Baer, J. (2004). The amusement park theoretical (APT) model of creativity. Korean Journal of Thinking and Problem Solving, 14, 15–25. Lonergan, D. C., Scott, G. M., & Mumford, M. D. (2004). Evaluative aspects of creative thought: Effects of idea appraisal and revision standards. Creativity Research Journal, 16, 231–246. Marion, E., Erez, M., & Naveh, E. (2004). Do personal characteristics and cultural values that promote innovation, quality, and efﬁciency compete or compliment each other. Journal of Organizational Behavior, 25, 175–199. Marta, S., Leritz, L. E., & Mumford, M. D. (2005). Leadership skills and group performance: Situational demands, behavioral requirements, and planning. Leadership Quarterly, 16, 97–120. Mumford, M. D., Connelly, M. S., & Gaddis, B. (2003). How creative leaders think: Experimental ﬁndings and cases. Leadership Quarterly, 14, 411–432. Mumford, M. D., Feldman, J. M., Hein, M. B., & Nago, D. J. (2001). Tradeoffs between ideas and structure: Individual versus group performance in creative problem-solving. Journal of Creative Behavior, 35, 1–23. Mumford, M. D., & Gustafson, S. B. (1988). Creativity syndrome: Integration, application, and innovation. Psychological Bulletin, 103, 27–43. Mumford, M. D., Scott, G. M., Gaddis, B., & Strange, J. M. (2002). Leading creative people: Orchestrating expertise and relationships. Leadership Quarterly, 13, 343–377. O’Connor, L. C. (1998). Market learning and radical innovation: A cross case comparison of eight radical innovation projects. Journal of Product Innovation Management, 15, 151–166. Osburn, H. K., & Mumford, M. D. (2004). Creativity and planning: Training interventions to develop a creative problem-solving skill. Norman, OK: University of Oklahoma. Perry-Smith, J. E., & Shalley, C. E. (2003). The social side of creativity: A static and dynamic social network perspective. Academy of Management Review, 25, 89–106. Sharma, A. (1999). Central dilemmas of managing innovation in large ﬁrms. California Management Review, 41, 65–85. Short, J. C., Palmer, T. B., & Ketchen, D. J. (2003). Multi-level inﬂuences on ﬁrm performance: Insights from the resource-based view and strategic groups research. In: F. J. Yammarino & F. Dansereau (Eds), Research in multi-level issues (Vol. II, pp. 155–187). Oxford, England: Elsevier. Sternberg, R. J. (this volume). We want creativity! No we don’t! In: F. Dansereau & F. J. Yammarino (Eds), Research in multi-level issues (Vol. 4). Oxford, England: Elsevier. Tushman, M. L., & O’Reilly, C. A. (1997). Winning through innovation. Cambridge, MA: Harvard Business School Press. Williams, W. M., & Yang, L. T. (1999). Organizational creativity. In: R. J. Sternberg (Ed.), Handbook of creativity (pp. 373–391). Cambridge, England: Cambridge University Press.

PART II: STRATEGIC PERFORMANCE

115

This page intentionally left blank

116

MULTI-LEVEL ISSUES FOR STRATEGIC MANAGEMENT RESEARCH: IMPLICATIONS FOR CREATING VALUE AND COMPETITIVE ADVANTAGE Paul Drnevich and Mark Shanley ABSTRACT Most research issues in strategic management are essentially problem focused. To one extent or another, these problems often span levels of analysis, may align with different performance metrics, and likely hold different implications from various theoretical perspectives. Despite these variations, research has generally approached questions by taking a single perspective or by contrasting one perspective with a single alternative rather than exploring integrative implications. As such, very few efforts have sought to consider the performance implications of using combined, integrated, or multi-level perspectives. Given this reality, what actually constitutes ‘‘good’’ performance, how performance is effectively measured, and how performance measures align with different perspectives remain thorny problems in strategic management research. This paper discusses potential extensions by which strategic management research and theory might begin to address these conflicts. We first consider the Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 117–161 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04005-1

117

118

PAUL DRNEVICH AND MARK SHANLEY

multi-level nature of strategic management phenomena, focusing in particular on competitive advantage and value creation as core concepts. We next present three approaches in which strategic management theories tend to link levels of analysis (transaction, management, and atmosphere). We then examine the implications arising from these multi-level approaches and conclude with suggestions for future research.

INTRODUCTION Most research topics within strategic management span multiple levels of analysis. For example, the importance of the interactions of individual ﬁrms within broader collectives such as strategic groups, industries, and sectors is a fundamental perspective of most lines of strategy ‘‘content’’ research and is inherently multi-level. Such research is not totally focused on either the ﬁrm or its context, but rather focuses on the firm’s behavior in its context. Other lines of strategy research move from the ﬁrm level to more ‘‘micro’’ perspectives. For example, top management teams have been a staple of ‘‘strategy process research’’ that involves individuals (executives), groups of individuals (top management teams) whose interactions can be characterized and tracked, and the even broader collective of formal relations (networks) in which a ﬁrm and its members are embedded. Process research, however, does not just link micro and macro ﬁrm-level perspectives, but also can include the role of the ﬁrm in its broader environmental context. For example, executives work not only within a team and their ﬁrm, but also within a network of their counterparts at other ﬁrms. As a consequence, levels of analysis in research on boards of directors could potentially range from that of the individual director to extended interﬁrm networks, institutional leadership, and business–government relations. Clearly, it is indeed difﬁcult to name a topic in strategic management that does not cross multiple levels to some degree. The implications of spanning of levels are therefore fundamental to the ﬁeld, its research agendas, and its theoretical insights. Whereas strategy topics span levels of analysis, academic studies of strategy (and of strategic managers) often consider only one or two levels, while controlling for other levels involved in the topic of interest. This approach may be a reasonable intellectual division of labor, as both research time and attention are limited. Furthermore, multi-level issues are ‘‘messy’’ and therefore difﬁcult to research well in a rigorous manner. Nevertheless, such

Multi-Level Issues for Strategic Management Research

119

practice may result in positions that devalue the multi-level aspects of a situation and thus control for the very features that are most interesting about a ﬁrm’s strategic decisions. Studies of general managers and strategic decision processes must focus on the perceptions of a small number of top respondents, either single key informants or top management team members. These studies must then move from largely individual-level data and results in an attempt to reach broader multi-level conclusions. The data requirements of such studies limit the number of ﬁrms that can be sampled, the type of data obtainable from such samples, and the ability of researchers to employ longitudinal designs (Miller, Burke, & Glick, 1998). As an alternative to ﬁeld studies, strategic process studies may employ experimental or quasi-experimental designs that control for multi-level inﬂuences and abstract away from actual ﬁrm settings (Sutcliffe & Zaheer, 1998). However, these types of designs permit broader generalizations only to the extent that they are reﬂective of conditions in real ﬁrms. An extreme example of such levels problems arises in studies of top managers, whose individual behaviors and even cognitions are taken as critical inputs to dynamics that affect entire ﬁrms and resonate into the broader business environments faced by ﬁrms (Hayward, Rindova, & Pollock, 2004). These types of issues arise in different ways in other areas such as sociological research. Population ecologists, for example, have focused on broad environmental views of populations and industry sectors to highlight the limited possibilities for successful adaptation and change by ﬁrms. This work emphasizes inertial properties and tendencies toward convergence that large populations of ﬁrms demonstrate and thus downplays the potential for strategic action (Hannan & Freeman, 1989; Carroll & Hannan, 2000). Similar views on the constraining power of environmental forces have been used by institutional theorists, who have emphasized the power of coercive, normative, and cultural systems on organizations. This emphasis has generally been tempered to recognize the potential for significant action by individual organizations (Scott, 2001, pp. 193–200). In addition, levels of analysis issues have arisen more explicitly in research on the embedded nature of strategic action (Granovetter, 1985; Emirbayer & Goodwin, 1994; Baum & Dutton, 1996). Similarly, economic studies of industries and markets have often de-emphasized the role of strategy in specific situations, instead focusing on the importance of regulation, industry structure, or market dynamics in the determination of prices and output levels. This approach is also problematic: While markets and industries can place constraints on ﬁrms, it is

120

PAUL DRNEVICH AND MARK SHANLEY

doubtful that the activities and decision-making of large ﬁrms will be rendered unimportant by such constraints. Nevertheless, economists have made some limited attempts over the years to examine the ﬂuid boundaries between ﬁrms and markets in an effort to theorize about ﬁrm decision-making (Coase, 1937; Penrose, 1959; Cyert & March, 1963; Richardson, 1972; Williamson, 1975). Yet, only recently has the idea of ‘‘management strategy’’ come into its own and begun to gain some legitimacy as a distinct area of study within economics (Milgrom & Roberts, 1988; Spulber, 1994, 2003). Thus, while the challenges posed by multi-level phenomena predate the study of strategy, the issues illustrated in related ﬁelds remain only partially addressed and warrant more consideration in strategic management research. Understanding the implications of multi-level phenomena on strategic management research does not just mean noting the relevance of multiple levels or assessing which level is most appropriate, but also involves theorizing about how levels interact and developing theoretical mechanisms to reﬂect those interactions. If strategy research is to effectively link organizational and environmental levels, then theory development should be driven in part by efforts to clarify the nature of these interactions across levels. This endeavor involves a variety of questions related to the role of ﬁrm resources, capabilities, and managerial choice and action in relation to environmental and industry contexts (Table 1 gives some examples of potential questions). Unfortunately, strategic management research tends to focus on these questions individually, or on positioning one question against another as alternatives, rather than on exploring their interaction. As such, theory development in strategy too often appears driven by disagreements regarding the multi-level nature of these issues rather than by their integration of different levels (Short, Palmer, & Ketchen, 2003; Madhok, 2002; Silverman, 1999). In this chapter, we discuss the problem-focused, multi-level nature of strategic management as well as implications for researchers stemming from

Table 1. 1. 2. 3. 4. 5.

How How How How How

Key Questions for Multi-Level Theory Development.

much inﬂuence does environmental context have on a ﬁrm’s performance? much inﬂuence does industry have on a ﬁrm’s performance? much inﬂuence do resources have on a ﬁrm’s performance? do managers translate a ﬁrm’s resources into capabilities for sustained performance? inﬂuential are the strategic choices of top managers at different levels?

Multi-Level Issues for Strategic Management Research

121

these issues. We argue that this crossing of levels of analysis is fundamental to the ﬁeld of strategy and especially to ideas of complexity, commitment, and sustainability. To illustrate this idea, we ﬁrst explore two concepts central to strategic management: competitive advantage and value creation. Next, to consider explicitly how strategic management theories address multi-level issues, we discuss the relative strengths and weaknesses of three mechanisms that strategic management theories have used to implicitly address multi-level issues: transactional linkages, linkage through managerial roles, and linkage by atmosphere. We then suggest some theoretical implications for problem-focused, multi-level work and conclude with suggestions for future research.

THE MULTI-LEVEL, PROBLEM-FOCUSED NATURE OF STRATEGY History and Background of the Topic and Issue The multi-level, problem-focused nature of strategy predates the growth of industrial organizations. Indeed, the origins of the concept of strategy in political and military contexts date back to the works of Sun Tzu around 400 B.C., if not earlier. Since then, military and political applications of strategy have evolved and expanded beyond traditional ideas of tactics, especially in such areas as planning, resource positioning, maneuver, and operations. While military and political applications differ significantly from their counterparts in economic settings, the focus for the concept has been on helping the decision-maker – whether a commander, prince, or manager – to make consequential decisions. In military applications, the concept of strategy has evolved to span multiple levels of analysis, including environmental, technological, and political components, as conditions have changed. As such, crafting strategy has involved focusing on objectives and problems simultaneously from multiple levels where the measures of success are also aligned by level. Considerations here include national and multinational resources and capabilities, which determine both strategic options and tactical capabilities of the nation-state. Similarly, changes in technology force nations to periodically reexamine their strategies and tactics as well as the changing information needs of their military units (Keegan, 2003). While the application of strategic concepts to economic contexts has been motivated by the problems faced by large ﬁrms, the ﬁeld has also developed

122

PAUL DRNEVICH AND MARK SHANLEY

around the efforts of various communities of scholars. Ideas contributing to strategy have originated in multiple academic disciplines, each with its own focus of attention. This multidisciplinary basis of academic strategy has contributed to the multi-level nature of the resulting body of knowledge. The applied problem-focused basis of strategic thinking, coupled with its location in multiple but overlapping academic disciplines, guarantees that explanations in strategic management will span levels of analysis and be ‘‘messy.’’ For example, economists have sought to better explain these difﬁculties with traditional industry and market models and to develop new approaches for modeling ﬁrm behavior. Sociologists, by contrast, have attempted to explain the behavior of the large public and private organizations that emerged in the 20th century. This effort spawned the study of bureaucracy and organization theory, as well as the study of the behaviors of large groups of organizations over time, and the ways in which the environments of organizations are themselves organized. Psychologists have displayed more interest in ‘‘micro’’-behavioral topics, such as how individuals and small groups make decisions, interact with one another, and generally behave in organizational contexts. This diverse theoretical background contributes to the multi-level, problem-focused nature of strategy and has implications for strategy research.

The Multi-Level, Problem-Focused Nature of Strategy Research Topics Why do strategic management topics span multiple levels of analysis? This is an intriguing aspect of the ﬁeld, specifically because it works against our inherent desire for elegant modeling and focused research designs. It also limits a researcher’s ability to articulate clear and powerful recommendations, and works against the ready accumulation of research ﬁndings within an academic community. The answer to this question appears to lie in the nature of the problems that strategy researchers study. ‘‘Strategic’’ topics tend to be problem-focused and complex (see Table 2). The problems that prompted the growth of studies in the ﬁeld of strategy were those related to the growth of large ﬁrms in modern economies (Chandler, 1977). Even though there is continuing interest in new ventures and small entrepreneurial ﬁrms, contemporary strategy research remains strongly oriented toward larger ﬁrms, for reasons of both theoretical interest and research practice (consider the large number of research studies relying on databases such as CRSP and COMPUSTAT, whose samples comprise large public companies with extensive reporting obligations).

Multi-Level Issues for Strategic Management Research

Table 2.

123

The Multi-Level, Problem-Focused Nature of Strategy Research.

‘‘Strategic’’ research topics tend to be problem-focused and complex Most of the issues facing large ﬁrms are inherently multi-level Firm capabilities developed for scanning and acting in these broader environments Multi-level sets of issues are the normal expectation rather than the exception ‘‘Strategic’’ research topics naturally span multiple levels of analysis

How does the problem-focused nature of strategy research topics relate to their spanning multiple levels of analysis? In fact, it has quite a bit to do with it, as most of the issues facing large ﬁrms are inherently multi-level. Initially, these ﬁrms had grown larger than most other ﬁrms, often by a wide margin. To accomplish such growth, a number of problems had to be solved that greatly complicated the ways in which ﬁrms needed to be considered and their actions evaluated. First, for such growth to be profitable, it needed to be accompanied by an internal division of labor and task specialization and a managerial hierarchy that, along with size, allowed economies of scale and scope to be realized. Second, these growing ﬁrms began consolidating their markets into oligopolies that permitted surviving members to establish sustainable strategies, experiment with collective action, and reap the beneﬁts of scale relative to the size of the market. The industries in which these ﬁrms developed have often possessed important scientific and technological aspects that forced ﬁrms to make substantial investments in R&D capabilities that supported existing products, developed new ones, and kept managers abreast of changes in the technological environment that might affect the ﬁrm. Finally, the growth and internal differentiation of these ﬁrms posed a new range of problems for ﬁrms regarding how best to relate to those actors in the larger environment who were affected by these changes, such as workers, local governments, and competitors. Firms needed to develop capabilities for scanning and acting in their broader environments, whether by lobbying, negotiating with unions, or complying with government regulations. Some ﬁrms, such as General Electric, have faced this array of problems from their very inception (Chandler, 2001); for most others, such challenges have evolved as ﬁrms have grown. As managers developed the capabilities of their ﬁrms and coordinated them, so that their responses to markets and competitors were coordinated with their internal workforce needs and productive assets, they found that their solutions committed their ﬁrms to a set of strategic choices that could be difﬁcult to change if conditions in the

124

PAUL DRNEVICH AND MARK SHANLEY

business environment did not turn out as planned. If these managers’ decisions were effective, however, this inertial multi-level commitment made it difﬁcult for their rivals to imitate them. For today’s mid-size and large ﬁrms, this multi-level set of issues is the normal expectation rather than the exception. A problem focus on strategic issues also ensures that the importance of different cross-level issues will change over time. As the problems facing the ﬁrm change, the ﬁrm’s ‘‘strategy’’ must change as well. Furthermore, the scientific knowledge and technologies underlying industries and sectors will change, sometimes gradually and sometimes sharply. Some ﬁrms will dominate their industries for a time, whereas others will decline, exit, be acquired, or perish. Industries will expand, mature, and then change as related industries develop and whole sectors shift. Political climates and regulatory regimes will also change, as evidenced through whole professions – such as those for regulatory affairs – that have developed as the need for ﬁrms to interact with their broader environment grows. Furthermore, technological contextual changes, such as the rise of the Internet, have increasingly enabled smaller ﬁrms to make management choices regarding environmental linkages formerly available only to larger ﬁrms (as an example, consider the growth of such intermediaries as Verticalnet.net, which facilitate buyer–supplier coordination in a range of industries and sectors). How ﬁrms align with their environments will likely require continual reexamination and adjustment over time as the environmental context changes. Multi-Level Implications for Performance Measurement Given the multi-level, problem-focused nature of strategic management, there has been a natural interest in whether the problems of interest to the ﬁeld are solved by ﬁrms and their management. This type of inquiry has led to an emphasis on ﬁrm performance and its measurement. This perspective has, in turn, raised further questions for research scholars in regard to what actually constitutes good performance and how, where, and when to measure it in a manner to support strategy research (see Table 3). Specifically, performance measurement questions include the following: What constitutes performance and what are the relevant dimensions of performance? How do we know what is good performance? Which levels of analysis should be used to assess performance (and over what time periods)?

Multi-Level Issues for Strategic Management Research

Table 3. 1. 2. 3. 4. 5. 6. 7.

125

Multi-Level Implications and Questions for Performance Measurement.

At which levels of analysis and time periods should performance be assessed? How do the specific actions of subunits contribute to overall ﬁrm performance? How should we evaluate and reward managers? What actually constitutes performance? What dimensions does performance include? What is good performance? What standards should we use to judge performance and draw comparisons across ﬁrms?

How do the specific actions of a ﬁrm’s subunits contribute to overall ﬁrm performance? How should we evaluate and reward managers based on measured performance? What standards should we use to judge both ﬁrm and managerial performance? How can we effectively draw comparisons across ﬁrms and management teams? These types of performance measurement questions have taken strategy scholars into ‘‘troubled waters’’ from which we have yet to emerge as a ﬁeld. A solution is needed to allow scholars to deal with the implications of performance and to enable them to address these types of performance measurement questions. The real question is how strategic management research should deal with these issues of performance measurement. Any detailed consideration of performance should include the overall results of the ﬁrm, its relative performance vis-a`-vis its competitors, and the performance of the ﬁrm’s various subunits. Of course, looking at these results together raises further levels of analysis issues. For example, is the combination of good subunit performance and lackluster corporate results, as reﬂected in the share price, an indicator of poor corporate management and an invitation to takeover? Or is it an indication of industry or sector dynamics that affect competitors as well? Current performance measurement methods make it difﬁcult for one to distinguish between these possible alternative situations, which pose a problem for both scholars and practitioners. Even when a ﬁrm’s performance measures are understandable, the overall goals of the ﬁrm, against which performance is measured, are predicated on assumptions about technologies, products, markets, and the nature of

126

PAUL DRNEVICH AND MARK SHANLEY

consumer demand, all of which are continually evolving. Therefore, assessing the performance of a single ﬁrm in such a context requires viewing it as a part of a larger dynamic whole as well as the sum of its constituent subunits. For example, corporate strategy decisions regarding mergers, executive compensation, or international expansion have sparked public policy debates on appropriate corporate structure and conduct within the broader social environment. While several studies in strategic management have examined multi-level determinants of performance, including industry, organizational scope and diversiﬁcation, and ﬁrm-level differences, little progress has been made toward developing a combined multi-level set of measures (Seth & Thomas, 1994; Kogut, 2000). Thus, strategic management remains rich with evolving, competing, and overlapping perspectives, each attempting to explain elements of the problem–action–performance questions of the ﬁeld. To illustrate the implications of adopting multi-level perspectives, the next section explores the inherent multi-level nature of two concepts central to research in strategic management: creating value and sustaining competitive advantage.

STRATEGY’S INHERENTLY MULTI-LEVEL CORE CONCEPTS: COMPETITIVE ADVANTAGE AND VALUE CREATION The Multi-Level Nature of Competitive Advantage Perhaps the most commonly accepted core concept among strategic management research perspectives is that of competitive advantage. When a ﬁrm (or business unit within a multibusiness ﬁrm) persistently earns more profit than the other ﬁrms with which it competes, the ﬁrm is viewed as having a competitive advantage in that market. Even such a basic definition as this suggests an inherent spanning of levels of analysis, because a ﬁrm’s profitability within a particular market depends on (1) market-level economics, (2) the abilities of individual ﬁrms to generate revenues to cover their costs, and (3) the relative skills of ﬁrms to do this more effectively than their competitors. Understanding competitive advantage here requires a careful analysis of a ﬁrm’s production efﬁciencies, R&D investments, and marketing skills. We would certainly expect that a ﬁrm that was ‘‘trying hard’’ to succeed in the marketplace would be investing in its resources and capabilities and

Multi-Level Issues for Strategic Management Research

127

attempting to use them as effectively as was possible. Of course, without resources and capabilities, it is unlikely that any ﬁrm will succeed. Resources and capabilities, however, are not logical requirements for success (perhaps necessary prerequisites, but not sufﬁcient by themselves). Furthermore, whether the mere existence of resources and capabilities in a ﬁrm leads to actual advantage will depend on what the ﬁrm’s competitors are doing and what the ﬁrm’s consumers value (Powell, 2001). Unless these three aspects of advantage (ﬁrm, competitors, and consumers) are deﬁned tautologically, such that superior resources and capabilities are deﬁned by relative profitability or that success with consumers equals marketing capability, then advantage must be seen as inherently multi-level and relational. Research relying on ideas of competitive advantage should therefore consider resource-, ﬁrm-, and industry-level implications. Table 4 provides an overview of the multiple levels of competitive advantage. Scholars have attempted to resolve some of these issues by studying the importance of ﬁrm- or market-level effects in explaining profitability (Schmalensee, 1985; Rumelt, 1991; McGahan & Porter, 1997; Brush & Bromiley, 1997). These studies have arrived at varying estimates of the relative contributions of business units, corporate parents, industry, and broader macroeconomic conditions to ﬁrm profitability – results suggesting contributions to profitability from multiple levels, as well as a large degree of the unexplained variance. While there is some accepted support for industry and corporate inﬂuences, issues of how multiple levels interact to inﬂuence profitability are unresolved, a point to which we will return later in this chapter. First, however, we will examine other multi-level inﬂuences on competitive advantage, looking next at industry. Industry analysis frameworks, such as Porter’s framework (1980), are based on the idea that industry conditions are important inﬂuences on ﬁrm profitability. This is undoubtedly correct to some degree – the average performance of ﬁrms in some industries is consistently higher than it is in other industries. In reality, knowing that ‘‘industry matters’’ in ﬁrm profitability tells us very little about how it matters or what ﬁrms should do in the face of Table 4.

The Multi-Level Nature of Competitive Advantage.

Firm resources and capabilities Firm strategy and managerial actions Competitor resources, capabilities, behavior, and actions Consumer demand and behaviors Industry/market macro-level structural and contextual characteristics

128

PAUL DRNEVICH AND MARK SHANLEY

industry effects. Even though it is fairly clear that knowing the industry in which a ﬁrm competes helps to explain its performance (Rueﬂi & Wiggins, 2003), few researchers agree on the size of this effect or would focus solely on the industry level of analysis in explaining ﬁrm performance. This is especially true where ﬁrm profitability varies both across and within industries over time, as is often the case. Also, because some ﬁrms consistently earn rates of return that exceed the average for their industries, this idea suggests that industry effects are not equally important for all competitors but may be most important for average or typical ﬁrms. Therefore, a ﬁrm’s competitors, competitive positioning, and the competitive dynamics of an industry must be closely considered. While a ﬁrm’s competitive advantage implies a favorable comparison with its competitors, it is a surprisingly open question just who those competitors happen to be. Specifically, should the identiﬁcation of ‘‘competitors’’ be narrowly restricted, so that only a ﬁrm’s longstanding and direct rivals are considered? This practice would be consistent with the daily experience of ﬁrms, but could prove too narrow in periods of significant industry change. Firms with too narrow a view of their industries could be more likely to experience strategic ‘‘blind spots’’ (Zajac & Bazerman, 1991). As an alternative to narrow, industry-specific definitions, competition could be more broadly deﬁned to include general categories of ﬁrms, such as in broad industry codes. Doing so may well be more appropriate than relying on immediate competitors, and could make use of existing data sources and classiﬁcations. In addition, it may allow for the inclusion of ﬁrms with similar resource and capability bases, which may be potential competitors, yet operate outside of normal industry definitions. Of course, too broad a definition of competitors may go well beyond the underlying context in which the ﬁrm actually operates and lead to an underestimation of a ﬁrm’s potential for strategic action. Too broad a grouping will keep a researcher from identifying an existing ﬁrm-level inﬂuence on performance by masking a ﬁrm’s true advantage relative to its competitors. Further considerations may need to be explored to help appropriately deﬁne the range of a ﬁrm’s competitors. One such consideration may be geography. Should competition be geographically delimited and, if so, what is the appropriate basis for identifying proximate competitors? Do all hospitals within a metropolitan area compete with one another on all products? Most likely they do not, although there may well be product overlap. What about the food stores in an area? Should a supermarket’s competitors be considered the hundreds of stores in the general metropolitan area or just the few stores in the immediate geographic area of a store (Phillips, 1960)? The

Multi-Level Issues for Strategic Management Research

129

view of competition and strategy that arises from such an analysis will certainly differ by how broadly or narrowly a ﬁrm’s comparison set is deﬁned. The same point may also hold for corporate-level actors, in that diversiﬁed ﬁrms that compete in global markets may differ sharply in their strategic decisions from otherwise comparable ﬁrms that compete in primarily domestic markets (Murtha & Lenway, 1994). As discussed earlier, strategic management research that considers multilevel perspectives will need to consider what constitutes ‘‘performance.’’ Historically, economic and strategic theories have tended to focus on costs when considering advantage. While ‘‘profit’’ in general is what remains after costs have been covered, the unambiguous identiﬁcation of the components of profit – including direct and indirect costs, and various types of returns – has proven difﬁcult. It has proved even more difﬁcult to examine, across a large sample, why ﬁrms create their particular arrays of products and services, and why consumers purchase those products and services. Specifically, just because a ﬁrm is efﬁcient does not imply that its products will sell, or that they will sell at a price that justiﬁes the ﬁrm’s investments in the means of production and distribution. Likewise, just because a ﬁrm develops new and improved products does not mean that these products will be valued by consumers. However, the types of values created by ﬁrms and the nature of demand facing ﬁrms have largely been treated as exogenous in strategy models. This practice, in turn, leads to the idea that ﬁrms need to be ‘‘lucky’’ in their initial product and service offerings, and links the ability of the ﬁrm to strategize with this good fortune (Barney, 1986). This concept increases the difﬁculties in understanding competitive advantage as well as the related idea of value creation.

The Multi-Level Nature of Value Creation It is bordering on the commonplace to say that value-creating activities are important for ﬁrm success and survival. They are particularly critical in rapidly changing environments, fast-cycle markets, and hypercompetitive industries in which ﬁrms may be unable to sustain current competitive advantages. The process by which organizations identify activities that contribute to value creation in ongoing activities remains unclear. It is also not well understood by management scholars how new sources of value are created or how organizations balance current value-generating activities and new value-creation prospects.

130

PAUL DRNEVICH AND MARK SHANLEY

The expression ‘‘value creation’’ is common in research on strategy, marketing, ﬁnance, and entrepreneurship. It is the business result that managers and entrepreneurs strive for and the standard by which business and venture performance is often assessed. Even with its popularity, however, what ‘‘value creation’’ actually means remains unclear. The basic intention in its use appears to be the positive function of ﬁrms in generating new products and services that prove attractive to consumers. This view is quite different from perspectives such as industry analyses that are interested in how ﬁrms ‘‘capture’’ the demand already existing in a situation, a description suggesting that value is merely redistributed rather than created. Alternatively, value creation can be analyzed as an overall performance result. In some studies, it refers to profits or the expectation of profits (as in ‘‘event’’ studies of stock price changes following some important event). In others, it concerns the amount of realized profit above ‘‘normal’’ market returns. Value creation can also refer to the processes and activities that lead to a ﬁrm’s business results. This sense is used by Porter (1985) when discussing the ‘‘value chain’’ as the activities by which ﬁrms add value to their inputs before they are sold to buyers and end users. In more entrepreneurial settings, the sense of value creation refers to new occasions for value, new products, and services that prove useful to consumers and commercially viable to developers, whether entrepreneurs or internal venture managers. Like the idea of competitive advantage, value creation forces a discussion of ﬁrm performance that is inherently multi-level (see Table 5). First, it considers ﬁrm results such that ﬁrms that perform well are seen as creating value. Second, considering how value is created highlights internal capabilities and resources, regardless of whether one is considering innovation or ongoing activities. Third, the concept of value creation brings in external constituencies to the ﬁrm; rather than competitors, the reference group for competitive advantage, the constituency here is consumers or end users. Ultimately, the source of value may be exogenous to ﬁrms, but it is possible to think about the elements of consumer demand in better ways that enhance our ability to understand ﬁrm strategies and performance. Progress in this direction has come from conceptualizing ideas such as value and its Table 5.

The Multi-level Nature of Value Creation.

Actual ﬁrm-level performance Internal ﬁrm resources and capabilities External constituencies to the ﬁrm such as consumers Industry/market macro-level structural and contextual characteristics

Multi-Level Issues for Strategic Management Research

131

creation in terms of perceived beneﬁts to users and consumer surplus. This perspective involves moving away from a ﬁrm-specific view of profit to one that focuses on the interactions between ﬁrms and consumers – a shift in the level of analysis (Brandenburger & Stuart, 1996; Besanko, Dranove, Shanley, & Schaefer, 2004). Suppose that consumers enter into transactions with the intent of making themselves better off than they were prior to the transaction. In such a case, a discussion of value-creation ideas can proceed with some fairly specific meanings. For example, the degree to which the consumer is actually better off from the transaction can be seen as the consumer’s surplus. The value (perceived beneﬁt) of a transaction to the consumer would be the maximum price that the consumer would be willing to pay, while the consumer surplus is the difference between that price and the price that the consumer actually paid. Carrying the accounting language further, the surplus can be thought of as the value of the transaction to consumers (on a variety of attributes) minus (1) the direct costs of the transaction, including installation and maintenance costs, and (2) any transaction costs associated with the product. When the consumer is a ﬁrm, consumer surplus as described here would be identical to the ﬁrm’s incremental profit. A seller must deliver a substantial surplus to consumers on a consistent basis to compete successfully. This is not the same as giving the consumer all of the value created net of costs, because then there would be no incentive for the producer to continue. The value created from an exchange needs to be divided between consumers and producers. Consumer surplus represents the portion of the value-created that the consumer captures. The seller receives the price paid and uses the revenue to buy labor, capital, materials, and other inputs. The producer’s profit (price minus costs) represents the portion of the value-created that the producer captures. Adding together consumer surplus and producer profit gives the total value-created.

Linking Value Creation to Competitive Advantage Seeing value creation in terms of interactions between ﬁrms and consumers permits the linking of value creation and competitive advantage. Competition among ﬁrms can be thought of as a process whereby ﬁrms, through their prices and product attributes, submit ‘‘bids’’ to consumers in an effort to secure their business. Consumers choose the ﬁrm that offers them the greatest surplus. On average, we would expect that a ﬁrm that offers a consumer less surplus than its rivals will lose the ﬁght for that consumer’s

132

PAUL DRNEVICH AND MARK SHANLEY

business. (Of course, it is difﬁcult for a ﬁrm to determine the mix of product and transaction characteristics to offer to consumers, as preferences can vary widely and are difﬁcult to ascertain in advance.) This view of competition has implications for pricing decisions, as consumer perceptions of value may be somewhat asymmetrical. Furthermore, a ﬁrm’s products can be very complex and different product offerings’ effects on consumer value are unclear. Firms that overestimate the willingness of consumers to trade-off price for quality risk, overpricing their products and losing market share. Conversely, ﬁrms that underestimate the willingness of consumers to trade-off price and quality may fail to capture market value. In conclusion, the importance of multi-level issues in strategic management is more than just a function of the historical growth of the ﬁeld or of its inherent multi-level, problem-focused nature. In this section, we discussed two of the basic concepts of the ﬁeld: competitive advantage and value creation. For each, we suggested that the core meaning of the concept implied a linkage of levels. Competitive advantage focuses on a ﬁrm’s capabilities relative to its competitors. Value creation relates to the ﬁrm’s capabilities and resources for providing products and services that meet the demands of consumers. Both of these core concepts have an embedded cross-level relationship between the ﬁrm, its resources and capabilities, and its environment. In the next sections, we explore these multi-level issues and their implications for strategic management research and develop a typology to address these issues. We conclude by discussing the potential implications and contributions of this work and offering some suggestions for future research.

WHAT’S THE REAL PROBLEM? ISSUES FOR MULTI-LEVEL STRATEGY RESEARCH Is it a problem that strategic management’s phenomena are inherently multi-level in nature? Why and how is it a problem that core concepts of strategic management, such as competitive advantage and value creation, are also multi-level? We believe that the ‘‘problem’’ derives not from the phenomena themselves, but from the difﬁculties that the presence of multiple levels of analysis poses for efforts to conceptualize, measure, and explain them. The fact that a phenomenon spans multiple levels of analysis means that multiple potential explanations can be offered at each level. How should one sort out and choose among these competing explanations? And

Multi-Level Issues for Strategic Management Research

133

how should one account for these potential multiple explanations at different levels of analysis? As an example, consider the performance of General Electric Corporation (GE) at the close of the 1990s. By consensus and by most measures, the ﬁrm’s results were extraordinary. Corporate revenues topped $100 billion with operating margins of nearly 17%. Net earnings had increased more than 500% since 1981 and had doubled since 1993. Earnings per share (EPS) were at record levels. GE was the ‘‘most admired’’ company (Fortune) in the United States and the ‘‘most respected company in the world’’ (Financial Times) and it continues to remain a legend among major diversiﬁed ﬁrms (for more information, see GE’s Two-Decade Transformation: Jack Welch’s Leadership. Harvard Business School, Case #9-399-150 (2000); also see Welch, 2001). How else might we interpret and explain GE’s results over this period? We could start by examining GE’s major business units and groups – more than 350 business segments in the year 2000. For each, we could fashion an industry analysis and consider the business strategy decision-making and market conditions that generated each unit’s results. However, the sheer number of units involved, as well as the complexity of GE’s products and markets, would likely render such an exercise counterproductive as a way of understanding the parent ﬁrm. Alternatively, we could use a picture of GE’s results that aggregated subunit results, but this approach would not be meaningful or useful to unit managers or others interested in particular businesses. While we might wish to begin our explanation of GE’s overall performance by looking at subunit results, it would soon be necessary to go beyond them and consider corporate strategic decisions such as mergers and acquisitions or new product and service initiatives, such as e-commerce. It would also be important to consider the processes by which corporate managers make decisions, plan and control operations, and select, train, and evaluate managers. While the immediate impact of these activities is difﬁcult to assess, they almost certainly affect business unit operations and thus contribute to GE’s overall performance. Along with corporate decisions and processes, anyone following GE in the 1980s and 1990s would have taken an interest in the leadership of its CEO, Jack Welch. Welch was renowned for his experimentation with GE’s strategies and processes, including the restructuring of corporate planning and control systems, the enhancement of executive development and training, the innovative use of stock options, and the fostering of efforts at reducing bureaucracy. He also pioneered GE’s moves into globalization,

134

PAUL DRNEVICH AND MARK SHANLEY

service businesses, and e-commerce. Welch’s stature was evident in the national media attention given to the search for his successor, which culminated in the selection of Jeff Immelt in November 2000. Finally, a consideration of GE in the 1990s would require a consideration of the broader economic and political context of the times. GE is so large and diversiﬁed that its issues tend to mirror those of the larger national and world economies. In the 1990s, this context was shaped by the end of the Cold War and changes in defense priorities, the growth of globalization, the deregulation of health care and ﬁnancial markets, the rise of the Internet, and the stock market and technology boom of 1999–2000. Given the multi-level issues and multiple potential explanations discussed above, how do we fully explain GE’s performance without lapsing into an enormous case study of a unique ﬁrm? Is the ﬁrm’s performance due to Jack Welch? Is it due to the growth of particular business areas, such as ﬁnancial services (GE Capital)? Who are GE’s competitors – against which ﬁrms should the company’s performance be compared? While GE is clearly an exceptional ﬁrm, these same issues are likely to apply to some degree in efforts to understand the strategic performance of many large ﬁrms. What is the best way to study strategic management at GE and, more generally, how should strategic management researchers take the potential for multiple explanations into account? How should the presence of multiple potential explanations be handled? Careful research designs will focus on a particular explanation and attempt to exclude alternative explanations. This idea lies at the heart of well-designed research (Campbell & Stanley, 1963), although it is absent all too frequently in published strategy studies. When explanations span levels, however, their complexity increases and large sample studies become less feasible, because competing explanations can be very different from one another and prove difﬁcult to research. For example, if one is attempting to explain a ﬁrm’s behaviors and its results in terms of its CEO’s characteristics or leadership behaviors, an alternative explanation may require a more involved examination of industry dynamics that may be independent of the CEO’s inﬂuence, but that coincides with his or her tenure. To address such alternatives, researchers often opt for industry or other control variables that permit them to isolate the statistical inﬂuences of a particular set of predictor variables. This approach is perfectly appropriate, provided that these control variables are not theoretically relevant to the issues under investigation. If they are theoretically important, however, it is inappropriate to control for them and doing so risks oversimpliﬁcation (for which management research is sometimes challenged). So

Multi-Level Issues for Strategic Management Research

135

how could one pursue effective strategy research while addressing these problems? One could start by recognizing the multiple levels inherent in a phenomenon. Once the multi-level nature of a strategy phenomenon is recognized, the question then arises as to which levels are the most important to study. In this regard, researchers may well have preferences rooted in their training and experience. For example, economists can be expected to look at industries and markets as a starting point for research, but might be less inclined to consider the personality or character of top managers as important objects of study. Psychologists, by contrast, might be more likely to look for explanations in the individual and group behaviors of decision-makers. Researchers from practitioner and consulting backgrounds might be drawn more to issues of control and implementation, whereas such issues might not hold much allure for researchers in traditional academic disciplines. It is doubtful that researcher predilections should provide the basis for sorting out multi-level situations. While some CEOs, for example, are very powerful and capable of inﬂuencing the situations in which their ﬁrms compete, it is doubtful that more than a relative handful of such individuals exist. In most situations, CEOs may inﬂuence but are unlikely to determine the outcome of a situation, independently of the vast array of other factors that inﬂuence their ﬁrms and the broader business and regulatory environments. Similarly, while every business is situated in some economic market, it is often the case that what is most interesting and important about a ﬁrm’s strategy concerns is less how the relevant markets function and more the ways in which the markets do not work efﬁciently. Which of several possible levels of analysis is most important in a situation is ultimately an empirical question, provided some consensus can be reached among researchers about what ‘‘important’’ means.

Prior Attempts at Addressing the Levels Issues: The Industry versus Strategy Debate Collectively, the stream of strategy studies that attempts to explain the relative importance of industry-, corporate-, and ﬁrm-level factors on ﬁrm performance represents an explicit attempt to sort out these types of relationships among levels of analysis (for a review, see Rueﬂi & Wiggins, 2003). This set of more than two-dozen studies originated with Schmalensee (1985) and includes work by Wernerfelt and Montgomery (1988), Rumelt (1991), McGahan and Porter (1997, 1999), Brush and Bromiley (1997), and others.

136

PAUL DRNEVICH AND MARK SHANLEY

Much of this work is methodological in focus. While a detailed discussion of this research stream is beyond the scope of this chapter, these studies do raise some issues that are relevant to our discussion. First, as this line of research has developed, differences in levels of analysis have become better articulated. What began primarily as a distinction between industry and ﬁrm effects soon developed to include corporate-level effects. More recent studies in this tradition have expanded this approach to consider the effects of business groups and strategic groups on performance (Khanna & Rivkin, 2001; Dranove, Peteraf, & Shanley, 1998). These studies have further substantiated the multi-level nature of the determinants of ﬁrm performance. While these studies have documented the inﬂuence of multiple levels, they have been less effective in explaining what these results mean for strategy. What does a ﬁnding of weak corporate effects mean? Does it indicate that corporate strategy is unimportant and that business units should be divested? Are industry-level effects equally important for all ﬁrms or just for some? Along with the extensive methodological debates addressed by these studies, criticisms of this line of work (Brush & Bromiley, 1997; Bowman & Helfat, 2001; Rueﬂi & Wiggins, 2003) have raised as an issue the need to clarify theory regarding the relationship of managerial decisions to the identiﬁed ‘‘importance’’ of effects at a given level. More thought needs to be given to the ways in which managers can inﬂuence ﬁrm performance and the ways in which this inﬂuence may be apparent at different levels of analysis – strategy theories need to clarify their treatment of these multi-level interactions. The need for clarifying the relationships among levels in this stream has also become apparent indirectly in the reactions of scholars to controversial ﬁndings in this line of research – in particular, Rumelt’s (1991) ﬁnding of a small effect of corporate strategy on ﬁrm performance. Indeed, this ﬁnding clearly motivated several subsequent studies, which largely consisted of efforts to either (1) replicate it or (2) challenge it and substantiate the importance of corporate-level effects. Such observed reactions in the literature suggest that researchers share an implicit assumption of the importance of corporate-level phenomena, which is not surprising given the origins of the ﬁeld and the problem-focused nature of its phenomena. They also raise the question of how the extent of current practice regarding some area of activity speaks to its ‘‘importance,’’ especially when practice appears to conﬂict with research results. Theoretical critiques of these studies are also noteworthy and reﬂect a more general concern about multi-level explanations, namely, that strategy

Multi-Level Issues for Strategic Management Research

137

theories need to incorporate multiple levels rather than choosing between them. While many researchers recognize the potential inﬂuence of multiple levels of analysis on performance, the nature of that inﬂuence is often left implicit. The more explicitly it is recognized that ﬁrms and their managers can exert their inﬂuence across levels, and in turn be inﬂuenced by activities at those levels, the less attractive a variance decomposition approach to questions of importance will appear.

Additional Implications of Theorizing across Multiple Levels of Analysis Analyzing the long-term strategic performance of ﬁrms in research-intensive industries provides another example of the complexity of theorizing across levels. One could begin with an explanation at the ﬁrm and industry levels and discuss strategy in terms of product–market dynamics. The story would quickly expand to the corporate level by recognizing that some activities, such as testing, government relations, and marketing, are more amenable to economies of scale than are others. It would then be necessary to broaden such an analysis to consider the state of knowledge regarding the technologies involved, as well as the institutional structure of regulation, innovation funding, and cooperative relationships among industry participants. In addition to considering the ﬁrm and its environment, explanations would need to consider a ﬁrm’s internal structures and processes – for example, whether its R&D authority is centralized or decentralized, or whether significant R&D projects are organized on a modular basis. An intergroup analysis of project teams could also be important, as could even more micro-level studies of the group/team characteristics and activities associated with successful R&D project outcomes. Baldwin and Clark (2000) provide this sort of multi-level approach in their analysis of the evolution of modular ideas in the computer industry. They document the industry’s transition from a period of dominance by IBM to its current fractionated state of innovation-driven competition. In doing so, they use ideas of modularity to link issues of product design, organizational architecture, outsourcing, innovativeness, industry structure, and regulation. Conceptualizing strategic situations to afﬁrm multiple levels of analysis, however, is the stuff of which paradoxes are made. As an example, consider the dichotomies that have long plagued strategy regarding environmental or market determinism and managerial choice. On the one hand, social contexts are very inﬂuential on their members, and efﬁcient markets will tend to

138

PAUL DRNEVICH AND MARK SHANLEY

nullify temporary individual advantages, as well as most pretensions toward long-term competitive advantage. On the other, efﬁcient markets presume market actors who are capable, are highly motivated, and believe that they can ‘‘beat the market.’’ How can both levels be jointly afﬁrmed? If markets are efﬁcient, how can reasonable individuals believe that they can beat them? If individuals can beat markets, how can the markets really be efﬁcient? While resolving these types of anomalies is beyond the scope of this chapter (if it is possible at all), we do think it is possible to improve our thinking about multi-level problems. Finally, we must recognize the cost and feasibility issues involved in conducting multi-level research. At the same time, we think it is both feasible and desirable to examine the ways in which strategic management theories link levels of analysis. While cross-level strategic issues are rarely explored on an explicit basis, they are considered implicitly in the general theoretical approaches in common use in the ﬁeld. We explore these more implicit linkages in the next section. Doing so will help researchers to make their cross-level assumptions explicit, which in turn will permit them to craft better research designs and obtain more useful results.

HOW DO STRATEGY THEORIES LINK LEVELS? If strategic management phenomena are inherently multi-level and problemfocused, and if this is at least implicitly recognized by researchers, then perhaps the real issue is not whether multiple levels of analysis are linked, but rather how they are linked in the dominant theoretical perspectives of the ﬁeld. What are the different conceptual mechanisms, or chains of reasoning, by which theories have linked different levels? Most often, they are fairly abstract and constitute chains of assumptions rather than testable propositions (indeed, a problem with propositions about multi-level effects is that they involve unobservable relationships for which the available data will be ambiguous and potentially speak to multiple levels at once, rendering testing difﬁcult). It is with such mechanisms that strategic theories add value over explanations focusing on a single particular level. In this section, we discuss three different conceptual mechanisms for linking levels of analysis and provide some examples of each (see Table 6). We will discuss the relative advantages and disadvantages of each mechanism as a means of linking levels. Of course, all three may not be equally effective in every situation, so we will therefore offer some suggestions as to which are likely to lead to better results in which situations.

Multi-Level Issues for Strategic Management Research

Table 6.

139

How Strategy Theories Link Levels.

Linkage by transaction Linkage by managerial role Linkage by atmosphere

Linkage by Transaction A ﬁrst way to link levels is with a common transaction or behavior that all of the actors in a domain can, in principle, perform. The term ‘‘transaction’’ here is taken very generally and can include such activities as informal social exchanges, market exchanges, long-term contracts, or mergers and strategic alliances. The classic example occurs with an economic market. A market consists of a number of participants, who can enter, transact, and exit according to their capabilities and the opportunities available to them. The linking transaction that is the unit of analysis here is the economic exchange, which involves a buyer and a seller. Sometimes, these exchanges are discussed in terms of ‘‘spot market’’ contracts. The ‘‘market’’ itself is apparent in the set of interactions among all buyers and sellers, from which the quantity of goods and services provided and the prices charged for goods are determined. As a focal point for analysis, one can begin at the level of an individual actor, trying to sell his or her goods and services or shopping around for what the individual wants to buy. One can then aggregate using the market transaction as the unit of analysis, which permits a focus on the interactions between buyers and sellers. This is akin to the distinction made in network analysis between focusing on individual network nodes or on the dyadic links between network members (Burt, 1992). If one uses individual actors or their transactions as the basis for analysis, the levels of analysis are linked by a process of aggregation. Here, the ‘‘micro’’ level consists of the individual transactions and their participants. At a more ‘‘macro’’ level is the market itself, which comprises the overall results of buying and selling, the presence or absence of persistent inequalities among buyers or sellers, and whether the market ‘‘space’’ is homogenous or heterogeneous with varied niches. Broader levels of analysis are linked to narrower ones through the aggregation of transaction results. Market ‘‘structure’’ arises from these interactions of market participants. Economic approaches to strategy (Porter, 1980, 1985) often use transaction ideas as the basis for linking levels of analysis within the ﬁrm and beyond, within product markets or as part of a larger corporate context.

140

PAUL DRNEVICH AND MARK SHANLEY

Transaction cost economics (TCE), especially as developed by Williamson (1975, 1985), also employs this approach, with its emphasis on the transaction as the unit of analysis. For example, ﬁrms make decisions either to perform an activity themselves (vertical integration) or to buy from a market specialist. Aggregating from the level of the transaction to the level of the ﬁrm, a ﬁrm’s strategy can be seen as the sum of its ‘‘make versus buy’’ decisions. For TCE, shifts in level of analysis also occur when simple aggregation is no longer feasible. For example, the shift from a business to a corporate level occurs when a ﬁrm becomes too large and complex to be organized functionally (Williamson, 1975). This is why multidivisional (or M-form) structures are introduced. Along with its coordination tasks, an important purpose for such a structure is to provide an ‘‘internal capital market’’ for divisions and subunits. The approach of agency theory (Jensen & Meckling, 1976; Eisenhardt, 1989) to levels of analysis is similar to that of TCE in its focus on the relationships between principals and agents as a basis for ﬁrm governance and interﬁrm relations. The resource-based view (RBV) of the ﬁrm also embodies what we call a transactional approach for linking levels. A ﬁrm’s activities comprise exchanges in which it is paid for various combinations of resources and capabilities that are valued by consumers (Wernerfelt, 1984; Barney, 1986; Peteraf, 1993). The RBV differs from more traditional economic approaches in which it emphasizes how a ﬁrm’s sustainable performance in generating profits (rents) stems from its possession of scarce, valuable, and difﬁcult-toimitate resources that are not actively traded and priced in the ﬁrm’s factor markets. This idea suggests market imperfections, in which no simple relationship exists between the behaviors of markets and the behaviors of participating ﬁrms. Lippman and Rumelt (2003) argue for a simpler ‘‘payments perspective’’ as a micro-foundation for a resource approach based on payments to ﬁrms for valued resources, without reference to whether those resources are priced in markets. Transaction linkages can be an attractive means for linking levels. They provide a simple and logical basis for theorizing that is linked with the behavior of individual actors. If we wish to know a corporation’s strategy for diversiﬁcation, for example, it is reasonable to consider the ﬁrm’s portfolio of businesses, especially any changes made during some period via merger, acquisition, alliance, divestiture, or spin-off. It is also plausible to analogize between levels regarding some economic transactions and social exchanges. Social relationships between two individuals, for example, have clear similarities with those of more ‘‘macro’’ actors, such as groups and

Multi-Level Issues for Strategic Management Research

141

organizations, and metaphors of marriages, courtship, conﬂict, and divorce are common ways to view mergers and acquisitions on the one hand or divestitures on the other. Theories that utilize transaction linkages are not without their difﬁculties, however. They must treat as exogenous systemic characteristics that are not reducible to the aggregated transactions of actors. For example, the rules by which an exchange system operates are often not determined by the participants and are exogenous to the transactions themselves. Furthermore, while market transactions can to some degree be aggregated, the structure of a given market may not be reducible to the aggregated actions of participants; instead, it may also take into account such factors as regulation, possession of critical resources, collusive action by dominant industry players, or historical idiosyncrasies, all of which constrain the actions of individual ﬁrms. These gaps are considered in terms of market failures in TCE. Industry analysis ideas also embody such gaps, in which the desirable strategic positions for ﬁrms within an industry are those where they are shielded from competition and imitation from rivals or new entrants. The more closely an industry’s conditions approach those of a competitive market, the less room individual ﬁrms have to achieve an advantage. Divergence from competitive conditions, however, is critical to these types of explanations, but is generally taken as exogenous to the behaviors of particular ﬁrms (for an exception, see Sutton, 1992). Building theories around dyadic relationships, exchanges, or transactions may render it difﬁcult to identify the locus of agency in a given situation. If our focus is on transactions, for example, how do we distinguish between transactions that are separate and distinct from those that are linked together as part of a larger program? If our unit of analysis is not necessarily where the critical decisions are being made, then either theory must be adjusted for particular and even idiosyncratic circumstances (and thus be less parsimonious); otherwise, each theory will lead to confusing results. Looking at a network of business units, for example, it may not always be clear whether the network actors are independent and autonomous or whether the network includes a clear leader who inﬂuences the behaviors of others. Similarly, in considering a ﬁrm’s merger activity, it is not clear whether mergers constitute separate decisions or are part of a larger corporate strategy embodying multiple mergers, divestitures, alliances, and internal initiatives. A related problem is identifying who in a complex corporate actor is actually making the decisions. Continuing the example of mergers, in some ﬁrms these decisions are made at the corporate level. In others, merger

142

PAUL DRNEVICH AND MARK SHANLEY

decisions are made by managers of groups, divisions, or even smaller business units. In addition, studies of merger performance as measured by changes in stock prices often embody heroic assumptions about the consistency with which corporate decisions about a merger translate into implementation decisions by lower-level managers. Without information on the intra-organizational details of merger integration, market evaluations may not be fully informative about merger performance. Evidence to date suggests that corporate representations do not consistently translate into implementation success. This consideration is especially critical for stock price studies of ﬁrm strategy, because internal corporate information is much less readily available to investors for all except the largest ﬁrms. For example, Kaplan, Mitchell, and Wruck (2000) report a case analysis of the discrepancies between the market evaluation of mergers and their implementation success. Bridging levels by transactions may also call the boundaries between levels into question, as well as the boundaries of the ﬁrm itself. For example, taking an agency perspective and viewing the ﬁrm as a ‘‘nexus of contracts’’ (Alchian & Demsetz, 1972) makes it difﬁcult to justify the existence of the ﬁrm or its authority structure when a contract-based market solution would be preferable. Why would such a costly structure be desirable if ﬁrms are simply the aggregation of dyadic contractual relationships among members, each of which can be understood and governed on its own terms? In a related vein, Miller (1990) argues that the transaction costs of organizing within a ﬁrm can be as troubling as the issues associated with external contracting. He ends up augmenting a transaction cost approach with an emphasis on culture and relationship building by general managers. Social exchange approaches to ﬁrm behavior (Pfeffer & Salancik, 1978; Pfeffer, 1987) also use transaction logic to link levels of analysis. They emphasize less formal social exchanges rather than contracts and other formal exchanges (Coleman, 1990; Levine & White, 1961; Blau, 1964). The idea here is the same as with economic exchanges, however, at least in terms of bridging levels. The order that is apparent within ﬁrms is explainable (at least in part) as the cumulative result of social exchanges among members. At the interﬁrm level, the activities of particular ﬁrms are linked through power–dependence relationships with other ﬁrms, regulatory organizations, and other stakeholders (Thompson, 1967). Similar issues to those noted with transaction cost industry approaches also arise for theories relying on social exchange. While it is possible to see a micro logic of interactions, many aspects of collective behavior seem to depend on macro factors that are not the cumulative result of dyadic

Multi-Level Issues for Strategic Management Research

143

exchanges. These relationships have frustrated efforts to links levels of analysis through social exchanges (Willer, 1999). For example, Burt’s (1992) theory of networks and structural holes acknowledges the transactional logic discussed previously. At the same time, the most important part of his analysis concerns the structural characteristics of networks, such as centrality and functional equivalence, and the ability of actors to control important network positions, or span ‘‘structural holes.’’ While actors and dyadic exchanges matter for understanding action, so too do the positions of actors within the network. Difﬁculties may also arise in identifying a locus of agency with exchange theories. Suppose that one observes an exchange between two parties. Without more information than the fact of an exchange, it is difﬁcult to tell who profited if there was an imbalance associated with it (Laumann & Knoke, 1987). Presumably both parties are willing, but one may beneﬁt more than the other. It is also difﬁcult to tell whether exchanges are distinct transactions or part of a larger history of existing exchange relationships between the parties. As with theories of economic transactions, theories of social exchange have trouble explaining the clarity and persistence of ﬁrm identity and boundaries. If ﬁrms are run by groups of powerful actors, each pursuing its own interests at the expense of other groups, then they are at constant risk of dissolution. In recognition of this fact, some theorists have enlarged the role of management and added the condition that managers must contribute to overall ﬁrm performance (Aldrich, 1999). This anticipates our second linking mechanism – linking levels through some form of managerial bridging role.

Linkage by Managerial Role A second way to link levels of analysis is through the delineation of a general managerial role. Firms are called upon to make decisions in a variety of areas, from supply management and production, to sales and marketing, to research and development. These decisions span levels of analysis as well, from micro-level dealings between employees and customers, to group and organization decisions concerning operations, to strategies for interacting with other ﬁrms, government agencies, and media organizations. While these decisions can be looked at and analyzed in isolation, the ‘‘ﬁrm’’ must attend to them all together in real time to determine how to allocate its attention. The job of the general manager, how these managers coordinate

144

PAUL DRNEVICH AND MARK SHANLEY

the varied decisions of their ﬁrms, has evolved over time as ﬁrms have evolved to meet these decision burdens. It should not be surprising that many strategic management theories have focused on the central role of the general manager as a way to link the diverse strategic decisions of a ﬁrm. In doing so, these theories serve to bridge levels of analysis between the ﬁrm and its broader environment. They also link levels within ﬁrms, such as in studies of planning and control processes, which consider how to link the divergent perspectives of corporate, divisional, and departmental levels to facilitate planning, capital budgeting, or internal venturing (Bower, 1970; Burgelman, 1983). The classic example of this use of a managerial linking role is in Barnard’s (1938) study of the functions of the executive. He saw the role of the top manager as that of distributing and balancing the inducements and contributions of all those actors who were necessary for the success of an enterprise in the broader environment (what some call stakeholders), whether they were employees, customers, suppliers, or buyers. This role represents a bridging of what we have come to know as organizational, competitor, and market levels, performed by an individual or small team, and it goes well beyond dyadic transactions with particular constituents. Barnard’s perspective has continued to powerfully inﬂuence strategy research. A full accounting of his inﬂuence is beyond the scope of this chapter, but some basic points can be made. His work inﬂuenced the development of the Business Policy Group at the Harvard Business School (Andrews, 1971) and continues to inﬂuence top management research (Donaldson & Lorsch, 1983; Kotter, 1982; Gabarro, 1985). Barnard also strongly inﬂuenced the work of March and Simon (1958), who adopted his inducements–contributions framework as a basis for their own frameworks. Cyert and March (1963) further developed ideas relating to general manager roles in their behavioral theory of the ﬁrm, in which managerial roles were exercised by a top management team (or dominant coalition) whose goals became the goals of the ﬁrm and whose actions aligned the organization to its uncertain environment. Finally, Barnard’s inﬂuence is seen in current popular approaches to general management (Collins, 2001). Two other scholars who have focused on general management as a critical component of their theories are Chandler (1962, 1977) and Penrose (1959, 1960). Chandler’s idea of strategy as a ﬁrm’s long-term goals and objectives and their implementation, coupled with his focus on the importance of professional managers and managerial hierarchies, make general management a central concept of his research. While Penrose (1959) is often associated with the RBV, she actually placed a ﬁrm’s managers in a central

Multi-Level Issues for Strategic Management Research

145

role that linked the ﬁrm and its capabilities with its environment. She saw ﬁrm growth as being governed by the interaction between internal resources and services (capabilities) in relation to external market opportunities. The task of overseeing and guiding this interaction falls to the ﬁrm’s managers, who use their specific knowledge of ﬁrm capabilities in conjunction with their vision of market opportunities to craft the most effective growth strategies and then adjust them on the basis of interactions with the ﬁrm’s environment (Penrose & Pitelis, 2002; Ghoshal, Hahn, & Moran, 2002). By linking the ﬁrm to its environment, managers enact the crucial level-bridging function for Penrose’s theory. Linking levels of analysis by managerial roles offers a number of practical and theoretical beneﬁts. Such a link is plausible and reﬂects the reality of top managers directing large ﬁrms. Top managers are highly trained and successful, well paid, highly visible, and often inﬂuential participants in their ﬁrms, industries, and communities. They are often articulate as well, and have ready explanations to offer regarding their contributions to their ﬁrms. It is reasonable to assume that these managers may be vital links between their ﬁrms and their environments. Incorporating an endogenous role for general management also permits a theorist the ﬂexibility with which to address the variety of behaviors that are apparent among real-world ﬁrms. Without such ﬂexibility, a theorist could be drawn into a contingency logic expecting an isomorphism of sorts between environmental characteristics and ﬁrms’ decisions. Incorporating a managerial role into a theory allows for a given environmental set of opportunities and constraints to be perceived quite diversely by different managers. In this sense, Child’s (1972) critique of structural contingency theories was not a rejection of the need for alignment with the environment, but rather a clariﬁcation of how such an alignment would come to pass through the strategic choices of a ﬁrm’s managers. It is the environmental conditions as perceived by managers that matter for the organization, and managers choose how to view the environment and presumably how to respond to its environmental demands (Hickson, Butler, Cray, Mallory, & Wilson, 1986). While this approach has its advantages as a way of bridging levels, the general management role is messy and difﬁcult to specify (Kotter, 1982). This complexity is to be expected, however. While the varied decisions facing a ﬁrm can be analyzed according to their respective economic and social logics, how these decisions ﬁt together will, of necessity, be more complex, disorganized, and idiosyncratic and less amenable to systematic analysis. As a result, a potential criticism of general management theories is that they do

146

PAUL DRNEVICH AND MARK SHANLEY

not resolve the levels problem as much as package it and compartmentalize it. Given the relatively low levels of accumulated knowledge on general managers (Bennis & Nanus, 1985), one is tempted to see research on general managers as identifying what we do not know rather than what we do know. This also suggests that linking levels by a focus on managerial roles may not be a panacea. A related problem with these theories is a potential confusion of causality. Because theories of general management and executive leadership link levels of analysis and embody the least rationalized aspects of a ﬁrm’s decisions, they may not be very helpful in explaining a ﬁrm’s results. Perhaps this is one reason why research on general management has produced so few accumulated results. For example, if a manager’s tenure coincides with a period of very good or very poor ﬁrm performance, what conclusions can be drawn regarding the role of the manager in the ﬁrm’s results? The ﬁrm’s incentive system will likely reward the manager for the good performance (although it may also reward him or her for poor performance). We can expect that many top managers will take credit for the good results and provide their own rationalizations for poor results. In any event, it will still be unclear what the true contribution of the manager was to his or her ﬁrm’s performance. While we know that general management and executive leadership are important, we are less clear regarding why or when they are important, which leads us to consider yet another mechanism – linkage by atmosphere.

Linkage by Atmosphere A third way to link levels is through the effects of a shared context of interaction or atmosphere. Several different aspects of social context are discussed here. The intuition for referring to them as ‘‘atmosphere’’ comes from Williamson (1975) and ultimately concerns Adam Smith’s idea of an ‘‘atmosphere conducive to trade.’’ The idea is that the more participants share such a background, the easier it will be to explain multi-level ﬁrm dynamics. In this sense, atmosphere constrains the strategic behavior of managers and their ﬁrms. The importance of context has a long tradition in economic sociology, where context is seen as not just constraining on actors but also constitutive in that it helps to determine what is important for actors and how they should go about pursuing it and interacting with others (Baum & Dutton, 1996; DiMaggio, 1994). Sometimes this context is discussed in terms of

Multi-Level Issues for Strategic Management Research

147

culture. Increasingly, the social context in which ﬁrms interact is taken more broadly to include the institutional setting, which includes not only culture, but also legal and regulatory regimes, normative structures such as status orderings, and common historical ties. Atmosphere here is not a distinct mechanism, but rather something that links levels by reducing the variety of behaviors across levels, thereby rendering the complexity of a multi-level situation less problematic. It is not a contextual determinism, but more a corrective to the ‘‘undersocialized’’ individualistic views of action that have been common in economic views of strategy (Granovetter, 1985). A particular atmosphere does not by itself imply a particular decision or transaction logic on the part of ﬁrms or their managers. As a consequence, it is likely best employed in conjunction with transactional or general manager logics. For example, the more homogenous the environment in a given nation, the easier it will be for general managers to learn about the requirements of their institutional structures and craft successful strategies to gain legitimacy (Aldrich, 1999, p. 229). Firms and industries do not just operate in the traditional space assumed by economic theories of strategy, but also within complex institutional structures that condition how industry participants think about their tasks, organize their behaviors, and respond to regulation by government and nongovernmental organizations (Scott, 2001). These structures can greatly constrain the behaviors of ﬁrms, whose managers actively seek to obtain social legitimacy for their ﬁrms (Oliver, 1997). The extent to which the potential for value creation for an industry or ﬁrm is a function of institutional constraints has yet to be determined, but it is likely to be high in most settings. At a minimum, a homogenous cultural/normative environment will likely reduce search and decision-making costs (Kreps, 1990). Even using atmosphere as a complementary means for bridging levels, however, has some inherent limitations and will likely be useful for studies of relatively stable industry situations. Institutional explanations have typically not been effective for explaining dynamic situations, or events such as drastic industry change, as in periods of deregulation (Hirsch, 1997). Moreover, as institutions become more inﬂuential and more heterogeneous, the problems they pose for ﬁrms will become more – rather than less – complex, making atmosphere a complicating – not simplifying – factor. This is especially the case in settings where ﬁrms can, either individually or collectively, undertake joint institutional strategies to improve their situations versus regulatory agencies and minimize the number of constraints in their business dealings. It is unclear whether such strategies are effective for

148

PAUL DRNEVICH AND MARK SHANLEY

improving performance (Schnatterly, 2003; Waddock & Graves, 1997; Fombrun & Shanley, 1990). Sorting Out the Mechanisms Strategic management theorists recognize the multi-level nature of their target phenomena and attempt to address levels of analysis issues in their research through a variety of conceptual mechanisms, which provide logics for explaining how the activities of ﬁrms and their managers lead to a variety of complex consequences that become apparent at different levels of analysis. These mechanisms include linkage by transactions/exchanges, linkage by managerial roles, and linkage by atmosphere. These ways of linking levels have their relative advantages and disadvantages. For example, transactional approaches have proven very amenable to modeling, but may also be more abstract and reductionist in dealing with the richness of the situations being explained. While some types of behaviors may aggregate well to macro levels of analysis, others may not, and many macro phenomena may not be reducible to exchanges or contracts. Conversely, focusing on managerial roles may reﬂect the ﬂuidity of corporate situations, but may also render an analysis less suitable for modeling. Raising the advantages and disadvantages of these approaches to linking levels should not suggest that one approach is ‘‘better’’ than the other approaches. Market and exchange theories have proven extraordinarily successful in economics, sociology, political science, and other areas. The relative advantages and disadvantages of these theories are well known and scholars continue to employ them productively. However, it is interesting to note that while exchange and market models are very important in strategy research, many of the fundamental views of strategic management and ﬁrm behavior remain oriented toward general managers. This perspective most likely reﬂects the continuing tension in which strategic management ﬁnds itself as an academic discipline studying a problem-focused applied subject matter of continuing interest to practitioners. Furthermore, it suggests that the most innovative theories may be those that work toward integrating transactional and managerial approaches (e.g., see Hermalin, 1998).

IMPLICATIONS FOR THEORY AND RESEARCH What are the implications of the preceding discussion? Why is it important to focus on the multi-level dimensions of strategy? Are the three linking

Multi-Level Issues for Strategic Management Research

149

approaches discussed previously equally plausible or useful for researchers? In this section, we will draw out some of the implications of our analysis and offer suggestions for researchers. We consider the beneﬁts of making explicit what has largely been an implicit consideration of levels of analysis issues. We also make suggestions for improving research designs on levels issues. We conclude by considering specific topic areas in which a more explicit consideration of levels of analysis issues would foster research progress.

Making Levels of Analysis Issues Explicit Our interest in this chapter has not been to introduce levels of analysis issues as concerns to which strategy researchers must attend. These issues have been present throughout the history of the ﬁeld, and most researchers have dealt with them from time to time. What has been done less frequently is to consider them explicitly, which is what we have tried to do here. How does considering these issues explicitly help research in strategic management? We offer some general observations and potential contribution in this section (see Table 7). Levels of Analysis Choices Matter in Strategy Research The conclusions one draws about a particular type of ﬁrm will depend on how one deﬁnes its served markets. Are markets small and local? Studies that deﬁne markets narrowly may be more likely to reach conclusions consistent with the potential for effective strategic action by ﬁrms, whereas studies deﬁning markets more broadly will come to conclusions more consistent with competitive assumptions. Similar points could be made about studies of the ﬁrm in its social context. Which type of error is more likely? That depends on the ‘‘true’’ market served by a ﬁrm or the ‘‘true’’ set of environmental inﬂuences on its managers. A researcher can only approximate these factors, subject to a range of inﬂuences. We think it would be better for such choices regarding the ﬁrm’s environment to be made explicitly rather than implicitly or on the basis of available data. Table 7.

Implications of Making Levels of Analysis Issues Explicit.

Levels of analysis choices matter in strategy research Approaches to dealing with levels of analysis matter Theoretical clariﬁcation helps one to make choices regarding levels problems Easy ﬁxes to levels problems in strategy may not exist

150

PAUL DRNEVICH AND MARK SHANLEY

Not only are industry and market definitions important, but the deﬁnition of levels presumed to inﬂuence decisions within the ﬁrm (or organization of interest) is also critical. Who is presumed to be making decisions within a ﬁrm about mergers and acquisitions? Will managers charged with making merger decisions be the same ones charged with implementing them? Does the practice of ﬁrms regarding mergers vary within an industry? Questionnaires that might be very appropriate for business-level managers in one ﬁrm might prove unintelligible to the same types of managers in other ﬁrms if these decisions are made elsewhere in the hierarchy. What if corporate and business unit managers inﬂuence merger decisions in different ways? Of course, researchers must make choices, and inevitably error is attached to any choice. Our point is simply that such choices should be considered explicitly. A response to this situation may be to handle concerns about levels through the use of control variables. However, if strategic management issues are inherently multi-level, then the use of control variables may be inappropriate and problematic. Controls are appropriate when the variable being controlled is inﬂuential but not central to the theoretical interests of the study; otherwise, controlling for a theoretically important variable is problematic to good research design. It is our contention that levels issues often are fundamental to the theoretical interests of strategy studies and thus should not be controlled. At a minimum, the rationales for the use of controls need to be elaborated more than is commonly found in current research.

Approaches to Dealing with Levels of Analysis Matter In our discussion of the different approaches for linking levels, we noted the strengths and weaknesses of each. It is doubtful that any of the three approaches is uniformly ‘‘best’’ for handling levels of analysis issues. However, this does not mean that these approaches are interchangeable. On the contrary, we believe it is likely that studies of the same phenomena that employ different approaches will reach different ﬁndings and perhaps come to very different conclusions. So which of these approaches should be used? As a starting point, we think it is necessary to explicitly consider how levels of analysis issues are to be handled, so as to ensure that a study’s results will be determined by the nature of the phenomena being studied and not unintentionally by the tastes of its researchers. The tastes of researchers certainly matter, but other factors do as well, including the objectives of a study, the nature of the available data, and the resources available for a study.

Multi-Level Issues for Strategic Management Research

151

How might the choice of one approach versus another matter in a study? We have already suggested that a transactional approach will focus on a particular transaction, exchange, or related interaction and then bridge levels through a process of aggregation. In contrast, an approach focusing on managerial roles will consider how the behavior of the individual or individuals exercising general management functions will serve to integrate the demands of multiple levels from a given situation. A comparison of these mechanisms suggests that studies, even of similar phenomena, will differ on a number of dimensions. Transactional studies would have relatively large sample sizes and employ relatively standardized data. For example, studies of mergers and acquisitions can use CRSP or SDC data, or even the FTC database, while studies of ﬁrm innovation behavior can access popular patent databases. Some basic hypotheses can be tested on fairly large samples with fairly standard data. In contrast, managerial studies will tend to have smaller samples. Data collection will be more difﬁcult and the data in general will be less standardized and less available. Some of the classic studies (Kotter, 1982) have two-dozen or fewer respondents. Studies of merger implementation have relied upon case data and smaller samples (Shanley & Correa, 1992; Kaplan et al., 2000). How would the results of different approaches differ for the same phenomena? It depends on what is being studied, of course, but it is difﬁcult to imagine that the results would not differ. Small-sample studies using data obtained from managers, for example, may be biased toward reﬂecting managerial interests in the topic of the study (Shanley & Correa, 1992). Regarding mergers, large-sample studies of various types have called merger performance into question (Porter, 1987; Ravenscraft & Scherer, 1987; Seth, 1990; Sirower, 1997). Of course, even with these results from academic studies, merger and acquisitions, driven by managerial decisions, remain popular. This tension between academic and practitioner approaches is not new, and the problem of levels is just one of several points on which these approaches may differ. Considering the levels problem explicitly may be useful for a variety of reasons. For example, it is not clear that academic and practitioner approaches to levels problems are fundamentally opposed to each other. Trying to understand how a market works, for example, does not imply that the question of how a participant in such a market should make decisions is unimportant. Similarly, understanding how institutional context operates in an industry setting does not mean that questions of ﬁrm agency or institutional change are unimportant. The problem arises when an approach is applied to the wrong question or when the conclusions from one

152

PAUL DRNEVICH AND MARK SHANLEY

study are applied uncritically to other contexts. This possibility may not be a fundamental issue of strategic theory, but it is an issue that can be addressed by making theoretical and study choices more explicitly. Theoretical Clarification Helps One to Make Choices Regarding Levels Problems One way in which the three approaches differ is in the elaboration of theory regarding the role of managers. The more detailed a researcher’s expectations are regarding the activities of managers and the ways in which they can inﬂuence their ﬁrm and its environment, the more likely it is that a managerial role approach to linking levels will be chosen. The less well developed the expectations regarding managerial action, and the more focused a researcher is on a focal set of common behaviors, the more likely it is that a transactional approach will prove effective. We are not suggesting that theoretical clariﬁcation will automatically lead to a choice of one approach over another, but rather that clariﬁcation will help match the mechanism selected to the needs and expectations of a study. This is not a terrible burden to place on strategy researchers. Whichever approach to handling levels is chosen, it should be reﬂected upon rather than chosen by default. Widely differing views in the ﬁeld dispute what constitutes ‘‘theory’’ – from those favoring a rigorous formalism with deduction and prediction, to those who view theory as more descriptive and sense-making, to those who view theory as the expected associations between a set of variables. Spending more time working through the theoretical expectations for cross-level interactions will likely prove useful whatever design is employed. Easy Fixes to Levels Problems in Strategy May Not Exist There is a temptation to consider levels of analysis issues as symptoms of strategy’s early stage of development as an area of social science research. The implication of doing so is to suggest that given sufﬁcient time and the accumulation of sufﬁcient research results, these ‘‘problems’’ will become better understood and less difﬁcult for researchers. Unfortunately, considering the nature of levels problems in strategy gives us cause to doubt this hopeful prognosis. To start with, the phenomena that are of most interest to strategic management scholars are inherently multi-level. The potential for strategic decisions to commit ﬁrms deep into the future and reduce their ﬂexibility in the face of change is also key to ideas of commitment and sustainability, and stems from the ability of managers to link their ﬁrm’s resources and

Multi-Level Issues for Strategic Management Research

153

capabilities to environmental conditions. A focus on the decisions and behaviors of top managers has been central to the ﬁeld since its inception and is unlikely to change. In addition, strategy has maintained its dual audience of academics and practitioners. This duality has ensured a continued focus on practitioner problems and their solutions, a continued tension between explanation and prescription in research, and a tendency toward more realism and less abstraction in strategy theories. All of these factors will work toward promoting more managerial approaches to levels issues, even as more traditional academic pressures may favor the beneﬁts of more rigorous transactional approaches.

CONCLUSIONS AND SUGGESTIONS FOR FUTURE RESEARCH Arguing that strategic management theory and research needs to recognize and embrace its inherent multi-level nature does not mean that existing theories are inadequate or that existing research approaches should be discontinued. For a wide range of topics, these theories, coupled with focused research designs, generate useful results. Pointing out the multi-level nature of strategy research does mean that for some topics more work across levels will be necessary if research results are to accumulate. How should progress be made on multi-level issues? One tactic is to conceptualize specific issues and problem areas in terms of how they involve multiple levels of analysis, as we have done with our focus on competitive advantage and value creation. Doing so forces the researcher to specify the issues involved in an area in a manner consistent with the requirements of multiple levels and thus recast how research can proceed. We believe that understanding the intersection of levels of analysis provides opportunities for theoretical insights (Coleman, 1990; Lonergan, 1970). It may also help avoid some of the theoretical problems, such as that of tautology, inherent in some more conventional approaches for which strategic management theories are frequently attacked (Camerer, 1985; Powell, 2001; Priem & Butler, 2001). Another way to proceed would be with the use of meta-theory or integrative efforts across levels of analysis. Along these lines, prior efforts to develop evolutionary views of strategic management theory (Kogut, 2000) are particularly useful because they provide us with a grounded review of the

154

PAUL DRNEVICH AND MARK SHANLEY

Table 8.

Suggestions for Future Research.

Value redistribution versus creation Competitive benchmarking, advantage, and distinctive competence Strategic pricing

dominant perspectives. Furthermore, contributions to the ﬁeld of strategy and other areas could no doubt be made by taking similar views of appreciative or integrative inquiry. Such approaches would assume that different theoretical traditions might beneﬁt from a compare-and-contrast exercise regarding how they viewed similar phenomena. For strategy research, for example, the interaction of industry economics and economic sociology would be enlightening. Of course, we are not arguing that theories should be integrated wherever possible. This view has long been a chimera in the social sciences, as seen in the rise and fall of such fads as systems theory. Replacing a number of inadequate theoretical bodies with a reduced number of more cumbersome ones is clearly not desirable. We would argue instead that efforts to address multi-level issues should be problem or issue driven and that their success should be determined on the grounds of usefulness and pragmatism. If such efforts to consider multi-level issues in a different way prove useful in accumulating research results and advancing inquiry, then they should proceed. Table 8 provides an overview of suggested areas for future research. Several areas of strategy research would beneﬁt from a prolonged consideration of levels of analysis issues. Some areas (mergers and acquisitions, innovation strategy, strategic groups) have already been mentioned in passing, but are too voluminous to consider in more detail here. The areas discussed below bear directly on the issues of value creation and competitive advantage that were discussed earlier.

Value Redistribution versus Creation If ‘‘value creation’’ is to mean more than just a new way of saying ‘‘profits,’’ then it must be possible, both theoretically and empirically, to distinguish between strategies that create value and those that just move value around among market actors. Do ﬁrm strategies actually create new value? It is likely that many successful strategies fail to create value and at best just redistribute value away from some products and uses to others, without providing any new beneﬁts for consumers.

Multi-Level Issues for Strategic Management Research

155

Market power explanations have contended that some aspects of interﬁrm transactions do not create new value and may actually destroy it, such as through collusion or anticompetitive pricing. Recent work on strategic groups (Dranove et al., 1998) has contended that strategic interactions lie at the heart of strategic group effects, calling into question whether strategic industry groups – even those whose existence can be ascertained – actually create any value at all. Work on market power aspects of strategies must also consider the industry conditions in which dominance strategies are sustainable (Shamsie, 2003).

Competitive Benchmarking, Advantage, and Distinctive Competence A related set of issues concerns the standards to be used for assessing the meaning of results. On the one hand, concerns about the RBV (and uniqueness), sustainability, and distinctive competence suggest that the bases for superior ﬁrm performance are local and highly individualized, and that performance standards should be similarly focused. On the other hand, competitive advantage and benchmarking ideas suggest that performance can be best assessed in terms of some reference set of other ﬁrms. This may be problematic, however, as the appropriate reference group may be far from clear and could vary widely depending on the performance dimension of interest. It suggests that ﬁrms looking at performance locally may miss the opportunity to see what performance gains are possible by not identifying the best reference ﬁrms. Both views are important, but their reconciliation is not obvious. Whether the comparison group for performance is deﬁned broadly or narrowly, it will always be necessary to consider how industry context inﬂuences performance. A tradition of research has assumed that industry conditions are exogenous to managers and constrain performance possibilities. There may also be possibilities for strategy to inﬂuence structure, either individually or collectively. In such a case, performance expectations and results will need to be considered differently. Furthermore, the technological and regulatory environment in which a ﬁrm operates may be sufﬁciently volatile that collective strategies may be required for successful adaptation and more traditional views of performance may prove inadequate. Assessing performance will prove even more problematic for diversiﬁed corporate actors. Strategic performance for these ﬁrms will include the issues raised previously, along with how the ﬁrm’s businesses interact. A further complicating factor is the reality that performance judgments for

156

PAUL DRNEVICH AND MARK SHANLEY

corporate actors will be generated by the interaction of corporate policies and procedures on the one hand, and the judgments of business unit managers on the other. Business unit managers will have choices regarding how they deﬁne their environments to corporate managers, while corporate managers will have choices in their oversight and motivation of business unit managers (Welch, 2001, p. 392). It will be necessary to know how different levels within a ﬁrm have interacted to appreciate the ﬁrm’s performance history.

Strategic Pricing Research on strategic pricing may provide a good vehicle for developing knowledge on value creation and competitive advantage. This research has not been as significant to date as other types of strategy research, perhaps because of the perception that prices are relatively easy to change and thus do not significantly commit the ﬁrm to a course of action. As a result, pricing decisions have not been considered as ‘‘strategic’’ decisions comparable to mergers and acquisitions, capacity expansions, new product launches, and related decisions requiring long-term commitments. In reality, ﬁrm prices may be stickier than many have thought. Moreover, the range of price changes will likely be limited greatly by significant investments of a ﬁrm in production capabilities on the one hand, and in product design and advertising on the other. If we begin to look at pricing strategically, then it becomes an inherently multi-level link between ﬁrm decisions and market definition; as such, models of price changes could be developed to reﬂect multi-level explanatory frameworks (Besanko, Dranove, & Shanley, 2001). Doing so will help us to begin viewing pricing as a strategic capability rather than as just an operating activity (Dutta, Zbaracki, & Bergen, 2003).

SUMMARY This chapter made the case that research issues in strategic management are essentially all problem focused, and that, to one extent or another, these problems usually span multiple levels of analysis. As such, they may align with different performance metrics and likely hold different performance implications from various perspectives – all of which may be important for strategy research. Most prior research has approached these issues from

Multi-Level Issues for Strategic Management Research

157

single perspectives, which in many cases may be problematic, especially in research concerned with ﬁrm performance issues such as competitive advantage and value creation. This chapter discussed potential approaches for addressing this conﬂict. We also considered the different ways in which strategy explanations link levels of analysis. We explored the multi-level performance implications of two concepts central to strategic management: competitive advantage and value creation. Finally, we offered a variety of potential solutions through several different approaches for future research to consider when addressing multi-level issues.

REFERENCES Alchian, A., & Demsetz, H. (1972). Production, information costs, and economic organization. American Economic Review, 62, 777–795. Aldrich, H. (1999). Organizations evolving. London: Sage. Andrews, K. (1971). The concept of corporate strategy. Homewood, IL: Irwin. Baldwin, C., & Clark, K. (2000). Design rules: The power of modularity. Cambridge, MA: MIT Press. Barnard, C. (1938). The functions of the executive. Cambridge, MA: Harvard University Press. Barney, J. (1986). Strategic factor markets: Expectations, luck, and business strategy. Management Science, 32, 1231–1241. Baum, J., & Dutton, J. (Eds) (1996). Advances in strategic management 1996: The embeddedness of strategy (Vol. 13). Greenwich, CT: JAI Press. Bennis, W., & Nanus, B. (1985). Leaders: The strategies for taking charge. New York: Harper & Row. Besanko, D., Dranove, D., & Shanley, M. (2001). Exploiting a cost advantage and coping with a cost disadvantage. Management Science, 47, 221–235. Besanko, D., Dranove, D., Shanley, M., & Schaefer, S. (2004). Economics of strategy (3rd ed.). New York: Wiley. Blau, P. (1964). Exchange and power in social life. New York: Wiley. Bower, J. (1970). Managing the resource allocation process. Boston: Harvard Graduate School of Business Administration. Bowman, E., & Helfat, C. (2001). Does corporate strategy matter? Strategic Management Journal, 22(1), 1–24. Brandenburger, A., & Stuart, H. (1996). Value-based business strategy. Journal of Economics and Management Strategy, 5, 5–24. Brush, T., & Bromiley, P. (1997). What does a small corporate effect mean? A variance components simulation of corporate and business effects. Strategic Management Journal, 18, 825–835. Burgelman, R. (1983). Corporate entrepreneurship: Insights from a process study. Management Science, 29, 1349–1364. Burt, R. (1992). Structural holes: The social structure of competition. Cambridge, MA: Harvard University Press.

158

PAUL DRNEVICH AND MARK SHANLEY

Camerer, C. (1985). Redirecting research in business policy and strategy. Strategic Management Journal, 6, 1–15. Campbell, D., & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Boston: Houghton Mifﬂin. Carroll, G., & Hannan, M. (2000). The demography of corporations and industries. Princeton, NJ: Princeton University Press. Chandler, A. (1962). Strategy and structure. Cambridge, MA: MIT Press. Chandler, A. (1977). The visible hand. Cambridge, MA: Belknap. Chandler, A. (2001). Inventing the electronic century. New York: Free Press. Child, J. (1972). Organization structure, environment, and performance – The role of strategic choice. Sociology, 6, 1–22. Coase, R. (1937). The nature of the ﬁrm. Econometrica, 4, 386–405. Coleman, J. (1990). Foundations of social theory. Cambridge, MA: Belknap. Collins, J. (2001). Good to great. New York: Harper Business. Cyert, R. M., & March, J. G. (1963). A behavioral theory of the firm. Englewood Cliffs, NJ: Prentice-Hall. DiMaggio, P. (1994). Culture and economy. In: N. Smelser & R. Swedberg (Eds), The handbook of economic sociology (pp. 27–57). Princeton, NJ: Princeton University Press. Donaldson, G., & Lorsch, J. (1983). Decision making at the top. New York: Harper & Row. Dranove, D., Peteraf, M., & Shanley, M. (1998). Do strategic groups exist? An economic framework for analysis. Strategic Management Journal, 19, 1029–1044. Dutta, S., Zbaracki, M., & Bergen, M. (2003). Pricing process as a capability: A resource-based perspective. Strategic Management Journal, 24, 615–630. Eisenhardt, K. (1989). Agency theory: An assessment and review. Academy of Management Review, 14, 57–74. Emirbayer, M., & Goodwin, J. (1994). Network analysis, culture, and the problem of agency. American Journal of Sociology, 99, 1411–1454. Fombrun, C., & Shanley, M. (1990). Reputation-building and corporate strategy. Academy of Management Journal, 33, 233–258. Gabarro, J. (1985). When a new manager takes charge. Harvard Business Review, May–June, 110–123. Ghoshal, S., Hahn, M., & Moran, P. (2002). Management competence, ﬁrm growth, and economic progress. In: C. Pitelis (Ed.), The growth of the firm, the legacy of Edith Penrose (pp. 279–303). Oxford, UK: Oxford University Press. Granovetter, M. (1985). Economic action and social structure: The problem of embeddedness. American Journal of Sociology, 91, 481–510. Hannan, M., & Freeman, J. (1989). Organizational ecology. Cambridge, MA: Harvard University Press. Hayward, M., Rindova, V., & Pollock, T. (2004). Believing one’s own press: The causes and consequences of CEO celebrity. Strategic Management Journal, 25, 637–653. Hermalin, B. (1998). Towards an economic theory of leadership: Leading by example. American Economic Review, 88, 1188–1206. Hickson, D., Butler, R., Cray, D., Mallory, G., & Wilson, D. (1986). Top decisions: Strategic decision making in organizations. San Francisco: Jossey-Bass. Hirsch, P. (1997). Review essay. Sociology without social structure, neoinstitutional theory meets Brave New World. American Journal of Sociology, 102, 1702–1723.

Multi-Level Issues for Strategic Management Research

159

Jensen, M., & Meckling, W. (1976). Theory of the ﬁrm: Managerial behavior, agency costs, and ownership structure. Journal of Financial Economics, 3, 305–360. Kaplan, S., Mitchell, M., & Wruck, K. (2000). A clinical exploration of value creation and destruction in acquisitions. In: S. Kaplan (Ed.), Mergers and productivity (pp. 179–227). Chicago: University of Chicago Press. Keegan, J. (2003). Intelligence in war: Knowledge of the enemy from Napoleon to Al-Qaeda. New York: Knopf. Khanna, T., & Rivkin, J. (2001). Estimating the performance effects of business groups in emerging markets. Strategic Management Journal, 22, 45–74. Kogut, B. (2000). The network as knowledge: Generative rules and the emergence of structure. Strategic Management Journal, 21, 405–425. Kotter, J. (1982). The general managers. New York: Free Press. Kreps, D. (1990). Corporate culture and economic theory. In: J. Alt & K. Shepsle (Eds), Perspectives on positive political economy. Cambridge, UK: Cambridge University Press. Laumann, E., & Knoke, D. (1987). The organizational state. Madison, WI: University of Wisconsin Press. Levine, S., & White, P. (1961). Exchange as a conceptual framework for the study of interorganizational relations. Administrative Science Quarterly, 5, 583–610. Lippman, S., & Rumelt, R. (2003). The payments perspective: Micro-foundations of resource analysis. Strategic Management Journal, 24, 903–927. Lonergan, B. (1970). Insight: A study of human understanding. New York: Philosophical Library. Madhok, A. (2002). Reassessing the fundamentals and beyond: Ronald Coase, the transaction cost and resource-based theories of the ﬁrm and the institutional structure of production. Strategic Management Journal, 23, 535–550. March, J. G., & Simon, H. A. (1958). Organizations. New York: Wiley. McGahan, A., & Porter, M. (1997). How much does industry matter, really? Strategic Management Journal, 18, 15–30. McGahan, A., & Porter, M. (1999). The performance of U.S. corporations: 1981–1994. Journal of Industrial Economics, 47, 373–398. Milgrom, P., & Roberts, J. (1988). Economic theories of the ﬁrm: Past, present, and future. Canadian Journal of Economics, 21, 444–458. Miller, G. (1990). Managerial dilemmas: Political leadership in hierarchies. In: K. Cook & M. Levi (Eds), The limits of rationality (pp. 324–347). Chicago: University of Chicago Press. Miller, C., Burke, L., & Glick, W. (1998). Cognitive diversity among upper-echelon executives: Implications for strategic decision processes. Strategic Management Journal, 19, 39–58. Murtha, T., & Lenway, S. (1994). Country capabilities and the strategic state: How national political institutions affect multinational corporations’ strategies. Strategic Management Journal, 15, 113–130. Oliver, C. (1997). Sustainable competitive advantage: Combining institutional and resourcebased views. Strategic Management Journal, 18, 697–713. Penrose, E. (1959). The theory of the growth of the firm. Oxford, UK: Oxford University Press. Penrose, E. (1960). The growth of the ﬁrm – A case study, the Hercules Power Company. Business History Review, 34, 1–23.

160

PAUL DRNEVICH AND MARK SHANLEY

Penrose, P., & Pitelis, C. (2002). Edith Elura Tilton Penrose: Life, contribution, and inﬂuence. In: C. Pitelis (Ed.), The growth of the firm: The legacy of Edith Penrose (pp. 17–36). Oxford, UK: Oxford University Press. Peteraf, M. (1993). The cornerstones of competitive advantage: A resource based view. Strategic Management Journal, 14, 179–191. Pfeffer, J. (1987). A resource dependence perspective on intercorporate relations. In: M. S. Mizruchi & M. Schwartz (Eds), Intercorporate relations: The structural analysis of business (pp. 25–55). Cambridge, UK: Cambridge University Press. Pfeffer, J., & Salancik, G. (1978). The external control of organizations. New York: Harper & Row. Phillips, A. (1960). A theory of interﬁrm organization. Quarterly Journal of Economics, 74, 602–613. Porter, M. (1980). Competitive strategy. New York: Free Press. Porter, M. (1985). Competitive advantage. New York: Free Press. Porter, M. (1987). From competitive advantage to corporate strategy. Harvard Business Review, 65, 43–59. Powell, T. (2001). Competitive advantage: Logical and philosophical considerations. Strategic Management Journal, 22, 875–888. Priem, R., & Butler, J. (2001). Is the resource-based view a useful perspective for strategic management research? Academy of Management Review, 26, 22–41. Ravenscraft, D., & Scherer, F. (1987). Mergers, sell-offs, and economic efficiency. Washington, DC: Brookings Institution. Richardson, G. (1972). The organization of industry. Economic Journal, 62, 883–896. Rueﬂi, T., & Wiggins, R. (2003). Industry, corporate, and segment effects and business performance: A non-parametric approach. Strategic Management Journal, 24, 861–880. Rumelt, R. (1991). How much does industry matter? Strategic Management Journal, 12, 167–185. Schmalensee, R. (1985). Do markets differ much? American Economic Review, 75, 341–351. Schnatterly, K. (2003). Increasing ﬁrm value through detection and prevention of white-collar crime. Strategic Management Journal, 24(7), 587–614. Scott, W. (2001). Institutions and organizations (2nd ed.). Thousand Oaks, CA: Sage. Seth, A. (1990). Sources of value creation in acquisition: An empirical investigation. Strategic Management Journal, 11(6), 431–446. Seth, A., & Thomas, H. (1994). Theories of the ﬁrm: Implications for strategy research. Journal of Management Studies, 31(2), 165–191. Shamsie, J. (2003). The context of dominance: An industry-driven framework for exploiting reputation. Strategic Management Journal, 24, 199–216. Shanley, M., & Correa, M. (1992). Perceived versus actual agreement in acquisition integration. Strategic Management Journal, 13, 245–266. Short, J., Palmer, T., & Ketchen, D. (2003). Multilevel inﬂuences on ﬁrm performance: Insights from the resource-based view and strategic groups research. In: F. Dansereau & F. Yammarino (Eds), Research in multilevel issues, (Vol. 2, pp. 155–187). New York: JAI Press. Silverman, B. (1999). Technological resources and the direction of corporate diversiﬁcation: Toward an integration of the resource-based view and transaction cost economics. Management Science, 45, 1109–1124. Sirower, M. (1997). The synergy trap. New York: Free Press.

Multi-Level Issues for Strategic Management Research

161

Spulber, D. (1994). Economic analysis and management strategy: A survey continued. Journal of Economics and Management Strategy, 3, 355–406. Spulber, D. (2003). Management strategy. New York: McGraw-Hill/Irwin. Sutcliffe, K., & Zaheer, A. (1998). Uncertainty in the transaction environment: An empirical test. Strategic Management Journal, 19, 1–23. Sutton, J. (1992). Sunk costs and market structure. Cambridge, MA: MIT Press. Thompson, J. (1967). Organizations in action. New York: McGraw-Hill. Waddock, S., & Graves, S. (1997). The corporate social performance–ﬁnancial performance link. Strategic Management Journal, 18, 303–319. Welch, J. (2001). Jack: Straight from the gut. New York: Warner-Business. Wernerfelt, B. (1984). The resource based view of the ﬁrm. Strategic Management Journal, 5, 171–180. Wernerfelt, B., & Montgomery, C. (1988). Tobin’s Q and the importance of focus in ﬁrm performance. American Economic Review, 78, 246–250. Willer, D. (1999). Network exchange theory. Westport, CT: Praeger. Williamson, O. (1975). Markets and hierarchies. New York: Free Press. Williamson, O. (1985). The economic institutions of capitalism. New York: Free Press. Zajac, E., & Bazerman, M. (1991). Blind spots in strategic decision making: The case of competitor analysis. Academy of Management Review, 16, 37–56.

This page intentionally left blank

162

DEVELOPING MULTI-LEVEL THEORY IN STRATEGIC MANAGEMENT: THE CASE OF MANAGERIAL TALENT AND COMPETITIVE ADVANTAGE Alison Mackey and Jay B. Barney ABSTRACT This chapter applies arguments advanced by Drnevich and Shanley (this volume) to the strategic leadership literature – an area of work where such multi-level analyses seem likely to be particularly appropriate. In an analysis of the relationship between managerial capabilities and firm performance, this chapter breaks from tradition in the strategic leadership literature by examining the interaction between three levels of analysis. In doing so, this chapter identifies the conditions under which leadership can be a source of competitive advantage for a firm, when labor markets will allocate managerial talent imperfectly across competing firms, and when managers will and will not be able to appropriate the rents their specific managerial talents might generate.

Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 163–175 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04006-3

163

164

ALISON MACKEY AND JAY B. BARNEY

INTRODUCTION Traditionally, work in strategic leadership has focused on the individual, group, organization, or the market as the unit of analysis. Rarely have the effects of one of these levels of analysis on another level been examined (Finkelstein & Hambrick, 1996). Yet, as Drnevich and Shanley (this volume) suggest, strategic leadership is precisely that kind of strategic phenomena that must be examined using multiple levels of analysis simultaneously. Consider, for example, the problem of establishing whether or not managers can create competitive advantages for ﬁrms (Finkelstein & Hambrick, 1996). A major problem with establishing the link between managers and competitive advantage has been that rents from a competitive advantage will not be observed in performance measures if they are appropriated by stakeholders other than the ﬁrm owners (Coff, 1999). In the context of managers as a source of competitive advantage, it is not hard to imagine a scenario in which the top managers of a ﬁrm possess critical knowledge or other resources that enable them to help a ﬁrm generate a competitive advantage, but that these managers also appropriate all the rents generated with this advantage. Thus, the interaction between at least three levels of analysis – the individual-level resources controlled by a manager, the industry-level competitive advantages derived from these resources, and the market-determined ability of a manager to appropriate the rents these competitive advantages generate – are important in understanding the relationship between managerial capabilities and ﬁrm performance. This discussion of strategic leadership begins by applying resource-based theory, at the level of the individual manager and the level of the ﬁrm, to identify conditions under which such leadership can be a source of competitive advantage for a ﬁrm. Then, the unit of analysis shifts to the market – in this case – the labor market, the mechanism by which these potential sources of competitive advantage are allocated across ﬁrms. Finally, the interaction between these individual resources, ﬁrm competitive advantages, and the labor markets through which managers are allocated to ﬁrms is used to describe conditions under which managers will and will not be able to appropriate the rents their specific managerial talents might generate.

CAN MANAGERIAL TALENT BE A SOURCE OF COMPETITIVE ADVANTAGE FOR FIRMS? Resources acquired in imperfectly competitive factor markets have the potential to generate competitive advantage (Barney, 1986). In fact, there is

Developing Multi-Level Theory in Strategic Management

165

good reason to expect that many managerial labor markets – and especially the labor market for senior managers – are imperfectly competitive. The value of managerial talent is often difﬁcult to ascertain, ex ante; exceptional talent is rare, costly to imitate, and without close substitutes. Firms that are able to acquire talent in imperfectly competitive labor markets may, sometimes, gain competitive advantages from doing so.

The Question of Value Managerial talent can only be a source of competitive advantage or sustained competitive advantage if managers are valuable in conceiving of or implementing successful strategies for organizations (Barney, 1991). Some theorists and many practitioners have argued that leadership – and especially leadership in a ﬁrm’s senior executive positions – has an important impact on the ability of a ﬁrm to conceive of and implement value creating strategies (Barnard, 1938; Collins, 2001; Collins & Porras, 1994; Drucker, 1954; Kotter, 1996). The activities of leaders affect the productivity of all individuals who work below them in the organizational hierarchy (i.e. scale effects) (Rosen, 1990). Leaders infuse the organization with its values (Selznick, 1957), formulate its purpose (Barnard, 1938), mold its culture (Schien, 1992), and direct its course of action (Tichey & Cohen, 1997). On the other hand, others have argued that leadership is unimportant because organizational and environmental constraints limit the inﬂuence that any one leader can have (Hall, 1977; Hannan & Freeman, 1989; Pfeffer & Salancik, 1978) and because executives tend to be homogeneous with respect to personal characteristics, socialization, and training processes (Finkelstein & Hambrick, 1996; March & March, 1977; Whitehill, 1991). Some empirical work suggests managers can substantially alter organizations (Mackey, 2004; Wasserman, Nohria, & Anand, 2001) while other work ﬁnds that the person who holds particular executive positions, on average, does not have a significant impact on a ﬁrm (Lieberson & O’Connor, 1972; Samuelson, Galbraith, & McGuire, 1985). This variance in leader effects may be reconciled by recognizing the varying levels of discretion that leaders inevitably have (Hambrick & Finkelstein, 1987). Thus, managers have the most potential for value creation in situations in which they have the highest managerial discretion. Hambrick and Finkelstein (1987) conceptualized managerial discretion as a function of environmental, organizational, and individual managerial characteristics. To understand the relationship between discretion and

166

ALISON MACKEY AND JAY B. BARNEY

competitive advantage it is important to differentiate between the portion of managerial discretion that resides within the firm (from the environmental and organizational characteristics) and the portion that resides in the manager (from the individual managerial characteristics). Henceforth, the portion of managerial discretion that resides within the ﬁrm will be referred to as managerial discretion and the portion of managerial discretion that resides within the manager will be referred to as managerial talent. This distinction is important because it will be shown that while managerial discretion within the ﬁrm is a valuable resource, the available managerial talent in the managerial labor market may not be.

The Question of Rarity Managerial talent will not be a source of sustained competitive advantage if many competing ﬁrms are able to acquire it in managerial labor markets. If managerial talent is a valuable but common resource, it will, however, still be a crucial resource for ﬁrms to gain competitive parity as well as to ensure organizational survival (Porter, 1980). If managerial talent is in fact rare, then many managers thought to be valuable resources will in fact turn out to lack the managerial talent needed to generate competitive advantage. Since managerial talent is difﬁcult to assess ex ante, ﬁrms re-evaluate the expected talent level of managers after being employed by the ﬁrm for some time. If managers are unable to generate competitive advantage for the ﬁrm, they will often be dismissed; thus, high turnover rates and short lengths of managerial tenure will be observed if managerial talent is a scarce resource. Empirical work in executive turnover conﬁrms that leaders (i.e. CEOs, top management teams, and directors) are significantly more likely to lose their jobs following poor ﬁrm performance (Coughlan & Schmidt, 1985; Finkelstein & Hambrick, 1996; Goldman, Hazarika, & Shivdasani, 2003; Huson, Parrino, & Starks, 2001; Kaplan, 1994; Warner, Watts, & Wruck, 1988). While this pattern is robust in the literature, it is not particularly powerful in explaining much of the variance in departure rates (Finkelstein & Hambrick, 1996). Thus, the performance–dismissal link has also been subject to considerable criticism. For example, Weisbach (1988) ﬁnds that for ﬁrms with outside dominated boards, moving from the top decile of performance to the bottom decile of performance raises the likelihood of turnover by only 5.7%. Huson et al. (2001) show that CEO turnover is concentrated in ﬁrms that fall into the

Developing Multi-Level Theory in Strategic Management

167

bottom decile of ﬁrm performance and argue that boards only ﬁre CEOs in extreme cases of poor performance. Goldman et al. (2003) demonstrate that board actions are sensitive to ﬁrm performance only when the absolute level of performance is low. For ﬁrms satisfying a performance threshold, there is no relationship between changes in performance and CEO turnover. Thus, the context of the poor performance is extremely important in that whether or not a turnover event will occur (e.g. whether the low performance is unexpected or persistent or whether the performance is just progressively deteriorating). Whether or not a ﬁrm dismisses a manager following poor performance is thus related to the expectations that were held ex ante about the manager’s level of managerial talent.

The Question of Imitability Valuable and rare managerial talent must also be imperfectly imitable for ﬁrms to accrue competitive advantage from its use (Barney, 1986; Lippman & Rumelt, 1982). If competitors can easily acquire managerial talent, it will not be a source of sustained competitive advantage for any ﬁrm. Isolating mechanisms (Rumelt, 1984) such as causal ambiguity (Lippman & Rumelt, 1982) concerning the link between the resource and subsequent competitive advantage and social complexity in the management and manipulation of the resource are two reasons managerial talent may be imperfectly imitable (Barney, 1991). Firms that own managerial talent, potential imitators, and the managers themselves can all be limited in their ability to understand the linkage between the managerial talent controlled by the ﬁrm and the ﬁrm’s sustained competitive advantage. When this linkage is poorly understood, it is not clear to the ﬁrm how much economic rent should be appropriated by the managerial talent; it is not clear to the imitators which managers should be acquired (assuming this is the resource responsible for the ﬁrm’s competitive advantage) and what the strategic future value of those managers will be in a new ﬁrm; nor is it clear to the manager herself how much value she brings to the ﬁrm. Thus, causal ambiguity prevents managers from appropriating their true value both within and outside of the ﬁrm. This is likely to occur even if managerial talent is not a specialized asset (Teece, 1987; Williamson, 1985). Causal ambiguity will not always be the reason that managerial talent is an imperfectly imitable resource. The social complexity of acquiring, managing, and/or inﬂuencing managerial talent is likely to be a significant

168

ALISON MACKEY AND JAY B. BARNEY

barrier for imitation (Barney, 1991). For example, General Electric’s unparalleled ﬁnancial success is generally attributed to the ﬁrm’s ability to develop great managerial talent. Numerous books and articles have been written about ‘‘The GE Way,’’ detailing the link between managerial talent and competitive advantage (e.g. Krames, 2001; Slater, 1998). Yet, the social complexity of developing/harnessing managerial talent to create competitive advantage has protected this resource from simple imitation. The Question of Substitutability The existence of strategically equivalent valuable, rare, and inimitable resources would undermine the competitive advantage-generating potential of managerial talent. Whether or not these substitutes exist for managerial talent depends upon the strategy being conceived of or implemented. It is not hard to imagine a context in which the implementation of a strategy is so constrained by environmental and organizational factors that managerial talent can be replaced by substitute resources such as strategic planning processes systematized in the ﬁrm’s organization (Pearce, Freeman, & Robinson, 1987). Strategic planning processes are, however, developed by individuals and hence may actually be representations of the level of managerial talent in the ﬁrm’s history. Although there may be instances in which managerial talent can be imitated or substituted, the imperfections in the market suggest that resource heterogeneity and immobility are likely to persist. The task facing ﬁrms is to improve their capabilities to recognize and evaluate managerial talent.

HOW DOES THE LABOR MARKET ALLOCATE MANAGERIAL TALENT? The previous discussion suggested that, in fact, managerial talent can sometimes be a source of sustained competitive advantage for a ﬁrm. However, this talent must be allocated to a ﬁrm, and thus, as suggested by Drnevich and Shanley (this volume), a full understanding of the impact of leadership on ﬁrm performance depends not only on the existence of managerial talent, at the individual level, but also on how this talent is allocated, at the level of the labor market. A competitive managerial labor market allocates talented individuals to higher-level positions in larger ﬁrms. The most talented managers, in

Developing Multi-Level Theory in Strategic Management

169

equilibrium, will earn the highest levels of compensation (Rosen, 1990) as larger ﬁrms pay higher executive wages. However, since competitive imperfections exist in most managerial labor markets, the most talented individuals will not necessarily hold the highest positions in the largest ﬁrms. In fact, the largest ﬁrms in very certain environments might not need high levels of managerial talent if the opportunity for managerial inﬂuence is low (Hambrick & Finkelstein, 1987). Managerial talent is only valuable to the extent that it can be used to conceive of and implement ﬁrm strategies (i.e. the extent of the discretion). The Coase Theorem (1960) implies that since managerial talent will be the most valuable to the ﬁrm with the most managerial discretion, this ﬁrm will be willing to pay the most for talent. Thus, in a world of perfect information about managerial talent, the managers with the most ability will be hired by the ﬁrms that can offer the most managerial discretion. Without perfect information, ﬁrms hire based on expectations of talent. The selection process in this imperfect labor market has been described as dysfunctional (Bennis & O’Toole, 2000) and wrought with cognitive biases (Khurana, 2004). Corporate boards often have divergent views about what traits ‘‘signal’’ managerial talent. Some observers and scholars have posited that individuals with managerial talent are charismatic and ‘‘move the human heart’’ (Bennis & O’Toole, 2000) while others have suggested that talented managers are able to ‘‘blend extreme personal humility with intense professional will’’ (Collins, 2001, p. 21). The bilateral information asymmetries that exist between ﬁrms and managers with respect to a manager’s talent level further complicate the managerial selection process. Even if those who ‘‘own’’ the talent know more about it than those who may buy it, the ‘‘owner’s’’ (i.e. manager’s) information will still be imperfect. Efforts to signal talent (i.e. performancebased compensation packages) will not adequately resolve the information asymmetries. The imperfect information regarding an individual’s level of managerial talent creates incentives for boards to rely on heuristics to make selection decisions. For example, boards have been found to attribute leadership qualities to individuals from high-status ﬁrms, highly performing ﬁrms, or those who have held the CEO position in the past (Khurana, 2004). Thus, boards seem to rely on the efﬁciency of the managerial labor market in making their selection decisions. Yet, the competitive imperfections in the market suggest that the allocation of talent throughout the managerial labor market will not be efﬁcient (i.e. the most talented managers will not necessarily occupy the most prestigious, well-compensated positions).

170

ALISON MACKEY AND JAY B. BARNEY

Adjustments in the managerial labor market occur as ﬁrms alter their theories for what ‘‘signals’’ managerial talent and as selection errors are identiﬁed. Although these adjustments cause ﬁrm-level productivity losses, these adjustments inevitably improve expectations concerning talent and ultimately decrease the imperfections in the managerial labor market. Additionally, the value to shareholders of devising efﬁcient mechanisms to identify the most talented managers are likely to be extremely large because of the scale effects of manager activities. When a ﬁrm acquires a manager, the manager’s compensation package partially reﬂects the probability that the ﬁrm has overestimated the manager’s value to the ﬁrm. If there were never any selection errors, the compensation for managers would be higher. Because there are at least some managers who will be worth less to the ﬁrm than expected, the average compensation of managers is discounted (Akerlof, 1970). Akerlof’s ‘‘Lemon Principle’’ does not completely apply to the managerial labor market for one very important reason – every ﬁrm has to buy a manager’s services; in Akerlof’s model, not everyone has to buy a used car. In Akerlof’s example of the used car market, owners of good used cars will not bring them to the used car market because they would be sold for less than their value. ‘‘Good’’ managers still participate in the managerial labor market because they have no outside option. Hence, ‘‘bad’’ managers do not drive out ‘‘good managers’’ from the managerial labor market.

WHO APPROPRIATES THE ECONOMIC VALUE FROM MANAGERIAL TALENT? The question of ‘‘who appropriates’’ any value created by managerial talent introduces the last level of analysis into the study of strategic leadership and ﬁrm performance. According to the managerial power view (Bebchuk, Fried, & Walker, 2002), executives are able to extract rents from shareholders through mechanisms such as stock options because executives control the board. Resource-based logic and stakeholder bargaining power theory suggest that managers possessing critical knowledge may be about to appropriate rents from shareholders (Coff, 1999). Yet, it is likely that managers will be unable to extract the full economic value that they create because of the information asymmetries in the managerial labor market and because of the causal ambiguity about the linkage between managerial behavior and competitive advantage.

Developing Multi-Level Theory in Strategic Management

171

Managerial Discretion From the ﬁrm’s perspective, it is the managerial discretion that resides within the ﬁrm (determined by the size and risk of the decisions managers must make) that allows the ﬁrm to realize value from managerial labor. It is difﬁcult to determine exactly how much value will be appropriated by the manager and how much will be appropriated by the ﬁrm, since this is a bargaining situation (Lippman & Rumelt, 2003), but, in general, the ﬁrm with the most managerial discretion will pay the most talented manager no more than the ﬁrm with the second most managerial discretion would gain from his services. This most talented manager’s compensation would also be no more than the compensation offered to the second most talented manager by the ﬁrm with the second most managerial discretion. Under information asymmetries, it will be the manager with the highest perceived value who is hired by the ﬁrm with the highest managerial discretion. Consider the situation in which all ﬁrms have the same amount of managerial discretion, but managers differ in their talent levels. In this case, the most talented managers will be paid the most because they would be able to create the most value from the same opportunity set and each ﬁrm would appropriate the same amount of value from managerial labor. If this were not the case, then ﬁrms would have incentives to hire away managers from ﬁrms that are gaining more value. Now suppose one ﬁrm increases its discretion (e.g. changes in board composition). This ﬁrm would not have to increase the compensation of the most talented manager in the market because his or her alternatives have not changed (i.e. no other ﬁrm would be willing to pay this manager more). In this situation all the excess value will accrue to the ﬁrm. If managerial talent is scarce in the labor market, ﬁrms with high discretion beneﬁt from increases in the talent available in the market and from increases in the heterogeneity of talent, but the changes would not help ﬁrms with less discretion. For example, suppose one ﬁrm has superior discretion, but all managers are identically talented. In this case, all managers will be paid their reservation salary, but the ﬁrm with superior discretion will accrue rents from their manager creating more value with more discretion at his disposal. Now suppose one manager gains superior talent. This manager would receive additional compensation since the acquisition of this manager would add value to any ﬁrm. This manager would not appropriate the full value added to the ﬁrm, however. This increase, however, would not be greater than the increase in value an ordinary ﬁrm would get from hiring the

172

ALISON MACKEY AND JAY B. BARNEY

superior manager over an ordinary manager. Since the superior ﬁrm will get more value from the superior manager than an ordinary ﬁrm would, the superior ﬁrm will gain at least this difference in ﬁrm values.

Management Development The total amount of managerial talent and the degree of heterogeneity in talent in the market and the total amount of and heterogeneity of managerial discretion that resides within each ﬁrm jointly determine the value from managerial labor and the price of that labor. Under information asymmetries, ﬁrms will increase the managerial talent available to them by investing in management development. Firms with the most managerial discretion will invest the most in management development. The heterogeneity in discretion within ﬁrms explains why ﬁrms would invest resources to develop managers’ general (non-ﬁrm-specific) abilities when such actions would seemingly raise the ability of managers to negotiate pay raises. Increasing the talent level of managers increases the value managers can create for ﬁrms more than it raises the compensation for the managers. Asymmetric information in the managerial labor market is one justiﬁcation for ﬁrms’ developing managers’ general (non-ﬁrm-specific) ability, when it seemingly increases the ability of managers to negotiate pay raises. If good managers are systematically paid less than the value they create, and bad managers are paid more than the value they create, then there is a level of cost for which it is in ﬁrms’ best interest to pay to develop managerial ability and pay higher salaries to these well-trained managers. The rarity of managerial talent inhibits a ﬁrm’s ability to gain from the managerial talent it employs. Hence, developing managerial talent is a way ﬁrms with discretion will partially allay the scarcity of talent.

CONCLUSION This chapter builds on the observations made by Drnevich and Shanley (this volume). Indeed, strategic leadership, and its impact on a ﬁrm’s competitive advantage and performance, is the kind of strategic phenomena where a failure to recognize the effects of multiple levels of analysis simultaneously is likely to lead to, at best, incomplete, and at worst, misleading conclusions. This chapter observes, ﬁrst, that individual management skills can

Developing Multi-Level Theory in Strategic Management

173

sometimes be a source of sustained competitive advantages for ﬁrms, second, that labor markets will generally allocate these kinds of resources imperfectly across competing ﬁrms, and ﬁnally, that the existence of heterogeneous management skills and imperfectly competitive labor markets, together, have an impact on how much of the rent generated by managerial talent will be appropriated by the ﬁrm, and how much will be appropriated by the manager. If, in this analysis, individual managers did not vary in their skills, then managerial talent would not be a source of competitive advantage for ﬁrms. Thus, the individual unit of analysis is essential for understanding the competitive effects of managerial talent – an effect realized at the ﬁrm level. However, if labor market dynamics are not also introduced into this analysis, then the impact that heterogeneous individual attributes and ﬁrm level competitive advantages would have on observed ﬁrm performance could also not be understood. Thus, just as suggested by Drnevich and Shanley (this volume), only when all three of these levels of analysis are considered simultaneously is it possible to understand the relationship between management talent and ﬁrm performance.

REFERENCES Akerlof, G. A. (1970). The market for ‘‘lemons’’: Quality uncertainty and the market mechanism. Quarterly Journal of Economics, 84, 488–500. Barnard, C. (1938). The functions of the executive. Cambridge, MA: Harvard University Press. Barney, J. B. (1986). Strategic factor markets: expectation, luck, and business strategy. Management Science, 32, 1231–1241. Barney, J. B. (1991). Firm resources and sustained competitive advantage. Journal of Management, 17, 99–120. Bebchuk, L. A., Fried, J. M., & Walker, D. I. (2002). Managerial power and rent extraction in the design of executive compensation. University of Chicago Law Review, 69, 751–846. Bennis, W., & O’Toole, J. (2000). Don’t hire the wrong CEO. Harvard Business Review, 78, 170–176. Coase, R. H. (1960). The problem of social cost. The Journal of Law and Economics, 3, 1–44. Coff, R. W. (1999). When competitive advantage doesn’t lead to performance: The resourcebased view and stakeholder bargaining power. Organization Science, 10, 119–133. Collins, J. C. (2001). Good to great: Why some companies make the leapyand others don’t. New York: Harper-Collins Publishers. Collins, J. C., & Porras, J. I. (1994). Built to last: Successful habits of visionary companies. New York: Harper-Collins Publishers. Coughlan, A. T., & Schmidt, R. M. (1985). Executive compensation, managerial turnover, and ﬁrm performance: An empirical investigation. Journal of Accounting and Economics, 7, 43–66.

174

ALISON MACKEY AND JAY B. BARNEY

Drucker, P. (1954). The principles of management. New York: Harper-Collins Publishers. Finkelstein, S., & Hambrick, D. C. (1996). Strategic leadership: Top executives and their effects on organizations. Minneapolis/St. Paul, MN: West’s Strategic Management Series. Goldman, E., Hazarika, S., & Shivdasani, A. (2003). What determines CEO turnover? Working Paper, Kenan-Flagler Business School, Chapel Hill, NC. Hall, R. H. (1977). Organizations: structure and process (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall. Hambrick, D. C., & Finkelstein, S. (1987). Managerial discretion: A bridge between polar views of organizations. In: L. L. Cummings & B. M. Staw (Eds), Research in organizational behavior, (Vol. 9, pp. 369–406). Greenwich, CN: JAI Press. Hannan, M. T., & Freeman, J. H. (1989). Organizational ecology. Cambridge, MA: Harvard University Press. Huson, R. M., Parrino, R., & Starks, L. T. (2001). Internal monitoring mechanisms and CEO turnover: A long-term perspective. Journal of Finance, 56, 2265–2297. Kaplan, N. S. (1994). Top executive rewards and ﬁrm performance: A comparison of Japan and the United States. Journal of Political Economy, 102, 510–546. Khurana, R. (2004). Constructing leadership: The social construction of charisma in the CEO succession process. Academy of Management Annual Meeting, New Orleans, LA. Kotter, J. P. (1996). Leading change. Cambridge, MA: Harvard Business School Press. Krames, J. A. (2001). The Jack Welch lexicon of leadership: Over 250 terms, concepts, strategies & initiatives of the legendary leader. New York: McGraw Hill. Lieberson, S., & O’Connor, J. F. (1972). Leadership and organizational performance: A study of large corporations. American Sociological Review, 37, 117–130. Lippman, S., & Rumelt, R. (1982). Uncertain imitability: An analysis of interﬁrm differences in efﬁciency under competition. Bell Journal of Economics, 13, 418–438. Lippman, S., & Rumelt, R. (2003). The payments perspective. Strategic Management Journal, 24, 903–927. Mackey, A. (2004). The executive effect on firm performance. Working Paper, Fisher College of Business, Ohio State University. March, J. C., & March, J. G. (1977). Almost random careers – the Wisconsin school superintendency, 1940–1972. Administrative Science Quarterly, 22, 377–409. Pearce, J. A., Freeman, E. B., & Robinson, R. B. (1987). The tenuous link between formal strategic planning and ﬁnancial performance. Academy of Management Review, 12, 658–675. Pfeffer, J., & Salancik, G. R. (1978). The external control of organizations: A resource-dependence perspective. New York: Harper & Row. Porter, M. E. (1980). Competitive strategy: Techniques for analyzing industries and competitors. New York: The Free Press. Rosen, S. (1990). Contracts and the market for executives. NBER Working Paper No. W3542. Cambridge, MA. Rumelt, R. (1984). Toward a strategic theory of the ﬁrm. In: R. Lamb (Ed.), Competitive strategic management (pp. 556–570). Englewood Cliffs, NJ: Prentice-Hall. Samuelson, B. A., Galbraith, C. S., & McGuire, J. W. (1985). Organizational performance and top-management turnover. Organization Studies, 6, 275–291. Schien, E. (1992). Organizational culture and leadership. San Francisco, CA: Jossey-Bass. Selznick, P. (1957). Leadership in administration: A sociological interpretation. New York: Harper & Row.

Developing Multi-Level Theory in Strategic Management

175

Slater, R. (1998). Jack Welch and the GE way. New York: McGraw-Hill. Teece, D. J. (1987). Profiting from technological innovation: Implications for integration, collaboration, licensing, and public policy. In: D. J. Teece (Ed.), The competitive challenge (pp. 185–219). New York: Harper & Row. Tichey, N. M., & Cohen, E. (1997). The leadership engine: How winning companies build leaders at every level. New York: Harper-Collins Publishers. Warner, J. B., Watts, R. L., & Wruck, K. H. (1988). Stock prices and top management changes. Journal of Financial Economics, 20, 461–492. Wasserman, N., Nohria, N., & Anand, B.N. (2001). When does leadership matter? The contingent opportunities view of CEO leadership. Strategy Unit Working Paper No. 02-04; Harvard Business School Working Paper No. 01-063. Weisbach, M. (1988). Outside directors and CEO turnover. Journal of Financial Economics, 20, 431–460. Whitehill, A. M. (1991). Japanese management: Tradition and transition. London: Routledge. Williamson, O. E. (1985). The economic institutions of capitalism. New York: Free Press.

This page intentionally left blank

176

A RESOURCE-BASED LENS ON VALUE CREATION, COMPETITIVE ADVANTAGE, AND MULTI-LEVEL ISSUES IN STRATEGIC MANAGEMENT RESEARCH Margaret Peteraf ABSTRACT Multi-level issues (Klein, Dansereau, & Hall (1994). Academy of Management Review, 195–229) are critical to both strategic management research and practice, and yet, we have few approaches for dealing with them both systemically and systematically. In this chapter, I take a resource-based approach to exploring multi-level linkages, suggesting that such an approach has wide applicability. A resource-based view (RBV) of competitive advantage and value creation illustrates the multi-level nature of these concepts and shows how the RBV is itself linked to the external market environment. The RBV also provides a way to link a variety of types of levels of analysis. These include different organizational levels of analysis, content and process linkages, and linkages across time.

Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 177–188 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04007-5

177

178

MARGARET PETERAF

INTRODUCTION As Drnevich and Shanley (this volume) argue compellingly, multi-level issues (Klein, Dansereau, & Hall, 1994) are critical to both strategic management research and practice, and yet, we have few approaches for dealing with them systemically and systematically. They suggest three such approaches that have considerable promise for linking levels of analysis: linkage by transaction, by managerial role, and by atmosphere. While they include the resource-based view (RBV) (Wernerfelt, 1984; Peteraf, 1993) as a type of transactional approach, I argue here that the RBV should be considered as a separate approach to providing multi-level linkages. Not only is it distinct from the transaction approach, but it goes well beyond this approach. In fact, as I suggest, the RBV has clear links to Drnevich and Shanley’s approaches based on managerial roles and on atmosphere as well. Like Drnevich and Shanley (this volume), I see two of the most central concepts in strategic management – competitive advantage and value creation – as inherently multi-level. By this I mean, as they do, that one cannot understand and evaluate these concepts without going beyond the focal ﬁrm level of analysis. A consideration of the focal ﬁrm in relation to other actors in the environment is essential. Like Drnevich and Shanley, I see a role for resources and capabilities in explaining both of these concepts. Similarly, I see a close connection between these two concepts and a beneﬁt for both theory and practice in understanding this linkage. Where I differ from Drnevich and Shanley is in the precise definition of these terms. In the next section, I offer a resource-based view (Wernerfelt, 1984; Peteraf, 1993) of competitive advantage and value creation that not only illustrates the multi-level nature of these concepts, but shows how the RBV is itself linked to the external market environment. Following that, I discuss the use of the RBV, more broadly, as a conceptual mechanism for linking various types of levels of analysis. These include different organizational levels of analysis, content and process linkages, and linkages across time. In the penultimate section, I show how the RBV connects to the three approaches to linkage suggested by Drnevich and Shanley (this volume). I conclude by noting some limitations to the range of application of the RBV.

THE CONCEPTS OF COMPETITIVE ADVANTAGE AND VALUE CREATION The term competitive advantage, while being one of the most commonly employed terms and most central concepts in strategic management, has no

A Resource-Based Lens on Value Creation

179

commonly accepted definition. Often, it is deﬁned in terms of superior ﬁnancial performance (Winter, 1995). Drnevich and Shanley (this volume, 14) take such an approach. They assert that, When a ﬁrm (or business unit within a multi-business ﬁrm) persistently earns more profit than the other ﬁrms with which it competes, the ﬁrm is viewed as having a competitive advantage in that market.

But as they make clear, this definition is not without problems. One problem is that profitability itself is not well deﬁned. Another is that the set of relevant competitors is not entirely clear. While this definition still has much to recommend it and is useful for certain applications, it is problematic from a resource-based perspective. One problem is that its speciﬁcation of ‘‘persistently’’ earning more profits confounds sustainable competitive advantage with the more basic concept of competitive advantage. Another is that it leads to tautological problems when used as part of the resource-based logic for sustainable advantage. Resolving the ﬁrst problem is easy. Decoupling the term competitive advantage from sustainable advantage simply allows us to consider the concept of competitive advantage apart from the issue of whether the advantage can be preserved. A competitive advantage may be sustainable or not. If it is an imitable competitive advantage, it is not sustainable and therefore likely to be short-lived (Peteraf, 1993; Winter, 1995). The second problem is resolved by deﬁning competitive advantage not in terms of relative profitability, but in terms of relative value created. This not only resolves the RBV-related tautology problem, but it ﬁnesses the question of how to deﬁne profitability. Moreover, it has the advantage of making the link between the concepts of competitive advantage and value creation even more explicit. In Peteraf and Barney (2003, p. 314), we deﬁne competitive advantage as follows: An enterprise has a competitive advantage if it is able to create more economic value than the marginal (breakeven) competitor in its product market.

By deﬁning competitive advantage in this fashion, we use the term to describe the relative performance of rivals in a given product market. This is consistent with the approach that Drnevich and Shanley (this volume) employ, but somewhat more specific. It clariﬁes which competitors are the relevant rivals for comparison’s sake (the breakeven ones). This reﬂects a resource-based concern with the potential for Ricardian rents. And while it does not offer specific guidance regarding the boundaries of the market, it

180

MARGARET PETERAF

may be used iteratively with alternative telescoping market definitions (Peteraf & Bergen, 2003). The precise meaning of this definition, of course, rests on an understanding of what it means to ‘‘create economic value.’’ Our definition of economic value corresponds exactly to Drnevich and Shanley’s (this volume, 21) expression ‘‘total value created.’’ We (Peteraf & Barney, 2003, p. 314) deﬁne economic value in the following way: The economic value created by an enterprise in the course of providing a good or service is the difference between the perceived beneﬁts gained by the purchasers of the good and the economic cost to the enterprise.

As Drnevich and Shanley make clear, this is equal to the sum of the consumer surplus and the producer surplus. It does not involve the price of the good. It captures the beneﬁts to society of producing net of the economic costs. Taken together, these two definitions (of competitive advantage and of economic value) provide a clear picture of what comprises a competitive advantage and how it may be achieved, in general terms. As Drnevich and Shanley emphasize, the concept of competitive advantage is naturally multilevel. It is a relative term, in which the benchmark is the breakeven rival. It therefore demands a comparison of the ﬁrm with other players in the industry environment. Similarly, the notion of value created calls for a multilevel view of the economic terrain, since it connects the ﬁrm with the customers and the demand environment. Taken together, these two definitions illustrate the linkage between value creation and competitive advantage that Drnevich and Shanley observe. But the linkage between these concepts as I deﬁne them is even tighter, since competitive advantage is deﬁned explicitly in terms of relative amounts of value created. And the fact that it is relative value created that matters implies a need for even greater attention to the linkage between levels of analysis. For relative value creation will depend upon the resources and capabilities of the rivals in the marketplace relative to those of the focal ﬁrm.

THE RBV AS A CONCEPTUAL MECHANISM FOR LINKING LEVELS OF ANALYSIS As a supplement to the three approaches suggested by Drnevich and Shanley (this volume), I offer a resource-based view of linkages across levels. First, note that a resource-based view of competitive advantage naturally

A Resource-Based Lens on Value Creation

181

links the ﬁrm to its market environment and the other players in that environment. As we make clear in Peteraf and Barney (2003), resource heterogeneity implies that some ﬁrms have resources that can generate more economic value than other ﬁrms. Relative economic value creation requires comparison across levels, from the ﬁrm and its internal capabilities to those of its rivals in the industry environment. The RBV provides a tool that can facilitate the comparison of ﬁrms with not only their direct rivals, but with substitute providers as well (Peteraf & Bergen, 2003). Similarly, the resource-based approach to economic value that we advocate calls for an assessment of customer needs, encouraging another crosslevel connection. Further, a resource-based consideration of the economic costs requires attention to players in the factor markets as well (Barney, 1986a). A full consideration of these various elements necessitates an examination of conditions over multiple levels throughout the entire vertical chain. The resource-based view, then, is one that spans levels of analysis by virtue of the perspective it takes and the kind of analysis it demands.

RBV and Organizational Levels of Analysis Not only does the RBV link the ﬁrm to the industry and to other players along the vertical chain, it links levels of analysis within the ﬁrm as well. What links these various levels of analysis is a common lens and the focus on the role of resources and capabilities. The resource-based view of competitive advantage and value creation, as described above, is concerned with analysis at the level of a single business or a single business unit, operating in a given market. The resource-based view of sustainable advantage (Dierickx & Cool, 1989; Barney, 1991; Peteraf, 1993) is similarly concerned with business-unit-level analysis. With only minor modiﬁcations, the RBV applies equally to corporate level analysis, providing a cogent explanation of corporate advantage (Peteraf, 1993; Collis & Montgomery, 1997). Just as value can be created from business level activities, so it can be created by resources, such as an umbrella brand, that are properly understood as corporate level resources. Just as a ﬁrm gains a competitive advantage by creating more value than its rivals in the marketplace, so corporate advantage comes from creating more value than other corporations in the market for corporate control. At the corporate level, the RBV has been used to understand not only the various types of diversiﬁcation strategies (Montgomery & Wernerfelt, 1988), but to predict the direction of ﬁrm growth and entry into new markets as well

182

MARGARET PETERAF

(Montgomery & Hariharan, 1991). Indeed, the roots of the RBV in Penrose’s (1959) work make this an obvious application. The RBV not only applies to different levels of analysis within an organization, it can be used as a lens to understand strategies that span organizations as well. Thus, it has been used to explain alliances (Eisenhardt & Schoonhoven, 1996) and interﬁrm networks (Lorenzoni & Lipparini, 1999). It has been used to explain geographic clusters of ﬁrms (Mathews, 2003) and strategic groups (Cool, Dierickx, & Martens, 1994). As before, what links these multiple levels of analysis is the central role played by resources and capabilities and the common lens.

RBV and Content/Process Linkages While the RBV applies to levels of analysis within the ﬁrm, it has been applied mostly to the content area of strategy research, reﬂecting a longstanding division between content-oriented and process-oriented research. This focus on content, however, has more to do with the path by which resource-based theory developed, and its ties to economic reasoning, than it does to any inherent limitation on its application. Indeed, as I have argued in Peteraf (2004), there are multiple opportunities for resource-based thinking to guide and inform process-oriented research. An example of this is the potential for linking the RBV to research in the Bower/Burgelman tradition on the resource allocation process, known as RAP (Bower, 1970; Burgelman, 1983). There are numerous points of intersections between these two research traditions that suggest the possibility for forging more explicit ties across multiple levels of analysis. Both, for example, focus centrally on resources and capabilities internal to the ﬁrm and their connection to strategic decision making. Both concern competitive forces, although at different levels of analysis: RBV with external competitive forces, RAP with internal competition. Both acknowledge the importance of tacit knowledge and have an afﬁnity to evolutionary models, although of different types: RBV with evolutionary economics (Nelson & Winter, 1982) and RAP with population ecology (Aldrich, 1979; Hannan & Freeman, 1984). Research on the resource allocation process has, for the most part, been focused only on issues internal to the ﬁrm, ignoring the relation of the ﬁrm to its external environment. Only recently have researchers begun to make the connection from the resource allocation process to players in the industry environment. Christensen and Bower (1996), for example, introduce

A Resource-Based Lens on Value Creation

183

the topic of how customers inﬂuence the resource allocation process, which Christensen (1997) extends in his widely cited book. Bringing a resourcebased view to the study of resource allocation processes can bridge these multiple levels of analysis further. The concern of the RBV with performance relative to rivals in the marketplace suggests a natural bridging mechanism. A question that remains unexplored is the question of how variation in resource allocation processes affects relative ﬁrm performance. Like other types of organizational routines, resource allocation processes may be valuable capabilities that help a ﬁrm gain a competitive edge, under some conditions. Whether and how such capabilities affect relative ﬁrm performance is an open question that the RBV can help answer. By tying the RAP model more directly to the performance implications of the processes described in the RAP ﬁeld studies, the RBV may provide a mechanism for connecting what until now have been separate spheres of analysis. One result of this may be to facilitate a better understanding of the normative implications of both the RAP and RBV models, thereby bridging another gap – that between research and practice.

RBV and Linkages Across Time Among the ways that the RBV aids in bridging multiple levels of analysis is in the linkage it provides for analysis across multiple dimensions of time. In its most basic form, the RBV provides a static equilibrium model that explains how a competitive advantage can arise and be sustained (Peteraf & Barney, 2003; Peteraf, 1993). This basic model provides the starting point for understanding competitive advantage in more dynamic contexts. The emerging work on the dynamic resource-based view make this clear (Helfat et al., forthcoming). Likewise, the basic conception of resources and capabilities as strategic assets (Amit & Schoemaker, 1993) serves as a starting point for considering higher-order capabilities (Collis, 1994). The connection from ordinary capabilities to dynamic capabilities (Teece, Pisano, & Schuen, 1997) as agents of change ﬂows naturally out of the basic resource-based model. While there are some who argue that dynamic capabilities is separable from the RBV, we take the perspective that it is best understood as an extension of the more basic framework (Helfat et al., forthcoming). The RBV serves as a conceptual mechanism for linking analysis at various points in time within the life of a ﬁrm. Its connection to evolutionary models

184

MARGARET PETERAF

is clear (see eg. Winter, 2003; Helfat & Raubitschek, 2000). It provides a basis for the view that history matters and that the resource endowment of a ﬁrm, when it is young, may shape its evolutionary trajectory for years to come (Helfat & Lieberman, 2002). It provides the basis for a conceptual understanding of the lifecycles of capabilities (Helfat & Peteraf, 2003). In illustrating how capabilities can outlive the ﬁrms in which they originally develop, the RBV suggests a mechanism for bridging multiple levels of analysis across generations of ﬁrms. While these examples are only a few of the many ways in which the RBV bridges multiple levels of time and space, they illustrate the general principle that the RBV provides the linking mechanism.

Links Between the RBV and Drnevich and Shanley’s Three Approaches While Drnevich and Shanley (this volume) fold the RBV into the category of linkage by transaction, we see the RBV as a far broader mechanism. Certainly, they are correct in asserting that the RBV embodies a transactional approach to linking levels. Market failures limiting the free ﬂow of exchange transactions is a part of the RBV story (Dierickx & Cool, 1989). But it is only a part. In my view, the more interesting part has to do with the resources and capabilities themselves, rather than the markets (or absence of markets) for their exchange. Resources serve to link levels of analysis between the ﬁrm and the market when there is an appropriate ﬁt between the environmental requirements and the potential of the resources to provide for those needs. They link levels of analysis within the ﬁrm when there is a ﬁt between the needs of the various business units and the ability of resources to meet those needs. That is the basis of true synergy. And they link levels of analysis across ﬁrms when there are commonalities or complementarities among the resources of a set of ﬁrms that spur cohesion among the ﬁrms in the set or drive competitive interaction. Thus, while I agree with Drnevich and Shanley’s characterization of the RBV as a transactional approach, I feel that this is a limited characterization. RBV is not only a transactional approach. The linking mechanism that it offers is part of a broader set. One indication of this is that the RBV can also be included as a part of the other two approaches to linkage that Drnevich and Shanley (this volume) provide. The ﬁrst of these is the linkage by managerial role. As Drnevich and Shanley explain, general managers directly link levels of analysis by virtue of their role in coordinating the various decisions of their ﬁrms. This ultimately bridges the levels of analyses between the ﬁrm and its environment, since the

A Resource-Based Lens on Value Creation

185

performance consequences of a lack of such coordination may be dire. In addition, it links levels within the ﬁrms, drawing together the divergent perspectives of lower and middle managers utilizing a variety of decision processes. The connection of this to the RBV is quite straightforward. As Castanias and Helfat (1991, 2001) have argued, managers may themselves be a critical resource of the ﬁrm. Their skills and capabilities inﬂuence not only the performance of the ﬁrm, but its entire developmental trajectory. As Adner and Helfat (2003) have argued, the dynamic capabilities of managers are important as well and have an important effect on shaping corporate strategy. And as Peteraf (2004) suggests, there is a potential for more RBVinspired work connecting the decision-making processes of managers at all levels within the ﬁrm to environmental outcomes. Note that even in this briefest of discussions of the RBV perspective on the managerial role, the cross connections among various levels of analysis (market level, ﬁrm level, corporate level, departmental level) are clear. Not only does the RBV offer the potential to span multiple levels of analysis within the ﬁrm and within the environment, it offers the potential to span multiple levels regarding process and time dimensions as well. Lastly, there is a connection between the RBV and Drnevich and Shanley’s third approach to linking levels – linkage by atmosphere. By atmosphere, Drnevich and Shanley (this volume) mean ‘‘a shared context of interaction’’ similar to Williamson’s (1975) use of the term. One obvious connection to the RBV is the close correspondence between atmosphere and culture, which are arguably synonymous. Barney’s (1986b) claim is that a ﬁrm’s culture is a resource and potentially one of the most valuable. As a socially complex, somewhat tacit resource, with path-dependent developmental properties, it is immune from ready imitation. Consequently, it may be a source not only of competitive advantage, but of sustainable advantage as well. If atmosphere is interpreted to have a broader meaning, then there may be yet another connection to the resource-based view of the ﬁrm. A shared context of interaction may refer to the organizational routines and processes by which the ﬁrm operates. And high-level organizational routines are capabilities, in resource-based parlance. Winter (2000, p. 983) deﬁnes an organizational capability as: A high level routine (or collection of routines) that, together with its implementing input ﬂows, confers upon the organization’s management a set of decision options for producing significant outputs of a particular type.

186

MARGARET PETERAF

As this definition suggests, organizational capabilities and routines may provide the context in which managerial decisions are made. In this way, they provide the kind of contextual linking mechanism that Drnevich and Shanley (this volume) have in mind.

CONCLUSIONS In sum, the RBV provides a way to draw the linkages across multiple levels of analysis. It connects the ﬁrm logically to the industry environment and other actors in that environment, such as rival ﬁrms. It links to levels of analysis up and down the vertical chain, connecting the ﬁrm with customers at one end and suppliers at the other. It draws together strategy at the business-unit level with strategy at the corporate level. It connects the concept of competitive advantage to the parallel concept of corporate advantage. In so doing, it ties the ﬁrm to other actors in the market for corporate control. It facilitates the commonalities across multiple modes of corporate strategy, from various types of diversiﬁcation, to alliances and networks of organizational actors. It spans the levels of analysis of the ﬁrm to that of the strategic group and the geographic cluster. In addition, it facilitates linking processes and activities within the ﬁrm at multiple levels, such as departments and divisions, with environmental outcomes and their normative implications. It provides a mechanism for linking levels that is separable from, but yet related to, the three approaches offered by Drnevich and Shanley (this volume). Despite what I view as the wide applicability and general nature of the RBV, I do not see it as an all-encompassing explanatory mechanism. There are clear areas to which it does not apply and others that may be better served by other types of lenses (Peteraf & Barney, 2003). It does not explain monopoly power, for example, or agency issues. It does not substitute for industry analysis, say, or transaction cost analysis. It cannot analyze interactions among ﬁrms in the manner of game theory. Other lenses sometimes substitute for and sometimes complement the RBV lens. What I suggest here is, simply, that the RBV can play a role in spanning levels of analysis that is remarkably broad and useful. As Drnevich and Shanley (this volume) acknowledge, there are advantages and disadvantages associated with each of the approaches to linking levels. My arguments here simply add one more such approach to the mix.

A Resource-Based Lens on Value Creation

187

REFERENCES Adner, R., & Helfat, C. (2003). Corporate effects and dynamic managerial capabilities. Strategic Management Journal, 24, 1011–1025. Aldrich, H. (1979). Organizations and environments. Englewood Cliffs, NJ: Prentice-Hall. Amit, R., & Schoemaker, P. (1993). Strategic assets and organizational rent. Strategic Management Journal, 14, 33–46. Barney, J. B. (1986a). Strategic factor markets: Expectations, luck, and business strategy. Management Science, 23, 1231–1241. Barney, J. B. (1986b). Organizational culture: Can it be a source of sustained competitive advantage? Academy of Management Review, 11, 656–665. Barney, J. B. (1991). Firm resources and sustained competitive advantage. Journal of Management, 17, 99–120. Bower, J. (1970). Managing the resource allocation process: A study of corporate planning and investment. Boston, MA: Harvard Business School Press. Burgelman, R. (1983). A process model of internal corporate venturing in the diversiﬁed major ﬁrm. Administrative Science Quarterly, 28, 223–244. Castanias, R., & Helfat, C. (1991). Managerial resources and rents. Journal of Management, 17, 155–171. Castanias, R., & Helfat, C. (2001). The managerial rents model: Theory and empirical analysis. Journal of Management, 27, 661–678. Collis, D. J. (1994). Research note: How valuable are organizational capabilities? Strategic Management Journal, 15, 143–152. Collis, D. J., & Montgomery, C. A. (1997). Corporate strategy: Resources and the scope of the firm. Boston, MA: Irwin, McGraw-Hill. Cool, K., Dierickx, I., & Martens, R. (1994). Asset stocks, strategic groups and rivalry. In: H. Daems & H. Thomas (Eds), Strategic groups, strategic moves and performance (pp. 219–234). Tarrytown, NY: Elsevier Science. Christensen, C. (1997). The innovator’s dilemma: When new technologies cause great firms to fail. Boston, MA: Harvard Business School Press. Christensen, C., & Bower, J. (1996). Customer power, strategic investment, and the failure of leading ﬁrms. Strategic Management Journal, 17, 197–218. Dierickx, I., & Cool, K. (1989). Asset stock accumulation and sustainability of competitive advantage. Management Science, 35, 1504–1511. Eisenhardt, K., & Schoonhoven, C. (1996). Resource-based view of strategic alliance formation: Strategic and social effects in entrepreneurial ﬁrms. Organization Science, 7, 136–150. Hannan, M., & Freeman, J. (1984). Organizational ecology. Cambridge, MA: Harvard University Press. Helfat, C., Finkelstein, S., Mitchell, W., Peteraf, M., Singh, H., Teece, D., & Winter, S. (Forthcoming) Dynamic capabilities and resource-based change. Oxford, UK: Blackwell. Helfat, C. E., & Lieberman, M. B. (2002). The birth of capabilities: Market entry and the importance of pre-history. Industrial and Corporate Change, 11, 725–760. Helfat, C., & Peteraf, M. (2003). The dynamic resource-based view: Capability lifecycles. Strategic Management Journal, 24, 997–1010. Helfat, C. E., & Raubitschek, R. S. (2000). Product sequencing: Co-evolution of knowledge, capabilities and products. Strategic Management Journal, 21, 961–980.

188

MARGARET PETERAF

Klein, K., Dansereau, F., & Hall, R. (1994). Levels issues in theory development, data collection and analysis. Academy of Management Review, 20, 195–229. Lorenzoni, G., & Lipparini, A. (1999). The leveraging of interﬁrm relationships as a distinctive organizational capability: A longitudinal study. Strategic Management Journal, 20, 317–338. Mathews, J. A. (2003). Competitive dynamics and economic learning: An extended resourcebased view. Industrial and Corporate Change, 12, 115–145. Montgomery, C., & Hariharan, S. (1991). Diversiﬁed expansion by large established ﬁrms. Journal of Economic Behavior and Organization, 15, 71–89. Montgomery, C. A., & Wernerfelt, B. (1988). Diversiﬁcation, ricardian rents, and tobin’s q. RAND Journal of Economics, 19, 623–632. Nelson, R. R., & Winter, S. G. (1982). An evolutionary theory of economic change. Cambridge, MA: Harvard University Press. Penrose, E. (1959). The theory of growth of the firm. New York: Wiley. Peteraf, M. (1993). The cornerstones of competitive advantage. Strategic Management Journal, 14, 179–191. Peteraf, M. (2004). Research complementarities: A resource-based view of the resource allocation process model (and vice versa). In: J. Bower & C. Gilbert (Eds), Strategy as resource allocation. Oxford: Oxford University Press. Peteraf, M., & Barney, J. (2003). Unraveling the resource-based tangle. Managerial and Decision Economics, 24, 309–323. Peteraf, M., & Bergen, M. (2003). Scanning dynamic competitive landscapes: A market-based and resource-based framework. Strategic Management Journal, 24, 1027–1041. Teece, D., Pisano, G., & Schuen, A. (1997). Dynamic capabilities and strategic management. Strategic Management Journal, 18, 509–533. Wernerfelt, B. (1984). A resource based view of the ﬁrm. Strategic Management Journal, 5, 171–180. Williamson, O. (1975). Markets and hierarchies. New York, NY: Free Press. Winter, S. (1995). Four R’s of profitability: Rents, resources, routines, and replication. In: C. A. Montgomery (Ed.), Resource-based and evolutionary theories of the firm (pp. 147–158). Boston, MA: Kluwer. Winter, S. G. (2000). The satisfying principle in capability learning. Strategic Management Journal, 21, 981–996. Winter, S. (2003). Understanding dynamic capabilities. Strategic Management Journal, 24, 991–995.

MULTI-LEVEL ISSUES FOR STRATEGIC MANAGEMENT RESEARCH: FURTHER REFLECTIONS Paul Drnevich and Mark Shanley ABSTRACT In this reply to the articles about ‘‘Multi-level Issues for Strategic Management Research: Implications for Creating Value and Competitive Advantage’’ (Drnevich and Shanley, this volume), we consider and applaud the applications of our ideas presented by Mackey and Barney (this volume) and Peteraf (this volume). We also note three general issues that arise from considering the two articles together: (1) the complexity of multi-level constructs of management; (2) the importance of strategic decision processes (and process approaches to strategy) in a multi-level approach; and (3) the need for dynamic approaches to bridge temporal periods in crafting strategic theories.

INTRODUCTION It is tremendously heartening to have such informative and insightful commentaries as those provided by Mackey and Barney (this volume) and Peteraf Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 189–194 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04008-7

189

190

PAUL DRNEVICH AND MARK SHANLEY

(this volume). It is a great compliment that scholars such as these found our ideas useful and stimulating of new research directions. As these chapters are interesting, carefully done, and no doubt have the potential to extend and contribute to the conversation of multi-level issues in strategic management research, we have few substantive axes to grind and hope to see these efforts further develop. To this end, we offer some suggestions to further the ongoing discussion of multi-level issues in strategy research that we tried to start in our chapter. As such, this response offers some reﬂections on each chapter, followed by a discussion of the implications of both for multi-level research, process studies, and the role of general managers in linking levels.

STRATEGIC LEADERSHIP Mackey and Barney (this volume) apply our arguments to the strategic leadership literature. This is an area rich with opportunities to make strong contributions. Historically, strategic leadership has been one of the most popular, and yet most frustrating areas of strategic management. Further, this is also the area in which ‘‘never have so many laboured so long to say so little’’ (Bennis & Nanus, 1985). The need for some new insights is quite clear. Mackey and Barney begin by situating the general manager within the resource-based view of the ﬁrm. They then consider the conditions under which strategic leadership can be a source of competitive advantage. Appreciating the potential for this advantage, however, also requires understanding the dynamics by which labor markets allocate managerial talent across ﬁrms. They conclude by looking at the interactions among these levels, to shed light on the conditions under which managers will be able to appropriate any rents that their talents might generate. Overall, their article provides a good illustration of the importance of how the managerial linking function may be used to bridge the resource and market levels and inﬂuence value creation and competitive advantage. Further, they illustrate the effective use of multiple levels with a clear demonstration of the distinct logics at work on each level and offer a good example of why a multi-level approach is both necessary and valuable. In their analysis, Mackey and Barney make use of the dynamic interaction between actors and the context in which they must perform. For example, managerial discretion can be understood as the interaction between the talent of managers and the opportunities for discretion offered by the ﬁrm. While managers may try to appropriate the value they generate for ﬁrms, their ability to do so also will be constrained by their environment. At a different

Multi-Level Issues for Strategic Management Research: Further Reflections

191

level of analysis, however, the ability of ﬁrms to capitalize on the potential for rents from managerial talent will depend on how those talents are allocated across ﬁrms by labor markets. Even if talent is available, ﬁrms will be constrained by the complexity of developing and deploying that talent. A particularly interesting issue in their article concerns substitutability. Specifically, how planning processes may constitute substitute resources for managerial talent. Beyond the question of whether planning processes are substitutes or complements to managerial talent is the broader one of whether ‘‘management’’ is best seen as a resource or a capability. There may well be varied ways to combine people, structures, and processes in a managerial division of labor so as to obtain comparable results for a ﬁrm. Further, the actor exercising the general management function in a ﬁrm may be an individual, a top management team, or a series of management teams linked together in an interactive organization design. Finally, a related question arising from Mackey and Barney concerns how the content of a business unit’s strategy changes as a function of how its managers are organized. The strategic decisions made by a ﬁrm will be associated with the managerial resources brought to bear on the decision, along with how those resources are organized and interact. For example, the incentive schemes in place for managers will inﬂuence how they deﬁne their markets, which advantages they choose to pursue, and the effort expended in pursuing them. Incentive schemes may also rule out certain types of arrangements (i.e. cross unit collaboration in a tight profit center arrangement) and the decisions that such arrangements are likely to produce.

A MULTI-LEVEL RESOURCE-BASED VIEW Peteraf (this volume) responds to the arguments in our chapter by reﬂecting upon the potentially broad applicability of the resource-based view (RBV) of the ﬁrm. She shows how the RBV has afﬁnities to all three linking approaches that we present. She also demonstrates how the RBV provides ways to link a variety of levels, including different organizational levels, content versus process distinctions, and temporal linkages. Peteraf’s article provides an insightful articulation of a multi-level view of the RBV that is consistent with recent work (Helfat et al., in press), which is developing a dynamic view of the RBV. The richness of the RBV had also occurred to us, for example in comparisons of Penrose’s (1959) work with that of later researchers (Rugman & Verbeke, 2002; Kor & Mahoney, 2004; Lockett & Thompson, 2004). Peteraf, however, provides a powerful

192

PAUL DRNEVICH AND MARK SHANLEY

statement of the potential of the perspective as a basis for improving theory. Some of the most interesting parts of the article concern her suggestions for improving RBV theorizing, for example in the use of the marginal or breakeven competitor as the benchmark basis for assessing value creation. This addition, while seemingly minor, both clariﬁes and links ideas of value creation and competitive advantage in useful ways. Peteraf’s observations on the potential for the RBV to link strategy process and content, as well as its potential to link temporal periods, are also insightful and will receive further comment in the concluding section of this response. Our differences with Peteraf on fundamentals are minor. We agree with her refocusing of competitive advantage on value creation, although we would add that there are problems involved with developing this approach to the value construct — just as there are problems involved with focusing on profit. While we might envisage somewhat stricter limits to the range of RBV applicability, we agree with Peteraf regarding the richness, dynamic nature, and potential for extension and enhancement of the RBV. One of the central premises of our chapter was that a full consideration of multiple level implications should cause us to rethink and potentially revise and extend some of our current and accepted notions of strategic management theory. As such, Peteraf’s article offers a strong contribution toward this challenge by illustrating how the RBV may be extended, and in places reconceptualized, to link multiple levels.

COMMON ISSUES Looking at the two articles together raises three sets of issues that receive at least some implicit consideration. The ﬁrst concerns the dual role of management as a resource/capability on the one hand and an integrator on the other. The second concerns the importance of strategic decision processes and ﬁrm structures as vehicles for linking levels both in theory and practice. Finally, the dynamic nature of large ﬁrms and their environments requires a recognition that the resources and capabilities of ﬁrms will change over time. This occurs along with changes in the business environment within which the ﬁrm operates. Collectively, these changes produce opportunities for value to be created. As such, strategic theories need to link temporal levels and thus incorporate ideas of evolution and learning. A ﬁrst issue concerns the complexity of management. Managerial leadership is a central construct for most aspects of strategic management and yet remains highly complex and in need of theoretical development. Management

Multi-Level Issues for Strategic Management Research: Further Reflections

193

talent is clearly a resource and a potential source of competitive advantage for ﬁrms, following RBV criteria. Skill at managing, however, is a capability that can arise from a variety of different ways of combining managers, processes, and structures with business environments. Which of the various resource and capability combinations a ﬁrm chooses will depend upon the choices of managers, who perceive an environment and then deploy ﬁrm resources in accordance with the requirements of that perceived environment. Coming to grips with a resource such as management that is at once part of the entire ﬁrm’s resource conﬁguration, and yet determines that conﬁguration through strategic decisions, is a major task for theory development. A second general issue arising from the articles is the centrality of strategic decision processes to management efforts to bridge levels in practice. This also suggests that improved theories of strategic process will prove critical for a truly multi-level approach to strategic management. Without a deep understanding of the process, what it is that managers do will remain a black box or the subject of anecdote and retrospection, while bridging levels via managerial roles will remain a redefinition of the problem in the form of a solution. Mackey and Barney imply this, while Peteraf is more explicit. Her call to revisit the work on resource-allocation processes by Bower (1970) and Burgelman (1983) is one with which we heartily agree. A third general issue suggested by the articles, concerns the need for dynamic multi-level approaches to strategic management. While it is nice to mention ideas of sustainable advantage and long-lived strategic resources, much in the ﬁrm’s environment and in its internal resource conﬁguration is subject to continual change. While resources may persist for a ﬁrm, their value may change, positively or negatively, as a result of exogenous environmental shocks. A ﬁrm’s success with a particular resource conﬁguration will prompt subsequent growth and complementary investments that subsequently constrain the ﬁrm when its environment changes, a situation explored in depth by Christensen (2000). A multi-level approach to strategic management must address the trajectories by which resources and capabilities change, whether due to internal dynamics or in response to environmental change. This reinforces our thinking in this chapter regarding the need for more focused and longitudinal research designs.

CONCLUSION We conclude by reafﬁrming our belief that the phenomena of interest to strategic management are multi-level in nature and that research to date has

194

PAUL DRNEVICH AND MARK SHANLEY

only begun to address what this means. In our chapter, we have tried to provide some clariﬁcation regarding multi-level issues in strategy and the ways in which strategic management theories have attempted to deal with those issues. It is our hope that researchers can use these ideas, along with those contained in the excellent chapters on our chapter by Mackey and Barney and by Peteraf to begin crafting more interesting and valuable multi-level strategy research.

ACKNOWLEDGMENTS We are grateful to Alison Mackey, Jay Barney, and Margaret Peteraf for their thoughtful and constructive commentaries on our work. We also thank the editors, Fred Dansereau and Francis Yammarino, for providing us with the opportunity to develop these ideas in much greater depth and detail than most journals would allow.

REFERENCES Bennis, W., & Nanus, B. (1985). Leaders: The strategies for taking charge. New York: Harper & Row. Bower, J. (1970). Managing the resource allocation process: A study of corporate planning and investment. Boston, MA: Harvard Business School Press. Burgelman, R. (1983). A process model of internal corporate venturing in the diversiﬁed major ﬁrm. Administrative Science Quarterly, 28, 223–244. Christensen, C. (2000). The innovator’s dilemma. New York: Harper Business. Helfat, C., Finkelstein, S., Mitchell, W., Peteraf, M., Singh, H., Teece, D., & Winter, S. (in press). Dynamic capabilities and resource-based change. Oxford, UK: Blackwell. Kor, Y., & Mahoney, J. (2004). Edith Penrose’s (1959) contributions to the resource-based view of strategic management. Journal of Management Studies, 41, 1. Lockett, A., & Thompson, S. (2004). Edith Penrose’s contributions to the resource-based view: An alternative perspective. Journal of Management Studies, 41, 1. Penrose, E. (1959). The theory of the growth of the firm. New York: Wiley.

PART III: UPPER ECHELONS

195

This page intentionally left blank

196

A MULTI-LEVEL ANALYSIS OF THE UPPER-ECHELONS MODEL Albert A. Cannella, Jr. and Tim R. Holcomb ABSTRACT The upper-echelons model of Hambrick and Mason ((1984). Academy of Management Review, 9, 193–206) launched a new area of research and provided the first overall theoretical framework for use in understanding how the experiences, backgrounds, and values of senior executives in organizations can influence the decisions that they make. The model is typically assumed to be what Rousseau ((1985). In: B. M. Staw, & L. L. Cumming (Eds), Research in organizational behavior (Vol. 7, pp. 1–37). Greenwich, CT: JAI Press) calls ‘‘multi-level,’’ as it describes how both individuals and top management teams (TMTs) make decisions in line with their preferences, biases, and values; the same model is applicable to both individuals and groups. However, the levels issues in the model have never been subjected to rigorous analysis. This chapter juxtaposes levels concepts and theories on the upper-echelons model, in an effort to highlight its strengths as well as its weaknesses. While the majority of researchers use the model to describe team-level decision making, the analysis presented here reveals that the model is inherently individuallevel in focus, and several important limitations must be overcome before the model will provide a full explanation of team-level decision making.

Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 197–237 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04009-9

197

198

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

INTRODUCTION Since 1980, the study of strategic leadership and top management teams (TMTs) has burgeoned (Boal & Hooijberg, 2001; Finkelstein & Hambrick, 1996). Much of this research draws from what is known as the ‘‘upperechelons’’ perspective, ﬁrst proposed by Hambrick and Mason (1984). In this view, organizations are perceived to be importantly inﬂuenced by the values, preferences, and experiences of their top managers. In essence, senior executives (the upper echelons) make decisions and take actions that are in accord with their personal preferences and biases. Because of the decisions they make and the strategies they develop, their organizations come to reﬂect their personalities, values, and beliefs. The upper-echelons approach builds on the premises of earlier strategic choice literature (Child, 1972), suggesting that managers lead their organizations by making strategic choices. The preferences, biases, and personalities of top managers manifest themselves at several stages of the decision process, from problem identiﬁcation to alternative generation to alternative selection. However, while much empirical research has been conducted in support of upper-echelons theory (see Finkelstein & Hambrick, 1996), we believe that the time has come for a careful analysis of the theory itself in light of a growing body of literature examining levels issues in organizational research. Organizations are rife with levels issues, and the explication of these issues remains a source of ongoing debate in the literature (House, Rousseau, & Thomas-Hunt, 1995; Klein, Dansereau, & Hall, 1994; Rousseau, 1985). Issues of level enter into the researcher’s choice of the unit of measurement (e.g., individual, group, organization) and selection of analytical methodology. Theories that do not adequately consider the role of levels in organizational phenomena risk potential biases that may confound interpretation of research results. Levels, in the context of multi-level theory and research, reﬂect the relations within and between organized systems; linkages between levels may exist that will contribute to a better understanding in the speciﬁcation of theory. For example, the CEO of any organization is its ﬁgurehead and presumably its most powerful leader. The upper-echelons model, however, argues that the TMT, under the leadership of the CEO, is where strategic decision making occurs (Hambrick & Mason, 1984). Therefore, researchers are immediately confronted with how the independent decision-making aspects of individual group members come together to form a unitary strategic decision. Once a decision is made, it must be implemented, in a process that

A Multi-Level Analysis of the Upper-Echelons Model

199

frequently involves large numbers of individuals at all hierarchical levels in the organization (Hambrick & Cannella, 1989). If we wish to track a strategic decision through to its performance implications, we must specify how a single decision leads to coordinated action among a relatively decentralized set of individuals, and then how those somewhat independent and perhaps even disparate actions result in a singular organization-level outcome (Klein & House, 1995; Waldman, Ramirez, House, & Puranam, 2001). In this chapter, we highlight theoretical issues arising from the misspeciﬁcation of theory across levels and consider several models designed to facilitate the study of levels-related phenomena in organizations. We believe that the upper-echelons model may offer a robust platform for dealing with levels issues in strategic decision making. At the same time, we suggest that much previous research on the upper-echelons model has failed to realistically consider the inﬂuence of levels issues, which are likely to affect strategic decision making. Another levels issue also should be considered with respect to the upper-echelons model – that of decision implementation – but we leave that issue to future study. We begin with a brief overview of Hambrick and Mason’s (1984) upperechelons theory and discuss some of the contributions that other authors have made since the original model was published. The intent here is not to provide an exhaustive review of the literature, but rather to highlight some basic assumptions that underlie the model and then note some commonalities in the ways the model has been formulated for testing. We then overview levels theory and highlight its key constructs and models. Finally, we juxtapose levels theory on the upper-echelons model to draw attention to some possible shortcomings and provide fresh insights for researchers interested in how individuals inﬂuence organization-level strategy and outcomes. Where possible, we provide suggestions for upper-echelons researchers who desire to add levels issues to their studies and for others who seek a more ﬁne-grained approach than the upper-echelons model, in its current conﬁguration, provides.

THE UPPER-ECHELONS MODEL The upper-echelons model is shown schematically in Fig. 1. This model follows closely the original perspective introduced by Hambrick and Mason (1984) and reviewed in detail by Finkelstein and Hambrick (1996). The essence of the model is that a decision maker’s unique ‘‘givens’’ – knowledge, values, biases, familiarities, and preferences – importantly inﬂuence

Organizational outcomes – both strategies and performance (effectiveness) – are viewed as reflections of the values and cognitive bases of powerful actors in the organization.

Executive Orientation Psychological Factors •Values •Cognitive Model •Other Personality Factors

Filtering Process

Limited Field of Vision

Selective Perception

Strategic Choices and

Organizational Performance

Interpretation Executive Behaviors

Observable Experiences • Age or tenure • Formal education • Functional Background • Other Factors

SOURCE: Adapted from Hambrick and Mason (1984) and Finkelstein and Hambrick (1996)

Fig. 1.

Classical Upper-Echelons Perspective: Strategic Choice Under Bounded Rationality. Source: Adapted from Hambrick and Mason (1984) and Finkelstein and Hambrick (1996).

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

Construed Reality

Strategic Situation (all potential environmental and organizational stimuli)

200

Classical UE Emphasis on either micro or macro research, i.e. the rational man or the dominant coalition View Current premise:

A Multi-Level Analysis of the Upper-Echelons Model

201

decision making. The model draws from the assumption of bounded rationality – the notion that there are far too many complexities in most strategic situations for complete rationality to exist, so decision makers must work within the bounds of their own intellects. The model explains how bounded rationality is manifested in decision situations and, therefore, how different executives can examine the same strategic situation and come to different conclusions about how to respond. Drawing from the Carnegie School approach to decision theory (Cyert & March, 1963; March & Simon, 1958), the upper-echelons model assumes that behavioral factors, rather than rational calculation, shape strategic decision making (see Fig. 1). In this way, complex choices are strongly inﬂuenced by the personal values and biases of the decision maker. The left side of Fig. 1 depicts the strategic situation, consisting of innumerable stimuli from both inside and outside the organization. The decision maker confronts these stimuli in decision situations. The center of the ﬁgure outlines the decision processes – the ﬁltering, interpretation, and construed reality that lead ultimately to strategic choice. On the right-hand side of the ﬁgure are the organizational outcomes that arise from the decisions made. Decision-maker demographics such as age, tenure, educational background, and functional background ﬁgure prominently in the upper-echelons model. Note, however, that the model does not propose that demographics cause decision outcomes or even that demographics are directly involved in any decision situation. Rather, demographics are used as measures of unobservable constructs such as values, cognitive models, personality factors such as risk-taking propensity or attitudes toward change, and other psychological factors that may inﬂuence decision behavior (Hambrick & Mason, 1984). The heart of the model begins with the ﬁltering process, describing how the decision maker’s psychological makeup directs attention toward particular stimuli and away from others. In effect, decision-maker ‘‘givens’’ (e.g., knowledge, preferences, values, biases) limit the ﬁeld of vision, making it more likely that executives will notice issues in the strategic situation that they personally care about, are familiar with, and have ready solutions for. Conversely, they are less likely to notice those issues that they do not understand, are not familiar with, and have not noticed in the past. Because decision makers are confronted with far too many stimuli to comprehend, each decision maker’s focus of attention will be strongly inﬂuenced by his or her psychological makeup, functional conditioning, and social network (Chattopadhyay, Glick, Miller, & Huber, 1999; Ocasio, 1997; Simon, 1945).

202

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

Of those stimuli that make it into the decision maker’s ﬁeld of vision, only part will actually receive direct attention. This is commonly referred to as ‘‘selective perception,’’ and is what Starbuck and Milliken (1988) call ‘‘noticing.’’ Finkelstein and Hambrick (1996) illustrate this concept with an example of a written report, noting that different managers will notice different aspects of the report’s information. Some of it will garner direct attention, some of it will be noticed only unconsciously, and much of it will not be noticed at all. Additionally, different managers will notice, pay attention to, and skip different parts of the report. In the interpretation portion of the ﬁltering process (illustrated in Fig. 1), those factors that the decision maker has directly noticed are processed further, and a meaning is assigned to them. This portion of the model has received a significant amount of study. For example, Dutton and Jackson (1987), drawing upon Staw, Sandelands, and Dutton’s (1981) threat-rigidity hypothesis, argue that managers are likely to interpret stimuli as either opportunities or threats; this classiﬁcation of stimuli inﬂuences both subsequent cognitive processing of the stimuli and responses to them. The interpretations become part of the decision-maker’s sense-making process (Starbuck & Milliken, 1988; Weick, 1995) and are absorbed into the person’s worldview (i.e., construed reality). In this way, decisions are strongly inﬂuenced by the prior experiences of the decision maker. Perhaps the most widely cited experiment on this part of the upper-echelons model is a study by Dearborn and Simon (1958) in which subjects (executives) were exposed to a very long and complex business case and asked to identify the key issues in the case. Executives’ interpretations of the issues tended to follow their functional expertise, with manufacturing executives tending to identify manufacturing problems, marketing executives tending to identify marketing problems, and so forth. Much later work has raised important questions about this study (e.g., Walsh, 1988, 1995), but the notion that a person’s psychological makeup will importantly inﬂuence his or her perceptions and interpretations has not been successfully challenged with an alternative theory. Many strategic situations are what Mischel (1977) calls ‘‘weak situations,’’ in that they are arguably open to a variety of interpretations. In these situations, the psychological makeup of decision makers can clearly be critical to the ultimate choices made. Notably, the ability to frame a strategic decision gives added importance to strategic leaders in the upper echelons of organizations (Fairhurst & Sarr, 1996; Hambrick & Cannella, 1989). Framing means that leaders in the upper reaches of the organization have already noticed, and made sense of, some issue in their ﬁeld of vision. They then direct

A Multi-Level Analysis of the Upper-Echelons Model

203

the attention of others (typically lower in the hierarchy) toward the issue, but also strongly inﬂuence (or perhaps even control) subsequent interpretations of it. This framing process can also go the other way, with those lower in the hierarchy framing some decisions of the topmost executives (Miller, 1991). This possibility opens the upper-echelons view to a variety of issues from a multi-level perspective, because control over the information that decision makers (at any level) see and the ability to ‘‘spin’’ or pre-interpret that information may be very inﬂuential in making the ﬁnal decision (Dutton, Frost, Worline, Lilius, & Kanov, 2002; Dutton & Jackson, 1987).

MULTI-LEVEL THEORY BUILDING Group phenomena inﬂuencing upper-echelons activity within organizations reﬂect properties of dynamic systems, with critical antecedents, processes, and outcomes conceptualized and measured at multiple levels of organizational analysis. Because researchers are beginning to realize that organizational phenomena are inherently multi-level, organizational studies are increasingly adopting a multi-level approach. Over the past two decades, several inﬂuential frameworks for multi-level research have emerged from the convergence of micro- and macro-level theories and from the formation of levels typologies to explain relations between individuals, dyads, work groups, and organizations (House et al., 1995; Klein et al., 1994; Rousseau, 1985; Sutton & Staw, 1995). Several authors have noted the importance of specifying the level of analysis at which constructs occur within a theoretical framework (Dansereau, Alutto, & Yammarino, 1984; Klein et al., 1994). For example, Rousseau (1985) noted how the issue of levels enters into the researcher’s choice of the unit and level of measurement and analysis (e.g., the individual, the dyad, the group, the department, or the organization). Consideration of multiple levels of analysis in research methods acknowledges the inherent predictive complexities of interactive inﬂuences and of hierarchical relationships at work across levels within organizations. The level of theory, the level of measurement, and the level of statistical analysis must all be congruent (Klein et al., 1994). Furthermore, the level of theory should effectively describe the target – specifically, individuals, dyads, teams, organizations, ﬁrms, or industry segments – that a researcher intends the theoretical framework to explain (Klein, Tosi, & Cannella, 1999). Thus, ﬁrst and foremost, the researcher must choose and justify a level of analysis for study.

204

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

Levels Issues in Theory Building Levels issues are deﬁned by the fact that the researcher must somehow consolidate units of analysis at one level to reﬂect a phenomenon at another level. For example, strategic decisions are deﬁned as those that affect the entire organization, and that are made in an effort to protect and enhance the health of the entire organization (Child, 1972). Reﬂecting this definition, the level of analysis for strategic decisions (under most approaches) is the organization. The upper-echelons model does not pose levels issues for decision making as long as only one person (e.g., the CEO) is involved, and that person has complete authority and responsibility for the decisions made. When more than one person is involved, however, the researcher must specify how these individuals come together to produce a single (group-level) decision. Toward that end, we describe several characterizations of the individuals that make up groups and consider how levels issues can arise in a TMT context. Individuality implies that there is virtually no ‘‘group-ness’’ with respect to the construct under consideration. With this characterization, the effect of group membership is immaterial. Klein et al. (1994) specify this level of theory as the independent level. Variation in the measurement of constructs is conceptualized simply as between-individuals variation. Hambrick (1994), for example, questions how well-integrated groups of top managers are and suggests that some TMTs may have little ‘‘team-ness’’ to them. When the individuals who constitute the group are characterized by individuality, the group has little or no inﬂuence on its members, and they are considered to be more or less independent. Likewise, the distinction of within-group and between-groups variation is also inconsequential (Dansereau et al., 1984). Homogeneity implies that group members are identical, or nearly identical, with respect to the construct examined. Under the assumption of homogeneity, levels issues are handled through averages or variances. Put differently, the assumption of homogeneity leads to the conclusion that the average of group member values on a particular measure is an accurate characterization of the group level. Highly homogeneous teams, for example, might be assumed to act much like a single person would if that person were characterized by the team’s average of members’ values on a key set of measures. Heterogeneity implies that the individuals in the group differ in important ways from one another, and especially relative to one another. A comparative process is at the center of individual-within-group constructs. Under

A Multi-Level Analysis of the Upper-Echelons Model

205

the heterogeneity assumption, individuals might make up subgroups that are internally homogeneous, but heterogeneous with respect to the other subgroups. For example, a vice president of marketing, who is a member of the TMT, is also in charge of his or her own department and is involved with direct reports and their subordinates. As a result, his or her views, values, and beliefs are likely to be inﬂuenced by participation in this subgroup (e.g., Waldman & Yammarino, 1999). In turn, these cognitive perceptions are assimilated by the executive and ultimately carried back into interactions with the TMT. Klein et al. (1994) describe this multi-level model as a crosslevel model, whereby causal relationships are formed between constructs at different levels. This consequence of hierarchical structures is significant because it can introduce tension and conﬂict into within-group interactions (Hambrick, 1994). Theories drawing on the heterogeneity assumption often predict that the effects of an independent variable on a dependent variable depend on the context (Klein et al., 1994). Heterogeneity could mean that any characterization of the group must consider how each individual differs from, or is similar to, every other member of the group. Although group members are considered to vary with respect to the theory’s construct, the group itself is deemed to be a meaningful entity. Mixed-effect models are theoretical models in which a single construct or intervention typically has effects at different levels of analysis (Klein et al., 1994). For example, Waldman and Yammarino (1999) present a model of charismatic leadership by CEOs in organizations that highlights the effect of social distance between leaders and groups within the organization on organizational cohesion, effort, and performance. In their model, they draw a series of distinctions between close versus distant leadership, and they examine the effect of CEO leadership behavior on TMT cohesion and effort and on intergroup cohesion and effort at lower hierarchical echelons within the ﬁrm. Their framework provides further insights into the complexities of these relationships and highlights the fact that the implications of the CEO for those close at hand may differ importantly from implications for those more distant. Temporal effects are described by Dansereau, Yammarino, and Kohles (1999). These authors describe how the fundamental characterizations of individuals with the group (individuality, homogeneity, heterogeneity) can change over time. It is well known that groups can shift from collections of disparate individuals (when the group is ﬁrst formed) to highly homogeneous teams after they have worked together for some time. We will return to this notion in our application of levels theory to the upperechelons model, discussing both the seasons and cycles of a CEO’s tenure

206

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

(Hambrick & Fukutomi, 1991), and the implications of CEO succession processes for TMT functioning (Shen & Cannella, 2002a, b).

LEVELS ISSUES IN UPPER-ECHELONS THEORY In the sections that follow, we consider levels issues related to the upperechelons model. This discussion is long and perhaps somewhat difﬁcult to follow, as some issues are relatively disjointed and complex. We have summarized the issues in Table 1, and reference to the table should help the reader follow our overall logic.

What is the Appropriate Level? First and foremost, when using a levels lens, the researcher must specify and theoretically justify what level is appropriate. Strategic management is always ultimately about overall organizational performance (Schendel & Hofer, 1979), but the upper-echelons approach is fundamentally about the decision-making behavior of executives, which is assumed to affect overall performance. The level of analysis for the upper-echelons approach tends to be either the CEO (e.g., Haveman, 1993; Miller & Toulouse, 1986; Yukl, 1989) or the TMT (e.g., Amason, 1996; Wiersema & Bantel, 1992), although some exceptions exist (Cannella & Shen, 2001; Reutzel & Cannella, 2004). Hambrick and Mason (1984) argue forcefully for the TMT as the key level of analysis, as noted earlier. In fact, most upper-echelons researchers agree with that conclusion (Hambrick, Finkelstein, & Mooney, 2003). Finkelstein and Hambrick (1996) offer four reasons why groups of top managers (i.e., the TMT) are likely to be a better level of analysis for study of decision making than individual executives: 1. Organizations are aggregations of individuals, each of whom has particular (and unique) goals, values, objectives, and preferences; the goals of individuals, even when they are part of a cohesive team, are frequently in conﬂict (Cyert & March, 1963; Weick, 1979). 2. Most descriptions of decision-making processes within organizations describe the activities of groups of individuals, not the activities of single individuals (e.g., Mintzberg, Raisinghani, & Theoret, 1976; Pettigrew, 1973; Roberto, 2003). 3. Predictive ability is typically enhanced when studies use group-level approaches rather than individual-level approaches.

A Multi-Level Analysis of the Upper-Echelons Model

Table 1.

207

A Summary of Levels Issues in Upper-Echelons Research.

Levels Issue

Appropriate level of analysis

Applied to the Upper-Echelons Model

Is the team or the individual (the CEO) the appropriate level of analysis? Is the upper-echelons model Hambrick and Mason (1984) a multi-level model? approach the model as if it were multi-level – the same processes at work at both individual and group levels

Conclusions

Most authors perceive the team to be the appropriate level, but there may be exceptions Careful consideration indicates that the upper-echelons model is inherently an individual-level model. Application to the group level requires rethinking some key issues in the model, in most situations The assumption of Individuality would The assumption of TMT individuality among TMT characterize situations in member independence would members which TMT members do not ordinarily not be tenable, but constitute a ‘‘team,’’ but in some situations the instead act in relative assumption may be independence. Individuality appropriate also requires a situation where the CEO is weak or has no clear vision The assumption of Homogeneity would The assumption of TMT homogeneity among TMT characterize situations in member homogeneity exists members which TMT members in contexts where the CEO is (including the CEO) respond very powerful and/or in identical fashion to autocratic, or has a very strategic stimuli strong vision coupled with strong charismatic relationships with TMT members. It may also characterize situations in which all TMT members are threatened or are otherwise expected to respond in a uniform way When heterogeneity is assumed, The assumption of Widely assumed in upperit is critical to understand the heterogeneity among echelons research. Key process through which TMT members questions are how individual-level concerns are heterogeneous and through manifested at the group level what processes are team – how the TMT makes member heterogeneity decisions reﬂected in team-level outcomes

208

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

Table 1. (Continued ) Levels Issue

Applied to the Upper-Echelons Model

TMT decision-making processes

Requires speciﬁcation of the nature and frequency of interactions among TMT members

The concept of framing

Framing describes how one person’s interpretation of a situation or phenomenon is transmitted to others, thereby inﬂuencing (perhaps controlling) the subsequent interpretation and processing of the issue

TMT heterogeneity: Who is The boundaries of the TMT on the TMT? and decision-making processes are important to specify in a levels framework

TMT heterogeneity: Political models of decision making

Political models of decision making describe situations of power and goal heterogeneity among parties to a decision

Conclusions

At best, researchers assume that TMT members meet weekly to discuss topics of concern. Regardless of the nature and frequency of meetings, it is unlikely that the team jointly experiences stimuli. More likely, stimuli are noticed and interpreted by individual TMT members, and then framed by those individuals for the remainder of the group Framing is likely to be a critical part of the upper-echelons model, and it has not received much research attention. Framing describes how individual TMT members – usually those with interests and expertise in the issue area – inﬂuence the thinking of other members regarding a particular issue Some authors (e.g., Roberto, 2003) argue that the TMT consists of a stable core and a dynamic periphery. In this approach, decision making is delegated to diverse groups depending on the issue and its ramiﬁcations. Core TMT members are represented on most decision-making groups. Political models are important to upper-echelons research, but their use is limited to situations where conﬂicts of interest exist among the members of the TMT. Political models pose further

A Multi-Level Analysis of the Upper-Echelons Model

209

Table 1. (Continued ) Levels Issue

TMT heterogeneity: The role of the CEO

TMT temporal dynamics: The seasons and cycles model

TMT temporal dynamics: CEO succession

Applied to the Upper-Echelons Model

Conclusions

challenges to the traditional upper-echelons model TMTs are unlikely to be groups The CEO’s vision is important of equals. The CEO exerts a to reducing heterogeneity disproportionate inﬂuence among TMT members. over the group and its Without a strong vision, processing heterogeneity among TMT members increases. CEOs are also guardians of decision processes for TMTs. If the CEO encourages open debate, heterogeneity among TMT members is likely to be healthy. If debate is discouraged, homogeneity is the likely result Hambrick and Fukutomi Each season has somewhat (1991) propose that CEOs different implications for the move through several level of TMT heterogeneity predictable stages as they go and interactions among through their tenures TMT members Succession introduces tensions Early and late in a CEO’s among TMT members, as tenure, the implications of they jockey for position the succession process are either as the successor CEO likely to be felt. Relay or as members of the new succession reduces these TMT tensions, leading to more homogeneity among TMT members

4. The interactions among group members are likely to be important to decision structures and outcomes and, therefore, are of important interest to researchers. Interestingly, the ﬁrst of these reasons (highlighting the heterogeneity in goals, values, and so forth, among TMT members) actually complicates the upper-echelons model, because of the complexity it introduces and the fact that the model says virtually nothing about this added complexity. Put differently, the model does not address how a heterogeneous group of people comes together and builds consensus around strategic decisions. Finkelstein and Hambrick’s (1996) reasoning suggests that heterogeneity

210

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

might best characterize the TMT, and that what Rousseau (1985) calls a ‘‘multi-level’’ model might best characterize strategic decision making. We will return to this issue later. The second and third reasons provided by Finkelstein and Hambrick certainly appear to be correct, but do not provide a theoretical rationale for selecting the TMT as the level of analysis. Only the fourth reason (referencing the importance of team member interactions and processes) moves us toward an explicit levels focus. This reason ties closely to calls from levels researchers reminding us that when we examine group-level outcomes (such as TMT strategic decisions), we must specify the processes that generate those outcomes (House et al., 1995). For our purposes, this statement means that we must describe how the psychological and experiential proﬁles of individual group members become realized in TMT-level decisions. Toward that end, we will apply an explicit levels lens to the upper-echelons model, starting with some rather simple assumptions about group composition and decision processes, and moving toward more complex assumptions after the simple ones have been examined.

The Upper-Echelons Model in a Multi-Level Framework Close examination of Fig. 1 indicates that the upper-echelons model itself does not separate individual and group-level decision making. Indeed, it applies the same orthodoxy to both, implying that the same – or parallel – processes are at work for both levels. Rousseau (1985) describes the relationship pattern in which generalizations may be made from phenomena observed at one level to those occurring at another level as a multi-level model. That is, parallel processes are at work at both the individual level and the TMT level. Put differently, the decision-making contexts for both individual executives and TMTs represent functionally similar phenomena that occur at different levels. While this notion is intuitively appealing in the original Hambrick and Mason (1984) manuscript, careful consideration of it raises important questions. As we move from left to right across the model illustrated in Fig. 1, several multi-level issues become apparent. First, teams will not experience decisions in exactly the same way that individuals do. Different team members will also have somewhat different executive orientations. The ﬁelds of vision of individual members will vary, requiring us to specify how the variegated limited ﬁelds of vision of individual executives yield a similar construct at the team level. Selective perception will be different for different members, as will interpretation and

A Multi-Level Analysis of the Upper-Echelons Model

211

construed reality. Thus, the model seems considerably less intuitively appealing if the team is the level of analysis, at least until the researcher speciﬁes the processes through which the team makes decisions. In some situations, a multi-level model does effectively describe strategic decision making. For example, the single-person TMT, where the CEO is extremely powerful, would render the TMT relatively impotent, shifting the level of analysis back to the individual level. Founder CEOs sometimes ﬁt this description (Haveman, 1993). Another case might be the highly autocratic leader who refuses to delegate decisions or to consider much input from others in making decisions. A ﬁnal case might be the ‘‘stale in the saddle’’ CEO (Miller, 1991), or the leader who has remained in charge for too long. In this setting, organizational routines have become relatively ﬁxed, and there is no impetus (or even tolerance) for change. Clearly, situations such as these exist, but they do not appear to describe the decisionmaking contexts of most organizations. Specifically, the large ﬁrms that are the focus of most upper-echelons research are not often characterized in these ways. Thus, we must move toward the use of what Rousseau (1985) calls a composition model, which speciﬁes how a group of individuals experiences a decision situation. We will characterize the upper-echelons model by alternately applying each of Klein et al.’s (1994) group member characterizations: individuality, homogeneity, or heterogeneity. Individuality among TMT Members The concept of individuality in characterizing a TMT is not very appealing on its face, because to be characterized in this way the members must be quite independent of one another and not relate very much to one another. Additionally, in a levels framework, individuality implies that group member inﬂuence is equal (Klein et al., 1994). Put differently, no specific team member has more inﬂuence than any other group member. Hambrick (1994) reminds researchers that some TMTs may experience little of the cohesion, camaraderie, or give-and-take of a team. Some are not much more than a collection of people with high-level positions in a single organization. However, true individuality would require that they not relate to or interact with one another at all, or feel any relationship to one another. We ﬁnd the extreme assumption to be untenable. Clearly, it is possible that a TMT may exhibit little ‘‘team-ness’’; when this is the case, the researcher should clearly state this assumption and justify it for the sample under study (Klein et al., 1994). We can think of several instances in which this assumption might be appropriate, and therefore accurately describe TMT functioning. A corporation

212

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

consisting of unrelated businesses brought together by a string of acquisitions might be one example. A ﬁrm that runs on strict ﬁnancial controls as opposed to strategic controls (Hill, Hitt, & Hoskisson, 1992) might operate similarly. With a ﬁnancial control system, clear and objective goals are set for all subunit leaders, and compensation and other resources are allocated strictly based on how well the goals are reached. Such a system reinforces individuality among the executive cadre, discourages cooperative behavior across subunits, and inclines executives to make decisions by considering only the context of their specific responsibilities, and nothing else (Hoskisson & Hitt, 1988; Varadarajan & Ramanujam, 1990). Of course, in this kind of situation, the members of the TMT might perceive one another as competitors, which would weaken significantly both the original intent of the upper-echelons model and the assumption of individuality, as described by Klein et al. (1994). Finally, TMTs whose members who are geographically separated and seldom have the opportunity to interact directly may come to act like collections of individuals rather than teams. Park, Lee, and Cannella (2004), for example, consider the physical proximity of TMT members in their assessment of how strategic decisions are made. TMTs whose members are not colocated might well be assumed to operate more independently than those whose members are located in the same place. Another setting in which the individuality assumption might be valid would be that in which highly decentralized decision making takes place within the TMT (Vancil, 1979). For example, a single executive may be put in charge of an initiative and be given wide leeway to pursue whatever action he or she deems appropriate. Sometimes, new product development in large organizations is conducted in this way, with different champions protecting and advocating a variety of new products until the products can demonstrate some potential in the marketplace (Hoffman & Hegarty, 1993; Howell & Higgins, 1990; Schilling & Hill, 1998). Of course, to analytically track such a model with a sample of organizations, the researcher must identify who is responsible for what decisions – a potentially nettlesome task. It seems unlikely that a sizable proportion of the large ﬁrms to which the upper-echelons model has been applied would fall into this category. Finally, to truly ﬁt the assumption of individuality, the CEO must be either removed from the decision or considered to be just another team member. This situation would exist in settings where the CEO delegates decision making to others. For example, there is little evidence that Jack Welch of General Electric (GE) forced decisions on his executives (Tichy & Charan, 1989). Instead, his approach seemed to be to assure that all aspects of a decision were considered, leaving the ﬁnal choice to others. For

A Multi-Level Analysis of the Upper-Echelons Model

213

many – perhaps most – decision situations, however, the CEO will exert a disproportionate inﬂuence on the ﬁnal choice. Homogeneity among TMT Members Clearly, TMTs are homogeneous on at least some dimensions. For example, the overwhelming majority are white males (Daily, Certo, & Dalton, 1999). If we take the homogeneity assumption to its logical extreme, the upperechelons model will work well as formulated. Staw (1981), for example, uses this assumption to argue that groups respond to threats much as individuals do. If all members of the decision-making group see the world in the same way, the TMT will act like a single person. However, this assumption undercuts one of the key rationales put forth by Finkelstein and Hambrick (1996) for selecting the TMT as the unit of analysis – namely, that organizations are made up of diverse individuals. Most discussions of TMT functioning assume some reﬂexivity and a discussion of issues among TMT members, which leads naturally to the assumption that TMT members are not completely homogeneous. Nonetheless, some decisions – especially those characterized by Staw et al. (1981) as significantly threatening to the entire group – are likely to be processed as if all team members are homogeneous. For example, a takeover attempt (hostile or otherwise) broadly threatens the entire group of top managers (Coffee, 1988; Davis & Stout, 1992; Hambrick & Cannella, 1993; Jensen, 1986; Stein, 1988; Turk, 1992), and it is reasonable to assume a fairly homogeneous response among all members of the upper echelons, at least initially (Walkling & Long, 1984). In another example, Shen and Cannella (2002a) describe how the threat of an outside succession brings together the non-CEO members of TMTs, as all tend to be threatened by the entry of a new CEO from outside the ﬁrm. A clear threat, which is consistently threatening to all TMT members, is likely to result in a team that responds as if the members were highly homogeneous. A second example of a situation in which team members might be treated as homogeneous might arise when considering a decision that has a strong institutional history within the ﬁrm. For example, after a disastrous failure of an earlier strategy, it will be quite difﬁcult for executives in the ﬁrm to conclude that they should reattempt some form of the earlier strategy. Their negative shared experience will lead executives to a very consistent conclusion that some other strategy will be more appropriate. Harley-Davidson, with its experience with small motorcycles (Teerlink, 2000), and Intel, with its experience with the reduced instruction set computer chip (RISC) (Afuah, 2001), are examples that come to mind. In both cases, a strong,

214

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

knee-jerk negative reaction met any strategy perceived to be similar to one that had been widely discredited within the ﬁrm. As the preceding examples indicate, there are some prominent situations wherein the assumption of homogeneity seems appropriate. Additionally, when researchers have made the assumption of homogeneity, they have usually been explicit about it and have argued persuasively for its veracity. While the limited examples described here are illustrative of homogeneity among TMT members and the consistency it brings, strategic decisions are not limited to responses to perceived threats or revisitations of previous failed strategies. They also include decisions about initiatives and opportunities, which would seem to be decision situations in which the assumption of homogeneity would be much less tenable. Some opportunities might be so obviously valuable in the eyes of all TMT members that they would respond as a homogeneous group, but this situation seems to be relatively rare. In general, for most strategic decision situations, and specifically for those in which several potential choices would appear to be viable (Mischel, 1977), the assumption of homogeneity seems unrealistic. Heterogeneity among TMT Members The assumption of team member heterogeneity clearly leads to the greatest complexity in analyzing the process through which TMTs arrive at decisions, but it also appears to be the most realistic for mainstream strategic decision-making settings. Furthermore, heterogeneity is the assumption that seems overwhelmingly to be selected by TMT researchers (e.g., Amason, 1996; Bantel & Jackson, 1989; Carpenter, 2002; Finkelstein & Hambrick, 1996; Henderson & Fredrickson, 2001; Kilduff, Angelmar, & Mehra, 2000; Simons, 1995; Wiersema & Bantel, 1992). As noted by Finkelstein and Hambrick (1996) organizations are made up of diverse individuals, who have diverse goals, preferences, values, and intentions. Additionally, it is widely recognized that some TMT members are more powerful than others (Finkelstein, 1992; Hambrick & Finkelstein, 1987). While the heterogeneity assumption leads to more complexity, it would also seem to be the most interesting and realistic approach. A careful reading of Hambrick and Mason (1984) indicates conclusively that they had this assumption in mind when they argued that the upper-echelons model would be most effective when applied to the TMT. Members of the TMT are very likely to differ on a number of important dimensions. For example, an executive’s functional background is likely to inﬂuence his or her beliefs primarily through exposure over time to different

A Multi-Level Analysis of the Upper-Echelons Model

215

situations or stimuli (Chattopadhyay et al., 1999; Starbuck & Milliken, 1988). The greater the length of time spent by a member of the TMT in a particular position or functional area, the more likely his or her beliefs and actions will be consistent with that functional area. Thus, when the TMT considers a strategic situation, different aspects of the situation are likely to come to life for different executives (Dearborn & Simon, 1958; Walsh, 1988). This idea is a foundational assumption for the original upper-echelons model and its subsequent formulations. Note that this statement is not meant as an endorsement of the use of demographic measures to assess executive cognition, a practice that has been critiqued by several researchers (Lawrence, 1988; Priem, Harrison, & Muir, 1995; Priem, Rasheed, & Kotulic, 1995). The assumption of heterogeneity, however, leads naturally to the questions of on what dimensions the heterogeneity is felt and how the group comes together to make a decision. Next, we describe several potential models of TMT processes that seem viable.

TMT Decision-Making Processes As a ﬁrst cut, we might assume that the TMT meets regularly, jointly experiencing the stimuli described by the upper-echelons model, and discussing openly what each perceives and what to do about it (if anything). This seems to be the assumption that Hambrick and Mason (1984) and many subsequent authors had in mind. However, this assumption raises significant questions, at least in its extreme form. For example, it is not clear how the TMT would jointly experience the strategic situation, stimuli, and so forth. TMT members have independent responsibilities, and they do not spend all of their time together. More likely, they meet weekly for a few hours and discuss issues that have already arisen. In other words, they meet to discuss issues that one or more of the team members have already noticed and considered to be of enough importance to raise in the group meeting. This is a very important context to consider for two reasons. First, it likely represents a realistic view of TMT functioning (Kotter, 1982; Mintzberg, 1973; Roberto, 2003). Team meetings are likely to be planned forums where either (1) an agenda is set and distributed ahead of time, thus restricting the issues considered, or (2) concerns, ideas, and issues that each member brings are raised and discussed openly. In either case, the entire left side of the upper-echelons model depicted in Fig. 1 (executive orientation and the ﬁltering process) needs to be carefully reconsidered, because the

216

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

team does not experience the strategic situation jointly, but rather considers issues raised (i.e., issues already noticed) by one or more TMT members. The challenges that individual-level sense making brings to the upperechelons model are that (1) the issues the team considers are likely to be more important to the member raising the issue than to other members, and (2) the remaining members’ ﬁelds of vision, selective perception, and interpretations are likely to be strongly inﬂuenced by the framing of the member who raises the issue. Certainly, at regular meetings, the TMT will review regular reports, and TMT members are likely to make regular reports to the group regarding areas of their specific responsibility. Here again, the team member raising an issue has already noticed it and made sense of it, and therefore that member has a very large effect on the group’s processing of the issue. This is a fundamental assumption of the work on strategic issue diagnosis (Dutton & Ashford, 1993; Dutton & Duncan, 1987; Dutton, Fahey, & Narayanan, 1983; Dutton & Jackson, 1987). In that work, the CEO (an individual) is the focus of analysis. The CEO notices or perceives issues and identiﬁes them as either threats or opportunities, then takes the issues to others in the organization, framing each issue as an opportunity or threat as it is communicated to others. Note that this process raises challenges for the left side of the upper-echelons model depicted in Fig. 1, as the team does not jointly experience and make sense of the environment, but rather considers issues that have already been identiﬁed and possibly researched deeply by others. Dutton and her colleagues believe that this effect is sufﬁciently strong to warrant selecting the individual (the CEO) as their unit of analysis, assuming that the CEO’s framing is so strong that the other TMT members will simply accept it. The assumption that only the CEO’s perception matters could be relaxed and the assumption of heterogeneity remains intact. It is likely that TMT members have diverse responsibilities and, therefore, that they will notice, study, and develop perspectives and frames of reference for issues before bringing them to the TMT. Because issues raised would likely fall within their own area of responsibility, their perceptions and perspectives are likely to carry significant weight. To better understand how this process would work, we turn to the concept of framing.

The Concept of Framing It is worthwhile, at this point, to discuss the concept of framing, which is deﬁned as the categorization or classiﬁcation of an issue. The framing

A Multi-Level Analysis of the Upper-Echelons Model

217

process bounds (frames) the issue and limits potential responses to it. Decision framing has received a good deal of attention in the decision-making literature (e.g., Fiol, 1994; Kogut & Kulatilaka, 2001; Nutt, 1993, 1998; Sitkin & Weingart, 1995; Tversky & Kahneman, 1986; Wiseman & GomezMejia, 1998). Tversky and Kahneman (1981, p. 453) describe a frame as ‘‘the decision-maker’s conception of acts, outcomes, and contingencies’’ associated with rational choice. They argue that the frame adopted is partly inﬂuenced by the decision maker’s interpretation of the situation and partly by his or her functional conditioning and psychological characteristics. Overwhelmingly, framing is considered by prior researchers to be an individual-level phenomenon. Much of the ‘‘ﬁltering process’’ of the upper-echelons model describes the process of framing. Most research on framing considers framing to be the categorization of issues as either opportunities or threats, following Kahneman and Tversky’s (1979) prospect theory. Prospect theory predicts that the response to a situation will depend importantly on the decision’s context, or how the decision is framed by the decision makers. The categories of threats and opportunities are important, but seem too limited for the strategic decision-making context. For example, Fairhurst and Sarr (1996) describe framing as a key leadership task, and go well beyond identifying issues as simply threats or opportunities. It is through framing that organizational leaders guide the thinking of their followers, helping them to construct perceptions of reality. While the Fairhurst and Sarr book is not deeply theoretical, it is important to note that such processes occur frequently in organizations and represent a key mechanism by which leaders lead (Hambrick & Cannella, 1989; Kotter, 1982; Mintzberg, 1973; Waldman & Yammarino, 1999). Framing and the Upper-Echelons Model The framing issue is important for many approaches to team-level decision making because, as noted earlier, it is likely that issues come before the TMT pre-framed. Again, this assumption would either skip the left side of the upper-echelons model or offer a different starting point for the ﬁltering process for those members of the TMT who are hearing the framed issue for the ﬁrst time. Playing this assumption out leads to the idea that many strategic issues are brought to the attention of the TMT by a member who has noticed them, made sense of them, and has a point of view about them that he or she wishes to persuade the other TMT members to adopt. Furthermore, decision making may well be delegated to others based on the framing that the issue receives. Issues framed as ‘‘ﬁnancial,’’ for example,

218

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

are not only more likely to be noticed and raised by the CFO, but also more likely to be delegated to the CFO. Additionally, the CFO’s opinion on issues framed as ‘‘ﬁnancial’’ is likely to carry disproportionate weight. Similarly, issues framed as ‘‘manufacturing’’ are likely to be heavily inﬂuenced by manufacturing executives, and so forth. Most formulations of the upper-echelons model suggest that more heterogeneous TMTs will lead to a broader consideration of decision issues and factors (Finkelstein & Hambrick, 1996). The notion of framing and the changes that we have outlined for the upper-echelons model seem to weaken that assumption somewhat. The earlier in the decision process the framing takes place, the more inﬂuence that individual psychological makeup will have. Once the decision is framed and some backup evidence presented, it seems as if TMT heterogeneity would have a much weaker effect. All is not lost, by any means, for existing theories of heterogeneity. More heterogeneous teams are still more likely to identify more issues in the strategic situation, frame them, and bring them back to the group. The theoretical logic differs a bit from the upper-echelons model presented in Fig. 1, but the end result might be the same: TMT heterogeneity leads to broader consideration of the issues. Additionally, at the construed reality and strategic choice stages, the heterogeneity of TMT members is likely to be manifested in ways that lead to a more complex construed reality if the TMT meets regularly and has an open discussion of the issues. The fact that team members are exposed to a broad array of ideas and issues from a heterogeneous group of others should help to widen their own horizons and experiences. In addition, other members of the TMT will consider how the strategic issue, noticed and pre-framed by another, will affect their own area of responsibility. Decisions would appear to be more thoroughly ‘‘vetted’’ by a heterogeneous team. These outcomes are clearly in line with a multi-level interpretation of the upper-echelons model. TMT Heterogeneity Considered in Depth In this section, we carry forward the assumption of TMT heterogeneity. Our discussion covers three issues: The speciﬁcation of who is a TMT member, and the implications of that choice (on the part of the researcher) for TMT processes and the upperechelons model. TMT processes and the level of heterogeneity among TMT members. The CEO’s role in TMT functioning.

A Multi-Level Analysis of the Upper-Echelons Model

219

TMT Heterogeneity: Who is on the TMT? Roberto (2003) raises an important issue in the study of TMT functioning: Most empirical formulations of TMTs involve the use of executive titles to classify specific executives as TMT members or nonmembers. For example, some studies consider all ofﬁcers of the company to be TMT members (Shen & Cannella, 2002b); some use all ofﬁcers above the level of vice president (Cannella, Fraser, & Lee, 1995; Hambrick & Cannella, 1993); and still others use all inside board members (Finkelstein, 1992; Finkelstein & Hambrick, 1990). Roberto argues that the correct identiﬁcation of TMT members may be critical to successful understanding of TMT functioning as well as understanding the decision outcomes that result. Specifically, Roberto (2003) argues that TMTs are often not unitary groups of executives, but rather consist of what he calls a stable core and dynamic periphery. That is, there are a handful of key TMT members, at least one of whom is involved in most strategic decisions of consequence. Ultimately, the composition of remaining members of the decision-making ‘‘team’’ depends on the decision to be made. Groups of executives with particular expertise are assembled for particular decisions, and most groups assembled for a particular decision will include senior executives plus at least some representation from groups much lower in the hierarchy. For example, an important manufacturing problem might involve one senior executive (a member of the stable core TMT) and a group of other executives and/or employees with expertise in manufacturing, or with some other expertise or perspective relevant to that specific decision. The upper-echelons model is not rendered obsolete with the assumption of a stable core and dynamic periphery. Clearly, even in this formulation, the selection of which issues need to be considered and who would be important to have involved in the decision process is made by the stable core of executives (Roberto, 2003). Therefore, through framing and initial decision making, the stable core of the TMT continues to exert a very strong inﬂuence on the broad decision context, even though decisions are largely delegated to others. Additionally, the inclusion of at least one member of the stable core on all key decisions assures that the stable core’s framing is maintained throughout the decision process or, if there is strong logic for changing that framing, the core member can relate the reframing back to the core TMT. In this way, the core TMT can learn from those with specific expertise in the ﬁrm, and those lower in the ﬁrm’s hierarchy can learn the perspectives, mindsets, and overall intentions of the core TMT members. Note that this process involves a two-way ﬂow of information and perspectives across levels. The approach also suggests an important bridge

220

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

between close and distant leadership on the part of the CEO (Waldman & Yammarino, 1999). The central challenge raised by Roberto (2003) is the fact that the demographic makeup of a presumed unitary TMT is used by researchers to make predictions about the decisions and performance implications that result. If no unitary TMT is actually responsible for all decisions, this approach would seem to present a large problem. Nevertheless, as noted earlier, if the key members of the TMT are accurately identiﬁed, much of the logic developed here would still apply because the core TMT framing takes place, and because the core TMT members are likely to be the most senior – and therefore the most inﬂuential – leaders in the ﬁrm. Decisions would likely reinforce the CEO’s vision (Waldman & Yammarino, 1999), because of the great power wielded by the core TMT and the fact that at least one core member is involved in all key decisions. Additionally, the close contact between the core TMT and others of lower hierarchical rank would seem to provide numerous opportunities for leadership and vision selling through framing (Fairhurst & Sarr, 1996). Put differently, the TMT inﬂuence on decision making should be very strong; hence, if the team members are accurately identiﬁed, many of the outcomes predicted by the upper-echelons model would likely be observed.

TMT Heterogeneity: Political Models of Decision Making Another important question to be resolved in upper-echelons research, given the assumption of heterogeneity, is the level and nature of the heterogeneity that accurately characterizes TMT members. Hambrick and Mason (1984) seem to assume (appropriately) a modest level of heterogeneity, and nearly all authors associated with the upper-echelons model assume that the goals of team members are consistent (for notable exceptions, see Bourgeois, 1980; Eisenhardt, Kahwajy, & Bourgeois, 1997). This is not surprising, considering that the primary roots of strategic management theory lie in economics and therefore thinking in the ﬁeld is strongly guided by assumptions of rationality and value maximization (Schendel & Hofer, 1979). Furthermore, the notion of CEO charisma and vision are important heterogeneity-reducing forces in organizations (Kets de Vries, 1998; Klein & House, 1995; Waldman & Yammarino, 1999). It is possible – and perhaps even likely in larger organizations and certain decision contexts – that a political model of strategic decision making would be appropriate. At the heart of any political model of decision making is heterogeneity in both power and goals among parties to a decision.

A Multi-Level Analysis of the Upper-Echelons Model

221

Eisenhardt and Bourgeois (1988, p. 738) describe politics as ‘‘the observable, but often covert, actions by which executives enhance their power to make decisions.’’ Many researchers argue that conﬂict is central to the concept of politics (March, 1962; Pfeffer, 1981). Political models of decision making usually describe large and infrequently occurring decision settings in which the interests of the parties to the decision are clearly in some conﬂict – settings such as CEO succession or CEO compensation. Political models have been used in strategic management studies (Cannella & Lubatkin, 1993; Finkelstein & Hambrick, 1989; Ocasio, 1994; Shen & Cannella, 2002a; Zajac & Westphal, 1996; Zald, 1965), although their use in general strategic decision making has been limited. Note that in CEO succession and compensation decisions, the interests of those charged with making the decision are in some conﬂict. However, political models are equally appropriate to situations in which important differences of opinion arise about courses of action. Reasonable people not only can, but frequently do, disagree about what is the appropriate course of action. In strategy settings, there is frequently no arbiter of last resort when differences of opinion arise, so political models of decision making make be quite appropriate. The approach is usually to determine the opinions of the person with the power to impose his or her will – typically, the CEO. In settings without a dominant power like a strong CEO, researchers often look for coalitions among those charged with making the decision. Coalitions reﬂect bargaining by the parties, as they strive to secure their favored courses of action (Cyert & March, 1963). Additionally, studies using political models often make important distinctions between executives and directors (members of the company’s board of directors) and note the conﬂicts of interest inherent in the relationship between the two groups (Finkelstein & Hambrick, 1989; Shen & Cannella, 2002a). Decision contexts such as day-to-day strategic decision making and decisions limited to the TMT (i.e., not involving board assent or approval) have received considerably less attention. Political models open the door to the assumption that goal congruity among TMT members may be neither complete nor universal, thereby greatly complicating the strategic decision-making process. For this reason, political models begin with the assumption of important heterogeneity among TMT members. To begin using some of these models, the researcher must consider the decision context and determine which TMT members (or groups of TMT members) interests are served by what decision alternatives – an onerous burden for some decision contexts. Most authors argue that politics is most evident when power is decentralized (Pfeffer, 1981). When no single TMT member has the power

222

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

to enforce his or her will on the others, TMT members are more likely to form coalitions that seek to inﬂuence decision making. As such, political models take us somewhat further from the traditional upper-echelons model, at least in its current formulations. Perhaps most challenging would be the notion of significant heterogeneity in overall goals among TMT members. With a political lens, issues would arise and would essentially be contested by groups of executives, depending on their particular goals and interests. Additionally, political models usually invoke some degree of negotiation, bargaining, and trade-offs among the parties (Ocasio, 1994). Political models require fairly clear and well-speciﬁed conﬂicts of interest or sources of disagreement among the parties (Eisenhardt & Bourgeois, 1988; Mintzberg, 1985; Pettigrew, 1973), and these settings are often outside the arena of what might be called ordinary strategic decision making. Political models represent another approach in which the left side of the upper-echelons model depicted in Fig. 1 (limited ﬁeld of vision, selective perception) is circumvented. The interpretation of issues is likely to be a source of contest and disagreement among the parties. Even more prominent might be disagreement about construed reality. It is easy to imagine situations in which executives might disagree on fundamental issues, such as the importance of incremental versus breakthrough innovation, or the extent to which a new technology poses a threat. These fundamental disagreements, where present and allowed to grow, could recur over and over, as the competing camps interpret what they see (the same facts) in light of their own worldview. In this way, decisions follow the construed reality of the powerful groups in organizations. The presence of a singular power in the organization, such as the CEO, likely minimizes the occurrence of this situation. In fact, Pfeffer (1981) argues that when power is highly centralized, conﬂict becomes submerged and the use of politics declines. In contrast, Eisenhardt and Bourgeois (1988, p. 743) found the use of politics was also tied to centralized decision making. The more powerful the CEO, for example, the greater the likelihood that TMT members will act to consolidate power and engage in ‘‘alliance and insurgency behavior’’ while the CEO acts to control or completely withhold information from the group. TMT Heterogeneity: The Role of the CEO The CEO is the top ofﬁcer in the ﬁrm and, correspondingly, is usually thought to be the most powerful and inﬂuential person in the ﬁrm (Vancil, 1987). The CEO is also regularly considered to be the architect of the TMT, in that members serve at his or her option (Finkelstein, 1992; Shen

A Multi-Level Analysis of the Upper-Echelons Model

223

& Cannella, 2002a). Therefore, TMT members may feel a strong obligation toward the CEO and be unwilling to challenge him or her (Lorsch, 1989). No treatment of TMT decision-making processes would be complete without consideration of the CEO’s role as separate from other top managers. The CEO’s role in TMT functioning might have been considered as part of overall heterogeneity. We provide this separate treatment, however, because discussions of heterogeneity in prior TMT research overwhelmingly emphasize dispersion measures among team member characteristics (Finkelstein & Hambrick, 1996). Our approach focuses on the concept of heterogeneity as the term is used in levels research (Klein et al., 1994). We discuss key issues relating to the CEO’s vision (or lack thereof) and the CEO as guardian of the interactions among TMT members (the processes used to make decisions). CEO Vision. The CEO is expected to set the vision and/or mission of the organization, which will strongly inﬂuence the context in which decisions by TMT members are made (Gabarro, 1987; Kotter, 1990; Vancil, 1987). As previously discussed, Waldman and Yammarino (1999) argue that heightened cohesion with the TMT ensues when a charismatic relationship develops between a CEO and TMT members. The CEO fosters this commitment by inspiring TMT members to wholeheartedly embrace his or her vision. Once the vision is established, it will be used to frame subsequent issues. This framing inﬂuence will be felt throughout the ﬁrm’s hierarchy, as issues raised will be tied to the vision to gain the CEO’s support for action. Additionally, executives in all parts of the hierarchy who want to boost their careers will likely watch carefully for issues that they can identify and articulate as important to the overall vision and mission of the CEO. In this way, they curry favor in the promotion process. These selfinterest drivers will produce important consistency in the mindsets or ‘‘construed realities’’ among managers at all levels, but especially among TMT members, making the TMT more homogeneous than it might be otherwise. This suggests that the more strongly the CEO holds to a vision and frames alternatives as building toward that vision (or detracting from it), the more the TMT members will come to be (or act as if they are) highly homogeneous. Similarly, CEO charisma will lead to reduced heterogeneity among TMT members (Klein & House, 1995; Waldman & Yammarino, 1999). Alternatively, the CEO who does not ‘‘take charge’’ (Gabarro, 1987) by strongly articulating a vision and course of action may leave a power vacuum in the organization that others might strive to ﬁll. In this situation, political coalitions are likely to form among TMT members and inﬂuential

224

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

others in the ﬁrm, with each group vying for the right to bring its own issues to the fore (Shen & Cannella, 2002a). It is possible – and perhaps likely – that political inﬁghting and struggles among TMT members will result from the presence of a weak CEO. Certainly, with a weak CEO in place (deﬁned as one with no clear vision or direction), TMT heterogeneity will be accentuated. The CEO as Guardian of TMT Process. The CEO exerts a powerful inﬂuence on TMT functioning by establishing the process through which decisions are made. For example, a CEO may encourage open discussion of issues, problems, and ideas raised by anyone in the organization, regardless of hierarchical position. Jack Welch of GE became known as this kind of leader – one who fostered lively and open debate on all issues before the top group (Tichy, 1989; Tichy & Charan, 1989). Alternatively, a powerful CEO may openly discourage debate and dialog, moving quickly to squelch any opposition to his or her choices (Miller, 1991; Miller & Droge, 1986). The CEO who encourages open debate and disagreement moves the TMT toward the assumption of more heterogeneity among members, whereas the CEO who frames open debate as dissent – or, even worse, as disloyalty – moves the TMT toward less heterogeneity among members. Therefore, we assert that only when the CEO encourages open debate and dialog among TMT members does a team-level approach to decision making truly improve the effectiveness of decision making, as per Finkelstein and Hambrick’s (1996) prediction. Under an autocratic or intolerant CEO, only the opinions and perspectives of one person really matter (i.e., the CEO). The CEO’s commitment to a single vision for the company’s strategy is not necessarily a requirement if he or she encourages open debate and dialog. Nevertheless, some consistent and objective methods for selecting preferred courses of action are essential. For example, Jack Welch clearly preferred market share, profitability, and growth as organizational outcomes, but was open to divergent views regarding how to actually achieve those objectives. At the same time, he did not tolerate disagreement about these ultimate objectives (Tichy, 1989; Tichy & Charan, 1989). Put differently, disagreement about means is not the same as disagreement about ends (Bourgeois, 1980). In situations where the CEO is both powerful and insistent on a full discussion and consideration of the issues (e.g., Jack Welch of GE), the TMT moves toward what we might think of as the ideal state envisioned by Hambrick and Mason (1984). Our earlier caution that issues raised are noticed and framed before the TMT considers them, and that the team does

A Multi-Level Analysis of the Upper-Echelons Model

225

not jointly experience and make sense of stimuli, stands even in this setting. However, a full discussion of the issues will likely cause those who raise issues to more fully consider the dimensions and ramiﬁcations, and the discussion is likely to highlight issues for further consideration. Decisions, when made, are likely reﬂective of the full set of facts that can be gathered around the decision setting, and those facts will be considered by all TMT members though their various backgrounds and value sets. Additionally, the open discussion among TMT members, with the CEO as guardian of the process rather than dictator of the outcome, will help to build a consistent and relatively up-to-date construed reality among TMT members. Rather than dedication to historical beliefs or blind commitment to one approach, the TMT is more likely to recognize and confront changed realities and to incorporate those new circumstances into their subsequent issue processing. It is difﬁcult to see how this endeavor could fail to improve the quality of decision making overall. Finally, with the CEO enforcing a full and open decision-making process, it is less likely that member disagreements and conﬂicts will turn personal, leading to disintegration and distrust among TMT members (Amason, 1996). Conﬂict must remain constructive if TMT interactions are to avoid negative consequences from it. Again, a powerful CEO who is dedicated to a fair and open discussion of issues will probably tend to keep the conﬂict constructive. Finally, some CEOs are aloof and distant, and these individuals are likely to have a correspondingly negative inﬂuence on TMT functioning. When it is unclear what the CEO wants or will support, it is naturally more difﬁcult for TMT members to hold open discussions and reach any sort of consensus. Additionally, TMT members may end up competing for the CEO’s attention. One example of the problems that this situation may produce is illustrated in Hambrick and Cannella’s (2004) study on CEOs who have COOs. The authors argue that the CEOs most likely to have COOs are those who do not like or feel conﬁdent in managing internal matters in their ﬁrms. For this reason, they elect to have COOs and delegate to those COOs all or most internal responsibilities. Because TMT members in this situation ﬁnd themselves further removed from the CEO and remain unsure of what the CEO will support or how the COO ﬁts into the picture, decision-making interactions among team members are likely to suffer. When leadership is unclear, turmoil in decision making is the likely result. In support of this notion, Hambrick and Cannella (2004) show that ﬁrms that separate the roles of CEO and COO (i.e., have separate people fulﬁll each role) are significantly poorer performers than those that do not separate these roles.

226

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

TMT Temporal Dynamics Dansereau et al. (1999) argue that groups frequently evolve over time, and this evolution is likely to have important levels-related consequences. For example, Shamir (1995) suggests that positive interactions between a CEO and his or her TMT over time lead to the development of trust. Other researchers have raised the possibility that cohesion between the CEO and TMT over time promotes the cascading of leadership behavior to lower echelons within the organization, thereby strengthening group cohesion and effort across the hierarchical echelons. In turn, this cohesion is believed to affect the goal orientation of groups and lead the organization toward a higher level of organizational performance (Waldman & Yammarino, 1999). In this section, we describe two approaches for considering this evolution of group characteristics over time: The seasons and cycles model of Hambrick and Fukutomi (1991), in which the behaviors of the CEO are argued to evolve in predictable ways over CEO tenure. The natural group evolution of TMT members over time, and the intrusion of conﬂicts of interests among team members both late in CEO tenure, as a successor is being selected, and early in a new CEO’s tenure, before his or her power has been consolidated (Shen & Cannella, 2002a). Two factors are likely to affect TMT heterogeneity during the tenure of the incumbent CEO. The ﬁrst of these is illustrated by the seasons and cycles model of CEO tenure, put forth by Hambrick and Fukutomi (1991). The second is caused by tensions among TMT members generated by the process of CEO succession. Each of these factors is discussed below. CEO Seasons and Cycles Hambrick and Fukutomi (1991) argue that CEO behavior changes in predictable ways over the course of the individual’s tenure as top ofﬁcer. Drawing on the work of earlier authors such as Gabarro (1987) and Kotter (1982), the ‘‘seasons and cycles’’ model describes changes on several critical dimensions over time – namely, commitment to the paradigm, task knowledge, information diversity, task interest, and power. The authors label the ﬁve seasons as (1) response to mandate, (2) experimentation, (3) selection of an enduring theme, (4) convergence, and (5) dysfunction, noting that not all CEOs last through all ﬁve of the seasons. Next, we discuss each season and its implications for TMT decision processing and interactions.

A Multi-Level Analysis of the Upper-Echelons Model

227

In the ‘‘response to mandate’’ stage, the new leader responds in ways that he or she perceives to be expected by those who made the selection decision (i.e., board members, large investors, and the predecessor CEO). Typically, the new leader seeks to remedy perceived inadequacies of the predecessor and follow any initial instructions or suggestions of the board of directors. Power is relatively low at this point, and there is a moderately strong commitment to respond to the perceived mandate. Task knowledge is low, but increases rapidly. Task interest is quite high, as the CEO is actively engaged in the new job. Information diversity is high, as the new leader is actively learning and seeking out divergent points of view while he or she builds an agenda (Kotter, 1982). During this stage, the TMT is either quite committed to the new CEO or plotting his or her replacement (Shen & Cannella, 2002a, b). TMT members are quite uncertain of their own futures in the company at this point, as they likely serve at the behest of the new leader and are typically anxious to please the new leader so that they can stay on and become full members of the TMT. Thus, TMT heterogeneity, though evident in the nature of the members, may be tempered to gain the approval of the CEO. At the same time, actions taken outside the group are more likely reﬂect the individuality of each member. The fact that the new leader seeks out information diversity encourages TMT members to speak honestly and openly, and it is likely that the new leader will identify several members of the group who will form a core TMT that will stay in place during the new leader’s entire tenure (Gabarro, 1987). During the ‘‘experimentation’’ stage, new CEOs who survive the response to mandate stage (which is quite likely) begin to experiment on the job, seeking the vision (or enduring theme) that will characterize their tenures. Miller and Shamsie (2001) characterize this life-cycle phase as the learning stage in which leaders ‘‘test the waters’’ and ﬁnd out more about their organization and market. In this stage, the CEO is again open to many sources of information, but the information is becoming increasingly more ﬁltered, as the new leader’s vision begins to take shape. The CEO’s commitment to the paradigm (vision) could be strong or weak. Task knowledge is growing rapidly, and the CEO’s power is increasing. During this stage, TMT members either become permanent or are replaced by the new leader as he or she seeks a stable group of executives to form an enduring TMT. The increased focus of this stage begins to reduce the heterogeneity of the TMT, though it may still be quite diverse. TMT members are developing work habits that will stay with the team permanently as well. As the enduring theme is developed, the team members form an ever tighter

228

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

group. In addition, the CEO’s decision-making style (open discussion and disagreement versus no dissent permitted) is becoming ﬁxed. In the ‘‘selection of an enduring theme’’ stage, new leaders work to capitalize on their past learning experience. They gain more conﬁdence and begin to select and communicate a uniﬁed vision or enduring theme. This theme will typically remain in place throughout the duration of the leader’s tenure. Information diversity is still present, but rapidly becoming ﬁltered. Considerably fewer sources of information are referenced, relative to the two earlier seasons. Tushman and Romanelli (1985) refer to this phase as a period of consolidation. The CEO’s power is increasing rapidly, and task knowledge is beginning to peak. The CEO is becoming quite committed to a unitary vision, and is looking to build support for that vision throughout the organization. TMT members who are not fully committed to the vision are expected to leave the organization. Notably, TMT functioning is becoming increasingly more regular and organized (whether characterized by open discussion or muted discussion). TMT members are growing sharply in homogeneity, as those who do not ﬁt the enduring theme are replaced or depart voluntarily. In the ‘‘convergence’’ stage, the theme has been selected and the new leader is fully committed to it. Those who disagree with it are removed. Dissent is unlikely on the theme, although some dissent may be permitted with respect to the means of achieving the theme. CEO power is high, but CEO task interest begins to decline, as the leader settles in for the remainder of the tenure. Even more information ﬁlters are put in place, and information diversity declines steadily. The TMT becomes much more homogeneous in this stage, and a large number of team members have regular and established interactions that lead to predictable outcomes. Likewise, actions taken by members outside the group tend to become more predictable. Even if a TMT member perceives a significant threat or opportunity, unless it ﬁts well within the established theme, it is unlikely to garner much attention. To a increasingly greater extent, the TMT acts like a group of highly homogeneous members. In the ‘‘dysfunction’’ stage, those CEOs who stay this long become complacent and lose touch with their markets (Miller & Shamsie, 2001). They are ‘‘stale in the saddle’’ (Miller, 1991), leading to the conclusion that continuation of the leader is actually harmful to the organization. At this point, the CEO is very powerful but brooks little or no dissent or debate. Executives tend to stick with a strategic formula beyond its usefulness to the organization. The leader’s task interest declines rapidly, as he or she no longer feels challenged by the job. Not surprisingly, this phase is marked by declining ﬁnancial performance. Interestingly, during this period, the TMT

A Multi-Level Analysis of the Upper-Echelons Model

229

may perceive that the board is likely to select an outsider as the new leader, as all current TMT members are strongly associated with the dysfunctional leader (Shen & Cannella, 2002a). This strong threat unites TMT members, and they may take action to remove the CEO in an effort to assure that one of them will become the new leader. The threat of an outsider succession arises because outsider successors are well known to replace more of the existing executive cadre than insider successors (Dalton & Kesner, 1983; Helmich & Brown, 1972). CEO Succession and Its Effects on Team Dynamics One ﬁnal issue is likely to have important temporal effects on the TMT and its functioning – the issue of CEO succession. Early in a new CEO’s tenure, it is likely that some individuals who were viable competitors for the position remain part of the TMT. Therefore, the new CEO must deal with the psychological implications of being passed over among what are surely key members of the new team. We say ‘‘surely key members’’ because the most senior members of the team would also be the most likely candidates for the CEO job (Cannella & Shen, 2001). The assumption that the TMT members serve at the pleasure of the CEO is widespread in the literature (e.g., Finkelstein & Hambrick, 1996). At the same time, empirical examination illustrates that the TMT does not resign en masse when a new CEO is appointed (Cannella & Lubatkin, 1993; Cannella & Shen, 2001; Helmich & Brown, 1972). Indeed, the ﬁrm often strives to keep senior executives in place so as not to lose the ﬁrm-specific talent that it has accumulated (Castanias & Helfat, 1991, 2001). Most of the time, and for most issues, the TMT is expected to work together as a highly uniﬁed group. When the time comes to choose a successor, however, team members may (at least for the succession issue) become competitors. Shen and Cannella (2002a) describe how senior executives, in a succession setting, want very much to become CEO, for both ﬁnancial and personal reasons. Being passed over is difﬁcult for them, and this rejection will surely complicate their interactions with the new leader and with each other. For all of these reasons, early in a new leader’s tenure, TMT members may be working out interpersonal issues with one another, and perhaps having a difﬁcult time functioning as a team. If the succession was the result of a relay, with an heir apparent serving in a grooming period of a few years, many of these issues may have been worked out by the time the CEO title passes to the successor (Cannella & Shen, 2001; Vancil, 1987). In the event of a ‘‘horse race,’’ where several candidates are pitted directly against one another, more exits among those who were passed over are expected (Vancil,

230

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

1987). Horse races accentuate the fact that the succession process has winners and losers, and it publicly identiﬁes both. In any event, early in a new CEO’s tenure, a number of points of disagreement among TMT members are likely to arise. This situation is ripe for dysfunctional conﬂict to interfere with functional conﬂict (Amason, 1996). Furthermore, this is a time when the new CEO is attempting to build and consolidate his or her power. For at least a year or two, regardless of the CEO’s style of leadership or whether he or she encourages open debate and dialog, the TMT is likely to be operating at a higher-than-optimal level of heterogeneity. Applying this factor to the upper-echelons model, the team may not notice key issues arising in the external and internal environment, or it may not pay attention to those issues or to others outside the team who try to raise them. Strategic decisions themselves may be surrounded by conﬂict, but the discord may be the remnants of affective conﬂict, rather than the constructive conﬂict needed for effective decision making. In many ways, the team will be heterogeneous, but the beneﬁts of heterogeneity will not be fully realized. In the middle years of a CEO’s tenure, the team should settle in, resolve the affective conﬂicts left over from the succession decision, and function more effectively. In the later years, when it is clear that a new successor is needed, the competition among team members will again sharpen (Vancil, 1987) as the TMT members position themselves to be selected as the next CEO. Of course, in real life, the situation is not always as clearly deﬁned as pictured here. Many ﬁrms have a single ‘‘favorite son’’ or a person whom all on the TMT expect to be the next CEO. This clear choice would likely reduce the conﬂict and tension among the TMT members as the succession decision arises. In contrast, if there is no clear successor, coalition behavior among TMT members would be expected (Shen & Cannella, 2002a). That is, team members may align themselves with candidates that they are close to, and work to place those candidates in the CEO position. Of course, the expectation is that the entire coalition will beneﬁt if its candidate is selected. Again, it is not likely that this competitive and political behavior will always be severe, but the competition for the top position can get out of hand, causing significant harm to both the TMT and the organization (Vancil, 1987). Keeping the TMT functioning effectively is one reason for the popularity of the relay succession.

DISCUSSION In this chapter, we have evaluated the upper-echelons model with a levels lens, noting several concerns about using the model with groups as well as

A Multi-Level Analysis of the Upper-Echelons Model

231

individuals. At the individual level, the model appears to be quite reﬂective of individual psychological responses. At the group level, the model is hampered by the fact that the group is unlikely to jointly experience and make sense of stimuli; instead, individual members will note issues and raise them to the larger group. This tendency has important implications for the framing of decisions and for the kind of action that is likely to be observed. Note that our discussion does not present sharp challenges to the predictions made by the upper-echelons model, but rather poses questions about the theoretical logic behind those predictions. We caution researchers using the upper-echelons model to carefully specify their assumptions about theoretical logic to assure that their results are not confounded with alternative explanations. We are upper-echelons researchers, and we intend to remain so. The model has spawned a huge array of research projects, and it has put managers back in the forefront of strategy research (Hambrick, 1989). Nevertheless, as research on the model accumulates, it is important to go beyond significant results to consider realistic pictures of how TMTs actually function and how strategic decisions are made in real-world ﬁrms. This chapter represents an attempt to provide some of that realism to the study of upper echelons, and we have tried to be complete in our discussion of what might stand in the way of a TMT functioning as predicted by the upper-echelons model. Finally, we have challenged the assumption that the team level is universally the best approach to use when studying upper echelons. We discussed a number of situations in which team-level analysis may, in fact, not predict outcomes as well as simply using the CEO’s psychological proﬁle and preferences. If the TMT is not co-located, or if it does not meet regularly as a group to discuss issues and scrutinize strategic decisions, then the predictions of the upper-echelons model are not likely to be observed. Additionally, if the CEO does not support open debate and dialog on the issues and choices, the team is not likely to act in the ways predicted by the upperechelons model. In all of these cases, the CEO’s individual-level preferences and biases will likely provide much stronger predictions about decisions about strategic decision making. We note that our discussion of levels issues in organizations has some parallels to the discussion of the Cuban missile crisis by Allison (1971). Allison explains the Cuban missile crisis using three very different theoretical models. His ﬁrst, the rational actor model, assumes that the organization acts like a single rational person. This approach is quite similar to our discussion of the homogeneous TMT, or the team led by a single dominant leader. Allison’s second model is the organizational process model, in which different

232

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

parts of the organization see the world differently and respond very differently to various stimuli. This model follows the assumption of heterogeneity in consolidating the actions taken to the organization level. Allison’s third model is a political model, in which different parts of the organization have different goals and objectives, and outcomes result from political processes including bargaining. All three models explain the ultimate outcome, but each successively explains in more detail the actual outcomes observed. Each theory also adds significantly more complexity relative to previous one(s). Understanding, however, requires an appreciation for the inherent complexity of strategic situations and organizational action.

REFERENCES Afuah, A. (2001). Dynamic boundaries of the ﬁrm: Are ﬁrms better off being vertically integrated in the face of a technological change? Academy of Management Journal, 44, 1211–1228. Allison, G. T. (1971). Essence of decision: Explaining the Cuban missile crisis. Boston, MA: Little, Brown. Amason, A. C. (1996). Distinguishing the effects of functional and dysfunctional conﬂict on strategic decision making: Resolving a paradox for top management teams. Academy of Management Journal, 39, 123–148. Bantel, K. A., & Jackson, S. E. (1989). Top management and innovations in banking: Does the composition of the top team make a difference? Strategic Management Journal, 10, 107–124. Boal, K. B., & Hooijberg, R. (2001). Strategic leadership research: Moving on. Leadership Quarterly, 11, 515–549. Bourgeois, L. J., III (1980). Performance and consensus. Strategic Management Journal, 1, 227–248. Cannella, A. A., Jr., Fraser, D. R., & Lee, D. S. (1995). Firm failure and managerial labor markets: Evidence from Texas banking. Journal of Financial Economics, 38, 185–210. Cannella, A. A., Jr., & Lubatkin, M. (1993). Succession as a sociopolitical process: Internal impediments to outsider selection. Academy of Management Journal, 36, 763–793. Cannella, A. A., Jr., & Shen, W. (2001). So close and yet so far: Promotion versus exit for CEO heirs apparent. Academy of Management Journal, 44, 252–270. Carpenter, M. A. (2002). The implications of strategy and social context for the relationship between top management team heterogeneity and ﬁrm performance. Strategic Management Journal, 23, 275–284. Castanias, R. P., & Helfat, C. E. (1991). Managerial resources and rents. Journal of Management, 17, 155–171. Castanias, R. P., & Helfat, C. E. (2001). The managerial rents model: Theory and empirical analysis. Journal of Management, 27, 661–678. Chattopadhyay, P., Glick, W. H., Miller, C. C., & Huber, G. P. (1999). Determinants of executive beliefs: Comparing functional conditioning and social inﬂuence. Strategic Management Journal, 20, 763–789.

A Multi-Level Analysis of the Upper-Echelons Model

233

Child, J. (1972). Organization structure, environment and performance: The role of strategic choice. Sociology, 6, 1–22. Coffee, J. C., Jr. (1988). Shareholders versus managers: Strain in the corporate web. In: J. C. Coffee Jr., L. Lowenstein & S. Rose-Ackerman (Eds), Knights, raiders and targets: The impact of the hostile takeover (pp. 77–134). New York: Oxford University Press. Cyert, R. M., & March, J. G. (1963). A behavioral theory of the firm. Englewood Cliffs, NJ: Prentice-Hall. Daily, C. M., Certo, S. T., & Dalton, D. R. (1999). A decade of corporate women: Some progress in the boardroom, none in the executive suite. Strategic Management Journal, 20, 93–99. Dalton, D. R., & Kesner, I. F. (1983). Inside/outside succession and organizational size: The pragmatics of executive replacement. Academy of Management Journal, 26, 736–742. Dansereau, F., Alutto, J., & Yammarino, F. J. (1984). Theory testing in organizational behavior: The varient approach. Englewood Cliffs, NJ: Prentice-Hall. Dansereau, F., Yammarino, F. J., & Kohles, J. C. (1999). Multiple levels of analysis from a longitudinal perspective: Some implications for theory building. Academy of Management Review, 24, 346–357. Davis, G. F., & Stout, S. K. (1992). Organization theory and the market for corporate control: A dynamic analysis of the characteristics of large takeover targets, 1980–1990. Administrative Science Quarterly, 37, 605–633. Dearborn, D. C., & Simon, H. A. (1958). Selective perception: A note on the departmental afﬁliations of executives. Sociometry, 21, 144–150. Dutton, J. E., & Ashford, S. J. (1993). Selling issues to top management. Academy of Management Review, 18, 397–428. Dutton, J. E., & Duncan, R. B. (1987). The creation of momentum for change through the process of strategic issue diagnosis. Strategic Management Journal, 8, 279–295. Dutton, J. E., Fahey, L., & Narayanan, V. K. (1983). Toward understanding strategic issue diagnosis. Strategic Management Journal, 4, 307–323. Dutton, J. E., Frost, P. J., Worline, M. C., Lilius, J. M., & Kanov, J. M. (2002). Leading in times of trauma. Harvard Business Review, 80, 54–62. Dutton, J. E., & Jackson, S. E. (1987). Categorizing strategic issues: Links to organizational action. Academy of Management Review, 12, 76–90. Eisenhardt, K. M., & Bourgeois, L. J. (1988). Politics of strategic decision making in highvelocity environments: Toward a midrange theory. Academy of Management Journal, 31, 737–770. Eisenhardt, K. M., Kahwajy, J. L., & Bourgeois, L. J., III. (1997). How management teams can have a good ﬁght. Harvard Business Review, 75, 77–85. Fairhurst, G. T., & Sarr, R. A. (1996). The art of framing: Managing the language of leadership. San Francisco: Jossey-Bass. Finkelstein, S. (1992). Power in top management teams: Dimensions, measurement, and validation. Academy of Management Journal, 35, 505–538. Finkelstein, S., & Hambrick, D. C. (1989). Chief executive compensation: A study of the intersection of markets and political processes. Strategic Management Journal, 10, 121–134. Finkelstein, S., & Hambrick, D. C. (1990). Top management team tenure and organizational outcomes: The moderating role of managerial discretion. Administrative Science Quarterly, 35, 484–503.

234

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

Finkelstein, S., & Hambrick, D. C. (1996). Strategic leadership: Top executives and their effects on organizations. Minneapolis, St. Paul: West Publishing. Fiol, C. M. (1994). Consensus, diversity, and learning in organizations. Organization Science, 5, 403–420. Gabarro, J. J. (1987). The dynamics of taking charge. Boston, MA: Harvard Business School Press. Hambrick, D. C. (1989). Guest editor’s introduction: Putting top managers back in the strategy picture. Strategic Management Journal, 10, 5–15. Hambrick, D. C. (1994). Top management groups: A conceptual integration and reconsideration of the ‘‘team’’ label. In: B. M. Staw & L. L. Cummings (Eds), Research in organizational behavior (pp. 171–214). Greenwich, CT: JAI Press. Hambrick, D. C., & Cannella, A. A., Jr. (1989). Strategy implementation as substance and selling. Academy of Management Executive, 3, 278–285. Hambrick, D. C., & Cannella, A. A., Jr. (1993). Relative standing: A framework for understanding departures of acquired executives. Academy of Management Journal, 36, 733–762. Hambrick, D. C., & Cannella, A. A., Jr. (2004). CEOs who have COOs: Contingency analysis of an unexplored structural form. Strategic Management Journal, 25, 1097–1119. Hambrick, D. C., & Finkelstein, S. (1987). Managerial discretion: A bridge between polar views of organizations. In: L. L. Cummings & B. M. Staw (Eds), Research in organizational behavior, (Vol. 9, pp. 369–406). Greenwich, CT: JAI Press. Hambrick, D. C., Finkelstein, S., & Mooney, A. C. (2003). Executive job demands: Missing link in upper-echelons theory. Working Paper. Hambrick, D. C., & Fukutomi, G. D. S. (1991). The seasons of a CEO’s tenure. Academy of Management Review, 16, 719–742. Hambrick, D. C., & Mason, P. (1984). Upper echelons: The organization as a reﬂection of its top managers. Academy of Management Review, 9, 193–206. Haveman, H. A. (1993). Ghosts of managers past: Managerial succession and organizational mortality. Academy of Management Journal, 36, 864–881. Helmich, D. L., & Brown, W. B. (1972). Successor type and organizational change in the corporate enterprise. Administrative Science Quarterly, 17, 371–378. Henderson, A. D., & Fredrickson, J. W. (2001). Top management team coordination needs and the CEO pay gap: A competitive test of economic and behavioral views. Academy of Management Journal, 44, 96–117. Hill, C. W. L., Hitt, M. A., & Hoskisson, R. E. (1992). Cooperative versus competitive structures in related and unrelated diversiﬁed ﬁrms. Organization Science, 3, 501–521. Hoffman, R. C., & Hegarty, W. H. (1993). Top management inﬂuence on innovations: Effects of executive characteristics and social culture. Journal of Management, 19, 549–574. Hoskisson, R. E., & Hitt, M. A. (1988). Strategic control systems and relative R&D investment in large multiproduct ﬁrms. Strategic Management Journal, 9, 605–621. House, R. J., Rousseau, D. M., & Thomas-Hunt, M. (1995). The meso paradigm: A framework for the integration of micro and macro organizational behavior. In: B. M. Staw & L. L. Cummings (Eds), Research in organizational behavior, (Vol. 17, pp. 71–114). Greenwich, CT: JAI Press. Howell, J. M., & Higgins, C. A. (1990). Champions of technological innovation. Administrative Science Quarterly, 35, 317–341.

A Multi-Level Analysis of the Upper-Echelons Model

235

Jensen, M. C. (1986). Agency costs of free cash ﬂow, corporate ﬁnance and takeovers. American Economic Review, 76, 323–329. Kahneman, D., & Tversky, A. (1979). Prospect theory: Analysis of decision making under risk. Econometrica, 47, 262–291. Kets de Vries, M. F. R. (1998). Charisma in action: The transformational abilities of Virgin’s Richard Branson and ABB’s Percy Barnevik. Organizational Dynamics, 26, 6–21. Kilduff, M. R., Angelmar, R., & Mehra, A. (2000). Top management team diversity and ﬁrm performance: Examining the role of cognitions. Organization Science, 11, 21–34. Klein, K. J., Dansereau, F., & Hall, R. J. (1994). Levels issues in theory development, data collection, and analysis. Academy of Management Review, 19, 195–229. Klein, K. J., & House, R. J. (1995). On ﬁre: Charismatic leadership and levels of analysis. Leadership Quarterly, 6, 183–198. Klein, K. J., Tosi, H. L., Jr., & Cannella, A. A., Jr. (1999). Multilevel theory building: Beneﬁts, barriers, and new developments. Academy of Management Review, 24, 243–248. Kogut, B., & Kulatilaka, N. (2001). Capabilities as real options. Organization Science, 12, 744–758. Kotter, J. P. (1982). The general managers. New York: Free Press. Kotter, J. P. (1990). What leaders really do. Harvard Business Review, 68, 103–111. Lawrence, B. S. (1988). New wrinkles in the theory of age: Demography, norms, and performance ratings. Academy of Management Journal, 31, 309–337. Lorsch, J. W. (1989). Pawns or potentates: The reality of America’s boards. Boston, MA: Harvard Business School Press. March, J. C., & Simon, H. A. (1958). Organizations. New York: Wiley. March, J. G. (1962). The business ﬁrm as a political coalition. Journal of Politics, 24, 662–678. Miller, D. (1991). Stale in the saddle: CEO tenure and the match between organization and environment. Management Science, 37, 34–52. Miller, D., & Droge, C. (1986). Psychological and traditional determinants of structure. Administrative Science Quarterly, 31, 539–560. Miller, D., & Shamsie, J. (2001). Learning across the life cycle: Experimentation and performance among the Hollywood studio heads. Strategic Management Journal, 22, 725–745. Miller, D., & Toulouse, J.-M. (1986). Chief executive personality and corporate strategy and structure in small ﬁrms. Management Science, 32, 1389–1409. Mintzberg, H. (1973). The nature of managerial work. New York: Harper & Row. Mintzberg, H. (1985). The organization as political arena. Journal of Management Studies, 22, 133–154. Mintzberg, H., Raisinghani, D., & Theoret, A. (1976). The structure of unstructured decision processes. Administrative Science Quarterly, 21, 246–275. Mischel, W. (1977). The interaction of person and situation. In: D. Magnusson & N. S. Endler (Eds), Personality at the crossroads: Current issues in interactional psychology (pp. 333–352). Hillsdale, NJ: Erlbaum. Nutt, P. C. (1993). The formulation processes and tactics used in organizational decision making. Organization Science, 4, 226–251. Nutt, P. C. (1998). How decision makers evaluate alternatives and the inﬂuence of complexity. Management Science, 44, 1148–1166. Ocasio, W. (1994). Political dynamics and the circulation of power: CEO succession in U.S. industrial corporations, 1960–1990. Administrative Science Quarterly, 39, 285–312.

236

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

Ocasio, W. (1997). Towards an attention-based view of the ﬁrm. Strategic Management Journal, 18, 187–206. Park, J.-H., Lee, H. U., & Cannella, A. A., Jr. (2004). Top management team diversity and firm performance: Examining the roles of external and internal contexts. Unpublished manuscript, Seoul, Korea. Pettigrew, A. (1973). The politics of organizational decision making. London: Tavistock. Pfeffer, J. (1981). Power in organizations. Boston, MA: Pitman. Priem, R. L., Harrison, D. A., & Muir, N. K. (1995). Structured conﬂict and consensus outcomes in group decision making. Journal of Management, 21, 691–710. Priem, R. L., Rasheed, A. M. A., & Kotulic, A. G. (1995). Rationality in strategic decision processes, environmental dynamism and ﬁrm performance. Journal of Management, 21, 913–929. Reutzel, C. R., & Cannella, A. A. Jr. (2004). A model of chief financial officer promotion and exit. Paper presented at the Annual meetings of the Academy of Management, New Orleans, LA. Roberto, M. A. (2003). The stable core and dynamic periphery in top management teams. Management Decision, 41, 120–131. Rousseau, D. M. (1985). Issues of level in organizational research: Multi-level and cross-level perspectives. In: B. M. Staw & L. L. Cummings (Eds), Research in organizational behavior, (Vol. 7, pp. 1–37). Greenwich, CT: JAI Press. Schendel, D. E., & Hofer, C. W. (Eds) (1979). Strategic management: A new view of business policy and planning. Boston: Little, Brown. Schilling, M. A., & Hill, C. W. L. (1998). Managing the new product development process: Strategic imperatives. Academy of Management Executive, 12, 67–81. Shamir, B. (1995). Social distance and charisma: Theoretical notes and an exploratory study. Leadership Quarterly, 6, 19–47. Shen, W., & Cannella, A. A., Jr. (2002a). Political dynamics within top management and their impacts on CEO dismissal followed by inside succession. Academy of Management Journal, 45, 1195–1208. Shen, W., & Cannella, A. A., Jr. (2002b). Revisiting the performance consequences of CEO succession: The impacts of successor type, postsuccession senior executive turnover, and departing CEO tenure. Academy of Management Journal, 45, 717–733. Simon, H. A. (1945). Administrative behavior. New York: Free Press. Simons, T. (1995). Top management team consensus, heterogeneity, and debate as contingent predictors of company performance: The complementarity of group structure and process. Academy of Management Best Papers Proceedings, 62–66. Sitkin, S. B., & Weingart, L. R. (1995). Determinants of risky decision-making behavior: A test of the mediating role of risk perceptions and propensity. Academy of Management Journal, 38, 1573–1592. Starbuck, W. H., & Milliken, F. (1988). Executives’ perceptual ﬁlters: What they notice and how they make sense. In: D. C. Hambrick (Ed.), The executive effect: Concepts and methods for studying top managers (pp. 35–65). Greenwich, CT: JAI Press. Staw, B. M. (1981). The escalation of commitment to a course of action. Academy of Management Review, 6, 55–100. Staw, B. M., Sandelands, L. E., & Dutton, J. E. (1981). Threat rigidity effects in organizational behavior: A multilevel analysis. Administrative Science Quarterly, 26, 501–524. Stein, J. C. (1988). Takeover threats and managerial myopia. Journal of Political Economy, 96, 61–80.

A Multi-Level Analysis of the Upper-Echelons Model

237

Sutton, R. I., & Staw, B. M. (1995). What theory is not. Administrative Science Quarterly, 40, 371–384. Teerlink, R. (2000). Harley’s leadership U-turn. Harvard Business Review, 78, 43–48. Tichy, N. M. (1989). GE’s Crotonville: A staging ground for corporate revolution. Academy of Management Executive, 3, 99–106. Tichy, N. M., & Charan, R. (1989). Speed, simplicity, self-conﬁdence: An interview with Jack Welch. Harvard Business Review, 67, 112–120. Turk, T. A. (1992). Takeover resistance, information leakage, and target ﬁrm value. Journal of Management, 18, 503–522. Tushman, M. L., & Romanelli, E. (1985). Organizational evolution: A metamorphosis model of convergence and reorientation. In: L. L. Cummings & B. M. Staw (Eds), Research in organizational behavior, (Vol. 7, pp. 171–222). Greenwich, CT: JAI Press. Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211, 453–458. Tversky, A., & Kahneman, D. (1986). Rational choice and the framing of decisions. Journal of Business, 59, S251–S278. Vancil, R. F. (1979). Decentralization: Managerial ambiguity by design. New York: Financial Executives Research Foundation. Vancil, R. F. (1987). Passing the baton. Boston, MA: Harvard Business School Press. Varadarajan, P. R., & Ramanujam, V. (1990). The corporate performance conundrum: A synthesis of contemporary views and an extension. Journal of Management Studies, 27, 463–483. Waldman, D. A., Ramirez, G. G., House, R. J., & Puranam, P. (2001). Does leadership matter? CEO leadership attributes and profitability under conditions of perceived environmental uncertainty. Academy of Management Journal, 44, 134–143. Waldman, D. A., & Yammarino, F. J. (1999). CEO charismatic leadership: Levels-of-management and levels-of-analysis effects. Academy of Management Review, 24, 266–285. Walkling, R. A., & Long, M. S. (1984). Agency theory, managerial welfare, and takeover bid resistance. RAND Journal of Economics, 15, 54–68. Walsh, J. P. (1988). Selectivity and selective perception: An investigation of managers’ belief structures and information processing. Academy of Management Journal, 31, 873–896. Walsh, J. P. (1995). Managerial and organizational cognition: Notes from a trip down memory lane. Organization Science, 6, 280–321. Weick, K. E. (1979). The social psychology of organizing. Reading, MA: Addison-Wesley. Weick, K. E. (1995). Sensemaking in organizations. Thousand Oaks, CA: Sage Publications. Wiersema, M. F., & Bantel, K. A. (1992). Top management team demography and corporate strategic change. Academy of Management Journal, 35, 91–121. Wiseman, R. M., & Gomez-Mejia, L. R. (1998). A behavioral agency model of managerial risk taking. Academy of Management Review, 23, 133–153. Yukl, G. (1989). Managerial leadership: A review of theory and research. Journal of Management, 15(2), 251–289. Zajac, E. J., & Westphal, J. D. (1996). Who shall rule after a CEO succession? The likelihood and direction of changes in CEO characteristics. Administrative Science Quarterly, 41, 507–529. Zald, M. N. (1965). Who shall rule? A political analysis of succession in a large welfare organization. Pacific Sociological Review, 8, 52–60.

This page intentionally left blank

238

MOVING (FINALLY) TOWARD A MULTI-LEVEL MODEL OF THE UPPER ECHELONS Mason A. Carpenter ABSTRACT This chapter examines Cannella and Holcomb’s (this volume) multi-level analysis of Hambrick and Mason’s ((1984). Upper echelons: The organization as a reflection of its top managers. Academy of Management Review, 9, 193–206) original upper echelons perspective, and the strategy formulation studies that have consequently employed that perspective. I highlight five key contributions made by Cannella and Holcomb (this volume), and suggest three supplementary avenues for inquiry. I close by arguing that if we are aiming to encourage researchers to move to a truly multi-level upper echelons model, then such a multi-level emphasis must encompass both strategy formulation and strategy implementation given their implicit interdependence.

INTRODUCTION It might come across as somewhat unorthodox for a chapter to open by offering kudos to a chapter’s authors. However, my ﬁrst comment that it is Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 239–247 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04010-5

239

240

MASON A. CARPENTER

about time that such significant time, care, and intelligence has been invested in an upper echelons perspective (UEP) review of this nature. My rationale for this starting point is that, despite the fact Hambrick and Mason’s (1984) UEP has been invoked in well over 500 refereed journal articles since its publication – ranging from management to psychology to economics – there have been only three systematic reviews of the UEP in the past 20 years (see Jackson, 1992; Finkelstein & Hambrick, 1996; Carpenter, Sanders, & Geletkanycz, 2004). Moreover, not only do Cannella and Holcomb (this volume) provide such a comprehensive overview, they also create new opportunities for future research by pointing out that the published UEP theory and research to date has operated as if it was based on group-level analysis, when in fact the operationalization of the model has predominantly resided at the individual level. I have always viewed the UEP as an inherently multi-level framework since research in this vein, including my own, often involves the external environment, board members, chief executives (CEOs), executives, and downstream outcomes. In reading Cannella and Holcomb (this volume) I was taken aback at the accuracy of their assessment of the mismatch between the UEP and its employment in research. On the one hand, this inconsistency between the individual-level emphasis of extant studies and the inherent multi-level nature of the UEP is problematic. On the other, the identiﬁcation of this inconsistency provides the type of challenge that can sow great prospects for valuable UEP research contributions for decades to come. In this chapter I start by noting ﬁve key contributions made by Cannella and Holcomb (this volume), and then move on to provide three additional suggestions as to how researchers might further change their research practices to achieve Cannella and Holcomb’s (this volume) objective that the UEP be multi-level in both theory and application. I close the chapter by reiterating Cannella and Holcomb’s (this volume) call for broadening the multi-level application of the UEP to include strategy implementation, in addition to strategy formulation.

KEY CONTRIBUTIONS Cannella and Holcomb (this volume) make ﬁve interrelated contributions in this piece that should be highlighted. First, one key assumption maintained in most UEP research, although appropriately brought into question by Cannella and Holcomb (this volume), is that top management team (TMT) members operate as a collection of equally empowered individuals, when in

Moving (Finally) Toward a Multi-Level Model of the Upper Echelons

241

fact there are various forces at play which are likely to make their impact on TMT decision making unequal. In the section on ‘‘Heterogeneity among TMT members,’’ Cannella and Holcomb (this volume) note that Hambrick and Mason (1984) suggest that the UEP essentially has this complexity at heart, but that most research employing the UEP has instead opted to adopt the equal empowerment assumption. The importance of showing how and when team members may differentially affect TMT processes is of course obvious when we anecdotally distinguish between CEOs, particularly those who are both the CEO and chairperson of the board, and other members of the TMT. Although some observers have decried that we are seeing the demise of the imperial CEOs (Martin, 2003), the CEOs clearly stand apart from the other executives in terms of their responsibilities, power, and inﬂuence. Moreover, even when they do not hold both the CEO and chairperson positions, increasingly they are the only inside executive on the board of directors. This structural difference, in and of itself, suggests that there are multiple levels of analysis implicit in even the most egalitarian TMTs. The second contribution, relating again to the CEO, is made when Cannella and Holcomb (this volume) revisit ‘‘CEO Seasons and Cycles’’ and ‘‘CEO Succession.’’ Given our own intimate experiences as academics serving or observing committee or department chairs and even deans, it is ironic how little the dimension of time (aside from tenure) has been incorporated into research on the UEP. From a practical standpoint, based on studies by Booz Allen and other consulting ﬁrms we know that the ‘‘honeymoon period’’ for new CEOs is substantially shorter than in years past, which by itself suggests that we should see multi-level effects manifest themselves more quickly, and perhaps more dramatically, in and around the TMT decision-making process. There is no reason to suspect that such dynamics will not spill over into the complexities surrounding CEO succession, as Cannella and Holcomb (this volume) suggest. A third contribution, starting even earlier in the TMT literature with Ancona and Nadler (1989), is to remind us that there are many possible designs that TMTs may adopt or evolve into. These designs, in turn, have implications for the composition, structure, and succession issues that arise, further buttressing Cannella and Holcomb’s (this volume) assertion that in even the simplest team designs there are multiple levels of analysis regarding the work, internal, and external relationships to be managed by the various actors concerned. Finkelstein (1992) looks at this issue empirically by suggesting that each individual TMT member has a different power base, and therefore those power differences will augment or decrease their unique

242

MASON A. CARPENTER

inﬂuence. Again, however, this view still treats TMT members as unitary actors, with the added nuance that they have differing levels of inﬂuence. Putting a slightly different spin on this perspective, Jackson (1992) notes that to the extent that a TMT is comprised of individuals with similar backgrounds, the degree of their collective inﬂuence increases. For example, one marketing person on the TMT will have less independent effect on decision making than would be the case if there were two or three with marketing backgrounds, and so on. I think that Jackson’s perspective, combined with Finkelstein (1992), is moving the UEP closer to the multilevel model to which Cannella and Holcomb (this volume) aspire. In many ways, this suggests that separately and jointly the relationship effects of TMT members should be more rightly compared to relationships and effects of political parties in coalition governments, rather than parties in simple one-person-one-vote democracies. The reader may miss Cannella and Holcomb’s (this volume) fourth key contribution in the section on ‘‘TMT Decision-Making Processes’’ because it comes across so subtly, and is implicit as opposed to explicit. I would characterize this contribution as suggesting a research study on ‘‘who sets the TMT and board meeting agendas,’’ and ‘‘who has the most inﬂuence on getting items on and off this agenda.’’ This section reminds us how little, yet, we actually understand the processes leading up to the TMT decision-making process and how little they are studied. I believe that the questions of how decisions are framed, and when, and by whom they are made become all the more important by the practical fact that board and TMT meetings are increasingly dominated with legal and bureaucratic issues related to corporate governance, and less and less by issues central to the strategic direction of the ﬁrm. If Hamel and Prahalad (1994) concluded that most TMTs spend less than 4% of their time developing a shared view of the ﬁrm’s future and its strategic intent, then imagine how dramatically that ﬁgure must have fallen a decade later in the face of today’s relentless assault on management’s time by bureaucratic governance minutia. Similarly, such scarcity of time gives even more dimensionality to the TMT decision-making process by virtue of who has inﬂuence over what gets on the TMT’s ultimate meeting agenda. It is also intriguing to consider, as Cannella and Holcomb (this volume) suggest, that one additional consequence of agenda framing may be that the effects of TMT heterogeneity become less pronounced once the strategic issues have been deﬁned. However, recall that Cannella and Holcomb (this volume) are predominantly focused on strategy formulation, and not on strategy implementation. So while it may be true that a set agenda could

Moving (Finally) Toward a Multi-Level Model of the Upper Echelons

243

decrease the effects of TMT heterogeneity on further changes in strategy, or even items on a meeting’s agenda, there is overwhelming evidence that heterogeneity will significantly impact the speed and effectiveness of strategy implementation (Hambrick, Cho, & Chen, 1996; Williams & O’Reilly, 1998). The ﬁfth contribution, which I will also use as a segue to the next section of my chapter, is that Cannella and Holcomb (this volume) bring the Roberto (2003) piece to the attention of UEP researchers. While not the ﬁrst researcher to make this observation (see Jackson, 1992, for instance), Roberto (2003) succinctly lays out the logic that TMTs are likely to be comprised of a stable core and a dynamic periphery. It is important to point out that his paper is written from a decision-making perspective, since Roberto’s (2003) argument is essentially that for any particular strategic decision there will always be a common core of decision makers. In contrast, for any given decision there is also likely to be a variance in the composition of decision makers according to the nature of the strategic decision being made. What is striking about Roberto’s (2003) work is that he backs it up with an empirical small-sample ﬁeld study combined with a large-sample survey study of 78 business unit managers from 73 different Fortune 500 ﬁrms. In this large-sample study he found that a small stable team of managers spent a great deal of time with strategy execution, and relatively little time with strategy formulation. At the same time, when strategic decisions were made, he showed that different sets of individuals worked with the CEO to make specific strategic choices during strategy formulation. One conclusion that Roberto (2003) drew from the ﬁeld and survey studies was that an executive’s involvement in any particular decision depended on their expertise. This means that measurement of the composition of the TMT would vary by decision. However, composition was also found be a function of prior personal working relationships or the degree to which it might affect the decision’s eventual implementation. These three TMT composition-determining factors all have important implications for Cannella and Holcomb’s (this volume) efforts to encourage moving away from a single level of analysis with regards to the UEP.

MOVING FURTHER FORWARD Beyond offering kudos to the authors, my chapter aims to address three additional issues not explicitly considered by Cannella and Holcomb (this

244

MASON A. CARPENTER

volume), but which are nonetheless highly pertinent to the multi-level perspective inherent in the UEP. These three issues are: (1) the UEP as both a theory and a methodology, (2) determining who is in the TMT, and (3) extending the definition of the UEP group boundaries to include the board of directors and external consultants. I close by reiterating and perhaps amplifying Cannella and Holcomb’s (this volume) call for research that considers decision implementation. After all, top managers spend about 20% of their time formulating strategies, versus 80% of their time implementing them, and formulation and implementation are inextricably intertwined. First, the UEP brought forth much more than a theory. The UEP also gave rise to a new methodology for empirically considering the impact of CEOs and TMTs on their ﬁrms’ strategies and performance. The impact of this new methodology is so dramatic that it pervades research ranging far aﬁeld of strategy and management including disciplines such as psychology and economics (see Carpenter et al., 2004, for a comprehensive review of this diffusion). The research noted above found over 500 refereed journal articles referring to the UEP through the end of 2003, and Hambrick and Mason (1984) in particular. Moreover, the citation rate of nearly 25 refereed journal articles per year appears to continue unabated, with at least 30% of those works citing Hambrick and Mason (1984) for its methodology, not its theory. Specifically, many new works are citing the UEP in support of using managerial demographic characteristics as proxies for underlying psychological and behavioral characteristics. Why is the distinction between the UEP as a theory or a methodology important? For one, theories are based on theoretical constructs and the relationships, sometimes causal, among them. In contrast, methodologies are aimed at allowing us to gauge those constructs with accuracy such that we may make inferences about the predictive or descriptive validity of the theory (Schwab, 1999). As demonstrated by Roberto (2003), if the methodology is operationalized using archival sources of data, such as Dunn and Bradstreet’s Reference Book of Corporate Managements or corporate proxy or 10-K statements, then it is assumed that the TMT as a theoretical construct (i.e., the dominant coalition) has been correctly identiﬁed. However, even when archival measures are used there is variance in how they are applied, which in turn has implications for multi-level analysis. For instance, Bunderson and Sutcliffe (2003) showed that different operationalizations of TMT functional heterogeneity had revealed different relationships with strategic outcomes. If such archival measures of the TMT are unreliable measures of the construct, regardless of how the measure is operationalized, then either no

Moving (Finally) Toward a Multi-Level Model of the Upper Echelons

245

results or spurious results could be a consequence. However, UEP researchers often argue that weak measures make for a strong test of theory. In some ways, Roberto’s (2003) ﬁnding of a stable core suggests that perhaps extant archival-based TMT studies are at least tapping aspects of the TMT construct. Whether or not the weak-measurement argument holds water continues to be a matter of academic debate. As a practical matter, authors typically point to their results as evidence supporting the use of archival measures. Second, a related and direct consequence of the UEP as a methodology (versus the UEP as a theory) is the need, from an empirical standpoint, to determine whether or not the research question drives the operational definition of the TMT or if the operational definition of the TMT drives the research question. From Cannella and Holcomb’s (this volume) and Roberto’s (2003) perspective, it appears that the appropriate definition of the TMT (as a construct) should be based on the strategic decision being considered (i.e., strategic decision making, another construct). Such constructs would imply an implicit theory that the composition of the TMT, correctly measured, will be reﬂected in certain patterns of strategic decision making. However, an equally plausible theory would hold that those who are at the highest levels of the organization are likely to have a marked impact on organizational decisions if they are strategic in nature. The latter theory, in its pure form, is the one applied most commonly in papers adopting the UEP. The fact that these two opposing views exist suggests a good research opportunity that will also contribute to the dialogue of the UEP and multilevel analysis. As Carpenter et al. observe, ‘‘A related source of study heterogeneity may rest with the use of archival versus survey data for the identiﬁcation of TMT characteristics. Since the sampled TMT members are meant to be representative of the dominant coalition construct, an important opportunity exists to compare the demographic proﬁles of TMTs calculated with public data (from proxies, directories, etc.) with those based on primary data provided by the same ﬁrm, typically the CEO. Such a study may show, for instance, whether the source of data matters. To the extent that there is agreement across data sources, the data provide a reasonable indicator of the underlying theoretical construct’’ (2004, p. 770). Such a study would move Roberto’s (2003) ideas one step forward, with the potential to resolve a debate that has been going on since the publication of Hambrick and Mason (1984). The third related issue is how broadly we cast our net in operationally deﬁning who does and does not constitute the TMT. It is on this point where

246

MASON A. CARPENTER

the concerns surrounding the UEP as methodology and Cannella and Holcomb’s (this volume) efforts to move the perspective to consider multiple, interrelated levels of analysis most closely coincide. Specifically, should the TMT be deﬁned to include all members of the board of directors, or a supra-TMT as deﬁned by Finkelstein and Hambrick (1996)? What about external but inﬂuential advisors such as consultants, investment bankers, or executives’ spouses? Just to give you some idea of the complexity and relevance of this issue, Jensen and Zajac (2004) tested the supra-TMT notion and found that the characteristics of CEOs, outside directors, and non-CEO TMT members generated very different strategy formulation outcomes (i.e., effects on diversiﬁcation and acquisitions). They ultimately concluded that it was problematic to use highly aggregated definitions of the TMT. Moreover, by disaggregating the TMT into subcomponents (e.g., in their case, a CEO with a ﬁnance background) they were able to show that the aggregate-level effects masked a positive ﬁnance CEO effect and a negative non-ﬁnance CEO effect. While their research does not address the inclusion of other inﬂuential stakeholders, at the very least it reafﬁrms the multiple levels of analysis inherent in the UEP. In closing, kudos to the authors aside, I must reiterate the call for UEP research that focuses on strategy execution, not simply strategy formulation and performance. Admittedly, strategy formulation and implementation are inextricably intertwined, so in some small way Cannella and Holcomb’s (this volume) consideration of the TMT strategic decision-making process leading up to formulation also has implications for implementation. The importance of this formulation/implementation interdependence is Roberto’s (2003) ﬁnding that TMT members were sometimes included in formulation decisions when they would have an inﬂuence, the eventual execution of the decision. However, Cannella and Holcomb (this volume) have only touched the tip of the formulation/implementation iceberg in their paper, as it relates to a multi-level analysis of the UEP. One of the implications of greater attention to implementation is that it requires going into the black box of what managers do. With few notable exceptions such as Eisenhardt and company (see Cannella and Holcomb, this volume) for some of these references, some scholars since Mintzberg (1973) and Kotter (1982) have delved deeply back into that box. Most things have likely remained unchanged among the upper echelons and their context. However, given the dramatic dynamism in the external environment, one is just as likely to expect that we would learn how more, as opposed to less, has actually changed inside the black box in terms of the antecedents and consequences of TMT composition for strategy formulation and

Moving (Finally) Toward a Multi-Level Model of the Upper Echelons

247

decision making. Cannella and Holcomb (this volume) provide us with the appropriate springboard for embarking on such an ambitious and comprehensive research effort.

REFERENCES Ancona, D., & Nadler, D. (1989). Top hats and executive tales: Designing the senior team. Sloan Management Review, 31, 19–28. Carpenter, M. A., Sanders, W. G., & Geletkanycz, M. A. (2004). Upper echelons research revisiting: Antecedents, elements, and consequences of TMT composition. Journal of Management, 30, 749–778. Finkelstein, S. (1992). Power in top management teams: Dimensions, measurement, and validation. Academy of Management Journal, 35, 505–538. Finkelstein, S., & Hambrick, D. C. (1996). Strategic leadership: Top executives and their effects on organizations. Minneapolis: West Publishing. Hambrick, D. C., Cho, T. S., & Chen, M. (1996). The inﬂuence of top management team heterogeneity on ﬁrms’ competitive moves. Administrative Science Quarterly, 41, 659–684. Hambrick, D. C., & Mason, P. M. (1984). Upper echelons: The organization as a reﬂection of its top managers. Academy of Management Review, 9, 193–206. Hamel, G., & Prahalad, C. K. (1994). Competing for the future. Boston: HBS Press. Jackson, S. (1992). Consequences of group composition for the interpersonal dynamics of strategic issue processing. In: P. Shrivastava, A. Huff & J. Dutton (Eds), Advances in strategic management (pp. 345–382). Greenwich, CT: JAI Press. Jensen, M., & Zajac, E. (2004). Corporate elites and corporate strategy: How demographic preferences and structural differences shape the scope of the ﬁrm. Strategic Management Journal, 25, 507–524. Kotter, J. P. (1982). The general managers. New York: Free Press. Martin, J. (2003). Rise of the new breed. CEO Magazine August/September, 191, 25–29. Mintzberg, H. (1973). The nature of managerial work. New York: Harper & Row. Roberto, M. A. (2003). The stable core and dynamic periphery in top management teams. Management Decision, 41(2), 120–131. Schwab, D. P. (1999). Research methods for organizational studies. NJ: Erlbaum. Williams, K. Y., & O’Reilly, C. A. (1998). Demography and diversity in organizations: A review of 40 years of research. In: L. L. Cummings & B. M. Staw (Eds), Research in organizational behavior, (Vol. 20, pp. 77–140). Greenwich, CN: JAI Press.

This page intentionally left blank

248

UPPER ECHELONS PERSPECTIVE AND MULTI-LEVEL ANALYSIS: A CASE OF THE CART BEFORE THE HORSE? Dan R. Dalton and Catherine M. Dalton ABSTRACT Cannella and Holcomb ((this volume). In: F. Dansereau & F. J. Yammarino (Eds), Research in multi-level issues (Vol. 4). Oxford, UK: Elsevier Science) are unconvinced that top management teams (TMTs) are the appropriate level of analysis for upper echelons research and are, accordingly, unenthusiastic about the promise of multi-level analysis for research of this type. We agree and discuss (1) the fragility of agency theory as it pertains to TMT research, (2) various issues pertaining to TMT turnover (or lack thereof), (3) paradoxes in practice and theory regarding TMT homogeneity/heterogeneity, (4) the absence of boards of directors in the upper echelons perspective, and (5) the implications of these issues on the theory/conceptualization of TMTs and of the research dedicated to them. We question whether the variables, as currently configured, relied on in this literature are sufficiently developed to adequately test an upper echelons perspective, or to sensibly warrant a multi-level analytical approach.

Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 249–261 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04011-7

249

250

DAN R. DALTON AND CATHERINE M. DALTON

INTRODUCTION Cannella and Holcomb (this volume) have adroitly reviewed the conceptual foundations as well as much of the extant research addressing the upper echelons perspective and its potential application to multi-level analysis. For a host of well delineated reasons, Cannella and Holcomb are unconvinced of the promise for multi-level analysis for this stream of research. Moreover, they remain skeptical that the top management team (TMT) or management group (Hambrick, 1994) is the appropriate level of analysis when examining the upper echelons perspective (for reviews of upper echelons research and TMTs see, Carpenter, Geletkanycz, & Sanders, 2004; Daily & Schwenk, 1996; Finkelstein & Hambrick, 1996; Jackson, 1992). We are entirely comfortable with Cannella and Holcomb’s conclusions. Let us confess from the onset, however, that we recognize the paradox of our position. We could argue, in parallel with Cannella and Holcomb, that the academy simply does not have the research foundation at any given level to rely on multi-level analyses for enhancing contributions to the upper echelons literature. That said, we recognize others may suggest that without multi-level analyses, we will never fully understand the upper echelons perspective. At some risk of conservatism bordering, in the view of some, on intransigence, we adopt the former perspective. As such, we frame our discussion in the literature of strategic studies, as did Cannella and Holcomb. In subsequent sections, we discuss the mounting fragility of agency theory as its suggested interventions continue to be unsupported empirically. We also review a number of paradoxes implicit in the homogeneity/heterogeneity TMT literature. In addition, we describe some unsettling aspects of the absence of boards of directors, and others, in the upper echelons perspective. We conclude that many TMT variables are unsuited for the testing of an upper echelons perspective and, therefore, do not warrant a multi-level analysis.

SO MANY QUESTIONS, FAR FEWER ANSWERS For us, there are a host of questions that, in concert, undermine our conﬁdence in TMTs as the appropriate empirical unit of analysis for research relying on the upper echelon perspective. It follows that, those concerns undermine our conﬁdence in a multi-level analytical approach for that research. The issues we raise are in no particular order, as we realize that each

Upper Echelons Perspective and Multi-Level Analysis

251

will have a different utility, or none at all, for those interested in these areas of research. We present these, each in turn, without elaborate transition except to note that they are relevant to us as we assess the promise of TMTs, the upper echelon perspective, and multi-level analyses.

The Fragility of Agency Theory It seems reasonable that TMTs, the upper echelon perspective, and related concepts (e.g., dominant coalitions, Cyert & March, 1963) have an underlying foundation of ﬁdelity to the enterprise. Carpenter et al. (2004, p. 750), for example, nicely capture the basis of the upper echelons framework: ‘‘ythe situation is enacted by the TMT, enactment leads to strategic choices, and those choices affect performance.’’ But, there is compelling evidence in recent years (Carpenter et al., 2004) that such ‘‘enactments,’’ while perhaps in the interests of the TMT and certainly for some of its members, may not be in the interests of the ﬁrm or its critical constituencies. Beyond that, recent legislation (e.g., Sarbanes-Oxley, 2002), guidelines by the various stock exchanges (e.g., Nasdaq, NYSE), and the activities of the Justice Department would suggest that these TMT ‘‘enactments’’ have not been uniformly celebrated. In sum, there would appear to be a generalized recognition that senior executives sometimes act with considerable self-interest. Moreover, some would argue that agency theory itself, if not defeated, is in full retreat. It is fair to say that agency theory has served as a dominant theme for strategic leadership research. Grounded largely in the seminal work of Berle and Means (1932), agency theory has been suggested as the means by which high-ranking corporate ofﬁcers may pursue courses of action consistent with their own interests, rather than the interests of owners (e.g., Eisenhardt, 1989; Jensen & Meckling, 1976; Shleifer & Vishny, 1997). It is this potential conﬂict of interests that is a central focus of corporate governance research, of which strategic leadership studies are an important component. Macey (1997, p. 602) explained that ‘‘corporate governance can be described as the processes by which investors attempt to minimize the transactions costs (Coase, 1937) and agency costs (Jensen & Meckling, 1976) associated with doing business within a ﬁrm.’’ On the basis of the extant literature, agency interventions including boards of directors’ oversight, equity holdings by managers and directors, equity holdings by institutional investors, and an efﬁcient market for corporate control, do not seem to be uniformly effective (e.g., Dalton, Daily, Certo, & Roengpitya, 2003; Dalton,

252

DAN R. DALTON AND CATHERINE M. DALTON

Daily, Ellstrand, & Johnson, 1998; Davis, Schoorman, & Donaldson, 1997; Lane, Cannella, & Lubatkin, 1998; Shleifer & Vishny, 1997; Walsh & Kosnik, 1993). As we noted, Sarbanes-Oxley, the new stock exchange guidelines, and the activities of the Justice Department would seem to illustrate public policy faith in the current promise of agency theory.

If TMTs Are the Key, Why Is the Lock So Rarely Opened? There continues to be extensive, multidisciplinary commentary and research on CEO succession/turnover (e.g., Engel, Hayes, & Wang, 2003; Farrell & Whidbee, 2003; Parrino, Sias, & Starks, 2003; Shen & Cannella, 2002, 2003; Wasserman, 2003; Zhang & Rajagopalan, 2004) and work, examining the relationship between several CEO ‘‘variables’’ and organizational outcomes (Cheng, 2004; Grinstein & Hribar, 2004; Hambrick & Cannella, 2004; McDonald & Westphal, 2003; Murphy & Zabojnik, 2004; Shen, 2003). This is a vast literature, in contrast to the more modest research exposure to TMT turnover/succession and organizational outcomes (for an exception, see Shen & Cannella, 2002). In what we readily admit is something less than a scientific survey, we conducted a computer-aided search (EBSCO – Business Source Premier) of the expression ‘‘CEO’’ and ‘‘chief executive ofﬁcer’’ in the title line for peerreviewed journals over a 15-year period (1989–2004). There were just over 1,400 hits. We did the same for ‘‘TMT’’ and ‘‘top management team’’ and 98 were noted. When we open the search to all sources (i.e., without using the ‘‘peer-review’’ ﬁlter), there are some 15,136 hits for ‘‘CEO’’ compared with 174 for ‘‘TMT/top management team.’’ While this disparity is not a dispositive statement on the potential contribution of TMTs to outcomes, it does provide a rather clear signal about the apparent preference in the research community for the promise of CEO research as compared to that focusing on TMTs. For us, there is yet another fascinating aspect of the relative contribution of CEO research and that of TMTs. A cursory reading of the extant literature suggests that many observers believe that CEOs matter. We hasten to add that the literature does not uniformly underscore the accuracy of that perception (e.g., Carpenter et al., 2004). Even so, the literature continues to articulate that attribution. Let us accept that members of TMTs are perceived as a group – an entity that meaningfully contributes to important organizational outcomes. The issue is not that some, or most, of these TMT members are able,

Upper Echelons Perspective and Multi-Level Analysis

253

high-ranking executives. Instead, the argument goes well beyond that. The literature suggests that there is something about the TMT, whose members in concert, provide something well beyond the individual contributions of those members – an important synergy enabling ﬁrms’ productivity. Why, then, have we never read about one ﬁrm hiring the CEO and the TMT from another ﬁrm? Why do we not observe one ﬁrm seeking new leadership, for whatever reason, obtaining the entire TMT from another ﬁrm? Academic departments do not hire entire faculties from one another; sports programs do not hire entire senior staffs from one another. In fairness, there may be an exception that warrants serious examination because, to our knowledge, it has never been studied. In mergers and acquisitions, one could reasonably argue that entire TMTs are, in fact, obtained. And, it is possible that in some cases an acquisition was positively evaluated, in part, because of the perceived quality of the TMT of the acquired ﬁrm, especially if the acquired ﬁrm is expected to operate independently. High levels of turnover for acquired ﬁrms’ TMTs, however, would certainly be a factor in this argument (Davis & Nair, 2003; Krug & Hegarty, 1997; Walsh, 1988, 1989). Setting those cases aside for the moment as the fate of ‘‘acquired’’ or ‘‘merged’’ executives is not uniformly satisfactory (at least in the minds of all the executives involved), there is another aspect of this TMT transition issue which is interesting. Perhaps we need not be distracted by the interests of other parties at all. Let us profoundly simplify this issue by focusing only on the will of the CEO. It is unassailable that companies hire CEOs from other ﬁrms; it is done routinely. If a TMT is a contributing entity, why then do CEOs who have been hired not insist on moving their senior staffs with them when they move to the new ﬁrm? We would be willing to bet, and confess that we have no data whatsoever to buttress our position, that a newly arriving CEO is far more likely to move with her or his administrative assistant than with any other executive in the prior ﬁrm, TMT member or otherwise. Now, that observation may be either an eloquent example of who is the more valuable contributor, or it may say something about the efﬁcacy of TMTs, at least as perceived by the transferring CEO. We would concede, however, that the ﬁrm would prefer to have a better TMT than an inferior one, whether or not the incumbent CEO would elect to ask it or any of its members to join her/him in moving to another ﬁrm. Accordingly, a ﬁrm may prefer certain qualities in a TMT, and seek to obtain or retain those qualities, whatever the incumbent CEO’s inclinations about asking others to join her/him may be.

254

DAN R. DALTON AND CATHERINE M. DALTON

Homogeneity/Heterogeneity of the TMT Much of the literature addressing TMTs focuses on the homogeneity or heterogeneity of the TMT and the extent to which those patterns are associated with organizational outcomes (for reviews, see, e.g., Carpenter et al., 2004; Daily & Schwenk, 1996; Finkelstein & Hambrick, 1996; Jackson, 1992; current examples include Boone, Olffen, Witteloostuijn, & Brabander, 2004; Carpenter, 2002; Ensley, Pearson, & Amason, 2002; Ferrier & Lyon, 2004; Kor, 2003). A review of this literature would seem to suggest that neither homogeneity nor heterogeneity is universally superior (e.g., Boone et al., 2004). Rather, the preference for either would seem to depend on a particular outcome of interest. While these choices are almost certainly more complex than any illustration will reﬂect, we hope the following illustration will underscore an interesting aspect of the empirical examinations of TMTs. Let us assume, if for no other reason than because it does have an intuitive appeal that TMT heterogeneity is clearly superior in the formulation stage, while TMT homogeneity is the better pattern in the implementation stage. But, the relevant literature does not address multiple TMTs in the same ﬁrm. A ﬁrm has a TMT and that there is only one leads to a series of issues. We could take the position that the TMT notion, then, is seriously compromised because a single TMT team can not be simultaneously both homogeneous and heterogeneous. This, of course, is particularly true if the operationalization of homogeneity/heterogeneity is largely based on demographic factors (e.g., experience, age, organizational tenure, educational level, functional background; see, e.g., Carroll & Harrison, 1998; Hambrick & Mason, 1984; Haveman, 1995; Pfeffer, 1983). Given this, there is only one ‘‘demographic’’ proﬁle of a TMT for any given ﬁrm. If that perspective is accepted, then a contingency approach to homogeneity/heterogeneity is moot. Alternatively, one might be comfortable suggesting that ﬁrms do, in fact, have more than one TMT. A CEO, for example, might be much more likely to engage one subset of her/his senior ofﬁcers when the issue is formulative and another subset for implementation. In fact, one could imagine a host of much-trusted subsets depending on the issue at hand (for some discussion on this point, see Roberto, 2003). We ﬁnd this position to be entirely satisfactory and do not doubt its accuracy for a moment. In fairness, however, whether any of these focused subgroups rise to the theoretical/conceptual notion of a TMT as most commonly expressed in the literature is an open question. We suspect that they do not.

Upper Echelons Perspective and Multi-Level Analysis

255

One could imagine, however, a scenario wherein multiple TMTs would be quite sensible. Consider, for example, a multinational company with a geographical structure in which there is a CEO for Latin America, for Asia, for Europe, for Africa, for Australia, and so forth. Given this structure, we would expect that the company CEO would consult with the geographical CEO and senior staff routinely when the issue at hand is local (e.g., an issue in Asia). By the same logic, one could imagine similar processes for companies that comprise largely independent business lines. Companies with structures similar to General Electric or Berkshire Hathaway come immediately to mind. The concept of multiple TMTs raises a host of issues. Would we, for example, expect the ‘‘corporate’’ TMT to be the more important or the TMTs operating at the business level? At the international/ regional level? Should the multiple TMTs be homogeneous/heterogeneous? There may be yet another view. Perhaps what the next generation of research should address is the ‘‘chameleon’’ qualities of senior executives. Maybe TMT members are entirely capable of extreme levels of heterogeneity in the formulation stage (i.e., argue vehemently against a proposed initiative) and yet unanimously accept a given decision and work tirelessly for its implementation. We suspect that, at their very best, this is a near ideal proﬁle for outstanding executives. Observers will likely disagree on which of these options, or combinations, is the more realistic. Few observers, however, would argue that TMT theory or the extant literature as currently conﬁgured addresses these issues.

There Are Some Implied ‘‘Buts’’ Here We suspect that our ruminations in the last section on homogeneity/heterogeneity generated some ‘‘Yes, butsy’’ Consider a few questions about homogeneity/heterogeneity and TMTs. Is the entire notion of outside CEO succession a strategy for those who apparently prefer heterogeneity? Is the whole generic notion of decentralization a comment on the perceived efﬁcacy of heterogeneity? Is it reasonable to expect that any group will move toward homogeneity over time? It would seem that extreme cases of individual heterogeneity (a TMT outlier if you will) would leave the group voluntarily in frustration, or be strongly encouraged to apply their skills elsewhere by the will of the group. Do group members over time become too inured to the history, culture, processes, practices, and procedures of the group? For this very reason, Sutton (2004) has argued, in a modestly different context, that board members with more than 5 years of service are no

256

DAN R. DALTON AND CATHERINE M. DALTON

longer independent. So, how would a ‘‘natural’’ movement towards homogeneity over time inform TMT dynamics? Cannella and Holcomb (this volume) make another compelling argument in the ‘‘yes, but’’ category. Is it true, at some level, that TMT members are competitors? Are they, in fact, eventual candidates for the CEO position in the focal organization? Are they also candidates for CEO positions in other ﬁrms? Is this a mini-agency theory problem? How will TMT members distinguish themselves among their peers? Does this tendency to distinguish oneself lead to heterogeneity? And, how do these dynamics inform TMT homogeneity/heterogeneity?

Where is the Board of Directors? There has been some discussion about the board of directors in the TMT/ upper echelons perspective (e.g., Cannella & Holcomb, this volume; Carpenter et al., 2004; Finkelstein & Hambrick, 1996; see especially, Daily & Schwenk, 1996) and its membership, or otherwise, in the TMT. We mention this only in the spirit that whatever view one may have embraced previously on this question may require some recalibrating. We now know, in the current corporate governance environment (e.g., Sarbanes-Oxley, 2002, stock exchange guidelines, activism of the Justice Department) that the nature of the board and its processes is radically and rapidly changing. There is an emphasis on ‘‘independence’’ of members and transactions. This focus decidedly does not embrace TMT membership on the board of directors. Conceptualizations of the upper echelons perspective are silent or barely whisper on structural issues of contemporary corporate governance. What, for example, are the TMT issues implicit in CEOs serving simultaneously as chairpersons of the board? Similarly, are, or should, the role and responsibilities of the lead director, the formal interface between outside board members and senior management, be an element in the TMT construct? And a presiding director? And an executive committee? An executive committee, for example, is a very interesting problem for TMT researchers. Sarbanes-Oxley and the guidelines of the major stock exchanges are silent on the ‘‘independence’’ requirements of executive committees. Therefore, executive committees can be comprised largely of inside directors (i.e., ofﬁcers of the company). Surely, such a group should be considered in the notion of TMTs. In fact, one could argue that an executive committee is the TMT for any corporation that employs such a committee.

Upper Echelons Perspective and Multi-Level Analysis

257

Approximately 50% of the largest U.S. corporations have executive committees (Kenny, 2004). Is the appropriate role of the contemporary board to serve as devil’s advocate, constantly challenging management’s views? Or, is the better course that of a resource board, an enabler of management? Or, more likely, does it depend? If so, on what? How shall we interpret the role of the chairperson of the board’s audit committee and her/his formal responsibility over the ﬁrm’s internal auditing function? Can a TMT ignore the roles of the board? Is it – or some subset of its members – not a member consistent with an upper echelons perspective? Consistent with a dominant coalition perspective? Or minimally, does the board have a place in a multi-level analysis?

And, We Will Resisty Somewhere near this point we could note that virtually every issue we have raised thus far is actually longitudinal, subject to routine and rapid change. And we could repeat the universal criticism that TMT measures in the extant literature are largely demographic and do not capture process. We could also join a chorus of observers who are concerned about the lack of consistency in measurement of the TMT concept. Because these issues have been discussed at length by others and, in any case, are not unique to TMT research, we will say no more about them.

CONCLUSION The concerns/questions that we have noted are (hopefully) illustrative, but by no means exhaustive. Twenty years ago, Hambrick and Mason (1984) signaled the currency of the upper echelons perspective. Subsequently, Hambrick (1989, p. 5) commented that the increase in research addressing the roles of senior executives in corporate outcomes was ‘‘inevitable’’ because strategic leaders ‘‘account for what happens to the organization.’’ Now, some years later, we learn that ‘‘ythe UE [upper echelons] stream is a ﬂourishing one. At the same time, it stands at an important crossroads’’ (Carpenter et al., 2004, p. 768). We could not agree more. While much has been accomplished, there is much to be done. We hope that our comments are not interpreted as critical of multi-level modeling and its great promise. From Rosseau’s (1985) early contributions through the truly ﬁrst-rate advances in multi-level analyses (e.g., Hox, 2002;

258

DAN R. DALTON AND CATHERINE M. DALTON

Little, Schnabel, & Baumert, 2000; Luke, 2004; Reis & Duan, 2003), it appears that we have the means to examine our TMT work in a more robust way. We, though, and evidently Cannella and Holcomb (this volume) as well, are not convinced that the state of TMT/upper echelons research is yet mature enough to concern ourselves with the multi-level analysis opportunity. To take that step comfortably, it seems that, like every analytical approach, we need sound theory and reliable variables. For now, however, we are not conﬁdent that the ﬁeld has achieved either.

ACKNOWLEDGEMENT We thank Cannella and Holcomb (this volume) for their outstanding work. Beyond that, though, we thank them because they made us think, and rethink, about TMTs, the upper echelon perspective, and related topics in a much more concentrated way than we previously had. And there is much more to think about as a new series of conceptualization and research underscores the centrality of strategic leadership, however, its elements may be operationalized.

REFERENCES Berle, A., & Means, G. (1932). The modern corporation and private property. New York: MacMillan Company. Boone, C., Van Olffen, W., Witteloostuijn, A., & De Brabander, B. (2004). The genesis of top management team diversity: Selective turnover among top management teams in Dutch newspaper publishing, 1970–1994. Academy of Management Journal, 4, 633–656. Cannella, A. A., & Holcomb, T. R. (this volume). A multi-level analysis of the upper-echelons model. In: F. Dansereau & F. J. Yammarino (Eds), Research in multi-level issues, Vol. 4, Oxford, UK: Elsevier Science. Carroll, G. R., & Harrison, J. R. (1998). Organizational demography and culture: Insights from a formal model and simulation. Administrative Science Quarterly, 43, 737–767. Carpenter, M. A. (2002). The implications of strategy and social context for the relationship between top management team heterogeneity and ﬁrm performance. Strategic Management Journal, 23, 275–284. Carpenter, M. A., Geletkanycz, M. A., & Sanders, W. G. (2004). Upper echelons research revisited: Antecedents, elements, and consequences of top management team composition. Journal of Management, 30, 749–778. Cheng, S. (2004). R&D expenditures and CEO compensation. The Accounting Review, 79, 305–328.

Upper Echelons Perspective and Multi-Level Analysis

259

Coase, R. (1937). The nature of the ﬁrm. Economica, 4, 386–405. Cyert, R. M., & March, J. G. (1963). A behavioral theory of the firm. Englewood Cliffs, NJ: Prentice-Hall. Daily, C. M., & Schwenk, C. (1996). Chief executive ofﬁcers, top management teams, and boards of directors: Congruent or countervailing forces? Journal of Management, 22, 185–208. Dalton, D. R., Daily, C. M., Certo, S. T., & Roengpitya, R. (2003). Meta-analyses of corporate ﬁnancial performance and the equity of CEOs, ofﬁcers, boards of directors, institutions and blockholders: Fusion or confusion? Academy of Management Journal, 46, 13–26. Dalton, D. R., Daily, C. M., Ellstrand, A. E., & Johnson, J. L. (1998). Meta-analytic reviews of board composition, leadership structure, and ﬁnancial performance. Strategic Management Journal, 19, 269–290. Davis, J. H., Schoorman, F. D., & Donaldson, L. (1997). Toward a stewardship theory of management. Academy of Management Review, 22, 20–47. Davis, R., & Nair, A. (2003). A note of top management turnover in international acquisitions. Management International Review, 43, 171–183. Eisenhardt, K. M. (1989). Agency theory: An assessment and review. Academy of Management Review, 14, 57–74. Engel, E., Hayes, R. M., & Wang, X. (2003). CEO turnover and properties of accounting information. Journal of Accounting & Economics, 36, 197–226. Ensley, M. D., Pearson, A. W., & Amason, A. C. (2002). Understanding the dynamics of new venture top management teams: Cohesion, conﬂict, and new venture performance. Journal of Business Venturing, 17, 365–386. Farrell, K. A., & Whidbee, D. A. (2003). Impact of ﬁrm performance on CEO turnover and replacement decisions. Journal of Accounting & Economics, 36, 165–196. Ferrier, W. J., & Lyon, D. W. (2004). Competitive repertoire simplicity and ﬁrm performance: The moderating role of top management team heterogeneity. Managerial & Decision Economics, 25, 317–327. Finkelstein, C., & Hambrick, D. C. (1996). Strategic leadership: Top executives and their effects on organizations. Minneapolis, MN: West Publishing. Grinstein, Y., & Hribar, P. (2004). CEO compensation and incentives: Evidence from M&A bonuses. Journal of Financial Economics, 73, 119–143. Hambrick, D. C. (1989). Putting top managers back in the strategy picture. Strategic Management Journal, 10, 5–15. Hambrick, D. C. (1994). Top management groups: A conceptual integration and reconsideration of the ‘‘team’’ label. In: B. Staw & L. L. Cummings (Eds), Research in organizational behavior, (Vol. 16, pp. 171–213). Beverly Hills, CA: JAI Press. Hambrick, D. C., & Cannella, A. A. (2004). CEOs who have COOs: Contingency analysis of an unexplored structural form. Strategic Management Journal, 25, 959–979. Hambrick, D. C., & Mason, P. A. (1984). Upper echelons: The organization as a reﬂection of its top managers. Academy of Management Review, 9, 193–206. Haveman, H. A. (1995). The demographic metabolism of organizations: Industry dynamics, turnover, and tenure distributions. Administrative Science Quarterly, 40, 586–618. Hox, J. (2002). Multi-level analysis. Mahwah, NJ: Lawrence Erlbaum. Jackson, S. (1992). Consequences of group composition for the interpersonal dynamics of strategic issue processing. In: P. Shrivastava, A. Huff & J. Dutton (Eds), Advances in strategic management (pp. 345–382). Greenwich, CT: JAI Press.

260

DAN R. DALTON AND CATHERINE M. DALTON

Jensen, M. C., & Meckling, W. H. (1976). Theory of the ﬁrm: Managerial behavior, agency costs and ownership structure. Journal of Financial Economics, 3, 305–360. Kenny, R. M. (2004). Executive committee: A vestigial appendage. Directors & Boards, 28(4), 43–46. Kor, Y. Y. (2003). Experience-based top management team competence and sustained growth. Organization Science, 14, 707–719. Krug, J. A., & Hegarty, W. H. (1997). Post-acquisition turnover among U.S. top management teams: An analysis of the effects of foreign vs. domestic acquisitions of U.S. targets. Strategic Management Journal, 18, 667–675. Lane, P. J., Cannella, A. A., & Lubatkin, M. H. (1998). Agency problems as antecedents to unrelated mergers and diversiﬁcation: Amihud and Lev reconsidered. Strategic Management Journal, 19, 555–578. Little, T. D., Schnabel, K. U., & Baumert, J. (2000). Modeling longitudinal and multi-level data. Mahwah, NJ: Lawrence Erlbaum. Luke, D. A. (2004). Multi-level modeling. Thousand Oaks, CA: Sage Publications. Macey, J. R. (1997). Institutional investors and corporate monitoring: A demand-side perspective. Managerial and Decision Economics, 18, 601–610. McDonald, M. L., & Westphal, J. D. (2003). Getting by with the advice of their friends: CEO’s advice networks and ﬁrms’ strategic responses to poor performance. Administrative Science Quarterly, 48, 1–32. Murphy, K. J., & Zabojnik, J. (2004). CEO pay and appointments: A market-based explanation for recent trends. American Economic Review, 94, 192–196. Parrino, R., Sias, R. W., & Starks, L. T. (2003). Voting with their feet: Institutional ownership changes around forced CEO turnover. Journal of Financial Economics, 68, 3–46. Pfeffer, J. (1983). Organizational demography. In: L. L. Cummings & B. M. Staw (Eds), Research in organizational behavior, (Vol. 5, pp. 299–357). Greenwich, CT: JAI Press. Reis, S. P., & Duan, N. (2003). Multi-level modeling. Mahwah, NJ: Lawrence Erlbaum. Roberto, M. A. (2003). The stable core and dynamic periphery in top management teams. Management Decision, 41, 120–131. Rosseau, D. M. (1985). Issues of level in organizational research: Multi-level and cross-level perspectives. In: B. M. Staw & L. L. Cummings (Eds), Research in organizational behavior, (Vol. 7, pp. 1–37). Greenwich, CT: JAI Press. Sarbanes-Oxley (2002). Sarbanes-Oxley Act of 2002, Public Law Number 107-204, 116 Statute 765. Shleifer, A., & Vishny, R. W. (1997). A survey of corporate governance. Journal of Finance, 52, 737–783. Shen, W. (2003). The dynamics of the CEO-board relationships: An evolutionary perspective. Academy of Management Review, 28, 466–476. Shen, W., & Cannella, A. A. (2002). Revisiting the performance consequences of CEO succession: The impacts of successor type, post-succession senior executive turnover and departing CEO tenure. Academy of Management Journal, 45, 717–733. Shen, W., & Cannella, A. A. (2003). Will succession planning increase shareholder wealth? The evidence from investor reactions to relay CEO succession. Strategic Management Journal, 24, 191–198. Sutton, G. (2004). Rules for rock-solid governance. Directors & Boards, 28(3), 18–21. Walsh, J. P. (1988). Top management turnover following mergers and acquisitions. Strategic Management Journal, 9, 173–183.

Upper Echelons Perspective and Multi-Level Analysis

261

Walsh, J. P. (1989). Doing a deal: Merger and acquisition negotiations and their impact upon target company top management turnover. Strategic Management Journal, 10, 307–322. Walsh, J. P., & Kosnik, R. D. (1993). Corporate raiders and their disciplinary role in the market for corporate control. Academy of Management Journal, 35, 671–700. Wasserman, N. (2003). Founder-CEO succession and the paradox of entrepreneurial success. Organization Science, 14, 149–172. Zhang, Y., & Rajagopalan, N. (2004). When the known devil is better than an unknown god: An empirical study of the antecedents and consequences of rely CEO succession. Academy of Management Journal, 47, 483–500.

This page intentionally left blank

262

A MULTI-LEVEL ANALYSIS OF THE UPPER-ECHELONS MODEL: PLANTING SEEDS FOR FUTURE RESEARCH Albert A. Cannella, Jr. and Tim R. Holcomb ABSTRACT We thank Carpenter and Dalton and Dalton for their insights on our earlier chapter, and on the promise (and perils) of upper-echelons research in general. We set out to closely examine the levels issues in Hambrick and Mason’s ((1984). Academy of Management Review, 9, 193–206.) original upper-echelons model, and the research initiatives that have applied this theoretical framework. We are encouraged by the initial reception that we have received from these authors. We continue to believe that top management teams (TMTs) are an important level of analysis for strategic leadership research, though the original upper-echelons model proposed by Hambrick and Mason cannot be directly applied at the team level. Our reply highlights several joint and individual concerns raised by the articles. We close by reiterating our call for continued analysis of the upper-echelons model.

Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 263–273 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04012-9

263

264

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

INTRODUCTION Let us begin by noting that both of us were quite excited to hear that Mason Carpenter, and Dan and Catherine Dalton had agreed to write articles about our chapter. Further, we were very happy with the articles that each provided. Both articles have helped to sharpen our own ideas about the upper-echelons perspective and the need for subsequent research that includes multi-level issues. In the ﬁrst section below, we respond to the issues raised by both articles. In the next two sections, we respond to issues unique to each article.

JOINT CONCERNS There is significant overlap across the two articles, and we will respond ﬁrst to the overlapping issues. Perhaps most prominent in both articles was the conclusion that there is a wide gap between upper-echelons theory and its application at the team level. Carpenter (this volume) was struck by the mismatch between upper-echelons theory and its use in empirical research. Dalton and Dalton (this volume) on the other hand, found that same mismatch so great that they believe that we have the cart before the horse. That is, we had called for multi-level research when our measures, methods, and theories seemingly are not up to the task. As we wrote the original chapter, we were also surprised by how poorly the upper-echelons perspective (UEP) ﬁts the basics of multi-level theorizing, but (like Carpenter) we are not giving up on TMT research just yet. In fact, should not good theory stimulate our inquisitive desire to do good research? We think so. In fact, Hambrick and Mason’s (1984) UEP has invoked curiosity among scholars for the past two decades. Clearly, the UEP has made us examine more closely the inﬂuence of the values, preferences, and experiences of top managers on the decisions they make. While the issues raised by our chapter as well as the articles are formidable, progress clearly has been made (see, e.g., Bourgeois & Eisenhardt, 1988; Bunderson & Sutcliffe, 2002; Pelled, Eisenhardt, & Xin, 1999; Pitcher & Smith, 2001). Much more research is possible, and we hope that it has been made more likely by our chapter and the articles. Both sets of articles, as well as our own chapter, call for more consideration of process in TMT research, and we are by no means the ﬁrst to make that call, as Dalton and Dalton (this volume) amply illustrate. We will have more to say on this issue later, but sufﬁce it to say, at this point, that we believe that a quantum

A Multi-Level Analysis of the Upper-Echelons Model

265

leap can be made with TMT research if a few simple process issues are dealt with. We think it best not to despair over the state of TMT research, but to lay out a theoretical standard that (we hope) future researchers will strive for, coupled with some suggestions about how researchers might proceed. The second issue with considerable overlap across the two articles was the lack of differentiation among TMT members in most empirical work. Carpenter (this volume) raises this as an issue of power, noting that empirical work on TMTs tends to assume equal power among all TMT members. He pushes the point about how unrealistic this is. Dalton and Dalton (this volume) on the other hand, ask where the CEO is in all of this. Clearly, the lack of differentiation between the CEO and members of the TMT is an important concern in TMT research. Carpenter’s point about power is apt, and parallels Finkelstein and Hambrick’s (1996) call from some years back. However, researchers should consider the multiple sources of power (e.g. Finkelstein, 1992), and be fully aware that, in terms of power, the CEO is likely to overwhelm everyone else on the TMT. For example, Cannella and Shen (2001), Reutzel and Cannella (2004), and Hambrick and Cannella (2004) all measured the power of nonCEO executives, and concluded that, relative to the CEO, these executives have little power. This is in spite of the fact that each of the above studies focused on very high-ranking TMT members (heirs apparent, CFOs (Chief Financial Ofﬁcers), and COOs (Chief Operating Ofﬁcers), respectively) and demonstrated results for power among non-CEO TMT members. While power measured through share ownership is reliable and convenient, we believe that consideration of expertise power (Finkelstein, 1992) is likely to be more fruitful. Expertise power would be particularly important when coupled with Roberto’s (2003) notion of the TMT as comprised of a stable core and dynamic periphery, with membership depending upon the issue under consideration. Dalton and Dalton (this volume) are surely correct when they assert that consideration of the CEO is lacking in TMT research. Indeed, such a condition only strengthens our view of the mismatch between the UEP and its application in research. However, our chapter provided (we hope) some basis for considering how the CEO and TMT might relate to each other, and we are not the ﬁrst to do so (e.g. Carpenter, Geletkanycz, & Sanders, 2004). We believe that it is simply not enough to say that the CEO is all powerful, especially when implementation issues are considered. Consequently, we (again) caution researchers using the UEP to carefully specify their assumptions about the theoretical logic they choose to use at each level of analysis.

266

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

Both Carpenter (this volume) and Dalton and Dalton (this volume) ask where the board is in all of this. We are temped to reply that the board is not, and probably should not, be very much involved in strategy formulation – the topic we limited our analysis to. As Dalton and Dalton (this volume) noted in their own article, any director who has been on the board for more than 5 years probably is no longer independent (Sutton, 2004). In our own view, ‘‘independent’’ directors must remain largely away from strategy formulation or they will not be able to objectively evaluate the strategy formulated. We believe, as did Fama and Jensen (1983), that managers are responsible for initiation and implementation, and directors are responsible for ratiﬁcation and monitoring. Having said the above, we would hasten to add that independent directors can and do inﬂuence strategic decision making, for example, through their advice-giving interactions with CEOs (Westphal, 1999). Likewise, their preferences regarding CEO appointments also inﬂuence the formation process and in setting overall strategic direction (Westphal & Fredrickson, 2001). Nonetheless, if independent directors are to be effective monitors (and that is their stated purpose, not to mention their sole responsibility if one believes the spirit of Sarbanes-Oxley), then they must leave decision initiation largely to managers. Now, we were not born yesterday, as they say. Clearly, some managers can (and some do) manipulate their boards. However, the widespread and (collectively) incredibly large investments in the stock market suggest that investors are still willing to place their bets with the monitoring functions of boards, backed up by a panoply of other monitoring devices (e.g. Fama, 1980). Carpenter’s (this volume) framing of this question about the boundaries of the TMT went beyond boards to other inﬂuential people from outside the ﬁrm. He asked about the roles of consultants and spouses as potential members of TMTs. In some ways, the social capital literature has begun to respond to some of these issues (see Carpenter et al., 2004). We hope his call spurs further research on both of these issues in the near future, though they were a bit beyond the scope of our original chapter. Finally, both Carpenter (this volume) and Dalton and Dalton (this volume) note that the UEP is both a theory and a methodology. While they differ a bit in their perspectives on this issue, we think that it is fair to say that the methodology is probably far more threatened than the theory. Simply put, the use of demographics as measures of far more specific though unobservable constructs is facing significant challenges to validity. Again, we are not the ﬁrst to make this point (e.g. Carpenter et al., 2004; Lawrence, 1997; Priem, Lyon, & Dess, 1999; West & Schwenk, 1996). While

A Multi-Level Analysis of the Upper-Echelons Model

267

demographic measures are highly reliable, precisely what they measure is often open to question (construct validity) and the same handful of demographic variables are used as indicators for an ever-widening variety of unobservable constructs. It has become more and more difﬁcult to argue that empirical research relying entirely on demographic measures is not confounded in terms of cause and effect.

CONCERNS SPECIFIC TO CARPENTER (THIS VOLUME) Carpenter (this volume) raises several important (and unique) points in his article. First, he notes that the UEP is inherently multilevel, and for reasons well beyond those mentioned in our original chapter. He is certainly correct in this notion, however researchers have not always treated it in true multilevel fashion, and we believe (as he does) that future researchers should do so. Toward that end, we would like to encourage future researchers to take advantage of the advances in research on charisma that we cited in our chapter (e.g. Kets de Vries, 1998; Klein & House, 1995; Waldman & Yammarino, 1999). The notion of charisma, coupled with the concept of framing, provide (we believe) important levers into how the CEO works through the TMT to achieve desired outcomes. Of course, some CEOs have little charisma, and some have weak visions (or no vision at all), both of which would considerably reduce their effects on organizations. However, these situations might also prompt a host of other team-level issues to arise. We were also pleased to see that Carpenter had highlighted the importance of agendas and agenda setting. We believe that this is an area in which upper-echelons researchers can make great strides without data collection becoming too onerous. For example, it is likely that many in the ﬁrm will be aware of the CEO’s decision making style or leadership style, or the presence or absence of agendas at meetings, and who exhibits power over agendas when they are present. A very short survey (2 or 3 questions) sent to a number of senior managers is quite likely, we believe, to elicit at least one usable response per company. Additionally, a similar methodology could be used to determine (in broad terms) whether the TMT meets regularly as a team, how it processes decisions or issues, or if indeed the ‘‘team’’ processes issues at all. These concepts are not complex, and we believe that coupling surveys with archival methods could go a long way toward furthering our

268

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

understanding of the roles and functions of TMTs, as well as settings in which TMTs are more or less inﬂuential. As we note below, and others have noted (e.g. Carpenter et al., 2004; Finkelstein & Hambrick, 1996) there is no reason to believe that TMTs are critical (i.e. the appropriate level of analysis) in every strategic context. Finally, Carpenter’s (this volume) call for research on implementation is timely and welcome. Levels issues in theory building are not limited to strategic decision making, nor do the potential causal relationships necessarily ‘‘behave’’ similarly between the formulation and the implementation stages. We limited our analysis to formulation because we simply did not have the space to consider implementation issues. However, any study that directly links TMT characteristics to organizational outcomes using the UEP runs the risk of confounding causal linkages. Put differently, if more diverse teams are associated with better ﬁrm-level performance, is it because they make better decisions, implement decisions better once made, or some combination of the two? Again, the mainstream methodologies that UEP researchers have traditionally used are unable to shed much light on these questions.

CONCERNS SPECIFIC TO DALTON AND DALTON (THIS VOLUME) You can always count on Dan and Catherine Dalton to be provocative. Yet given that, we were surprised to ﬁnd that they (Dalton & Dalton, this volume) interpreted our paper as indicating that we did not believe that TMTs are the appropriate unit of analysis for upper-echelons research. We raised a number of questions about the current state of research, and for the use of the upper-echelons model, as formulated by Hambrick and Mason (1984) with TMTs. Our questions covered both theory and method. However, we do not believe that our chapter should be interpreted as questioning the appropriateness of the TMT as the unit of analysis. Rather, we believe that before we can make much research progress on TMTs, issues of process must be included, and the role of the CEO versus the team must be clariﬁed. We certainly hope that our chapter is not widely interpreted as indicating that TMTs are an inappropriate unit of analysis for UEP research. To the contrary, we believe the TMT should be included in a robust theoretical context that facilitates the study of levels-related phenomena in strategic decision making.

A Multi-Level Analysis of the Upper-Echelons Model

269

Dalton and Dalton (this volume) point out that executives often act out of self-interest, and do not necessarily have the best interests of the ﬁrm in mind as they go about their work. While we agree with that notion, we do not wish to carry it too far. Lane, Cannella and Lubatkin (1998) pointed out that while agency theory is valuable, its applicability to questions of corporate strategy is not always clear or appropriate. In order for agency theory to have any explanatory power whatsoever, there must be sharp conﬂicts of interest present. Like Lane et al. we recognize that some strategic situations pose sharp conﬂicts of interest, but we do not believe that many strategic situations can be readily characterized in this way. While Amihud and Lev (1981) and others (e.g. Denis, Denis, & Sarin, 1997) believe that personal risk reduction drives at least some decision making in upper echelons, we are not persuaded that the reduction of employment risk is a central concern in much strategic decision making. In our view, for many strategic decision settings, what is good for the company is also good for the executives. Clearly, in executive compensation decisions, there are sharp conﬂicts of interest, and agency theory’s application to executive compensation is highly appropriate. However, when evaluating a new product introduction, a strategic alliance, a response to competitors, or a myriad other decision settings, the interests of executives and those of shareholders are not necessarily in sharp conﬂict, and are likely not in conﬂict at all. We believe that few executives would interpret poor ﬁrm performance as in their personal interest. Our own experiences with executives lead us to believe that self-interest is important. Indeed, since Smith (1776) this conclusion has been widespread and seldom challenged. However, Smith also recognized that in a great many situations, when individuals pursue self-interest, the interests of the many are furthered. We wish to be quite clear here, at the risk of repetitiveness. Executives are not angels and we do not expect them to be. However, before the concerns voiced by Dalton and Dalton (this volume) about executive self-interest will be strongly heeded by researchers, it must be shown that most decision situations involve sharp conﬂicts of interests between executives and shareholders, and we simply do not believe that this is broadly the case. Dalton and Dalton (this volume) raise very interesting and important questions about executive turnover, and we hope that the issues they raise will spur further research. However, many of the points they raise have not been subject to empirical investigation. For example, why do CEOs not take their teams with them when they change employers? We believe that careful research might well show that CEOs often do recruit others that they have worked with when they join new ﬁrms. For example, the minivan, which was

270

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

an extremely successful innovation for Chrysler, was initially developed at Ford. However, the executive who championed the development was unable to persuade Ford to introduce a minivan. When Lee Iacocca was ﬁred at Ford and joined Chrysler, he recruited this executive to Chrysler, and (as they say) the rest is history (Iacocca & Novak, 1984). Further, Bill Parcells, a professional football coach well known for his turnaround of dismal performing football teams, has a loyal group of assistant coaches who tend to follow him to new assignments (Fagenson-Eland & Parcells, 2001). Beyond the lack of empirical research, two other issues might stand in the way of CEOs and teams moving together to new ﬁrms. First, CEOs do not often move from one ﬁrm to another. More often, it is those who are passed over for CEO who move to take positions at other ﬁrms. Second, Castanias and Helfat (1991, 2001) explain that ﬁrm-specific human capital poses an important impediment to executive movement from ﬁrm to ﬁrm, and this impediment exists for CEOs as well as senior executives. It is quite likely that a CEO who is recruited by another ﬁrm will be unable to persuade members of the TMT to join him or her, as they may feel their own prospects are better at the original ﬁrm. Certainly, the CEO’s movement to another ﬁrm opens promotion possibilities for the remaining executives. In sum, it would not be surprising to ﬁnd that teams of executives do not move from one ﬁrm to another, and that fact may not pose a challenge for the selection of TMTs as the appropriate level of analysis for UEP researchers. Regarding the importance of executives below the CEO, Bertrand and Schoar (2003), in a very interesting and large-scale analysis, demonstrate that while CEO effects are stronger, non-CEO members of the executive cadre can have significant ﬁrm-level effects. Put differently, executives below the CEO matter, and are worthy of study. Finally, Dalton and Dalton (this volume) ask why there are so many studies of CEOs and so few of TMTs? This is an important question, and one that we hope is remedied by future research. However, there are a lot of reasons why research on CEOs would be more popular than research on TMTs. First, there is a well-documented tendency to attribute organizational outcomes to the actions and intentions of individuals (Chen & Meindl, 1991; Meindl & Ehrlich, 1987). There is little reason to believe that researchers (or reviewers) will be immune to these tendencies. Second, it is much easier to make clear theoretical predictions about CEOs than TMTs, and a large part of our original chapter describes why this is the case. Third, methodologically, it is much more challenging to study TMTs than CEOs. Data collection is much more complex, and the missing data problem frequently looms large. Establishing reliable and valid team-level measures is

A Multi-Level Analysis of the Upper-Echelons Model

271

also much more challenging, as we noted in our earlier chapter. Finally, passing muster with reviewers is a daunting challenge for any research, but the added complexities of TMT theory and method simply provide more targets for the slings and arrows of reviewers. In sum, it should be no surprise that CEO research is far more common than TMT research. Again, however, we hope that our chapter and the two responses to it will spur further interest in TMTs. In closing, we thank Mason Carpenter, and Dan and Catherine Dalton for their insightful articles. Their perspectives caused us to think through a number of issues that were not covered in the original chapter. While we do not agree with all the points raised, their articles clariﬁed our own perspectives on the issues related to the UEP. Notwithstanding its limitations, we continue to believe that the UEP, if framed correctly, offers a robust platform for dealing with levels issues in strategic decision making. Further, our exchange and other debates that follow will pave the way, we hope, for a lot of future research, as the articles raised or clariﬁed a number of important issues for prospective UEP researchers.

REFERENCES Amihud, Y., & Lev, B. (1981). Risk reduction as a managerial motive for conglomerate mergers. Bell Journal of Economics, 12, 605–617. Bertrand, M., & Schoar, A. (2003). Managing with style: The effect of managers on ﬁrm policies. Quarterly Journal of Economics, 118, 1169–1208. Bourgeois, L. J., III, & Eisenhardt, K. M. (1988). Strategic decision processes in high velocity environments: Four cases in the microcomputer industry. Management Science, 34, 816–835. Bunderson, J. S., & Sutcliffe, K. M. (2002). Comparing alternative conceptualizations of functional diversity in management teams: Process and performance effects. Academy of Management Journal, 45, 875–893. Cannella, A. A., Jr., & Shen, W. (2001). So close and yet so far: Promotion versus exit for CEO heirs apparent. Academy of Management Journal, 44, 252–270. Carpenter, M. A. (this volume). Moving (ﬁnally) toward a multi-level model of the upper echelons. In: A. Dansereau, & F. J. Yammarino (Eds), Multi-level issues in strategy and methods (Research in multi-level issues, Vol. 4) (Vol. 4). Oxford, UK: Elsevier Science. Carpenter, M. A., Geletkanycz, M. A., & Sanders, W. G. (2004). The upper-echelons revisited: Antecedents, elements, and consequences of top management team composition. Journal of Management, 60, 749–778. Castanias, R. P., & Helfat, C. E. (1991). Managerial resources and rents. Journal of Management, 17, 155–171. Castanias, R. P., & Helfat, C. E. (2001). The managerial rents model: Theory and empirical analysis. Journal of Management, 27, 661–678.

272

ALBERT A. CANNELLA, JR. AND TIM R. HOLCOMB

Chen, C. C., & Meindl, J. R. (1991). The construction of leadership images in the popular press: The case of Donald Burr and People Express. Administrative Science Quarterly, 36, 521–551. Dalton, D. R., & Dalton, C. R. (this volume). Upper echelons perspective and multi-level analysis: A case of the cart before the horse? In: A. Dansereau, & F. J. Yammarino (Eds), Multi-level issues in strategy and methods (Research in multi-level issues, Vol. 4) (Vol. 4). Oxford, UK: Elsevier Science. Denis, D. J., Denis, D. K., & Sarin, A. (1997). Ownership structure and top management turnover. Journal of Financial Economics, 45, 193–221. Fagenson-Eland, E., & Parcells, B. (2001). The National Football League’s Bill Parcells on winning, leading, and turning around teams. Academy of Management Executive, 15, 48–55. Fama, E. F. (1980). Agency problems and the theory of the ﬁrm. Journal of Political Economy, 88, 288–307. Fama, E. F., & Jensen, M. C. (1983). Separation of ownership and control. Journal of Law and Economics, 26, 301–325. Finkelstein, S. (1992). Power in top management teams: Dimensions, measurement, and validation. Academy of Management Journal, 35, 505–538. Finkelstein, S., & Hambrick, D. C. (1996). Strategic leadership: Top executives and their effects on organizations. Minneapolis/St. Paul: West Pub. Co. Hambrick, D. C., & Cannella, A. A., Jr. (2004). CEOs who have COOs: Contingency analysis of an unexplored structural form. Strategic Management Journal, 25, 959–979. Hambrick, D. C., & Mason, P. (1984). Upper echelons: The organization as a reﬂection of its top managers. Academy of Management Review, 9, 193–206. Iacocca, L., & Novak, W. (1984). Iacocca: An autobiography. New York: Bantam. Kets de Vries, M. F. R. (1998). Charisma in action: The transformational abilities of Virgin’s Richard Branson and ABB’s Percy Barnevik. Organizational Dynamics, 26, 6–21. Klein, K. J., & House, R. J. (1995). On ﬁre: Charismatic leadership and levels of analysis. Leadership Quarterly, 6, 183–198. Lane, P. J., Cannella, A. A., Jr., & Lubatkin, M. H. (1998). Agency problems as antecedents to unrelated mergers and diversiﬁcation: Amihud and Lev reconsidered. Strategic Management Journal, 19, 555–578. Lawrence, B. S. (1997). The black box of organizational demography. Organization Science, 8, 1–22. Meindl, J. R., & Ehrlich, S. B. (1987). The romance of leadership and the evaluation of organizational performance. Academy of Management Journal, 30, 91–109. Pelled, L. H., Eisenhardt, K. M., & Xin, K. R. (1999). Exploring the black box: An analysis of work group diversity, conﬂict, and performance. Administrative Science Quarterly, 44, 1–28. Pitcher, P., & Smith, A. D. (2001). Top management team heterogeneity: Personality, power, and proxies. Organization Science, 12, 1–18. Priem, R. L., Lyon, D. W., & Dess, G. G. (1999). Inherent limitations of demographic proxies in top management team heterogeneity research. Journal of Management, 25, 935–953. Reutzel, C. R., & Cannella, A. A., Jr. (2004). A model of Chief Financial Ofﬁcer promotion and exit. Paper presented at the annual meetings of the Academy of Management, New Orleans, LA.

A Multi-Level Analysis of the Upper-Echelons Model

273

Roberto, M. A. (2003). The stable core and dynamic periphery in top management teams. Management Decision, 41, 120–131. Smith, A. (1776). An inquiry into the nature and causes of the wealth of nations. London: W. Strahan and T. Cadell. Sutton, G. (2004). Rules for rock-solid governance. Directors & Boards, 28, 18–21. Waldman, D. A., & Yammarino, F. J. (1999). CEO charismatic leadership: Levels-of-management and levels-of-analysis effects. Academy of Management Review, 24, 266–285. West, C. T., Jr., & Schwenk, C. R. (1996). Top management team strategic consensus, demographic homogeneity and ﬁrm performance: A report of resounding nonﬁndings. Strategic Management Journal, 17, 571–576. Westphal, J. D. (1999). Collaboration in the boardroom: Behavioral and performance consequences of CEO-board social ties. Academy of Management Journal, 42, 7–24. Westphal, J. D., & Fredrickson, J. W. (2001). Who directs strategic change? Director experience, the selection of new CEOs, and change in corporate strategy. Strategic Management Journal, 22, 1113–1137.

This page intentionally left blank

274

PART IV: LATENT GROWTH MODELING

275

This page intentionally left blank

276

MULTIVARIATE LATENT GROWTH MODELS: READING THE COVARIANCE MATRIX FOR MULTI-LEVEL INTERPRETATIONS Kai S. Cortina, Hans Anand Pant and Joanne Smith-Darden ABSTRACT Over the last decade, latent growth modeling (LGM) utilizing hierarchical linear models or structural equation models has become a widely applied approach in the analysis of change. By analyzing two or more variables simultaneously, the current method provides a straightforward generalization of this idea. From a theory of change perspective, this chapter demonstrates ways to prescreen the covariance matrix in repeated measurement, which allows for the identification of major trends in the data prior to running the multivariate LGM. A three-step approach is suggested and explained using an empirical study published in the Journal of Applied Psychology.

Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 277–318 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04013-0

277

278

KAI S. CORTINA ET AL.

INTRODUCTION Over the past decade, latent growth modeling (LGM) has become a common statistical tool to analyze longitudinal data. Its major advantage – reducing the complexity of development over time – makes it an attractive alternative to other standard statistical approaches, such as analysis of variance or autoregressive path analysis. Recently, applied research has started to use LGM to simultaneously analyze changes in two or more variables over time. Multivariate latent growth modeling (MLGM) makes it possible to analyze the mutual inﬂuences of more than one variable over time and enables researchers to better understand dynamic processes. It will certainly become a very useful tool for many research domains in the social sciences, including those ﬁelds of applied psychology that, by deﬁnition, are interested in change over time (see Dansereau & Yammarino, 2003; Dansereau, Yammarino, & Kohles, 1999). From a multi-level perspective, the change model can be integrated as part of the hierarchical data structure. Change over time is thus conceptualized as being nested within individuals, adding another level to the model. This expanded model proves a strong tool with which the researcher can analyze the effects of higher-order variables on change. To provide a nontechnical understanding of MLGM, it is helpful to express LGM and MGLM using the concepts and terminology of hierarchical linear modeling (HLM). After a brief introduction of the general idea of LGM, we will explain the relation between HLM and LGM. We will then analytically demonstrate why it is helpful to look at the empirical covariance matrix prior to specifying an MLGM.

LATENT GROWTH MODELING: BASIC CONCEPT LGM assumes that the observed change over time follows – at least approximately – a mathematical function that is based on a limited number of parameters. A common and very simple LGM assumes that the change over time for each observational unit follows a linear trend. Whether the scenario involves students’ learning progress in mathematics over the course of one school year, companies’ increases in sales, or the change in percentage of GNP that countries spend on the military, a linear model might capture the underlying development sufﬁciently well. In this case, the linear process over the observational period is analyzed using two parameters: the intercept

Multivariate Latent Growth Models

279

(i.e., the differences between units at the ﬁrst measurement point) and the slope (i.e., the growth rate). Both parameters can differ between units, so the statistical analysis focuses on identifying causal factors for both. If it is unlikely that the development over time follows a simple linear function (e.g., governments’ military expenditures), more complex growth curves have to be considered that capture, in addition to the linear trend, curvilinear saturation effects (by including a quadratic term) or cyclical trends (by including more complex trend components). Because many theories in the social sciences imply linear or curvilinear developmental patterns, it is not surprising that the polynomial function is commonly applied when the shape of the function is determined by three parameters (intercept, linear, and quadratic component). The statistical analysis consists of two steps. First, the adequacy of the chosen function is inspected. Second, individual differences in each parameter of the function are analyzed for their association with other variables.

Nested Data Structure The analysis of LGM is based on the assumption that repeated measurements are equivalent to a data structure commonly described as ‘‘nested.’’ ‘‘Nested’’ means that the units of analysis are clustered in a nonignorable way, as is common within organizations. For example, students in schools are clustered in classrooms as a learning group with a speciﬁc teacher who might be more efﬁcient than a teacher for another classroom in the same school. Therefore, the learning progress of students from the same classroom is more similar than the learning progress of students from different classrooms. This concept violates an important assumption of most statistical procedures; they assume that the empirical sample is drawn as a sample of independent observational units. As in the classroom example, the independence assumption is often violated in ﬁeld research. In organizational psychology, for example, collecting data on the workplace climate experienced by employees has to take into account the fact that persons work together in workgroups in which they share supervisors, colleagues, tasks, and so on. Two people in the study who belong to the same workgroup are more likely to give a similar assessment of their work environment than two employees of the same companies who work in different departments. The statistical analysis, therefore, must account for this so-called nested structure. It makes a difference for a sample

280

KAI S. CORTINA ET AL.

of 100 whether we sampled 5 workers from each of 20 workgroups or 20 workers from 5 workgroups. The analysis of nested data structures has presented a challenge for statistics in the social sciences for a long time. Owing to major breakthroughs in statistical estimation procedures implemented in user-friendly software (e.g., Raudenbush & Bryk, 2002), the analysis of nested data structures has become very convenient and efﬁcient. The gist of the approach is to distinguish the levels of analysis and to derive and test hypotheses separately for each level. As an example, assume we want to test whether gender differences exist in the perceived working atmosphere. Knowing that the gender ratio varies dramatically across departments and along the management hierarchy, it would be misleading to simply take a random sample of all employees and test for signiﬁcant gender differences. Therefore, it is necessary to be more speciﬁc with our research question. What we really want to know is whether men and women differ in their perceptions of the workplace climate within each workgroup. In other words, do they differ in their perceptions despite the fact that they are assessing the same work environment? More generally speaking, we are interested in ﬁnding individual variables (in this case, gender) that explain the variance within the workgroup. In HLM, this research question is posed on the ﬁrst level (i.e., the individual level). On the second level (in this case, the workgroup level), we might be interested in differences in workplace climate depending on the gender of the manager to whom the workgroup reports. Here, we are looking for variables that explain the variance between workgroups. Note that the number of independent units of observations differs for the two research questions. While we analyze employees as the unit of analysis on level 1, we analyze workgroups as the unit of analysis on level 2. Although we might appear to be testing two gender effects (gender of employee, gender of superior) so as to test both research questions, the sample sizes are different in that they vary by unit of analysis (by level). In addition to main effects, the nested data structure allows for testing of cross-level interactions. In the preceding example, the researcher might be interested in whether the gender discrepancy in the perception of the workplace climate is less pronounced if the immediate superior is a woman (perhaps based on a theory of gender differences in management styles). The hypothesis would be that the gender effect within the workgroup (level 1) is moderated by a variable on level 2. For a more technical account of the underlying statistical model, see Raudenbush and Bryk (2002).

Multivariate Latent Growth Models

281

Repeated Measurement as Nested Data Structure Although it is not immediately obvious, many longitudinal data sets can be considered nested in a similar way as individuals are nested within workgroups or school classes. Conceptually, repeated observations are nested data within individuals. Just as members of a workgroup tend to be more similar in their responses, so repeated measurements from the same person are likely to be more similar than repeated measures from different persons. As in the workgroup example, the variance between can be distinguished from the variance within: Variability in the outcome variable across measurement points counts as the variance within the higher level units (in this example, persons), which is different from the variance between persons or – more accurately – from the variance of the persons’ mean scores (average across measurements for each person). The ﬁnal missing link between the nested data structure and the latent growth model is the introduction of growth parameters as predictors of the variance within – that is, of the variability across measurement points within the individual. In the workplace climate example, we used employees’ gender as predictor for the variance within. As will be derived more formally later in this chapter, we do not have real variables as predictors in the repeated measurement model, but rather use dummy codes that specify the assumed latent growth process. This step is necessary because the formula to calculate variance is not sensitive for the sequence of values over time. To see how this works, consider the following example. In a four-wave study on the improvement in the workplace climate after an intervention, the within variance for person a, with the values 1.5 at the ﬁrst measurement point, 2.6 at the second, 3.7 on the third, and 4.9 at the fourth, is identical with the variance for person b, with the sequence 3.7, 1.5, 4.9, and 2.6. While person a’s measurements show almost perfectly linear growth, no clear pattern can be discerned for person b. If we introduce a dummy variable for linear increase, such as 1, 2, 3, and 4 for the ﬁrst, second, third, and fourth observations (linear trend), we would get an almost perfect correlation between linear trend and value sequence for person a and a correlation close to zero for person b. Formally, this relationship is expressed as a regression equation, with the measurement at a given time point serving as the dependent variable and the linear dummy code as predictor. We observe variance in the linear component on the level of individuals, which now deﬁnes level 2 (because level 1 is the level of repeated measurements). On level 2, it is now possible to identify predictors of variance in the linear

282

KAI S. CORTINA ET AL.

component and the regression intercept that reﬂects interindividual differences. For instance, the researcher might be interested in comparing the linear change for men and women independent of the general effect that men and women differ in their assessments of the workplace climate. A signiﬁcant effect of gender on the intercept would capture the differences prior to the intervention, while a signiﬁcant effect of gender on the linear trend would indicate that men and women differ in their perceptions of change of climate over the course of the study. Note that this effect of gender on the linear trend deﬁnes a cross-level interaction, because gender moderates the (linear) inﬂuence of time on the climate assessment. In practical application with three and more observations, there is no reason to limit the analysis of change to the linear trend component. It is possible to create a quadratic trend component for an improved prediction of variance over time. Taking the square of the linear coefﬁcients (1, 4, 9, 16) deﬁnes a dummy variable for the quadratic trend that can be included as a second predictor for the regression equation for level 1. Note, however, that this simple technique to create nonlinear trend components is of limited value because it creates multicollinearity of the dummy codes (for a discussion and alternatives, see Pedhazur, 1997).

Three-Level Models Obviously, the nested structure of repeated measurement within persons and the nested structure of persons within workgroups can be integrated into a three-level model, which features repeated measurement as level 1, the person as level 2, and the workgroup as level 3. This model provides a very effective statistical tool to disentangle the effects of institutional variables, variables on the individual level, and the interaction of institutional and individual variables on developmental processes within organizations. Bryk and Raudenbush (1988) provide a detailed example in the area of school research. Conceptually, there is no limit in analyzing nested data structures that include more than three levels of nesting. Workgroups, for example, are nested within departments, which in turn are nested within companies. Companies are nested within industrial sectors, and so forth. For most research projects, however, it is unlikely that the database is sufﬁcient to reliably estimate coefﬁcients for higher levels, because sample sizes at higher levels are typically very small.

Multivariate Latent Growth Models

283

Multivariate Repeated Measurement Earlier, we linked analysis of repeated measurement and hierarchical data structures and demonstrated the integration of both concepts. This concept involves a generalization of the two-level hierarchical model to the threelevel hierarchical model. Based on the two-level repeated measurement model, we will now discuss how the LGM can be generalized for applications where multiple dependent variables are analyzed in their development over time. As mentioned earlier, one major focus of LGM is to identify predictors of the variance in trend coefﬁcients – in particular, the linear trend component. In the example, we used employees’ gender as a possible predictor. In fact, in traditional LGM, independent variables could be all sorts of sociodemographic variables, experimental conditions, or scale scores, as long as they can be assumed to be stable over time or stable for at least the observational period of the study. Stability of predictors of change is not necessary, however, because change in one variable could potentially be inﬂuenced by the change characteristics of another variable. This case, in which the development over time is analyzed simultaneously for two or more variables, is commonly referred to as MLGM (see Chan, Ramey, Ramey, & Schmitt, 2000; MacCallum & Kim, 2000). In MLGM, the statistical estimation of the correlation matrix of the growth parameters lies at the core of the analysis. For example, if the linear trend of two variables is analyzed simultaneously, four latent growth variables (intercept and linear trend for both variables) and their intercorrelations help to answer theoretically relevant questions: 1. Are the two constructs related at the onset of the study (intercept– intercept correlation)? 2. Do those individuals who started off at a higher level gain more over time (intercept–slope correlation within construct)? 3. Is the linear trend in one variable associated with the initial value of the other (intercept–slope correlation across construct)? 4. Are both linear trends associated (slope–slope correlation)? Similar to other statistical techniques in the analysis of longitudinal data, MLGM analysis is based on the covariance matrix and the mean values of the variables involved, although the rationale of the approach might suggest that individual growth curves are being analyzed. If the correlations, variances, and means are analyzed in a MLGM, should it not be possible to

284

KAI S. CORTINA ET AL.

see major trends by simply looking at these data? For example, if the two variables are correlated in the beginning (intercept–intercept correlation) and have correlated liner trends (slope–slope correlation), would it not imply that the correlation between the two variables at each time point increases over time? In the remainder of this chapter, we will explore these questions systematically, starting with the simple change model that serves as the basis for latent growth modeling. Using simulated data, we will demonstrate that it is, in fact, possible to identify major MLGM trends prior to the statistical analysis. In many cases, a systematic inspection of the covariance matrix helps to identify irregularities in the data when the MLGM estimates linear functions for all variables under investigation. We will suggest simple guidelines for checking a covariance matrix, concluding with an illustration of the use of these guidelines with a data set taken from a published study.

THE CONCEPT OF CHANGE IN LONGITUDINAL RESEARCH In the social sciences, we often use ‘‘analysis of longitudinal data’’ and ‘‘analysis of change’’ as interchangeable terms. Common sense understanding of the term ‘‘change’’ implies the repeated measurement of an entity on the same scale (‘‘Judith is 2 inches taller today than a year ago’’). In the case of two time points, the difference between pretest (time 1) and post-test (time 2) deﬁnes the change score. Longitudinal studies in the social sciences are often based on repeated measures using different instruments/ scales to measure the same construct over time. In this scenario, individual change scores are not deﬁned and longitudinal analyses can be performed only using association measures based on standardization (i.e., Pearson’s correlation coefﬁcients). The correlation matrix of all variables relevant to the investigation (repeated measures and predictor variables) is then analyzed using multiple regression analysis, where the post-test score is the dependent variable and the pretest score is one of the predictor variables (autoregressive multiple regression, AMR). Because the pretest score is held constant in this analysis, a signiﬁcant regression weight for a predictor, other than the pretest measure, is usually interpreted as a causal factor of ‘‘unusual change.’’ More precisely, the signiﬁcant variable predicts unusual shifts of the relative position in the distribution from pretest to post-test.

Multivariate Latent Growth Models

285

A simple example helps to illustrate this point. Suppose a researcher wants to investigate whether women, after graduation from business school, not only start with lower salaries at their ﬁrst jobs than their male counterparts, but also experience lower income increases over the course of the next 5 years. Income information for participants was collected annually. Assuming that everyone is paid in U.S. currency, the statistical analysis is straightforward, as the dependent variable (income) is measured on the same scale. But what if participants begin to work in different countries and receive paychecks in various currencies for which the researcher does not know the exchange rates (different scales)? To make the analogy more accurate, let us assume that the participants move together to a different country every year. Using AMR, the researcher is able to test both hypotheses without knowing the exchange rates. The ﬁrst hypothesis would be tested using a simple two-group comparison of males’ and females’ ﬁrst post-graduate salary (controlling for other relevant variables, such as GPA (Grade Point Average), industrial branch, or age). The second hypothesis, the analysis of the gender-speciﬁc change rate, requires a more complicated statistical analysis because change scores cannot be calculated. In this case, instead of calculating individual change scores, we would use the income distribution for each measurement point as the basis for analysis. At each time point, the distribution is rescaled in such a way that the means and standard deviations are the same (usually standardized with means rescaled to zero and the standard deviation to 1). If the gender gap widens over time, it is reﬂected in an increase of mean differences between men and women in the transformed (standardized) income variables. This exempliﬁes the rationale of the AMR in a nontechnical manner. Note that we are able to test the hypotheses, but we are unable to draw any conclusions about the overall development of income in the sample. All graduates might continuously earn more money over time or make less and less money each year. Perhaps some are more successful in gaining income increases than others whose income stagnates or decreases. The standardization results in an average of zero at each time point, resulting in a nil development. By deﬁnition, we are also unable to make use of the differences in observed variance over time. Presumably due to its versatility and simple speciﬁcation in statistical software, AMR has become a popular statistical approach in analyzing longitudinal data. This is true even in those cases where the data were collected using identical empirical scales. The fallback to the weaker statistical model (i.e., discarding the variance information) is unnecessary

286

KAI S. CORTINA ET AL.

and can make interpretation of empirical ﬁndings ambiguous (Raudenbush & Bryk, 1987). While the advantages of a real analysis of change are not always obvious when only one dependent variable and its development are investigated in a simple pretest/post-test study, the conceptual limitations of AMR become obvious when we want to understand developmental trends across more than two time points and when the researcher is also interested in the simultaneous development of two or more variables. When three waves of data collection are realized, AMR with the third time point as the dependent variable not only controls for the second time point as a predictor, but can also include the ﬁrst measurement point to account for variance that is not explained by the scores from the second wave. If both prior measurements are signiﬁcant predictors, the autoregressive process (second-order AMR) is often difﬁcult to interpret because it alludes to nonlinearity in the change process under investigation. Somewhat paradoxically, the nonlinearity cannot be further analyzed in the AMR framework, because it implies the assumption of a constant measurement scale. With more measurement occasions, complex autoregressive structures can emerge, turning a sensible interpretation into a challenging task. In the event that the measurement scale remains the same over time, there would be no reason to discard the variance information. With three or more repeated measurements, it is often helpful to reduce the complexity of the change model by analyzing variability in trajectory parameters. Although path analytic techniques make it very convenient to generalize the AMR to the analysis of more than one variable over time, a conceptual limitation exists in that the aim of the analysis is to explain the variances of each dependent variable separately for each measurement point rather than analyzing ongoing trends that persist over the entire observational period. When it is obvious, or empirically defensible to assume, that the measurement scale is the same for all repeated observations, LGM is a very powerful statistical approach. This modeling technique was ﬁrst advocated by Meredith and Tisak (1990). Pivotal to the LGM approach is the analysis of variance in the parameters that describe the trajectories over time rather than the analysis of longitudinal data with respect to the interindividual differences at each measurement point. The LGM assumes – and takes advantage of the fact – that the researcher has some theoretical idea about the general pattern or shape of the change over the time period under observation. This theoretical input into the data analysis reduces the complexity of the analysis dramatically, particularly if the research focus lies

Multivariate Latent Growth Models

287

on the analysis of (curvi-) linear development when data were collected on many (at least three) occasions. In case of a strictly linear model, the longitudinal analysis is basically reduced to two parameters for each variable (intercept and slope), irrespective of the number of observation points. Whether the assumed model of the developmental process accurately describes the actual change process in the population becomes part of the empirical model test. Over the past decade or so, LGM has become very popular, especially in those disciplines in the social sciences in which empirical longitudinal research is based mainly on nonexperimental studies, such as developmental psychology (e.g., McArdle & Bell, 2000), psychopathology (Curran, Stice, & Chassin, 1999; Duncan, Duncan, & Hops, 1994; Stoolmiller, 1994), and organizational psychology (Chan & Schmitt, 2000). Different variants and reﬁnements of the approach have been suggested, particularly with respect to integrating a ﬂexible measurement error structure into the model (for an overview, see Chan, 2001; Duncan, Duncan, Strycker, Li, & Alpert, 1999). The popularity of LGM is partly due to the fact that its realization was demonstrated almost simultaneously in two developed statistical frameworks. Bryk and Raudenbush (1988) used HLM to demonstrate the usefulness of the polynomial decomposition of change over time. In the framework of structural equation modeling (SEM), McArdle (1988) was a pioneer in estimating models that integrated covariance and mean scores based on the LGM idea. In both statistical frameworks, LGM was developed before the seminal papers by Meredith and Tisak (1990) and Willett and Sayer (1994) were published. Although the generalization of LGM to the simultaneous analysis of two and more variables over time is straightforward in the SEM as well as in the HLM framework (MacCallum & Kim, 2000; MacCallum, Kim, Malarkey, & Kiecolt-Glaser, 1997), MLGM is not very common in the applied literature compared to its ‘‘competitor,’’ the cross-lagged structural equation model, where mutual inﬂuences of variables over time are speciﬁed as regression coefﬁcients across waves (Schnabel, 1996). One of the shortcomings of LGM is its inability to take measurement error into account (Chan, 2001). The idea of latent factors measured by several indicators, as is standard in structural equation models, does not immediately lend itself to be combined with LGM. It is not by chance that the term ‘‘latent’’ has a different meaning in LGM than in factor analysis approaches. Recently, however, Sayer and Cumsille (2001) have demonstrated how both approaches can be combined. It is likely that we will see more applications of MGLM controlling for measurement error in the future.

288

KAI S. CORTINA ET AL.

Theory of Change and LGM in HLM Recent suggestions from methodologists about ways to overcome shortcomings in the use of LGM and MLGM and to integrate the autoregressive and latent growth models (Curran & Bollen, 2001) should not lead to the misconception that no substantial difference exists between autoregressive and LGM. Unlike the longitudinal autoregression approach, LGM can be derived directly from a common sense notion of change. In this section, we brieﬂy review the basic algebra and demonstrate why variance and variance development over time provide relevant information for longitudinal analysis of data that are measured on the same scale. The core idea of a change model is simple. Formalizing the colloquial notion of change, we suppose that a measurement score y1i of a given person i is a composite of his or her score at a prior measurement point (y0i) and the difference between the two scores (change score d10i ) y1i ¼ y0i þ d10i

(1)

For a further measurement point, we deﬁne y2i ¼ y1i þ d21i ¼ y0i þ d10i þ d21i

(2)

If we assume that the development of person i is linear and the time interval between measurement points is the same, the change score is the same as well ðdi ¼ d10i ¼ d21i Þ: Eq. (2) simpliﬁes to y2i ¼ y0i þ 2 di

(3)

For the general case of occasion t ðt ¼ 0; 1; 2; . . . ; kÞ; we can write yti ¼ y0i þ di t

(4)

Note that in Eq. (4) the score of a person i is at any time determined by two parameters: the initial value (intercept) yi0 and the person-speciﬁc growth rate (or slope) di : In the notation of HLM, the symbol p0 is used for the intercept and the symbol p1 for the slope. Following Raudenbush and Bryk (2002, p. 163), Eq. (4) can be rewritten as follows: yti ¼ p0i þ p1i ati þ eti

(5)

Instead of the simple time variable used in Eq. (4), the formula uses ati because the general HLM allows the measurement points to vary from person to person, which is not relevant for the purpose of this chapter. The formula also includes a random error term, eti ; to take into consideration the fact that even if individual developmental trajectories exactly follow a linear line, some random components usually distort the measurement slightly.

Multivariate Latent Growth Models

289

For the sake of simplicity, we will drop the random component from the formula because it is not relevant for the argument. As a consequence, Eq. (5) simpliﬁes to yti ¼ p0i þ p1i at

(6)

This simple formula makes sense only if p0 and p1 are measured on the same metric. Otherwise, the scale of yt is not deﬁned. Note that Eq. (6) already reduces the complexity of the information in the original data because it speciﬁes only two parameters for each individual instead of k observations. While it might be of some interest for the researcher to test whether this ‘‘latent’’ linear model adequately describes the data at hand, the major purpose of the empirical investigation usually is to ﬁnd predictor variables that explain the speciﬁc score of the individual in terms of both parameters, intercept and slope. If our data set includes a predictor variable x (e.g., a personal characteristic such as level of education), we can formulate two separate regression equations for both parameters. In the notation of HLM p0i ¼ b00 þ b01 xi þ r0i p1i ¼ b10 þ b11 xi þ r1i

ð7Þ

In HLM, these equations constitute a higher level (level 2) because a stable person-level characteristic is used to predict the change parameters on level 1. Although the equations look very similar to the level 1 Eq. (6), the interpretation of the coefﬁcients is very different. The beta weights have no person index (i), which means that they are the same for all subjects. Unlike in the level 1 Eq. (6), the scale of x is different from the scales of both p parameters. As a consequence, it is convenient in many cases to use the standardized version because the coefﬁcients b00 and b10 have no substantial meaning. The coefﬁcients b01 and b11 then reﬂect the effects of x on the intercept and the slope, respectively. In the MLGM, where we express the development of two variables, A and B, simultaneously as linear functions, it is also possible to use the linear parameters as mutual predictors because they vary across individuals but not over time. From the two level 1 equations yAti ¼ pA0i þ pA1i at yBti ¼ pB0i þ pB1i at

ð8Þ

it would be possible, for example, for the latent growth process for variable A to specify pA1i ¼ bA00 þ bA01 pB0i þ rA0i

(9)

290

KAI S. CORTINA ET AL.

In this case, the linear growth in variable A is predicted by the intercept of variable B. Eq. (9) is only one equation out of four on level 2, however, and it is algebraically possible to include the intercept and slope of variable B and even the intercept of A so that the regression on the slope of A can be meaningful. This is particularly the case when both intercepts are correlated and the purpose of the analysis is to test whether the intercept of B inﬂuences the slope of A, even if we control for the intercept of A. In practical applications, of course, it is wise to be parsimonious in the model speciﬁcations on level 2; some predictions are logically questionable. For instance, it is difﬁcult to explain how the slope of B can affect the intercept of A. It is always a good idea in MLGM to ﬁrst estimate the correlation matrix for the latent growth parameters and only then to decide on theoretical ground whether a regression equation on level 2 would lead to a more precise test of a speciﬁc hypothesis.

PREDICTING VARIANCE DEVELOPMENT In Eqs. (1)–(9), change was deﬁned on the individual level and the regression equations were speciﬁed accordingly. Nonetheless, in the statistical analysis of change, we are not able to analyze change for each individual or their trajectories over time. Instead, interindividual variability is analyzed based on the variance of the distribution. The aim of the statistical analysis is the prediction/explanation of variance. Note that this is quite a leap, as the variance captures speciﬁc information only about the variability of a parameter. It is exhaustive only in case of a normal distribution. Although most students who took an introductory statistics class are able to calculate the variance, it is not common knowledge that variance algebra also provides rules to calculate the variance of sums (or differences) of two variables based on both variances and the covariance. It is very instructive to use basic variance algebra to see how the variance of a variable develops over time if the change model given in Eq. (6) holds. Obviously, the repeated measurement is deﬁned as a composite of two variables, the intercept and the slope. Therefore, the variance of the variable at the second time point is the sum of the variance of the variable at the ﬁrst measurement (intercept), the variance of the slope, and two times the covariance of intercept and slope. Formally, we write s2y1 ¼ s2p0 þ s2p1 þ 2 sp0 =p1

(10)

Multivariate Latent Growth Models

291

where s2p1 is the variance of the difference score and sp0 =p1 the covariance between initial score and change. Eq. (10) is strictly derived from Eq. (6) for at ¼ 1 ð) y1i ¼ p0i þ p1i Þ: Therefore, it deﬁnes the fundamental variance decomposition in the analysis of change from pretest to post-test. For simplicity, we do not distinguish between sample and population statistics throughout this chapter, because inferential statistics are not of major relevance for the argument. For the same reason, the distinction between true score and measurement error is not introduced here. If error terms would be considered as independent distortion terms, the measurement error would not appear in Eq. (10) because, by deﬁnition, it contributes to neither the variance of the second time point nor the covariance between intercept and slope. The measurement error for the second time point would simply be added to the right side of the equation and render the association between the ﬁrst and second time points nondeterministic. In the examples presented later in this chapter, we will use measurement error and their ‘‘attenuation’’ effect to make the examples more realistic. Eq. (10) reveals some interesting relationships when we assume that people (or any other unit of analysis) in most empirical studies differ in their change scores (i.e., s2p1 40). In this case, the variance of the variable must increase over time if the change scores and initial scores are independent (i.e., if sp0 =p1 ¼ 0). Contrary to this expectation (often called the ‘‘fanspread’’ effect), we generally do not see a steady variance increase in longitudinal studies in the social sciences. If the observed variance does not increase, we can immediately conclude that a negative covariance between initial value and change must exist which ‘‘compensates’’ for the variance in the change scores that is added to the variance of the initial score and which, by deﬁnition, cannot be negative. Note that this conclusion also holds if the variable is measured with additional error variance, as long as the amount of error variance is the same at both measurement points and the error terms are uncorrelated. For practical purposes, LGM is not a very helpful statistical tool if only two measurement points are realized because the linear change model in Eq. (6) is trivially correct and no inferential statistics can be calculated. Estimation and signiﬁcance testing of parameters in a linear model require at least three measurements. It is, therefore, useful to generalize the variance development for more than two measurement points. To keep the algebra simple when we expand Eq. (10) to reﬂect the variance development until the second follow-up, we assume that the slope parameter is exactly the same for each individual, which also means that the covariance between intercept and

292

KAI S. CORTINA ET AL.

slope remains constant over time. Therefore, the correlation between the intercept and the ﬁrst slope (between ﬁrst measurement and ﬁrst follow-up) is the same as the correlation between the intercept and the second slope (between ﬁrst and second follow-up). Given that the covariance of a variable with itself is its variance (e.g., sp1 =p1 ¼ s2p1 ), the following variance decomposition can be derived for the third measurement point ða ¼ 2Þ s2y2 ¼ s2y1 þ s2p1 þ 2 sy1 =p1 ¼ ½s2p0 þ s2p1 þ 2 sp0 =p1 þ s2p1 þ 2 sðp0 þp1 Þ=p1 ¼ s2p0 þ s2p1 þ 2 sp0 =p1 þ s2p1 þ 2 ðsp0 =p1 þ sp1 =p1 Þ ¼ s2p0 þ s2p1 þ 2 sp0 =p1 þ s2p1 þ 2 sp0 =p1 þ 2 s2p1 ¼ s2p0 þ 4 s2p1 þ 4 sp0 =p1

ð11Þ

As Eq. (11) demonstrates, the variance of any future measurement point in a linear model can always be decomposed into three components: (1) initial variance, (2) variance of the change scores, and (3) covariance of intercept/change. If the change is not strictly linear, which means that the change scores vary across time, Eq. (11) becomes more complex, because in this case the covariance between intercept and change is likely to be different for the second time interval. In addition, the covariance between the ﬁrst and second change scores is no longer equal to the variance of the slope, which renders a rather cumbersome formula. If we stick to the linear model, however, Eq. (11) can be easily rewritten for the general case of k measurement points s2yk1 ¼ s2p0 þ ðk 1Þ2 s2p1 þ 2 ðk 1Þ sp0 =p1

(12)

(Note that k refers to the total number of measurement points – that is, starting the ﬁrst measurement as k ¼ 1: In HLM notation, a refers to the number of repeated measures with the ﬁrst measurement noted as a ¼ 0: Therefore, k ¼ a þ 1:) It goes beyond the scope of this chapter to explicitly derive the variance decomposition for a model that includes a linear, a quadratic, and even higher-order polynomial terms. In such a case, the individual slope parameters do not remain constant over time. The linear component then expresses the average of the slopes. A certain slope between two measurement points – between the fourth and ﬁfth observations – is then partitioned into the linear component p¯ 1 and the nonlinear remainder ðp1ð54Þ p¯ 1 Þ; which, in turn, is analyzed for higher-order polynomial components (e.g., quadratic, cubic, or other trends). For each additional

Multivariate Latent Growth Models

293

trend component, another parameter is added to the model (for a discussion of alternative parameterizations, see Raudenbush & Bryk, 2002). In the quadratic case, for example, we could use the following parameterization: yti ¼ p0i þ p1i at þ p2i a2t

(13)

If the time lags between observations are equal (e.g., measurement every year), the linear trend component (0, 1, 2, 3,y, a) is squared to deﬁne a quadratic trend (0, 1, 4, 9,y, a2). In latent growth analysis, the three parameter variances and their correlations are of theoretical interest. In reality, theoretical predictions of higher-order polynomial components beyond a linear or quadratic component (e.g., saturation processes) are rare in the social sciences, and the statistical analysis of the quadratic component is usually limited to prove its existence and test for interindividual variability. The interpretation of correlates with complex change components (cubic and higher polynomial trends) runs into epistemological difﬁculties because it implies, for example, that the sign of the association between predictor and development ﬂips during the observational period.

Variance–Covariance Structure in Linear Latent Growth Development So far, only variance development for the linear model has been discussed. Applying variance algebra, however, it is easy to derive the entire variance– covariance matrix (VCM) for repeated measurements solely from the individual change model introduced in Eq. (6). This consideration is important for the statistical analysis in LGM, because the empirical VCM calculated from a sample is the ‘‘raw’’ material that is used to estimate the (unknown) parameters of the underlying growth model. Note that we again assume that the development for each individual is strictly linear and can, therefore, be described with two parameters. If there are only two time points (pre–post design), the covariance between the ﬁrst and second measurements is identical to the sum of the variance of the ﬁrst measurement (variance of the intercept s2p0 ) and the covariance of intercept and slope (sp0 =p1 ). If there are more than two time points, the variance algebra becomes more complicated. We abstain from deriving each element, choosing instead to summarize the formulas in Table 1. In addition to the formulas needed to calculate the VCM for the ﬁrst three measurements, the table provides the formulas to calculate the covariance between prior time points and the kth measurement point. Note that every element of the repeated measurement covariance matrix is

294

KAI S. CORTINA ET AL.

Table 1.

Structure of the Variance–Covariance Matrix for Repeated Measurement with Underlying Linear Growth.

Intercept ðk ¼ 1Þ

First Follow-Up ðk ¼ 2Þ

k¼1

s2p0

k¼2

s2p0 þ 1 sp0 =p1

s2p0 þ s2p1 þ 2 sp0 =p1

k¼3

s2p0 þ 2 sp0 =p1 ^ s2p0 þ ðk 1Þ sp0 =p1

s2p0 þ 2 s2p1 þ 3 sp0 =p1 ^ s2p0 þ ðk 1Þ s2p1 þk sp0 =p1

^ k¼k

Second Follow-Up ðk ¼ 3Þ

s2p0 þ 4 s2p1 þ 4 sp0 =p1 ^ s2p0 þ 2 ðk 1Þ s2p1 þðk þ 1Þ sp0 =p1

y

y

kth Follow-Up ðk ¼ kÞ

s2p0 þ ðk 1Þ2 s2p1 þ2 ðk 1Þ sp0 =p1

expressed as a linear composite of the three core elements of the change model: (1) variance of the intercept (s2p0 ); (2) variance of the slope (s2p1 ); and (3) covariance intercept–slope ð2 sp0 =p1 Þ: The elements in Table 1 are derived from the change model deﬁned in Eq. (6), which is deterministic in the sense that if a linear function adequately describes the longitudinal development for all individuals under observation and we know the three relevant parameters, the variances and covariances can be exactly calculated. Logically, the reverse prediction is valid as well: If the true population VCM is known and the strictly linear model is accurate, the three latent growth parameters can be determined algebraically. In practice, the population VCM is unknown and is estimated by an empirical VCM calculated from a sample of a certain size. Additional imprecision arises in the likely case that the linear model might not exactly ﬁt the underlying change process but does serve as a fairly appropriate approximation. As mentioned earlier, the strength of the reverse calculation (from an empirical VCM to the growth parameters) is further weakened if the variables under observation are subject to measurement error. Applying statistical estimation theory, we are nevertheless able to calculate estimates for the three theoretically relevant parameters that have certain statistical properties. Software usually calculates maximum-likelihood or generalized least-squares estimates for the growth parameters with standard errors, which can be used for signiﬁcance and hypothesis testing. For the remainder of this chapter, we are particularly interested in the speciﬁcs of the statistical estimation process. Indeed, the differences between

Multivariate Latent Growth Models

295

the statistical algorithms tend to be more of academic interest than to have practical relevance for the applied researcher. The estimates they produce are very similar. If the parameter estimates vary substantially in size between procedures, that outcome sheds doubt on the validity of the entire model. Instead, we want to demonstrate how the algebraic background unfolded earlier helps us to roughly analyze latent growth processes by informed inspection of the VCM. Because it is straightforward to generalize the rationale to the MLGM, we demonstrate this effect for the simplest multivariate case: two variables measured over three time points with an underlying linear developmental pattern. Before moving on to the multivariate case, it is instructive to see how the elements of the structural VCM in Table 1 are inﬂuenced by standardization. This consideration has practical importance because many researchers are used to calculating and interpreting correlation matrices instead of covariance matrices. This practice is justiﬁed in cases where the measurement scales have no substantial meaning or do not stand in sensible relationship to each other. As emphasized earlier, neither condition is met in linear change models. Unfortunately, transforming the VCM into a correlation matrix not only gives away important information about the variance development (because all elements in the diagonal in the matrix in Table 1 are rescaled to 1), but also destroys the regularities in the covariance matrix. The correlation coefﬁcient is calculated by dividing the covariance of two variables by the product of both standard deviations. Using the formulas in the Table 1, we can, for example, calculate the correlation between the ﬁrst and second time points as follows: s2p þ 1 sp0 =p1 s2p0 þ sp0 =p1 ﬃ ﬃ¼ qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ ry1=2 qﬃﬃﬃﬃﬃﬃ q0ﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ s2p0 s2p0 þ s2p1 þ 2 sp0 =p1 sp0 s2p0 þ s2p1 þ 2 sp0 =p1

(14)

The covariance is divided by the product of the square roots of both variances (ﬁrst two diagonal elements in Table 1). As the formula reveals, all three linear growth parameters are involved in the calculation in a rather complex way. Although it is not easy to predict how the correlations will change if one of the elements changes, some conclusions can be drawn: 1. If and only if, the slope variance is zero (i.e., is a constant), the correlation between repeated measures becomes 1. In this case, the formula reduces to sp0 =sp0 because all covariances with a constant are zero. 2. The larger the variance of the slope, the smaller the correlation between repeated measures.

296

KAI S. CORTINA ET AL.

3. The effect of the intercept–slope covariance is difﬁcult to predict because it is part of the numerator and the denominator. Even more important for empirical research is the negative reverse conclusion that can be drawn for Eq. (14): Besides the trivial conclusion that a correlation of 1 indicates constant slopes, little can be inferred from the correlation coefﬁcients between measurement points about the underlying latent growth structure. This is primarily due to the fact that the ‘‘diluting’’ effect of the slope variance in the denominator is potentially counteracted by a possibly negative intercept–slope covariance. Regularities in the structure of the linear growth process can be seen only in the covariance matrix. In most empirical studies, the correlation matrix conceals more than it reveals.

MLGM and the Empirical Covariance Matrix In both statistical approaches that are commonly used to estimate LGM, it is relatively easy to analyze the latent growth for more than one variable simultaneously. MacCallum and collaborators (MacCallum & Kim, 2000; MacCallum et al., 1997) demonstrate the speciﬁcation of a MLGM for the same data set in both HLM and SEM. They also discuss some of the differences between both approaches (see also Schulenberg & Maggs, 2001). As mentioned earlier, it is wise to be parsimonious in the speciﬁcation of the latent growth process in MLGM. Besides the fact that multivariate statistical analyses are generally prone to an inﬂation in the number of parameters, it is easy in MLGM to cross epistemological boundaries. While it might be a simple matter to provide theory-driven predictions about the correlation of intercepts and linear trends for two variables, most researchers would be hard-pressed to explain correlations of higher-order polynomials across variables. At least in the social sciences, theories are usually not speciﬁc enough to give a sound explanation for a phenomenon such as the correlation between the cubic trend in variable A and the quadratic trend in variable B. It is not surprising that practical applications of MLGM typically conﬁne the multivariate analysis to linear latent growth models and include higher-order polynomials only on an exploratory basis. Similar to the derivations for the univariate repeated measurement case given earlier, we now want to demonstrate that variance algebra is helpful in disentangling the covariance structure for two variables repeatedly measured over time. Although limited to the case of two variables measured

Multivariate Latent Growth Models

297

at three occasions and to a strictly linear model, it is a straightforward matter to generalize the following considerations for more measurement points and more than two variables. Assume we have two variables A and B measured at three occasions ðk ¼ 1; 2; 3Þ: If for both variables the simple linear individual change model deﬁned by Eq. (6) holds, we can infer that the variance and covariance development over time for each variable separately is determined by six variance components (for both variables: variance of intercept, variance of slope, covariance of intercept and slope). While this determines the 3 3 covariance matrix for each variable over the three measurement points separately, a part of the [3+3] [3+3] covariance matrix of all six variables involved in the MLGM remains undetermined. The formulas given in Table 1 do not help to calculate, for instance, the correlation between the ﬁrst measurement point in variable A and the third measurement point in variable B. Intuitively, one might guess that these cross-variable/cross-time covariances depend on a mix of the six latent variance components mentioned above. In MLGM, four additional latent variance components need to be considered. This becomes immediately clear when we look at the covariance matrix of the intercepts and slopes for both variables (see Table 2). To distinguish between the parameters for the two variables, A or B was added to the indices. In addition to the univariate latent components discussed earlier, we are interested in the latent covariance between both intercepts ðsAp0 =Bp0 Þ and the covariance of both slopes ðsAp1 =Bp1 Þ: We also have to consider the covariance between intercept A and slope B ðsAp0 =Bp1 Þ and the covariance between intercept B and slope A ðsAp1 =Bp0 Þ: Note that the latter terms are not identical. In many applications of MLGM, calculating the elements of Table 2 given an empirical data set and testing them for signiﬁcance is the ultimate goal of the analysis. For this matrix, standardizing often makes sense because the scales of A and B do not have to be the same. The resulting correlation matrix answers theoretically relevant questions. It might be of some theoretical interest, for example, that in a study of economic development in 15 OECD (Organization of Economic Cooperation and Development) countries, the linear increase in federal debt over time correlates positively with the linear increase in unemployment rate – despite the fact that both variables are negatively correlated at the onset of the study. Loosely speaking, the purpose of the MLGM in the two-variable case is to ‘‘condense’’ the empirical covariance matrix to the 4 4 matrix given in Table 2. The size of the empirical covariance matrix depends on the number

298

KAI S. CORTINA ET AL.

Table 2.

Intercept A Slope A Intercept B Slope B

Variance–Covariance Matrix of Linear Latent Growth Parameters for Two Variables. Intercept A

Slope A

Intercept B

Slope B

s2Ap0 sAp0 =Ap1 sAp0 =Bp0 sAp0 =Bp1

s2Ap1 sAp1 =Bp0 sAp1 =Bp1

s2Bp0 sBp0 =Bp1

s2Bp1

of measurement points. The more measurement points realized, the more reduction of complexity is achieved using the MLGM. With only three measurement points, as in the numerical examples given later in this chapter, the beneﬁt is not as noticeable: The 21 nonredundant elements of the 6 6 empirical covariance matrix are reduced to 10 nonredundant elements in Table 2. However, the number of theoretical elements in a linear MLGM remains constant irrespective of the number of repeated measurements. In the case of 6 measurement points, for example, the 10 latent elements explain the structural composition of a matrix containing 78 elements. In Table 1, we demonstrated how each element of a 3 3 repeatedmeasurement matrix for one variable could be expressed as a composite of the three univariate LGM variance components. For the following generalization of these conceptual considerations to the case of a 6 6 covariance matrix of two variables at three measurement points, it is helpful for the sake of clarity to replace the 10 variance components deﬁned in Table 2 with simpler symbols. Therefore, we deﬁne a ¼ s2Ap0

d ¼ s2Bp0

g ¼ sAp0 =Bp0

s2Ap1

s2Bp1

h ¼ sAp1 =Bp1 i ¼ sAp0 =Bp1

b¼ c ¼ sAp0 =Ap1

e¼ f ¼ sBp0 =Bp1

(15)

j ¼ sAp1 =Bp0 Applying variance algebra to all elements of the covariance matrix, the structural decomposition of the covariance matrix shown in Table 3 holds. Note that the submatrix of the ﬁrst three columns and ﬁrst three rows in Table 3 is identical to Table 1 for variable A when the letters are replaced with the respective variance components. The same is true for variable B in

Multivariate Latent Growth Models

299

the lower-right triangle. Re-emerging in the joint multivariate covariance matrix are the nine elements within the highlighted frame, consisting of all ‘‘cross-wave-cross-variable’’ (CWCV) covariances. A closer look at Table 3 reveals a couple of interesting aspects, which, as we will show later in two examples, are very helpful for a systematic inspection of any empirical covariance matrix. First, all CWCV elements are additive composites of the latent components g, h, i, and j. This, in turn, means that they do not depend on a, b, c, d, e, or f. The CWCV part of the matrix is independent from the variances of the intercepts and slopes and the intercept–slope covariance for each variable. Therefore, it is possible to detect tendencies in the covariance between the two variables independent from the analysis of the latent growth parameters that involve each variable separately. This independence does not mean that the relevant latent growth elements for the CWCV matrix (g, h, i, j) can vary completely independent from other elements. It is easy to show, for example, that the covariance between the slopes of A and B (h) must be zero if either slope variance (b or d) is zero. For practical purposes, these algebraic associations are not relevant because we interpret and analyze a given empirical covariance matrix. If we were to construct a covariance matrix based on latent growth parameters of our own choosing, ignoring them would yield an ill-conditioned covariance matrix.

Table 3.

Multivariate Variances and Covariance Matrix for t. Variable A

K¼1 Variable K ¼1 Variable K ¼2 Variable K ¼3 Variable K ¼1 Variable K ¼2 Variable K ¼3

K¼2

Variable B K¼3

K¼1

K ¼2

A

a

A

aþc

a þ b þ 2c

A

a þ 2c

a þ 2b þ 3c

a þ 4b þ 4c

B

g

gþj

g þ 2j

d

B

gþi

gþhþiþj

g þ 2h þ i þ 2j

d þf

d þ e þ 2f

B

g þ 2i

g þ 2h þ 2i þ j

g þ 4h þ 2i þ 2j

d þ 2f

d þ 2e þ 3f

K ¼3

d þ 4e þ 4f

300

KAI S. CORTINA ET AL.

Second, the ‘‘inner structure’’ of the CWCV matrix is very similar to the covariance structure of the repeated measurements for each variable over time (i.e., the covariances between ﬁrst, second, and third measurement points for variable A or B). The similarity becomes obvious when we look at the special case where the covariance of (intercept A, slope B) is identical to the covariance of (intercept B, slope A). In this case ði ¼ jÞ; the CWCV is symmetrical. The formulas to calculate the development of the covariance between A and B within each time point become structurally identical to the formulas to calculate the variance development for each variable. For the covariance between A and B at k ¼ 2; for example, the formula g þ h þ i þ j becomes g þ h þ 2i; for k ¼ 3; the formula g þ 4h þ 2i þ 2j simpliﬁes to g þ 4h þ 4i: In the case that the CWCV is not symmetrical (i.e., the covariances between the intercepts and slopes are not the same), the covariance development is still strictly additive. As derived earlier for the univariate LGM, a decrease of the covariance between A and B in followups indicates negative latent growth parameters. However, while a univariate variance decrease clearly means that the intercept–slope covariance is negative, a decrease in the covariance can also be due to a negative covariance (slope A, slope B). Third, the CWCV matrix immediately reveals the sign for both crossvariable intercept slope covariances (i and j) by simply comparing two elements. The covariance of A at k ¼ 1 and B at k ¼ 2 is the sum of the covariance of A and B at k ¼ 1 (g) and the covariance of the intercept A and the slope B (i). If this covariance is smaller than g (the element above it in the matrix), it follows that i is negative. If the difference is positive, then i must be positive as well. In fact, the difference between both elements is the exact value of i. Similarly, if the element in the matrix next to the right of g (the covariance of A at k ¼ 2 and B ¼ 1) is smaller [larger] than g, it follows that j is negative [positive]. Again, the absolute value of j is given by the difference of both elements. Fourth, once the values of i and j are calculated, it becomes possible to determine the value of h by subtracting i, j, and g from the covariance of A/B at k ¼ 2 (which is deﬁned as g þ h þ i þ j). Fifth, looking at the variance and covariances of later measurement points is redundant when we look at an ideal matrix where the latent growth process for both variables is exactly linear. In empirical practice, additional comparisons are helpful to detect deviances from the linearity assumption. It is an empirical question, for example, as to whether the covariance between A at k ¼ 1 and B at k ¼ 3 is, in fact, as expected, equal to g þ 2i:

Multivariate Latent Growth Models

301

Latent Growth and a Given Empirical Covariance Matrix In previous sections, we ‘‘reconstructed’’ the covariance matrix analytically given that the latent growth model is linear and the parameters of the univariate or multivariate change are known. In empirical practice, the reverse is the more usual situation: The covariance matrix at hand should be used to make inferences about the unknown latent growth parameters. This chapter demonstrates that we can learn a great deal from a systematic look at the covariance matrix and, in fact, roughly estimate the values of the latent growth parameters without actually running a LGM or MLGM. Understanding the covariance structure of a given data set can also come in handy when interpreting ﬁndings of the statistical analysis. With a little practice, it becomes possible to locate misﬁt (nonlinearity) in the covariance matrix. As a ﬁrst step in this direction, this section discusses a sequence of artiﬁcial data sets to familiarize the reader with the rules to systematically check a multivariate covariance matrix. These rules will be summarized at the end of this section and will ﬁnally be applied to a real-word example in the last section. In passing, we explain why the correlation matrix of the variables is of little value for understanding the underlying latent growth process. Our starting point is the same as that used earlier: Two variables (A and B) are measured at three time points (k ¼ 3). Although a linear change model implies a linear trend for the mean scores as well (including a ﬂat line for no change), the development of mean values can be omitted here because it is easy to determine the direction and shape of the average scores over time. Central for LGM and for our consideration is the covariance or correlation matrix of the six resulting variables ðAK1 ; AK2 ; AK3 ; BK1 ; BK2 ; BK3 Þ: The objective of the multivariate latent growth analysis is to identify the ten underlying latent growth parameters as listed in Eq. (15):

The The The The The The

variance of the intercepts of both variables variance of the slopes of both variables covariance of intercept and slope within each variable covariance of both intercepts covariance of the slopes cross-variable intercept–slope covariance

Case 1: Independent Growth Perhaps the simplest model of theoretical interest is a model where all possible covariances are zero. The only relevant variance components are

302

KAI S. CORTINA ET AL.

the variance of the intercept and the variance of the slope for each variable, which are mutually uncorrelated. If we assume that the variance of A at the ﬁrst measurement point is 100 ðs2AK1 ¼ s2Ap0 ¼ 100Þ and the variance of the change over time is 5 ðs2Ap1 ¼ 5Þ; it follows from Eq. (10) that the variance of A at the second measurement (k ¼ 2) is s2AK2 ¼ 105: According to Eq. (11), the variance at the third measurement can be calculated as s2AK3 ¼ 100 þ 22 5 þ 0 ¼ 120: For simplicity, we assume the same intercept and change rate apply to variable B (i.e., s2BK1 ¼ s2Bp0 ¼ 100 and s2Bp1 ¼ 5), which results in s2BK2 ¼ 105 and s2BK3 ¼ 120: To make this example more realistic, we assume that both variables are measured with an additional (independent) error variance of VarðÞ ¼ 20 at each measurement occasion (equivalent to a reliability of 0.83 at time 1). Table 4 shows the covariance and correlation matrix for the independent growth model. The observed variances [true variance plus error variance – for example, for the ﬁrst time point: VarðAK1 Þ ¼ 100 þ 20 ¼ 120; for the second time point: VarðAK2 Þ ¼ 105 þ 20 ¼ 125] are the diagonal elements; the covariances are listed in the subdiagonal triangle with the corresponding correlation coefﬁcients in the supradiagonal triangle. Note that the true score variance at K1 [VarðtK1 Þ ¼ 100] is identical to both the covariance between times 1 and 2 and the covariance between times 1 and 3. However, because the variance increases but the covariance stays the same, the correlations with the ﬁrst measurement differ. In this case, they decrease over time (from 0.82 to 0.77). In the example, the adjacent correlations tend to slightly increase over time: The correlation between the ﬁrst and second measurement points is r ¼ 0:82; the correlation between times 2 and 3 is r ¼ 0:83: The correlation between times 3 and 4 (not presented) would be r ¼ 0:86: In this speciﬁc case, where the correlation between intercept and slope is exactly zero, the adjacent correlation must remain the same or increase over time. This logic, however, cannot be reversed. If we observe the adjacent correlation coefﬁcients to increase, we cannot infer that the intercept and slope are independent because this trend also occurs if they are positively correlated (see Case 2). Note that (independent) measurement error adds variance to the diagonal, but leaves all covariances unaffected. The correlations, in contrast, are calculated by dividing the covariance by the product of the standard deviations of the two respective variables and are, therefore, lowered by measurement error (attenuation effect).

Multivariate Latent Growth Models

Table 4.

AK1 AK2 AK3 BK1 BK2 BK3

303

Variances, Covariances, and Correlations for Case 1.

AK1

AK2

AK3

BK1

BK2

BK3

120 100 100 0 0 0

0.82 125 110 0 0 0

0.77 0.83 140 0 0 0

0.00 0.00 0.00 120 100 100

0.00 0.00 0.00 0.82 125 110

0.00 0.00 0.00 0.77 0.83 140

Case 2: Intercept Correlation Table 5 shows the same situation as depicted in Table 4 but with the additional assumption that the two measured variables are moderately correlated at the ﬁrst time point. However, the slope parameters remain independent; that is, they correlate with neither the intercepts nor each other. The initial covariance between A and B at time 1 ðsAp0=Bp0 ¼ sAK1BK1 ¼ 75Þ is retained at times 2 and 3 ðsAK1BK1 ¼ sAK2BK2 ¼ sAK3BK3 Þ: However, because the variance of both variables increases steadily, the concurrent correlations decrease, from the initial value of rAK1=KB1 ¼ 0:62 to rAK2=BK2 ¼ 0:60 and ﬁnally to rAK3=BK3 ¼ 0:54: At ﬁrst glance, it seems reasonable to predict that the concurrent correlation between A and B will always decrease from time 1 to 3, because both slopes add variance to the variables but do not contribute to the covariance between them, which in turn reduces the correlation. Although Table 5 conﬁrms this prediction, this is not necessarily the case if a negative intercept–slope variance exists, as it reduces the variance of the variables over time (as we will see in Case 4). The development of the concurrent correlation of A and B has no clear interpretation. Case 3: Intercept Correlation and Within-Variable Intercept–Slope Correlation Both positive and negative intercept–slope correlations are common phenomena in longitudinal analysis. A positive correlation, for example, is often found in experimental learning studies, where those students who start with higher scores tend to beneﬁt more from instruction. A negative correlation between intercept and slope typically occurs, for example, when the empirical scale has an upper limit and causes a ceiling effect; the potential gains of an individual are limited by the distance to the upper boundary. If the intercept–slope correlation is positive, the resulting

304

KAI S. CORTINA ET AL.

Table 5.

AK1 AK2 AK3 BK1 BK2 BK3

Variances, Covariances, and Correlations for Case 2.

AK1

AK2

AK3

BK1

BK2

BK3

120 100 100 75 75 75

0.82 125 110 75 75 75

0.77 0.83 140 75 75 75

0.62 0.61 0.58 120 100 100

0.61 0.60 0.57 0.82 125 110

0.58 0.57 0.54 0.77 0.83 140

covariance (or correlation) pattern is similar to the one given for Case 1. Owing to the accelerated increase in variance over time, the concurrent correlation wanes faster. In the case that the variance of a variable decreases over time, the intercept and slope must be negatively correlated (as can be derived from Eq. (10)). In our example, even a relatively small negative correlation of 0.31 results in shrinkage of the variance (see Table 6). The reverse conclusion is not true, however. Even if the variance increases over time, the correlation can still be negative as long as the variance in change ðsp1 Þ is more than two times larger than the negative covariance. Note that apart from the difference in variance development, a negative intercept–slope correlation often does not lead to negative values in the covariance or correlation matrix and is, therefore, less obvious. Given the formulas in Table 1, the value of the negative intercept–slope covariance can be immediately calculated for Table 6 when we analyze the development of the covariance of the ﬁrst measurement point with both consecutive time points. The covariance between the ﬁrst and second time points ðsAK1=AK2 ¼ 93Þ is larger than the covariance between the ﬁrst and third time points ðsAK1=AK3 ¼ 86Þ: The difference (sAK1=AK3 – sAK1=AK2 ¼ –7) is the negative covariance. Note that the difference between the variance of A at the ﬁrst time [VarðAK1 Þ ¼ 120] and the covariance at the ﬁrst and second times ðsAK1=AK2 ¼ 93Þ is much larger due to the additional measurement error. If measurement errors must be considered but can be assumed to be uncorrelated, the comparison described above is more adequate. Table 6 also reveals that the within-variable intercept–slope correlation does not affect the concurrent covariances of the two variables over time (they remain unchanged at sAKk=BKk ¼ 75), but it does affect the concurrent correlations: Unlike in Case 2, the correlations now tend to increase over time. This trend reﬂects the fact that the covariance (the numerator in the

Multivariate Latent Growth Models

Table 6.

AK1 AK2 AK3 BK1 BK2 BK3

305

Variances, Covariances, and Correlations for Case 3.

AK1

AK2

AK3

BK1

BK2

BK3

120 93 86 75 75 75

0.81 111 89 75 75 75

0.74 0.80 112 75 75 75

0.62 0.65 0.65 120 93 86

0.65 0.68 0.67 0.81 111 89

0.65 0.67 0.67 0.74 0.80 112

formula to calculate the correlation) is the same, but the denominator (the product of the standard deviations of both variables) decreases. Accordingly, the decline in the correlation shown in Case 2 would become more pronounced, if the intercept and the slope were positively correlated. In the example, a positive correlation of 0.31 would result in rA1B1 ¼ 0:62; rA2B2 ¼ 0:53; rA3B3 ¼ 0:45: At this point, it is obvious that the concurrent correlation between two variables is not a valid indicator of any underlying tendencies in the multivariate change model, because conceptually different change parameters can alter its value in opposite directions. Case 4: The (Almost) Everything Case When we introduce a positive covariance between the slopes to the latent growth process as discussed in Case 3, the weakness of the concurrent correlation as an indicator of growth trends becomes even more obvious. This positive slope covariance has a similar (positive) effect on the concurrent correlation as a negative within-variable intercept–slope covariance. However, in the covariance matrix, these two covariances affect different matrix elements and can, therefore, be easily distinguished. Unlike in Cases 2 and 3 (Table 7), the concurrent covariances between A and B now increase over time (from sAK1=BK1 ¼ 75 to sAK2=BK2 ¼ 77:5 to sAK3=BK3 ¼ 85). In case of a negative slope–slope correlation, the covariance would decrease over time and the correlation would, in turn, decrease (other things held constant). It should be noted that this model is not the most complex model possible. The cross-variable intercept–slope covariances (i.e., the covariance between the intercept of variable A and the slope of variable B, and vice versa) are still assumed to be zero. As mentioned in the analysis of Table 3, this can be derived from the fact that sAK1BK1 ¼ sAK1BK2 ¼ sAK1BK3 ¼ sAK2BK1 ¼ sAK3BK1 ¼ 75: If the covariance would increase starting from sAK1BK1 to the right in the CVCW submatrix, the covariance (slope A, intercept B) is

306

KAI S. CORTINA ET AL.

Table 7.

AK1 AK2 AK3 BK1 BK2 BK3

Variances, Covariances, and Correlations for Case 4.

AK1

AK2

AK3

BK1

BK2

BK3

120 93 86 75 75 75

0.81 111 89 75 77.5 80

0.74 0.80 112 75 80 85

0.62 0.65 0.65 120 93 86

0.65 0.70 0.72 0.81 111 89

0.65 0.72 0.76 0.74 0.80 112

positive; if the covariance would decline, it is negative. If we go vertically down from sAK1BK1 ; we can detect both positive and negative covariances between slope B and intercept A.

PRACTICAL RECOMMENDATIONS: A THREE-STEP APPROACH Despite the fact that software for LGM is readily available, its application is error-prone and minor ﬂaws in the syntax can easily lead to highly distorted conclusions. This problem is, of course, exacerbated in the multivariate case. An easy and robust technique to get at least a rough idea about the major developmental trends beforehand or a simple plausibility check of ﬁndings is certainly a useful tool. As derived analytically and demonstrated with an artiﬁcial data example, we suggest a three-step approach (TSA) of how to ‘‘read’’ the covariance matrix prior to undertaking statistical analysis. We will ultimately apply these rules to a published multivariate latent growth analysis and show how strikingly similar the simple estimates and the ﬁnal HLM estimates are. In applied MLGM with two variables, variance of the intercepts and slopes are estimated for both variables as well as all possible covariances between these four variables. For our consideration, the variance for both intercepts and their covariance are of minor importance because they are identical to the variances and covariance of the ﬁrst measurement point in the case that the linear change model holds perfectly. This was trivially true in the cases presented earlier, because the covariance matrices were constructed so that they ﬁt the model perfectly. In practical empirical research, a discrepancy between intercept variance and empirical variance

Multivariate Latent Growth Models

307

at the ﬁrst time point indicates a misﬁt between theoretical model and observed data. Similarly, a correlation estimate for the intercept in latent growth analysis that differs substantially from the simple correlation of both variables at time 1 can indicate that one or both latent growth models are nonlinear. As repeatedly emphasized, the correlation matrix, in general, and the development of the concurrent correlations, in particular, do not provide any reliable information about the underlying latent growth processes – except for special cases that are arguably rare in practical research. The concurrent correlation can, in fact, be highly misleading: In their ﬁve-wave, two-variable MLGM analysis, for example, MacCallum et al. (1997) reported a strong positive slope–slope correlation (r ¼ 0:87) for levels of two stress hormones combined with a positive correlation of both parameters at time 1 (r ¼ 0:29). One would expect a high and even increasing correlation between both hormone levels over time. Surprisingly, a reanalysis of their reported data reveals that the initial correlation of r ¼ 0:25 at the ﬁrst measurement point becomes continually smaller and even negative at the fourth and ﬁfth measurement points. The explanation for this phenomenon is simple: The effect of the slope–slope covariance on the correlation between measures of the two hormones over time was cancelled out by the antagonistic negative intercept–slope covariance within each variable (r ¼ 0:70 and 0:63). Hence, the ultimate ﬁrst advice (Step 0) when looking at the data for latent growth analysis is

Step 0: Forget the Correlation Matrix. All Relevant Information is in the Covariance Matrix To simplify the next set of rules, it is helpful to arbitrarily number the elements of the covariance matrix. We can then refer to elements of the matrix without writing out their meaning. Table 8 serves as reference table. Element 1 (E1) stands for the variance of the variable A at the ﬁrst measurement point, element 13 (E13) refers to the covariance between the third measurement point of A and the second measurement point of B, and so on. The associations between the latent growth components within the variables are logically independent from the covariance of these components across both variables. It is, therefore, a good idea to start with the analysis of the separate within-variable latent growth parameters.

308

KAI S. CORTINA ET AL.

Table 8.

AK1 AK2 AK3 BK1 BK2 BK3

Numbers for the Elements of the Covariance Matrix.

AK1

AK2

AK3

BK1

BK2

BK3

E1 E2 E4 E7 E11 E16

E3 E5 E8 E12 E17

E6 E9 E13 E18

E10 E14 E19

E15 E20

E21

Step 1: Analyze the Variance Development of Each Variable Over Time If the variance does not increase or has a tendency to decrease, the intercept–slope covariance must be negative. If the variance increases, the intercept–slope correlation can be (slightly) negative, zero, or positive. If the variables are associated with some measurement error, the best estimate for the covariance is the difference between E2 and E4 ð^sAp0 =Ap1 ¼ E4 E2Þ and between E14 and E19, respectively ð^sBp0 =Bp1 ¼ E19 E14Þ: Once both intercept–slope covariances are calculated, it is easy to estimate the variance of the slopes. We simply subtract – for each variable separately – the covariance twice from the variance at the second measurement point. The difference between the result and the variance at time 1 is the estimate of the slope variance. Formally s^2Ap1 ¼ E3 E1 2^sAp0 =Ap1 ¼ E3 E1 2ðE4 E2Þ s^2Bp1 ¼ E15 E10 2^sBp0 =Bp1 ¼ E15 E10 2ðE19 E14Þ Note that this procedure yields accurate estimates only if we can assume that the variance of the measurement error is the same at both time points. However, this assumption can be relaxed in practical application when reliability information is available for each measurement point. The reliability coefﬁcient can be used to calculate the error variance separately for each occasion. This also helps to estimate the true variances of A and B at the ﬁrst time point, which becomes relevant when we wish to transform the ﬁnal covariance matrix of the latent growth parameter into a correlation matrix. At the end of Step 1, we have estimates for all six univariate latent growth parameters. We can now focus on the cross-variable growth parameters. As Table 3 shows, the covariance of the intercepts is trivially available in the

Multivariate Latent Growth Models

309

covariance matrix. It is element E7 in the reference matrix in Table 8. We can therefore determine both cross-variable intercept–slope covariances by applying a similar procedure as in Step 1. For this step, measurement error is irrelevant because the covariances are not affected by error (if the measurement errors are independent).

Step 2: Calculate the Cross-Variable Intercept–Slope Covariances Although in practice a more reﬁned strategy should be used, all that is needed for Step 2 are two simple subtractions: s^Ap0 =Bp1 ¼ E8 E7 s^Ap1=Bp0 ¼ E11 E7 The more reﬁned strategy takes advantage of the fact, that in the case of three measurement points, we have more information available to calculate the estimates. We could also calculate s^Ap0 =Bp1 ¼ E9 E8 and s^Ap1 =Bp0 ¼ E16 E11: If the estimates differ from the preceding calculations, the average might be the best guess. At this point, the only parameter missing is perhaps the most important one: the covariance of the two slopes. Looking at Table 3, it is obvious that we can estimate this coefﬁcient in the last step.

Step 3: Estimate the Slope–Slope Covariance Using the Result from Step 2 Although we could develop a more reﬁned version that takes advantage of information redundancy, we give the simplest formula based on the covariance of the two variables at time 2 (E12) s^Ap1 =Bp1 ¼ E12 E7 s^Ap0 =Bp1 s^Ap1 =Bp0 ¼ E12 E7 ðE8 E7Þ ðE11 E7Þ ¼ E12 E7 E8 þ E7 E11 þ E7 ¼ E12 þ E7 E8 E11 It is straightforward to use this TSA in cases where more than three measurement points are available. As long as the underlying growth process is linear for both variables in the sense of the change model deﬁned in Eq. (10), additional measurement points can help to reﬁne the estimates by calculating them using different time intervals and averaging the results.

310

KAI S. CORTINA ET AL.

The TSA provides good estimates for the covariance matrix of the latent growth parameters. Because the variables involved are not measured on the same scale, however, the researcher is often interested in standardizing the matrix so as to obtain the correlations. This is directly possible if the variables are measured without error variance. If the reliabilities of the measures are known or estimated, the variances should be multiplied by their respective reliabilities prior to the standardization. Otherwise, the correlations between the parameters will be underestimated. The lower the reliability, the more pronounced the negative bias. At the same time, the signs of the coefﬁcients do not change.

The Three-Step Approach in the Context of Multi-Level Data When the multivariate latent growth model is part of a broader multi-level structure – for example, a structure in which individuals are nested in clusters (e.g., employees as part of a workgroup) – the TSA can be used to explore the effects of higher-order variables on all latent growth parameters discussed earlier, including the correlations between intercepts and slopes of two and more variables. The basic idea is to analyze the covariance matrices separately for subsamples deﬁned by higher-order variables. For example, if the researcher wants to know whether the improvement in work climate after an intervention occurs more rapidly in workgroups with a high percentage of women, the TSA can be applied by comparing the longitudinal VCM calculated for two subgroups deﬁned by a higher-level variable. In this case, the TSA is performed for workgroups having a relatively high percentage of women and is then contrasted with the analysis for those employees who belong to a workgroup having a low percentage of women. The distinction between both subsamples could be based on median-split of the gender ratio of the workgroups, but could also be based on extremegroup comparison (e.g., top 25% versus bottom 25% of the distribution). The latter approach is recommended only if data for many higher-order units are available, as the information for half of the sample would be excluded. In most cases, the sample size on higher levels is critical, so that median-split is the preferred approach. Although the TSA does not provide any inferential statistics, it nevertheless gives the researcher an idea of whether it would be worthwhile to pursue the hypothesis. If the TSA estimate for the growth parameter is smaller in the sample of workgroups with a higher percentage of female coworkers, the hypothesis would be rejected.

Multivariate Latent Growth Models

311

In the same way, it is possible to eyeball the effect of all higher-order variables in the model. This holds not only for the linear growth parameter, but also for all univariate or multivariate latent growth parameters (e.g., the correlation of slope parameters of two variables). Interaction terms of higherorder variables can further be explored when the respective variables are dichotomized or otherwise categorized. For example, a researcher might hypothesize that the faster climate improvement rate in workgroups with a high percentage of female workers would be more pronounced for workgroups headed by a female supervisor. In terms of analysis of variances, this situation would be described as an ordinal interaction. To explore this hypothesis using the TSA, we would divide both workgroups into those with male supervisors and those with female supervisors, resulting in four groups. The four longitudinal VCMs would be analyzed separately, which would allow us to compare the growth parameter estimates: The differences between workgroups with high and low percentages of female coworkers should be more pronounced for the groups with female supervisors. With this type of subgroup analysis, all intercepts and slopes and their covariances can be compared between groups. Again, the purpose of this analysis is not to replace the analysis using HLM but rather to supplement it with a systematic prescreening that helps to understand the underlying data structure. Example: Newcomer’s Advice Seeking and Relationship Building with Coworkers To illustrate the practical use of the TSA, we use parts of the data of a multivariate latent change analysis reported in Chan and Schmitt (2000). In this study, the authors investigated the dynamic in the interaction of new coworkers with their peers (in this case, other graduate students) and their supervisors (faculty members). Among other ﬁndings, technical information seeking from coworkers (TS) decreased over time, as predicted. The construct was measured with a four-item scale as suggested by Morrison (1993) (example item: ‘‘How frequently do you ask other graduate students how to perform a speciﬁc aspect of your work?’’). Over the same time period, relationship building (RB) – measured with an adapted four-item scale introduced by Ashford and Black (1996) – decreased as well (example item: ‘‘To what extent have you attended social gatherings organized by other graduate students in your program?’’). For both variables, a linear latent growth model described the change over time adequately. Therefore, the correlation of four parameters (intercept and slope for both variables) captures the multivariate dynamic over time. As reported, both latent intercepts had a correlate of r ¼ 0:32; indicating that

312

KAI S. CORTINA ET AL.

newcomers who seek technical support from coworkers tend to be more active in building social relationships with them as well. Interestingly, the slope parameters are negatively correlated, with r ¼ 0:24: Although not signiﬁcant in the study (with a sample size of N ¼ 146), this ﬁnding might indicate that those individuals who intensify their information seeking from their fellow students tend to reduce their socializing activities. The full correlation matrix of the four latent growth parameters is given in Table 9. For both variables, the intercept–slope correlation is (signiﬁcantly) negative; that is, students with higher initial scores in TS tend to show faster decreases in this behavior than students with a lower score at the ﬁrst measurement occasion. For RB, this effect was even more pronounced. For the purpose of this chapter, we want to systematically investigate how well these ﬁndings of the multivariate latent growth analysis can be predicted by a systematic look at the covariance matrix using the TSA. Although Chan and Schmitt (2000) did not report the zero-order correlations and standard deviations, it is possible to recalculate the covariance matrix for the observed variables based on the information given in Table 9, the reported variances for both variables at time 1 (s2 ¼ 0:8284 for TS, s2 ¼ 0:9484 for RB), and the reliability. To keep the analysis simple, we used the reported average reliability of 0.83 as a proxy and calculated the error variances for both variables (TS: VarðÞ ¼ 0:16; RB: VarðÞ ¼ 0:19). We assumed that the error variance remains the same over time, implicitly deﬁning a ‘‘congeneric’’ measurement model (Jo¨reskog, 1971). As in the original paper, no error covariance was assumed for repeated measures. It is worth mentioning that more complex error structures can be integrated in LGM, particularly in the SEM framework (Chan, 2001). Of course, change in the measurement properties of the observed variables over time runs counter to the idea of equivalent measurement and makes it harder to justify the assumption that the

Table 9.

Empirical Findings of the MLGM Taken from Chan and Schmitt (2000, p. 29).

Correlations

Intercept TS

Slope TS

Intercept RB

Slope RB

Intercept TS Slope TS Intercept RB Slope RB

1 0.35 0.32 0.17

1 0.10 0.24

1 0.54

1

Multivariate Latent Growth Models

313

measurement scale remains the same over time – a core prerequisite for latent growth modeling. The reproduced covariance matrix and the correlations (above the diagonal) of the repeated measurements are given in Table 10. To keep the analysis comparable to the examples presented earlier, only the ﬁrst three measurement points are given, although four were employed in the actual study. Table 10 gives the predicted mean scores, which we calculated from the reported mean score at the ﬁrst time point (M ¼ 2:82 and 3:31) and the average linear change rate per month (M ¼ 0:077 and 0:0717). The mean scores are not of central importance here, because it is easy to eyeball whether the mean development is approximately linear. Note, however, that a linear growth in the mean score is a necessary but not sufﬁcient condition for a latent linear process. It is intriguing to compare Table 10 with Table 9 before we start the TSA. As Table 9 shows, all but two correlations of the latent growth parameters are negative. None of the covariance or correlation coefﬁcients in Table 10 has a negative sign, however. It may be that this discrepancy is part of the fascination with LGM; from an ‘‘innocent’’-looking covariance matrix, LGM extracts strong associations between growth parameters. From the extensive explications given earlier, it should be clear that negative covariances on the level of latent growth parameters correspond to a decrease in certain covariance elements over time. In fact, even without calculating the exact numbers, the signs of most latent growth covariances are easy to see in Table 10: 1. The variance for both variables tends to decrease over time (bold values

Table 10. Means, Covariances, and Correlations among Three Measurement Points for TS and RB. Cov/Cor

TS 1

TS 2

TS 3

RB 1

RB 2

RB 3

TS 1 TS 2 TS 3 RB 1 RB 2 RB 3

0.988 0.749 0.670 0.284 0.312 0.341

0.80 0.890 0.712 0.258 0.275 0.292

0.71 0.79 0.914 0.233 0.238 0.244

0.27 0.26 0.23 1.138 0.853 0.758

0.32 0.29 0.25 0.81 0.983 0.732

0.36 0.33 0.27 0.75 0.78 0.896

Means

2.820

2.743

2.666

3.310

3.238

3.167

314

KAI S. CORTINA ET AL.

in the diagonal). This indicates negative intercept–slope covariance within each variable. 2. The covariance between TS and RB decreases from left to right in the ﬁrst part of the CVCW submatrix (0.284, 0.258, 0.233). From this, we can infer that the covariance between intercept RB and slope TS is negative. 3. The covariance between TS and RB increases in the ﬁrst column of the CVCW submatrix downward (0.284, 0.312, 0.341). From this, we can infer that the covariance between intercept TS and slope RB is positive. 4. The slope–slope covariance cannot be eyeballed in this case because it depends on whether the positive and negative covariances inferred in items 2 and 3 cancel each other out. We can see that the positive effect in item 3 appears to be stronger. This, in turn, would mean that the diagonal element in the CVCW submatrix (bold) would increase if the slope–slope covariance is zero or positive. However, it decreases, which is possible only if the slope–slope covariance is negative. Applying the TSA, we can calculate the ﬁgures more precisely: Step 1: s^Ap0 =Ap1 ¼ E4 E2 ¼ 0:670 0:749 ¼ 0:079 s^Bp0 =Bp1 ¼ E19 E14 ¼ 0:095 s^2Ap1 ¼ E3 E1 2^sAp0 =Ap1 ¼ 0:89 0:988 2ð0:079Þ ¼ 0:06 s^2Bp1 ¼ E15 E10 2sBp0 =Bp1 ¼ 0:983 1:138 2ð0:095Þ ¼ 0:035 Step 2: s^Ap0 =Bp1 ¼ E8 E7 ¼ 0:258 0:284 ¼ 0:026 s^Ap1 =Bp0 ¼ E11 E7 ¼ 0:028 Step 3: s^Ap1 =Bp1 ¼ E12 E7 s^Ap0 =Bp1 s^Ap1 =Bp0 ¼ 0:275 0:284 ð0:026Þ 0:028 ¼ 0:011: To see how well these simple calculations reproduce the correlation matrix resulting from a MLGM in HLM, we can calculate the correlation matrix based on the TSA covariance estimates and compare it with the correlation matrix given in Table 9. We use the raw variance estimates for TS and RB at the ﬁrst measurement point as given in Table 10, including the relevant measurement errors (0.988 and 1.138). This exercise gives us an impression of how distorted the TSA estimates are if we ignore the unreliability. Table 11 provides these estimates and, in parentheses, the correlation coefﬁcients if the error variances are taken into consideration. Comparing these coefﬁcients with the coefﬁcients reported by Chan and Schmitt (2000), our TSA estimates are strikingly accurate when the error

Multivariate Latent Growth Models

Table 11.

315

Covariance of the Latent Growth Parameters Based on the Three-Step Approach.

Correlations

Intercept TS

Slope TS

Intercept RB

Slope RB

Intercept TS Slope TS Intercept RB Slope RB

1 0.324 (0.354) 0.267 (0.320) 0.149 (0.163)

1 0.10 (0.109) 0.238 (0.238)

1 0.472 (0.517)

1

variance is controlled. As expected, the correlations tend to be underestimated when the unreliability is not taken into account. However, the differences are not very dramatic, suggesting that ignoring the unreliability still provides an informative estimate of the latent growth structure. This is at least true if the reliability, as in this case, lies in a range that is usually considered acceptable (approximately 0.80).

Limitations of the Three-Step Approach The TSA is certainly a very helpful heuristic to immediately understand the underlying latent growth process in a given data set. Throughout the chapter, the emphasis of the derivation of the TSA focused on the practical side, in an effort to develop a tool that is easy to apply. Nevertheless, some caveats need to be kept in mind when using this approach routinely with real data sets. In the delineation of the algebraic structure as well as in the development of the examples, we used a ‘‘top-down’’ procedure from the latent growth model to the covariance matrix. Only in the last example was a more realistic ‘‘bottom-up’’ rationale applied. The top-down perspective produced clear covariance patterns, which we can expect when the latent model is, in fact, true. More often than not, however, a linear model is assumed to hold only approximately. In this case, the covariance structure based on the strictly linear latent model may be distorted by more complex growth components, which the researcher wants to ignore. The empirical covariance matrix may then deviate substantially from the model-implied covariance matrix. As a consequence, the TSA-calculated values could differ more substantially from HLM or SEM estimates. If the differences are very pronounced, the modelﬁt of the latent growth model will also be very low, as indicated, for example, by a high chi-square statistic in SEM or a low reliability for the latent growth

316

KAI S. CORTINA ET AL.

parameters in HLM. Therefore, the results from the LGM must be interpreted carefully whether they are calculated with the TSA or otherwise. Another caveat is related to sample ﬂuctuations. Throughout this chapter, we deliberately ignored the distinction between sample statistics and population parameters. In practical applications of the (M)LGM, it is important to get a sense of the stability of the ﬁndings across samples – that is, to make judgments regarding how likely it is that the ﬁndings can be reproduced. Using the TSA, we get quick estimates of the parameters, but no standard errors for them. Therefore, the TSA cannot replace a real latent growth analysis using, for example, maximum-likelihood estimates, but it is certainly a useful supplement.

ACKNOWLEDGMENTS We dedicate this chapter to Professor Ingeborg Stelzl, of University of Marburg, Germany, in acknowledgment of her lasting inﬂuence on the methodological orientation of Kai Cortina and Hans Anand Pant. We also thank her for the valuable comments on an earlier draft of the chapter and give her credit for an algebraic proof she provided for one of the claims we make in the text.

REFERENCES Ashford, S. J., & Black, J. S. (1996). Proactivity during organizational entry: The role of desire for control. Journal of Applied Psychology, 81, 199–214. Bryk, A. S., & Raudenbush, S. W. (1988). Toward a more appropriate conceptualization of research on school effects: A three-level hierarchical linear model. American Journal of Education, 97, 65–108. Chan, D. (2001). Latent growth modeling. In: F. Drasgow & N. Schmitt (Eds), Measuring and analyzing behavior in organizations (pp. 302–349). New York: JosseyBass. Chan, D., Ramey, S., Ramey, C., & Schmitt, N. (2000). Modeling intraindividual changes in children’s social skills at home and at school: A multivariate latent growth approach to understanding between-settings differences in children’s social skill development. Multivariate Behavioral Research, 35(3), 365–396. Chan, D., & Schmitt, N. (2000). Interindividual differences in intraindividual changes in proactivity during organizational entry. A latent growth modeling approach to understanding newcomer adaptation. Journal of Applied Psychology, 85(2), 190–210. Curran, P. J., & Bollen, K. A. (2001). The best of both worlds: Combining autoregressive and latent curve models. In: L. M. Collins & A. G. Sayer (Eds), New methods for the analysis of change (pp. 107–135). Washington, DC: APA.

Multivariate Latent Growth Models

317

Curran, P. J., Stice, E., & Chassin, L. (1999). The relation between adolescent alcohol use and peer alcohol use: A longitudinal random coefﬁcient model. Journal of Consulting and Clinical Psychology, 65, 130–140. Dansereau, F., & Yammarino, F. J. (2003). Longitudinal processes in groups and organizations. Group and Organization Management, 28, 312–314. Dansereau, F., Yammarino, F. J., & Kohles, J. (1999). Multiple levels of analysis from a longitudinal perspective: Some implications for theory building. Academy of Management Review, 24, 346–357. Duncan, S. C., Duncan, T. E., Strycker, L. A., Li, F., & Alpert, A. (1999). An introduction to latent variable growth curve modeling: Concepts, issues, and applications. Mahwah, NJ: Erlbaum. Duncan, T. E., Duncan, S. C., & Hops, H. (1994). The effects of family cohesiveness and peer encouragement on the development of adolescent alcohol use: A cohort-sequential approach to the analysis of longitudinal data. Journal of Studies on Alcohol, 55, 588–599. Jo¨reskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109–133. MacCallum, R. C., & Kim, C. (2000). Modeling multivariate change. In: T. Little, K. Schnabel & J. Baumert (Eds), Modeling longitudinal and multilevel data (pp. 215–253). Mahwah, NJ: Erlbaum. MacCallum, R. C., Kim, C., Malarkey, W. B., & Kiecolt-Glaser, J. K. (1997). Studying multivariate change using multilevel models and latent curve models. Multivariate Behavioral Research, 32(3), 215–253. McArdle, J. J. (1988). Dynamic but structural equation modeling of repeated measures. In: J. R. Nesselroade & R. B. Cattell (Eds), Handbook of multivariate experimental psychology. New York: Plenum. McArdle, J. J., & Bell, R. Q. (2000). An introduction to latent growth models for developmental data analysis. In: T. D. Little, K. U. Schnabel & J. Baumert (Eds), Modeling longitudinal and multilevel data (pp. 69–107). Mahwah, NJ: Erlbaum. Meredith, W., & Tisak, J. (1990). Latent curve analysis. Psychometrika, 55, 107–122. Morrison, E. W. (1993). Newcomer information seeking: Exploring types, modes, sources, and outcomes. Academy of Management Journal, 36, 557–589. Pedhazur, E. J. (1997). Multiple regression in behavioral research: Explanation and prediction. Fort Worth: Harcourt College Publishers. Raudenbush, S. W., & Bryk, A. S. (1987). Examining correlates of diversity. Journal of Educational Statistics, 12(3), 241–269. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage. Sayer, A. G., & Cumsille, P. E. (2001). Second-order latent growth models. In: L. M. Collins & A. G. Sayer (Eds), New methods for the analysis of change (pp. 179–200). Washington, DC: APA. Schnabel, K. (1996). Latent difference models as alternatives to modeling residuals of crosslagged effects. In: U. Engel & J. Reinecke (Eds), Analysis of change: Advanced techniques in panel data analysis (pp. 251–278). Berlin/New York: Walter de Gruyter. Schulenberg, J., & Maggs, J. L. (2001). Moving targets: Modeling developmental trajectories of adolescent alcohol misuse, individual and peer risk factors and intervention effects. Applied Developmental Science, 5(4), 237–253.

318

KAI S. CORTINA ET AL.

Stoolmiller, M. (1994). Antisocial behavior, delinquent peer association and unsupervised wandering for boys: Growth and change from childhood to early adolescence. Multivariate Behavioral Research, 29, 263–288. Willett, J. B., & Sayer, A. G. (1994). Using covariance structure analysis to detect correlates and predictors of individual change over time. Psychological Bulletin, 116(2), 363–381.

MULTIVARIATE LATENT GROWTH MODELING: ISSUES ON PRELIMINARY DATA ANALYSES David Chan ABSTRACT Multivariate latent growth modeling (multivariate LGM) provides a flexible data analytic framework for representing and assessing crossdomain (i.e., between-constructs) relationships in intraindividual changes over time, which also allows incorporation of multiple levels of analysis. Using the chapter by Cortina, Pant, and Smith-Darden (this volume) as a point of departure, this chapter discusses important preliminary data analysis and interpretation issues prior to performing multivariate LGM analyses.

INTRODUCTION Much of the methodological discussion on multi-level research (e.g., Chan, 1998a; Hannan, 1971; Hofmann, 2002; James, Demaree, & Wolf, 1984; Kozlowski & Klein, 2000; Ostroff, 1993; Robinson, 1950; Rousseau, 1985) has focused on the conventional type of multi-level data where individuals (Level 1) are nested within groups (Level 2). One type of multi-level data where the multi-level structure is less obvious is longitudinal data obtained Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 319–334 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04014-2

319

320

DAVID CHAN

from measurements repeated on the same individuals over time. In such data, a multi-level structure is established with the repeated observations over time (Level 1) nested within individuals (Level 2). Although there are some additional complexities involved in data analyses, the basic logical structure of such longitudinal data is similar to the conventional multi-level data. But unlike the conventional multi-level analysis of hierarchical data where the concern is with interindividual differences associated with group membership, multi-level analysis of longitudinal data is concerned with modeling intraindividual change over time. Multi-level regression models typically employed in conventional multilevel analysis can also be used to analyze intraindividual changes over time (see Bryk & Raudenbush, 1992; Hofmann, Jacobs, & Baratta, 1993). However, the issues of changes over time are often very complex and may involve facets of change over time (e.g., conceptual changes in the constructs, changes in calibration of measurement, various types of time-related errorcovariance structures) that are not readily handled by multi-level regression models. In modeling change over time, the primary purposes are describing the nature of the trajectory of change and attempting to account for the interindividual differences in the functional forms or parameters of the trajectories by relating them to explanatory variables that may be in the form of experimentally manipulated or naturally occurring groups, time-invariant predictors, time-varying correlates or the trajectories of a different variable. Latent growth modeling (LGM) and its extensions, which are implemented in a structural equation modeling framework, are well suited to address these issues. In Chan (1998b), I provided a review of these issues and an exposition of the application of LGM techniques, as well as an overview comparison between latent variable models and multi-level regression models. The chapter by Cortina et al. (this volume) discussed the topic of multivariate LGM analysis, which is, as the name implies, a multivariate extension of the basic logic of the univariate LGM analysis. Multivariate LGM is a recent important methodological advance in longitudinal and, less obvious but no less important, multi-level research. Specifically, multivariate LGM provides a ﬂexible data analytic framework for representing and assessing cross-domain (i.e., between-constructs) relationships in intraindividual changes over time, which also allows for incorporation of multiple levels of analysis. Given the importance of multivariate LGM and that methodological exposition of the data analytic framework is only beginning to emerge in recent years, any methodological writings on the topic is a potentially welcomed addition to the literature. On the other hand, precisely because of the importance of and the infant (or at best adolescent) stage of

Multivariate Latent Growth Modeling

321

development (both in methodology and substantive applications) in multivariate LGM, it is critical that fundamental questions be adequately addressed and practical recommendations on its use be adequately grounded in sound conceptual, measurement, and statistical logic. It was with this spirit that I wrote and I hope that with the same spirit the reader would approach my chapter about Cortina et al.’s chapter. Several points in Cortina et al.’s chapter, especially the proposed approach to prescreening data, need to be addressed. Cortina et al. began with a review of the basic concepts and terminology in growth modeling. In the review, they made several comments regarding the inadequacy of difference scores and autoregressive models as approaches to the study of intraindividual change over time. As these issues have been elaborated by many researchers including myself, I will only briefly react to these comments (for details, see Chan, 1998b; Rogosa, 1995). What is new in Cortina et al.’s chapter is their proposed Three-Step-Approach (TSA) as a prescreening procedure to be conducted prior to performing actual multivariate LGM analyses. The proposed procedure, as I understand it, raises several questions that lead me to be much less optimistic than the authors are in terms of the procedure’s heuristic value and even appropriateness as a prescreening tool prior to actual multivariate LGM analyses, and these questions are the focus of my chapter.

DIFFERENCE SCORES, AUTOREGRESSIVE MODELS, MEASUREMENT SCALES AND INTRAINDIVIDUAL CHANGE Before discussing Cortina et al.’s TSA, let us briefly consider the issues of difference scores and autoregressive models as methods for analyzing intraindividual change over time. Cortina et al. correctly noted that difference sores and autoregressive models are not well suited for the assessment of intraindividual change. The authors discussed these inadequacies in the context of the limitations of correlations associated with the failure to capture information on variances. Although the separate points made by Cortina et al. on difference scores, autoregressive models, and limitations of correlations are not factually incorrect, I think there is a need to clarify some of these issues and link them more coherently to the assessment of intraindividual change over time. The ﬁrst point to note is that there is nothing inherently wrong with using difference scores obtained between two time points to index change, if the

322

DAVID CHAN

scores across the measurement occasions in fact represent the same construct and do so with the same precision (i.e., measurement invariance over time exists). Rogosa and others (e.g., Rogosa, Brandt, & Zimowski, 1982; Rogosa & Willett, 1983, 1985; Zimmerman & Williams, 1982) have shown that many of the criticisms of difference scores were founded in misconception and they have demonstrated that the difference score can be an intuitive and unbiased simple measure of individual growth (change). As I as well as others have noted (e.g., Chan, 1998b; Rogosa, 1995), the real problem with using the traditional difference score to represent intraindividual change over time has to do with the limitations of the two-wave study design that produces the difference score rather than with any inherent deﬁciency in the difference score itself. Specifically, even though the difference score can be an unbiased measure of individual change, only very limited information on individual change over time can be obtained from a difference score analysis of data obtained from two-wave designs. Two-wave designs are problematic for assessing change over time because measurements are repeated only two points in time and, as such, they provide no precise information on intraindividual change over time. That is, no precise individual growth curves can be plotted because the most complex functional form that can be ﬁtted is a straight line passing through the two data points representing the two observations over time for a single individual. There is no way to evaluate the adequacy of the straight-line functional form for the growth (i.e., change over time). In addition, there is no way to compare the ﬁt between the straight line function with other growth curves such as quadratic or cubic curves that could pass perfectly through the same two data points (Rogosa, 1995). The two-wave design represents two snapshots of a continuous growth process (Bryk & Weisberg, 1977). Without the ability to assess the functional form of the trajectory that most adequately describes intraindividual change over time, the simple difference score analysis of data from two time points clearly fails to answer many fundamental questions on change over time (for details on these questions, see Chan, 1998b). In short, the problem with using the difference score analysis to represent intraindividual change is one of study design associated with the number of measurement occasions (only two time points) in repeated measurement, and not one of inherent statistical deﬁciency in the difference score as an estimate of change. Indeed, if the difference score is inherently deﬁcient, then any trajectory in a LGM analysis cannot be very meaningful either, since the meaningfulness of the functional form of the trajectory and its associated parameters presupposes that true change has occurred between

Multivariate Latent Growth Modeling

323

two adjacent time points when the difference in scores on the variable between the two points is statistically significant. Another point made by Cortina et al. that is worth clarifying concerns the use of autoregressive models to assess intraindividual change. Autoregressive models, regardless whether they incorporate latent variables (i.e., autoregressive latent variable models which were not explicitly discussed in Cortina et al.) or not (i.e., autoregressive multiple regression models which were discussed in Cortina et al.), are inadequate for the analysis of longitudinal data representing intraindividual change over time. Autoregressive models estimate scores on a variable based on values of the same variable. Researchers who argued for the inclusion of autoregressive models in the longitudinal modeling of intraindividual change claimed that the autoregressive effect (the ‘‘effect’’ of the Time 1 measure on the Time 2 measure of the same variable) is a legitimate competing explanation for an observed effect and therefore must be included before causal inferences can be made regarding the inﬂuence of other predictors of change over time. But the inclusion of autoregressive effects in longitudinal modeling of intraindividual change is problematic because they tend to remove all potentially important predictors of change except those that predict changes in rank order of the observations over time. For example, in a monotonically stable growth process in which all individuals increase at a constant rate (i.e., linearly) while maintaining the same rank order, the important predictors of the individual slopes would be eliminated with the inclusion of autoregressive effects (Chan, 2002). More generally, the autoregressive model fails when intraindividual change is accompanied by high-rank-order stability over time. In addition, it is conceptually questionable to construe the autoregressive effect as a parameter representing a true causal effect and researchers have argued that the autoregressive effect is in fact a stability coefﬁcient representing the boundary or initial values of the system (for more details on problems associated with including autoregressive effects in longitudinal modeling of intraindividual change, see Rogosa & Willett, 1985; Stoolmiller & Bank, 1995). In discussing autoregressive models and elsewhere in the chapter, the authors devoted a considerable amount of attention to show that analysis of correlations of repeated measurements may lead to results that are different from those obtained from analysis of variances and covariances. But this should not be surprising since the correlation matrix is obtained by standardizing the variance–covariance matrix. In standardizing the variance–covariance matrix, the variances of all variables are transformed to be equal (to 1.00) to obtain the correlation matrix. To the extent that the differences

324

DAVID CHAN

in observed variances across two or more variables (or across repeated measurements of the same variable) are due to true score differences, a transformation that imposes equality of variances will lead to a distortion of the summary of the empirical data (which is used as input to the LGM analysis) and hence produce misleading results in the LGM analysis. In fact, using the correlation matrix as data input, as opposed to using the variance– covariance matrix, is problematic for all data analysis techniques (and not just LGM) when differences in observed variances represent true score differences. In the context of analysis of interindividual differences in intraindividual changes over time (e.g., LGM analyses), the problem of using correlations is simply instantiated in the form of misleading results due to incorrect elimination of true differences in variances across time by ﬁxing the variances to equality (i.e., equal to 1.00). There are also some other points made in Cortina et al.’s chapter that are worth clarifying. The authors claimed that ‘‘longitudinal studies in the social sciences are often based on repeated measures using different instruments/ scales to measure the same construct over time.’’ I wonder if the authors have done any systematic review to support this claim. From my personal experiences of reviewing (although not a systematic formal review) longitudinal studies in the published literature as well as those submitted to journals for publication, my guess is that the vast majority of longitudinal studies used the identical measure in repeated measurement occasions when assessing the same construct over time. This is especially so when the researcher begins a longitudinal study knowing which construct to track over time. Of course, different measures or items over time may be used to represent the same construct if the researcher in fact decides on what construct to track in a post hoc manner after examining the measures available in a longitudinal dataset or when practical considerations led to removal of some items or measures at some time points in the repeated measurement (and the researcher therefore looks for an alternative measure that happens to be available which may measure the same construct). Cortina et al. used the example of different money currencies across time to illustrate the case of same construct (money) being assessed using different measurement scales over time. In this example, there is no conceptual dispute that the different money currencies (metrics) are different measures of the same construct of money. By the definitions of money and money currencies, there is no need to provide a statistical justiﬁcation for the construct validity of the metric used at any of the time points of the repeated measurements. However, money is not representative of the focal constructs in many social science studies which are psychological constructs that do not have the

Multivariate Latent Growth Modeling

325

deﬁnitional advantage (that money has) that removes the need for construct validity evidence. When different items are used across time to measure the same construct, we have a serious problem of confounding across-time differences due to different item content and across-time differences due to changes in measurement properties of the same item content. Only the second type of across-time differences, but not the ﬁrst type, can be statistically assessed using established methods of analyzing measurement invariance of responses across time (Chan, 1998b; Vandenberg & Lance, 2000). Even if we have independently obtained validity evidence that the different measures/items across time (e.g., Test A at Time 1 and Test B at Time 2) measure the same construct, the item parameters are not directly comparable across time since item content differs across time. In short, when analyzing how a construct undergoes intraindividual changes over time, it is critical that the identical measure(s) be used across all the time points in the repeated measurement to assess the same construct. Only in doing so can we assess measurement invariance of responses across time to establish if the same construct is being assessed and assessed with the same precision across time (which is a prerequisite for LGM analysis, see Chan, 1998b), which in turn establish if the intraindividual change trajectory composed by the scores on the same measure at different time points represent changes in values on the same construct.

ISSUES CONCERNING CORTINA ET AL.’S THREE-STEP APPROACH Whereas the above issues on intraindividual change have been elaborated by several researchers before Cortina et al., the proposed TSA as a prescreening procedure to be conducted prior to performing actual multivariate LGM analyses is a somewhat novel aspect of the assessment of intraindividual change. It is somewhat novel because while the specific proposed prescreening procedure is novel, the more general idea of conducting systematic preliminary analyses prior to LGM analyses is not new. In Chan (1998b), I proposed an integrative two-phase uniﬁed framework in which Phase 1 consisted of systematic preliminary analyses (to be conducted prior to actual LGM analyses) to: (a) test the measurement invariance assumptions of LGM analyses (which are not addressed by Cortina et al.’s TSA) and (b) perform exploratory and guidance functions by identifying likely types of or constraints on the possible functional forms of intraindividual change

326

DAVID CHAN

trajectory and the cross-domain (i.e., between-constructs) relationships between trajectories in the sample under study, which in turn provide a guide to subsequent LGM analyses (which are similar to the intended purpose of Cortina et al.’s TSA). While the TSA requires less effort in data analysis than Phase 1 of Chan’s (1998b) uniﬁed framework, I argue below that TSA fails to address many of the critical preliminary questions relevant at the stage prior to performing actual LGM analyses and could lead to misleading inferences. Of course, with regard to the preliminary analyses prior to actual LGM analyses (univariate or multivariate), the issue is not about prescreening data to detect outliers, transcription errors, and so forth as these are good data cleaning practices that we expect researchers to adopt prior to conducting the actual focal data analyses. Instead, according to Cortina et al. the prescreening is to identify the major trends in growth and the proposed TSA procedure is purportedly ‘‘a very helpful heuristic to immediately understand the underlying latent growth process in a given data set.’’ I agree with Cortina et al. that it is helpful to examine the empirical covariance matrix of the repeated measurements prior to specifying and testing multivariate LGMs. I also agree that it is possible and desirable (and I would add ‘‘necessary’’ in most situations) to conduct preliminary analyses prior to actual LGM analyses with the aim to help identify major trends in intraindividual change and relationships between different change trajectories. However, I disagree that the TSA procedure proposed by Cortina et al. provides an adequate and useful way to achieve this aim. In addition, I argue that there are other important aims of preliminary analyses that cannot be achieved by the TSA. The problems with the TSA procedure may be classiﬁed into three types namely, failure to incorporate information on means, failure to test the assumption of measurement invariance of responses over time and across groups, and failure to incorporate measurement error at each time point and across-time error covariance structures. The following sections discuss each of these problems in the context of identifying true patterns of intraindividual change. Finally, the problems associated with applying the TSA procedure to multi-level modeling of longitudinal data are discussed.

Failure to Incorporate Information on Means In LGM analyses, the growth models ﬁtted are models with structured means (i.e., the models include estimates of the intercept factor mean and the slope/shape factor mean) and the data input is the augmented

Multivariate Latent Growth Modeling

327

moment matrix, which consisted of both the variance–covariance matrix and the vector of means of the observed variables. Clearly, the means of the observed variables in the repeated measurement and their measurement errors in estimating the true mean scores are an integral part of the data that will directly affect the LGM results depicting the functional form of the trajectory of intraindividual change and therefore the between-construct relationships of these changes as well. Despite its importance in determining the ﬁt of the ﬁnal LGM model, the information provided by the vector of means is ignored in the preliminary analysis prior to actual LGM analysis if we apply Cortina et al.’s TSA procedure. I ﬁnd this omission extremely perplexing. As explained in Phase 1 of Chan’s (1998b) uniﬁed procedure, the mean of the latent factor (i.e., focal construct) at each time point (estimated from the information from the vector of observed means) provides critical information in the preliminary analyses (together with the information from the variance–covariance matrix of repeated observations) to help identify the likely types of or constraints on the possible functional forms of intraindividual change trajectory and the cross-domain (i.e., between-constructs) relationships between trajectories in the sample under study, which in turn provide a guide to subsequent LGM analyses. Failure to Test the Assumption of Measurement Invariance over Time and across Groups Toward the end of the section titled The Concept of Change in Longitudinal Research in Cortina et al., the authors noted that the basic LGM analysis may be reﬁned by integrating measurement error structures into the growth model and correctly noted that the latent growth factors (i.e., intercept and slope/shape factors) in the basic LGM analysis and the latent factors in factor analysis have different meanings. Here, I presume that Cortina et al. were referring to the fact that the latent growth factors were chronometric factors formed by ﬁxing or estimating basis coefﬁcients for the observed variables in repeated measurement, whereas the latent factors in factor analysis are constructs formed by ﬁxing or estimating factor loadings for each set of multiple observed variables jointly representing a single latent factor and explicitly modeling the measurement errors of the observed variables. Cortina et al. then cited Sayer and Cumsille (2001) who demonstrated how standard structural equation modeling that incorporates measurement errors can be combined with standard LGM analyses. In fact, prior to Sayer and Cumsille (2001), Chan (1998b) had proposed an integrative two-phase data analytic framework that fully incorporates measurement error structures into both the preliminary analyses and the actual LGM analyses. The

328

DAVID CHAN

omission of Chan (1998b) in Cortina et al.’s chapter is noteworthy because the TSA procedure, which is preliminary to actual LGM analyses, fails to incorporate measurement error, whereas Chan (1998b) addresses the importance of incorporating measurement error even in the preliminary analyses prior to actual LGM analyses. Why is it critical to incorporate measurement error, even in the preliminary analysis prior to actual LGM analysis? The reasons have to do with establishing measurement invariance of responses over time and across groups as a prerequisite for LGM analysis, as well as separating true change/stability from observed changes/stability due to the inﬂuence of measurement errors. Cortina et al. appear to be cognizant of at least some of the reasons, since they correctly noted in their chapter that measurement invariance over time is a core prerequisite of LGM analysis and the possibility of distortion of results is due to measurement errors. Yet, the TSA approach does not explicitly incorporate measurement errors and the analysis assumes, without testing, measurement invariance of responses over time (and across groups in the case of multiple-group analysis). This is because TSA analyzes the focal variable at each time point as an observed variable and instead of a latent (common) factor with multiple indicators (i.e., multiple items, item parcels, or scales). If the assumption of measurement invariance (over time and across groups) is in fact violated in the dataset, the results of the TSA procedure will be misleading. This is because the interpretation of results of the TSA assumes that any observed changes are due to changes involving differences in values on the same construct being measured and measured with the same precision across time points/groups (i.e., alpha changes), but not changes involving calibration differences across time points/groups (i.e., beta changes) or changes involving conceptual differences across time points/groups (i.e., gamma changes) (see Chan, 1998b). To put it simply, the TSA procedure assumes, without testing, that the only relevant changes that occur across time are true quantitative changes on the same construct that has been measured with equal precision across time points. The procedure, by itself, is not able to detect true qualitative changes involving calibration of measurement or conceptualization of the construct domain. The longitudinal mean and covariance structures analyses (LMACS) in Phase 1 of the uniﬁed procedure proposed by Chan (1998b) explicitly addressed the above issues by modeling the focal variable as a latent construct represented by multiple indicators, repeatedly measured at each time point, and conducting a systematic hierarchy of tests to establish across-time or

Multivariate Latent Growth Modeling

329

between-groups measurement invariance in terms of equality of the respective item (indicator) parameters. One may counterargue that we could simply conduct LMACS or similar measurement invariance analyses prior to applying the TSA procedure. But this will not make sense since we will now have to perform a distinct type of preliminary analysis before using the TSA procedure, which is meant to be a preliminary analysis prior to actual LGM analysis. More fundamentally, the LMACS may, in addition to testing for measurement invariance, be used to identify constraints on major trends of intraindividual change (for details, see Chan, 1998b) and it does so more adequately than TSA allows because it allows the clear isolation of true score variance–covariance and error score variance–variance. This point becomes more evident in the following section when we consider in more detail the TSA’s failure to incorporate measurement error and error covariance structures.

Failure to Incorporate Measurement Error at Each Time Point and AcrossTime Error Covariance Structures The TSA’s failure to incorporate measurement errors at each time point is problematic because it assumes that observed score variance accurately represents true score variance. To the extent that measurement errors are substantial, the violation of this assumption leads to inaccurate preliminary estimates of true intraindividual change (e.g., slope factor) and true betweenconstruct differences in these changes (e.g., slope–slope factor association). In addition, across-time error covariances affect the accuracy of observed covariances of repeated observations as estimates of true score covariances and hence the accuracy of the function form of the trajectory as well as the estimate of the true intercept-slope/shape association. These problems are directly addressed in the Chan’s (1998b) LMACS procedure, which allows modeling of across-time covariances, so as to decompose observed covariances of repeated measurements into true covariance and error covariance. Specifically, in the uniﬁed analytic framework proposed by Chan (1998b), the LMACS procedure, which is a preliminary phase prior to conducting actual LGM analyses, uses multiple indicators to represent the focal construct at each time point (note: similarly, the Phase 2 procedure of the uniﬁed framework uses multiple-indicator LGM analysis). As noted in Chan (1998b), the use of multiple indicators in LMACS as well as in LGM analysis is important because it allows both random and nonrandom measurement errors to be taken into account when modeling changes over time.

330

DAVID CHAN

Specifically, the use of multiple indicators to assess the focal construct allows reliable (nonrandom) variance to be partitioned into true score common (construct) variance and true score unique variance. True score unique variance is nonrandom and it refers to that portion of observed variance in a measure (or item) that is not shared with other measures (or items) of the same construct. In the longitudinal design for assessing intraindividual change, the same measures are repeatedly administered over time. A failure to partition nonrandom variance into true construct variance and unique variance leads to distorted (inﬂated) estimates of true change in the focal construct over time.

SOME ADDITIONAL ISSUES There are several apparently more minor points on Cortina et al.’s arguments for the validity or usefulness of the TSA procedure that require clariﬁcation. These points although apparently minor, are reﬂections of the more fundamental problems associated with failures to explicitly incorporate issues of measurement invariance and measurement errors in preliminary analyses as well as actual LGM analyses. For example, at the beginning of the section on practical recommendations using the TSA, Cortina et al. stated that ‘‘In practical empirical research, a discrepancy between intercept variance and empirical variance at the ﬁrst time point indicates a misﬁt between theoretical model and observed data.’’ This statement is true only if there is no measurement error such that the observed score variance at initial status (i.e., Time 1) is in fact equal to the true score (construct) variance. As another example, Cortina et al. noted, when describing Step 1 of the TSA, that the procedure yields accurate estimates if we can assume that the measurement error variance is the same across time points or, alternatively, compute the measurement error variance separately at each time point using the reliability information available for that time point. Now, the assumption of equal measurement error variance across time points needs to be empirically tested since its violation would affect the accuracy of the estimates produced by the TSA, but this test is not incorporated in the TSA procedure. On the other hand, if we have internal consistency reliability information available from the data in the study, it is almost the case that we also have data at the item level (where the mean or total score of the multiple items is used as the scale score to represent the observed variable at each time point). But that implies we have data to represent the focal variable at each time point as a latent construct (i.e., common factor)

Multivariate Latent Growth Modeling

331

assessed by multiple indicators (items), instead of as a single manifest variable. With this multiple-indicator representation at each time point, which is used in the LMACS procedure described in Chan (1998b), it is possible to explicitly model and empirically estimate the measurement error variances, as well as any specific within-time or across-time item error covariance structures, provided the model is identiﬁed. Moreover, measurement invariance of responses across time or groups in terms of equality of factor loadings or error variances/covariances can be directly tested, unlike the TSA procedure, which assumes, without testing, equality of error variances (and in fact, also equality of factor loadings). It is noteworthy that the problems with the TSA procedure (as I have explicated so far in this chapter) that are relevant to preliminary analysis even for the univariate LGM analysis are applicable and compounded when multiple focal variables are involved (i.e., in multivariate LGM analysis), since the validity of the parameter estimates of between-construct associations in a multivariate latent growth model is strongly dependent on the adequate speciﬁcation of the intraindividual change in the component univariate latent growth models that were combined to form the multivariate model (Stoolmiller, 1994).

Problems of Applying TSA to Multi-Level Models of Longitudinal Data Finally, Cortina et al. also described how the TSA may be applied in the context of more complex multi-level data structures where there are more than two levels in the basic longitudinal data (i.e., when Level 3 (e.g., groups) or higher levels (e.g., organizations) exists in addition to Level 2 represented by individuals and Level 1 represented by repeated observations over time). The basic idea is to analyze the covariance matrices separately for the subsamples (e.g., groups) deﬁned by the higher-order (e.g., grouplevel) variable. If I understand it correctly, the application of the TSA procedure in this context involves comparing the corresponding estimates across the subsamples simply by ‘‘eyeballing’’ the effects of higher-order variables without performing any inferential statistical analysis. The problems with the TSA procedure for the single-group analysis are applicable and compounded when multiple groups are involved. For example, without testing for measurement invariance of responses across groups (and not just across time points), we do not know if the between-group differences in estimates observed by the eyeball are in fact meaningful since we cannot be certain that the between-group differences are due to only alpha changes and

332

DAVID CHAN

not beta or gamma changes across groups. Also, without explicitly modeling measurement errors in each group, the ‘‘eyeball analysis’’ in the TSA approach confounds true differences across groups and artifactual differences due to differential reliability across groups. The LMACS analysis in Phase 1 of the uniﬁed procedure in Chan (1998b) addresses these problems directly because multiple-group LMACS analyses can be conducted to assess measurement invariance across groups and model any between-group measurement error structures derived a priori or post hoc after ﬁtting a series of models to the data in an exploratory fashion. Moreover, the results from the multiple-group LMACS procedure provide the preliminary empirical basis to guide the researcher in making choices of which specific multiple-group latent growth models, among an inﬁnite number of possible models, to ﬁt to the data in the subsequent actual LGM analyses (Chan, 1998b). A more complicated problem exists if the measurement errors in one or more of the groups examined in fact consisted of systematic errors due to construct contamination in addition to random measurement errors. In such situations, the observed between-group differences in estimates can be either underestimates or overestimates of the true between-group differences, depending on the nature of the relationship between the extraneous construct and the focal construct in each group as well as the amount of systematic error that exists in each group. Although it has not been previously noted in the published literature, it is possible to directly address these issues by extending the multiple-group LMACS analysis to incorporate systematic errors in the latent measurement model, using the LISREL All-Y-model speciﬁcation introduced in Joreskog and Sorbom (1989) and applied by others (Chan, 2001; Schmitt, Pulakos, Nason, & Whitney, 1996; Williams & Anderson, 1994), to simultaneously model the focal constructs and the extraneous constructs assumed to be assessed by the relevant repeated measures in each group.

CONCLUSION In conclusion, I am much less optimistic than Cortina et al. in the evaluation of the TSA as a proposed preliminary analysis procedure prior to conducting actual multivariate LGM analysis. Given the issues that I have explicated in this chapter, I cannot agree with Cortina et al. that ‘‘the TSA is certainly a very helpful heuristic to immediately understand the underlying latent growth process in a given dataset.’’ Cortina et al. may be correct that the TSA is ‘‘a tool that is easy to apply’’, but I must admit that I ﬁnd it

Multivariate Latent Growth Modeling

333

difﬁcult to even agree that the TSA is ‘‘certainly a useful supplement’’ to LGM analysis, insofar as there is a tendency for the preliminary procedure to produce misleading inferences to the extent that measurement errors or across-time item error covariances are substantial or the assumption of measurement invariance across time points or groups is violated. Finally, because of the problems discussed in this chapter, I am not optimistic that the TSA can provide us useful or even accurate preliminary information on construct relationships across multiple levels of analysis. From a multi-level perspective, this limitation is noteworthy because it means that the proposed preliminary analysis prior to multivariate LGM is inadequate as a ﬁrst guide to identify accurate constraints on the many types of potential complex multi-level effects such as cross-levels effects or interaction effects on interindividual differences in intraindividual changes over time that involve variables at different levels of analysis. Issues on preliminary analysis prior to growth modeling, especially multivariate LGM involving multiple levels of analysis, are certainly important and difﬁcult to address. I hope that some of the ideas discussed in this chapter will provide a springboard for Cortina et al. and other researchers to develop defensible and practically useful preliminary analysis procedures that will truly be a valuable heuristic for guiding subsequent modeling of longitudinal processes, especially for complex processes involving multiple focal variables and multiple levels of analysis.

REFERENCES Bryk, A., & Raudenbush, S. W. (1992). Hierarchical linear models. Thousand Oaks, CA: Sage. Bryk, A. S., & Weisberg, H. I. (1977). Use of the nonequivalent control group design when subjects are growing. Psychological Bulletin, 85, 950–962. Chan, D. (1998a). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234–246. Chan, D. (1998b). The conceptualization of change over time: An integrative approach incorporating longitudinal means and covariance structures analysis (LMACS) and multiple indicator latent growth modeling (MLGM). Organizational Research Methods, 1, 421–483. Chan, D. (2001). Method effects of positive affectivity, negative affectivity, and impression management in self-reports of work attitudes. Human Performance, 14, 77–96. Chan, D. (2002). Longitudinal modeling. In: S. Rogelberg (Ed.), Handbook of research methods in industrial and organizational psychology (pp. 412–430). Oxford: Blackwell. Hannan, M. T. (1971). Aggregation and disaggregation in sociology. Lexington, MA: Heath. Hofmann, D. A. (2002). Issues in multi-level research: Theory development, measurement, and analysis. In: S. Rogelberg (Ed.), Handbook of research methods in industrial and organizational psychology (pp. 247–274). Oxford: Blackwell.

334

DAVID CHAN

Hofmann, D. A., Jacobs, R., & Baratta, J. (1993). Dynamic criteria and the measurement of change. Journal of Applied Psychology, 78, 194–204. James, L. R., Demaree, R. G., & Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69, 85–98. Joreskog, K. G., & Sorbom, D. (1989). LISREL 7: Analysis of linear structural relationships by the method of maximum likelihood. Chicago: National Educational Resources. Kozlowski, S. W. J., & Klein, K. J. (2000). A multi-level approach to theory and research in organizations: Contextual, temporal, and emergent processes. In: K. J. Klein & S. W. J. Kozlowski (Eds), Multi-level theory, research, and methods in organizations (pp. 3–90). San Francisco: Jossey–Bass. Ostroff, C. (1993). Comparing correlations based on individual level and aggregated data. Journal of Applied Psychology, 78, 569–582. Robinson, W. S. (1950). Ecological correlations and the behavior of individuals. American Sociological Review, 15, 351–357. Rogosa, D. R. (1995). Myths and methods: ‘‘Myths about longitudinal research’’ plus supplemental questions. In: J. M. Gottman (Ed.), The analysis of change (pp. 3–66). Hillsdale, NJ: Lawrence Erlbaum Associates. Rogosa, D. R., Brandt, D., & Zimowski, M. (1982). A growth curve approach to the measurement of change. Psychological Bulletin, 92, 726–748. Rogosa, D. R., & Willett, J. B. (1983). Demonstrating the reliability of the difference score in the measurement of change. Journal of Educational Measurement, 20, 335–343. Rogosa, D. R., & Willett, J. B. (1985). Understanding correlates of change by modeling individual differences in growth. Psychometrika, 50, 203–228. Rousseau, D. M. (1985). Issues of level in organizational research: Multi-level and cross-level perspectives. In: B. M. Staw & L. L. Cummings (Eds), Research in organizational behavior (pp. 1–7). Greenwich, CT: JAI Press. Sayer, A. G., & Cumsille, P. E. (2001). Second-order latent growth models. In: L. M. Collins & A. G. Sayer (Eds), New methods for the analysis of change (pp. 179–200). Washington, DC: APA. Schmitt, N., Pulakos, E. D., Nason, E., & Whitney, D. J. (1996). Likability and similarity as potential sources of predictor-related criterion bias in validation research. Organizational Behavior and Human Decision Processes, 68, 272–286. Stoolmiller, M. (1994). Antisocial behavior, delinquent peer association and unsupervised wandering for boys: Growth and change from childhood to early adolescence. Multivariate Behavioral Research, 29, 263–288. Stoolmiller, M., & Bank, L. (1995). Autoregressive effects in structural equation models: We see some problems. In: J. M. Gottman (Ed.), The analysis of change. New Jersey: LEA. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–69. Williams, L. J., & Anderson, S. E. (1994). An alternative approach to method effects by using latent-variable models: Applications in organizational behavior research. Journal of Applied Psychology, 79, 323–331. Zimmerman, D. W., & Williams, R. H. (1982). Gain scores in research can be highly reliable. Journal of Educational Measurement, 19, 149–154.

A NOTE ON THE COMPUTER GENERATION OF MEAN AND COVARIANCE EXPECTATIONS IN LATENT GROWTH CURVE ANALYSIS Kevin J. Grimm and John J. McArdle ABSTRACT Every ‘‘structural model’’ is defined by the set of covariance and mean expectations. These expectations are the source of parameter estimates, fit statistics, and substantive interpretation. The recent chapter by Cortina, Pant, and Smith-Darden ((this volume). In: F. Dansereau & F. J. Yammarino (Eds), Research in multi-level issues (Vol. 4). Oxford, England: Elsevier) shows how a formal investigation of the data covariance matrix of longitudinal data can lead to an improved understanding of the estimates of covariance terms among linear growth models. The investigations presented by Cortina et al. (this volume) are reasonable and potentially informative for researchers using linear change growth models. However, it is quite common for behavioral researchers to consider more complex models, in which case a variety of more complex techniques for the calculation of expectations will be needed. In this chapter we demonstrate how available computer programs, such as Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 335–364 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04015-4

335

336

KEVIN J. GRIMM AND JOHN J. MCARDLE

Maple, can be used to automatically create algebraic expectations for the means and the covariances of every structural model. The examples presented here can be used for a latent growth model of any complexity, including linear and nonlinear processes, and any number of longitudinal measurements.

INTRODUCTION All structural equation models have expectations for the covariance and mean matrices of the observed variables. These expectations are used to calculate parameter estimates for structural models by minimizing the difference between the expected covariances and means of the model and the covariances and means of the data. These expectations are an important aspect of the calculation of ﬁt indices since the indices are based on the size of the difference between data and expected covariances and means. The recent chapter by Cortina, Pant, and Smith-Darden (this volume) promotes the investigation of the data covariance matrix when considering uni- and bivariate linear growth models. This investigation allows researchers to calculate estimates or approximations of the covariances of the growth factors, and was intended to assist researchers in understanding the practical programming of the growth model. In this chapter, we use available computer software to verify the results presented by Cortina et al. (this volume). We also use the same software to extend the principles of expectations to deal with more complex longitudinal models. The examples we give include growth curves that have more occasions and are nonlinear. The covariance and mean expectations of a structural model can be calculated in a number of ways and we will focus this paper on the automatic generation of expectations using the Maple computer software (Maplesoft, 2004). Maple is one of a number of recent computer programs that can perform symbolic matrix algebra. There are a large number of matrix options that can be used to create structural expectations (e.g., McDonald, 1978; Jo¨reskog & So¨rbom, 1978; McArdle, 1980). In this presentation, we use Reticular Action Model (RAM) notation based on the general method of path-tracing rules (RAM; McArdle, 1980; McArdle & McDonald, 1984; McArdle & Boker, 1990; Boker, McArdle, & Neale, 2002; McArdle, 2004). RAM is a technique of specifying a structural equation model with three matrices, which are sufﬁcient to calculate the expectations and all have a one-to-one mapping onto the structural diagram, and this makes structural expectations relatively easy to derive and understand.

A Note on the Computer Generation of Mean and Covariance Expectations

337

In the ﬁrst section, we demonstrate how Maple can be used to calculate the covariance and mean expectations of the models presented by Cortina et al. (this volume). We show how Cortina and colleagues are accurate and how their method is potentially informative for researchers using linear change growth models. In the next section, we extend the same methodology for structural expectations of more complex growth models since it is quite common for behavioral researchers to consider such models. The examples presented here can be used for a latent growth model of any complexity, including linear and nonlinear processes, and any number of longitudinal measurements. It is important to note this technique of generating expectations is useful for all structural models and is not limited to growth models. The ﬁnal section discusses a technique for generating expectations from the structural model itself.

LATENT GROWTH MODELS IN RAM NOTATION Let us assume we are trying to understand the expectations of a latent growth model written for observations on n ¼ 1 to N people at t ¼ 1 to T consecutive occasions, Y ½t: Following the notation of Meredith and Tisak (1990), McArdle (2001), and McArdle and Bell (2000), we can write the traditional linear growth model as Y ½tn ¼ y0n þ y1n B½t þ e½tn

(1)

where the uppercase variables are observed, the lowercase variables are unobserved (or latent), and the B½t represents a basis for the timing of the observations (described below). The three unobserved variables represent an initial level ðy0 Þ; a systematic slope with respect to the time basis ðy1 Þ; and a random error term ðe½tÞ: The ﬁrst two components are each assumed to have a mean ðm0 ; m1 Þ; a variance ðs20 ; s21 Þ; and a covariance ðs0;1 Þ or correlation ðr0;1 Þ; but the error term has a zero mean, a constant variance ðs2e Þ; and is uncorrelated with all other components. In one of the simplest cases, we assume the basis B½t ¼ ðt 1Þ; then the model describes a linear trend over time in group and individuals using latent components. If we assume we have just T ¼ 3 occasions, we can write this model in more speciﬁc detail as Y ½1n ¼ y0n þ y1n 0 þ e½1n Y ½2n ¼ y0n þ y1n 1 þ e½2n Y ½3n ¼ y0n þ y1n 2 þ e½3n

ð2Þ

338

KEVIN J. GRIMM AND JOHN J. MCARDLE

In this case, we can see the general pattern of the scores and parameters, and we can re-deﬁne this model in terms of a system of matrices. The matrices for RAM notation will now be described with an example of a three-occasion linear growth model without means. The Maple script for this example is Script 1 in the appendix and the path diagram is contained in Fig. 1. In RAM notations there are three matrices deﬁned as the filter (F), the asymmetric (A), and the symmetric (S). The F matrix is a ﬁxed binary matrix used to separate the manifest variables from the observed variables. This matrix is a rectangular matrix with as many rows as manifest variables and as many columns as total variables. The F matrix contains ones in the elements of the matrix corresponding to each manifest variable and zeros everywhere else. For example, the F matrix for a linear growth model with three repeated measurements (Y1, Y2, Y3) and two growth factors ðy0 ; y1 Þ is 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 The matrix is 3 5 since there are three manifest variables ðY 1; Y 2; Y 3Þ and ﬁve total variables ðY 1; Y 2; Y 3;y0 ; y1 Þ: The ones separate the manifest variables ðY 1; Y 2; Y 3Þ from the latent variables ðy0 ; y1 Þ or the squares from the circles in a structural diagram. σ0,1 σ02

y1

y0

0 1 2

Y1

Y2

Y3

σe 2

σe 2

σe 2

Fig. 1.

Three-Occasion Linear Growth Model.

σ12

A Note on the Computer Generation of Mean and Covariance Expectations

339

The second matrix is the A matrix and corresponds to all one-headed or asymmetric arrows. This matrix is square containing all of the variables in the model. An arrow starts at the variable column and ends at the variable row. The matrix contains either the ﬁxed values or the names of the estimated parameter. In this example, all of the one-headed arrows have ﬁxed values since we are modeling linear growth. These values are put into the A matrix. The A matrix for a linear growth model with three repeated measurement occasions is 0 0 0 1 0 0 0 0 1 1 0 0 0 1 2 0 0 0 0 0 0 0 0 0 0 since there are three one-headed arrows with unit weights beginning at the intercept (column y0 ) and ending at the three manifest variables (rows 1, 2, and 3), and three one-headed arrows with weights of 0, 1, and 2 starting from the slope (column y1 ) and ending at the three manifest variables (rows 1, 2, and 3). Finally, the S matrix contains all slings or two-headed arrows. This matrix is square, symmetric, and contains the values or names of estimated parameters as the A matrix. For our linear growth model, the S matrix is 2 se 0 0 0 0 0 s2 0 0 0 e 0 0 s2e 0 0 0 0 0 s2 s 0;1 0 0 0 0 s0;1 s21 since there is a constant residual variance ðs2e Þ; an intercept variance ðs20 Þ; a slope variance ðs21 Þ; and a covariance between the intercept and slope ðs0;1 Þ: Now that the three matrices are deﬁned and entered, we can calculate the expected covariance matrix. Without further elaboration (McArdle & McDonald, 1984), we can calculate the expected covariances matrix using the general matrix operations S ¼ F ðI AÞ1 SðI AÞ10 F 0

(3)

Using the Maple program for the linear growth model with three measurement occasions, the expected covariance for the three manifest

340

variables is 2 se þ s20 s2 þ s 0;1 0 2 s0 þ 2s0;1

KEVIN J. GRIMM AND JOHN J. MCARDLE

s20 þ s0;1 s2e þ s20 þ 2s0;1 þ s21 s20 þ 3s0;1 þ 2s21

s20 þ 3s0;1 þ 2s21 s2e þ s20 þ 4s0;1 þ 4s21 s20 þ 2s0;1

This matrix is the same as the ﬁrst three rows and columns of the matrix contained in Table 1 of the chapter by Cortina and colleagues. From the expected covariance matrix we can calculate the expected change in variance for consecutive measurement, E½y2; y2 E½y1; y1; E½y3; y3 E½y2; y2: Calculating these differences E½y2; y2 E½y1; y1 ¼ 2s0;1 þ s21 E½y3; y3 E½y2; y2 ¼ 2s0;1 þ 3s21

ð4Þ

we ﬁnd that the expected change in variance for a linear growth model is not constant, but if we extend to more time points there is a pattern to the changes in expected variance. The multiplier of the covariance term remains constant, but the multiplier of the variance of the slope increases by two with each consecutive difference. The Maple computer program was used to generate these expectations. The three RAM matrices were entered into Maple and the LinearAlgebra (linalg) package, which deﬁnes a set of commands to perform linear algebra, was used to derive the expectations. Once the basic series of commands is deﬁned in Maple, the program can easily be manipulated to derive the expectations with more occasions (i.e. four occasions), a different type of growth model or any structural equation model. One convenient way to deﬁne the RAM matrices is to start with the diagram of the structural model with all of the manifest and latent variables, the directional arrows indicating regressions or loadings, and the slings denoting all covariance terms (variance and covariances). It becomes relatively simple to create the RAM matrices once the structural model is drawn. One thing to remember is that one-headed arrows begin from the column and end in the row of the A matrix. Scripts 2 and 3 in the appendix are Maple program scripts for the four-occasion model and a more general model with k occasions. These scripts show the ﬂexibility of the Maple program for accommodating more occasions of measurement. Fig. 2 is a structural model of a bivariate growth model with four occasions of measurement and Script 4 in the appendix is the accompanying Maple script for generating the covariance expectations for linear growth models. These expectations are the same as those generated by Cortina and

A Note on the Computer Generation of Mean and Covariance Expectations σey2

σey2

Y1

Y2

y0

σx02

X1

Y4

y1 σy12 σx0,y1

σx1,y1

X2 σex2

σex,ey

σx12

σx0,x1

x0

Fig. 2.

Y3

σy0,x1

σx0,y0

σey2

σy0,y1

σy02

σex2

σey2

341

x1

X3 σex2

X4 σex2

Four-Occasion Bivariate Growth Model with Correlated Growth Factors.

colleagues if the speciﬁc covariance between measurements within an occasion were zero. When using Maple to generate expectations, it becomes easy to evaluate changes in expectations under different model conditions. For example, it is easy to change the covariance between intercepts to zero and follow the changes in the expectations.

EXPECTATIONS OF MEANS AND COVARIANCES The previous sections, including ﬁgures and Maple scripts, have focused on the expectations of covariances of linear growth models; however these sections ignore the means, which are an important aspect of latent growth modeling. The expectations for means can be generated simultaneously with the expectations for covariances in the same manner using RAM notation. The same three matrices need to be deﬁned and the only change is the addition of a constant vector, which is denoted as a triangle in the structural

342

KEVIN J. GRIMM AND JOHN J. MCARDLE

model. Means can be modeled in all structural modeling programs by analyzing the moment matrix (or raw data) and including a constant vector. The constant has a mean of one, standard deviation of zero, zero covariance with all variables in the model, and unit variance. The constant is treated as a manifest variable in RAM notation. Fig. 3 contains a path diagram of a linear growth curve with four occasions with means. The means of the growth factors are one-headed arrows from the constant to the growth factors and that is how they are input into the RAM matrices. There is one change in the interpretation when calculating the expected means and covariances of a structural model. The calculation of F ðI AÞ1 SðI AÞ10 F 0 is now the average cross-products (O), instead of the expected covariances (S). The expected covariances (S) is formed by subtracting the square (MM0 ) of the expected mean vector from O. The RAM matrices (F, A, S) for calculating the expected means and covariances for the four-occasion linear growth model follows (note: Y 1; Y 2; Y 3; Y 4; k; y0 ; y1 ) and the accompanying Maple program script is Script 5 in the appendix.

σ0,1 µ0

σ02

µ1 1

y0

y1

σ12

0 1

Y1 σe 2

Fig. 3.

Y2 σe 2

Y3 σe 2

2

3

Y4 σe 2

Four-Occasion Linear Growth Model with Means.

A Note on the Computer Generation of Mean and Covariance Expectations

0

0

0

0

1

0

0

0

0 0

1 0

0 1

0 0

0

0

0

1

0 0 0 0 0 0 0 0 0 0

0 0 0 0

0 0

0 0

1 1

0 0

0

0

1

0 0 0 0

0 0

0 0

1 0

0 0 0 0

0 0

m0 m1

0 0

0 s2e

0 0

0 0

0 0

0 0

0

s2e

0

0

0

0 0

0 0

s2e 0

0 1

0 0

0 0

0 0

0 0

0 0

s20 s0;1

1 0 F ¼ 0 0 0 0 0 0 A ¼ 0 0 0 0 2 se 0 0 S¼ 0 0 0 0

343

0 1 2 3 0 0 0 0 0 0 s0:1 s21 0 0

As previously stated, O is the average cross-products and the expectations of each manifest variable with the constant (last row or last column) is the vector of expected means (M). Multiplication of this vector by itself leads to a mean-square matrix (MM0 ), and subtracting this from the expected crossproducts (O) leaves the expected covariance matrix (S). Expectations of Latent Growth Models Based on Latent Difference Scores Many researchers, including Cortina et al. (this volume), have investigated the expected covariance matrix when ﬁtting linear growth models. However, these principles can be easily extended to include latent growth curves which do not have a linear basis. The Maple scripts can be easily changed to calculate the expected covariances and means for more complex growth model. For example, if expectations were desired for a growth model with a

344

KEVIN J. GRIMM AND JOHN J. MCARDLE

latent basis, there would only be two changes to make (from Script 5). Any changes in the basis ðB½tÞ of the latent growth model is reﬂected in the A matrix. In this case the values to the two middle occasions from the slope would be changed from 1 and 2 to a2 and a3 : The calculations follow and the changes would be apparent in the expectations. Proceeding with the focus of this response we present the expected means and covariances for a series of more complex growth models based on latent difference scores (LDS; McArdle, 2001; McArdle & Hamagami, 2001; Hamagami & McArdle, 2001). The expectations for these models are slightly more cumbersome to generate as there are more latent variables, but the calculations are the same. The ﬁrst LDS model is the constant change model, which posits an initial level for each participant from which a series of constant changes occur. In the constant change model, the amount of change occurring between consecutive occasions is the same, which result in a linear growth pattern. The constant change LDS model is structurally equivalent to the linear growth model, which means the structural expectations (covariances and means) are the same. A structural model of the constant change LDS model is contained in Fig. 4 and the Maple script for calculating the expected covariances and means is Script 6 in the

σ0, 1 µ0

σ02

Fig. 4.

1

y0

σe 2

µ1 y1

∆y1

∆y2

∆y3

y1

y2

y3

y4

Y1

Y2

Y3

Y4

σe 2

σe 2

σ 12

σe 2

Four-Occasion Constant Change LDS Growth Model with Means.

A Note on the Computer Generation of Mean and Covariance Expectations

345

appendix. The expectations in Script 6 in the appendix are the same as in Script 5 in the appendix showing the equivalence in the structural models. The complexity of structural expectations (Script 7 in the appendix) increase signiﬁcantly as two more estimated parameters (a and b) are added to this model as in Fig. 5. The constraints, a ¼ 1 and b ¼ 0 were relaxed making the expectations much more complex. This model hypothesizes an initial level from which a series of changes occur. In this model the changes have two sources, which are a constant amount with weight a and an amount proportional to the score at the previous occasion with the weight b: This model is the dual change model as there are two sources of the change as shown in the following equation: DY ½tn ¼ ay y1n þ by y½t 1n

(5)

Finally, moving to the bivariate case of the dual change latent difference score model (Fig. 6; Script 8 in the appendix), the expectations get unreasonably long. However, Maple has no problem in generating the expectations once the three RAM matrices are deﬁned.

σ0, 1 µ0

σ02

µ1 1

y0

y1 α

∆y1

σe 2

Fig. 5.

∆y2 β

β

α

α ∆y3

β

y1

y2

y3

y4

Y1

Y2

Y3

Y4

σe 2

σ12

σe 2

σe 2

Four-Occasion Dual Change LDS Growth Model with Means.

346

KEVIN J. GRIMM AND JOHN J. MCARDLE σy02

υx1,y0

σey2

y0 σy1,y0

µy1 σx0,y0

1

µx0

µx1

Y3

Y4

y1

y2

y3

y4

βy

σx1,y1

Fig. 6.

∆y3

γxy

γxy

γyx

γyx

γyx

βx

σx1,x0

βy ∆y2]

γxy

x1

σx02

βy ∆y1

y1

x0

σey2

Y2

σx12

σx0,y1

σey2

Y1

σy12

µy0

σey2

∆x1

βx

∆x2

βx

σex,ey

∆x3

x1

x2

x3

x4

X1

X2

X3

X4

σex2

σex2

σex2

σex2

Four-Occasion Bivariate Dual Change LDS Growth Model with Means.

EXPECTATIONS FROM THE RAM PATH DIAGRAM The ﬁnal part of this response is to discuss a method for obtaining covariance and mean expectations based on the structural diagram and the path-tracing rules (McArdle, 2004). There are only a few rules to path tracing which are necessary to calculate the expected covariances and means of a structural equation model. We will ﬁrst start with the direction of movement through a structural equation model. The ﬁrst movement begins in the opposite direction of a one-headed arrow (or through a sling in shorthand notation). Movement can continue in this opposite direction of oneheaded arrows as many times as necessary until reaching a sling (variance or covariance). A sling is considered a turning point and only one turn can be made. Movement then continues in the direction of one-headed arrows as many times as necessary to reach the destination. The values of the paths along one journey are multiplied and the values of all possible journeys are summed to calculate expectations. For variances, the expectation is the sum of all paths leading from the variable and ending at the same variable.

A Note on the Computer Generation of Mean and Covariance Expectations

347

For covariances, the expectation is the sum of all paths leading from the ﬁrst variable to the second (not through the constant), and ﬁnally for means, the expectation is the sum of all paths leading from the variable to the constant. For example, in our linear growth model (Fig. 3) the expected variance of Y 1 is the sum of ﬁve paths. The ﬁrst path is through the residual variance of Y 1 ðs2e Þ: The remaining paths go through the growth factors. The ﬁrst path through a growth factor is strictly through the intercept ð1 s20 1Þ; the second is strictly through the slope ð0 s21 0Þ; and two paths through the covariance between intercept and slope (1 s0;1 0; 0 s0;1 1). The sum of these paths and the expected variance of Y 1 is s20 þ s2e (If average crossproducts are desired there are four paths through the constant: 1 m0 1 m0 1; 1 m0 1 m1 0; 0 m1 1 m0 1; 0 m1 1 m1 0:). The expected covariance of Y 1 and Y 2 is the sum of four paths, all of which travel through the growth factors. The ﬁrst goes through the intercept ð1 s20 1Þ; second through the slope ð0 s21 1Þ; and the ﬁnal two travel through the covariance of intercept and slope (1 s0;1 1; 0 s0;1 1). The sum of these paths is s20 þ s0;1 : Finally, the expected mean of Y 1 is the sum of two paths. The ﬁrst path is through the intercept ð1 m0 Þ and the second is through the slope ð0 m1 Þ and summed we ﬁnd the expected mean of Y 1 is m0 : Through these path-tracing rules it is relatively easy to see how standard deviations could be estimated instead of variances in structural modeling. It is also simple to generate the expectations of latent variables. Fig. 7 is a path diagram of the four-occasion linear growth model with standard deviations estimated rather than variances. This model is structurally equivalent to the model in Fig. 4 since the expectations are the same. For example, the expected covariance of the intercept and slope is s0 r0;1 s1 ; which is the equivalent to s0;1 : In this model, it is easier to understand the residual variance and its relationship to the expected variance of Y1. The difﬁculty with the path-tracing rules is ﬁnding all of the different paths. For example, there are at least 19 different paths to determine the expected variance of Y3 for the LDS dual change growth model in Fig. 5.

CONCLUDING REMARKS The chapter by Cortina et al. (this volume) effectively uses structural expectations to assist the novice in understanding the basis of linear growth models. In this article, we have gone further along these lines by showing how an automated investigation of the expected mean and covariance matrix can be helpful for researchers looking for a relatively quick way to understand the

348

KEVIN J. GRIMM AND JOHN J. MCARDLE

ρ0, 1 y0*

y1*

σ0

σ1 µ0

µ1 1

y0

y1

0 1

Y1 σe eY1

Fig. 7.

Y2 σe eY2

Y3 σe eY3

2

3

Y4 σe eY4

Four-Occasion Linear Growth Model with Standard Deviations Estimated Instead of Variances.

role of estimates for estimated parameters in any longitudinal structural model. It is important for researchers to know how the expectations are calculated and to know the expectations for any structural equation model the researchers are considering. The techniques we use to automatically generate expectations based on RAM notation using the Maple program make it possible to study any structural equation model in this way. It is important for researchers to know what to expect (in terms of estimates and ﬁt statistics) from ﬁtting structural models. When using longitudinal data, another important inspection of any data can be made through plot of the model and the data. A longitudinal plot informs the investigator about outliers (miscodes), missing data patterns, the shape of the growth, and even changes in variance over time. The univariate plot has one limitation compared to the method proposed – plotting the data in a univariate framework when multivariate growth is investigated does not provide information in the covariances among growth factors of the different variables. However, a variety of other techniques can be used for more complex multivariate trajectories (see McArdle, 2001; McArdle & Hamagami, 2001).

A Note on the Computer Generation of Mean and Covariance Expectations

349

ACKNOWLEDGMENTS Kevin J. Grimm is supported by a National Institute on Aging Training Grant received by John R. Nesselroade: NIA T32 AG20500-01. John J. McArdle is supported by grants from the National Institute on Aging: AG-007137.

REFERENCES Boker, S. M., McArdle, J. J., & Neale, M. (2002). An algorithm for the hierarchical organization of path diagrams and calculation of components of expected covariance. Structural Equation Modeling, 9, 174–194. Cortina, K., Pant, H., & Smith-Darden, J. (this volume). Multivariate latent growth models: Reading the covariance matrix for multi-level interpretations. In: F. Dansereau & F. J. Yammarino (Eds), Research in multi-level issues (Vol. 4). Oxford, England: Elsevier. Hamagami, F., & McArdle, J. J. (2001). Advanced studies of individual differences linear dynamic models for longitudinal data analysis. In: G. A. Marcoulides & R. E. Schumacker (Eds), New developments in structural equation modeling (pp. 203–246). Mahwah, NJ: Erlbaum. Jo¨reskog, K. G., & So¨rbom, D. (1978). LISREL-IV: Analysis of linear structural relationships by the method of maximum likelihood. Chicago: National Educational Resources. McArdle, J. J. (1980). Causal modeling applied to psychonomic systems simulation. Behavior Research Methods and Instrumentation, 12, 193–209. McArdle, J. J. (2001). A latent difference score approach to longitudinal dynamic structural analyses. In: R. Cudeck, S. du Toit & D. Sorbom (Eds), Structural equation modeling: Present and future (pp. 342–380). Lincolnwood, IL: Scientiﬁc Software International. McArdle, J. J. (2004). The RAM notation for structural equation modeling. In: A. Maydeau & J. J. McArdle (Eds), Contemporary Psychometrics. Mahwah, NJ: Erlbaum. McArdle, J. J., & Bell, R. Q. (2000). Recent trends in modeling longitudinal data by latent growth curve methods. In: T. D. Little, K. U. Schnabel & J. Baumert (Eds), Modeling longitudinal and multiple-group data: Practical issues, applied approaches, and scientific examples (pp. 69–108). Mahwah, NJ: Erlbaum. McArdle, J. J., & Boker, S. M. (1990). RAMpath: A computer program for automatic path diagrams. Hillsdale, NJ: Lawrence Erlbaum Publishers. McArdle, J. J., & Hamagami, F. (2001). Latent difference score structural models for linear dynamic analyses with incomplete longitudinal data. In: L. Collins & A. Sayer (Eds), New methods for the analysis of change (pp. 139–175). Washington, DC: American Psychological Association. McArdle, J. J., & McDonald, R. P. (1984). Some algebraic properties of the reticular action model for moment structures. British Journal of Mathematical & Statistical Psychology, 37, 234–251. McDonald, R. P. (1978). A simple comprehensive model for the analysis of covariance structures. British Journal of Mathematical Statistical Psychology, 31, 59–72. Meredith, W., & Tisak, J. (1990). Latent curve analysis. Psychometrika, 55, 107–122.

350

KEVIN J. GRIMM AND JOHN J. MCARDLE

APPENDIX. THE EIGHT SCRIPTS Script 1: Latent Growth Model – Linear Basis (Three Occasions)

A Note on the Computer Generation of Mean and Covariance Expectations

351

352

KEVIN J. GRIMM AND JOHN J. MCARDLE

Script 2: Latent Growth Model – Linear Basis (Four Occasions)

A Note on the Computer Generation of Mean and Covariance Expectations

Script 3: Latent Growth Model – Linear Basis (k Occasions)

353

354

KEVIN J. GRIMM AND JOHN J. MCARDLE

Script 4: Bivariate Latent Growth Model – Linear (Four Occasions)

A Note on the Computer Generation of Mean and Covariance Expectations

355

356

KEVIN J. GRIMM AND JOHN J. MCARDLE

A Note on the Computer Generation of Mean and Covariance Expectations

Script 5: Latent Growth Model – Linear Basis with Means (Four Occasions)

357

358

KEVIN J. GRIMM AND JOHN J. MCARDLE

A Note on the Computer Generation of Mean and Covariance Expectations

359

360

KEVIN J. GRIMM AND JOHN J. MCARDLE

Script 6: Latent Growth Model – Latent Difference Score Constant Change with Means (Four Occasions)

A Note on the Computer Generation of Mean and Covariance Expectations

361

362

KEVIN J. GRIMM AND JOHN J. MCARDLE

Script 7: Latent Growth Model – Latent Difference Score Dual Change Model (Four Occasions)

A Note on the Computer Generation of Mean and Covariance Expectations

363

Script 8: Latent Growth Model – Latent Difference Score Bivariate Dual Change Model with Means (Four Occasions)

364

KEVIN J. GRIMM AND JOHN J. MCARDLE

THE VALUE OF HEURISTICS: CLARIFYING THE PURPOSE OF THREE-STEP-APPROACH TO ANALYZE MULTIVARIATE LATENT GROWTH MODELS Kai S. Cortina, Hans Anand Pant and Joanne Smith-Darden ABSTRACT In response to the Three-Step-Approach (TSA) that Cortina, Pant, and Smith-Darden (this volume) have suggested, Chan (this volume) expressed his reservations regarding the usefulness of a procedure that explicitly ignores measurement considerations and does not include mean scores. In this reply, we argue that the purpose of TSA is heuristic in nature and does not involve statistical testing of assumptions. In this spirit, the software, illustrated by Grimm and McArdle (this volume), rounds out our more conceptual considerations.

Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 365–372 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved. ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04016-6

365

366

KAI S. CORTINA ET AL.

INTRODUCTION pﬃﬃﬃﬃﬃ What is 95? It is easy to use a calculator to ﬁnd out that the right answer is 9.757. In all likelihood, you learned in school that it is not a bad idea to get a ballpark estimate of the solution before starting to crunch numbers. Obviously, you can infer that the solution lies between 9 and 10 – even if your calculator is out of juice. It is in this heuristic spirit that we are advocating the use of the Three-Step-Approach (TSA) as a preliminary step toward a multivariate latent growth analysis. Like every good heuristic (Polya, 1954), TSA is a way of looking at problems and casting about for solutions. It does not provide the ﬁnal answers, but gives an idea of what to expect. In this brief reply, we will ﬁrst comment on Grimm and McArdle’s (this volume) extension of our original contribution and then address three major concerns that Chan (this volume) raised in his commentary. We will argue that Chan’s suggestions lend themselves to a sequence of statistical testing, which is not the intention of a heuristic procedure like TSA.

GRIMM AND MCARDLE’S DATA GENERATION SOFTWARE It goes without saying that critical comments trigger more responses than positive feedback and extensions of the original idea. For this reason our comments on Grimm and McArdle’s suggestions are brief. In essence, we ﬁnd their ideas stimulating, and the software they illustrate will certainly facilitate heuristic analysis of multivariate latent growth models (MLGMs). While our contribution (Cortina, Pant, & Smith-Darden, this volume) discusses the conceptual model of linear growth in two variables and the use of TSA for this special case, Grimm and McArdle introduce a convenient way to expand the TSA idea to more complex designs using the ‘‘Maple’’ software. This is certainly an important step, as our illustration was limited to the basic model. MLGM is not limited with respect to the number of variables and measurement points analyzed simultaneously and, perhaps more importantly, the complexity of the hypothesized growth curve. Even a heuristic prescreening procedure like TSA immediately becomes rather cumbersome without the help of a software tool when, for example, an orthogonal quadratic term is added. Owing to space constraints, Grimm and McArdle could not elaborate on the Maple syntax. We hope it is easy to learn because it seems to us that it

The Value of Heuristics

367

would make an ideal teaching tool in advanced statistic courses on latent growth modeling (LGM). It is a common attitude among students and researchers to simply apply multivariate statistics rather than try to understand them conceptually. It makes Maple all the more appealing if it can help to prevent this ‘‘cookbook’’ approach to complex statistical analysis.

CHAN’S MAJOR CONCERNS Most of the critical comments raised by Chan (this volume) are based on a comparison of TSA with the Longitudinal Means And Covariance Structures analysis (LMACS)–MLGM strategy he proposed (Chan, 1998). He is skeptical about the usefulness of TSA if ‘‘measurement errors and acrosstime error covariances are substantial or the assumption of measurement invariance across time-points or groups is violated.’’ We could not agree more – but this comment misses the major purpose of the TSA approach. Pitting the two approaches against each other is like arguing that the ballpark estimate above (between 9 and 10) is less accurate than 9.757. MLGM is a statistically complex endeavor which, in addition, becomes epistemologically challenging when the researcher co-analyzes the dynamics of more than two variables and/or considers growth models that involve higherorder polynomials. In most cases, determining the correlation matrix of all growth parameters is the central goal of the analysis. TSA simply provides a ‘‘sneak preview’’ of this matrix, derived directly from the covariance matrix of the repeated observations of the variables. After some general considerations on repeated measurements that are only loosely related to our original contribution (on difference score and autoregression), Chan (this volume) summarizes his reservation against the TSA heuristic in three central points: (a) ‘‘yfailure to incorporate information on means’’ (b) ‘‘yfailure to test the assumption of measurement invariances,’’ and (c) ‘‘yfailure to incorporate measurement error at each time point.’’ We will discuss the second and third points together as they are related issues. We will comment on the incorporation of mean scores after the measurement problem as part of our argument follows from the latter. Consideration of Measurement Error and Invariance of Measurement As proposed by Cortina et al. (this volume), TSA can be applied if repeated measurement information on more than two occasions is available for two

368

KAI S. CORTINA ET AL.

or more variables, be it merely as observed variables or as factor scores that account for measurement error. Chan’s main concerns revolve around the measurement error in LGM. To date, however, most published applications of (uni- or multivariate) LGM use observed variables over time and not latent variables derived in a conﬁrmatory factor analytic model. As discussed in our original contribution, it is unfortunate that the term ‘‘latent’’ has two different meanings in complex latent growth analysis. Chan is referring to the all-inclusive approach that could be dubbed as ‘‘latent–latent growth model’’, i.e., latent variables (factors) combined with latent (polynomial) growth parameters. It is certainly an invaluable advantage to have a set of indicators, instead of only one score, for each measurement point. But we see no categorical reason to question the validity of LGMs without measurement models if researchers are aware of the caveats, and many reviewers of top-tiered journals apparently feel the same way. As mentioned before, TSA is a versatile heuristic technique that can be used in both situations, and we are surprised that Chan felt compelled to comment extensively on the measurement issue given that we mention it at several points and even refer to one of his more recent publications, e.g., ‘‘One of the shortcomings of LGM is that it does not take measurement error into account (Chan, 2001).’’ What might be more controversial is our statement that ‘‘The idea of latent factors measured by several indicators, as it is standard in structural equation models, does not immediately lend itself to be combined with LGM.’’ It is clearly beyond the scope of this brief commentary to discuss this issue in depth, so a brief sketch of the argument must sufﬁce. In terms of classical test theory (CTT), the measurement model that Chan implicitly refers to is known as ‘‘congeneric’’ (Jo¨reskog, 1971). Even for very complex error structures, as often found in repeated measurement models (i.e., correlation of measurement errors of indicators over time or within measurement points), one basic tenet of CTT must hold universally: The expected value for the error term of each variable (p) at each time point (t) for each individual (i) must be zero, i.e., Eðepti Þ ¼ 0: It follows that, at each measurement point, the mean score for a given sample is the best estimate of the population mean score. Measurement error variance adds to the variance of the true scores around the mean score but does not – by definition – affect the point estimate of the mean score. An (unsaturated) latent growth model (e.g., a linear model with four measurement points) adds constraints to the estimate of the (then latent) mean scores because they must fall on the ﬁtted linear line. However, the discrepancy of the sequence of mean scores over time from the hypothesized

The Value of Heuristics

369

linear line is independent of the reliability. These considerations do not contradict Chan’s argument that having both aspects included in one model is the better model. But they highlight that, in general, a LGM without a separate measurement model provides accurate estimates of the latent growth parameters if the growth model itself is properly speciﬁed. Adding measurement models and hence controlling for measurement error mainly reduces the standard errors of the parameter estimates, but not the estimates themselves. From the perspective of inferential statistics, including measurement models increases the power, and this certainly makes their inclusion worthwhile. But this is not crucial for a heuristic strategy like TSA that does not involve statistical testing. In order to get a ballpark idea about the latent growth coefﬁcients and their correlations, it is more important that the quick-and-easy estimates calculated in TSA are approximately consistent and unbiased. We agree with Chan that a more complex error structure could change the covariance matrix of growth parameters, but for simple error structures (uncorrelated errors), ignoring measurement error only results in the diagonal elements of this covariance matrix (variances of growth parameter) being inﬂated; all off-diagonal elements remain the same. For the correlation matrix of growth parameters, this means that TSA renders a lower-bound estimate of the true parameters (with correct signs), which is certainly of heuristic value. As an aside, we would like to emphasize that algebraically, an LGM implicitly speciﬁes a measurement model in the sense of CTT even without a ‘‘latent–latent’’ model. This becomes obvious when we consider the special case of an LGM when the variance of all growth parameters except for the intercept is zero. In this case, all individual values are stable over time (except for a constant gain or loss that is identical for all subjects), and the covariance matrix of the repeated measurements becomes statistically identical to a latent factor model with measurement points serving as indicators. Without elaborating on the inferential aspects of these considerations, it sufﬁces here to say that adding variance components for further polynomial trend coefﬁcients alters this effect but does not eliminate it, as long as degrees of freedom remain in the latent part of the model. This is important to keep in mind, as advocates of ‘‘latent–latent growth models’’ like Chan (1998) and Sayer and Cumsille (2001) assume rather than prove that there is substantial power gained by analyzing a ‘‘latent–latent growth model’’ instead of a simple LGM. The power gain is likely to be strong for models where the number of measurement points is close to the number of growth parameters, but is less in situations where a large number

370

KAI S. CORTINA ET AL.

of measurement points (say, 10) are paired with just a few trend components (say, intercept and linear slope). In the latter case, the researcher might get a very strong and less complex model by using average scores calculated from the indicator set at each time point. At several points in his comments, Chan (this volume) asks for statistical tests. This request clearly misses the central purpose of the TSA: to prescreen the covariance matrix prior to inferential analyses. Chan’s (1998) LMACS–MLGM approach makes hypothesis testing an integral part of the measurement analysis. This is clearly a very reasonable approach to data analysis. What he is really arguing for here, however, is the full-ﬂedged data analysis, not a heuristic or preliminary analysis. His approach produces – using the example we began with – the exact 9.757, not the ballpark estimate ‘‘between 9 and 10.’’ There is much to say in favor of the LMACS–MLGM approach, but not so much on a fast and easy prescreening heuristic like TSA. Succinctly put, for Chan the term ‘‘preliminary analysis’’ involves testing measurement assumptions. For us this is the ﬁrst step of the real data analysis, not a prescreening procedure. Establishing measurement invariance across measurement points is certainly an important step of the data analysis, but nobody gets hurt if we leave this question aside to get a quick glimpse at how the trajectories of the variables covary. As a heuristic, TSA deliberately ignores significance concerns and simply asks how the covariance of the latent variables would look like if we assume that all measurement requirements were met. From his concluding remarks, one gets the impression that Chan considers this a bad idea. Needless to say, we do believe that TSA can come in handy as an analytical tool in many empirical situations where ‘‘modeling’’ is the operative word. Probably in multivariate analysis in general, but certainly in the context of advanced structural equation modeling (SEM) or hierarchical linear modeling (HLM), it is rarely possible to have theory-derived hypotheses for all parts of the model. Technically speaking, every omitted path in an SEM model or unspeciﬁed cross-level interaction in HLM would require theoretical justiﬁcation. Researchers are often unaware that the alpha-inﬂation caused by the uncontrolled number of interpreted statistical tests in a complex multivariate model makes inferential conclusions extremely questionable. Under these circumstances, significance tests turn into heuristic rules, telling us when to acknowledge that a certain coefﬁcient likely exists in the population. The chosen p value, however, has lost its inferential meaning. Even a prescribed sequence of nested models, which is the gist of Chan’s LMACS– MLGM approach, may help minimize this problem – but does not cure it. Chan seems to be aware of this quandary, stating carefully that his

The Value of Heuristics

371

LMACS–MLGM approach provides a ‘‘basis to guide the researcher in making choices’’ between models. The rigidity of the rules of inferential statistics almost inevitably must be relaxed in multivariate analysis. Otherwise, there would be little beneﬁt from multivariate approaches (for further discussion, see Schnabel, Little, & Baumert, 2000). The real litmus test for complex multivariate models lies in complete replication. With only one sample at hand, what begins as theory-guided ‘‘modeling’’ typically turns into partially data-driven model trimming. This, of course, is not the story told in most Method sections of journal articles, because most reviewers are reluctant to accept an honest account of the real analytic progression. It is this ambiguity inherent in complex statistical approaches that makes heuristic tools and strategies so important.

Consideration of Mean Scores Chan (this volume) feels very strongly about the inclusion of means in LGMs – and so do we, although Chan seems to believe otherwise. It is not easy to comment on an erroneous presumption. But the misunderstanding on Chan’s part is likely due to his general skepticism about any heuristic prescreening procedures. As mentioned above, the TSA approach involves no inferential tests, as its purpose is to obtain rough estimates of the critical parameters of the speciﬁed LGM. For the MLGM with two variables and linear trajectories – the focus of our contribution – the inspection of the mean scores is an almost trivial exercise: one simply looks at them. If the mean development is fairly linear, this suggests that a linear model could be a reasonable approximation. The best guess of the latent linear increase (or decrease) over time is the average of the differences between adjacent time points. As this seemed pretty common-sense, we wrote ‘‘Although a linear change model implies a linear trend for the mean scores as well y the development of mean values can be omitted here because it is easy to determine direction and shape of the average scores over time.’’ Admittedly, a closer look at the mean score development over time becomes an issue in TSA when the growth trajectories for the variables involve higher-order polynomials. While most researchers might be able to eyeball linear and even quadratic trends of the mean scores over time, it is less obvious how the mean score would change when growth trajectory includes a cubic or quadratic trend component. For this reason, we believe that the software introduced by Grimm and McArdle (this volume) is very helpful in further

372

KAI S. CORTINA ET AL.

simplifying the TSA approach for more sophisticated MLGMs, as it integrates the mean values in the matrix algebra.

CONCLUSION In sum, Chan’s reservations against TSA as a heuristic strategy seem primarily based on his skepticism about any form of rough screening of the complex data that a MLGM implies. TSA is a relatively simple heuristic procedure, which does not replace the real analysis; ‘‘between 9 and 10’’ is not meant to replace the ﬁnal answer of 9:757; instead it is a reasonable guess of what to expect. As Grimm and McArdle point out in their concluding remarks, TSA might be particularly useful to help novices understand the intricacies of MLGM. We could not wish for more.

REFERENCES Chan, D. (1998). The conceptualization of change over time: An integrative approach incorporating longitudinal means and covariance structures analysis (LMACS) and multiple indicator latent growth modeling (MLGM). Organizational Research Methods, 1, 421–483. Chan, D. (2001). Latent growth modeling. In: F. Drasgow & N. Schmitt (Eds), Measuring and analyzing behavior in organizations (pp. 302–349). New York: Jossey-Bass. Chan, D. (this volume). Multivariate latent growth modeling: Issues on preliminary data analysis. In: F. Dansereau & F. J. Yammarino (Eds), Research in multi-level issues (Vol. 4). Oxford, England: Elsevier. Grimm K. J., & McArdle, J. J. (this volume). A note on the computer generation of mean and covariance expectations in latent growth curve analysis. In: F. Dansereau & F. J. Yammarino (Eds), Research in multi-level issues (Vol. 4). Oxford, England: Elsevier. Jo¨reskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrika, 36, 109–133. Polya, G. (1954). Mathematics and plausible reasoning. Princeton, NJ: Princeton University Press. Sayer, A. G., & Cumsille, P. E. (2001). Second-order latent growth models. In: L. M. Collins & A. G. Sayer (Eds), New methods for the analysis of change (pp. 179–200). Washington, DC: APA. Schnabel, K., Little, T. D., & Baumert, J. (2000). Modeling longitudinal and multi-level data. In: T. D. Little, K. Schnabel & J. Baumert (Eds), Modeling longitudinal and multi-level data: Practical issues, applied approaches, and specific examples (pp. 9–14). Mahwah, NJ: Erlbaum.

PART V: INTRA-CLASS CORRELATION

373

This page intentionally left blank

374

SIGNIFICANCE TESTS FOR DIFFERENCES BETWEEN DEPENDENT INTRACLASS CORRELATION COEFFICIENTS (ICCs) Ayala Cohen and Etti Doveh ABSTRACT Most multi-level studies are cross-sectional and focus on a certain point in time, though various changes within levels may occur over time. This chapter presents a statistical method for assessing whether the degree of interdependency within a group has changed over time, using the intraclass correlation coefficient (ICC) as an indicator of the degree of homogeneity within the groups. It then shows how to apply this method using the SAS MIXED procedure. The problem was motivated by a study in which 120 subjects were divided into 40 groups of three. In a portion of the study, collective efficacy was the dependent variable measured for each subject under four different conditions (two levels of task interdependence at two points in time). ICC was used as a measure of group homogeneity with respect to collective efficacy, and the problem was how to compare the dependent ICCs associated with the different conditions.

Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 375–420 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04017-8

375

376

AYALA COHEN AND ETTI DOVEH

INTRODUCTION Multi-level theory and research have become common in the ﬁeld of industrial and organizational psychology (Klein & Kozlowski, 2000). Data collected within organizations are multi-level in nature. Most of the studies that collect such data are cross-sectional and focus on a certain point in time. Nevertheless, changes in levels may occur over time. Individuals who are initially independent of each other may form groups of interdependent members, or vice versa. In other words, the homogeneity within the group may increase or decrease over time. Longitudinal multi-level studies can identify these shifts, thereby improving our understanding of changing organizational processes (Dansereau, Yammarino, & Kohles, 1999). This chapter explores the question of whether groups form or dissolve over time, presenting a statistical method to assess whether the degree of interdependency among individuals in a group has changed over time. The authors’ research was motivated by a study in which 120 subjects were divided into 40 groups of three. In a portion of the study, collective efﬁcacy was the dependent variable measured for each subject under four different conditions – namely, two levels of task interdependence at two separate points in time. The intraclass correlation coefﬁcient (ICC) was used as a measure of group homogeneity with respect to collective efﬁcacy, and the problem was how to compare the dependent ICCs associated with the different conditions. The evaluation of ICC is one of several techniques based on ANOVA that were developed to determine whether lower-level data are sufﬁciently homogeneous to justify aggregation (e.g., Bartko, 1976; Shrout & Fleiss, 1979; James, 1982; McGraw & Wong, 1996). Another such procedure is the WABA method (Dansereau, Alutto, & Yammarino, 1984). Klein, Bliese, and Kozolowski (2000) showed that ICC and WABA (Within And Between Analysis) lead to very similar conclusions. A different approach for measuring group members’ similarity was introduced by James, Demaree, and Wolf (1984), who proposed rwg(j) as a measure of interrater agreement. The rwg(j) index compares observed variances within a group to an expected variance derived from random responses. In Bliese (2000), interested readers can ﬁnd an overview of what ICC, rwg(j) and the correlation ratio Z2 reveal about the group-level properties of data. Jung and Sosik (2003) used the WABA method to assess group homogeneity at two points in time, in a longitudinal multi-level study that examined collective efﬁcacy and group potency from a level of analysis perspective. Their results showed that initially group coherence was not

Significance Tests for Differences between Dependent Intraclass

377

strong, but that a very strong whole group effect emerged after the group received feedback on its performance. Jung and Sosik’s assessment was not based on a formal statistical test, however, as none is currently available. The objective here is to introduce formal statistical methods that make it possible to draw inferences about differences in the homogeneity of groups observed under different conditions. ICC, which is used as an indicator of group homogeneity, is actually the proportion of variance attributable to the groups. It contrasts the between-group variance with the within-group variance. In most studies where ICC has been applied, it has been used as a descriptive tool. The currently available formal inference methods on ICC remain limited. They make it possible only to test the significance of ICC and to construct conﬁdence intervals. This chapter reviews the methods of inference on ICC and introduces methods for testing the significance of differences between two dependent ICCs. These methods are applicable for longitudinal multi-level studies. All the inference methods are illustrated with SAS software (GLM and MIXED procedures). Readers who are less familiar with the SAS MIXED procedure are referred to Singer (1998). The ﬁrst section reviews the definition and inference methods on ICC (1,1), which is the most commonly used intraclass correlation form. The second section shows how the SAS software package can be used to draw inferences about one or more correlated ICCs. These methods are illustrated with examples that are based on simulated data. The third section describes the longitudinal study that motivated the research presented in this chapter. Collective efﬁcacy was measured under four conditions (low and high levels of task interdependence at two different points in time). ICC was used as a measure of group homogeneity with respect to collective efﬁcacy, and the problem was how to compare the ICCs associated with the different conditions. The fourth section describes the application of the proposed statistical methodology to this study. Under low task interdependence, group homogeneity with regard to collective efﬁcacy was stable in the study; under high task interdependence, it increased with time. A discussion and a summary are presented in the ﬁfth section. The codes of the SAS procedures with explanations are included in the appendix to this chapter.

REVIEW OF ICC Several definitions of ICC exist, with each depending on the assumptions made about the sampling design. For this reason, users of ICC must specify

378

AYALA COHEN AND ETTI DOVEH

an additive variance model appropriate to the sample design. McGraw and Wong (1996) discuss in detail 10 forms of ICC, six of which were described by Shrout and Fleiss (1979). This chapter deals with the intraclass correlation that McGraw and Wong (1996) denoted by ICC (1,1), but which is now usually denoted as ICC (1) (e.g., James, 1982; Bliese, 1998). The methods presented here can be generalized to the other versions of ICC. The basic underlying ANOVA model used to deﬁne ICC (1) is known as the variance component model. Let Yij denote measurements on team members within groups. There are n groups, which are not necessarily equal in size. The size of the ith group, i ¼ 1; . . . ; n; is denoted by ki. The model is Y ij ¼ m þ ri þ wij

(1)

where i ¼ 1; . . . ; n; and j ¼ 1; . . . ; ki. All members of the ith group share the same value of ri. The ri values are random, independent, and normally distributed, with a mean of 0 and a variance of s2r (the between-groups variance). The term wij denotes the individual component. The wij values are assumed to be random, independent, and normally distributed, with a mean of 0 and a variance of s2w (s2w is the within-group variance). The effects ri and wij are independent. For the purposes of this chapter, we will denote the intraclass correlation coefﬁcient ICC (1) simply as ICC. It is deﬁned as follows: ICC ¼

s2r

s2r þ s2w

(2)

ICC is equal to the correlation between any two Y measurements of team members who are in the same group. Both SAS and SPSS provide software for calculating this correlation. Yaffee (1998) and Nichols (1998) explain how to compute ICC with Version 8 of SPSS. Guidelines for using the SAS macro for computing ICC are given at http://www.host.utexas.edu/cc/faqs/stat/sas/sas77.html. In these references and in most textbooks, the discussion of ICC considers data that have the structure of an n k table. The n rows of the table correspond to the n groups, and the k columns correspond to the k subjects nested within the groups. The ICC estimates are based on mean squares obtained by applying an ANOVA model to these data. The groups are assumed to be a random sample taken from a larger population, so the group effect in the ANOVA is always modeled as a random effect. In all cases, ICC is the proportion of variance that is associated with the different groups.

Significance Tests for Differences between Dependent Intraclass

379

The following formulas for ICC (1,1) are presented by McGraw and Wong (1996) and are applicable for balanced data – that is, when the groups are of equal size. Let MSr denote the between-rows mean squares, and MSw the within-rows mean squares. Then, the estimated ICC is ICC ¼

MSr MS w MSr þ ðk 1ÞMS w

(3)

The conﬁdence interval formulas for the ICC population values are based on the F statistic, which tests the significance of the row effect in the ANOVA model. Let Fobserved denote the row effect F from ANOVA Eq. (1), and let FTable denote the ð1 a=2Þ 100th percentile of the F distribution with ðn21Þ numerator degrees of freedom and nðk21Þ denominator degrees of freedom. Then, the lower bound of the conﬁdence interval is F L 1=½F L þ ðk 1Þ; where F L ¼ F observed =F Table ; and the upper bound is F U 1=½F U þ ðk 1Þ; where F U ¼ F observed F Table : As noted by Bliese and Halverson (1998), articles that discuss the calculation of ICC rarely address the issue of unequal group sizes. Haggard (1958) recommended in this case replacing k in Eq. (3) with P 2 X k 1 (4) k¼ ki P i ki ðn 1Þ where ki denotes 2the size of the P ith group. ThePvalue of k can also be s expressed as k k ; where k ¼ ki =n and s2k ¼ ðki kÞ2 =ðn 1Þ: When nk all the groups are of equal size, k ¼ ki and s2k ¼ 0: From Eq. (3), one can obtain an equation that expresses ICC as a function of the F ANOVA statistic: ICC ¼

F 1 F þk1

(5)

This relationship holds both in the balanced and the unbalanced design. In the latter case, k is the value deﬁned in Eq. (4). Eq. (5) shows (1) that high F values lead to high ICC values and (2) that the size of the groups affects the relationship between F and ICC.

Inference on ICC The last 3 decades have seen a great deal of progress in the area of inference on variances in random effects models with unbalanced designs. Owing to the availability of high-power computers, variance parameters are easily

380

AYALA COHEN AND ETTI DOVEH

estimated by using maximum likelihood (ML) or residual (also called restricted) maximum likelihood (REML) estimation methods (Searle, Casella, & McCulloch, 1992). The main difference between these two methods is that, unlike ML, REML takes into account the loss of degrees of freedom that follows from estimating the regression parameters in the ANOVA model. Consequently, ML estimators have a downward bias, while REML estimators are unbiased. When the number of groups is large – according to one rule of thumb, ‘‘large’’ is n430 (Snijders & Bosker, 1999) – the difference between REML and ML variance estimates is practically insignificant. For balanced, mixed ANOVA models, the REML estimate is identical to the classical ANOVA estimate. In general, REML and ML differ little with respect to the estimates of the ﬁxed part. Any multi-level software, including SAS, provides both ML and REML estimates. The estimated ICC can then be derived by replacing the variances s2r and s2w in Eq. (2) with their corresponding estimates. An important question is how to design an experiment so as to obtain a smaller variance for the estimated ICC. In other words, given n subjects for an experiment, how should the researcher divide the subjects into groups? Are numerous small groups more or less desirable than a few large ones? When Snijders and Bosker (1999) investigated this question, they found that larger groups are preferable when the value of ICC is close to 1, whereas smaller groups are preferable for very small values of ICC (close to 0). In practice, the researcher does not know the value of ICC, but often he or she has some prior knowledge about its range. The Sign of ICC One alternative approach to deﬁning ICC is not based on ANOVA Eq. (1). It preserves the concept of ICC as a measure of the correlation between two Y measurements of team members in the same group, but it allows this correlation to be negative. Under this approach, the ANOVA model is Y ij ¼ m þ eij

(6)

where i ¼ 1; . . . ; n and j ¼ 1; . . . ; ki : The random variables eij are assumed to be normally distributed with a mean of zero, as were the wij variables of Eq. (1). However, unlike the wij variables of Eq. (1), which are assumed to be independent, the eij variables are modeled as being correlated. The assumption is that the correlation ðeij ; eim Þ ¼ r; where jam and j, m ¼ 1; . . . ; ki ; and the correlation ðeij ; egm Þ ¼ 0 where iag and i, g ¼ 1; . . . ; n: In other words, the measurements corresponding to members of the same group are

Significance Tests for Differences between Dependent Intraclass

381

correlated, and the same correlation ICC ¼ r holds between all pairs of members within the same group. There is no correlation between measurements of members in different groups. Both Eqs. (1) and (6) are hierarchical models with a correlation between subjects in the lower level. In the variance component Eq. (1), the correlation is induced through the random group effect ri, which is common to members of the same group. By definition, the correlation in this model is non-negative. In Eq. (6), however, the correlation is modeled through the covariance of the errors, and it can obtain negative values. Eq. (6) is therefore more general and should be used when the groups may be heterogeneous (negative ICC). When r is positive, Eqs. (1) and (6) are identical.

ANALYZING ICC BASED ON SAS PROCEDURES (GLM AND MIXED) The statistical methods that we propose to compare two or more ICCs are based on ANOVA models that are more complex than Eqs. (1) and (6). To make these methods easier to understand, we illustrate ﬁrst how to apply SAS software to draw inferences about a single ICC. Then, we extend this procedure to compare two ICCs. In each case, ﬁrst the relevant ANOVA model is formulated, then inferences are drawn about the ICC from the SAS output. All of the examples presented here are based on simulated data, and they proceed from simple to more complex problems. The SAS codes for all the problems, and detailed explanations of the various options, appear in the appendix to this chapter. The ﬁrst three examples illustrate methods of inference on a single ICC. In each case the total sample size is the same, but the three examples differ in their design. In Examples 1 and 2, the groups are balanced (equal in size); in Example 3, the groups have varying sizes. In Examples 1 and 3, the population ICC equals 0.8, indicating that there is a strong resemblance between individuals within each group. In Example 2, the ICC is zero, meaning that there is no group effect. Examples 4 and 5 compare two correlated ICCs. This procedure makes it possible to assess whether a change has occurred in the homogeneity of the groups. Example 4 illustrates a case in which no group effect is discernible, and no change occurs over time. Example 5 deals with a case in which group homogeneity changes over time, from less homogeneous ðICC ¼ 0Þ to more homogeneous (ICC40).

382

AYALA COHEN AND ETTI DOVEH

Example 1 (Equal Group Sizes, ICC40) Design and Implementation For the ﬁrst example, we generated data according to Eq. (1). We arbitrarily chose the values s2r ¼ 4 for the group effect and s2w ¼ 1 for the residuals wij. Thus, the generated data represent a population of groups for which the ICC is r¼

s2r

s2r 4 ¼ ¼ 0:8 þ s2w 5

We simulated measurements with n ¼ 10 groups, each of size k ¼ 15; so that the data set included 150 observations. The input data (found in the appendix) form a 10 15 table. An examination of these data reveals that measurements associated with each group are relatively similar. Table 1 presents the resulting ANOVA table. The estimated ICC value according to Eq. (5) is ICC ¼

F 1 68:17 1 ¼ ¼ 0:817 F þ ðk 1Þ 68:17 þ 14

Since for ð1 a=2Þ 100 ¼ 97:5; ðn 1Þ ¼ 9; nðk 1Þ ¼ 140; and F Table ¼ 2:206; the corresponding 0.95 conﬁdence interval is [0.666, 0.909]. As expected, the interval includes the true population value of 0.8. Next, we apply the MIXED procedure of SAS (Littell, Milliken, Stroup, & Wolﬁnger, 1996), which produces REML estimates of the two variances s2r and s2w : Table 2 displays the results. The ICC estimate is obtained by substituting the estimates of the between-groups and within-group variances (s2r ; s2w ) in Eq. (2): ICC ¼

Table 1.

3:6520 ¼ 0:817 3:6520 þ 0:8156

The ANOVA Table (Example 1).

Source

DF

Sum of Squares

Mean Square

F

p4F

Model Error

9 140

500.36456 114.18357

55.59606 0.81559

68.17

o0.0001

Significance Tests for Differences between Dependent Intraclass

Table 2.

383

Estimated Covariance Parameters (Example 1).

Cov Parm

Estimate

Standard Error

Z

p

Covariance Within

3.6520 0.8156

1.7472 0.0975

2.09 8.37

0.0183 o0.0001

Analysis The MIXED and GLM procedures produced the same variance estimates, and consequently the same ICC estimate (0.817). This principle will always hold when group sizes are equal and when the F ratio is greater than 1 (Donner, 1986). From Table 2 we can also conclude that the ICC is significant, because the hypothesis that s2r ¼ 0 is rejected. The p value of the test is based on the Z statistic – that is, the ratio of the estimate divided by its standard error (known as a Wald statistic). The value of p is larger than that obtained by applying the more precise F test. It is known that Wald tests for variances are of limited value. The sampling distribution of variance estimates is highly skewed, and the convergence to normality holds for very large samples. We therefore applied the deviance test. This large-sample test is also based on asymptotic theory, but it performs better than the Wald test. The deviance measures the lack of ﬁt between the model and the data. Its definition depends on whether the REML or ML method is used to estimate the parameters. In the former case, the deviance is deﬁned as the negative of twice the natural log of the residual likelihood; in the latter, it is deﬁned as the negative of twice the natural log of the likelihood (not the residual likelihood). In either case, the deviance value displayed in the MIXED output can be used to compare nested models. (The default option in SAS is REML.) The deviance test resembles the F test applied in multiple regression analysis to compare nested models that differ in their number of explanatory variables. In classical multiple regression analysis, the F test essentially compares the residual sum of squares of the two nested models. A small difference between the residual sum of squares indicates that the goodnessof-ﬁt of the model with fewer explanatory variables is equal to that of the extended model. Similarly, in the deviance test, a small difference between the deviances of two nested models indicates that the goodness-of-ﬁt of the model with fewer parameters is equal to that of the extended model. For testing the hypothesis r ¼ ICC ¼ 0; Example 1 compares the deviance of

384

AYALA COHEN AND ETTI DOVEH

the ﬁtted Eq. (1) [or Eq. (6)] with the deviance of the model that is ﬁtted under the null hypothesis. Another distinction to be made is whether the test should be considered one- or two-sided. In the framework of the variance component Eq. (1), the test is considered one-sided: By definition, s2r X0: When Eq. (6) is the underlying model, a two-sided alternative becomes meaningful. When the test is one-sided, then under the null hypothesis the tested parameter s2r is on the boundary of the parameter space; as a result, the standard theory on REML or ML estimators cannot be applied (e.g., Verbeke & Molenberghs, 1997). This restriction does not necessarily apply to the present case, as ICC in general can obtain negative values. Let D1 denote the deviance obtained by ﬁtting Eq. (6), and D0 denote the deviance obtained by ﬁtting this model under the hypothesis r ¼ ICC ¼ 0: Under the null hypothesis, all observations are identically distributed and no group effect occurs. When the test is two-sided, the difference D ¼ ðD0 D1 Þ is asymptotically distributed w2 ð1Þ: When it is one-sided, so that the parameter under the null hypothesis is on the boundary, the null distribution of D is not w2 ð1Þ but rather a mixture of w2 : In this case, the value of p associated with the test r ¼ ICC ¼ 0 is half of the tail probability of D in a w2 ð1Þ distribution (for example, if D ¼ 3:84; then p ¼ 0:025). This principle was derived by Self and Liang (1987). Snijders and Bosker (1999) offer some intuitive arguments for why the tail probability should be divided by 2 to derive the value of p. The SAS output for every ﬁtted model includes the value of 2Res log likelihood. The value of D is equal to the difference between (–2Res log likelihood) of the restricted model (under the null hypothesis) and (–2Res log likelihood) of the full model. Note that in this example, the model under the null hypothesis (known as the ‘‘empty model’’) is Y ij ¼ m þ wij

(7)

where i ¼ 1; . . . ; n and j ¼ 1; . . . ; k: When Eq. (1) and the model under the null hypothesis were compared in Example 1, a large value of D ¼ 203:5 resulted, meaning that the value of p is practically zero. The analysis thus leads to the correct conclusion – namely, that the ICC is significant. What happens if we ﬁt ANOVA model (6)? We then specify in the SAS code that the ﬁtted covariance structure should be compound symmetry. In other words, the same covariance (correlation) should hold between measurements of each pair of subjects in the same group, and the variance of the measurement should be the same for all subjects.

Significance Tests for Differences between Dependent Intraclass

385

Comparing the outputs of the MIXED procedure for ANOVA Eqs. (1) and (6), one can see that the same estimates are obtained for the variances s2r and s2w : However, the p value of the Wald Z statistic is computed for a onesided alternative in the ﬁrst case (using the RANDOM statement in SAS), and for a two-sided alternative in the second case (using the REPEATED statement in SAS). This difference arises because in the second case, the covariance need not be positive. An advantage in using the REPEATED statement is that the output includes the result of the deviance test. In this example, the values of p of the deviance test and the exact F test are the same and much smaller than the p value of the Wald test. Example 2 (Equal Group Sizes, ICC ¼ 0) Design and Implementation In Example 2, data from 150 measurements were simulated according to Eq. (1), with subjects being divided into n ¼ 10 groups of k ¼ 15: In this case, however, s2r ¼ 0; making the population ICC equal to 0. Thus, there is no group effect. An examination of the data for this example (found in the appendix) reveals that the measurements within any given group are not necessarily more similar than the measurements between groups. Table 3 represents the ANOVA table obtained using the GLM procedure of SAS. As expected, the large p value indicates that there is no group effect. When we apply Eq. (5) to estimate the ICC, we obtain a negative value. Applying the MIXED procedure produces Table 4. Here, the ﬁtting algorithm fails to obtain a positive estimate for the group effect. By definition s2r X0; so the estimate of s2r is set at zero and we do not need to estimate its standard error. Table 5 shows the estimated covariance structure obtained by ﬁtting Eq. (6) to the data of Example 2. Analysis According to this output, the estimated covariance between measurements of subjects in the same group is negative, but statistically nonsignificant

Table 3.

The ANOVA Table (Example 2).

Source

DF

Sum of Squares

Mean Square

F

p4F

Model Error

9 140

5.280921 114.18357

0.586769 0.81559

0.72

0.6904

386

Table 4.

AYALA COHEN AND ETTI DOVEH

Estimated Covariance Parameters Fitting Eq. (1) (Example 2).

Cov Parm

Estimate

Standard Error

Covariance Within

0 0.8018

0.09289

Table 5.

Z

p

0.000

Estimated Covariance Parameters Fitting Eq. (6) (Example 2).

Cov Parm

Estimate

Standard Error

Z

p

Covariance Within

0.01526 0.8156

0.01955 0.09748

0.78

0.4353 o0.0001

ðp ¼ 0:435Þ: Thus, we obtain the correct conclusion that the population ICC equals zero and there is no group effect.

Example 3 (Unequal Group Sizes, ICC40) Design and Implementation As in Example 1, the data in Example 3 were generated according to Eq. (1) with s2r ¼ 4 and s2w ¼ 1; so they represent a population of groups for which the ICC has the value ICC ¼

s2r

s2r 4 ¼ ¼ 0:8 2 þ sw 5

Once again, the simulation involved 10 groups and 150 observations, but this time the sizes of the groups varied, ranging from k ¼ 6 to 24 (the size of the ith group was ki ¼ 4 þ 2i). Table 6 displays the ANOVA results obtained with the GLM program. Because the design in this example is unbalanced, we use Eq. (4) to calculate the value of k for Eq. (5) (the calculation yields k ¼ 14:755). ICC ¼

F 1 103:63 1 ¼ ¼ 0:874 F þ ðk 1Þ 103:63 þ 13:755

The resulting 0.95 conﬁdence interval is [0.757, 0.939]. As expected, the interval includes the true population value of 0.8. The results produced by the MIXED procedure can be found in Table 7.

Significance Tests for Differences between Dependent Intraclass

Table 6.

387

The ANOVA Table (Example 3).

Source

DF

Sum of Squares

Mean Square

F

p4F

Model Error

9 140

947.81594 142.27460

105.31288 1.01625

103.63

o0.0001

Table 7.

Estimated Covariance Parameters (Example 3).

Cov Parm

Estimate

Standard Error

Z

p

Covariance Within

6.1226 1.0161

2.9174 0.1214

2.10 8.37

0.0179 o0.0001

The ICC estimate is easily obtained by replacing the variances s2r and s2w in Eq. (2) with their corresponding estimates: ICC ¼

6:1226 ¼ 0:858 6:1226 þ 1:0161

Analysis The two procedures – MIXED and GLM – produced essentially the same residual variance estimate s2w (1.016), and the same ICC estimate. The differences between the results are negligible. However, an advantage of the MIXED-based ICC estimate over the ANOVA-based estimate is that regardless of whether the groups are balanced or unbalanced, the ICC is easily obtained by substituting the REML variance estimates in Eq. (2), which deﬁnes the ICC.

Example 4 (ICC ¼ 0, No Change Over Time) Design and Implementation For Example 4, we simulated data corresponding to measurements on 30 equal-sized groups of three at two different points in time. The simulated data were designed to show no change over time and no group effect ðICC ¼ 0Þ: To compare two ICCs, we extend the underlying basic ANOVA model and take into account the ‘‘repeated measures’’ design. The extended model is a genuine MIXED model, because it includes ﬁxed effects (time), random

388

AYALA COHEN AND ETTI DOVEH

effects (groups and individuals), and interactions. Based on the MIXED ﬁtted models, we can draw inferences about the ICCs corresponding to the two measurement times. The ANOVA (MIXED) model that describes these measurements is Y iðjÞt ¼ m þ ðtimeÞt þ ðgrpÞi þ ðidÞjðiÞ þ ðgrp timeÞit þ eiðjÞt

(8)

where i ¼ 1; . . . ; 40; j ¼ 1; 2; 3; and t ¼ 1; 2: Time is the only ﬁxed effect in the model (time ¼ 1; 2). Both group (grp) and individual (id) are random effects with variances of s2g and s2ind ; respectively. The notation j(i) indicates that the individual effect is nested within the group effect; the data structure of repeated measures is expressed by (id)j(i). Contributions to the variance of Y come from three sources: the variance between groups, the variance between individuals, and the random unexplained variance expressed by e. The errors are assumed to be independent and normally distributed with a mean of zero and a variance of s2e ; which is assumed to be the same for both times. The within-group variance (that is, the component of variance that is not due to variance between groups) is s2w ¼ s2ind þ s2e : The model includes two interaction terms ðgrp timeÞit ; with t ¼ 1; 2 corresponding to the two times. Each interaction is random, with a mean of zero and a variance of s2gt ; t ¼ 1; 2: The two ICCs corresponding to t ¼ 1; 2 are as follows: ðICCÞt ¼

s2g þ s2gt . s2g þ s2gt þ s2w

(9)

Eq. (8) implies the following covariance structure. The covariance between measurements of different subjects in the same group i(j), i(j0 ) at different times (t, t0 ) stems from the common group effect: CovðY iðjÞt ; Y iðj0 Þt0 Þ ¼ s2g

(10)

There is an additional contribution to the covariance between measurements of different subjects in the same group i(j), i(j0 ) when both measurements refer to the same time. The addition, which depends on t, is s2gt : Therefore, CovðY iðjÞt ; Y iðj 0 Þt Þ ¼ s2g þ s2gt

(11)

The covariance between measurements of the same subject at different times (t, t0 ) is affected by the common group effect and the individual subject effect. Therefore, CovðY iðjÞt ; Y iðjÞt0 Þ ¼ s2g þ s2ind

(12)

Significance Tests for Differences between Dependent Intraclass

389

The variance of the measurement at time t for subject i(j) includes four independent terms: VarðY iðjÞt Þ ¼ s2ind þ s2g þ s2gt þ s2e

(13)

We used the following parameters in simulating Example 4: s2g ¼ 0; ¼ s2g;t¼2 ¼ 0; s2ind ¼ 3; s2e ¼ 4: Therefore, the true (population) ICCs in this example are equal to zero: ðICCÞ1 ¼ ðICCÞ2 ¼ 0: s2g;t¼1

Analysis Two questions now arise: 1. Is there a significant difference between the two (dependent) ICCs? 2. Are the ICCs significantly different from zero? In regard to the ﬁrst question, we see from Eq. (9) that equality holds if and only if s2g;t¼1 ¼ s2g;t¼2 : In the SAS MIXED procedure, we use the option that allows the covariances to be negative (the option ‘‘nobound’’). Thus, when the variances that induce covariances (correlations) between observations are actually equal to zero [Eqs. (10)–(12)], negative estimates of ‘‘variances’’ become possible. Such estimates correspond to negative, but nonsignificant sample correlations. From the SAS output (see the appendix), we obtained the estimated ICCs associated with the two measurement times. t ¼ 1: estimated ICC ¼ 0:005 t ¼ 2: estimated ICC ¼ 0:040 Table 8 displays the estimated variances, and Table 9 shows the asymptotic estimates of the variances and covariances of the estimates in Table 8. The estimated covariances are used if the Wald test is applied to compare the two ICCs. According to the output, s^ 2g ¼ 0:4596; s^ 2ind ¼ 1:5910; and Table 8. Cov Parm id(grp) grp grp time grp time Residual

time

1 2

Estimated Covariance Parameters (Example 4). Estimate

Standard Error

Z

p

1.5910 0.4596 0.4287 0.2111 4.3658

0.7960 0.5094 0.5146 0.5467 0.7971

2.00 0.90 0.83 0.39 5.48

0.0456 0.367 0.4048 0.6994 o0.0001

390

AYALA COHEN AND ETTI DOVEH

Table 9.

id(grp) grp grp time grp time Residual

Asymptotic Covariance Matrix of the Variance Estimates (Example 4). id(grp)

grp

grp time 1

Grp time 2

Residual

0.6336

0.2112 0.2595

0.1059 0.0868 0.2648

0.1059 0.0719 0.0371 0.2989

0.3177 0.1059 0.2118 0. 2118 0.6353

s^ 2e ¼ 4:3658: The estimated values s^ 2gt for both t ¼ 1 and 2 are negative, but they are (as they should be) nonsignificant. There are two ways to test the significance of the difference between the ICCs associated with the two time periods (equivalent to the significance of the difference s2g;t¼1 s2g;t¼2 ). The ﬁrst – and more accurate – method is the deviance test, which compares the deviance of the current model with that of the model under the null hypothesis. The model under the null hypothesis includes only one interaction term ðgrp timeÞ: Y iðjÞt ¼ m þ ðtimeÞt þ ðgrpÞi þ ðidÞjðiÞ þ ðgrp timeÞi þ eiðjÞt

(15)

where i ¼ 1; . . . ; 40; j ¼ 1; 2; 3; and t ¼ 1; 2: The second, less accurate method is the Wald test, which employs the asymptotic covariance matrix of the variance estimates. Here, the test statistic s^ 2g;ðt¼1Þ s^ 2g;ðt¼2Þ Z ¼ qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ ^ s^ 2g;ðt¼1Þ ; s^ 2g;ðt¼2Þ Þ V^arðs^ 2g;ðt¼1Þ Þ þ V^arðs^ 2g;ðt¼2Þ Þ 2Covð under the null hypothesis is N(0,1). Substituting the results from Tables 8 and 9, we obtain 0:4287 ð0:2111Þ Z ¼ pﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ ¼ 0:311 0:2648 þ 0:2989 2ð0:0371Þ This calculation is given for purposes of illustration only, as its results cannot be considered reliable. We therefore consider only the results of the deviance test. In the full model, –2Res log likelihood is equal to 824.1; in the model under the null hypothesis, it is equal to 824.2 – a most definitely nonsignificant difference ðp ¼ 0:755Þ: We thus obtain the correct conclusion that

Significance Tests for Differences between Dependent Intraclass

391

no difference in homogeneity exists between the two time periods (the level of analysis remained stable). The estimated ICC common to the two times is equal to 0.023. The next step is to determine the significance of the common estimated ICC. For this purpose, we again apply the deviance test. The null hypothesis is that the groups consist of independent individuals. Under this hypothesis the model is Y iðjÞt ¼ m þ ðtimeÞt þ ðidÞjðiÞ þ eiðjÞt

(16)

where i ¼ 1; . . . ; 40; j ¼ 1; 2; 3; and t ¼ 1; 2: This model includes a ﬁxed time effect and a random individual effect, which induces the correlation between repeated measurements of the same individual. It does not include a group effect. In Eq. (15), (–2Res log likelihood) ¼ 824.2; in the model under the null hypothesis [Eq. (16)], (–2Res log likelihood) ¼ 825.2. Under the null hypothesis, the difference D is asymptotically distributed w2 ð1Þ; because Eq. (16) has one fewer random component than Eq. (15). The result ðp ¼ 0:307Þ indicates that the ICC common to both times is equal to zero. We thus obtain the correct conclusion that no group effect occurs at either time. Example 5 (ICC ¼ 0 to ICC40) Design and Implementation For Example 5, as in Example 4, we simulated data corresponding to measurements on 30 groups of three taken at two points in time. Here, however, the ICC changes over time, from 0 to 0.571. The parameters used were as follows: s2g ¼ 0; s2g;t¼1 ¼ 0; s2g;t¼2 ¼ 4; s2ind ¼ 3; and s2e ¼ 4: Therefore, ðICCÞ1 ¼ 0; while ðICCÞ2 ¼

s2g þ s2gt 4 ¼ 0:571 ¼ 2 2 2 sg þ sgt þ sw 4 þ 3

From the SAS output (see the appendix), we obtained the estimated ICCs associated with the two times: t ¼ 1: estimated ICC ¼ 0:005 t ¼ 2: estimated ICC ¼ 0:479 Table 10 shows the estimated variances. According to the output, s^ 2g ¼ 0:1039; s^ 2ind ¼ 1:5910; and s^ 2e ¼ 4:3658: For t ¼ 1; s^ 2gt ¼ 0:1348; for t ¼ 2;

392

AYALA COHEN AND ETTI DOVEH

Table 10. Cov Parm id(grp) grp grp time grp time Residual

time

1 2

Estimated Covariance Parameters (Example 5). Estimate

Standard Error

Z

p

1.5910 0.1039 0.1348 5.5755 4.3658

0.7960 0.7715 0.8702 1. 9980 0.7971

2.00 0.13 0.15 2.79 5.48

0.0456 0.8928 0.8769 0.0053 o0.0001

s^ 2gt ¼ 5:5755: Once again, the estimated value s^ 2g is negative, but it is (as it should be) not significant. Analysis We applied the deviance test to compare the ICCs associated with the two time periods. In the full model, –2Res log likelihood is equal to 865.8; in the model under the null hypothesis, it is equal to 877.5. The difference is highly significant ðp ¼ 0:0006Þ: We thus obtain the correct conclusion that homogeneity increased significantly between the two points in time (the level of analysis changed).

DESCRIPTION OF THE MOTIVATING EXAMPLE In the study that motivated the current research (Katz & Erez, in press), selfefﬁcacy and collective efﬁcacy were examined under different degrees of task interdependence. To test one of their hypotheses, the researchers needed to assess the significance of differences between correlated ICCs. Here, we limit our description to the part of the study that pertains to this hypothesis. The study was designed to examine the role played by task interdependence in determining collective efﬁcacy as a shared group-level concept (Katz & Erez, in press). Collective efﬁcacy reﬂects the shared beliefs of group members about their group’s capacity to be motivated, mobilize its cognitive resources, and take action to meet given situational demands (Bandura, 1997). Task interdependence is a contextual factor that inﬂuences the level of interaction among team members. It is determined by the degree to which different functions within the group are linked, and the coordination required from the team (Kozlowski, Gully, Nason, & Smith, 1999). In groups exhibiting very low interdependence, each team member contributes to the team output without the need for direct interaction with other members. In

Significance Tests for Differences between Dependent Intraclass

393

groups with high interdependence, the output of one member becomes the input of another member. Under conditions of high task interdependence, team members interact closely with one another, and so have opportunities to develop similar perceptions of collective efﬁcacy. Katz and Erez (2004) hypothesized that homogeneous perceptions of collective efﬁcacy would emerge as team members gained experience working together under conditions of high – but not low – task interdependence. Their subjects included 120 engineering students who were paid for their participation in the study. The task was a management simulation of performance appraisal (used by Saavedra, Early, & Van Dyne, 1993; see pp. 65–66). It required groups of ‘‘managers’’ to recommend variable meritbased bonuses for ﬁctitious employees in different departments on the basis of written descriptions. The participants were randomly divided into 40 three-person work teams. Team members were then randomly assigned different roles in the group. The experiment itself was divided into two sessions. Half of the teams worked under conditions of low task interdependence in session 1 and high task interdependence in session 2; the other 20 teams did the opposite. Each session included (1) a 5-min practice trial; (2) initial self- and collective efﬁcacy questionnaires; (3) a 10-min performance trial; and (4) subsequent self-efﬁcacy and collective efﬁcacy questionnaires. There were no pauses between sessions, and the entire procedure took about 1 h. Under conditions of low task interdependence, the members of each group were assigned different sets of employees and told to perform the appraisals on their own. Each team’s performance was measured as the total number of evaluations completed by the three members of the team. The high-interdependence portion of the study was designed at the level of reciprocal interdependence. Under this condition, two group members worked together to perform the task, while the third member facilitated their activities. The ﬁnal evaluation was accomplished only with the cooperation of all three team members, and team performance was measured as the number of forms completed by the team. Collective efﬁcacy was measured in the following way. First, each participant was asked to estimate his or her team’s ability to perform the task at different levels (namely, to evaluate different numbers of employees, ranging from 12 to 26, in 10 min). The sum of positive responses made up the collective efﬁcacy magnitude scale. Second, participants rated their conﬁdence that their team would achieve each level on scale from 1 (not certain at all) to 10 (very certain). The sum of the conﬁdence scores made up the collective efﬁcacy strength scale. As the two efﬁcacy scales were highly

394

AYALA COHEN AND ETTI DOVEH

correlated ðr ¼ 0:94Þ; only the collective efﬁcacy strength scale was ultimately employed because of its higher variance. These scales were used to measure collective efﬁcacy under the condition of low task interdependence. Under the condition of high interdependence, the scales were modiﬁed, taking into consideration the fact that each team member completed only a portion of each evaluation. ICCs derived from ANOVA models, where collective efﬁcacy served as the dependent variable, were used to evaluate the homogeneity of team members’ perceptions of collective efﬁcacy. For each team member (nested within his or her group), measurements were made for each combination of the two ﬁxed effects: time (initial or subsequent) and task interdependence (low or high). Thus, an ICC value was associated with each of the four experimental conditions. Note that, if we were interested in only one of these four conditions, we would display the data as an n k (40 3) matrix, and the underlying ANOVA model would be Eq. (1) or Eq. (6). A conﬁdence interval and/or test significance rating could then be determined using the methods illustrated for Examples 1–3. In reality, this exercise would provide no information about the homogeneity among team members under different circumstances and, in particular, over time. To compare two or more of the four ICCs, we apply MIXED models that take into account the ‘‘repeated measures’’ design. These models include ﬁxed effects (time and/or task), random effects (teams and individuals), and interactions. Based on the MIXED ﬁtted models, we can draw inferences on the ICCs corresponding to the four different conditions.

Analyzing Correlated ICCs for the Motivating Example The ﬁrst, exploratory stage in the analysis involves ﬁtting ANOVA Eq. (1) separately to each of the four conditions. The next step is to perform two pairwise comparisons. First, we compare the ICCs corresponding to the two points in time under the condition of low task interdependence, which reveals that there is no significant difference between these ICCs (that is, group homogeneity did not change over time under this condition). Next, we compare the two ICCs corresponding to the two task levels at the initial time; as before, no statistically significant difference exists between the two ICCs (meaning that the groups exhibited the same degree of homogeneity under the low- and high-task interdependence, at the initial time).

Significance Tests for Differences between Dependent Intraclass

395

In the third step of the analysis, we take the pairwise comparisons a step further and show that no statistically significant difference exists between the three ICCs analyzed previously. Finally, we show that there is a significant difference between these three ICCs and the fourth ICC corresponding to subsequent time and high task interdependence. The results in Table 11 were obtained by ﬁtting Eq. (1) to each of the four conditions. In each case, we applied both the GLM and the MIXED procedures. Note that because the alternative is one-sided in the F test, the value of p of this test is essentially half of the p value of the deviance test. In the current problem, we should consider a two-sided test, as interdependence can arise either because the groups are heterogeneous (negative correlation among members) or because they are homogeneous (positive correlation among members). The results in Table 11 on their own – that is, without the benefit of formal statistical comparison – suggest two conclusions: 1. Under the condition of low task interdependence, the group effect on collective efﬁcacy was initially not significant, but subsequently was significant (that is, these groups appear to have gained in coherence over time). 2. Under conditions of high task interdependence, significant group effects can be observed at both times (the level of analysis remained stable). These conclusions are not supported by formal statistical inference, as will become apparent when we apply significance tests to comparisons of the empirical ICCs.

Table 11.

Results of Fitting Eq. (1) to Each of the Four Conditions. Estimated p Value of p Value of ICC F Test Deviance Test

Condition 1 Low task interdependence, initial time Condition 2 Low task interdependence, subsequent time Condition 3 High task interdependence, initial time Condition 4 High task interdependence, subsequent time

s^ 2w

0.091

0.158

0.380

366.923

0.187

0.0241

0.054

307.73

0.287

0.0016

0.0032

327.81

0.331

0.0003

0.0007

209.44

396

AYALA COHEN AND ETTI DOVEH

In this context, it is of interest to compare the estimated residual variances under the different conditions. At a high level of task interdependence, though only a small difference exists between the estimated ICCs associated with the two times, a large difference between the residual variances is apparent. We shall refer to this point later on.

First Pairwise Comparison (Conditions 1 and 2) We begin by comparing the ICCs corresponding to the two points in time under conditions of low task interdependence. The estimated ICCs were 0.113 and 0.1452 for the initial and subsequent times, respectively. These estimates differ from those obtained when each ICC was estimated separately – 0.091 and 0.187, respectively. It is not unusual for regression estimates based on pooled data to differ from those based on subsections. The estimated variances (see Table 12) are s^ 2g ¼ 18:980; s^ 2gt ¼ 24:740; for t ¼ 1; and 38.838, for t ¼ 2; s^ 2ind ¼ 127:36; and s^ 2e ¼ 213:01: Clearly, there is a relatively large contribution from the individual variance, whereas the variance between groups is relatively small. The null hypothesis (no change over time) was tested by comparing the deviance of the full model, Eq. (8), with that of the restricted model under the null hypothesis, Eq. (15). The two deviances are essentially the same, differing only after the second decimal place. Consequently, we may conclude that no increase in homogeneity occurred between the initial and subsequent periods under conditions of low task interdependence (i.e., the level of analysis remained stable). The question then becomes whether the estimated ICC common to the two points in time – which equals 0.1299 – indicates homogeneity. The tested null hypothesis is that the groups are noncohesive ðICC ¼ 0Þ:

Table 12. Cov Parm id(grp) grp grp time grp time Residual

Estimates of Variances (Low Task Interdependence, Two Times). time

1 2

Estimate

Standard Error

Z

p

127.36 18.980 24.740 38.838 213.01

41.271 31.252 35.280 36.900 34.083

3.08 0.61 0.70 1.05 6.25

0.001 0.271 0.242 0.146 o0.0001

Significance Tests for Differences between Dependent Intraclass

397

The deviance obtained by ﬁtting Eq. (16) is D0 ¼ 2061:1; while the deviance obtained via Eq. (15) is D1 ¼ 2057:5: Under the null hypothesis, the difference is asymptotically distributed w2 ð2Þ; because Eq. (16) has two fewer random components than Eq. (15). The result, p ¼ 0:165; indicates that no group effect can be discerned under conditions of low interdependence at either time. Second Pairwise Comparison (Conditions 1 and 3) Next, we need to determine whether the ICCs for the two levels of task interdependence at the initial time are significantly different. The corresponding ANOVA (MIXED) model is similar to Eq. (8) (for two points in time and one task level). The current model is Y iðjÞtask ¼ m þ ðtaskÞtask þ ðgrpÞi þ ðsubjÞjðiÞ þ ðgrp taskÞi;task þ eiðjÞtask (17) where i ¼ 1; . . . ; 40; j ¼ 1; 2; 3; and task ¼ L; H: The task (L ¼ low; H ¼ high) is the only ﬁxed effect in this model. Each of the two interactions ðgrp taskÞi;task is random, with a mean of zero and a variance of s2g;task : The two ICCs corresponding to task ¼ L; H are ðICCÞtask ¼

s2g þ s2g;task s2g þ s2g;task þ s2w

(18)

The significance of the difference between the ICCs associated with the two task levels is equivalent to the significance of the difference: ðs2g;task¼L Þ ðs2g;task¼H Þ: When we apply the deviance test, we obtain p ¼ 0:11: The estimated ICCs for the low and high task levels based on Eq. (17) are 0.11 and 0.266, respectively. Although the difference between these two ﬁgures seems large, with p ¼ 0:11 this difference cannot be considered statistically significant. The common estimate under the null hypothesis is 0.198. Three-Way Comparison (Conditions 1, 2, and 3) Neither of the two pairwise comparisons revealed any significant difference between the analyzed ICCs. In a three-way comparison, we test the equality of these three ICCs simultaneously. The procedure employed in this test closely resembles that of the pairwise comparison. The underlying ANOVA model [note the similarity to Eq. (8)] is Y iðjÞt ¼ m þ ðconÞcon þ ðgrpÞi þ ðsubjÞjðiÞ þ ðgrp conÞi;con þ eiðjÞ;con

(19)

where i ¼ 1; . . . ; 40; j ¼ 1; 2; 3; and con ¼ 1; 2; 3: Eq. (19) includes the ﬁxed effect con (condition), and the three interaction terms ðgrp conÞi;cont ; where

398

AYALA COHEN AND ETTI DOVEH

con ¼ 1; 2; 3 corresponds to the three possible states of this effect. Each of these interactions is random, with a mean of zero and a variance of s2g;con : The three ICCs corresponding to con ¼ 1; 2; 3 are ðICCÞcon ¼

s2g þ s2g;con s2g þ s2g;con þ s2w

(20)

We address two questions: Is there a significant difference between the three ICCs? Are the ICCs significantly different from zero? First, we apply the deviance test. Under the null hypothesis ðs2g;con¼1 Þ ¼ ðs2g;con¼2 Þ ¼ ðs2g;con¼3 Þ; D is asymptotically distributed as w2 ð2Þ: The value p ¼ 0:223 indicates that no statistically significant difference exists between the three correlated ICCs. In the full model, the ICCs for the three conditions are estimated at 0.103, 0.196, and 0.309, respectively. As in the second pairwise comparison, the differences among these ICCs seem large; however, with a p value of 0.22, they cannot be considered statistically significant. The pooled estimate under the null hypothesis is 0.20 and is significant (po0.001). The analysis at this stage indicates that the degree of homogeneity within the groups under these three conditions – low task interdependence at both initial and subsequent times, and high task interdependence at the initial time – is essentially the same. Note, however, that the results of the threeway analysis show some homogeneity (positive ICC) among group members under each condition, in contrast to the results when pairs of conditions were compared. This difference arises because by pooling the data for the three conditions we gained statistical power. Conclusion (Conditions 1–4) So far, we have not touched upon the ICC corresponding to the fourth condition – high task interdependence and subsequent time. Recall that when reporting on the individual analysis for each of the four conditions, we pointed out that the estimated within-group variance was much smaller under this condition than under the others. (A formal test, described later in this chapter, conﬁrmed that the error variance in condition 4 was not equal to the error variance in the other conditions.) Thus, Eqs. (8) and (15) are not adequate for further analysis unless we extend the model to include different error variances.

Significance Tests for Differences between Dependent Intraclass

399

To compare the pooled ICC of conditions 1–3 with that of condition 4, the underlying ANOVA model is Y iðjÞ;tðmÞ ¼ m þ ðconÞtðmÞ þ ðgrpÞi þ ðsubjÞjðiÞ þ ðgrp catÞim þ eiðjÞ;tðmÞ

(21)

where i ¼ 1; . . . ; 40; j ¼ 1; 2; 3; t ¼ 1; 2; 3; 4; and m ¼ 1; 2: The model includes the ﬁxed effect con for the four conditions. This effect is nested in the ﬁxed effect, cat, which has two levels: m ¼ 1 includes the three conditions 1–3, while m ¼ 2 includes the fourth condition. The two interaction terms ðgrp catÞim ; m ¼ 1; 2 correspond to the pooled ICC and that of condition 4. Each of these interactions is random, with a mean of zero and a variance of s2g;m : In contrast to the previous models, where the errors were assumed to be independent and normally distributed, with a mean of zero and a constant variance of s2e ; Eq. (21) assumes two error variances: VarðeiðjÞ; tðmÞÞ ¼ s2em ; m ¼ 1; 2: The respective within-group variances are s2wm ¼ s2ind þ s2em : We want to test the equality of the two ICCs: ðICCÞm ¼

s2g þ s2g;m s2g þ s2g;m þ s2wm

(22)

In the previous comparisons, the ICCs associated with the different conditions could differ only due to differences in the (group condition) interaction terms. Now, however, the ICCs may differ in both their numerators and their denominators and still be equal. Therefore, the test that compares these two ICCs cannot be reduced to a comparison between two interactions. In the current example, however, the problem is simpliﬁed. As we shall show, there is no statistically significant difference between the numerators of the two ICCs (the test of the hypothesis s2g;1 ¼ s2g;2 yields a large p value), while there is a statistically significant difference in their denominators. When we ﬁt Eq. (21), the estimated variance s^ 2w;1 is 424.26, but for the second category is much smaller: 295.92. (The test of the hypothesis s2w;1 ¼ s2w;2 produces a very small p value.) Applying the deviance test that compares the deviances from Eq. (21) and the model under the null hypothesis s2g;1 ¼ s2g;2 ; we obtain p ¼ 0:654: This result supports the conjecture that s2g;1 ¼ s2g;2 : In summary, we may draw the following conclusions: 1. All four conditions produce a statistically significant group effect, but this effect is relatively small ðICC ¼ 0:19Þ under conditions of low task interdependence at both times and high task interdependence at the initial time.

400

AYALA COHEN AND ETTI DOVEH

2. Perceptions of collective efﬁcacy grew considerably more similar ðICC ¼ 0:33Þ as team members gained experience working together under conditions of high – but not low – task interdependence.

DISCUSSION This section has presented a statistical analysis of a study that examined the role of task interdependence on the emergence of collective efﬁcacy as a shared group-level concept. Based on the suggested methodology for comparing ICCs, the results of this study indicate that the homogeneity of perceptions of collective efﬁcacy increased as team members gained experience working together under conditions of high – but not low – task interdependence. This conclusion, based on a simultaneous comparison of the four ICCs, differed dramatically from the conclusion suggested by a simple-minded comparison of the four ICCs when each one was estimated separately. In the analysis presented here, for three of the four conditions, the assumption of homo-scedasticity for the errors was justiﬁed. For those three cases, testing the equality of the ICCs were equivalent to testing the equality of the interaction error terms. When this assumption was invalid, ICCs could differ in their respective numerators and denominators, yet still be equal. For example, when ðICCÞm ¼ ðs2g þ s2g;m Þ=ðs2g þ s2g;m þ s2wm Þ; as in Eq. (22), then perhaps s2g;1 as2g;2 and s2w;1 as2w;2 ; but ðICCÞ1 ¼ ðICCÞ2 : In the study described here, this problem was simpliﬁed because we could show that the hypothesis s2g;1 ¼ s2g;2 cannot be rejected, so s2w;1 as2w;2 implies that the ICCs cannot be equal. In general, a special methodology should be developed to test the equality of ICCs such as those deﬁned in Eq. (22). Despite its undesired properties, the Wald test is often considered for this purpose. Even this procedure is not easily applicable here, however, as it requires us to derive the standard errors and correlation of the two ICCs. The delta method could be adopted for this derivation, but this possibility is beyond the scope of the current work. In the current study, the focus was on ICC (1,1), usually denoted as ICC (1). In some multi-level studies, ICC (2) is the indicator of interest. ICC (2) indicates whether group means are relatively different from each other (Bliese, 1998). Recall that ICC (2) can be expressed as a function of ICC (1): ICCð2Þ ¼

k½ICCð1Þ 1 þ ðk 1Þ½ICCð1Þ

Significance Tests for Differences between Dependent Intraclass

401

Thus, testing the equality of ICC (2) is actually equivalent to testing the equality of ICC (1). The ﬂexibility of MIXED models enables us to perform significance tests on the other forms of ICC. Finally, note that all the inference methods presented in this chapter are based on inferences on variances. As is true for inferences in structural estimation methods and in factor analysis, the asymptotic results hold only for large samples. Only a few studies aimed at investigating the small-sample properties for inference in MIXED models have been performed thus far. In addition, some modiﬁcations of ML estimation have been suggested to obtain faster convergence of the estimates. However, these analyses concern only the ﬁxed parameters in the MIXED models (Zucker, Lieberman, & Manor, 2000). In the study described here, the sample was not large. With a larger sample that would lead to higher power, we would likely have found additional significant differences among the ICCs.

CONCLUSION Organizational phenomena, particularly levels of analysis, do not necessarily remain stable over time. Individuals within groups may change from being independent of one another to forming homogeneous or heterogeneous groups. It is also possible that homogeneous groups will dissolve over time to become independent individuals. Longitudinal multi-level analysis can capture such changes, thereby enhancing our understanding of group phenomena. Relatively, few longitudinal multi-level studies have been conducted to uncover how group processes evolve from a level of analysis perspective. The assessment of changes in these few studies was based on descriptive rather than formal statistical methods. This chapter presented a formal statistical significance test for the differences in group homogeneity. This methodology extends the current methods of multi-level analysis and should help in understanding the dynamics of organizational processes.

ACKNOWLEDGMENT We thank Tal Katz and Miriam Erez for introducing us to the problem and providing us with their data.

402

AYALA COHEN AND ETTI DOVEH

REFERENCES Bandura, A. (1997). Self-efficacy: The exercise of control. New York: W. H. Freeman & Co. Bartko, J. J. (1976). On various intraclass correlation reliability coefﬁcients. Psychological Bulletin, 83, 762–765. Bliese, P. D. (1998). Group size, ICC values, and group-level correlations: A simulation. Organizational Research Methods, 1, 355–373. Bliese, P. D. (2000). Within group agreement, non-independence and reliability: Implications for data and analysis. In: K. J. Klein & S. W. J. Kozlowski (Eds), Multilevel theory, research and methods in organizations: Foundations, extensions, and new directions (pp. 349–381). San Francisco: Jossey-Bass. Bliese, P. D., & Halverson, R. R. (1998). Group size and measures of group level properties: An examination of eta-squared and ICC values. Journal of Management, 24(2), 157–172. Dansereau, F., Alutto, J. A., & Yammarino, F. J. (1984). Theory testing in organizational behavior: The varient approach. Englewood Cliffs, NJ: Prentice-Hall. Dansereau, F., Yammarino, F. J., & Kohles, J. C. (1999). Multiple levels of analysis from a longitudinal perspective: Some implications for theory building. Academy of Management Review, 24(2), 346–357. Donner, A. (1986). A review of inference procedures for the intraclass correlation coefﬁcient in the one-way random effects model. International Statistical Review, 54, 67–82. Haggard, E. A. (1958). Intraclass correlation and the analysis of variance. New York: Dryden Press. James, L. R. (1982). Aggregation bias in estimates of perceptual agreement. Journal of Applied Psychology, 67, 219–229. James, L. R., Demaree, R. G., & Wolf, G. (1984). Estimating within-groups interrater reliability with and without response bias. Journal of Applied Psychology, 69, 85–98. Jung, D. I., & Sosik, J. J. (2003). Group potency and collective efﬁcacy. Group and Organization Management, 28(3), 366–391. Katz, T. Y., & Erez, M. (in press). When collective- and self-efﬁcacy affect team performance: The moderating role of task interdependence. Small Group Research. Klein, K. J., Bliese, P. D., Kozlowski, S. W. J., et al. (2000). Multilevel analytical techniques: Commonalities, differences, and continuing questions. In: K. J. Klein & S. W. J. Kozlowski (Eds), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 512–553). San Francisco: Jossey-Bass. Klein, K. J., & Kozlowski, S. W. J. (Eds) (2000). Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions. San Francisco: Jossey-Bass. Kozlowski, S. W. J., Gully, S. M., Nason, E. R., & Smith, E. M. (1999). Developing adaptive teams: A theory of compilation and performance across levels and time. In: D. R. Ilgen & E. D. Pulak (Eds), The changing nature of performance: Implications for staffing, motivation and development (pp. 240–292). San Francisco: Jossey-Bass. Littell, R. C., Milliken, G. A., Stroup, W. W., & Wolﬁnger, R. (1996). SAS system for mixed models. Cary, NC: SAS Institute. McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intra-class correlation coefﬁcients. Psychological Methods, 1(1), 30–46. Nichols, D. (1998). SPSS, Inc. Choosing an intraclass correlation coefﬁcient. Retrieved from http://www.utexas.edu/cc/faqs/stat/spss/spss4.html.

Significance Tests for Differences between Dependent Intraclass

403

Saavedra, R., Early, P. C., & Van Dyne, L. (1993). Complex interdependence in task-performing groups. Journal of Applied Psychology, 78(1), 61–72. Searle, S. R., Casella, G., & McCulloch, C. E. (1992). Variance components. New York: Wiley. Self, G. S., & Liang, K. Y. (1987). Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. Journal of the American Statistical Association, 82, 605–610. Shrout, P. E., & Fleiss, J. L. (1979). Intra-class correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. Singer, J. D. (1998). Using SAS proc mixed to ﬁt multilevel models and individual growth models. Journal of Educational and Behavioral Statistics, 24, 323–355. Snijders, T., & Bosker, R. (1999). Multilevel analysis. London: Sage. Verbeke, G., & Molenberghs, G. (Eds) (1997). Linear mixed models in practice: A SAS-oriented approach. New York: Springer. Yaffee, R. A. (1998). SPSS, Inc. Enhancement of reliability analysis: Application of intraclass correlations with SPSS/Windows v.8. Retrieved from http://www.nyu.edu/its/socsci/ Docs/intracls.html. Zucker, D., Lieberman, O., & Manor, O. (2000). Improved small-sample inference in the mixed linear model: Bartlett correction and adjusted likelihood. Journal of the Royal Statistical Society B, 62, 827–838.

APPENDIX This appendix includes the codes of all SAS procedures used in the analysis described in this chapter. The order of the presentation here follows the order in the main text. Example 1 (Equal Group Sizes, ICC40) The data were generated using Eq. (1). The following SAS code was used to generate the data: data a; retain Seed_1 246747; sigma ¼ 1; sigma_a ¼ 2; do grp ¼ 1 to 10; call rannor (Seed_1,a); a ¼ sigma_a*a; do obs ¼ 1 to 15; call rannor (Seed_1,epsilon);

404

AYALA COHEN AND ETTI DOVEH

epsilon ¼ sigma*epsilon; z ¼ a+epsilon; output; end; end; run; The data obtained with this simulation are displayed such that each row corresponds to a ‘‘group’’ that includes 15 members. Obs grp ID1 ID2 ID3 ID4 ID5 ID6 ID7 ID8 1 1 0.08968 -0.73023 -1.20417 0.35627 -1.91655 -1.57565 -0.71382 1.23525 2 2 –3.06100 -4.38360 -3.39400 -3.92607 -2.04502 -4.20994 -1.60973 -1.56045 3 3 –1.28271 -0.22708 -0.14956 0.34937 -2.38245 -0.97863 -2.67926 -0.54901 4 4 2.87823 1.79217 1.96036 1.09509 2.89686 1.50053 2.53906 1.45891 5 5 –0.25081 -1.87867 -2.06639 -3.31802 -0.43392 -3.05161 -1.85367 0.00272 6 6 –2.39177 -0.55574 -3.72145 -3.75917 -2.66872 -1.98958 -4.03275 -3.21674 7 7 –5.31581 -2.24441 -3.67209 -3.59959 -3.20976 -4.27289 -4.08331 -3.14919 8 8 5.53139 3.92981 6.06098 6.06907 3.56440 4.06018 6.56192 2.65637 9 9 3.68976 2.06648 3.85300 1.47316 2.18797 0.38801 2.55766 3.04723 10 10 -1.07224 0.07266 -0.39859 0.33359 0.00746 -0.41335 -0.66929 -1.86896 Obs ID9 ID10 ID11 ID12 ID13 ID14 ID15 1 0.06259 -0.53130 0.44692 0.46493 -0.66998 0.88804 1.58205 2 -3.98722 -2.62355 -2.53165 -3.15014 -2.59481 -4.38357 -1.59771 3 0.96835 0.16614 -0.52950 -2.32484 -0.10282 -0.97190 0.14031 4 2.31691 3.37704 0.79340 1.62762 3.36259 2.24188 1.99053 5 0.00780 -2.01856 -3.33033 -3.43154 -1.77227 -2.03597 -0.53066 6 -2.71247 -4.06046 -3.76086 -3.31806 -2.34105 -2.45435 -3.42824 7 -3.58874 -2.63681 -2.01215 -3.42735 -2.60500 -2.91265 -3.40342 8 3.81613 3.32291 5.69408 4.82287 5.11570 3.44213 3.41475 9 3.87672 3.96609 1.38008 3.23260 3.12966 0.90798 1.60130 10 -0.75789 -0.06527 -0.64755 -0.71689 -0.20326 -0.55837 -2.64868

The GLM program of SAS was run with the following code: title ‘ Example 1 : Equal group size’; proc glm data ¼ a; class grp; model z ¼ grp /solution; run;

Significance Tests for Differences between Dependent Intraclass

405

The MIXED procedure of SAS (Littell et al., 1996), which produces REML estimates of the two variances s2r and s2w ; was run with the following code: proc mixed data ¼ a covtest; class grp; model z ¼ ; random int/subject ¼ grp; run; The output was obtained by adding the option covtest. Note that in the MIXED output notation, the intercept covariance parameter corresponds to the variance denoted by s2r ; while the residual covariance parameter corresponds to that denoted by s2w : When Eq. (1) was ﬁtted, the relevant MIXED output was Fit Statistics -2 Res Log Likelihood

435.5

We ﬁtted the ‘‘empty model’’ to the same data [Eq. (7)] following with the SAS code: proc mixed data ¼ a covtest; model z ¼ /solution ; run; The relevant output was Fit Statistics -2 Res Log Likelihood

639.0

D ¼ 639:0 435:5 ¼ 203:5 is extremely large, so the value of p is practically zero. We obtain the correct conclusion – namely, that the ICC is significant. The estimated ﬁxed effect m^ is the same in both the ‘‘empty model’’ and the full model (which included the random effect): Solution for Fixed Effects Standard Effect Error DF t Value Pr 4t Intercept 0.1958 0.6088 9 0.32 0.7551

Estimate

406

AYALA COHEN AND ETTI DOVEH

Instead of ANOVA Eq. (1), Eq. (6) could be ﬁtted. The following is the relevant SAS code: title2 ‘‘Model with repeated effect (type ¼ CS)’’; proc mixed data ¼ a covtest; class grp; model z ¼ ; repeated /subject ¼ grp type ¼ cs; run; The repeated statement with the option subject ¼ grp enables us to specify a covariance structure between subjects within the same group. The option type ¼ cs speciﬁes that the ﬁtted covariance structure should be compound symmetry, meaning that the same covariance (correlation) holds between measurements of each pair of subjects in the same group, and the variance of the measurement is the same for all subjects. A portion of the output appears here: Cov Parm CS Residual

Covariance Parameter Subject Standard Estimate Grp 3.6520 0.8156

Estimates Error Z Value Pr Z 1.7472 2.09 0.0366 0.09748 8.37 o.0001

Fit Statistics -2 Res Log Likelihood

43

Null Model Likelihood Ratio Test DF 1

Chi-Square 203.50

Example 2 (Equal Group Sizes, ICC ¼ 0) The SAS code follows: data a; retain Seed_1 246747; sigma ¼ 1; sigma_a ¼ 0; do grp ¼ 1 to 10;

Pr4ChiSq o.0001

Significance Tests for Differences between Dependent Intraclass

407

call rannor (Seed_1,a); a ¼ sigma_a*a; do obs ¼ 1 to 15; call rannor (Seed_1,epsilon); epsilon ¼ sigma*epsilon; z ¼ a+epsilon; output; end; end; run; As before, the data are displayed such that each row corresponds to a ‘‘group’’ of 15 members: Equal group sizes, Population ICC ¼ 0 Obs grp ID1 ID2 ID3 ID4 ID5 ID6 ID7 ID8 1 1 0.66243 -0.15748 -0.63142 0.92902 -1.34380 -1.00289 -0.14107 1.80800 2 2 0.23183 -1.09076 -0.10117 -0.63323 1.24781 -0.91710 1.68310 1.73239 3 3 -0.31396 0.74167 0.81919 1.31812 -1.41370 -0.00988 -1.71051 0.41974 4 4 0.87712 -0.20895 -0.04076 -0.90603 0.89575 -0.50058 0.53794 -0.54221 5 5 1.63609 0.00823 -0.17949 -1.43112 1.45298 -1.16471 0.03323 1.88962 6 6 -0.05004 1.78600 -1.37971 -1.41744 -0.32698 0.35216 -1.69102 -0.87501 7 7 -2.35238 0.71902 -0.70866 -0.63616 -0.24633 -1.30946 -1.11988 -0.18576 8 8 1.13173 -0.46985 1.66132 1.66940 -0.83526 -0.33949 2.16226 -1.74330 9 9 1.16446 -0.45882 1.32770 -1.05214 -0.33733 -2.13729 0.03236 0.52193 10 10 -0.47590 0.66900 0.19775 0.92993 0.60380 0.18299 -0.07295 -1.27262 Obs ID9 ID10 ID11 ID12 ID13 ID14 ID15 1 0.63534 0.04146 1.01967 1.03769 -0.09723 1.46079 2.15481 2 -0.69438 0.66929 0.76118 0.14270 0.69803 –1.09073 1.69512 3 1.93710 1.13489 0.43924 -1.35610 0.86593 –0.00315 1.10906 4 0.31580 1.37592 -1.20772 -0.37350 1.36148 0.24077 -0.01058 5 1.89470 -0.13166 -1.44343 -1.54464 0.11463 -0.14907 1.35624 6 -0.37074 -1.71873 -1.41913 -0.97632 0.00069 -0.11262 -1.08651 7 -0.62531 0.32662 0.95128 -0.46392 0.35843 0.05078 -0.44000 8 -0.58353 -1.07675 1.29441 0.42320 0.71604 -0.95754 -0.98492 9 1.35142 1.44079 -1.14522 0.70730 0.60436 –1.61732 -0.92400 10 -0.16156 0.53106 -0.05121 -0.12055 0.39308 0.03797 -2.05234

408

AYALA COHEN AND ETTI DOVEH

The following results were obtained using the GLM procedure: The GLM Dependent Variable: z Sum of Source DF Squares Model 9 5.2809208 Error 140 114.1835763

Procedure

Mean Square 0.5867690 0.8155970

F Value 0.72

Pr4F 0.6904

As expected, the large value of p indicates that there is no group effect. When we apply Eq. (3) to estimate the ICC, we obtain a negative value. With the MIXED procedure, using the option random int/subject ¼ grp, we obtain the following results:

Cov Parm Intercept Residual

Covariance Parameter Estimates Subject Standard Estimate Error Grp 0 . 0.8018 0.09289

Z Value . 8.63

Pr Z . 0.000

Fit Statistics 2 Res Log Likelihood

394.9

With the repeated option, we obtain the following results: Covariance Parameter Cov Parm Subject Standard Estimate CS grp 0.01526 Residual 0.8156

DF 1

Estimates Error Z Value Pr 0.01955 0.78 0.4353 0.09748 8.37 o.0001

Fit Statistics 2 Res Log Likelihood 394.5 Null Model Likelihood Ratio Test Chi-Square Pr4ChiSq 0.42 0.5185

The algorithm for ﬁtting Eq. (6) can also be applied with the random intercept rather than the repeated option, when the option nobound is added to the code. This option removes the restriction that s2r X0: In other words, these two SAS codes produce the same output:

Significance Tests for Differences between Dependent Intraclass

Title1 ‘‘Model with random intercept’’; proc mixed data ¼ a covtest nobound; class grp; model z ¼ ; random int/subject ¼ grp; run; title2 ‘‘Model with repeated effect (type ¼ CS)’’; proc mixed data ¼ a covtest; class grp; model z ¼ ; repeated /subject ¼ grp type ¼ cs; run;

Example 3 (Unequal Group Sizes, ICC40) The following SAS code was used to generate the data: data a; retain Seed_1 246747; sigma ¼ 1; sigma_a ¼ 2; do grp ¼ 1 to 10; call rannor (Seed_1,a); a ¼ sigma_a*a; do obs ¼ 1 to 4+grp*2; call rannor (Seed_1,epsilon); epsilon ¼ sigma*epsilon; z ¼ a+epsilon; output; end; end; run;

409

410

AYALA COHEN AND ETTI DOVEH

The output of the MIXED procedure with the RANDOM option is as follows: Covariance Parameter Estimates Cov Parm Subject Standard Estimate Error Z Value Pr Z Intercept Grp 6.1226 2.9174 2.10 0.0179 Residual 1.0161 0.1214 8.37 o.0001

Example 4 (ICC ¼ 0, No Change Over Time) The following SAS code was used to generate the data and to ﬁt Eq. (8): *¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼; * Example 4: 2 times, no change ; *¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼; title1 ‘‘Example 4: Comparing 2 Times (both ICC ¼ 0)’’; options nodate; proc delete data ¼ a;run; data a; retain Seed_1 8246747; sigma_e ¼ 2; *std of epsilon; sigma_s ¼ sqrt(3); *std of subject effect; sigma_g ¼ 0; *std of group effect at all timescreates correlation between subject i at time 1 to subject j at time 2; sigma_g1 ¼ 0; *std of group effect at time 1; Sigma_g2 ¼ 0; *std of group effect at time 2; time_1 ¼ 4; *1st time fixed effect; time_2 ¼ 6; *2nd time fixed effect; n_groups ¼ 30; *number of groups; k_in_grp ¼ 3; *number of subjects at each group; do grp ¼ 1 to n_groups; call rannor (Seed_1,a_g); call rannor (Seed_1,a_gt1); call rannor (Seed_1,a_gt2); rand_grp ¼ sigma_g*a_g; rand_grp_t1 ¼ sigma_g1*a_gt1; rand_grp_t2 ¼ sigma_g2*a_gt2;

Significance Tests for Differences between Dependent Intraclass

411

do id ¼ 1 to k_in_grp; call rannor (Seed_1,e1); e1 ¼ sigma_e*e1; *random error for subject no at time 1; call rannor (Seed_1,e2); e2 ¼ sigma_e*e2; *random error for subject no at time 2; call rannor (Seed_1,e_s); s ¼ sigma_s*e_s; *random subject effect for subject no; time ¼ 1; z ¼ time_1+s+ rand_grp +rand_grp_t1+e1; output; time ¼ 2; z ¼ time_2+s+ rand_grp +rand_grp_t2+e2; output; end; end; run; proc sort data ¼ a; by grp time id;run; options ls ¼ 75 ps ¼ 60; title2 ‘‘1st 15 obs’’; run; ods output FitStatistics ¼ FuRes; *Keep -2 Res Log Likelihood of 1st model; title2 ‘‘Full model, time 1+2 together’’; proc mixed data ¼ a covtest asycov nobound; class grp time id; model z ¼ time /solution ddfm ¼ sat; random int /subject ¼ id(grp); random int /subject ¼ grp vcorr ¼ 1 v; random int /subject ¼ time*grp group ¼ time vcorr ¼ 1 v ¼ 1; run;

412

AYALA COHEN AND ETTI DOVEH

The correlation matrix estimated for these data is as follows:

Row 1 2 3 4 5 6

Col1 1.0000 0.005157 0.005157 0.3364 0.07539 0.07539

Estimated V Correlation Matrix for grp 1 Col2 Col3 Col4 Col5 0.005157 0.005157 0.3364 0.07539 1.0000 0.005157 0.07539 0.3364 0.005157 1.0000 0.07539 0.07539 0.07539 0.07539 1.0000 0.04004 0.3364 0.07539 0.04004 1.0000 0.07539 0.3364 0.04004 0.04004

Col6 0.07539 0.07539 0.3364 0.04004 0.04004 1.0000

The estimated correlation matrix consists of 3 3 blocks. Both of the diagonal blocks display the estimated ICCs. The upper-left block corresponds to the initial time, and the lower-right block to time 2. Accordingly, t ¼ 1: estimated ICC ¼ 0.005157 t ¼ 2: estimated ICC ¼ 0.04004 The output also includes the following: Fit Statistics 2 Res Log Likelihood

824.1

The following SAS code was used for ﬁtting the model under the null hypothesis that the ICCs at both times are equal [Eq. (15)]: ods output FitStatistics ¼ SmRes; *Keep -2 Res Log Likelihood of 2nd model; title2 ‘‘Small model, time 1+2,(ICC time1) ¼ (ICC time2)’’ ; proc mixed data ¼ a covtest asycov nobound; class grp time id ; model z ¼ time /solution ddfm ¼ sat; random int /subject ¼ id(grp) ; random int /subject ¼ grp vcorr ¼ 1 v ; random int /subject ¼ time*grp vcorr ¼ 1 v ¼ 1; run; The output includes the correlation matrix under the null hypothesis. In contrast to the previous output, here the two correlation matrices (3 3 blocks) coincide. Both diagonal blocks display the estimated common ICCs ¼ 0.02291.

Significance Tests for Differences between Dependent Intraclass

Row 1 2 3 4 5 6

Col1 1.0000 0.02291 0.02291 0.3364 0.07538 0.07538

Estimated Correlation Matrix for grp 1 Col2 Col3 Col4 Col5 0.02291 0.02291 0.3364 0.07538 1.0000 0.02291 0.07538 0.3364 0.02291 1.0000 0.07538 0.07538 0.07538 0.07538 1.0000 0.02291 0.3364 0.07538 0.02291 1.0000 0.07538 0.3364 0.02291 0.02291

413

Col6 0.07538 0.07538 0.3364 0.02291 0.02291 1.0000

The output also includes the following: Fit Statistics -2 Res Log Likelihood

824.2

The next SAS code provides the p value of the deviance test for the null hypothesis that ICC at time 1 equals the ICC at time 2: title2 ‘‘Calculate Pvalue—(ICC time1) ¼ (ICC time2)’’; data Pv; merge Fures(rename ¼ (value ¼ Fu)) Smres(rename ¼ (value ¼ Sm)) ; by Descr; where Descr eq ‘‘-2 Res Log Likelihood’’; Pval ¼ 1-probchi(Sm-Fu,1); run; proc print; run; According to the following output, the null hypothesis is not rejected:

Obs 1

Example 4: Comparing 2 Times (both ICC ¼ 0) Calculate Pvalue - (ICC time1) ¼ (ICC time2) Descr Fu Sm Pval -2 Res Log Likelihood 824.1 824.2 0.75519

The following SAS code was used for ﬁtting the model under the null hypothesis that the ICCs at both times are equal to zero [Eq. (16)]: ods output FitStatistics ¼ SmRes0; *Keep -2 Res Log Likelihood of 2nd model; title2 ‘‘Smaller model, time 1+2,(ICC time1) ¼ (ICC time2) ¼ 0’’;

414

AYALA COHEN AND ETTI DOVEH

proc mixed data ¼ a covtest asycov nobound; class grp time id; model z ¼ time /solution ddfm ¼ sat; random int /subject ¼ id(grp) vcorr ¼ 1 v ¼ 1 ; run; The next SAS code provides the p value of the deviance test for the null hypothesis that both ICCs are equal to zero: title2 ‘‘Calculate Pvalue - (ICC time1) ¼ (ICC time2) vc all ¼ 0’’; data Pv; merge Smres(rename ¼ (value ¼ Sm)) Smres0(rename ¼ (value ¼ Sm0)) ; by Descr; where Descr eq ‘‘-2 Res Log Likelihood’’; Pval ¼ 1-probchi(Sm0-Sm,1); run; proc print; run; According to the following output, the null hypothesis is not rejected: Example 4: Comparing 2 Times (both ICC ¼ 0) Calculate Pvalue—(ICC time1) ¼ (ICC time2) vc all ¼ 0 Obs Descr Sm Sm0 Pval 1 -2 Res Log Likelihood 824.2 825.2 0.30767

Example 5 (ICC ¼ 0 to ICC40) The following SAS code was used to generate the data: *Example 5: 2 times, increase in icc ; *¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼; *¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼ ¼; title1 ‘‘Example 5: Comparing 2 Times (Increase in ICC)’’; options nodate; proc delete data ¼ a;run; data a; retain Seed_1 8246747;

Significance Tests for Differences between Dependent Intraclass

415

sigma_e ¼ 2; *std of epsilon; sigma_s ¼ sqrt(3); *std of subject effect; sigma_g ¼ 0; *std of group effect at all timescreates correlation between subject i at time 1 to subject j at time 2; sigma_g1 ¼ 0; *std of group effect at time 1; sigma_g2 ¼ 2; *std of group effect at time 2; time_1 ¼ 4; *1st time fixed effect; time_2 ¼ 6; *2nd time fixed effect; n_groups ¼ 30; *number of groups; k_in_grp ¼ 3; *number of subjects at each group; do grp ¼ 1 to n_groups; call rannor (Seed_1,a_g); call rannor (Seed_1,a_gt1); call rannor (Seed_1,a_gt2); rand_grp ¼ sigma_g*a_g; rand_grp_t1 ¼ sigma_g1*a_gt1; rand_grp_t2 ¼ sigma_g2*a_gt2; do id ¼ 1 to k_in_grp; call rannor (Seed_1,e1); e1 ¼ sigma_e*e1; *random error for subject no at time 1; call rannor (Seed_1,e2); e2 ¼ sigma_e*e2; *random error for subject no at time 2; call rannor (Seed_1,e_s); s ¼ sigma_s*e_s; *random subject effect for subject no; time ¼ 1; z ¼ time_1+s+ rand_grp +rand_grp_t1+e1; output; time ¼ 2; z ¼ time_2+s+ rand_grp +rand_grp_t2+e2; output; end; end; run; proc sort data ¼ a; by grp time id;run; options ls ¼ 75 ps ¼ 60;

416

AYALA COHEN AND ETTI DOVEH

title2 ‘‘1st 15 obs’’; proc print data ¼ a (obs ¼ 15); var grp time id z time_1 time_2 s rand_grp rand_grp_t1 rand_grp_t2 e1 e2; run;

SAS for the Motivating Example The following two SAS procedures were applied for each of the four datasets: proc glm data ¼ efficacy; class grp; model y ¼ grp; run; Proc mixed data ¼ efficacy covtest; class grp; model y ¼ ; repeated /subject ¼ grp type ¼ cs rcorr; run; The ﬁt of Eq. (8) was performed with the following SAS code: title ‘‘2 times together - low task interdependence’’; data efficacy_2; set efficacy; y ¼ L_efficacy_t1; time ¼ 1;output; y ¼ L_efficacy_t2; time ¼ 2;output; run; proc sort data ¼ efficacy_2; by grp time subj;run; title ‘‘mixed for 2 times together low task interdependence’’; proc mixed data ¼ efficacy_2 covtest asycov; class grp time id; model y ¼ time; random int /subject ¼ id(grp); random int /subject ¼ grp; random int /subject ¼ time*grp group ¼ time vcorr ¼ 1; run;

Significance Tests for Differences between Dependent Intraclass

417

Following are relevant portions of the MIXED output: 1 2 3 4 5 6

1.0000 0.1138 0.1138 0.3742 0.04853 0.04853

Estimated Correlation Matrix 0.1138 0.1138 0.3742 1.0000 0.1138 0.04853 0.1138 1.0000 0.04853 0.04853 0.04853 1.0000 0.3742 0.04853 0.1452 0.04853 0.3742 0.1452

0.04853 0.3742 0.04853 0.1452 1.0000 0.1452

0.04853 0.04853 0.3742 0.1452 0.1452 1.0000

The estimated correlation matrix consists of 3 3 blocks. Both diagonal blocks display the estimated ICCs. The upper-left block corresponds to the initial time, and the lower-right block to time 2. Accordingly, Low task interdependence, time ¼ initial: estimated ICC ¼ 0.1138. Low task interdependence, time ¼ subsequent: estimated ICC ¼ 0.1452.

Cov Parm Intercept Intercept Intercept Intercept Residual

Covariance Parameter Estimates Subject Standard Error Z Value Group Estimate subj(grp) 127.36 41.2710 3.09 grp 18.9797 31.2521 0.61 grp*time 24.7398 35.2804 0.70 time 1 grp*time 38.8378 36.9005 1.05 time 2 213.01 34.0831 6.25

Pr Z 0.0010 0.271 0.242 0.146 o.0001

According to the preceding output: s^ 2ind ¼ 127:36 s^ 2gt ¼ 24:740;

s^ 2g ¼ 18:980

t ¼ 1; and 38:838;

for t ¼ 2

s^ 2e ¼ 213:01 Asymptotic Covariance Matrix of Estimates Row Cov Parm CovP1 CovP2 CovP3 CovP4 CovP5 1 Intercept 1703.30 -565.27 180.20 175.23 -584.49 2 Intercept -565.27 976.69 -362.47 -309.44 198.1 3 Intercept 180.20 -362.47 1244.71 87.0337 -394.97 4 Intercept 175.23 -309.44 87.0337 1361.65 -385.64 5 Residual -584.49 198.13 -394.97 -385.64 1161.66

418

AYALA COHEN AND ETTI DOVEH

The SAS code for ﬁtting the model for the deviance test under the null hypothesis is as follows: title ‘‘mixed restricted model for 2 times together, low task interdependence’’; proc mixed data ¼ efficacy_2 covtest asycov; class grp time id; model y ¼ time; random int /subject ¼ id(grp) ; random int /subject ¼ grp ; random int /subject ¼ time*grp vcorr ¼ 1; run; This code differs from the code of the full model by just one statement; that is, it includes random int /subject ¼ time*grp vcorr ¼ 1; instead of random int /subject ¼ time*grp group ¼ time vcorr ¼ 1; The relevant portion of the MIXED output for the restricted model follows:

Row 1 2 3 4 5 6

Col1 1.0000 0.1299 0.1299 0.3740 0.04839 0.04839

Estimated Correlation Matrix Col2 Col3 Col4 0.1299 0.1299 0.3740 1.0000 0.1299 0.04839 0.1299 1.0000 0.04839 0.04839 0.04839 1.0000 0.3740 0.04839 0.1299 0.04839 0.3740 0.1299

Col5 0.04839 0.3740 0.04839 0.1299 1.0000 0.1299

Col6 0.04839 0.04839 0.3740 0.1299 0.1299 1.0000

The estimated correlation matrix consists again of 3 3 blocks. However, in the restricted model, unlike in the full model, the two diagonal blocks are identical and both display the estimated ICC common to the two times. Accordingly, the pooled estimated ICC equals 0.1299. The next step is the significance test for the common estimated ICC, which according to the ﬁtted Eq. (15) equals 0.1299. The SAS code for ﬁtting this model is as follows:

Significance Tests for Differences between Dependent Intraclass

419

title ‘‘low task, time 1+2 together - ICC time1 ¼ time2 ICC ¼ 0’’; proc mixed data ¼ a covtest asycov nobound; class grp time id ; model y ¼ time /solution ddfm ¼ sat; random int /subject ¼ id(grp) ; run; Under the null hypothesis, D is asymptotically distributed as w2 ð2Þ; because Eq. (16) has two fewer random components than Eq. (15). pval ¼ 1-probchi(2061.1-2057.5 ,2) ¼ 0.165 The p value indicates that there is no group effect under conditions of low task interdependence at either time. In the ﬁnal stage of the analysis, we compare the ICC of the fourth condition (high task interdependence, subsequent time) with the pooled ICC of the other three conditions. We use the following notation for the SAS code: task ¼ low time ¼ initial con ¼ 1 cat ¼ 1 task ¼ low time ¼ subsequent con ¼ 2 cat ¼ 1 task ¼ high time ¼ initial con ¼ 3 cat ¼ 1 task ¼ high time ¼ subsequent con ¼ 4 cat ¼ 2 The following SAS code was applied to ﬁt Eq. (21): title ‘‘mixed for 3 cases vs 4th together - full model’’; proc mixed data ¼ A covtest asycov nobound; class grp task no con cat; model y ¼ con /solution ddfm ¼ sat; random int /subject ¼ no(grp) ; random int /subject ¼ grp vcorr ¼ 1 v ; random int /subject ¼ con*grp group ¼ cat vcorr ¼ 1 v ¼ 1; repeated /type ¼ vc subject ¼ grp group ¼ cat r; run; The repeated statement with type ¼ vc and the group ¼ cat option speciﬁes a heterogeneous variance model (different variances for each ‘‘cat’’). The output included the following estimated 12 12 covariance matrix. The matrix consists of 3 3 blocks corresponding to the structure of three team members in each group.

420

AYALA COHEN AND ETTI DOVEH

Estimated V Matrix for grp Row Col1 Col2 Col3 Col4 Col5 Col6 Col7 1 424.26 80.4839 80.4839 92.2328 -1.9318 -1.9318 92.2328 2 80.4839 424.26 80.4839 -1.9318 92.2328 -1.9318 -1.9318 3 80.4839 80.4839 424.26 -1.9318 -1.9318 92.2328 -1.9318 4 92.2328 -1.9318 -1.9318 424.26 80.4839 80.4839 92.2328 5 -1.9318 92.2328 -1.9318 80.4839 424.26 80.4839 -1.9318 6 -1.9318 -1.9318 92.2328 80.4839 80.4839 424.26 -1.9318 7 92.2328 -1.9318 -1.9318 92.2328 -1.9318 -1.9318 424.26 8 -1.9318 92.2328 -1.9318 -1.9318 92.2328 -1.9318 80.4839 9 -1.9318 -1.9318 92.2328 -1.9318 -1.9318 92.2328 80.4839 10 92.2328 -1.9318 -1.9318 92.2328 -1.9318 -1.9318 92.2328 11 -1.9318 92.2328 -1.9318 -1.9318 92.2328 -1.9318 -1.9318 12 -1.9318 -1.9318 92.2328 -1.9318 -1.9318 92.2328 -1.9318 Row 1 2 3 4 5 6 7 8 9 10 11 12

Col8 -1.9318 92.2328 -1.9318 -1.9318 92.2328 -1.9318 80.4839 424.26 80.4839 -1.9318 92.2328 -1.9318

Col9 -1.9318 -1.9318 92.2328 -1.9318 -1.9318 92.2328 80.4839 80.4839 424.26 -1.9318 -1.9318 92.2328

Col10 92.2328 -1.9318 -1.9318 92.2328 -1.9318 -1.9318 92.2328 -1.9318 -1.9318 295.92 97.7259 97.7259

Col11 -1.9318 92.2328 -1.9318 -1.9318 92.2328 -1.9318 -1.9318 92.2328 -1.9318 97.7259 295.92 97.7259

Col12 -1.9318 -1.9318 92.2328 -1.9318 -1.9318 92.2328 -1.9318 -1.9318 92.2328 97.7259 97.7259 295.92

The output shows that the estimated variance for the ﬁrst category was 424.26, while that for the second category was much smaller – 295.92. The SAS code used to ﬁt the ‘‘restricted’’ model, assuming s2g;1 ¼ s2g;2 ; differs from the previous code by only one statement. In the line random int /subject ¼ con*grp group ¼ cat vcorr ¼ 1 v ¼ 1 group ¼ cat should be omitted. Finally, using the model that justiﬁably assumes s2g;1 ¼ s2g;2 ; we used the following SAS code to ﬁt a ‘‘restricted’’ model to test the hypothesis s2w;1 ¼ s2w;2 : This code differs from the previous code in one respect: group ¼ cat should be omitted from the line repeated /type ¼ vc subject ¼ grp group ¼ cat r.

INTERPRETING CHANGES IN ICCs: TO AGREE OR NOT TO AGREE, THAT IS THE QUESTION Paul J. Hanges and Julie S. Lyon ABSTRACT In this chapter, we discuss Cohen and Doveh’s (this volume) proposed protocol for testing differences in intra-class correlation coefficients (ICCs). We believe that there are many research questions that can be addressed by this procedure. We provide several potential examples of using this procedure at the individual, group, and organizational/society levels of analysis. We do, however, raise concerns about interpreting the ICC as an index of within-group homogeneity.

INTRODUCTION In this chapter, we review and comment on Cohen and Doveh’s (this volume) proposed protocol for testing changes in the degree of homogeneity within a group over time. The Cohen and Doveh protocol uses the intraclass correlation coefﬁcient (ICC) as an index of homogeneity, and their protocol enables researchers to make inferences regarding changes in homogeneity over time. Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 421–431 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04018-X

421

422

PAUL J. HANGES AND JULIE S. LYON

The ICC is a widely used statistic in the applied sciences. In particular, it is increasingly used by multi-level researchers. Basically, the ICC is an estimate of the association between some categorical/nominal variable (e.g., dyad, group, organization, society) and some continuous variable (e.g., culture, leadership, group potency). While many different ICC statistics have been developed (e.g., Shrout & Fleiss, 1979), Cohen and Doveh focus on the intra-class correlation coefﬁcient denoted as ICC(1) or ICC(1,1), although they note that their protocol can be extended to other ICCs as well. The magnitude of all ICCs reﬂects a ‘‘signal’’ to ‘‘noise’’ ratio. The ‘‘signal’’ in this case refers to the differences between the groups whereas the ‘‘noise’’ refers to the variability of the dependent variable within the groups. Holding the differences between groups constant (i.e., holding signal constant), the ICC increases as the amount of noise within groups decreases. Similarly, holding the magnitude within group variability constant (i.e., holding noise constant), the ICC will increase as the magnitude of the differences between groups increases. ICCs can be interpreted in many different ways. In generalizability theory (Shavelson & Webb, 1991), ICC is used and interpreted as an index of reliability. That is, the ICC indicates the extent to which the various categories (e.g., groups, organizations) can be differentiated on the continuous variable (Shavelson & Webb, 1991). This interpretation of the ICC as a reliability coefﬁcient is accurate in terms of the signal-to-noise nature of the ICC statistic. Reliability of measurement can be improved by reducing noise (e.g., increasing the number of scale items, increasing the number of respondents) or by increasing the magnitude difference among the items (e.g., improving experimental design, selecting groups from a more diverse population). ICC has also been interpreted as an index of group homogeneity. This interpretation is frequently implied by the way multi-level researchers use the ICC. Specifically, multi-level researchers use the ICC (along with other statistics such as Z2 and rwg) to assess whether their measures can be meaningfully aggregated to some group level of analysis. This aggregation is usually accomplished by averaging group members’ responses. In order for this average to be a meaningful representation of the group, the measured construct has to be what Kozlowski and Klein (2000) called a convergent construct. Convergent constructs are ones in which group members’ responses center about a single value. With such constructs, the ICC provides a useful index of the extent to which the group members are sufﬁciently homogeneous to justify aggregation. While the magnitude of the ICC and

Interpreting Changes in ICCs

423

other statistics (e.g., E-statistic) necessary to justify aggregation is debatable (Dansereau, Alutto, & Yammarino, 1984; Klein et al., 2000), the use of these statistics to justify aggregation is not (e.g., Bliese, 2000; Klein et al., 2000). Surprisingly, even though the ICC plays an increasingly important role in the applied sciences, only recently there has been attempts to explain how to test hypotheses about this statistic. In the past, researchers only qualitatively assessed whether ICC values differed from one another. It was never clear how to compare the different ICC values. McGraw and Wong (1996) provide procedures for calculating conﬁdence intervals and conducting F-tests on ICC values. Cohen and Doveh have added to these procedures by suggesting an interesting protocol that is based on Restricted Maximum Likelihood (REML) estimation methods. The REML method is increasingly being used in a variety of statistical tools such as structural equation modeling (Muthe´n & Muthe´n, 2004), item response theory (Mislevy & Bock, 1983), and hierarchical or mixed modeling (Raudenbush & Bryk, 2001). The Cohen and Doveh protocol can be run on readily available statistical packages (e.g., SPSS, HLM, SAS). Overall, there are many positive aspects to the protocol developed by Cohen and Doveh. These authors demonstrated their method by using simulated and actual group-level data. We believe, however, that their protocol extends beyond group-level research. In the next section, we will discuss several other possible applications of their approach.

APPLICABILITY OF THE PROTOCOL As indicated in the previous section, we believe that the protocol developed by Cohen and Doveh can be applied to situations other than small group research. Indeed, we believe that this protocol could be applied to study micro-level as well as macro-level issues.

Individual-Level Applications Issues regarding changes in the degree of consistency and reliability of measurement are not limited to group-level phenomena. It is also of interest to scientists typically interested in individual-level phenomena. For example, one issue that repeatedly appears in the Industrial/Organizational Psychology literature concerns the stability/variability of performance over time (Ployhart & Hakel, 1998; Barrett, Caldwell, & Alexander, 1985). Indeed, all

424

PAUL J. HANGES AND JULIE S. LYON

personnel decisions are predicated on the belief that long-term performance can be predicted. Thus, the amount of variability in performance has direct implications for the accuracy of all personnel decisions. In the early years of applied psychology, performance models were based on classical test theory in which it was assumed that individual-level performance is stable. However, this performance stability assumption was soon questioned (Ghiselli, 1956) and by 1968, Wernimont and Campbell were so convinced that individual performance lacked stability and they warned that it was ‘‘dangerous to apply the classic validation model and attempt to generalize from a one-time criterion measure to an appreciable time span of job behavior’’ (p. 374). Although such an extreme view of performance instability is no longer believed, this issue is not resolved and regularly reemerges in the applied literature (e.g., Guion & Gibson, 1988; Hanges & Gettman, 2004; Hanges, Schneider, & Niles, 1990; Hofmann, Jacobs, & Baratta, 1993; Hofmann, Jacobs, & Gerras, 1992). Unfortunately, little research exists on interventions that will maximize or minimize stability of individual performance over time. The Cohen and Doveh protocol is useful for assessing this issue. The effectiveness of different interventions (e.g., goal setting, training, feedback, selection) for increasing or decreasing individual stability can be assessed by examining the ICC changes under these various interventions.

Group-Level Applications Cohen and Doveh discussed one longitudinal application of their protocol with group research. Clearly, however, the protocol could be used to address other group-level questions. For example, the effectiveness of interventions designed to help groups build a team-orientation could be tested with the Cohen and Doveh protocol. Also, factors that increase the degree to which team members share mental models (e.g., training) could be explored with this technique (Klimoski & Mohammed, 1994). Research could also examine the impact of different types of interdependence (e.g., pooled, sequential, reciprocal, intensive, resource, outcome, structural) on collective efﬁcacy or other outcomes (Tesluk, Mathieu, Zaccaro, & Marks, 1997; Wageman, 2001). Finally, the literature on virtual teams clearly suggests that certain conditions (e.g., relationship building exercises; face-to-face meetings; realistic expectations about the difﬁculty of working virtually) might increase the effectiveness of such teams (Hanges, Lyon, & Dorfman, in press; Warkentin & Beranek, 1999; Maznevski & Chudoba, 2000). Some virtual

Interpreting Changes in ICCs

425

teams perform well despite interpersonal conﬂicts and low trust (Aubert & Kelsey, 2003). Agreement on team attitudinal dimensions could be indicative of the team’s viability, or the extent to which they are willing to work together in the future (Hackman, 1987; Kozlowski & Bell, 2003). Those teams that fall into a ‘‘danger zone’’ with a change (especially a dichotomizing) of attitudes could also be targeted for some intervention. These practical recommendations can also be tested using this procedure. Clearly, the Cohen and Doveh protocol can be widely applied to a number of different situations in group-level research.

Organizational- and Societal-Level Applications Finally, there are a number of ways that this technique can be applied to organizational- and societal-level research. For example, Gelfand (2000) and Gelfand, Raver, and Lim (2004) hypothesized that societies can be scaled on a tightness–looseness dimension. In contrast to loose societies, a tight society is one that has multiple, strong norms and one in which there is little tolerance for deviance from these norms. Thus, one question that could be addressed is the extent to which the inﬂuence of organizational-level differences (e.g., organizational culture) on some phenomena of interest (e.g., employee morale, organizational productivity) increases or decreases depending upon the tightness or looseness of a society. We would predict that the organizational-level ICCs from tight societies would be smaller than the organizational-level ICCs from loose societies. Another possible application of Cohen and Doveh’s protocol comes from the Global Leadership and Organizational Behavioral Effectiveness (GLOBE) research project (House, Hanges, Javidan, Dorfman, & Gupta, 2004). The GLOBE project is a multi-phase, multi-national study seeking to understand the complex relationship between leadership, organizational culture, and societal culture. These authors found that while the broader culture of the society inﬂuences the type of organizational culture within that society, the magnitude of this inﬂuence significantly varied as a function of organizational industry (Brodbeck, Hanges, Dickson, Gupta, & Dorfman, 2004). In particular, societal culture inﬂuenced organizational culture in organizations from the food services industry. However, organizational culture was relatively immune from the inﬂuence of societal culture in organizations from the ﬁnancial services. Once again, Cohen and Doveh’s protocol would provide a direct test of the magnitude of the societal effect (i.e., societal ICCs) across these different industries.

426

PAUL J. HANGES AND JULIE S. LYON

Overall, our discussion suggests that the Cohen and Doveh protocol has a widespread applicability. We believe that the authors have made a substantial contribution to the ﬁeld with their detailed discussion of their protocol. However, even though we believe that the Cohen and Doveh procedure is useful, we do have one major caveat regarding the procedure. We will discuss this issue in the next section.

PROTOCOL CAVEAT Many different questions remain about the protocol. For example, what is the statistical power of this technique? How many individuals within groups and how many groups are needed to detect changes in group homogeneity? What happens to the statistical power of the technique when the assumptions of the REML estimation procedure are violated? Overall, there are many questions to explore in future studies. We will leave these more specific questions for future exploration. Instead, in the remainder of this chapter, we want to focus on an important conceptual issue. In particular, one issue of particular concern is the interpretation of the ICC as an index of homogeneity. At best, ICCs are ambiguous measures of group cohesiveness. To understand why this is true, it is important to review the distinction between within-group agreement and reliability. Bliese (2000) clariﬁed the distinctions among within-group agreement and reliability. Within-group agreement is demonstrated when group members are more similar to each other than expected by chance. Cohen and Doveh use the term homogeneity and we consider this to be equivalent to within-group agreement. Within-group agreement only increases as the extent to which group members’ responses converge (i.e., become more homogeneous). One measure of within-group agreement is rwg (James, Demaree, & Wolf, 1984; James et al., 1993). Reliability, on the other hand, refers to consistency of group members’ responses (Kozlowski & Hattrup, 1992). As indicated earlier, reliability coefﬁcients reﬂect the extent to which a measure has sufﬁcient amount of signal to overcome the noise in the measure. ICCs are signal-tonoise ratios and thus, are reliability coefﬁcients. Reliability coefﬁcients can increase by increasing within-group homogeneity or by increasing betweengroup differences. The ambiguous interpretation of the ICC as an index of homogeneity is precisely because ICCs can increase by either increases in within-group homogeneity or by increases in between-group differences. Unless additional analyses are conducted (i.e., explicitly testing that the group means have not changed over time or condition), the interpretation of

Interpreting Changes in ICCs

427

a change in an ICC as an indication of improved group homogeneity is inappropriate. In their empirical example, Cohen and Doveh illustrated their technique by exploring changes in the homogeneity of collective efﬁcacy over time. Interestingly, Bliese (2000) specifically discussed collective efﬁcacy and strongly suggested that the aggregatability of such constructs needs to be demonstrated using statistics explicitly designed to assess within-group agreement (e.g., rwg) and to assess reliability (using the appropriate ICC statistic – in this case, Bliese (2000) suggests that the ICC(2) statistic be used because it is important to obtain an estimate of group-mean reliability.). In the Cohen and Doveh’s example, ad hoc teams were only expected to experience collective efﬁcacy after working together over time on an interdependent task. Cohen and Doveh interpreted the significant changes in the ICCs as a reﬂection of increases in group homogeneity with regard to the collective efﬁcacy construct. However, it is also likely that the within group agreement of the various groups could have remained the same or that increased tensions caused factions among the group members. Thus, a viable alternative explanation could be that the observed changes in the ICC are attributable to the groups becoming increasingly different in their level of collective efﬁcacy as opposed to becoming more cohesive and homogeneous. To illustrate this point, we conducted two Monte-Carlo simulations. In both of these simulations, the basic scenario was that 15 employees from 10 different organizations provide a ﬁrst assessment of some construct at Time 1 and then after some organizational intervention, they provide a second assessment of the same construct. (Even though our simulation was designed with equal numbers of employees within organizations, the same result will be obtained when an unequal number of employees are sampled across organizations.) We simulated this scenario by randomly selecting observations from a normal distribution for each of the 10 organizations and then randomly selecting observations from a normal distribution for the 15 employees from each organization. We estimated the variance components at the between- and within-organization level of analysis for each time period by specifying the REML estimation procedure in SPSS’s (Version 12.0.0) variance components analysis subroutine. For each simulation, we replicated our dataset 100 times. In our ﬁrst simulation, we simulated the case in which the organizational intervention causes between-organizations differences to increase but does not affect the within-organization agreement among the employees. Table 1 shows the results of this simulation separated by time. Consistent with the

428

PAUL J. HANGES AND JULIE S. LYON

Table 1. Simulation 1 Results: Intervention Increases Organizational Differences but Left Employee Level Variance Unchanged across Time.

Within organizational variance Between organizational variance ICC

Time 1 Average

Time 2 Average

3.95 0.01 0.01

4.05 3.97 0.46

Table 2. Simulation 2 Results: Intervention Increased Organizational Differences and Polarized Employees within Organizations across Time.

Within organizational variance Between organizational variance ICC

Time 1 Average

Time 2 Average

1.00 0.01 0.01

2.03 3.54 0.60

structure of the simulation, this table shows that the within-organizational variance remained the same over the two time periods while the betweenorganizational variance increased over time. More importantly, comparing the ICCs for the two time periods shows that despite the fact that homogeneity within group did not change over time, the ICC at Time 2 (0.46) was larger than the ICC at Time 1 (0.01). In the second simulation, we simulated the case in which the organizational intervention increased both the between- and within-organizational variances. In other words, the intervention polarized the employees within each organization. More specifically, this polarization was simulated by creating three groups of employees for each organization at Time 2. Seven of the employees were one standard deviation below the organization’s mean, seven were one standard deviation above the mean, and only one employee was centered around the organization’s mean. Table 2 shows the results of this simulation separated by time. Consistent with the structure of the simulation, this table shows that both the between- and within-organizational variances increased over time. The important result, however, is that despite the fact that homogeneity within the organization actually decreased over time, the ICC at Time 2 (0.60) was still larger than the ICC at Time 1 (0.01). Both of these simulations demonstrate that the ICC can increase regardless of what happens to the homogeneity within-groups. Indeed, in the

Interpreting Changes in ICCs

429

second simulation, the organizational employees actually become polarized (i.e., what the WABA procedure calls a part relationship emerged over time). In summary, it is clear that Cohen and Doveh have created a protocol that provides ambiguous evidence of group homogeneity. While there is nothing wrong statistically with their protocol, it is important to note that conceptually their test of ICC does not convincingly demonstrate that groups working on an interdependent task become more homogenous over time. A true test of homogeneity would explicitly test (using the F ratio) whether the variance of each group changed over time.

CONCLUSIONS In summary, we believe that Cohen and Doveh have made a contribution to applied sciences by developing their protocol. We believe that there are many research areas beyond longitudinal group-level research that could apply this technique. The clearest interpretation of the ICC is one of reliability, however. Changes in group homogeneity can only be cleanly interpreted when additional analyses are conducted.

REFERENCES Aubert, B. A., & Kelsey, B. L. (2003). Further understanding of trust and performance in virtual teams. Small Group Research, 34, 575–618. Barrett, G. V., Caldwell, M. S., & Alexander, R. A. (1985). The concept of dynamic criteria: A critical reanalysis. Personnel Psychology, 38, 41–56. Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In: K. J. Klein & S. W. J. Kozlowski (Eds), Multi-level theory, research, and methods in organizations: Foundations, extensions and new directions (pp. 349–381). San Francisco: Jossey-Bass. Brodbeck, F., Hanges, P. J., Dickson, M. W., Gupta, V., & Dorfman, P. W. (2004). Societal culture and industrial sector inﬂuences on organizational culture. In: R. J. House, P. J. Hanges, M. Javidan, P. W. Dorfman & V. Gupta (Eds), Leadership, culture, and organizations: The GLOBE study of 62 societies (pp. 654–668). Thousand Oaks, CA: Sage Publications. Dansereau, F., Alutto, J., & Yammarino, F. (1984). Theory testing in organizational behavior: The varient approach. Englewood Cliffs, NJ: Prentice Hall. Gelfand, M. J. (2000). Cultural tightness-looseness: A multi-level investigation. National Science Foundation Grant #9910760. Gelfand, M. J., Raver, J., & Lim, B. C. (2004). Culture and accountability in organizations: Variations in forms of social control across cultures. Human Resource Management Review, 14, 135–160.

430

PAUL J. HANGES AND JULIE S. LYON

Ghiselli, E. E. (1956). Dimensional problems of criteria. Journal of Applied Psychology, 40, 1–4. Guion, R. M., & Gibson, W. M. (1988). Personnel selection and placement. Annual Review of Psychology, 39, 349–374. Hackman, J. R. (1987). The design of work teams. In: J. Lorsch (Ed.), Handbook of organizational behavior (pp. 315–342). New York: Prentice Hall. Hanges, P. J., & Gettman, H. (2004). A comparison of test-focused and criterion-focused banding methods: Back to the future? In: H. Aguinis (Ed.), Test score banding in human resource selection: Legal, technical, and societal issues (pp. 29–48). West Port, CT: Praeger. Hanges, P. J., Lyon, J., & Dorfman, P. (in press). Managing an international research team: Lessons from project GLOBE. In: D. Shapiro, M. A. Von Glinow, & J. Cheung (Eds), Managing multinational teams: Cultural, organizational, and national influences. Amsterdam: Elsevier Press. Hanges, P. J., Schneider, B., & Niles, K. (1990). Stability of performance: An interactionist perspective. Journal of Applied Psychology, 75, 658–667. Hofmann, D. A., Jacobs, R., & Baratta, J. E. (1993). Dynamic criteria and the measurement of change. Journal of Applied Psychology, 78, 194–204. Hofmann, D. A., Jacobs, R., & Gerras, J. E. (1992). Mapping individual performance over time. Journal of Applied Psychology, 77, 185–195. House, R. J., Hanges, P. J., Javidan, M., Dorfman, P. W., & Gupta, V. (2004). Leadership, culture, and organizations: The GLOBE study of 62 societies. Thousand Oaks, CA: Sage Publications. James, L. R., Demaree, R. J., & Wolf, G. (1984). Estimating within-group interrater reliability with and without response bias. Journal of Applied Psychology, 69, 85–98. James, L. R., Demaree, R. J., & Wolf, G. (1993). rwg: An assessment of within-group interrater agreement. Journal of Applied Psychology, 78, 306–309. Klein, K. J., Bliese, P. D., Kozlowski, S. W. J., Dansereau, F., Gavin, M. B., Grifﬁn, M. A., Hofmann, D. A., James, L. R., Yammarino, F. J., & Bligh, M. C. (2000). Multi-level analytic techniques: Commonalities, differences, and continuing questions. In: K. J. Klein & S. W. J. Kozlowski (Eds), Multi-level theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 512–556). San Francisco: Jossey-Bass. Klimoski, R., & Mohammed, S. (1994). Team mental model: Construct or metaphor? Journal of Management, 20, 403–437. Kozlowski, S. W. J., & Bell, B. S. (2003). Work groups and teams in organizations. In: W. C. Borman, D. R. Ilgen & R. J. Klimoski (Eds), Handbook of psychology: Industrial and organizational psychology, (Vol. 12, pp. 333–375). New York: Wiley. Kozlowski, S. W. J., & Hattrup, K. (1992). A disagreement about within-group agreement: Disentangling issues of consistency versus consensus. Journal of Applied Psychology, 77, 161–167. Kozlowski, S. W. J., & Klein, K. J. (2000). A multi-level approach to theory and research in organizations: Contextual, temporal, and emergent processes. In: K. J. Klein & S. W. J. Kozlowski (Eds), Multi-level theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 3–90). San Francisco, CA: Jossey-Bass. Maznevski, M., & Chudoba, K. (2000). Bridging space over time: Global virtual team dynamics and effectiveness. Organization Science, 11, 473–492.

Interpreting Changes in ICCs

431

McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefﬁcients. Psychological Methods, 1, 30–46. Mislevy, R.J., & Bock, R.D. (1983). BILOG: Item analysis and test scoring with binary logistic models [computer program]. Mooresville, IN: Scientific Software, Inc. Muthe´n, L. K., & Muthe´n, B. (2004). Mplus user’s guide (3rd ed.). Los Angeles, CA: Muthe´n & Muthe´n. Ployhart, R. E., & Hakel, M. D. (1998). The substantive nature of performance variability: Predicting inter-individual differences in intra-individual performance. Personnel Psychology, 51, 859–902. Raudenbush, S. W., & Bryk, A. S. (2001). Hierarchical linear models: Applications and data analysis methods (Advanced quantitative techniques in the social sciences) (2nd ed.). Thousand Oaks, CA: Sage. Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing reliability. Psychological Bulletin, 86, 420–428. Tesluk, P., Mathieu, J. E., Zaccaro, S. J., & Marks, M. (1997). Task and aggregation issues in the analysis and assessment of team performance. In: M. T. Brannick, E. Salas & C. Prince (Eds), Team performance and assessment and measurement (pp. 197–225). Mahwah, NJ: LEA. Wageman, R. (2001). The meaning of interdependence. In: M. E. Turner (Ed.), Groups at work: Theory and research (pp. 197–218). Mahwah, NJ: Erlbaum. Warkentin, M., & Beranek, P. M. (1999). Training to improve virtual team communication. Information Systems Journal, 9, 271–289.

This page intentionally left blank

432

A MODEL SELECTION APPROACH TO TESTING DEPENDENT ICCs Wolfgang Viechtbauer and David Budescu ABSTRACT In this commentary, we discuss the tutorial by Cohen and Doveh (this volume) on using multi-level models to estimate and test hypotheses about (dependent) intra-class correlations (ICCs). The goal in this context is to find subsets of homogeneous ICCs across time points and/or experimental tasks. We suggest that one should approach this model selection problem by (a) specifying the set of all possible models, (b) using a systematic top-down selection strategy that avoids possible inconsistent and intransitive patterns in the results, and (c) using the entire dataset available when fitting each model. Our goal is to discuss the larger framework in which the specific models considered by Cohen and Doveh in their analysis are embedded, which should be useful in guiding researchers faced with a similar analysis task.

INTRODUCTION Cohen and Doveh (this volume) provide a useful tutorial for using multilevel models to estimate and test hypotheses about (dependent) intra-class correlations (ICCs). Speciﬁcally, the authors show how to specify a model Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 433–454 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04019-1

433

434

WOLFGANG VIECHTBAUER AND DAVID BUDESCU

that allows the ICC to differ over time and/or experimental tasks and then illustrate how hypotheses about changes in the ICC (i.e., in group homogeneity) can be examined by testing the equivalence of certain variance components in the model. Many readers can beneﬁt from their straightforward and easy to read introductions, the (simulated and real) examples, and their careful and detailed description of the computer programs (and their outputs) used in the analyses. In the present commentary, we will focus mostly on the analysis of the ‘‘motivating example.’’ In this study, participants were pre- and posttested under two different experimental tasks, leading to four different experimental conditions. Therefore, if the ICC depends on the experimental condition, then four possible ICCs are at most possible (one for each condition), implying that the ICC depends not only on the experimental task, but also on time. This most general model, however, might be unnecessarily complex. By constraining subsets (or all) of the four ICCs equal to each other, we obtain 14 possible subset models, each of which might provide a more parsimonious description of the data. The problem then is the selection of one of these models. We illustrate how each of these models can be ﬁt using the SAS MIXED procedure and propose a systematic model selection strategy that differs to some extent from the one taken by Cohen and Doveh (this volume). First of all, we choose a top-down approach, starting with the most complex model and then examine the feasibility of reductions in model complexity. We recommend a top-down approach, because we consider it more prudent to err on the side of selecting a model that is too complex, instead of one that is too simple. Second, we advocate using the full dataset available when ﬁtting each model, as opposed to using only that part of the data that is just sufﬁcient for establishing the homogeneity of various ICCs. Third, by outlining (and ﬁtting) all possible models, one can easily see how the various models are nested within each other. Accordingly, one can choose a sequence of model comparisons that avoids inconsistent and intransitive patterns (such as ﬁnding that the ﬁrst and second and the second and third ICC are homogeneous, but not the ﬁrst and third). Finally, given the computational ease with which mixed models can nowadays be ﬁtted, it is quite feasible to simply ﬁt all possible models and select based on, for example, Akaike’s information criterion (AIC) the one that strikes an optimal balance between model ﬁt and complexity (this approach would be consistent with the min(AIC) criterion discussed by Dayton, 2003; Bozdogan, 1987). However, whether one approaches the model selection problem through a series of deviance (i.e., likelihood ratio) tests involving properly nested

A Model Selection Approach to Testing Dependent ICCs

435

models or employs an information criterion such as the AIC is less critical. More importantly, one should clearly deﬁne the set of possible models under consideration, which guides either procedure and facilitates the interpretation of the results. In essence, our purpose in writing this commentary is not to critique the speciﬁc approach chosen by Cohen and Doveh (in fact, their conclusions for the motivating example coincide with those obtained by us), but instead to provide some suggestions and guidelines for researchers faced with a similar analysis task. To be more precise in discussing the various models, we will use the notation of the general linear mixed-effects model (GLMM) in our commentary. We therefore deﬁne the GLMM in the next section and then apply its notation to the one-way random-effects model and the two-factor model with repeated measures on one factor for illustration purposes. Next, the motivating example is introduced in the same manner and the various subset models are deﬁned. The selection of one of these models is then a straightforward matter of applying either a series of deviance tests or employing the min(AIC) criterion. However, before settling on one model, we ﬁrst consider a set of more general models in which the residual variances may also depend on the condition level. In the same manner as before, we again deﬁne and ﬁt all possible models and then select a ﬁnal model. However, model selection with heterogeneous residual variances involves some additional complexities (in particular, for testing the homogeneity of ICCs) which are discussed in the next to last section. Some general comments conclude our commentary.

THE GENERAL LINEAR MIXED-EFFECTS MODEL To discuss the various models presented by Cohen and Doveh (this volume) and in our commentary, it is helpful to write out in explicit detail the models and the implied covariance structures using the notation of the GLMM. The general form of GLMMs (e.g., Searle, Casella, & McCulloch, 1992) is given by y ¼ Xb þ Zc þ e where y is the vector of observations, X the design matrix for the ﬁxed effects parameters contained in b; Z the design matrix for the random effects contained in c; and e the vector of random residuals. We assume E½c ¼ 0; E½e ¼ 0; and Cov½c; e ¼ 0: Deﬁne D as the covariance matrix of the random effects parameters in c and R as the covariance matrix of e. Then

436

WOLFGANG VIECHTBAUER AND DAVID BUDESCU

Var

c e

¼

D

0

0

R

and V, the covariance matrix of y, is equal to V ¼ ZDZ 0 þ R:

One-Way Random-Effects Model As a simple example, consider ﬁrst the one-way random-effects model that is discussed by Cohen and Doveh (this volume) and given by the authors in Eq. (1). For i ¼ 1; . . . ; n groups with j ¼ 1; . . . ; ki members, the model can be written as Y ij ¼ m þ ri þ wij where ri Nð0; s2r Þ; wij Nð0; s2w Þ; and all the terms are independent. Then for n ¼ 2 and k1 ¼ k2 ¼ 2 (the ‘‘smallest’’ case that still illustrates the important features of this model), we can write the model in GLMM notation as shown below: 3 3 2 3 3 2 2 2 Y 11 w11 1 0 " # 1 6 1 0 7 r1 6 Y 12 7 6 1 7 6 w12 7 7 7 6 7 7 6 6 6 þ6 7 7 ¼ 6 7½m þ 6 7 6 4 0 1 5 r2 4 Y 21 5 4 1 5 4 w21 5 1 0 1 Y 22 w22 y ¼ Xb þ Zc þ e Then " D¼

s2r

0

0

s2r

0 s2w

0 0

0 0

s2w 0

#

and 2

s2w 6 6 0 R¼6 6 0 4 0

3 0 7 0 7 7 0 7 5 s2w

Therefore, V, the implied variance–covariance matrix for this model, is given by

A Model Selection Approach to Testing Dependent ICCs

437

We note, as expected, that observations from different groups are uncorrelated (since the upper right and lower left (2 2) blocks of V consist entirely of zeros), while the correlation of observations from the same group is given by ICC ¼

s2r s2r þ s2w

Two-Factor Model with Repeated Measures on One Factor Next, consider the model used in examples 4 and 5. The model, as given by Cohen and Doveh (this volume) in Eq. (8), is repeated below: Y iðjÞt ¼ m þ ðtimeÞt þ ðgrpÞi þ ðidÞjðiÞ þ ðgrp timeÞit þ eiðjÞt where Yi(j)t denotes the tth observation from subject j in group i. This model is commonly used to analyze data from two-factor designs with repeated measures on one factor. Discussions of this model can be found, for example, in Kirk (1995), Maxwell and Delaney (2004), and Neter, Kutner, Nachtsheim, and Wasserman (1996). This type of experimental design also falls under the general category of split-plot designs. Time (the repeated or within-subjects factor) is considered a ﬁxed effect in this model. The group effect (the between-subjects factor) and the subject effect are random with variances s2g and s2s ; respectively. The group by time interaction is also random and the variance of this effect can depend on the time point. Consequently, if there are T different time points, then in the most general case, we would need T different parameters, namely s2gt1 ; . . . ; s2gtT ; to describe the variances of the interaction effect. Finally, let s2e denote the variance of the residuals. Now assume we observe j ¼ 1; 2 subjects in i ¼ 1; 2 groups at t ¼ 1; 2 time points or measurement occasions. This is again the ‘‘smallest’’ case that still illustrates the important features of this model. Writing the model in GLMM notation yields:

438

WOLFGANG VIECHTBAUER AND DAVID BUDESCU

A Model Selection Approach to Testing Dependent ICCs

439

Then

and R ¼ s2e I 8 where I8 denotes an (8 8) identity matrix. Therefore, V can be written in block form as

where " A¼

s2g þ s2gt1 þ s2W

s2g þ s2s

s2g þ s2s

s2g þ s2gt2 þ s2W

#

and " B¼

s2g þ s2gt1

s2g

s2g

s2g þ s2gt2

#

with s2W ¼ s2s þ s2e : We see based on V that observations from different groups are uncorrelated (note the (4 4) block of zeros in the upper right and lower left of V). Note that A represents the covariance matrix for

440

WOLFGANG VIECHTBAUER AND DAVID BUDESCU

observations coming from a single individual (across the two time points), while B represents the covariance matrix for observations from different individuals in the same group (across the two time points). Hence, the correlation between observations from individuals within the same group is given by ICC 1 ¼

ICC 2 ¼

s2g þ s2gt1 s2g þ s2gt1 þ s2W s2g þ s2gt2 s2g þ s2gt2 þ s2W

ðat time point 1Þ

ðat time point 2Þ

THE MOTIVATING EXAMPLE Now consider the motivating example discussed by Cohen and Doveh (this volume). Here Yi(j)c denotes an observation from subject j ¼ 1; 2; 3 in group i ¼ 1; . . . ; 40 and condition c ¼ 1; . . . ; 4: The model is exactly of the same form as the previous one, except that the repeated factor ‘‘condition’’ now has four levels (i.e., low interdependence task at pre-test, low interdependence task at post-test, high interdependence task at pre-test, high interdependence task at post-test). Adapting the notation slightly to reﬂect this, the model can then be speciﬁed as follows: Y iðjÞc ¼ m þ ðconÞc þ ðgrpÞi þ ðsubjÞjðiÞ þ ðgrp conÞic þ eiðjÞc The variances of the group and subject effects will still be denoted by s2g and s2s ; respectively, while the variances of the group by condition interaction effect will be denoted, in the most general case, by s2gc1 ; . . . ; s2gc4 (for the four measurement occasions). We will again write out the model in GLMM notation. However, we will do so only for the three subjects in the ﬁrst group. As noted above, observations from different groups are independent and therefore looking at a single group is sufﬁcient for our purposes. Thus, for the ﬁrst group, we write

A Model Selection Approach to Testing Dependent ICCs

441

442

WOLFGANG VIECHTBAUER AND DAVID BUDESCU

Then

and R ¼ s2e I 12 where I12 denotes a (12 12) identity matrix. The form of V for the ﬁrst (or any other) group is now given by 3 2 A B B 7 6 V ¼ 4B A B5 B where 2 6 6 6 A¼6 6 4

B

A

s2g þ s2gc1 þ s2W

s2g þ s2s

s2g þ s2s

s2g þ s2s

s2g þ s2s

s2g þ s2gc2 þ s2W

s2g þ s2s

s2g þ s2s

s2g þ s2s

s2g þ s2s

s2g þ s2gc3 þ s2W

s2g þ s2s

s2g þ s2s

s2g þ s2s

s2g þ s2s

s2g þ s2gc4 þ s2W

3 7 7 7 7 7 5

and 2 6 6 6 B¼6 6 4

s2g þ s2gc1

s2g

s2g

s2g

s2g

s2g þ s2gc2

s2g

s2g

s2g

s2g

s2g þ s2gc3

s2g

s2g

s2g

s2g

s2g þ s2gc4

3 7 7 7 7 7 5

with s2W ¼ s2s þ s2e : As before, A is the covariance matrix for observations coming from a single individual (across the four conditions), while B is the covariance matrix for observations from two different individuals in the

A Model Selection Approach to Testing Dependent ICCs

443

same group (across the four conditions). Therefore, observations from individuals within the same group are correlated as follows: ICC 1 ¼

ICC 2 ¼

ICC 3 ¼

ICC 4 ¼

s2g þ s2gc1 s2g þ s2gc1 þ s2W s2g þ s2gc2 s2g þ s2gc2 þ s2W s2g þ s2gc3 s2g þ s2gc3 þ s2W s2g þ s2gc4

s2g þ s2gc4 þ s2W

ðlow interdependence task at pre-testÞ

ðlow interdependence task at post-testÞ

ðhigh interdependence task at pre-testÞ

ðhigh interdependence task at post-testÞ

Changes in group homogeneity can now be assessed by examining whether the ICC changes over time and/or experimental tasks.

Specification of all Possible Subset Models The model deﬁned in the previous section was not actually considered by Cohen and Doveh (this volume) in their analysis. Instead, using a bottomup approach, Cohen and Doveh examined a series of simpler models, in which at least some of the four variance components for the interaction effect (i.e., s2gc1 ; . . . ; s2gc4 ) were constrained to be equal to each other – and by implication, so were the corresponding ICCs (e.g., s2gc1 ¼ s2gc2 implies that ICC 1 ¼ ICC 2 ). We can in fact consider a large number of possible models, depending on the number of constraints we want to impose on the variance components. Table 1 provides a listing of all possible models. For example, in model 8, s2gc1 ¼ s2gc2 and s2gc3 ¼ s2gc4 and consequently ICC 1 ¼ ICC 2 and ICC 3 ¼ ICC 4 : In other words, observations from individuals in the same group show the same degree of correlation at the pre- and post-test, but the correlation may differ depending on task type. Hence, this model implies no change in group homogeneity across the two time points, but possibly across the two task types. The various constraints are imposed on model 1 by simple changes to the D matrix. For example, for model 8, we let

444

WOLFGANG VIECHTBAUER AND DAVID BUDESCU

Table 1.

Model Model Model Model Model Model Model Model Model Model Model Model Model Model Model

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Listing of all Possible Models Depending on the Equality Constraints among the Interaction Variances. s2gc1

s2gc2

s2gc3

s2gc4

No. of Free Parameters

a a a a a a a a a a a a a b a

b a b b b b b a b b a a b a a

c b a c b c c b a b a b a a a

d c c a c b c b b a b a a a a

4 3 3 3 3 3 3 2 2 2 2 2 2 2 1

Note: Variance components with the same letter are constrained to be equal to each other.

and therefore 2 2 sg þ s2gca þ s2W 6 6 s2g þ s2s 6 A¼6 s2g þ s2s 6 4 s2g þ s2s

s2g þ s2s

s2g þ s2s

s2g þ s2s

s2g þ s2gca þ s2W

s2g þ s2s

s2g þ s2s

s2g þ s2s

s2g þ s2gcb þ s2W

s2g þ s2s

s2g þ s2s

s2g þ s2s

s2g þ s2gcb þ s2W

3 7 7 7 7 7 5

A Model Selection Approach to Testing Dependent ICCs

445

and 2 6 6 6 B¼6 6 4

s2g þ s2gca

s2g

s2g

s2g

s2g

s2g þ s2gca

s2g

s2g

s2g

s2g

s2g þ s2gcb

s2g

s2g

s2g

s2g

s2g þ s2gcb

3 7 7 7 7 7 5

from which follows that ICC 1 ¼ ICC 2 ¼

s2g þ s2gca s2g þ s2gca þ s2W ðlow interdependence task at pre- and post-testÞ

ICC 3 ¼ ICC 4 ¼

s2g þ s2gcb s2g þ s2gcb þ s2W ðhigh interdependence task at pre- and post-testÞ

Therefore, the various models specify all the different ways how group homogeneity may change over time and/or experimental tasks. These 15 possible models can be classiﬁed into ﬁve groups, as a function of the number of free parameters being estimated: (a) the full model (model 1 in the table) including four ICCs (one for each combination of task by time), (b) six subset models (models 2–7 in the table) with only three ICCs (assuming that two of the four ICCs are equal), (c) three subset models (models 8–10) with only two ICCs (assuming that two pairs of ICCs are equal), (d) four subset models (models 11–14) with only two ICCs (assuming that three of the four ICCs are equal), and (e) a one-parameter model (model 15) assuming equality of all four ICCs. Also note that this hierarchy involves models that are properly nested within each other (e.g., all the models are properly nested within model 1 and the constraints for model 8 are a proper subset of those for model 2) and models that are not properly nested within each other (e.g., models 9 and 10 are not nested in model 2). It is possible to ﬁt each of the 15 models using the same approach and software (SAS code for ﬁtting the 15 models is given in the appendix), although clearly some models make more intuitive sense than others. For example, the constraints deﬁning model 8 (equal ICCs under low interdependence at both times and equal ICCs under high interdependence at both times) and model 9 (equal ICCs at the pre-test under both levels of interdependence and equal ICCs at the post-test under both levels of

446

Table 2.

WOLFGANG VIECHTBAUER AND DAVID BUDESCU

Estimates of Variance Components and Standard Errors for Model 1.

Variance Component Id(grp) grp

Group s2s s2g

Estimate

Standard Error

86.3716 1.2464

22.9022 17.6657

grp cond

s2gc1

1

59.0292

36.0487

grp cond

s2gc2

2

82.6210

40.4430

grp cond

s2gc3

3

grp cond

s2gc4

4

Residual

s2e

135.28 69.2271 216.91

52.6998 38.1404 19.9877

interdependence) are naturally appealing, while the constraints of model 10 make less sense and are less compelling. One particular aspect of Cohen and Doveh’s approach (this volume) calls for special attention. While Cohen and Doveh are examining a variety of simpler models, they are actually considering only subsets of the entire dataset that are relevant for establishing the homogeneity of various variance components. For example, in the ﬁrst step, Cohen and Doveh examine whether s2gc1 ¼ s2gc2 : Consequently, only the pre- and post-test data from the low interdependence task are used. This is similar to comparing model 1 with model 2, but instead of including all the data and leaving s2gc3 and s2gc4 free to vary (as model 2 would allow), the data for the high interdependence task are excluded. Similarly, only subsets of the entire dataset are used when examining whether s2gc1 ¼ s2gc3 and whether s2gc1 ¼ s2gc2 ¼ s2gc3 : An advantage of using the full dataset is that the remaining variance components (i.e., s2g ; s2s and s2e ) are estimated more efﬁciently, thereby also leading to more efﬁcient ICC estimates. For example, Table 12 in Cohen and Doveh (this volume) shows the estimated variance components and associated standard errors when only using the pre- and post-test data from the low interdependence task (and allowing the ICCs for the pre- and posttest data to differ). When using the full dataset and ﬁtting model 1 (which estimates all four ICCs without imposing any constraints), then the estimates as shown in Table 2 are obtained. Comparing these estimates with those given by Cohen and Doveh, we note that the standard errors for s^ 2s ; s^ 2g and s^ 2e are smaller when using the full dataset. However, considering that the residual variances were later found to be heterogeneous (i.e., s2e also appears to depend on the condition level), it is

A Model Selection Approach to Testing Dependent ICCs

Table 3.

Model Model Model Model Model Model Model Model Model Model Model Model Model Model Model

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

447

ICCs under all Possible Models Depending on the Equality Constraints among the Interaction Variances.

abcd aabc abac abca abbc abcb abcc aabb abab abba aaab aaba abaa baaa aaaa

ICC1

ICC2

ICC3

ICC4

2llR

AIC

0.16 0.19 0.24 0.17 0.17 0.16 0.16 0.19 0.24 0.17 0.24 0.19 0.22 0.16 0.22

0.21 0.19 0.23 0.21 0.26 0.20 0.21 0.19 0.20 0.26 0.24 0.19 0.22 0.24 0.22

0.31 0.31 0.24 0.31 0.26 0.31 0.25 0.25 0.24 0.26 0.24 0.31 0.22 0.24 0.22

0.18 0.18 0.17 0.17 0.18 0.20 0.25 0.25 0.20 0.17 0.17 0.19 0.22 0.24 0.22

4105.3 4105.6 4106.9 4105.4 4106.0 4105.4 4106.6 4106.8 4107.0 4106.0 4106.9 4105.6 4107.4 4106.7 4107.4

4119.3 4117.6 4118.9 4117.4 4118.0 4117.4 4118.6 4116.8 4117.0 4116.0 4116.9 4115.6 4117.4 4116.7 4115.4

Note: 2ll R ¼ 22 times the restricted log-likelihood; AIC ¼ Akaike’s information criterion.

understandable why Cohen and Doveh (this volume) chose to focus on subsets of the data initially. By focusing on subsets in which the residual variances appear to be roughly homogeneous, the problem of having to model heterogeneous residual variances is circumvented at ﬁrst. Nevertheless, we have some reservations about the approach chosen by Cohen and Doveh (this volume). They ﬁrst test whether s2gc1 ¼ s2gc2 (the ﬁrst pair-wise comparison) and s2gc1 ¼ s2gc3 (the second pair-wise comparison) and subsequently test whether s2gc1 ¼ s2gc2 ¼ s2gc3 (the three-way comparison). With this approach, one may reject s2gc1 ¼ s2gc2 ¼ s2gc3 after having previously established that s2gc1 ¼ s2gc2 and s2gc1 ¼ s2gc3 : Such inconsistent and intransitive patterns have long been a thorny issue for multiple comparisons procedures (e.g., Shaffer, 1995). By using Table 1 to choose a sequence of properly nested models, one can avoid such contradictions.

Model Selection In Table 3, we are providing a listing of the ICCs for the four conditions under the 15 models speciﬁed in Table 1. Also given are the value of 2 times the restricted log-likelihood (2llR) and AIC for each model. The AIC

448

WOLFGANG VIECHTBAUER AND DAVID BUDESCU

is here deﬁned as AIC ¼ 2ll R þ 2q where q is number of variance components in the model (Wolﬁnger, 1993). Smaller values of 2llR indicate a better ﬁtting model, but the ﬁt of a model can always be improved, at least marginally, by increasing the complexity of the model. The AIC is a measure of model ﬁt that includes a penalty for increasing the model complexity. Therefore, models with a small AIC strike a good balance between ﬁt and complexity (Bozdogan, 1987). Using the deviance test, we ﬁnd that model 15 (which constrains all ICCs equal to each other) ﬁts just as well as model 1 (which leaves all four ICCs unconstrained): D ¼ 4107:4 4105:3 ¼ 2:1; which under the null hypothesis follows a w2 distribution with 3 degrees of freedom and therefore is far from being signiﬁcant (p ¼ 0:55). In fact, considering that the smallest possible critical value for model comparisons is 3.84 (for a w2 distribution with df ¼ 1; i.e., for imposing a single constraint), any model comparison is going to be not signiﬁcant (for the models in Table 3, the largest value of D is 2.1). Also note that the AIC of model 15 is the smallest, indicating that this model strikes the best balance between ﬁt and complexity. Therefore, while the estimates from model 1 seem to indicate approximately homogeneous ICCs in conditions 1, 2, and 4 and a slightly higher ICC in condition 3, the model comparison shows that model 15 ﬁts just as well, indicating homogeneous ICCs across the four conditions.

Models with Heterogeneous Residual Variances However, the real ‘‘gist’’ of the example is missed by the analysis so far. As was mentioned before, the residual variance ðs2e Þ may also depend on the condition level. Accordingly, we can consider an even more general model, where R is now given by three diagonal blocks of 3 2 2 se1 0 0 0 7 6 6 0 s2e 0 0 7 2 7 6 7 6 0 s2e3 0 7 6 0 5 4 0 0 0 s2e4 instead of R ¼ s2e I 12 : Now V has the same structure as before, except that the s2W values along the diagonal of A are given by s2W 1 ¼ s2s þ s2e1 ; . . . ; s2W 4 ¼ s2s þ s2e4 ; depending on the condition level. The four ICCs

A Model Selection Approach to Testing Dependent ICCs

449

are then equal to: ICC 1 ¼

ICC 2 ¼

ICC 3 ¼

ICC 4 ¼

s2g þ s2gc1 s2g þ s2gc1 þ s2W 1 s2g þ s2gc2 s2g þ s2gc2 þ s2W 2 s2g þ s2gc3 s2g þ s2gc3 þ s2W 3 s2g þ s2gc4

s2g þ s2gc4 þ s2W 4

ðlow interdependence task at pre-testÞ

ðlow interdependence task at post-testÞ

ðhigh interdependence task at pre-testÞ

ðhigh interdependence task at post-testÞ

Analogously to the interaction variances, we can let all or only subsets of the residual variances be heterogeneous. We will make the simplifying ‘‘matching homogeneity’’ assumption that if s2gci ¼ s2gcj ; then s2ei ¼ s2ej for all pairs i,j. We do this for two reasons. First of all, without this assumption, the number of possible models would amount to 15 15 ¼ 225 (since we could combine 15 different types of constraints on the residual variances with each of the 15 models in Table 1). The simplifying assumption reduces the total number of models down to 15. However, more importantly, if we can simultaneously show that s2gci ¼ s2gcj and s2ei ¼ s2ej ; then this again implies ICC i ¼ ICC j : In other words, the assumption allows us to establish the homogeneity of various ICCs through corresponding reductions in the model complexity (more about this issue will be said below).

Model Selection Results for ﬁtting each of these models with heterogeneous residual variances are given in Table 4 (SAS code can again be found in the appendix) and a comparison with Table 3 demonstrates that the results generally differ quite substantially from those obtained under the assumption of homogeneous residual variances (except for model 15(h) in Table 4, which is identical to model 15 in Table 3 by deﬁnition). Model 1(h) (the most general model with heterogeneous residual variances) ﬁts signiﬁcantly better than model 1 (the most general model with homogeneous residual variances): D ¼ 4105:3 4090:1 ¼ 15:2; which is

450

WOLFGANG VIECHTBAUER AND DAVID BUDESCU

Table 4. ICCs under all Possible Models Depending on the Equality Constraints among the Interaction and Residual Variances.

Model Model Model Model Model Model Model Model Model Model Model Model Model Model Model

1(h) 2(h) 3(h) 4(h) 5(h) 6(h) 7(h) 8(h) 9(h) 10(h) 11(h) 12(h) 13(h) 14(h) 15(h)

abcd aabc abac abca abbc abcb abcc aabb abab abba aaab aaba abaa baaa aaaa

ICC1

ICC2

ICC3

ICC4

2llR

AIC

0.05 0.14 0.16 0.19 0.06 0.07 0.07 0.14 0.16 0.19 0.19 0.20 0.22 0.07 0.22

0.23 0.14 0.24 0.23 0.24 0.28 0.22 0.14 0.28 0.24 0.19 0.20 0.24 0.27 0.22

0.25 0.26 0.16 0.25 0.24 0.24 0.30 0.30 0.16 0.24 0.19 0.25 0.22 0.27 0.22

0.35 0.34 0.33 0.19 0.34 0.28 0.30 0.30 0.28 0.19 0.33 0.20 0.22 0.27 0.22

4090.1 4093.3 4092.2 4103.6 4091.6 4095.3 4101.9 4104.1 4097.2 4105.3 4094.6 4103.8 4107.2 4102.4 4107.4

4110.1 4109.3 4108.2 4119.6 4107.6 4111.3 4117.9 4116.1 4109.2 4117.3 4106.6 4115.8 4119.2 4114.4 4115.4

See Note in Table 3.

distributed w2 with 3 degrees of freedom under the null hypothesis that s2e1 ¼ s2e2 ¼ s2e3 ¼ s2e4 and therefore p ¼ 0:002: Also, the AIC of model 1(h) clearly falls below that of model 1, indicating that the increase in model complexity due to heterogeneous residual variances is outweighed by an even larger increase in model ﬁt. Finally, note that the resulting ICCs under model 1(h) provide a very different picture than those obtained under model 1: an ICC of almost zero under condition 1, similar ICCs under conditions 2 and 3, and a slightly higher ICC in condition 4. Having determined that heterogeneous residual variances are indicated, we can again use a hierarchical model selection approach (this time using Table 4) to ﬁnd a more parsimonious model. Analogously to the approach taken by Cohen and Doveh (this volume), we ﬁrst compare model 1(h) against model 2(h) and ﬁnd that the simpler model ﬁts just as well (D ¼ 4093:3 4090:1 ¼ 3:2; df ¼ 2; p ¼ 0:36). At this point, we can consider models 8(h), 11(h), and 12(h), which are all properly nested within model 2(h). It turns out that model 11(h) ﬁts just as well as model 2(h) (D ¼ 4094:6 4093:3 ¼ 1:3; df ¼ 2; p ¼ 0:52), while models 8(h) and 12(h) ﬁt signiﬁcantly worse (p ¼ 0:005 in both comparisons). Finally, model 15(h) ﬁts signiﬁcantly worse than model 11(h) (p ¼ 0:002) and therefore the hierarchical selection approach stops at this point. Note also that model 1

A Model Selection Approach to Testing Dependent ICCs

451

1(h) has the smallest AIC value of all the models considered. In essence, model 1 1(h) implies that group homogeneity does not change from pre- to post-test in the low interdependence task, but increases over time in the high-interdependence task. Additional Complexity due to Heterogeneous Residual Variances It is important to point out that this hierarchical selection approach focuses at each step on the question of model ﬁt, but does not directly test the equality of the various ICCs. Speciﬁcally, as Cohen and Doveh (this volume) note, the introduction of heterogeneous residual variances complicates matters with respect to directly testing the equality of ICCs. For homogeneous residual variances, we could test whether two ICCs are equal to each other by simply testing the equality of the corresponding interaction variances. This no longer works when the residual variances are heterogeneous. For example, s2gc1 ¼ s2gc2 no longer implies that ICC 1 ¼ ICC 2 : At the same time, simultaneously testing whether s2gc1 ¼ s2gc2 and s2W 1 ¼ s2W 2 ; (to determine whether ICC 1 ¼ ICC 2 ) is too restrictive, because it is possible that s2gc1 as2gc2 and s2W 1 as2W 2 ; but ICC 1 ¼ ICC 2 : For example: ICC 1 ¼

ICC 2 ¼

s2g þ s2gc1 s2g þ s2gc1 þ s2W 1 s2g þ s2gc2 s2g þ s2gc2 þ s2W 2

¼

100 þ 40 ¼ 0:25 100 þ 40 þ 420

¼

100 þ 80 ¼ 0:25 100 þ 80 þ 540

Similarly, it might be the case in the motivating example that s2gc1 ¼ s2gc2 ¼ s2gc3 as2gc4 and s2e1 ¼ s2e2 ¼ s2e3 as2e4 ; but ICC 1 ¼ ¼ ICC 4 : In other words, all four ICCs may be homogeneous, despite the fact that model 11(h) ﬁts signiﬁcantly better than model 15(h). As Cohen and Doveh (this volume) point out, there is currently no straightforward way of directly testing the equality of ICCs in this case. However, as long as we can reduce the model complexity by simultaneously imposing the same constraints on the residual and the interaction variances, we suggest that this additional complexity is of minor concern. Speciﬁcally, if the model complexity can be reduced through such simultaneous constraints (e.g., when going from model 1(h) to model 2(h) and from model 2(h) to model 11(h)), then we are not only ﬁnding models whose ﬁt is not signiﬁcantly worse than that of the more complex models, but we are also demonstrating the homogeneity of subsets of ICCs.

452

WOLFGANG VIECHTBAUER AND DAVID BUDESCU

However, once the model complexity cannot be reduced further by simultaneously imposing constraints on the residual and the interaction variances, then we must proceed in a different manner. In particular, we can conduct two separate tests, as Cohen and Doveh (this volume) do. It turns out that model 11(h) can be further simpliﬁed, since constraining all four interaction variances equal to each other (i.e., s2gc1 ¼ s2gc2 ¼ s2gc3 ¼ s2gc4 ) does not reduce the ﬁt of the model substantially (p ¼ 0:654), while a test of s2e1 ¼ s2e2 ¼ s2e3 ¼ s2e4 does turn out to be signiﬁcant (po0:001). Consequently, the ﬁnal ICCs are given by ICC 1 ¼ ICC 2 ¼ ICC 3 ¼

s2g þ s2gca s2g þ s2gca þ s2W a ðlow interdependence task at pre- and post-test and high interdependence task at pre-testÞ

ICC 4 ¼

s2g þ s2gca s2g þ s2gca þ s2W b

ðhigh interdependence task at post-testÞ

We therefore have indirectly determined that the triplet of homogeneous ICCs is signiﬁcantly different from ICC4.

CONCLUSION Cohen and Doveh (this volume) should be congratulated for their excellent discussion of the topic of comparing dependent ICCs. In the present chapter we have shown explicitly how the motivating example can be extended and framed in the context of the general linear mixed-effects model. The suggested approach stresses the representation of all the hypotheses being tested as explicit model comparisons and can be construed as a search process whose goal is, essentially, the identiﬁcation of the best-ﬁtting model. This approach leads naturally to a consideration of all possible models (e.g., Table 1) and highlights the commonalities as well as the key differences between them. By using the same standard parameter estimation technique on all the data in the sample for all the models being considered, this approach is (a) more powerful and efﬁcient, (b) minimizes the chances of inconsistencies in inferences across tests, and (c) provides the researchers with a deeper and more complete understanding of the results.

A Model Selection Approach to Testing Dependent ICCs

453

In this particular case, we selected the same ﬁnal model and arrived at the same conclusions with respect to the motivating example as Cohen and Doveh (this volume) did in their analysis, which is nice and reassuring. However, this is not guaranteed to be the case in all circumstances. We therefore consider it important to take into account the approach discussed in our commentary when examining data of the type described herein.

REFERENCES Bozdogan, H. (1987). Model selection and Akaike’s information criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52, 345–370. Dayton, C. M. (2003). Information criteria for pairwise comparisons. Psychological Methods, 8, 61–71. Kirk, R. E. (1995). Experimental design: Procedures for the behavioral sciences (3rd ed.). Paciﬁc Grove, CA: Brooks/Cole. Maxwell, S. E., & Delaney, H. D. (2004). Designing experiments and analyzing data: A model comparison perspective (2nd ed.). Mahwah, NY: Lawrence Erlbaum. Neter, J., Kutner, M. H., Nachtsheim, C. J., & Wasserman, W. (1996). Applied linear statistical models (4th ed.). Chicago: Irwin. Searle, S. R., Casella, G., & McCulloch, C. E. (1992). Variance components. New York: Wiley. Shaffer, J. P. (1995). Multiple hypothesis testing. Annual Review of Psychology, 46, 561–584. Wolﬁnger, R. (1993). Covariance structure selection in general mixed models. Communications in Statistics: Simulation and Computation, 22, 1079–1106.

APPENDIX SAS code for ﬁtting the various models presented in this commentary are given below. Assume that the data for the motivating example are contained in a plain text ﬁle (‘‘data.txt’’) with the following structure: 1 1 1 1

1 1 1 1

low1 low2 high1 high2

44 30 17 45

y

The ﬁrst number is the id number of the subject (1,y,120), the second value indicates the group number (1,y,40), the third variable indicates the condition level (low1, low2, high1, and high2), and the ﬁnal value indicates the subject’s response. Then the 15 models with homogeneous residual variances can be ﬁt with the following code:

454

WOLFGANG VIECHTBAUER AND DAVID BUDESCU

data example; infile data.txt; input id grp cond $ y; run; data example; set example; if cond ¼ ‘low1’ then cat ¼ ‘a’; if cond ¼ ‘low2’ then cat ¼ ‘b’; if cond ¼ ‘high1’ then cat ¼ ‘c’; if cond ¼ ‘high2’ then cat ¼ ‘d’; run; proc mixed data ¼ example covtest nobound; class grp cond id cat; model y ¼ cond; random int /subject ¼ id(grp); random int /subject ¼ grp vcorr; random int /subject ¼ cond*grp group ¼ cat; run;

By adjusting the letters in the various if-statements according to Table 1, one can impose the various equality constraints (the code as given above would ﬁt model 1). In order to ﬁt the 15 models with heterogeneous residual variances, one simply needs to add repeated /type ¼ vc subject ¼ grp group ¼ cat;

to the proc mixed step (at any point below the line starting with ‘‘model’’).

MORE ON THE COMPARISON OF INTRA-CLASS CORRELATION COEFFICIENTS (ICCs) AS MEASURES OF HOMOGENEITY Ayala Cohen and Etti Doveh ABSTRACT This article is a response to the two articles about our chapter (Cohen & Doveh, this volume). The first article was written by Viechtbauer and Budescu and the second written by Hanges and Lyon (both in this volume). The main contribution in the first article relates to the statistical methodology, while in the second article the authors introduce further applications to our method and discuss the interpretability of intra-class correlation coefficients (ICC). We concur with most of the ideas expressed in these articles and elaborate on some of the points raised in them.

INTRODUCTION This note is a response to the two articles on our Chapter (Cohen & Doveh, this volume). The ﬁrst article was by Viechtbauer and Budescu, the second Multi-Level Issues in Strategy and Methods Research in Multi-Level Issues, Volume 4, 455–459 Copyright r 2005 by Elsevier Ltd. All rights of reproduction in any form reserved ISSN: 1475-9144/doi:10.1016/S1475-9144(05)04020-8

455

456

AYALA COHEN AND ETTI DOVEH

was by Hanges and Lyon (both in this volume). Both articles provide valuable addenda to our chapter, particularly since each one addresses different issues. The main contribution in the ﬁrst article relates to the statistical methodology, while in the second article the authors introduce further applications to our method and discuss the interpretability of ICC.

REPLY TO VIECHTBAUER AND BUDESCU The motivation in writing our manuscript was beyond providing a solution to the problem of comparing correlated ICCs. In our manuscript, which Viechtbauer and Budescu as well as we consider to be a useful tutorial, we start with relatively simple models, we then use for each model only its relevant part of the whole data. This is a bottom-up analysis, which is particularly helpful for gaining insight into the properties of the data, for example, whether or not there are differences between the error variances under different conditions. Viechtbauer and Budescu (this volume) adopted a top-down approach for their analysis. We intentionally avoided using matrix notation, considering readers who might be less mathematically oriented. Viechtbauer and Budescu (this volume) used matrix notation, which is more compact, in order to express the General Linear Mixed Model (GLMM). While we examined only a subset of all possible models, considering only subsets of the entire data set, they analyzed the full available data and examined all possible models. They adopted the Akaike Information Criterion (AIC) for choosing the ‘‘best’’-ﬁtted model. As they pointed out, AIC is just one of the possible criteria. The use of AIC is not limited to MIXED models and extends to other statistical model-ﬁtting problems. It is, for example, one of the several goodness-of-ﬁt measures used in Structural Equation Modeling (SEM). Similar to the variety of goodness-of-ﬁt measures applicable in SEM, other criteria could be adopted for MIXED models also, which may not lead to the same goodness-of-ﬁt rankings of models. It is not surprising that in our example, the conclusions obtained by Viechtbauer and Budescu (this volume) coincide with ours. However, in general it is not guaranteed that the two different approaches, bottom-up vs. top-down, will lead to the same conclusions. Similar differences sometimes occur when different stepwise algorithms are used to obtain the ‘‘best’’ﬁtting model in regression analysis. Our algorithm and the algorithm proposed by Viechtbauer and Budescu (this volume) share the same problem of multiple testing. Also, Viechtbauer

Comparison of (ICCs) as Measures of Homogeneity

457

and Budescu (this volume), as well as we, recognized that we infer on the ICCs indirectly by comparing their numerators and denominators. In cases when ICCs differ in both their numerators and denominators, our method cannot evaluate the significance of the differences among these ICCs. It may occur that two ICCs will differ in both their numerators and denominators, while still being equal. This problem remains open for future research.

REPLY TO HANGES AND LYON There are two contributions in the article by Hanges and Lyon (this volume). The ﬁrst is their description of various applications for which our method for comparing ICCs is useful. They demonstrate that its use is not limited to group-level phenomena, but also to individual-level studies such as the assessment of individual stability under different conditions. The second contribution is their discussion concerning the interpretability of ICC as an index of homogeneity. In the introduction of our paper we mentioned that there are several approaches to measure group members’ similarity. Hanges and Lyon (this volume), and we refer to Bliese (2000), who provided an overview of what ICC, rWG(J) and the correlation ratio Z2 reveal about the group-level properties of data. The question whether rWG(J) rather than ICC should be used to assess homogeneity (in an example such as that given in our paper) is still debatable . Both ICC and rWG(J) have their advantages and disadvantages, and they often provide useful and complementary information about the within-group agreement. Therefore, researchers sometimes use both of them. For example, Hofmann and Stetzer (1996) examined the role of organizational group factors as antecedents to the accident sequence. Before investigating their hypotheses, they investigated the viability of the group-level variables. In order to demonstrate within-group agreement, they used rWG(J), but in addition they applied oneway analyses and computed intra-class correlations. They obtained high values of rWG(J) for their scales (medians above 0.90), but very low ICC values. They concluded on the basis of the high rWG(J) values that their scales appeared to have sufﬁcient within-group homogeneity to warrant aggregation. They explained that the work teams were all sampled from the same organization, therefore the low ICC values resulted from attenuated between-group variance. However, this between-group variance, as indicated by the one-way ANOVAs, was significant suggesting that there was both adequate within-group agreement as well as meaningful between-group variance in the aggregated measures. This example demonstrates how both

458

AYALA COHEN AND ETTI DOVEH

measures, ICC and rWG(J), provide useful and complementary information about the within-group agreement. In our response to the ﬁrst article (see above), we noted that one of the disadvantages of our method is that we infer on the ICCs indirectly. The significance of the differences between the ICCs is evaluated by comparing their numerators and denominators. This disadvantage turns out to be an advantage with respect to the problem raised by Hanges and Lyon (this volume). They claim that ‘‘a true test of homogeneity would explicitly test (using the F ratio) whether the variance of each group changed over time’’. In our procedure, since we examine separately the numerator and denominators of the ICCs, we actually test whether the variance within group changed over time. In the example in our Chapter, the ﬁnal conclusion that the homogeneity increased was based on the significant decrease of the within variance, while the between variance did not change. Next we refer to the comment by Hanges and Lyon (this volume) that our method is based on the restricted maximum likelihood method (REML). We should clarify that our hypothesis tests on the variances are based on the MIXED model, which includes random as well as ﬁxed effects (as implied by its name). The unknown various variance components can be estimated either by the maximum likelihood method (ML), or by the REML method. In Section 2 of our Chapter in this volume, we briefly compare the two methods. In principle, both methods can be used in our procedure, and in practice, with moderate to large sample sizes; and there will be no difference between the estimates obtained by both methods.

CONCLUSION The question whether indices of agreement differ from one another has been previously answered qualitatively. It was not clear how to compare different ICCs or rWG(J) values. We presented a procedure that enables one to compare different ICCs. We concur with Hanges and Lyon (this volume) that there are still many open questions to explore with regard to our procedure, and in particular, on the statistical power of this technique. Another challenge is to develop a method for comparing rWG(J) values.

Comparison of (ICCs) as Measures of Homogeneity

459

REFERENCES Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In: K. J. Klein & S. W. J. Kozlowski (Eds), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 349–381). San Francisco: Jossey-Bass. Hofmann, D. A., & Stetzer, A. (1996). A cross level investigation of factors inﬂuencing unsafe behaviors and accidents. Personnel Psychology, 49, 337–339.

This page intentionally left blank

460

PART VI: ABOUT THE AUTHORS

461

This page intentionally left blank

462

ABOUT THE AUTHORS Jay Barney is a Professor of Management and holds the Bank One Chair for Excellence in Corporate Strategy at the Max M. Fisher College of Business, The Ohio State University. He received his undergraduate degree from Brigham Young University, and his master’s and doctorate from Yale University. He taught at the Anderson Graduate School of Management at UCLA and Texas A&M University before joining the faculty at Ohio State in 1994, where Professor Barney teaches organizational strategy and policy to MBA and Ph.D. students. Kathleen Boies, Ph.D., is an Assistant Professor of Management at the John Molson School of Business, Concordia University. She received her doctoral degree in Industrial/Organizational Psychology from the University of Western Ontario in 2003. Her current research interests include leadership and trust, particularly as they relate to team performance, creativity, and innovation. David Budescu, Ph.D., is a Professor in the Quantitative Division of the Department of Psychology at the University of Illinois. His research interests are behavioral decision theory, judgment and communication of uncertainty, and behavioral statistics. Albert A. Cannella Jr. is the Hahnco Companies Professor of Strategic Management at Arizona State University. He received his Ph.D. from Columbia University in 1991, and served on the faculty of Texas A&M University until 2004. He served as Chair of the Business Policy and Strategy division of the Academy of Management in 2001, and as Associate Editor of the Academy of Management Review from 1999 to 2002. He currently serves on the editorial review boards of the Academy of Management Journal and the Journal of Management. His research interests include executives, executive careers, leadership, succession, and the links between executive actions and organizational outcomes. Mason A. Carpenter is the Keller Associate Professor of Strategic Management and Executive Director of the center for Strategic Management in the 463

464

ABOUT THE AUTHORS

Life and Engineering Sciences at the University of Wisconsin-Madison. He received his Ph.D. in Strategic Management and Organization Science from the University of Texas at Austin in 1997. His research concerns top management teams, corporate governance, the strategic management of global ﬁrms, and global new ventures. David Chan is a Professor at the Singapore Management University and consultant to the Prime Minister’s Ofﬁce and several organizations in Singapore. He serves on seven editorial boards. He is recipient of the Distinguished Early Career Contributions Award, William Owens Scholarly Achievement Award, and Edwin Ghiselli Innovative Research Design Award. Ayala Cohen, Ph.D., is a faculty member in the Faculty of Industrial Engineering & Management at the Technion-Israel Institute of Technology. She is head of the Statistics Laboratory that operates within the Technion. Her main research area is Applied Multivariate Analysis. Kai Schnabel Cortina received his Ph.D. in Psychology from the Free University of Berlin, Germany, in 1996. Prior to becoming Assistant Professor at the University of Michigan, he worked for the Max-Planck Institute for Human Development as a senior investigator on a 12-year longitudinal study. His particular focus was on the impact of educational institutions on life-span development from middle childhood to early adulthood. He coedited a book on longitudinal and multilevel data modeling and has published on students’ cognitive and motivational development. Catherine M. Dalton (Ph.D., Indiana University) holds the David H. Jacobs Chair of Strategic Management in the Kelley School of Business, Indiana University. She also serves as Editor of the Business Horizons, Research Director of the Institute for Corporate Governance, and Fellow in the Randall L. Tobias Center for Leadership Excellence. Professor Dalton’s research focuses largely on corporate governance. She also practices corporate governance through her service on the board of directors of Brightpoint, Inc. where she chairs the Corporate Governance and Nominating Committee. Dan R. Dalton is the founding Director of the Institute for Corporate Governance, Harold A. Poling Chair of Strategic Management, and Dean Emeritus of the Kelley School of Business, Indiana University. He received

About the Authors

465

his Ph.D. from the University of California, Irvine. Professor Dalton is widely published, with over 250 articles in corporate governance, business strategy, law, and ethics. Additionally, his work has been frequently featured in the business and ﬁnancial press including, the Business Week, Wall Street Journal, Fortune, Economist, Financial Times, Boston Globe, Chicago Tribune, Los Angeles Times, New York Times, and the Washington Post. Professor Dalton regularly addresses public, corporate, and industry groups on corporate governance issues. Etti Doveh, Ph.D., is a statistician. She is a member of the Statistics Laboratory staff in the Faculty of Industrial Engineering & Management at the Technion-Israel Institute of Technology. Her main research area is Applied Multivariate Analysis. Paul Drnevich is a doctoral student in Strategic Management at Purdue’s Krannert School. He has extensive management and consulting experience with such ﬁrms as McKinsey, PWC Consulting, and Ford. Paul’s dissertation research interests focus on how ﬁrms use information technology (IT) resources to develop organizational capabilities and competitive advantage. Kevin J. Grimm, M.A., is currently a graduate student in Psychology at the University of Virginia (Charlottesville, Virginia, USA) working with Jack McArdle and John Nesselroade. Kevin’s central methodological focus is on longitudinal structural equation models including growth curve analysis, dynamic models, and growth mixture modeling. His substantive research is concentrated in the growth and decline of cognitive abilities with an emphasis on the interrelationships in the development of academic abilities and multiple growth patterns in development. Paul J. Hanges (Ph.D., University of Akron) is a Professor and Coordinator of the University of Maryland’s I/O Psychology program. His research focuses on cross-cultural leadership, research methodology, and personnel selection. He was a co-principal investigator of the GLOBE project. Currently, he is on the editorial board of the Journal of Applied Psychology and is a fellow of the Society of Industrial and Organizational Psychology. Tim R. Holcomb is a doctoral candidate in Strategic Management at Texas A&M University. He has 20 years of strategic consulting, entrepreneurial and international management experience. His research interests include executive leadership and top management teams, executive hubris and

466

ABOUT THE AUTHORS

optimism, international growth strategies, and how ﬁrms effectively navigate in the evolving competitive landscape. Jane M. Howell holds the Taylor/Mingay Chair in Management and is Professor of Organizational Behavior at the Ivey Business School at The University of Western Ontario. She received her Ph.D. in Business Administration from The University of British Columbia. Her current research interests include champions of innovation, crisis leadership, and charismatic/transformational leadership. Samuel T. Hunter is a doctoral candidate in the Industrial and Organizational Psychology program at the University of Oklahoma. His major research interests include creativity, the management of innovation, and leadership. Julie Stella Lyon is a Ph.D. candidate in Industrial/Organizational Psychology at the University of Maryland. She holds a bachelor’s degree in Psychology from North Carolina State University and a master’s degree in Psychology from the University of Maryland. Her research interests include selection, distributed leadership, and virtual teams. Alison Mackey is a doctoral candidate in the Department of Management and Human Resources at The Ohio State University. She received her M.A. in Organizational Behavior from Brigham Young University. Her research interests include strategic leadership and managerial labor markets. John J. (Jack) McArdle, Ph.D., is currently Professor of Psychology at the University of Virginia (Charlottesville, Virginia, USA) where he has been teaching Quantitative Methods since 1984 and he is now the director of the Jefferson Psychometric Laboratory. McArdle’s research has been focused on age-sensitive methods for psychological and educational measurement and longitudinal data analysis, including published work in the area of factor analysis, growth curve analysis, and dynamic modeling of adult cognitive abilities, and a book with Richard Woodcock entitled, ‘‘Human Cognitive Abilities in Theory and Practice’’ (Erlbaum, 1998). He has been elected President of the Society of Multivariate Experimental Psychology (1993–94), President of the Federation of Behavioral, Psychological and Cognitive Sciences (1996–1999), and Secretary of the Council of Scientific Society Presidents (CSSP, 2000–2002). In 2003 he was named Lansdowne Professor of the University of Victoria, Jacob Cohen Lecturer of Columbia

About the Authors

467

University, and Best Academic Researcher, National Collegiate Athletic Association. Michael D. Mumford, Ph.D., is University Professor of Industrial and Organizational Psychology, Director of the Center for Applied Behavioral Studies, and Professor of Management, at the University of Oklahoma where he directs the doctoral program in Industrial and Organizational Psychology. He received his doctoral degree from the University of Georgia in 1983. Dr. Mumford has published more than 150 articles on creativity, leadership, planning, integrity, and job analysis. He is a fellow of the American Psychological Association (Divisions 3, 5, and 14), the American Psychological Society, and the Society for Industrial and Organizational Psychology. He serves on the editorial boards of the Creativity Research Journal, the Journal of Creative Behavior, and the Leadership Quarterly. He has received more than $20 million in grant and contract funding. Hans Anand Pant received his Ph.D. in Psychology from the Free University of Berlin, Germany, in 1998, where he holds a position as Assistant Professor in Evaluation Research. He is currently a Fulbright Scholar at the University of Michigan. The focus of his work lies in the application of multi-level approaches in Epidemiology and Public Health Research. Margaret Peteraf is an internationally recognized scholar of Strategic Management in the Tuck School at Dartmouth. Her paper, ‘‘The Cornerstones of Competitive Advantage: A Resource-Based View,’’ in the Strategic Management Journal (1993), won the Strategic Management Journal Best Paper Prize in 1999 for its deep inﬂuence on the ﬁeld. She is also known for her work on strategic groups. She is presently the Chair of the BPS Division of the Academy of Management. In addition she serves as Chair of the Competitive Strategy Group of the Strategic Management Society. Mark Shanley is an Associate Professor of Strategic Management at Purdue’s Krannert School. He taught previously at Northwestern and the University of Chicago and has published extensively on strategic management topics. His current research interests include mergers and alliances, health care strategies under deregulation, and strategic industry groups. Joanne P. Smith-Darden is currently a fourth-year joint doctoral student in Social Work and Developmental Psychology at the University of Michigan. She holds several graduate degrees that guide her research interests in the

468

ABOUT THE AUTHORS

etiology of sexually abusive behavior. Additional research interests focus on adolescent cognitive and motivational development. Robert Sternberg is IBM Professor of Psychology and Education at Yale University. He is the former President of the American Psychological Association and four of its Divisions. He is the former Editor of Contemporary Psychology and Psychological Bulletin and the author or coauthor of over 950 books, book chapters, and articles. He is a fellow of numerous professional organizations and is a major contributor to the areas of creativity and intelligence. Dr. Wolfgang Viechtbauer is an Assistant Professor in the Department for Methodology and Statistics in the Faculty for Health Sciences at the University of Maastricht. His research interests include mixed-effects models, multi-level modeling, longitudinal data analysis, and meta-analysis.