The Measure of Service Learning: Research Scales to Assess Student Experiences

The Measure of Service Learning

The Measure of Service Learning Research Scales to Assess Student Experiences

Robert G. Bringle Mindy A. Phillips Michael Hudson

American Psychological Association, Washington, DC

Copyright © 2004 by the American Psychological Association. All rights reserved. Except as permitted under the United States Copyright Act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written permission of the publisher. Published by American Psychological Association 750 First Street, NE Washington, DC 20002 www.apa.org To order APA Order Department P.O. Box 92984 Washington, DC 20090-2984 Tel: (800) 374-2721 Direct: (202) 336-5510 Fax: (202) 336-5502 TDD/TTY: (202) 336-6123 Online: www.apa.org/books/ E-mail: [email protected]

In the U.K., Europe, Africa, and the Middle East, copies may be ordered from American Psychological Association 3 Henrietta Street Covent Garden, London WC2E 8LU England

Typeset in Goudy by World Composition Services, Inc., Sterling, VA Printer: Sheridan Books, Ann Arbor, Ml Cover Designer: Michael Hentges Design, Alexandria, VA Technical/Production Editor: Rosemary Moulton The opinions and statements published are the responsibility of the authors, and such opinions and statements do not necessarily represent the policies of the American Psychological Association. Library of Congress Cataloging-in-Publication Data Bringle, Robert G. The measure of service learning : research scales to assess student experiences / Robert G. Bringle, Mindy A. Phillips, and Michael Hudson, p. cm. Includes bibliographical references and index. ISBN 1-59147-077-3 1. Student service—United States—Evaluation. 2. College students—Rating of—United States. 3. Evaluation research (Social action programs)—United States. I. Phillips, Mindy A. II. Hudson, Michael, 1966III. Title. LC220.5.B75 2004 379'.015'0973—dc22

2003016273 British Library Cataloguing-in-Publication Data A CIP record is available from the British Library. Printed in the United States of America First Edition

CONTENTS

Preface

ix

Acknowledgments

mi

I. Introduction to Service Learning and Research

1

Chapter 1.

Understanding Service Learning Service Learning as a Pedagogy Research on Service Learning Summary

3 5 7 8

Chapter 2.

Overview of Scientific Research and Measurement The Nature of Inquiry Deduction Induction: Generalizability Measurement Characteristics of Good Standardized Scales Summary

11 12 13 14 16 17 23

Chapter 3.

The Use of Scales in Service Learning Research Scales: Existing Versus Modifying Versus Developing Inclusion and Exclusion Criteria for Scales Scale Entries Summary

25

II.

Research Scales

Chapter 4.

Motives and Values Volunteer Functions Inventory

25 26 29 30 33 35 35

Chapter 5.

Motivation to Volunteer Scale Public Service Motivation Scale The Goal and Mode Values Inventories Survey of Interpersonal Values Personal Social Values

40 44 47 55 59

Moral Development Defining Issues Test Sociomoral Reflection Objective Measure Measure of Moral Orientation Prosocial Reasoning Objective Measure— Adult Version Revised Moral Authority Scale Ethics Position Questionnaire Visions of Morality Scale

63 63 67 72 77 81 85 88

Chapter 6.

Self and Self-Concept 97 Self-Esteem Scale 97 Community Service Self-Efficacy Scale 100 Self-Efficacy Scale 103 Confidence Subscale of the Erwin Identity Scale .... 107 Emotional Empathic Tendency Scale 110 Interpersonal Reactivity Index 114 Texas Social Behavior Inventory—Short Form 119 Hope Scale 126 Dean Alienation Scale 130 Selfism 134 Social Avoidance and Distress Scale 139

Chapter 7.

Student Development Student Developmental Task and Lifestyle Assessment Learning for Self-Understanding Scale Problem-Solving Inventory Career Decision-Making Self-Efficacy Scale

143

Attitudes AIDS Caregiver Scale Civic Action Community Service Involvement Preference Inventory Community Service Attitudes Scale Global Belief in a Just World Scale Life Orientation Test Revised Universal Orientation Scale

165 165 169

Chapter 8.

vi

CONTENTS

143 153 155 160

173 177 183 186 189

Social Dominance Orientation Scale Civic Attitudes Chapter 9.

193 197

Critical Thinking Watson-Glaser Critical Thinking Appraisal Scale of Intellectual Development California Critical Thinking Skills Test Cornell Critical Thinking Test

201 201 205 207 210

Glossary

215

Index

219

About the Authors

229

CONTENTS

viz

PREFACE

This volume is foremost a resource for researchers and program evaluators who are interested in developing a broader and deeper understanding of the value of service learning. Hopefully, this compilation of scales that measure key constructs associated with the experiences of students in service learning classes will stimulate innovative research and the development of additional scales that are consistent with the rationale that is provided for the advantages of multi-item measures. Those with an interest in conducting research on service learning have a tremendous opportunity to develop significant programs of research that can improve the practice of service learning, test theories about learning and student development, improve the quality of all instruction in higher education, and contribute to the evolution of the public purposes of higher education. This volume grew out of programs that were developed in Indiana during the 1990s. Indiana Campus Compact is a consortium of college and university presidents who endorse the public and civic missions of higher education. Through its programs, Indiana Campus Compact has helped member institutions engage students in community service, helped faculty in the development of service learning courses that integrate community service into the curriculum, and helped presidents and administrative staff members in institutional reform support the civic components of their mission. As part of its work, Indiana Campus Compact initiated the Universities as Citizens project in Fall 1996 to explore Ernest L. Boyer's (1996) ideas about connecting the rich resources of campuses "to our most pressing social, civic, and ethical problems, to our children, to our schools, to our teachers, to our cities" (pp. 19-20). President Clinton identified Ernest Boyer as one of the nation's most dedicated and influential educational

reformers. Boyer served as a college dean and chancellor, U.S. Commissioner of Education, and president of the Carnegie Foundation for the Advancement of Teaching. In reflecting on Boyer's contributions, Glassick (1999) noted that a common theme across Boyer's leadership positions was the call for educators to connect their work to the broader community. Boyer (1996) challenged higher education to find "a larger purpose, a larger sense of mission" (p. 20). At the heart of the Universities as Citizens project was the commitment to discuss, envision, and critically examine the implications for taking seriously Boyer's challenge for campuses to develop the scholarship of engagement. The Universities as Citizens project was conceived by Richard Games, executive director of Indiana Campus Compact, and developed by a planning committee that included Kendall Lankford from Indiana Campus Compact, Robert G. Bringle and Julie A. Hatcher from Indiana University-Purdue University Indianapolis, JoAnn Campbell from Indiana University Bloomington, and Cathy Ludlum Foos from Indiana University East. The first year of the project, funded by Cinergy Foundation, engaged Indiana campuses in planning activities and colloquia. Through a multiyear grant from the Corporation for National and Community Service, the Universities as Citizens project was extended in 1997 to include Illinois, Michigan, and Ohio Campus Compacts. Activities of the Universities as Citizens project included a series of colloquia and institutes. Indiana Campus Compact also supported campuses to examine practical next steps to promote institutional change to increase involvement of faculty, students, and administrative staff in communities by convening institutional teams at summer institutes and by providing institutional grants to encourage campus exploration and implementation of institutional change directed at different facets of the scholarship of engagement. Indiana Campus Compact also initiated the Universities as Citizens Higher Education Series, a series of publications coedited by Robert G. Bringle and Julie A. Hatcher that provide resources to higher education to enhance the quality of programs associated with community engagement. Establishing Universities as Citizens: Towards the Scholarship of Engagement (Rothman, 1998) describes the thought-provoking combination of deliberation and action that resulted from the first year of colloquia, summer institutes, and institutional development grants. In Colleges and Universities as Citizens (Bringle, Games, & Malloy, 1999), prominent scholars develop Boyer's ideas in key areas of the work of the academy and probe the possibilities and risks of community engagement by providing "guidance for paradigmatic change, [raising] aspirations for excellence in higher education, and [provoking] institutional change in how higher education conceptualizes its purpose and evaluates its accomplishments" (p. 17). PREFACE

Three publications in the Universities as Citizens Higher Education Series focus on service learning. Service Learning Tip Sheets: A Faculty Resource Guide (Hatcher, 1998) and Campus-Based Workshop Curriculum Guide (Foos & Hatcher, 1999) provide important resources to practitioners for developing, implementing, and improving service learning courses. The present volume, The Measure of Service Learning: Research Scales to Assess Student Experiences, is intended for program evaluators and researchers who can inform the practice of service learning through their scholarly work. The growth in service learning classes on campuses since the 1990s is both remarkable and paradoxical because it has been accompanied by a rather meager knowledge base (Eyler & Giles, 1999; Giles & Eyler, 1998) that is only slowly growing. The acute need for research on service learning has not gone unnoticed. Wingspread conferences were conducted in 1991 and 1993 to develop research agendas for service learning. In addition, Giles and Eyler (1998), the Research Advisory Council convened by Campus Compact during 1997-1998, and the Campus Compact's Presidents' Declaration on the Civic Responsibility of Higher Education (Boyte & Hollander, 1999) each provided agendas to stimulate research about salient questions. In a special issue of the Michigan Journal of Community Service Learning devoted to assessment and research, Bringle and Hatcher (2000) advocated that service learning practitioners must devote more resources to conducting systematic, scientific assessment of service learning outcomes across students, faculty, institutions, and communities. They noted the following: Although the experiences and anecdotes of service-learning practitioners, students, and community partners are meaningful and persuasive to some audiences, conducting systematic scientific research with meaningful indicators of educational outcomes represents a public, peer-reviewed, and replicable exercise that is important for increasing confidence among practitioners and for providing a justification to those who are in positions to support its expansion and recognition, (pp. 2-3)

The highly successful International Conferences on Service Learning Research held in fall 2001 and 2002 provided evidence that the interest in research on service learning is growing. The work presented at these conferences gives testimony to an expansion in both the quantity and the quality of research activities (see Billig & Furco, 2002). In addition, there was impressive participation by graduate students from a variety of disciplines, which indicates that future professionals are preparing to devote a significant portion of their training and careers to research on service learning. This volume is foremost a compilation of scales that measure constructs associated with students in service learning classes. In addition, this volume PREFACE

xi

provides an explanation of the characteristics that multiple-item scales should possess in order for them to be used in research. We trust that the book fulfills its major intent—to be an educational and practical resource that fosters innovative and informative research that leads to a broader and deeper understanding of service learning in higher education.

REFERENCES Billig, S. H., & Furco, A. (Eds.). (2002). Service-learning: Through a multidisdplinary lens. Greenwich, CT: Information Age. Boyer, E. L. (1996). The scholarship of engagement. Journal of Public Service and Outreach, 1, 11-20. Boyte, H., & Hollander, E. (1999). Wingspread declaration on the civic responsibility of research universities. Providence, RI: Campus Compact. Bringle, R. G., Games, R., & Malloy, E. A. (Eds.). (1999). Colleges and universities as citizens. Needham Heights, MA: Allyn & Bacon. Bringle, R. G., & Hatcher, J. A. (2000, Fall). Assessment: Meaningful measurement of theory-based service-learning outcomes. Michigan Journal of Community Service Learning, 68-75. Eyler, J., &. Giles, D. E., Jr. (1999). Where's the learning in service-learning7. San Francisco: Jossey-Bass. Foos, C. L., & Hatcher, J. A. (1999). Campus-based workshop curriculum guide. Indianapolis, IN: Indiana Campus Compact. Giles, D. E., Jr., & Eyler, J. (1998). A service-learning research agenda for the next 5 years. In R. Rhoads &. J. Howard (Eds.), Academic service learning: A pedagogy of action and reflection (pp. 65-72). San Francisco: Jossey-Bass. Glassick, C. I. (1999). Ernest L. Boyer: Colleges and universities as citizens: Issues and perspectives. In R. G. Bringle, R. Games, & E. A. Malloy (Eds.), Colleges and universities as citizens (pp. 17-30). Needham Heights, MA: Allyn & Bacon. Hatcher, J. A. (1998). Service learning tips sheets: A faculty resource guide. Indianapolis, IN: Indiana Campus Compact. Rothman, M. (1998). Establishing universities as citizens: Towards the scholarship of engagement. Indianapolis, IN: Indiana Campus Compact.

xii

PREFACE

ACKNOWLEDGMENTS

Through a symbiotic relationship, the Indiana University-Purdue University Indianapolis (IUPUI) Center for Service and Learning and Indiana Campus Compact have proven the value of thoughtful collaborative work. As executive director of Indiana Campus Compact, Richard Games provided Indiana Campus Compact with creative and well-managed programs. For example, he created the Faculty Fellows Program that is now being emulated in other states and on campuses, and he provided input into the national agenda through the Campus Compact board of directors and the Research Advisory Council. In addition, his leadership and vision have also extended the agenda of Indiana Campus Compact beyond service learning. The impetus for the University as Citizens project and, subsequently, this book have their origins in Rich's understanding of the potential for change in higher education and how to take steps to deepen and strengthen universitycommunity partnerships. We are grateful to Rich for his support and encouragement as a colleague and friend. We also appreciate the financial support that was provided by Indiana Campus Compact for researching and writing this book. In addition, the staff of Indiana Campus Compact, including Kendall Lankford and the late Brian Hiltunen, also contributed directly to the Universities as Citizens project and indirectly to this book. Julie A. Hatcher played a significant role in the Universities as Citizens project as part of the advisory group and as coeditor of the Universities as Citizens Higher Education Series. Most important, though, has been her enduring contribution as associate director of the Center for Service and Learning to the work of developing a model program of service learning, community service, and civic engagement on the IUPUI campus. She makes notable contributions not only to campus-based programs but also to the intellectual work that has guided program development, consultation with xni

other campuses, and scholarship that is focused on service learning and civic engagement. She has read drafts of the manuscript, provided constructive feedback, and offered encouragement. We especially want to thank her for her invaluable support. Barbara Cambridge wears many hats that include the following: professor of English and associate dean of faculties at IUPUI, vice president and former director of the Assessment Forum at the American Association for Higher Education, and director of the Carnegie Teaching Academy Campus Program. Barbara's work has made many contributions to activities associated with service and service learning on the IUPUI campus, to the Universities as Citizens project, and nationally through her many activities. We also acknowledge the role she has played as an intellectual, professional, and personal mentor for work that has surrounded this book and for her inspiration to persevere. The following individuals read drafts of part or all of the manuscript, and the book benefited from critical comments from these colleagues: Michelle Dunlap, Connecticut College; John McGrew, IUPUI Department of Psychology; Joe Ferrari, Psychology Department, DePaul University; Scott Evenbeck, IUPUI University College; John Saltmarsh, Campus Compact; and Kendall Lankford, Indiana Campus Compact. In addition, Susan McHenry and Laura Smith contributed many valuable hours to the project by providing editorial assistance. We also extend our gratitude to authors and publishers of scales for their cooperation. We each are especially appreciative of those persons close to us who provided love, support, and encouragement. Any project of this magnitude presents social as well as intellectual challenges. The writing has been coordinated around births, travels, studies, new jobs, and family transitions— all of which caused competing demands and compromises. We extend our sincere appreciation to each of those close to us, with a special note of appreciation to Rebecca Bringle for her understanding and for making room for the work on this book and other projects.

xiv

ACKNOWLEDGMENTS

I INTRODUCTION TO SERVICE LEARNING AND RESEARCH

1 UNDERSTANDING SERVICE LEARNING

In an American Psychological Association (APA) award address, Altman (1996) reflected on the past and present state of higher education and proposed a model for the future. In Altman's view, along with foundational knowledge (content and cross-disciplinary knowledge) and professional knowledge (practitioner skills and content), socially responsive knowledge should be an integral part of the undergraduate curriculum. How can the challenge of educating future generations include socially responsive knowledge in a manner that is pedagogically sound? How can undergraduate education prepare students for active participation in democratic processes in their communities? How can students acquire the philanthropic habits that will enrich their lives and contribute to their communities? The importance of answering these questions is heightened by a related question: "Why do we need more than a vocational education? In part, because we live more than a vocational life: we live a larger civic life and we have to be educated for it" (Mathews, 1995, p. 70). Educators in higher education, including psychologists, are interested in identifying increasingly better ways to achieve educational goals, including socially responsive knowledge and civic skills. John Dewey, who taught psychology and served as president of the APA, provided the theoretical underpinnings for understanding good instruction. He specified four conditions that maximize the potential for inquiry-based learning to be educative:

(a) generate interest in the learner, (b) be intrinsically worthwhile to the learner, (c) present problems that awaken new curiosity and create a demand for information, and (d) cover a considerable time span and be capable of fostering development over time (Giles & Eyler 1994)- Dewey (1933) valued the perplexity that students encounter when faced with new circumstances and difficult challenges in real-life situations: Thinking begins in what may fairly enough be called a forked-road situation, a situation that is ambiguous, that presents a dilemma, that proposes alternatives. .. - Demand for the solution of a perplexity is the steadying and guiding factor in the entire process of reflection, (p. H)

During the intervening 70 years, research has established a knowledge base for elements that are known to enhance depth of understanding in the learning process. These include (a) active learning; (b) frequent feedback from experts, students, or others (e.g., community practitioners) that is provided in nonthreatening ways; (c) collaboration; (d) cognitive apprenticeship (i.e., a mentor with whom students can discuss and learn generalization of principles, transfer of knowledge between theory and practice, and how to analyze perplexing circumstances); and (e) practical application in which students are involved in tasks that have real consequences but have a safety net for high-stakes mistakes (Marchese, 1997). Not surprisingly, these elements are compatible with Dewey's analysis and other discipline-based approaches to good pedagogy. For example, in a review of developments in undergraduate instruction in psychology since 1951, Brewer (1997) concluded that the "Principles for Quality Undergraduate Psychology Programs" (McGovern & Reich, 1996) developed from the St. Mary's Conference in 1991 is the "closest APA has come to providing specific direction about undergraduate programs in psychology" (p. 439). Their template defines common goals for psychology education: • multiple opportunities for students to be active and collaborative learners; • research projects to help students learn the science of psychology; • fieldwork, practica, and community service experiences to help students learn the applications of psychology; • an emphasis on learning across the curriculum about ethical issues and values; and • multiple courses and research methods which heighten students' understanding of diversity in behavior. (McGovern & Reich, 1996, p. 225). INTRODUCTION TO SERVICE LEARNING AND RESEARCH

From multiple perspectives, then, there is convergence across the past century on the qualities that are integral to using the dialectic between theory and practice for designing effective learning environments.

SERVICE LEARNING AS A PEDAGOGY Within this context, it is understandable that educators are attracted to a pedagogy like service learning, which engages students and faculty in their communities in educationally meaningful ways. Service learning1 is defined as a course-based, credit-bearing educational experience in which students (a) participate in an organized service activity that meets identified community needs and (b) reflect on the service activity in such a way as to gain further understanding of course content, a broader appreciation of the discipline, and an enhanced sense of civic responsibility. (Bringle & Hatcher, 1995, p. 112)

The service component in a course can be designed as an option to a traditional academic assignment (e.g., research paper), a requirement for all students, a disciplinary capstone course, or a 4th credit service learning option in a 3-credit semester hour course (Enos & Troppe, 1996). In service learning classes, students do not receive academic credit for engaging in community service; they receive academic credit for the learning that occurs as a result of the service experience (Howard, 1993). Service learning classes engage students in service activities that simultaneously attain two goals: The service activities (a) are beneficial to the community stakeholders (e.g., agency, clients, neighborhood residents) and (b) meet the instructor's educational objectives. The service learning class, then, establishes reciprocal relationships in which both the community and the campus invest and benefit. Thus, well-executed service learning represents a coordinated partnership between the campus and the community, with the instructor tailoring the service experience to the educational agenda and community representatives ensuring that the students' community service is consistent with their goals (Zlotkowski, 1999). In addition, the service experience provides a rich text from which academic lessons are learned through the interplay between theory and practice. The educational outcomes are derived from community service through reflection activities (e.g., journals, small group discussions, directed writing; see Eyler, Giles, &. Schmiede, 1996). The presumption is that community service does not necessarily, in and of itself, produce learning. 'Concepts in italics are defined in the Glossary.

UNDERSTANDING SERVICE LEARNING

Reflection activities provide the bridge between the community service activities and the educational content of the course. When properly designed and implemented, reflection activities direct the student's attention to new interpretations of events and provide a means through which the community service can be studied, analyzed, and interpreted much like a text is read and studied for deeper understanding. Well-designed reflection activities should (a) intentionally link the service experience to course-based learning objectives, (b) be structured, (c) occur regularly, (d) allow feedback and assessment, and (e) include the clarification of values (Bringle &. Hatcher, 1999; Hatcher & Bringle, 1997). Critical reflective thought, then, can add new meaning to service experiences, enrich the academic content of the course, and develop students' ability to take informed actions in the future (Dewey, 1916, 1933; Hatcher, 1997). As a form of experiential education, service learning shares similarities with internships, field education, practica, and voluntary service. Furco (1996) placed these forms of education on a continuum. At one end of the continuum are internships and practica with their primary focus on the students' career development. At the other end are volunteer activities, in which the emphasis is on civic involvement and the services provided to recipients. Furco (1996) located service learning in the middle of the continuum and states that it is unique in its "intention to equally benefit the provider and the recipient of the service as well as to ensure equal focus on both the service being provided and the learning that is occurring" (p. 5). However, what is most distinctive about a service learning course (in contrast to a practicum) is having civic education be a deliberate educational goal through which students develop an understanding of their current and future role in their communities. Altman (1996) noted that socially responsive knowledge has not been emphasized in higher education, and he suggested that service learning is the preferred pedagogy for achieving socially responsive knowledge. There are three goals for socially responsive knowledge: "first, to educate students in the problems of society; second, have them experience and understand first-hand social issues in their community; and third, and most important, give students the experiences and skills to act on social problems" (Altman, 1996, pp. 375-376). Service learning engages students in challenging community service settings that confront the application of course content to new situations, expose them to unfamiliar issues and to perspectives different from their own, and encourage them to consider how solutions can be developed for difficult societal issues. The promise is that service learning can produce enriched forms of learning that transcend traditional contentbased mastery and allow students to develop new ways of thinking and acting that are integrated with their personal values. There is increasing evidence that service learning is effective in helping students develop socially INTRODUCTION TO SERVICE LEARNING AND RESEARCH

responsive knowledge as well as facilitating learning in the more traditional domains of content and skills, such as the capacity to view phenomenon from multiple perspectives and to apply knowledge developed in one setting to other settings (Eyler & Giles, 1999; Eyler, Giles, Stenson, & Gray; 2001). Service learning represents a paradigm shift in higher education because it heightens the role that students can assume as constructors of knowledge. Furthermore, service learning shifts the role of the instructor from the center of instruction to the facilitator of learning that occurs outside the classroom. Part of the strength of service learning is illustrated in its compatibility with other pedagogical trends in education, such as collaborative learning, problem-based learning, and diversity education. Service learning is also compatible with other changes taking place in higher education that Rice (1996) has observed: (a) a shift from a focus on teaching to student learning; (b) moving from an emphasis on autonomous, individualistic work to collaborative, interdisciplinary work; and (c) a change from the isolated character of higher education to a more public and democratic approach to academic work. RESEARCH ON SERVICE LEARNING The growth in service learning classes on campuses during the 1990s is remarkable. Thousands of service learning classes have been developed across the entire spectrum of higher education. Interestingly, this growth of service learning has occurred with only a rather meager knowledge base from which to persuade educators and administrators about its benefits (Eyler & Giles, 1999; Giles & Eyler, 1998). The importance of high-quality systematic research, however, is not limited to benefiting future students in service learning classes through better course design. Bringle and Hatcher (2000) contended that scientific research on service learning provides a significant and necessary component of work that practitioners in service learning must more frequently use (a) to develop theory that explains the process and outcomes of service learning, (b) to improve the practice of implementing service learning courses and programs, (c) to facilitate the development of a culture of evidence and assessment on campuses, (d) to offer a justification for increased allocation of campus resources to service and service learning, and (e) to provide a basis for developing policy associated with the institutionalization of service learning in higher education, (p. 74)

Conducting good research begins with asking well-informed questions for which the answers have broad implications. At its best, the research enterprise represents an interplay between theory and research. Bringle UNDERSTANDING SERVICE LEARNING

7

(2002) discussed the importance of theory to guiding the identification, development, and clarity of research questions so that information collected has implications for other courses, different circumstances, and even other types of instruction. This volume is devoted to scales that can be used in research on students in service learning classes. We hope that advocating for the increased use of multiple-item scales will aid researchers in producing a knowledge base that further establishes the strengths of service learning, the conditions under which the optimal outcomes will be realized, the elements in course and program design that detract from realizing desired goals, and the types of students and settings that are well matched to particular strategies and outcomes. Although this book is primarily a collection of one type of research tool, multiple-item scales must be understood within the larger context of the process of conducting and reporting meaningful research. Therefore, chapter 2 provides an overview of the research process to discuss the interplay between theory and research and how dealing with measurement issues is only one component of a more extensive and complex process. Because focusing on scales runs the risk of suggesting that a scale's selection is the major task in conducting research, chapter 2 discusses the deductive and inductive processes that surround the measurement of constructs. Thus, the characteristics of good measurement (reliability, construct validity) are put within the context of other qualities (qualities of good theories, external validity) that must be considered when designing and conducting good research. Less experienced researchers may need to expand their study beyond the brief presentation in chapter 2 to other sources that provide more detail on theory and research methodologies to develop additional skills and understanding. In addition, chapter 2 provides a description of particular attributes of scales (e.g., reliability, construct validity) that are important for understanding how to measure theoretical constructs effectively. Chapter 3 discusses in more detail the nature of multiple-item scales, how the scales in this volume were selected for this volume, and the role these scales can play in developing research to answer different types of questions. Because chapters 2 and 3 discuss many terms that are used to describe and assess scales when they are presented in chapters 4-9, a Glossary compiles the definitions of key terms. The Glossary allows readers less familiar with these terms to refer to the definitions as they encounter the terms in the descriptions of scales.

SUMMARY What matters most in higher education is student learning for deeper understanding and students reaching their educational objectives. More INTRODUCTION TO SERVICE LEARNING AND RESEARCH

extensive, higher quality research is important for establishing when, for whom, how, and why service learning produces intended outcomes. Scientific research holds great potential to establish service learning as an integral component of the work of higher education. Research needs to establish that service learning contains the critical elements that promote good academic learning: active engagement, frequent feedback, collaboration, cognitive apprenticeship, and practical application (Marchese, 1997). Then service learning will be secure in assuming a more central position in higher education, alongside laboratories, internships, and problem-based learning.

REFERENCES Altman, I. (1996). Higher education and psychology in the millennium. American Psychologist, 51, 371-378. Brewer, C. L. (1997). Undergraduate education in psychology: Will the mermaids sing? American Psychologist, 52, 434-441. Bringle, R. G. (2002, October). Enhancing theory-based research on service learning. Keynote address presented at the Second International Conference on Service Learning Research, Nashville, TN. Bringle, R. G., & Hatcher, J. A. (1995). A service-learning curriculum for faculty. Michigan Journal of Community Service Learning, 2, 112-122. Bringle, R. G., & Hatcher, J. A. (1999). Reflection in service learning: Making meaning of experience. Educational Horizons, 77, 179-185. Bringle, R. G., & Hatcher, J. A. (2000, Fall). Assessment: Meaningful measurement of theory-based service-learning outcomes. Michigan Journal of Community Service Learning, 68-75. Dewey, J. (1916). Democracy and education. New York: Macmillan. Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process. Boston: D.C. Heath. Enos, S. L., & Troppe, M. L. (1996). Service-learning in the curriculum. In B. Jacoby &. Associates (Eds.), Service-learning in higher education (pp. 156-181). San Francisco: Jossey-Bass. Eyler, J., &. Giles, D. E., Jr. (1999). Where's the learning in service-learning1. San Francisco: Jossey-Bass. Eyler, J., Giles, D. E., & Schmiede, A. (1996). A practitioner's guide to reflection in service-learning: Student voices and reflections. Nashville, TN: Vanderbilt University. Eyler, J., Giles, D. E., Jr., Stenson, C. M., & Gray, C. J. (2001). At a glance: What we know about the effects of service-learning on college students, faculty, institutions and communities, 1993-2000 (3rd ed.). Nashville, TN: Vanderbilt University.

UNDERSTANDING

SERVICE LEARNING

9

Furco, A. (1996). Service-learning: A balanced approach to experiential education. In Corporation for National Service, Expanding boundaries: Serving and learning (pp. 2-6). Columbia, MD: Cooperative Education Association. Hatcher, J. A. (1997). The moral dimensions of John Dewey's philosophy: Implications for undergraduate education. Michigan Journal of Community Service Learn' ing, 4, 22-29. Hatcher, J. A., & Bringle, R. G. (1997). Reflections: Bridging the gap between service and learning. Journal of College Teaching, 45, 153-158. Howard, J. (1993). Community service in the curriculum. In J. Howard (Ed.), PRAXIS I: A faculty casebook on community service learning (pp. 3-12). Ann Arbor, MI: OCSL Press. Giles, D. E., & Eyler, J. (1994). The theoretical roots of service-learning in John Dewey: Towards a theory of service-learning. Michigan Journal of Community Service Learning, I, 77-85. Giles, D. E., Jr., & Eyler, J. (1998). A service-learning research agenda for the next 5 years. In R. Rhoads & J. Howard (Eds.), Academic service learning: A pedagogy of action and reflection (pp. 65-72). San Francisco: Jossey-Bass. Marchese, T. J. (1997). The new conversations about learning: Insights from neuroscience and anthropology, cognitive studies and work-place studies. In Assessing impact: Evidence and action (pp. 79-95). Washington, DC: American Association for Higher Education. Mathews, D. (1995). The politics of diversity and the politics of difference: Are academics and the public out of sync? Higher Education Exchange, 66-71. McGovern, T. V., & Reich, J. N. (1996). A comment on the Quality Principles. American Psychologist, 51, 252-255. Rice, R. E. (1996, January). Making a place for the new American scholar. Paper presented at the American Association for Higher Education Conference on Faculty Roles and Rewards, Atlanta, GA. Zlotkowski, E. (1999). Pedagogy and engagement. In R. Bringle, R. Games, & E. Malloy (Eds.), Colleges and universities as citizens (pp. 96-120). Boston: Allyn 6 Bacon.

10

INTRODUCTION TO SERVICE LEARNING AND RESEARCH

2 OVERVIEW OF SCIENTIFIC RESEARCH AND MEASUREMENT

Reflective practitioners in education are curious about the consequences that their work has on students, faculty members, the curriculum, the community, their institutions, and themselves. Systematically collecting information positions educators to make informed decisions that can improve their craft, enhance their understanding about why certain outcomes were achieved, and increase the confidence with which they represent their work to others, including colleagues and the general public. Collecting information about outcomes associated with teaching a course can be done in a variety of ways, including structured interviews, casual conversations with students after class, classroom assessment techniques, student portfolios, peer observation, student evaluations, course examinations, and reading student journals. In addition, practitioners can conduct formal research that uses standardized scales to measure constructs that are related to or part of intended educational outcomes. Although any of these methods may be useful, some provide more meaningful information than others do for specific purposes. Thus, course examinations are appropriate for assessing student learning and assigning grades (Walvoord & Anderson, 1998); classroom assessment techniques provide useful feedback on the effectiveness of specific modules of instruction (Angelo & Cross, 1993); and standardized scales are particularly useful when conducting program evaluation, research, and multicampus studies (Bringle & Hatcher, 2000). 11

This chapter provides an overview of how the interplay between theory and research can be analyzed into issues about the meaningfulness of theoretical analysis, the deductive process of translating concepts into researchable questions, gathering meaningful information, and generalizing those findings to other circumstances and to theories. More detailed attention is given to the rationale for multiple-item scales and the advantages that they offer to research on service learning.

THE NATURE OF INQUIRY Gaining knowledge that informs the practice of service learning is critical to the role that service learning can assume in higher education. Developing a knowledge base will determine whether service learning is a passing pedagogical fad or becomes an integral, enduring, and supported feature of the educational landscape. Such knowledge can be obtained in many ways. At the most general level, there are numerous forms of inquiry and types of evidence on which people have relied to make inferences, guide action, and persuade others. These include (a) experience and anecdotal information (including observation of the experiences of others); (b) intuition; (c) dogma, authority, and experts; (d) consensus; and (d) logical reasoning. Scientific research is distinguished from casual observation and other forms of inquiry in that scientific methods (a) are empirical (i.e., rely on observable events), (b) construct hypotheses that can be falsified, (c) gather information according to a set of procedural rules (i.e., to be systematic and follow methodological principles), (d) control for alternative explanations, (e) are public and open to scrutiny (i.e., results are shared, methods can be replicated, results are available for evaluation), and (e) aspire to be amoral or value free (i.e., acknowledge and attempt to overcome biases, be objective). In addition to its methods, the effectiveness and meaningfulness of scientific research is tied to its theoretical context. Figure 2.1 illustrates two types of connections between scientific research and theory. Hypotheses may be deduced from a theory and evaluated through a particular research study. Furthermore, a particular set of results or observations can form the basis for inductively generalizing broad principles that provide a basis for understanding phenomena. Thus, the belief that the purpose of scientific research is to collect data for its own sake is a myth. Scientific research is better viewed as the collection of data for the clarification, evaluation, and development of theory (Bringle, 2002). Theories represent cognitive and linguistic templates that are laid upon phenomena. The value of theories is that they clarify, simplify, and provide context within which diverse observations, information, inferences, and 12

INTRODUCTION TO SERVICE LEARNING AND RESEARCH

Theory Research Figure 2.1. The relationship between theory and research.

connections can be interpreted and understood. Theories are explanations for events that provide a rich set of heuristics through which to explore auxiliary phenomena, boundary conditions (i.e., when the theory is not applicable), and alternative conceptual frameworks. Theory and research are equally important to the process of accumulating knowledge through the scientific method. Where the process begins is inconsequential. Nonetheless, the presumption is that there is a symbiotic relationship between theory and research, such that theory guides the research process and research results arbitrate an evaluation and refinement of the theory (Bringle, 2002).

DEDUCTION Deduction refers to the process of going from general, abstract constructs to specific, concrete manifestations of the construct. Most typically, translating theoretical propositions into specific research questions and procedures for answering the questions is a deductive process. Constructs are abstract or hypothetical entities (i.e., concepts) that make sense out of a diverse set of phenomena. For example, consider critical thinking. The verbal, written, or behavioral manifestations of critical thinking may be apparent to an observer; they can help teachers identify differential growth among students; and they provide a means for rank ordering persons on attributes associated with critical thinking- However, the construct of critical thinking is not directly accessible; it is inferred from a coherent set of behavioral and verbal OVERVIEW OF SCIENTIFIC RESEARCH AND MEASUREMENT

13

manifestations that are presumed to indicate and represent the construct. Theorists can map the conceptual domain and identify attributes that are presumed to be indicative of good or poor critical thinking, but the construct, itself, does not exist in a tangible way. Theories are comprised of statements about the nature of constructs, their manifestations (cognitive, affective, behavioral), and the relationship between constructs. Theories ask and answer why questions. For example, a theory would explain why a particular course quality (e.g., written reflection on alternative problem-solving strategies for a community issue) produces a particular outcome (e.g., enhanced self-efficacy on community issues). Theories represent cognitive and linguistic templates that are laid upon phenomena (see Bringle, 2002). Variables are the concrete manifestations of a construct that are either (a) quantitative, in that they vary in intensity or degree (e.g., attitude, satisfaction), or (b) qualitative, in that they differ in kind (e.g., gender, types of instruction). There are potentially many variables that can be associated with a particular construct. A researcher may choose to study only one variable from the set of variables associated with a construct because that variable is more appropriate, more meaningful, and more clearly aligned with the outcomes expected from a particular intervention (see Figure 2.2). A researcher could also choose to measure more than one variable and evaluate the degree of convergence among them. Operationalization refers to the specific way in which a variable is measured (e.g., observation of behavior, scale) or manipulated (e.g., implementing a service and traditional section of a course, varying the type of reflection activities that two sections complete) in research. There are typically multiple ways to operationalize a variable, and a researcher must select among them. For example, civic skills can be measured by observing a student at a service site, analyzing journal entries, and presenting civic problems to a student and asking experts to code solutions proposed by a student. The adequacy of each operationalization depends on the degree to which it corresponds to the underlying construct that is embedded within a theoretical context. Good operationalizations should also be replicable by other researchers. The progression from construct to variable to operationalization, thus, goes from abstract to more concrete, from broad to more specific, and from theoretical to the empirical referents (see Figure 2.3).

INDUCTION: GENERALIZABILITY Induction is the generalization of broad conclusions or theoretical propositions from specific facts or findings. For example, generalizing from particular research findings from a single service learning class to a general principle about learning is an inductive process. The quality of induction relies on 14

INTRODUCTION TO SERVICE LEARNING AND RESEARCH

Construct

Variable ^

Variable

Operationalization j

Operationalization 2

Operationalization

Civic Education

Civic Attitudes

Coding Journal Entries for Civic Understanding

Civic Skills

Observing Student's Behavior at Civic Tasks

Civic Motives

Ratings of Student's Civic Skills by Agency Personnel

Figure 2.2. Constructs, variables, and operationalizations.

Theory Construct Variable

1

Operationalization Research

Figure 2.3. Translating constructs into operationalizations.

OVERVIEW OF SCIENTIFIC RESEARCH AND MEASUREMENT

15

evaluating the degree to which the specific information forms an appropriate basis for a broad conclusion. Because a broad inductive conclusion exceeds the information on which they are based, the evaluation of the meaningfulness of inductions will be a matter of degree or likelihood. External validity is the degree to which research findings generalize. There are several aspects of the evidence that determine the confidence with which generalizations can be made. The problem of induction in research depends heavily on the degree to which the sample generalizes to a broader set of circumstances and attributes. First, the less redundant the sample is, the broader the generalization can be. For example, if respondents are heterogeneous, broader generalizations are more likely to be accurate than if the respondents are highly similar. If the sample of students comes from a variety of types of colleges and universities, then there is more confidence that the results generalize to college students. If the sample comes from different regions of the country and the students are enrolled in different types of service learning classes (e.g., English, history, accounting), then the confidence also increases. However, if the research is based on one service learning class, or one institution, or one type of institution, then there is less confidence in the broad conclusion that a finding holds for all college students. Confidence in a generalization can be increased by appropriately limiting the generalization to reflect those restrictions (e.g., "Structured, regular written reflection produced more learning among psychology students at a private liberal arts college than did free-form written reflection"). Generalizability is also a function of the similarity between the sample and the population to which the generalization is to be made. The analysis of induction applies to all facets of the research enterprise (e.g., operationalizations, settings, times), not just respondents. For example, the same principles apply to how constructs are operationalized: Is there a set of heterogeneous, yet representative operationalizations that is large in number? When possible, researchers measure the same construct using different measurement procedures (e.g., self-report, archival records, observation) and different sources of information (e.g., teacher's report of student learning, student's report of learning). Positive results across different types of operationalizations increase confidence in making generalizations about a finding.

MEASUREMENT Interviews One of the simplest axioms of assessment would seem to be that if you want to know something about people, ask them! However, researchers 16

INTRODUCTION TO SERVICE LEARNING AND RESEARCH

understand the difficulties and biases that are inherent in this simple prescription. Face-to-face interviews require preparation, including some level of specification that describes the purpose of the interview. Interviews may be structured, with a predetermined set of questions being asked in a particular order, or they may be unstructured, with any questions being asked. One risk inherent in face-to-face interviews is biases that are associated with attributes of the interviewer (e.g., gender, age, status, race). There can also be inaccuracies generated by the manner in which interviewers execute the interview (e.g., not following directions, asking questions out of order, tone of voice, improvisation). Because face-to-face interviews can be complex data collection projects, they are often very expensive to conduct (e.g., training interviewers, travel to respondents). Under many circumstances, telephone interviews can reduce these costs and eliminate many of the effects attributable to interviewer characteristics. In addition, supervisors can monitor the interviewer's procedures during telephone interviews more easily than typical face-to-face interviews conducted in the field. Standardized Scales A scale is nothing more than a structured interview on paper. The questions and prompts can be open-ended with no specified format or categories for responding ("What are the biggest issues facing our community?"), or the responses can be structured (e.g., strongfy agree, strongly disagree). One of the foremost advantages of a paper-and-pencil scale is that all sources of variability attributable to characteristics of the interviewer are eliminated. The expenses of the interviewer (e.g., training, interview time, coding) are also eliminated. Once a standardized scale is incorporated in a questionnaire, the questionnaire can be rather inexpensively copied and distributed to potential respondents. Although return rates may be low, high rates of participation are possible with questionnaires, particularly when procedures are used that reflect good practice (Dillman, 1978).

CHARACTERISTICS OF GOOD STANDARDIZED SCALES Standardized scales are a product of a measurement strategy that has its roots in the deductive translation of a construct into a set of measurement procedures (Nunnally, 1967). A paper-and-pencil standardized scale is intended to measure one construct (e.g., attitude toward elderly persons, self-efficacy). As such, a scale should display qualities consistent with the assumptions of being unidimensional and measuring only one construct. Although most questionnaires are composed of several scales, each individual scale is a multiple-item measure of only one construct. Measurement is the OVERVIEW- OF SCIENTIFIC RESEARCH AND MEASUREMENT

17

process through which numbers are assigned to variations in a construct or attribute. A good scale, then, should produce numbers that correspond to individual differences in the attribute. Reliability One criterion of a good measure is that it yields consistent numbers when the attribute is stable. There are two ways of estimating consistency or reliability: (a) temporal consistency, which is assessed through test-retest and parallel forms procedures; and (b) internal consistency, which is assessed through split-half reliability and coefficient alpha. Temporal consistency works well for attributes that are assumed to be stable for a period of time. However, there are numerous factors that can potentially undermine an estimate of reliability based on both the test-retest and the parallel forms methods. Even if the attribute is presumed to be stable, environmental conditions may be different across the two administrations (e.g., quiet one time, noisy the next), and the motivation, mood, and health of respondents can vary. Each of these would result in an underestimate of the consistency of the scale (i.e., different rank orders of individuals even though the attribute being measured is constant). Is there a method of assessing reliability when respondents are available only once? One possibility would be to administer, if there are two forms, both alternate forms of the scale on a single occasion and correlate the two sets of scores. What is most noteworthy about this procedure is that it avoids all of the contaminating factors for estimating reliability from the parallel forms method when the two forms are given on different occasions (e.g., environmental variation, mood, health, motivation), except carryover effects and equivalence of the forms. If only one form of the scale is available, an alternative procedure is to administer the scale and, after the data are collected, divide the scale into two subscales. Items could be divided into subscales many ways; usually this is done randomly or by assigning the oddnumbered items to one subscale and the even-numbered items to the other subscale. This is equivalent to generating two alternative forms that contain half as many items as the original scale. The correlation between the two subscale scores obtained on a single administration is referred to as split-half reliability and is considered an estimate of the temporal consistency of the scale. Even though split-half reliability does not conform to the intuitive notion of reliability as temporal consistency, it is an estimate of temporal consistency that avoids all of the confounding problems associated with the test-retest and parallel forms methods. The correlation of the two sets of scores from the subscales is an estimate of the reliability of the whole scale. However, the correlation is between two scales that are half the length of the entire scale. Because 18

INTRODUCTION TO SERVICE LEARNING AND RESEARCH

longer scales are generally more reliable, all other considerations being equal, this correlation underestimates the reliability of the longer, whole scale. The Spearman-Brown prophecy formula provides a correction for length and an estimate of the reliability of the whole scale based on the correlation between the two halves. The split-half method of estimating reliability eliminates carryover effects as a confounding variable because items are selected from different positions on the scale and the subscale score averages across the order in which the items were answered. Furthermore, the way in which the subscales are generated makes it unlikely that the two subscales differ significantly in content. One issue associated with the split-half method of estimating reliability is that there are multiple ways of splitting the scale in half, each of which would give a different estimate of reliability. An alternative approach would be to split the scale into thirds. Each third could then be correlated with each of the other thirds, yielding three estimates of reliability. The average of these three correlations, with the Spearman-Brown prophecy formula correcting the estimate for length, would provide a more accurate estimate of the reliability of the scale than would a single split-half estimate. Extending this procedure, the scale could be split into four subscales, and each subscale score could be correlated with every other subscale, and the six correlations could be averaged. The final extension of this method would be to treat each item on a scale as a subscale, correlate each item with every other item, average those correlations, and apply the Spearman-Brown prophecy formula. This approach to estimating reliability is the basis for Cronbactis coefficient alpha, which is an estimate of reliability that takes into account the number of items on a scale and the degree to which items are correlated with each other (Cronbach, 1951). The Cronbach alpha coefficient is a computational variation of the Kuder-Richardson (K-R) formulas; the KR20 is an equivalent formula for scales that have a dichotomous responses format (e.g., "like me" or "not like me"; correct or incorrect). To understand why having more items is better than having fewer items or a single item to measure a construct, consider Nunnally's (1967) example of measuring the spelling ability of a group of children. If we gave them one word to spell, umpire, we could divide the group into good spellers and poor spellers, based on their performance. One of the problems with this procedure, in addition to only dividing the group into two subgroups (i.e., good spellers who were correct and bad spellers who were incorrect), is that any particular spelling word that might be used, like umpire, has a flaw. Any single item measures two qualities: (a) the attribute that is of interest (in this case spelling ability), and (b) attributes that are unique to that particular item (e.g., familiarity with sports), which is called specificity. An alternative approach to measuring spelling ability would be to have the children spell every word in the English language, an impractical OVERVIEW OF SCIENTIFIC RESEARCH AND MEASUREMENT

19

alternative. Clearly, the answer to the problem is to have the children spell a subset of words. But not all subsets are equally good. Words like heteroscedasticity and homunculus are poor choices because they possess too much specificity, only persons with highly specialized training would be able to spell them, and the base rate of spelling the words correctly is very low. A subset of words that includes umpire, baseball, diamond, and stadium suffers from the problem that all of the words possess the same type of specificity. Every item suffers from specificity, but some items suffer more than others do. Another issue with sampling words is their ability to measure different aspects of the construct. Words like cat, bat, mat, and rat are poor choices because they are highly redundant. Thus, there are good and inferior means through which a conceptual domain is sampled. In general, the more items that are included on the scale, the less significant specificity of each item becomes because the scale sums across items with different types of specificity. Therefore, other considerations being equal, longer tests are more reliable because specificity as a source of error becomes less significant. "Other things being equal" refers to the quality of the items being used to increase length. Adding items of equal or better quality will increase reliability. Adding items of inferior quality will not increase reliability and may decrease reliability. There are also practical limits related to the administration of a scale and the motivation of respondents that restrict scale length. In general, for research, reliability coefficients above .80 are considered acceptable, reliability coefficients in the .70 range are considered marginally acceptable, and reliability coefficients less than .70 are considered suspect and will underestimate the true relationship between two variables (Nunnally, 1967). Validity In addition to the importance of measures that give consistent readings of the same attribute, measures should also give meaningful information. Validity is the ability of a procedure to measure the construct that it is suppose to measure. For many constructs (e.g., racial tolerance, critical thinking, civic responsibility), there is no single criterion against which scores can be compared. Construct validity refers to the degree to which a particular measure of a construct produces scores that confirm the expectations contained in the theory. A construct takes meaning from the theory that specifies the constructs conceptual nature, manifestations of the construct (e.g., what is said, thought, felt, or done by those who possess the attribute), and confirmation of hypothesized relationships of the measure to other constructs (Cronbach & Meehl, 1955). For example, the construct

20

INTRODUCTION TO SERVICE LEARNING AND RESEARCH

of social justice can have different meanings depending on whether it is embedded in a theoretical perspective that is moral, psychological, or economic. The validity of a scale that measures one meaning of social justice is connected to its theory, and its construct validity is dependent on how well the scale measures that particular meaning and conforms to the predictions of that theory. A different scale that comes from a different theoretical perspective must similarly be evaluated in terms of its theoretical context and predictions. Thus, a scale may be a valid measure of a psychologically based perspective of social justice but not a valid measure of an economically defined view of the construct. There is no single index of construct validity; it is evaluated across numerous research studies that are conducted using a particular measure. Many sources of evidence contribute to establish a scale's construct validity, including the following: • Temporal consistency: evaluation of the reliability of the scale scores across time, such as test-retest and alternative forms reliability; • Internal consistency: the degree to which the items are unidimensional, including split-half reliability and coefficient alpha; • Face validity: judgments of the degree to which the scale contains items that appear to reflect the appropriate content; • Content validity: how well the scale items conform to a representative sample of a clearly defined conceptual domain; • Predictive and concurrent validity: how well the scale predicts a criterion measure; • Convergent validity: empirical evidence that scores on the scale correlate with scores from measures of the same construct or similar constructs that use either the same measurement procedure or a different measurement procedure; • Discriminant validity: empirical evidence that scores on the scale are uncorrelated with scores from measures of a construct (e.g., social desirability response bias) posited to be unrelated that use the same measurement procedure or a different measurement procedure; • Factor structure: /actor analysis or principal-components analysis of the inter-item correlations of a scale to determine (a) if it has only one factor and is unidimensional, (b) if it has clusters of items that are identified as factors, (c) the degree to which those factors are correlated or have an oblique factor structure (i.e., facets of the same construct) or (d) uncorrelated or have an orthogonal factor structure after farimax rotation (i.e.,

OVERVIEW' OF SCIENTIFIC RESEARCH AND MEASUREMENT

21

multidimensional), and (e) the degree to which the factor structure is consistent with the theory. Factor loadings are the correlation between an item and a factor; • Known groups comparisons: comparisons of scores that are obtained from groups of individuals that are known to differ on the attribute being measured by the scale; and • Experimental manipulation: demonstration that experimental manipulation of the attribute produces differences, after the manipulation, in scores on the scale obtained from the experimental and control (unmanipulated) groups. The evaluation of the validity of a measure of a construct must occur within the theoretical context of the construct. Thus, a measure of a construct is not deemed "valid." A scale is evaluated as a valid measure of a construct for a particular meaning of the construct. Thus, validity is contextspecific, is situation-specific, and is established for a particular use of a measure. Scales and Other Measurement Procedures This volume focuses exclusively on multiple-item scales. Standardized scales are tools that constitute only one part of the overall research and program evaluation strategy in which service learning practitioners may be involved (Bringle & Hatcher, 2000). These types of measures are important because they are easily transportable, they are efficient and effective ways of obtaining information, and they are readily coded and analyzed. We are not, however, advocating for the exclusive use of scales in research on service learning. As research is being designed, researchers should consider alternative forms of measurement, such as observation, interviews, qualitative data, physiological measures, focus groups, and archival information. More importantly, there are great benefits to researchers using multiple sources of data and multiple methods for collecting data so that converging results can be established (see Gelmon, Holland, Driscoll, Spring, &. Kerrigan, 2001). The description of multiple-indicator indices of constructs most obviously applies to paper-and-pencil scales. However, the measurement principles described in this chapter are not limited to only that type of measurement. Single- and multiple-indicator indices can be taken from observations of behaviors, reports by key persons (e.g., site supervisors), samples of journal entries, physiological measures of empathic reactions, ratings of job performance, and archival measures (e.g., transcripts). In all cases, these indices are improved by having multiple indicators. Thus, a single observation of a behavior possesses specificity just like a single item on a scale. For example, 22

INTRODUCTION TO SERVICE LEARNING AND RESEARCH

Epstein (1979) demonstrated that observations of students' punctuality at class increased in reliability when they were taken over multiple class meetings in contrast to when they were based on only one class meeting.

SUMMARY There are multiple sources of information that service learning practitioners can use to inform the development of their work and to portray its value to other audiences. Different types of inquiry (reflection, experimental, survey, qualitative, correlational) have both strengths and weaknesses, and the meaningfulness of any one source or type of information varies depending on the appropriateness of its application. Scientific research represents a particular type of systematic inquiry incorporating inductive and deductive inferential processes between theories and research procedures. Theories contain statements about the nature of constructs and the relationships among constructs. Research on constructs needs to use measurement procedures that provide representative samples of the conceptual domain that are accurate (i.e., error free), consistent across time (i.e., reliable), and coherent (i.e., internally consistent). Although single-item measures may be appropriate in some isolated instances, multiple-item measures are more likely to achieve these ends than single-item measures. When a multiple-item measure meets these criteria, then its validity can be assessed by examining the content of the items, comparing scores with empirical criteria, and evaluating the construct validity of the measurement procedure across a program of research. These criteria can establish the meaningfulness of the scores and guide the appropriate use of the measurement procedure.

REFERENCES Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers (2nd ed.). San Francisco: Jossey-Bass. Bringle, R. G. (2002, October). Enhancing theory-based research on service learning. Paper presented at the Second International Conference on Service Learning Research, Nashville, TN. Bringle, R. G., & Hatcher, J. A. (2000, Fall). Assessment: Meaningful measurement of theory-based service-learning outcomes. Michigan Journal of Community Service Learning, 68—75. Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika,16, 297-334. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-301.

OVERVIEW OF SCIENTIFIC RESEARCH AND MEASUREMENT

23

Dillman, D. A. (1978). Mail and telephone surveys: The total design method. New York: Wiley. Epstein, S. (1979). The stability of behavior: I. On predicting most of the people much of the time. Journal of Personality and Social Psychology, 37, 1097-1126. Gelmon, S. B., Holland, B. A., Driscoll, A., Spring, A., & Kerrigan, S. (2001). Assessing service-learning and civic engagement: Principles and techniques. Providence, RI: Campus Compact. Nunnally, J. C. (1967). Psychometric theory. New York: McGraw Hill. Walvoord, B. E., & Anderson, V. J. (1998). Effective grading : A tool for learning and assessment. San Francisco: Jossey-Bass.

24

INTRODUCTION TO SERVICE LEARNING AND RESEARCH

3 THE USE OF SCALES IN SERVICE LEARNING RESEARCH

Program evaluation and research have a long history in education (Sansone, Morf, & Panter, 2003). To some extent, the existing literature on measurement and design issues can be applied to service learning with good results. However, service learning has special characteristics such as unique aspects of the pedagogy (e.g., reflection) that warrant additional resources for good research. This is particularly the case for unique outcomes expected from service learning classes (changes in values, moral development, civic outcomes; Shumer, 2000)

SCALES: EXISTING VERSUS MODIFYING VERSUS DEVELOPING Using existing scales has many advantages over developing original scales. Existing scales take less time to incorporate in research, are usually prepared by researchers who have professional expertise, may have norms available against which a particular sample can be compared, and have a known record of psychometric qualities (although these may vary from sample to sample). Existing scales may also be modified to suit a particular research context or question associated with a service learning class. Adapting a scale has the advantage that most of the work has been completed and the resulting 25

scale may be more appropriate than the original scale. Modifying a scale, however, runs the risk of changing a known quantity in unknown ways. Developing an original scale allows a researcher to design a scale for a specific need. However, developing a new scale takes time and resources (e.g., literature review, pilot testing), requires knowledge of scale development procedures, and runs the risk that the result may not be a good scale. The rudiments for scale construction are found in the nature of construct validity. Constructing a good scale requires articulating a clear theoretical statement for the construct, delineating the content domain, writing items that adequately sample the domain, evaluating the unidimensionality and factor structure of a preliminary pool of items, and obtaining evidence of construct validity (e.g., Fink & Kosecoff, 1998).

INCLUSION AND EXCLUSION CRITERIA FOR SCALES In considering scales to be included in this volume, all published empirical research on service learning in higher education, some of the empirical research in K—12, dissertations that had been conducted on service learning, and empirical research from cognate areas (e.g., psychology, sociology, education) were examined. Multiple postings were made requesting scales on the Colorado State University service learning listserv (http:// www.csf.colorado.edu/sl/), and members of the Campus Compact Research Advisory Council were asked to contribute scales that have been used in service learning research. In spite of these efforts, this compilation was not taken from an exhaustive pool of potential scales. Two primary criteria were used to select scales for this volume from the empirical literature that was examined: (a) relevance to service learning and (b) the quality of supporting evidence. Relevance to Service Learning The content domains were selected because they include constructs that were examined in past research on service learning or they are present in or related to conceptual discussions of service learning (e.g., Eyler & Giles, 1999; Giles &. Eyler, 1998). These domains are not exhaustive for service learning, nor are the scales that are described in a particular domain the only ones that exist. For selected scales that have not been used in past research on service learning, the relevance of a scale's content was a judgment based on our impression of how it might be useful. Some scales were selected not because they are appropriate as an outcome measure but because they might prove interesting as a moderator or mediating variable. 26

INTRODUCTION TO SERVICE LEARNING AND RESEARCH

A mediating variable describes an intervening variable that is assumed to explain the relationship between two variables (Baron & Kenny, 1986). Mediating variables are useful for exploring theoretical explanations for why a relationship exists between two variables. For example, it might be hypothesized that structured reflection activities generate better learning among novice students because those activities provide better conceptual maps of the content domain than do unstructured reflection activities. In this case, the nature of the conceptual map is assumed to mediate or explain the relationship between reflection and learning. If the learning outcomes differ because of the type of reflection activities for novice students, then it is expected that a measure of conceptual maps will also differ for the two groups. Baron and Kenny (1986) described three conditions for determining if a variable is a mediating variable of two other variables: (a) a significant correlation between the antecedent variable and the mediating variable, (b) a significant correlation between the mediating variable and the outcome variable, and (c) a significant reduction in (or disappearance of) the correlation between the antecedent and outcome variable when the effects of the mediating variable are partialed out. A moderator variable, which describes an "it depends" relationship, specifies the conditions for which the relationship holds and other conditions for which a different relationship holds (Baron & Kenny, 1986). As such, moderator variables are useful for exploring the boundary conditions for a phenomenon. For example, structured reflection activities could be hypothe' sized to produce better learning for novice service learning students than do unstructured reflection activities, but the opposite is the case for experienced service learning students. The hypothesis is positing that one variable moderates the impact of the other variable. In this example, the impact of the type of reflection on learning depends on the background of the learner. Thus, moderator variables, which identify interaction effects, and mediating variables, which explain the relationship between two variables, are particularly useful in developing knowledge, clarifying theory, and informing practice. The volume does not include scales to measure the academic performance of students in service learning classes. Achievement scales of this type are much too course-specific to be readily transportable. The book also does not contain scales that measure student satisfaction for a service learning class. Furthermore, the focus of this book is exclusively on the processes and outcomes of students. No attempt has been made to compile scales to study other aspects of service learning such as community impact (Cruz & Giles, 2000), institutional attributes (Holland, 2000), and faculty (e.g., motives, professional development, outcomes; Driscoll, 2000). Practitioners must determine the appropriateness of a construct and scale. This assessment should consider the design of the course (e.g., USE OF SCALES IN SERVICE LEARNING RESEARCH

27

educational goals), the implementation of the course (e.g., selection of course activities and reflection assignments), and expected outcomes. A sound rationale that relates goals, course activities, and outcomes must exist for an expected outcome prior to the selection of a scale to measure that outcome. Research and program evaluation directed at service learning will benefit immensely when more practitioners develop multiple-item measures of constructs that target specific outcomes of service learning (e.g., civic skills, civic responsibility, citizenship), for which few scales could be found. Some constructs are conceptualized as being stable across time and situations (e.g., self-esteem, self-efficacy). These constructs are traitlike in nature, and the measure considers the construct in a global sense. In contrast, there are other constructs that are assumed to be more malleable and sensitive to environmental influences (e.g., attitude, mood). These constructs are statelike in that the measure focuses on a particular time and setting. Scales measuring traitlike constructs may be of interest to researchers as precursors that predispose individuals to behave, think, or feel in particular ways during a service learning experience. In this case, their role may be as a moderator variable that clarifies a set of conditions under which certain effects take place. In contrast to this use, a measure of a traitlike construct would be inappropriate as a pre-, posttest dependent variable if there is little reason to assume that a brief educational experience (i.e., a course) would change a trait (e.g., self-esteem). Malleable, state-dependent constructs would be more appropriate for dependent variables in this case (e.g., attitudes). The nature of some constructs may vary in the degree to which they are construed as being traitlike or statelike in nature. For example, selfesteem is assumed to be a relatively stable attribute that is a function of a lifetime of experiences. However, one could conceive a construct that is the person's self-esteem as a student. In this case, the construct might more reasonably be influenced by experiences during a course or semester. Furthermore, the construct could be the person's self-esteem as a student in a particular class; that is, how favorably or unfavorably does the person feel as a student during this service learning class? In this case, the focus of the construct is more situation specific and the expectations for the construct being influenced by the experience are more reasonable than with the global, traitlike construal. Thus, scales that are designed to measure traitlike constructs can be customized to focus the attention on state-oriented aspects of the attribute. Quality of Supporting Evidence Scales were selected not only because of their relevance to research on service learning but also because they are multiple-item measures of 28

INTRODUCTION TO SERVICE LEARNING AND RESEARCH

specified constructs that possess minimally acceptable psychometric characteristics (i.e., reliability, validity). There are numerous scales that were located in research on service learning that are multiple-item measures, but there was an absence of supporting evidence that the collection of items was intended to measure a specific construct. There are numerous scales, particularly ones developed as part of dissertation research, that appeared relevant but did not have sufficient supporting evidence for inclusion. Researchers may want to consult these dissertations; a comprehensive holding exists in the Payton Philanthropic Studies Library on the Indiana University-Purdue University Indianapolis campus (www.ulib.iupui.edu/ special/ppsl.html). In some cases, these scales that did not have supporting evidence for reliability and validity when we examined them may acquire evidence supporting reliability and validity in the future. There are also scales that have been developed for measuring outcomes in K-12 service learning classes. In some cases, this volume includes scales that can be used in precollege and collegiate settings. A complementary compilation of measurement instruments for K-12 exists at http://cart.rmcdenver.com/ and researchers may wish to consult that resource. In addition, there are many examples of single-item measures of unknown integrity that have been used in service learning research. These items are sometimes referred to as a "scale." For example, Markus, Howard, and King (1993) present what is called a Social Responsibility Scale. However, these items have no supporting evidence that they are a coherent multiple-item measure (e.g., coefficient alpha) and the items were analyzed as separate dependent variables.

SCALE ENTRIES Researchers must select scales based on the appropriateness of the scale to the broader research questions that are being asked. To assist researchers, the entries in chapters 4-9 provide a description of each scale that identifies the construct that is measured, a description of the scale, and some of the evidence about the psychometric properties of the scale that could be identified in existing published and unpublished (e.g., dissertations) research. In most cases, the descriptions have been shared with authors and the entries have been edited and updated according to their suggestions. The purpose of these entries is to provide sufficient information to guide researchers in identifying potentially useful scales. However, researchers who are selecting scales are advised to consult the primary sources to learn more about the theory and research that is critical to determining if a particular scales is appropriate. Potential users of these scales must also be aware that additional research may be available that would influence their decision to select a scale. USE OF SCALES IN SERVICE LEARNING RESEARCH

29

Some scales that are included are unpublished. In those cases, the author's contact information at the time of publication is given. When possible, the complete scale is provided and has been done so with the permission of the author or copyright holder. In a few cases, either no permission could be obtained or the scale is distributed commercially; for these cases, sample items and information about obtaining the complete scale are provided.

SUMMARY Although the scientific approach that emphasizes quantitative measurement may seem too structured and too narrow (see Shumer, 2000), accumulating this type of research is important (a) to practitioners as a means of improving knowledge and confidence and (b) to external audiences to increase their appreciation for how service learning can help the academy do its most important work: educating students to be better informed, more fully functioning, and active individuals in their communities. The scales contained in this volume are but one tool to be used by practitioners to engage in scientific research. Researchers must use good judgment and creativity to determine which tools are appropriate (e.g., scales, observations, archival measures, interviews, qualitative methods), which procedures are best suited to investigate a particular question, what inferences should be drawn from their research, and what are the best ways to communicate the results to multiple audiences. However, our belief is that the field will be well served when more scientific research is conducted using multiple-item scales. For, as Thorton (1999) noted, "Science is one of the glories of our world. It is the work of the analytic mind in love with life, and its poetry is mathematics. Science continuously reveals wonder, if we let it" (p. 26).

REFERENCES Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173-1182. Cruz, N. I., & Giles, D. E., Jr. (2000, Fall). Where's the community in servicelearning research? Michigan Journal of Community Service Learning, 28-34Driscoll, A. (2000, Fall). Study faculty and service-learning: Directions for inquiry and development. Michigan Journal of Community Service Learning, 35-41. Eyler, J., & Giles, D. E., Jr. (1999). Where's the teaming in service-learning^ San Francisco: Jossey-Bass.

30

INTRODUCTION TO SERVICE LEARNING AND RESEARCH

Fink, A., & Kosecoff, J. (1998). How to conduct surveys: A step-by-step guide (2nd ed.). Thousand Oaks, CA: Sage. Giles, D. E., Jr., & Eyler, J. (1998). A service-learning research agenda for the next 5 years. In R. Rhoads & J. Howard (Eds.), Academic service learning: A pedagogy of action and reflection (pp. 65-72). San Francisco: Jossey-Bass. Holland, B. A. (2000, Fall). Institutional impacts and organizational issues related to service-learning. Michigan journal of Community Service Learning, 52-60. Markus, G. B., Howard, J. P. F., & King, D. C. (1993). Integrating community service and classroom instruction enhances learning: Results from an experiment. Educational Evaluation and Policy Analysis, 15, 410-419. Sansone, C., Morf, C. C., & Panter, A. T. (2003). The sage handbook of methods in social psychology. Thousand Oaks, CA: Sage. Shumer, R. (2000, Fall). Science or storytelling: How should we conduct and report service-learning research? [Special issue]. Michigan Journal of Community Service Learning, 76-83. Thorton, J. (1999). A field guide to the soul: A down-to-earth handbook of spiritual practice. New York: Bell Tower.

USE OF SCALES IN SERVICE LEARNING RESEARCH

3]

4 MOTIVES AND VALUES

VOLUNTEER FUNCTIONS INVENTORY Source Clary, E. G., Snyder, M., Ridge, R. D., Copeland, J., Stukas, A. A., Haugen, ]., & Miene, P. (1998). Understanding and assessing the motivations of volunteers: A functional approach. Journal of Personality and Social Psychology, 74, 1516-1530. Construct The Volunteer Functions Inventory (VFI) assesses six functions that are served through volunteer activity: values, understanding, social, career, protective, and enhancement. Description The VFI is conceptually based on a theoretical approach that has been most prominent in the study of attitudes (Clary & Snyder, 1991). The functional approach examines the different functions that are served by an attitude, cognition, relationship, or behavior. Extrapolating from this conceptual theme, this scale is based on a functional examination of how 35

the act of volunteering can reflect different motives for an individual because the behavior serves different functions. The VFI consists of six subscales corresponding to the following functions: • Values: the degree to which volunteering expresses altruistic and humanitarian concern for others; • Understanding: the degree to which volunteering provides opportunities for new learning experiences and to use knowledge, skills, and abilities; • Social: the degree to which volunteering allows the person to be with friends and receive the recognition of others; • Career: the degree to which volunteering promotes clarity about vocational choices; • Protective: the degree to which volunteering allows the person to avoid guilt and better cope with personal problems; and • Enhancement: the degree to which volunteering promotes an individual's sense of personal growth and positive feelings. Each subscale consists of five items that assess the individual's degree of that motivation in making decisions to volunteer. Answers are recorded on a 7-point response scale anchored with not at all important/occur ate to extremely important/accurate. Subscale scores are obtained by calculating the mean score across the five items in each subscale, with a higher score indicating a greater sense of importance associated with the motivation. Estimated Time to Administer Fifteen minutes. Reliability Temporal Consistency

Clary, Snyder, Ridge, et al. (1998) reported 4-week test-retest reliabilities of .78 (values), .77 (understanding and enhancement), .68 (social and career), and .64 (protective). Internal Consistency Across three studies, Clary, Snyder, Ridge, et al. (1998) reported all subscales demonstrated coefficient alphas greater than .80. Factor Structure

Factor analyses of the scale have produced good factor structures indicating that the items are measuring distinct types of motives for volunteering.

36

RESEARCH SCALES

Furthermore, the factor structure demonstrates consistency across studies (Clary, Snyder, Ridge, et al, 1998). Validity Known Groups A significant correlation (-.53) was found for age of volunteers and career scores on the VFI (Clary, Snyder, & Ridge, 1992). This finding indicates that older volunteers were not motivated to volunteer for a career, presumably because they were already settled in careers or retired. Convergent Validity Clary, Snyder, Ridge, et al. (1998) created six advertisements that were tailored to appeal to the six psychological functions measured by the VFI. With the exception of the social advertisement, each of the advertisements was most strongly correlated with responses to the corresponding subscale of the VFI. They also found that volunteers who reported receiving benefits from their participation in community service that corresponded to their strong motive were more satisfied with their service and intended to continue to volunteer in the short and long term, when compared with those who did not receive functionally relevant benefits. Discriminant Validity The VFI was compared with Jackson's (1974) Personality Research Form (PRF). Generally, VFI subscales did not correlate with PRF scales. Furthermore, most VFI subscales were found to be better predictors than the PRF scale of advertisements designed to appeal to various motives to volunteer. Gender Differences Switzer, Switzer, Stukas, and Baker (1999) reported that female respondents scored significantly higher on the Values subscale than male respondents. Switzer et al. stated that women tended to score higher than men on all the subscales of the VFI, and the small sample size may account for the inability to detect statistically significant differences between men and women. Comments Many educators assume that student motives can be transformed across one or more service learning classes (Astin &. Sax, 1998; Driscoll, Holland, MOTIVES AND VALUES

37

Gelmon, & Kerrigan, 1996). The VFI scale is a psychometrically sound scale that can be a useful resource for those interested in studying motivations for engaging in volunteer activities and for studying changes in motivation that occur as a result of community service and service learning. Clary, Snyder, and Stukas (1998) discussed the relevance of the functional approach to motives that sustain volunteering. They speculated, for example, that volunteers who have a motive base that is balanced across the types of motives are more likely to persist than those for whom the motive base is strong for one motive and weaker for all others (e.g., the pure altruist). How motives measured by the VFI are related to student recruitment, service experiences, reflection strategies, and learning outcomes present interesting research possibilities. Different types of service learning classes may appeal to and reward students with different types of motives. These motives could include humanitarian concerns, increasing the students' academic knowledge, providing opportunities to make connections with other students, clarifying career decisions, and obtaining experiences that will make the student attractive to employers. When these attributes of service learning experiences are measured and their effects monitored, then research will be able to clarify how the nature of the service learning experience interacts with the motivational profiles of students to produce (or not produce) specific outcomes. References Astin, A. W., & Sax, L. J. (1998). How undergraduates are affected by service participation. Journal of College Student Development, 39, 251-263. Clary, E. G., & Snyder, M. (1991). A functional analysis of altruism and prosocial behavior: The case of volunteerism. In M. Clark (Ed.), Review of personality and social psychology (Vol. 12, pp. 119-148). Newbury Park, CA: Sage. Clary, E. G., Snyder, M., & Ridge, R. (1992). Volunteers' motivations: A functional strategy for the recruitment, placement, and retention of volunteers. Nonprofit Management and Leadership, 2, 333-350. Clary, E. G., Snyder, M., Ridge, R. D., Copeland, J., Stukas, A. A., Haugen, J., &. Miene, P. (1998). Understanding and assessing the motivations of volunteers: A functional approach. Journal of Personality and Social Psychology, 74, 15161530. Clary, E. G., Snyder, M., & Stukas, A. (1998). Service-learning and psychology: Lessons from the psychology of volunteers' motivations. In R. G. Bringle & D. K. Duffy (Eds.), With service in mind: Concepts and models for service-learning in psychology (pp. 35-50). Washington, DC: American Association for Higher Education. Driscoll, A., Holland, B., Gelmon, S., & Kerrigan, S. (1996). An assessment model for service-learning: Comprehensive case studies of impact on faculty, students,

38

RESEARCH SCALES

community, and institutions. Michigan Journal of Community Service Learning, 3, 66-71. Jackson, D. N. (1974). Personality Research Form (2nd ed.). Port Huron, Ml: Research Psychologists Press. Switzer, C. L, Switzer, G. E., Stukas, A. A., & Baker, C. E. (1999). Medical student motivations to volunteer: Gender differences and comparisons to other volunteers. Journal of Prevention and Intervention in the Community, ] 8, 53-64.

Scale Directions: If you have done volunteer work before or are currently doing volunteer work, then, using the 7-point scale below, please indicate how important or accurate each of the following possible reasons for volunteering is for you. If you have not been a volunteer before, then, using the 7-point scale below, please indicate how important or accurate each of the following reasons for volunteering would be for you. 1 2 not at all important/accurate for you Rating

3

4

5

6

7 extremely important/accurate for you

Reason

1. Volunteering can help me get my foot in the door at a place where I would like to work. 2. My friends volunteer. 3. I am concerned about those less fortunate than myself. 4. People I'm close to want me to volunteer. 5. Volunteering makes me feel important. 6. People I know share an interest in community service. 7. No matter how bad I've been feeling, volunteering helps me to forget about it. 8. I am genuinely concerned about the particular group I am serving. 9. By volunteering, I feel less lonely. 10. I can make new contacts that might help my business or career. 11. Doing volunteer work relieves me of some of the guilt over being more fortunate than others. 12. I can learn more about the cause for which I am working. 13. Volunteering increases my self-esteem. 14- Volunteering allows me to gain a new perspective on things. 15. Volunteering allows me to explore different career options. 16. I feel compassion toward people in need. MOTIVES AND VALUES

39

.17. Others with whom I am close place a high value on community service. . 18. Volunteering lets me learn through direct "hands on" experience. . 19. I feel it is important to help others. . 20. Volunteering helps me work through my own personal problems. .21. Volunteering will help me succeed in my chosen profession. . 22. I can do something for a cause that is important to me. . 23. Volunteering is an important activity to the people I know best. . 24. Volunteering is a good escape from my own troubles. . 25. I can learn how to deal with a variety of people. . 26. Volunteering makes me feel needed. .27. Volunteering makes me feel better about myself. . 28. Volunteering experience will look good on my resume. . 29. Volunteering is a way to make new friends. . 30. I can explore my own strengths. Social (Items 2, 4, 6, 17, 23) Values (Items 3, 8, 16, 19, 22) Career (Items 1, 10, 15, 21, 28) Understanding (Items 12, 14, 18, 25, 30) Esteem (Items 5, 13, 26, 27, 29) Protective (Items 7, 9, 11, 20, 24) Note. From "Understanding and Assessing the Motivations of Volunteers: A Functional Approach," by E. G. Clary, M. Snyder, R. D. Ridge, J. Copeland, A. A. Stukas, J. Haugen, and P. Miene, 1998, Journal of Personality and Social Psychology, 74, p. 1520 (Table 1). Copyright 1998 by the American Psychological Association. Reprinted with permission.

MOTIVATION TO VOLUNTEER SCALE Source Cnaan, R. A., & Goldberg-Glen, R. S. (1991). Measuring motivation to volunteer in human services. Journal of Applied Behavioral Science, 27, 269284. Construct The Motivation to Volunteer (MTV) scale measures motivations to volunteer among "those who habitually provide direct care in human services across a range of programs" (Cnaan &. Goldberg-Glen, 1991, p. 282). 40

RESEARCH SCALES

Description The 22-item MTV scale was developed from a factor analysis of an original grouping of 28 items. The scale uses a 5-point response format anchored by not important at all to very important. Estimated Time to Administer Ten to fifteen minutes. Reliability Temporal Consistency No information has been located. Internal Consistency Coefficient alpha was .86 for the 22 items (Cnaan & GoldbergGlen, 1991). Factor Structure Cnaan and Goldberg-Glen (1991) conducted numerous factor analyses and concluded that the 22 items that compose the final version of the MTV scale were univariate. However, Trudeau and Devlin (1996) reported a fivefactor solution. Validity Known Groups Cnaan and Goldberg-Glen (1991) compared scores on the MTV scale for individuals who volunteered in human services and nonvolunteers. Scores were significantly higher for the volunteer group than the nonvolunteer group, although Trudeau and Devlin (1996) failed to replicate this difference. In addition, there was a significant correlation between scores and number of hours per month devoted to volunteer work (Cnaan & Goldberg-Glen, 1991). Convergent Validity The instrument was based on the content analysis of the 28 motives derived from a literature review. Discriminant Validity No information has been located.

MOTIVES AND VALUES

41

Gender Differences No gender differences are reported by Cnaan and Goldberg'Glen (1991). Trudeau and Devlin (1996) reported that female respondents scored higher than male respondents on the Altruistic subscale. Comments The MTV scale provides an alternative to Clary et al.'s (1998) approach to measuring volunteer motives. The items were generated from a review of the literature and reduced through factor analysis to a unidimensional scale. Cnaan and Goldberg-Glen (1991) contrasted this inductive approach to the theory-driven approach of Clary and his colleagues (Clary et al., 1998). The factor analyses to date have been conducted with minimally sufficient numbers of participants to be confident about the stability of the items as unidimensional or multidimensional. The scale provided includes all 28 original items. According to Cnaan and Goldberg-Glen (1991), Items 2, 3, 5, 6, 7, and 21 did not load into the unidimensional factor created by the remaining items. Trudeau and Devlin (1996) used the 22-item survey found in the Goldberg-Glen study. However, they also included Item 7 in their study although no statistical evidence warranted the inclusion. Users of this scale should be aware of these inconsistencies and evaluate the scale's characteristics after collecting data. References Clary, E. G., Snyder, M., Ridge, R. D., Copeland, J., Stukas, A. A., Haugen, J., & Miene, P. (1998). Understanding and assessing the motivations of volunteers: A functional approach. Journal of Personality and Social Psychology, 74, 15161530. Cnaan, R., & Goldberg-Glen, R. (1991). Measuring motivation to volunteer in human services. Journal of Applied Behavioral Science, 27, 269-284. Trudeau, K. J., & Devlin, A. S. (1996). College students and community service: Who, with whom, and why! Journal of Applied Social Psychology, 26,1867-1888.

Scale If you are currently volunteering, please indicate to what extent each motive contributed to your decision to volunteer. If you have not volunteered please indicate to what extent each motive would influence your future decision to volunteer. Please rate each item on the following 5-point scale.

42

RESEARCH SCALES

1 not important at all

2

3

4

5 very important

1. It is God's expectation that people will help each other. 2. I adhere to the agency's specific goals. 3. If I did not volunteer there would be no one to carry out this volunteer work. 4. I did not have anything else to do with my time. 5. I was lonely. 6. I have more free time (i.e., kids have left home, retired, widowed, divorced). 7. I wanted to gain some practical experience toward paid employment (or new career). 8. I wanted to broaden my horizons. 9. Being involved with this agency is considered prestigious. 10. Volunteering for others makes me feel better about myself. 11. Volunteering in this agency provides challenging activities. 12. Most people in my community volunteer. 13. Helping people in need improves my attitude regarding my own life situation. 14. Volunteering creates a better society. 15. My employer-school expect their employees-students to provide volunteer community service. 16. Volunteering is an opportunity to change social injustices. 17. Volunteering is an opportunity to develop relationships with others. 18. Volunteering is an opportunity to work with different age groups. 19. Volunteering is an opportunity to do something worthwhile. 20. Volunteering is an opportunity to return good fortune. 21. A relative or friend is/was a client of this agency. 22. I have past experience providing similar service. 23. I am able to relate better to the patients/residents situation because of my own similar experience. 24- This volunteering gives me an opportunity to vary my weekly activities. 25. Previous contact with professionals in this agency. 26. Volunteering for this agency enables it to provide more care for less money. 27. It's a way to continue a family tradition of helping in need. 28. This is an excellent educational experience.

MOTIVES AND VALUES

43

Note. From the Motivation to Volunteer Scale. Copyright 1991 by Ram A. Cnaan and Robin S. Goldberg-Glen. Reprinted with permission of the authors. PUBLIC SERVICE MOTIVATION SCALE Source Perry, J. (1996). Measuring public service motivation: An assessment of construct reliability and validity. Journal of Public Administration Research and Theory, 6, 5-22. Construct Public service motivation represents one's "disposition to respond to motives grounded primarily or uniquely in public institutions" (Perry, 1993). The Public Service Motivation Scale (PSMS) assesses an individual's level on four motives for public service: attraction to public policy making, commitment to the public interest, self-sacrifice, and compassion. Description The PSMS uses 24 items to measure four dimensions associated with public service: • Attraction to Public Policy Making: the degree to which formulating good public policy is viewed as exciting, dramatic, and selffulfilling (three items); • Commitment to the Public Interest: the degree to which the person is motivated to pursue a career that allows serving in the public interest (five items); • Compassion: the degree to which an individual experiences moral and affective regard for needy persons (eight items); and • Self'Sacrifice: the degree to which a person puts the outcomes of public service before personal benefits (eight items). Estimated Time to Administer Fifteen to twenty minutes. Reliability Temporal Consistency No information has been located. 44

RESEARCH SCALES

Internal Consistency Coefficient alpha for the entire 24-item scale was .90. Coefficient alpha for each of the four subscales ranged from .69 to .74 (Perry, 1996). Factor Structure

A confirmatory factor analysis differentiated the four factors of motives for public service (Perry, 1996). Validity Known Groups No information has been located. Convergent Validity

The Self-Sacrifice subscale was positively related to self-reported performance (Perry, 1993, Table 3). In addition, the Compassion subscale was positively related to religious socialization, family socialization, and professional identification. Discriminant Validity Age was not significantly related to the Attraction to Policy Making, Compassion, or Self-Sacrifice subscales (Perry, 1997). Gender Differences Perry (1997) found that male respondents scored higher on the Commitment to the Public Interest and Self-Sacrifice subscales than did female respondents. Comments The nature of public service in this measure includes both government and nonprofit work. The original scale contained 40 items. However, after initial analyses were conducted, 5 items were dropped because of low variances and correlations to the overall scale. Also, 11 items that showed an association with multiple dimensions were eliminated. Even so, this scale presents an alternative means for tracking motives. Morton (1995) identified three types of service (personal, program, advocacy and policy), and this type of scale measures motives across all of these areas. The scale provides an interesting measure for studying motivation for career choices by students.

MOTIVES AND VALUES

45

References Morton, K. (1995). The irony of service: Charity, project, and social change in service-learning. Michigan journal of Community Service Learning, 2, 19-32. Perry, J. (1993, October). Public service motivation: Construct measurement and validation. Paper presented at the National Public Management Research Conference, Madison, WI. Perry, J. (1996). Measuring public service motivation: An assessment of construct reliability and validity, journal of Public Administration and Research, 6, 5-22. Perry, J. (1997). Antecedents of public service motivation. Journal of Public Administration Research and Theory, 7, 181-197.

Scale Rate on the following scale: 1 disagree

2

3

4

5 agree

1. Making a difference in society means more to me than personal achievements. 2. I am rarely moved by the plight of the underprivileged.* 3. Most social problems are too vital to do without. 4. It is difficult for me to contain my feelings when I see people in distress. 5. 1 believe in putting duty before self. 6. Doing well financially is definitely more important to me than doing good deeds.* 7. To me, patriotism includes seeing to the welfare of others. 8. Much of what I do is for a cause bigger than myself. 9. I seldom think about the welfare of people whom I don't know personally.* 10. Politics is a dirty word.* 11. Serving citizens would give me a good feeling even if no one paid me for it. 12. I am often reminded by daily events about how dependent we are on one another. 13. It is hard to get me genuinely interested in what is going on in my community.* 14- I feel people should give back to society more than they get from it. 15. I am one of those rare people who would risk personal loss to help someone else. 16. I unselfishly contribute to my community. 46

RESEARCH SCALES

.17. I have little compassion for people in need who are unwilling to take the first step to help themselves.* . 18. I am prepared to make enormous sacrifices for the good of society. . 19. The give and take of public policy making doesn't appeal to me.* . 20. Meaningful public service is very important to me. .21. I don't care much for politicians.* . 22. I would prefer seeing public officials do what is best for the whole community even if it harmed my interests. .23. 1 consider public service my civic duty. . 24- There are few public programs that I wholeheartedly support.* * reverse-scored Attraction to Public Policy (Items 10, 19, 21) Commitment to Public Interest (Items 13, 16, 20, 22, 23) Compassion (Items 2, 3, 4, 7, 9, 12, 17, 24) Self-Sacrifice (Items 1, 5, 6, 8, 11, 14, 15, 18)

Note. From the Public Service Motivation Scale. Copyright 1996 by James L. Perry. Reprinted with permission of the author.

THE GOAL AND MODE VALUES INVENTORIES Source Braithwaite, V. A., & Law, H. G. (1985). Structure of human values: Testing the adequacy of the Rokeach Value Survey. Journal of Personality and Social Psychology, 49, 250-263. Construct A revision of the Rokeach Values Survey (Rokeach, 1973, 1979), the Goal and Mode Values Inventories measure two components: • Personal goals: international harmony and equality, national strength and order, traditional religiosity, personal growth and harmony, physical well-being, secure and satisfying interpersonal relationships, social standing, and social stimulation; and • Modes of conduct: a positive orientation to others, competence and effectiveness, propriety in dress and manners, religious commitment, assertiveness, and getting ahead. MOTIVES AND VALUES

47

Description The Rokeach Values Survey (Rokeach, 1973, 1979) requires respondents to rank order 18 end states of existence (terminal values) and 18 modes of conduct (instrumental values) in terms of each value's importance as a guiding principle in life. Both the response format and the item content have been criticized (see Braithwaite & Law, 1985, for a summary). Braithwaite and Law used a community sample to expand the content domain of the items and used factor analyses to organize items into personal goals, modes of conduct, and social goals. They also used a 7-point asymmetrical response format. For personal goals and modes of conduct, the following response choices were used: 1=1 reject this as a guiding principle in my life. 2 = 1 am inclined to reject mis as a guiding principle in my life. 3 = 1 neither reject nor accept this as a guiding principle in my life. 4 = 1 am inclined to accept this as a guiding principle in my life. 5=1 accept this as important as a guiding principle in my life. 6 = 1 accept this as very important as a guiding principle in my life. 1 = I accept this as of the greatest importance as a guiding principle in my life. Because social goals do not apply directly to one's daily life, a variation is used for that response format that refers to principles that guide your judgments and actions. Estimated Time to Administer Twenty to thirty minutes. Reliability Temporal Consistency Braithwaite and Law (1985) reported the following.

Variable 1. International harmony and equality 2. National strength and order 3. Personal growth and inner harmony 4. Physical well-being

48

RESEARCH SCALES

No. items

Test-retest reliability (over 4-week period)

10 4

.73 .81

6 3

.70 .71

5. Secure and satisfying interpersonal relationships 6. Social standing 7. Social stimulation 8. Traditional religiosity 9. Religious commitment 10. Positive orientation toward others 11. Competence and effectiveness 12. Propriety in dress 13. Assertiveness 14. Getting ahead

5 3 2 4 4

.71 .77 .58 .93 -81

13

.80

13 7 3 2

.81 .84 .68 .72

Internal Consistency

Braithwaite and Law (1985) reported the following. Variable 1. International harmony and equality 2. National strength and order 3. Personal growth and inner harmony 4. Physical well-being 5. Secure and satisfying interpersonal relationships 6. Social standing 7. Social stimulation 8. Traditional religiosity 9. Religious commitment 10. Positive orientation toward others 11. Competence and effectiveness 12. Propriety in dress 13. Assertiveness 14. Getting ahead

No. items

Alpha reliability

10 4

.85, .83 .81, .83

6 3

.74, -73 .79, .74

5 3 2 4 4

.70, .68 .70, .65 .53, .53 .75, .70 .66, .66

13

.89, .88

13 7 3 2

.89, .83, .67, .66,

.88 .82 .72 .59

Factor Analysis

The identification of subscales was based on factor analyses that replicated reasonably well across different samples (Braithwaite & Law, 1985).

MOTIVES AND VALUES

49

Validity Known Groups The Goal and Mode Inventories were found to distinguish between supporters of left- and right-wing economic beliefs (Heaven, 1991). Convergent Validity Braithwaite and Law (1985) used items from the Rokeach Value Survey as marker items and correlated them with the obtained scales. The correlations supported the correspondence between many of Rokeach's items and the obtained scales. Discriminant Validity No information has been located. Gender Differences No information has been located. Comments Braithwaite and Law's Goal and Mode Values Inventories appear to improve on the Rokeach Value Survey both in form and content. The shift from ranking to rating is an improvement because it shifts the assessment from relative differences to an absolute measure. Thus, using Rokeach's method, two very closely rated values that are adjacent in importance for a respondent receive an N and N - 1 ranking. The degree of similarity (or difference) is not detectable with a ranking. However, Braithwaite and Law's response format assesses those similarities and differences through a rating procedure. The content of this scale can, generally, also be regarded as an improvement. Whereas Rokeach's domain was "intuitively" selected (quoted in Braithwaite & Law, 1985, p. 251), Braithwaite and Law conducted intensive semistructured interviews with 115 adults selected from the electoral roll from a demographically heterogeneous section of Brisbane, Australia. Furthermore, they used appropriate psychometric criteria for sorting through a large pool of items to obtain the resulting scale. There are some discrepancies between the Rokeach Value Survey and Braithwaite and Law's inventory. For example, Physical Well-Being had no counterpart in the Rokeach Value Survey. Also, there were numerous items from Rokeach's scale that did not result in multiple item scales in Braithwaite and Law's inventory. 50

RESEARCH SCALES

Although the scale represents an improvement, there are some reservations about its use. There is rather limited construct validity evidence for the components of the inventory. Several of the constructs measured with two or three items have low coefficient alphas and low temporal stability. However, some of the constructs may be of interest to those conducting research on service learning. The Positive Orientation to Others scale might be an appropriate index of how service has deepened students' regard for others (e.g., tolerance, forgiving, considerate, trusting, generosity). The International Harmony and Equality scale could fit an international service experience or a domestic service experience with culturally distinct groups. The Traditional Religiosity and Religious Commitment scales can be used to explore spiritual development and its relationship to volunteering during college. No research was found that determines the degree to which these scales are measuring a socially desirable response set. Only one item is stated in a negative manner that requires reverse scoring; therefore, researchers should be aware of this possibility. References Braithwaite, V. A., & Law, H.G. (1985). Structure of human values: Testing the adequacy of the Rokeach Value Survey. Journal of Personality and Social Psychology, 49, 250-263. Heaven, P. L. (1991). Economic beliefs and human values: Further evidence of the two-value model? Journal of Social Psychology, 130, 583-589. Rokeach, M. (1973). The nature of human values. New York: Free Press. Rokeach, M. (1979). Understanding human values. New York: Free Press.

Scale Part I Instructions: Listed below are 23 goals that various people have used as guiding principles in their lives. By goal, we mean any state of affairs that a person may wish to preserve or keep as is. Please indicate the extent to which you accept or reject each of these goals as a principle for you to live by. Do this by circling one of the following numbers which you will find listed below each question.

1

2

I reject this

I am inclined to reject this

3 I neither reject nor accept

4 I am inclined to accept this

5 I accept this as important

6 I accept this as very important

7 I accept this as of the greatest importance

MOTIVES AND VALUES

51

You are to circle the one that is closest to your own feelings about that particular goal as a guiding principle in your life. Before you start, quickly read through the entire list of goals in Part 1 to get a feel for how to score your answers. Remember there are no right or wrong answers. When you have completed Part 1 go on to Part 2 and then Part 3. TRADITIONAL RELIGIOSITY Salvation (being saved from your sins and at peace with God) Religious or Mystical Experience (being at one with God or the universe) Upholding Traditional Sexual Moral Standards (opposing sexual permissiveness and pornography) Sexual Intimacy (having a satisfying sexual relationship)* PERSONAL GROWTH AND INNER HARMONY Self-Knowledge or Self-Insight (being more aware of what sort of person you are) The Pursuit of Knowledge (always trying to find out new things about the world we live in) Inner Harmony (feeling free of conflict within yourself) Self-Improvement (striving to be a better person) Wisdom (having a mature understanding of life) Self-Respect (believing in your own worth) PHYSICAL WELL-BEING Physical Development (being physically fit) Good Health (physical well-being) Physical Exercise (taking part in energetic activity) SECURE AND SATISFYING INTERPERSONAL RELATIONSHIPS Mature Love (having a relationship a deep and lasting affection) True Friendship (having genuine and close friends) Personal Support (knowing that there is someone to take care of you) Security for Loved Ones (taking care of loved ones) Acceptance by Others (feeling that you belong) SOCIAL STANDING Recognition by the Community (having high standing in community) Economic Prosperity (being financially well off) Authority (having power to influence others and control decisions) 52

RESEARCH SCALES

SOCIAL STIMULATION An Active Social Life (mixing with other people) An Exciting Life (a life full of new experiences or adventures) * reverse-scored Note. Only the items that belong to the scales are reproduced here. Part 2

Instructions: Below are listed 42 ways of behaving. Please indicate the extent to which you accept or reject each way of behaving as a guiding principle in your life, in the same way as you did in Part 1. Again quickly read through the entire list before you start. Remember there are no right or wrong answers.

1

2

I reject this

I am inclined to reject this

3 I neither reject nor accept

4 I am inclined to accept this

5 I accept this as important

6 I accept this as very important

7 I accept this as of the greatest importance

A POSITIVE ORIENTATION TO OTHERS Tolerant (accepting others even though they may be different from you) Helpful (always ready to assist others) Forgiving (willing to pardon others) Giving Others a Fair Go (giving others a chance) Tactful (being able to deal with touchy situations without offending others) Considerate (being thoughtful of other people's feelings) Cooperative (being able to work in harmony with others) Loving (showing genuine affection) Trusting (having faith in others) Grateful (being appreciative) Understanding (able to share another's feelings) Friendly (being neighborly) Generous (sharing what you have with others) COMPETENCE AND EFFECTIVENESS Bright (being quick thinking) Adaptable (adjusting to change easily) Competent (being capable) Resourceful (being clever at finding ways to achieve a goal) Self-Disciplined (being self-controlled) Efficient (always using the best method to get the best results) Realistic (seeing each situation as it really is) MOTIVES AND VALUES

53

Knowledgeable (being well informed) Persevering (not giving up in spite of difficulties) Progressive (being prepared to accept and support new things) Conscientiousness (being hardworking) Logical (being rational) Showing Foresight (thinking and seeing ahead) PROPRIETY IN DRESS AND MANNERS Polite (being well-mannered) Patriotic (being loyal to your country) Prompt (being on time) Refined (never being course or vulgar) Clean (not having dirty habits) Neat (being tidy) Reliable (being dependable) RELIGIOUS COMMITMENT Committed (being dedicated to a cause) Devout (following your religious faith conscientiously) Self-Sacrificing (putting the interest of others before your own) Idealistic (living according to how things should be rather than or in addition to how things are) ASSERTIVENESS Standing Up for Your Beliefs (defending your beliefs no matter who opposes them) Having Your Say (confidently stating your opinions) Determined (standing by your decisions firmly) GETTING AHEAD Ambitious (being eager to do well) Competitive (always trying to do better than others) Part 3 Instructions: Below are 14 goals that refer to our society, our nation, and to people in general. Although most of us do not directly affect the course of national affairs, we all have principles or standards we prize highly in our society. We use these standards to make judgments about national policies and about world and community events, and at times, we may use them to guide our actions (e.g., when we join certain organizations or when we vote in elections). Please indicate the extent to which you reject or accept each of the following as principles that guide your judgments and actions, in the same 54

RESEARCH SCALES

way as you did previously. Again, quickly read through the entire list before you start. Remember there are no right or wrong answers.

1

2

I reject this

I am inclined to reject this

3 I neither reject nor accept

4 1 am inclined to accept this

5 I accept this as important

6 I accept this as very important

7 I accept this as of the greatest importance

INTERNATIONAL HARMONY AND EQUALITY A Good Life for Others (improving the welfare of all people in need) Rule by the People (involvement by all citizens in making decisions that affect their community) International Cooperation (having all nations working together to help each other) Social Progress and Social Reform (readiness to change our way of life for the better) A World at Peace (being free from war and conflict) A World of Beauty (having the beauty of nature and the arts: music, literature, art, etc.) Human Dignity (allowing each individual to be treated as someone of worth) Equal Opportunity for All (giving everyone an equal chance in life) Greater Economic Equality (lessening the gap between the rich and the poor) Preserving the Natural Environment (preventing the destruction of nature's beauty and resources) NATIONAL STRENGTH AND ORDER National Greatness (being a united, strong, independent, and powerful nation) National Economic Development (having greater economic progress and prosperity for the nation) The Rule of Law (punishing the guilty and protecting the innocent) National Security (protection of your nation from enemies) Note. From the Goal and Mode Values Inventories. Copyright 1985 by Valerie A. Braithwaite. Reprinted with permission of the author.

SURVEY OF INTERPERSONAL VALUES Source Gordon, L. V. (1960). Survey of Interpersonal Values. Chicago: Reid London House. MOTIVES AND VALUES

55

The cost to order the Survey of Interpersonal Values ranges from $54-00 to $64.00 per package of 25 tests, depending on the amount of packages purchased. The scoring stencil is available for $20.00, profile sheets (100 per package) are available for $70.00, and the Examiners Manual is available for $28.00. A start-up kit can be purchased at a discount and includes the Examiner's Manual, 25 test booklets, 100 profile sheets, and the scoring stencil. The survey can be obtained by contacting the publisher: Reid London House http://www.reidlondonhouse.com/reidlondonhouse/tests/siv.html [email protected] (800) 922-7343 (312) 938-9200 Construct The Survey of Interpersonal Values (SIV) measures an individual's values by asking the respondent to indicate the relative importance of various statements about social relationships. Description The SIV consists of 30 sets of three statements or triads. Each statement within each set represents a different value dimension. Within each set, respondents are asked to rate what they feel are the most important and least important statements. Scoring consists of assigning a "2" to items marked as most important, "1" if unmarked, and "0" if indicated as least important. Scale scores are obtained from the sum of the statement weights. The SIV measures six values: • Support: the feeling of receiving understanding, encouragement, and kindness from others; • Conformity: acting in a politically correct manner, following the rules; • Recognition: feeling of being admired and important; • Independence: freedom to make one's own decisions; • Benevolence: helping others, being generous; and • Leadership: having authority over others, holding a position of power. Estimated Time to Administer Fifteen minutes. 56

RESEARCH SCALES

Reliability Temporal Consistency

Based on a college student sample, test-retest reliability (10 days) was found to range from .78 (Recognition) to .89 (Independence). The testretest reliability for intervals ranging from 12 weeks to 1 year for samples of Peace Corps volunteers, hospital corps medics, and medical students ranged from .55 (Recognition and Leadership) to .82 (Independence), with most correlations above .64 (Gordon, 1960). Internal Consistency Coefficient alphas for the SIV based on samples of high school and college students ranged from .71 (Recognition) to .86 (Independence, Benevolence, Conformity) (Gordon, 1960). Factor Structure

Gordon (1976) stated that factor analysis was used in the development of the scales and that it supports the six-scale structure. Kikuchi and Gordon (1966) identified the same six scales in a factor analysis of the measure in a Japanese population. Validity Known Groups

Gordon (1976) reported a 1963 study that used the SIV to evaluate differences among resident fraternity members, students in dormitories, and off-campus students. Fraternity members scored significantly higher on the Recognition scale and lower on the Benevolence and Independence scales than did students residing off-campus or in dormitories. Convergent Validity

The SIV correlates significantly with other values and personality construct measures such as the Edwards Personal Preference Schedule (Edwards, 1959), Leary Interpersonal Checklist (Leary, 1957), and Work Environment Preference Schedule (Gordon, 1973). Further validation studies and correlational tables are provided in Gordon (1976). Discriminant Validity Gordon (1976) reported that the SIV is largely unrelated to cognitive ability measures such as the General Aptitude Test Battery (Hartigan &. Wigdor, 1989), Junior Scholastic Aptitude Test, and Watson-Glaser MOT7VES AND VALUES

57

Critical Thinking Appraisal (Watson & Glaser, 1980). Further validation studies and correlational tables are provided in Gordon (1976). Gender Differences Gordon (1976) reported that typically men score higher on the Recognition, Independence, and Leadership scales and women score higher on the Support, Conformity, and Benevolence scales. Comments The SIV provides a measure of a spectrum of values that may be of interest to researchers. Each of the scales represents an attribute of students that could potentially be related to how and why students are engaged in service experiences and how they are affected by the experiences. References Edwards, A. L. (1959). Manual for the Edwards Personal Preference Schedule—Revised. New York: Psychological Corporation. Gordon, L.V. (1960). Survey of Interpersonal Values. Chicago: Science Research Associates. Gordon, L. V. (1973). Manual, Work Environment Preference Schedule. New York: Psychological Corporation. Gordon, L.V. (1976). Survey of Interpersonal Values. Chicago: Science Research Associates. Hartigan, J. A., & Wigdor, A. K. (Eds.). (1989). Fairness in employment testing: Validity, generalization, minority issues, and the General Aptitude Test Battery. Washington, DC: National Academic Press. Kikuchi, A., & Gordon, L. V. (1966). Evaluation and cross-cultural application of a Japanese form of the Survey of Interpersonal Values, journal of Social Psychology, 69, 185-195. Leary, T. (1957). Manual for interpersonal diagnosis of personality. Berkeley, CA: Kaiser Permanente. Watson, G., & Glaser, E. (1980). Watson-GIoser Critical Thinking Appraisal. San Antonio, TX: Psychological Corporation.

Sample Items Examine each set. Within each set of statements, find the one statement of the three which represents what you consider to be most important to you. Select the box beside that statement in the column headed M (most). Of the two remaining statements, choose the one which represents what you 58

RESEARCH SCALES

consider to be least important to you. Select the box beside that statement in the column headed L (least). 1. To To To 2. To To To

be free to do as I choose have others agree with me make friends with the unfortunate be in a position of not having to follow orders follow rules and regulations closely have people notice what I do

M D D D D D D

L D D D D D D

PERSONAL SOCIAL VALUES Source Mabry, J. B. (1998). Pedagogical variations in service-learning and student outcomes: How time, contact, and reflection matter. Michigan Journal of Community Service Learning, 5, 32-47. Construct The Personal Social Values scale assesses personal social values as they relate to community service. Description The 4-item scale assesses the personal importance of influencing social values and helping those in need. Responses on the first three items are rated on a 4-point scale ranging from not important to essential. The fourth item is rated on a 5-point scale ranging from lowest 10% to Highest 10%. Estimated Time to Administer Five minutes. Reliability Temporal Consistency No information has been located. Internal Consistency Coefficient alpha for the Personal Social Values scale was .63 prior to the service learning course and .78 at the end of the course (Mabry, 1998). MOTIVES AND VALUES

59

Factor Structure Factor analysis with varimax rotation found a single factor (Mabry, 1998). Validity Known Groups Male respondents showed significant increases in personal social values from pre- to postservice learning experience. Although there were no significant changes for female respondents, the findings demonstrated a positive change trend (Mabry, 1998). Convergent Validity No information has been located. Discriminant Validity No information has been located. Gender Differences Female respondents scored significantly higher on the precourse personal social values scale than did male respondents (Mabry, 1998). Comments One of the most important objectives of service leaning is "learning to serve." Mabry's Personal Social Values scale is a short and simple measure of global motives and values associated with helping that was developed as part of research on reflection in service learning. To date, it has rather sparse psychometric support, but the nature of the scale and evidence in the initial study suggest that it can be a useful research instrument. References Mabry, J. B. (1998). Pedagogical variations in service-learning and student outcomes: How time, contact, and reflection matter. Michigan Journal of Community Service Learning, 5, 32-47.

Scale Indicate the importance to you personally of: 60

RESEARCH SCALES

Not important

1 . Influencing social values.3 2. Finding a career that provides an opportunity to be helpful to others or useful t o society.13 3 . Helping others w h o a r e in difficulty.3

Somewhat important

1

Essential

2

3

4

2

3

4 34

Average

Above average

Highest 10%

3

4

5

1 1

Very important

2

Rate yourself compared to others your age on: Lowest 10%

4 . Commitment t o serving your community.3

1

Below average

2

Items in the above scale come from the following sources as indicated: Astin, A., & Sax, L. (1996). Cooperative Institutional Research Program (CIRP) freshman survey. Los Angeles: University of California, Los Angeles, Higher Education Research Institute. b Markus, G., Howard, J., & King, D. (1993). Integrating community service and classroom instruction enhances learning: Results from an experiment. Educational Evaluation and Policy Analysis, 15, 410-419.

a

Note. From "Pedagogical Variations in Service-Learning and Student Outcomes: How Time, Contact, and Reflection Matter," by J. B. Mabry, 1998, Michigan Journal of Community Service Learning, 5, p. 46 (Appendix 1). Copyright 1998 by OCLS Press, University of Michigan. Reprinted with permission.

MOTIVES AND VALUES

61

5 MORAL DEVELOPMENT

DEFINING ISSUES TEST Source Rest, J. (1990). DIT Manual (3rd ed.). Minneapolis, MN: Center for the Study of Ethical Development. Rest, J., & Narvaez, D. (1998). Supplement to guide for DIT-l. Minneapolis, MN: Center for the Study of Ethical Development. An information package containing general information about the Defining Issues Test (DIT), Center for the Study of Ethical Development, recent publications and papers, and procedures for ordering and scoring the DIT is available from: Center for the Study of Ethical Development 206 Burton Hall 178 Pillsbury Dr. University of Minnesota Minneapolis, MN 55455 Each use of the DIT must be cleared through the Center. Predoctoral students may use the DIT at no cost by copying the scale from the Manual and hand scoring the scale. A Manual for the DIT is available for $25.00. To control use and prevent abuse, orders for the scale must be accompanied by a letter that includes the following information: 63

1. Name, address, and telephone number of the investigator(s) 2. Institutional affiliation 3. Characteristics of the sample to be tested (sample size, demographics) 4. Purpose of using the scale (main hypotheses) Copies of the scale must be ordered from the Center. As an example of price, 100 copies of the DIT, which includes printed answer sheets, instruction booklets, scoring service, mailing, a copy of the Guide and Report, costs $181 plus 10% for shipping and handling. Construct The DIT measures moral judgment by asking respondents to evaluate various ethical considerations including action and reasons behind a choice that is related to moral dilemmas. The moral judgments that are presented to respondents are based on Kohlberg's (1984) theory of moral development (Rest, 1986). Description Respondents are presented with six moral dilemmas (three moral dilemmas can be used in a shortened version). For each dilemma, 12 statements are presented that represent different levels of moral judgment. Each statement is rated in terms of its importance to the decision that the respondent has made about the dilemma. The respondent then selects from among the 12 statements the four most important issues and ranks them from most important to least important. Scoring responses can yield many different indices of moral judgment. The current recommendation is that the P score or the N2 score be used. The P score is the weighted sum of ranks given to Stage 5 and Stage 6 items and is an indication of the importance the respondent ascribes to moral reasoning at the principled level. The N2 score is an improvement on the P scoring procedure and is based on equal weighting of each story and weighting the rating data three times as much as the ranking data. A meta-analysis (i.e., a statistical evaluation of the patterns of past research) indicates that N2 scores outperform P scores in reanalyses of research (see Rest & Narvaez, 1998). Validity Scores The DIT has two methods of checking the validity of scores obtained from respondents. An M score is obtained from item choices that are "lofty 64

RESEARCH SCALES

sounding but meaningless" and is indicative of the respondents' tendency "to endorse statements for their pretentiousness rather than or in addition to meaning" (Rest, 1990, p. 4.2). It is recommended that respondents with raw scores two standard deviations above the mean (8 or higher) be discarded from subsequent analyses. A second validity score is obtained from a computation of the respondent's consistency. For both too much inconsistency and too much consistency, it is recommended that a respondent be deleted from subsequent analyses. Estimated Time to Administer Thirty to forty minutes. Reliability Temporal Consistency Rest (1990) reported that test-retest reliabilities gathered from studies conducted by Davison and Robbins (1978) for the P and D scores are in the high .70s or .80s. Internal Consistency

Rest (1990) reported that coefficient alpha for the P and D scores are in the high .70s (Davison & Robbins, 1978). Factor Structure

Analyses were conducted on the five stage scores yielding support for the proposed hierarchical structure of the instrument (Ma & Cheung, 1996). Validity Known Groups Statistically significant (p < .05) differences were found among groups of PhD students in moral philosophy and political science, ninth graders, and other high school and college students (Rest, 1990). McGeorge (1975) manipulated instructions to respondents. One group was instructed to "fake good" and respond to the scale in such a way as to show "the highest principles of justice." A second group was instructed to "fake bad." A third group was given the standard instructions for the scale. The results indicated that "fake bad" instructions produced lower scores than the other two groups, but the "fake good" and control groups did not differ. On the basis of this and other research, Rest (1990) concluded that "the DIT is eliciting a person's best notion of justice and fairness" (p. 5.6). MORAL DEVELOPMENT

65

Convergent Validity Rest (1990) stated that correlations of the D1T with other measures of moral reasoning go up to the .60s and .70s, averaging about .50s, whereas correlations with cognitive development and intelligence measures range from the .20s to .50s (p. 5.5). Discriminant Validity The DIT has been shown to significantly predict behavior despite controlling for or partialing out other variables such as IQ, age, and socioeconomic status (Rest, 1990). Gender Differences Analyses suggest no significant gender differences exist on scores obtained from the DIT (Rest, 1979). However, White (1999) found that women scored significantly higher on the DIT than did men. Comments The DIT is a widely used measure of moral development. It is based on Kohlberg's (1984) theory, which has shaped the study of moral development. Although the subject matter of some of the items is dated (e.g., Vietnam War), it is still a good measure for those interested in using moral development as a construct in research. Most probably, this measure would be used as a moderator variable because moral development is viewed as a rather stable or traitlike construct. However, it could be used as an outcome variable if the time span or impact of the intervention warranted expected changes (e.g., across 4 years of college, the use of certain types of reflection activities). For example, Boss (1994) and Gorman (1994) found that students in service learning classes with structured discussions on moral issues related to their service experience scored higher on the DIT than students in a similar class without the service component. References Boss, J. A. (1994). The effect of community service work on the moral development of college ethics students. Journal of Moral Education, 23, 183-198. Davison, M., & Robbins, S. (1978). The reliability and validity of objective indices of moral development. Applied Psychological Measurement, 2, 391-403. Gorman, M. (1994). Service experience and the moral development of college students. Religious Education, 9, 422-431.

66

RESEARCH SCALES

Kohlberg, L. (1984). Essays on moral development: Vol. 2. The psychology of moral development. San Francisco: Harper & Row. Ma, H. K., & Cheung, C. (1996). A Cross-cultural study of moral stage structure in Hong Kong Chinese, English and Americans. Journal of Cross-Cultural Psychology, 27, 700-713. McGeorge, C. (1975). The susceptibility to faking of the Defining Issues Test of moral development. Developmental Psychology, 11, 108. Rest, J. R. (1979). Development in judging moral issues. Minneapolis: University of Minnesota Press. Rest, J. R. (1986). Moral development: Advances in research and theory. New York: Praeger Press. Rest, ]. R. (1990). DIT manual (3rd ed.). Minneapolis, MN: Center for the Study of Ethical Development. Rest, J. R., & Narvaez, D. (1998). Supplement to guide for DlT-l. Minneapolis, MN: Center for the Study of Ethical Development. White, R. D. (1999). Are women more ethical? Recent findings on the effects of gender upon moral development. Journal of Public Administration Research and Theory, 9, 459-471.

SOCIOMORAL RERECTION OBJECTIVE MEASURE Source Gibbs, J. C., Arnold, K. D., Morgan, R. L., Schwartz, E. S., Gavaghan, M. P., & Tappan, M. B. (1984). Construction and validation of a multiplechoice measure of moral reasoning. Child Development, 55, 527-536. The scoring manual for the SROM can be obtained from: John C. Gibbs Psychology Department Ohio State University 219 Townshend Hall Columbus, OH 43210 Construct The Sociomoral Reflection Objective Measure (SROM) provides a paper-and-pencil measure of Kohlbergian moral stages (Kohlberg, 1984). Description The SROM presents two moral dilemmas to respondents and a series of items corresponding to eight sociomoral norms: affiliation (marriage and

MORAL DEVELOPMENT

67

friendship), life, law, legal justice, conscience, family affiliation, contract, and property. Options are given that correspond to Kohlbergian sociomoral stages and a SROM score is computed. The SROM score is a continuous score ranging from 100 to 500 that can, if desired, be transformed into a global stage score (see scoring manual for transformation, p. 4). Estimated Time to Administer Fifty minutes. Reliability Temporal Consistency

Gibbs et al. (1984) reported a 2-week test-retest reliability of .82 for a sample of 7th-grade, lOth-grade, and college respondents. The reliability for the college respondents was .80. Internal Consistency

Gibbs et al. (1984) reported coefficient alphas of .84 and .77. Factor Structure

No information has been located. Validity Known Groups Gibbs et al. (1984) found significant differences between a group of male and female delinquents (9th-, 10th-, and llth-grade respondents) and a comparably aged nondelinquent group. The difference was still significant after controlling for the effects of age and socioeconomic status, but not after controlling for IQ. Convergent Validity

The SROM results were compared with the Moral Judgment Interview (Colby, Kohlberg, Gibbs, & Lieberman, 1983), an interview procedure that probes respondents' moral reasoning regarding moral dilemmas. A correlation of .66 was found (Gibbs et al., 1984). The SROM correlated .77 and .73 for two samples that were given the Sociomoral Reflection Measure (Gibbs & Widaman, 1982), a "production-task measure of moral reasoning whereby subjects must justify their prescriptive decisions regarding moral problems and associated normative values" (p. 529). 68

RESEARCH SCALES

Discriminant Validity The SROM had a nonsignificant correlation with the Crowne and Marlowe (1964) measure of social desirability (Gibbs et al., 1984). Gender Differences Gibbs et al. (1984) reported that there were no gender main effects or interactions. Comments Like the DIT, the SROM may be useful in research studying the effects of community service taking place over a long period of time (e.g., 4 years of college). The SROM provides an alternative to the DIT for measuring Kohlbergian moral development. The SROM score is continuous but can be converted to stage designations. There are four steps involved in scoring the SROM. These are described in detail in the scoring manual along with a copy of the score sheet. Because of the detailed instructions involved in scoring the SROM, it is recommended that a copy of the manual and scoring sheet be obtained from the authors for use of this scale. Two related measures of moral judgment maturity are also available (for a review, see Gibbs, Basinger, & Grime, in press). If administration time is at a premium, using the Sociomoral Reflection Objective Measure— Short Form (SROM-SF; Basinger & Gibbs, 1987) is an option. The Sociomoral Reflection Measure—Short Form (SRM-SF; Basinger, Gibbs, & Fuller, 1995), which is an 11-item production measure and is presented in Gibbs, Basinger, and Fuller (1992), makes the lowest reading demand. References Basinger, K. S., & Gibbs, J. C. (1987). Validation of the Sociomoral Reflection Objective Measure—Short Form. Psychological Reports, 61, 139-146. Basinger, K. S., Gibbs, J. C., & Fuller, D. (1995). Context and the measurement of moral judgment. International Journal of Behavioral Development, 18, 537-556. Colby, A., Kohlberg, L, Gibbs, J. C., &. Lieberman, M. (1983). A longitudinal study of moral judgment. Monographs of the Society for Research in Child Development, 48(1-2, Serial No. 200). Crowne, D. P., & Marlowe, D. (1964). The approval motive. New York: Wiley. Gibbs, J. C., Arnold, K. D., Morgan, R. L., Schwartz, E. S., Gavaghan, M. P., & Tappan, M. B. (1984). Construction and validation of a multiple-choice measure of moral reasoning. Child Development, 55, 527-536. Gibbs, J. C., Basinger, K. S., & Fuller, D. (1992). Moral maturity: Measuring the development of sociomoral reflection. Hillsdale, NJ: Erlbaum.

MORAL DEVELOPMENT

69

Gibbs, J. C., & Widaman, K. F. (1982). Measuring the development of sociomoral reflection. Englewood Cliffs, NJ: Prentice-Hall. Gibbs, J. C., Basinger, K. S., & Grime, R. L. (in press). Moral judgment maturity. In S. J. Lopez & C. R. Snyder (Eds.), Handbook of positive psychological assessment. Washington, DC: American Psychological Association. Kohlberg, L. (1984). Essays on moral development: Vol. 2. The psychology of moral development. San Francisco: Harper & Row. Sample Items Instructions In this booklet are two social problems with questions for you to answer. We are asking the questions not just to find out your opinions about what should be done in the problems, but also to understand why you have those opinions. Please answer all the questions. Problem One In Europe, a woman was near death from a special kind of cancer. There was one drug that the doctors thought might save her. It was a form of radium that a druggist in the same town had recently discovered. The drug was expensive to make, but the druggist wanted people to pay ten times what the drug cost him to make. The sick woman's husband, Heinz, went to everyone he knew to borrow the money, but he could only get together about half of what the druggist wanted. Heinz told the druggist that his wife was dying, and asked him to sell it cheaper or to let him pay later. But the druggist said, "No, I discovered the drug and I'm going to make money from it." So the only way Heinz could get the drug would be to break into the druggist's store and steal the drug. Heinz has a problem. He should help his wife and save her life. But, on the other hand, the only way he could get the drug she needs would be to break the law by stealing the drug. What should Heinz do? should he steal / should he not steal / not sure (circle one) Why?

Let's change things about the problem and see if you still have the opinion you circled above (should steal, should not steal, or not sure). Also, 70

RESEARCH SCALES

we want to find out about the things you think are important in this and other problems, especially why you think those things are important. Please answer all the questions. 1. What if Heinz's wife asks him to steal the drug for her? Should Heinz: steal / not steal / not sure (circle one)? la. How important is it for a husband to do what his wife asks, to save her by stealing, even when he isn't sure whether that's the best thing to do? very important / important / not important (circle one)? Ib. Let's say you had to give a reason WHY it is IMPORTANT for a husband to do that. What reason would you give? Are any of the following reasons close to the one you would give? (If a reason is too hard to understand, seems silly, or makes no sense, just circle "not close," or "not sure.") a) because it's his wife, and she told him to do it, so he should do what his wife says. close / not close / not sure (circle one) b) because he married her and if he didn't want to help her, why did he marry her in the first place? close / not close / not sure (circle one) c) because they may have formed together a deep mutual commitment. close / not close / not sure (circle one) d) because the husband is expected to help his wife through sickness and health. close / not close / not sure (circle one) e) because he cannot recognize her without acceptance. close / not close / not sure (circle one) f) because he has accepted a responsibility as her husband. close / not close / not sure (circle one) Ic. Of all the above reasons, the reason which is the closest to the reason that you would give (or the least far off from the reason that you would give) is: a b c d e f (circle one) 2. What if the person dying isn't Heinz's wife, but instead is a friend (and the friend can get no one else to help)? Should Heinz: steal / not steal / not sure (circle one)? MORAL DEVELOPMENT

71

2a. How important is it to do everything you can, even break the law, to save the life of a friend? very important / important / not important (circle one) 2b. Let's say you had to give a reason WHY it is IMPORTANT for you to do that. What reason would you give? Are any of the following reasons close to the one you would give? (Treat these questions just as you did the last one. If a reason is too hard to understand, seems silly, or makes no sense, just circle "not close" or "not sure.") Note. From the Sociomoral Reflection Objective Measure (SROM). Copyright 1984 by John C. Gibbs. Reprinted with permission of the author.

MEASURE OF MORAL ORIENTATION Source Liddell, D. L, Halpin, G., & Halpin, W. G. (1992). The Measure of Moral Orientation: Measuring the ethics and care and justice. Journal of Colkge Student Development, 33, 325-330. Construct The Measure of Moral Orientation (MMO) was developed to measure the moral orientation for care, which is based on the work of Gilligan (1978), and justice, which is based on the work of Kohlberg (1971). Description The MMO is divided into two components, the first of which contains nine moral dilemmas that are typical of traditional-age college students. Each dilemma consists of a short story of conflict, followed by questions regarding possible solutions. Respondents are asked to respond to each dilemma using a 4-point response scale anchored with 1 = strongly agree and 4 = strong^ disagree. Across the nine dilemmas, 69 items are included in the measure: 34 items address issues of justice and 35 items address care issues. In addition, a 14-item self-descriptive section provides an assessment of a respondent's thoughts and feelings of being caring or just. Thus, scores are obtained in four areas: Care, Justice, Self-Description of Care, and SelfDescription of Justice. Scores are summed across responses with higher scores indicating stronger orientations. 72

RESEARCH SCALES

Time of Administration Twenty-five minutes. Reliability Temporal Consistency Liddell and Davis (1996) reported the following 3-week test-retest reliability: Care (.85), Justice (.79), Self-Description of Care (.70), and SelfDescription of Justice (.67). Internal Consistency Coefficient alphas for each of the four subscales were computed from two large samples by Liddell, Halpin, and Halpin (1992) and Liddell and Davis (1996): Care (.84 and .83, respectively), Justice (.73 and .70), SelfDescription of Care (.59 and .70), and Self-Description of Justice (.60 and .67). Factor Structure Liddell and Davis (1996) computed a Care and a Justice score for each of the nine moral dilemmas and conducted a factor analysis on those scores forcing a two-factor solution and using a least-squares fit. The factor loadings for the first factor generally supported the a priori classification of the caring items: All loadings were above .30, and only one item had a higher loading on the second factor. However, the factor loadings for the second factor failed to support the conclusion that they are clearly a coherent and distinct set of scores for the dilemmas that measure Justice: Only four of nine factor loadings were above .30, two were less than .10, and three scores had higher factor loadings on the first, rather than or in addition to, the second factor. The correlation between the two factors was .11, supporting the conclusion that they represent distinct constructs. No factor analysis was reported for the self-descriptive items. Validity Known Groups No information has been located. Convergent and Discriminant Validity In a comparison of the correlations between the four scales (Liddell & Davis, 1996; Liddell et al., 1992), the correlations between Care and Justice were .17 and .28, respectively, indicating discriminant validity. In MORAL DEVELOPMENT

73

addition, the correlations between the Care and Self-Description of Care (.32 and .50) and Justice and the Self-Description of Justice (.22 and .39) were generally higher, indicating convergent validity. The correlations between Self-Description of Care and Self-Description of Justice are high enough (-.44 and -.32) to indicate that these two scales share common method variance due to similar measurement procedures. Liddell et al. (1992) correlated the four scores from the MMO with a Feeling score from the Myers-Briggs Type Indicator (Myers & McCaulley, 1985). Feeling scores were found to differentiate the Care and Justice components of the MMO: Care (.36), Justice (-.13), Self-Description of Care (.34), and Self-Description of Justice (-.33). Liddell and Davis (1996) reported the following correlations between a World View Questionnaire (Stander & Jensen, 1993) that measures a person's preference for having a worldview that encompasses caring for others and the MMO scores: Care (.27), Justice (—.12), Self-Description of Care (.30), and Self-Description of Justice (-.43). The DIT (Rest, 1986) is derived from Kohlberg's theory and measures moral judgment—judging which action is morally right or wrong. The P score of the DIT "locates a subject in terms of a continuous number representing the developmental continuum (more like Kohlberg's Moral Maturity Index)" (Rest, 1990, p. 4-1). The MMO scores had the following correlations with the P score of the DIT: Care (-.23), Justice (-.37), Self-Description of Care (-.29), and Self-Description of Justice (-.27). These correlations are interpreted by Liddell and Davis as supporting the differentiation between both the theoretical intent of both scales and the operationalizations. They concluded that the DIT is a measure of moral judgment (i.e., deciding which judgment is morally right or wrong), whereas the MMO is a measurement of moral sensitivity (i.e., interpreting the situation). They also noted that the DIT "presents the reader with hypothetical moral stories that may be somewhat remote from the experience of contemporary college students" (Liddell & Davis, 1996, p. 488). In contrast, the MMO was designed to present dilemmas that are closer to the experience of college students, the dilemmas are written in the first person, and the respondent is asked to assume the role of the person in the story. Additionally, Liddell (1998) reported that scores on the Care and Justice constructs of the MMO were related to interview scores indicating strong Care and Justice voices, respectively. However, the latter relationship was not significant. Gender Differences Liddell, Halpin, and Halpin (1993) and Liddell and Davis (1996) reported that female respondents scored significantly higher on the Care scale. 74

RESEARCH SCALES

Comments The MMO provides an interesting alternative to the DIT as a measure related to moral development. The Care component of the MMO seems to reflect a distinct aspect of morality that, for some settings, might be viewed as a more likely candidate for outcomes in service learning classes (e.g., those that involve face-to-face contact) than the Judgment component measured by the DIT. Each can be appropriate, and researchers are advised to consider their differences when selecting between them. Based on the psychometric information to date, the Care component of the MMO seems to be better measured (e.g., internal consistency, temporal consistency, factor structure) than the Justice component. References Gilligan, C. (1978). In a different voice: Women's conception of self and morality. In Stage theories of cognitive and moral development (Reprint No. 13). Cambridge, MA: Harvard Educational Review. Kohlberg, L. (1971). From is to ought: How to commit the naturalistic fallacy and get away with it in the study of moral development. In T. Mischel (Ed.), Cognitive development and epistemology (pp. 151-235). New York: Academic Press. Liddell, D. L. (1998). Comparison of semistructured interviews with a quantitative measure of moral orientation. Journal of College Student Development, 39, 169— 178. Liddell, D. L, & Davis, T. L. (1996). The Measure of Moral Orientation: Reliability and validity evidence. Journal of College Student Development, 37, 485-493. Liddell, D. L., Halpin, G., & Halpin, W. G. (1992). The Measure of Moral Orientation: Measuring the ethics and care and justice. Journal of College Student Development, 33, 325-330. Liddell, D. L., Halpin, G., & Halpin, W. G. (1993). Men, women, and moral orientation: Accounting for our differences. NASPA Journal, 30, 138-144. Myers, I. B., & McCaulley, M. H. (1985). A guide to the development and use of the Myers-Briggs Type Indicator. Palo Alto: CA: Consulting Psychologists Press. Rest, J. R. (1986). Moral development: Advances in research and theory. New York: Praeger Press. Rest, J. R. (1990). D7T manual (3rd ed.). Minneapolis, MN: Center for the Study of Ethical Development. Stander, V., & Jensen, L. (1993). The relationship of value orientation to moral cognition: Gender and cultural difference in the U.S. and China explored. Journal of Cross-Cultural Psychology, 24, 42-52.

MORAL DEVELOPMENT

75

Sample Items Directions: Complete these questions on the computer answer sheet. Do not write in your name on the answer sheet. Do not write in the test booklet. Your responses will remain completely confidential. Read each dilemma and assume the role of the person in the dilemma. Rate each response which follows the dilemma as to whether you strong!} agree, somewhat agree, somewhat disagree, or strongly disagree with that response. There are no right or wrong answers to these questions. Your responses to these statements will help us determine how difficult decisions are made. Read each statement carefully, but don't spend too much time debating your responses. Usually our initial reaction is the most accurate one. Part I: Dilemmas

PARENTS My parents after 30 years of a somewhat rocky marriage, are going through a divorce. My mother has been involved with another man for several years and had decided to leave the marriage. She seems very happy with her decision. Each of my parents wants me to spend semester break at his or her particular home, but my father will be very upset if I go to my mother's because her "friend" will be there. 1 2 3 4

= strongfy agree = somewhat agree = somewhat disagree = strong!} disagree

1. I have the right to spend time with whomever I want. 2. I have an obligation to both my parents right now. Part II: Self-Description Please rate following statements as to how closely they describe your thoughts and feelings about yourself most of the time. 1 2 3 4

= strong!} agree = somewhat agree = somewhat disagree = strong^ disagree

3. In practically all situations, I make decisions based upon the principles and rules rather than, or in addition to, upon who is involved. 76

RESEARCH SCALES

Note. From the Measure of Moral Orientation. Copyright 1992 by Debora L. Liddle. Reprinted with permission of the author.

PROSOCIAL REASONING OBJECTIVE MEASUREADULT VERSION Source Carlo, G. (1997). Administration and scoring manual for the adolescent and adult versions of the Prosodal Reasoning Objective Measure (PROM and PROM-R). Lincoln: University of Nebraska, Department of Psychology. The manual and scale can be obtained from the author at the following address: Gustavo Carlo Department of Psychology University of Nebraska-Lincoln Lincoln, NE 68588-0308 Email: [email protected] Construct The Prosocial Reasoning Objective Measure—Adult Version (PROMR; Carlo, 1997) assesses prosocial moral reasoning on five categories of moral reasoning: hedonistic, needs oriented, approval oriented, stereotyped, and internalized. Description The PROM-R presents seven stories that describe moral dilemmas. Each story has a male or female character, so different versions are administered to male and female respondents. The respondent first indicates which behavioral choice the character should make and then rates how important each of nine reasons was in making that decision about what the character should do on a 7-point scale ranging from not at all to greatly. The nine reasons represent two each for hedonistic, approval oriented, and internalized, and one each for needs oriented and stereotypic categories. In addition, there is one item each to detect responses indicative of lying and respondents not paying attention to the item content. The PROM-R is scored by computing a raw score for each of the five response categories and then dividing each category total by the grand total across the five categories to obtain a proportion score. The proportion score MORAL DEVELOPMENT

77

is recommended over the raw score for analyses because it corrects for skewed distributions of raw scores within categories. Researchers should consider eliminating respondents who score two standard deviations above the mean on the lie/nonsense items (Carlo, Eisenberg, & Knight, 1992). Estimated Time to Administer Twenty to thirty minutes. Reliability Temporal Consistency

No assessment of temporal reliability of the PROM-R has been located. For the adolescent version of the PROM, 3-week test-retest reliabilities ranged from .70 to .79 (Carlo et al., 1992). Internal Consistency

Based on samples of Anglo American and Brazilian college students (Carlo, Roesch, &. Koller, 1999), the following coefficient alphas were obtained: hedonistic (.61 and .68, respectively), approval-oriented (.88 and .89), needs-oriented (.67 and .59), stereotypic (.70 and .75), and internalized (.70 and .57). Factor Structure

No information has been located. Validity Known Groups In regards to the adolescent version of the PROM, results showed expected age differences. Specifically, older participants reported more internalized reasoning and less approval orientations than younger participants (Carlo et al., 1992). Convergent Validity

Most of the validation data results are only available for the adolescent version of the PROM. There is a great deal of similarity between the adolescent PROM and the adult PROM-R; the differences are that the PROM-R (adult version) has a 7-point response format rather than a 5point response format, and the adult version has nine rather than six items that measure reasons. Carlo et al. (1992) found that the PROM internalized reasoning scores correlated with three scores from the Interpersonal Reactivity Questionnaire: perspective taking (.34), sympathy (.31), and personal 78

RESEARCH SCALES

distress when seeing the need for help (-.32). Carlo, Koller, Eisenberg, DaSilva, and Frohlich (1996) reported that peer ratings of helpfulness and generosity were positively related to internalized reasoning scores (.27) and negatively related to hedonistic reasoning scores (-.20). Discriminant Validity No zero-order or partial correlations (controlling for grade level and gender) were found between the PROM and social desirability (except for needs-oriented reasoning, which is negatively related to social desirability). Furthermore, no significant correlation was found between the PROM and vocabulary scores (Carlo et al., 1992). Gender Differences Carlo et al. (1999) contained two samples of both Anglo American and Brazilian adult respondents. In Study 1, for the two combined samples, male respondents reported significantly more approval-oriented reasoning and female respondents reported more stereotypic moral reasoning. In Study 2, for the combined samples, female respondents reported significantly more stereotypic and internalized moral reasoning compared with male respondents. In addition, male respondents reported marginally significant approval-oriented reasoning compared with female respondents. Comments The PROM-R is based on a scale with acceptable psychometric characteristics and provides an alternative measure for moral development that is not based on Kohlbergian theory (Kohlberg, 1984). The content areas of the PROM-R may be of particular interest within the context of service learning for tracking changes in students during one-semester courses or long involvement in a service learning curriculum. References Carlo, G. (1997). Administration and scoring manual for the adolescent and adult versions of the Prosocial Reasoning Objective measure (PROM and PROM-R). Lincoln: University of Nebraska, Department of Psychology. Carlo, G., Eisenberg, N., & Knight, G. P. (1992). An objective measure of adolescents' prosocial moral reasoning. Journal of Research on Adolescence ,2,331 -349. Carlo, G., Koller, S. H., Eisenberg, N., DaSilva, M. S., & Frohlich, C. B. (1996). A cross-national study on the relations among prosocial moral reasoning, gender role orientations, and prosocial behaviors. Developmental Psychology, 32, 231-240.

MORAL DEVELOPMENT

79

Carlo, G., Roesch, S. C., & Koller, S. (1999). Cross-national and gender similarities and differences in prosocial moral reasoning between Brazilian and EuropeanAmerican college students. Interamerican Journal of Psychology, 33, 151-172. Kohlberg, L. (1984). Essays on moral development: Vol. 2. The psychology of moral development. San Francisco: Harper & Row. Scale: Sample Story—Blood Donation Story A young man named Tony had a very unusual type of blood. One day right after Tony had begun school and was accepted on the baseball team, a doctor called Tony to ask him to give a large amount of blood to a boy who was very sick and needed more blood of the same kind as Tony's to get well. Because Tony was the only person in the town with the sick boy's type of blood, and since this was a rare and serious sickness, the blood would have to be given a number of times over a period of several weeks. So, if Tony agreed to give his blood, he would have to go into the hospital for several weeks. Being in the hospital would make Tony feel weak for a while, he would lose his spot on the team, and he would be very far behind in school or work. What should Tony do? (Check one) Tony should give blood Not sure Tony should not give blood How important were each of the following reasons in making your decision? Please rate each reason using the following 7-point scale (Circle your answers):

Not at all

1

Somewhat

2

3

4

Greatly

5

6

7 a. it depends whether Tony thinks that helping is nice or not

1

2

3

4

5

6

7 b. it depends on Tony's unidimensional approach to social classes

1

2

3

4

5

6

7 c. it depends whether Tony believes his friends or parents will like what he does

1

2

3

4

5

6

7 d. it depends whether Tony feels that losing his spot on the team is important or not (continuued)

80

RESEARCH SCALES

Not at all

1

Somewhat

2

3

4

5

Greatly

6

7 e. it depends whether Tony can understand how badly the other boy is feeling

1

2

3

4

5

6

7 f. it depends how sick the other boy will get

1

2

3

4

5

6

7 g. it depends whether Tony would feel badly because the person would still be ill

1

2

3

4

5

6

7 h. it depends whether his friends will be disappointed if he didn't help

1

2

3

4

5

6

7

i. it depends on how far behind Tony will get in school or work

Note. From Prosocial Reasoning Objective Measure—Adult Version. Copyright by Gustavo Carlo. Reprinted with permission of the author.

REVISED MORAL AUTHORITY SCALE Source White, F. A. (1997). Measuring the content of moral judgment development: The Revised Moral Authority Scales (MAS-R). Social Behavior and Personality, 25, 321-334. A copy of the entire scale is available from: Fiona White, Ph.D. School of Psychology University of Sydney Sydney 2006, Australia Email: [email protected] Construct The Revised Moral Authority Scales (MAS-R) measures the "attributed level of influence of different sources of moral authority in moral decision making" (White, 1997, p. 321). MORAL DEVELOPMENT

Description The MAS-R presents six moral issues. First, respondents are asked to indicate whether they agree with the issue as presented, and they are then asked to write a response to "why." Finally, using a 10-point scale anchored with no influence to strong influence, respondents rate the degree to which each of the following sources influenced their moral judgment: (a) selfinterest; (b) family; (c) teachers, friends, and the media; (d) society's welfare; and (e) equality. Scores can be computed across the six issues for each source of influence. Estimated Time to Administer Fifteen minutes. Reliability Temporal Consistency

White (1997) reported the following 4-week test-retest reliabilities: self-interest (.96), family (.98), education (.97), society's welfare (.95), and equality (.95). Internal Consistency

White (1997) reported the following coefficient alphas: self-interest (.88), family (.93), education (.93), society's welfare (.75), and equality (.82). Factor Structure

No information has been located. Validity Known Groups Some differences related to Australian political affiliation were found by White (1997): "Right wing and Left wing respondents attributed significantly more influence to the Society source in their moral decision making than did respondents claiming no political affiliation" (p. 331). Convergent Validity

White (1997) correlated the subscales of the MAS-R with the DIT (Rest, 1979) and the Vision of Morality Scale (Shelton & McAdams, 1990). The correlations were scattered and weak. Because of conceptual distinctions between the aspects of morality that are being measured by the respective 82

RESEARCH SCALES

scales, it is not clear if the weak correlations are evidence of convergent validity or if the weak correlations, along with the many nonsignificant correlations, are evidence of discriminant validity. Known Groups The MAS-R has been able to discriminate between various groups ascribing to different religious and political orientations (see White, 2000; White, Howie, & Perz, 2000). Gender Differences White et al. (2000) found that female respondents scored higher on both the Family and Equality sources of moral authority than did male respondents. Comments The MAS-R resulted from an attempt to improve the MAS developed by White (1996). The scales are very similar, except that the rating format was expanded from a l-to-5 to a l-to-10 format and some of the response choices were rewritten. Like the MAS, White's MAS-R was constructed to measure a distinct aspect of morality. Whereas the DIT items deal with Kohlbergian (Kohlberg, 1984) reasons for making a decision, the MAS items involve the subject's perception of the influence of Henry's (1983) sources of moral authority, thus allowing the content of that subject's moral judgments to be characterized. (White, 1996, p. 426)

Despite the weak construct validity evidence, the scale has some appealing qualities. The issues that are presented are done so in a straightforward and simple manner. The scale is simple to administer and score. It also provides a measure of the relative importance of different sources of influence on moral issues, an issue that might be of interest to service learning practitioners, particularly those conducting longitudinal research. References Henry, R. (1983). The psychodynamic foundations of morality. New York: McGraw-Hill. Kohlberg, L. (1984). Essays on moral development: Vol. 2. The psychology of moral development. San Francisco: Harper & Row. Rest, J. R. (1979). Development of judging moral issues. Minneapolis: University of Minnesota Press.

MORAL DEVELOPMENT

83

Shelton, C. M., & McAdams, D. P. (1990). In search of an everyday morality: The development of a measure. Adolescence, 25, 923-943. White, F. A. (1996). Sources of influence in moral thought: The new Moral Authority Scale. Journal of Moral Education, 25, 421-439. White, F. A. (1997). Measuring the content of moral judgment development: The Revised Moral Authority Scales (MAS-R). Social Behavior and Personality, 25, 321-334. White, F. A. (2000). The Revised Moral Authority Scale (MAS-R). In J. Maltby, C. A. Lewis, & A. P. Hill (Eds.), Commissioned reviews on 300 psychological tests (pp. 746-748). Lampeter, Wales, UK: Edwin Mellen Press. White, F. A., Howie, P., & Perz, J. (2000). Family processes and moral thought in two contrasting samples of school aged adolescents. Ethics and Behaviour, 10, 199-214. Scale: Sample Item 1. Should people who break the law (such as stealing, speeding, etc.) be punished? Yes / No / Can't decide (Please circle one) Why?

Rate the amount of influence of each statement on your opinion:

0

1

No influence

2

Almost no influence

3

4

Little influence

5 Moderate influence

6

7

Quite a strong influence

8

9

A very strong influence

10 A powerful influence

a) the idea that everyone should try to make a society a better place has on my opinion b) the idea the every person is worthy of some justice has opinion

on my

c) my family's beliefs and expectations about certain laws have my opinion

on

d) my friend's, the media and/or teacher's beliefs about certain laws have on my opinion e) the idea that it satisfies my own interests has f) (other)

84

RESEARCH SCALES

on my opinion

ETHICS POSITION QUESTIONNAIRE Source Forsyth, D. R. (1980). A taxonomy of ethical ideologies. Journal of Personality and Social Psychology, 39, 175-184. Construct

The Ethics Position Questionnaire (EPQ) was designed to "facilitate the classification of individuals according to ethical ideology" (Forsyth, 1980, p. 177). Description

The EPQ consists of two scales: Idealism and "rejection of moral principles in favor of Relativism" (Forsyth, 1980, p. 177). Respondents are asked to indicate their agreement with 20 items on a 9-point scale ranging from completely disagree to completely agree. Scores for each subscale are computed from the mean of the item ratings. Scores are then compared to categorize an individual into one of four types of approaches to ethical decision making: • Situationists: those high in both idealism and relativism. These individuals analyze each act in accordance to the individual situation. • Absolutists: high in idealism, low in relativism. These individuals always follow universal moral codes. • Subjectivists: low in idealism, high in relativism. Judgments are made on personal beliefs and perspective rather than or in addition to universal moral principles. • Exceptionists: low in both idealism and relativism. Judgments are based on moral rules. However, exceptions to the rules are allowed. Estimated Time to Administer

Fifteen minutes. Reliability Temporal Consistency

Based on a subgroup of the sample of college psychology students, 2-week test-retest reliabilities for the two subscales were .67 for Idealism and .66 for Relativism (Forsyth, 1980). MORAL DEVELOPMENT

85

Internal Consistency

Coefficient alphas for the two subscales were .80 for Idealism and .73 for Relativism (Forsyth, 1980). Factor Structure On the basis of item and factor analysis, Forsyth (1980) constructed a 27-item measure of the two subscales. Following further testing to delete items that were confusing or reduced internal consistency of the measure, 20 items representing the two scales were retained for the final scale. Validity Known Groups Older individuals scored lower on both the Idealism and Relativism scales than younger individuals, as predicted (Forsyth, 1980). Convergent Validity The Relativism score was correlated (-.31) with the Survey of Ethical Attitudes (Hogan, 1970). In other words, those individuals who refused to use a strict moral code for decisions tended to endorse behavior fitted to each specific situation (Forsyth, 1980, p. 181). Discriminant Validity Rest (1979) reported that scores on the EPQ did not correlate with scores on the DIT. This illustrates the uniqueness of the EPQ on identifying issues related to Idealism and Relativism in moral decision making and not reliance on self-generated values as the DIT measures. Gender Differences Forsyth and Nye (1990) found no differences in scores on the EPQ based on gender. Comments Forsyth and Nye (1990) suggested the typology of the EPQ assumes discontinuity in the concepts where none may exist. Nevertheless, the scale provides measures of Idealism and Relativism as aspects of moral development. Particularly for service learning classes that emphasize moral issues, measuring changes in these two constructs may be of interest. Furthermore, examining the differential influences of service learning experiences for those students with Idealism or Relativism might increase understanding of why different outcomes occur for different types of students and how to structure differently the service learning experiences for the two types of students. 86

RESEARCH SCALES

References Forsyth, D. R. (1980). A taxonomy of ethical ideologies. Journal of Personality and Social Psychology, 39, 175-184. Forsyth, D. R., &. Nye, J. L. (1990). Personal moral philosophies and moral choice. Journal of Research in Personality, 24, 398-414. Hogan, R. (1970). A dimension of moral judgment. Journal of Clinical and Counseling Psychology, 35, 205-212. Rest, J. (1979). Development in judging moral issues. Minneapolis: University of Minnesota Press.

Scale Instructions: You will find a series of general statements listed below. Each represents a commonly held opinion and there are no right or wrong answers. You will probably disagree with some items and agree with others. We are interested in the extent to which you agree or disagree with such matters of opinion. Please read each statement carefully. Then indicate the extent to which you agree or disagree by placing in front of the statement the number corresponding to your feelings, where: 1 2 3 4

= completely disagree = largely disagree 5 = neither agree or disagree = moderately disagree = slightly disagree

6 1 8 9

= slightly agree = moderately agree = largely agree = completely agree

1. A person should make certain that their actions never intentionally harm another even to a small degree. 2. Risks to another should never be tolerated, irrespective of how small the risks might be. 3. The existence of potential harm to others is always wrong, irrespective of the benefits to be gained. 4- One should never psychologically or physically harm another person. 5. One should not perform an action which might in any way threaten the dignity and welfare of another individual. 6. If an action could harm an innocent other, then it should not be done. 7. Deciding whether or not to perform an act by balancing the positive consequences of the act against the negative consequences of the act is immoral. MORAL DEVELOPMENT

87

8. The dignity and welfare of people should be the most important concern in any society. 9. It is never necessary to sacrifice the welfare of others. 10. Moral actions are those which closely match the ideals of the most "perfect" action. 11. There are no ethical principles that are so important that they should be part of any code of ethics. 12. What is ethical varies from one situation and society to another. 13. Moral standards should be seen as being individualistic; what one person considers to be moral may be judged to be immoral by another person. 14- Different types of moralities cannot be compared as to "rightness." 15. Questions of what is ethical for everyone can never be resolved since what is moral or immoral is up to the individual. 16. Moral standards are simply personal rules which indicate how a person should behave, and are not to be applied in making judgments of others. 17. Ethical considerations in interpersonal relations are so complex that individuals should be allowed to formulate their own individual codes. 18. Rigidly codifying an ethical position that prevents certain types of actions could stand in the way of better human relations and adjustment. 19. No rule concerning lying can be formulated; whether a lie is permissible or not permissible totally depends upon the situation. 20. Whether a lie is judged to be moral or immoral depends upon the circumstances surrounding the action. The idealism score is obtained by taking the mean of Items 1-10. The relativism score is obtained by taking the mean of Items 11-20. Note. From "A Taxonomy of Ethical Ideologies," by D. R. Forsyth, 1980, Journal of Personality and Social Psychology, 39, p. 178 (Table 2). Copyright 1980 by the American Psychological Association. Reprinted with permission. VISIONS OF MORALITY SCALE Source Shelton, C. M., & McAdams, D. P. (1990). In search of an everyday morality: The development of a measure. Adolescence, 25, 923-943. Construct The Visions of Morality Scale (VMS) measures moral thinking associated with everyday life. RESEARCH SCALES

Description The VMS is designed to "explore an empathic foundation for morality by relating everyday morality to various dimensions of empathy" (Shelton & McAdams, 1990, p. 926). The VMS comprises three subscales: • Private: prosocial behavior that occurs without the party aware of the behavior or no relationship existing with the beneficiary of the behavior, • Interpersonal: prosocial behavior that is aimed toward a known beneficiary, and • Social: prosocial behavior that is aimed at addressing societal concerns. Individuals are asked to rate their response to 45 situations. Ratings are given on a 7-point scale ranging from I would definitely do what the statement says I do to I definitely would not do what the statement says I do. Scores for each subscale and the overall scale are computed from the sum of the ratings. Estimated Time to Administer Twenty to twenty-five minutes. Reliability Temporal Consistency No information has been located. Internal Consistency No information has been located. Factor Structure No information has been located. Validity Known Groups Students who were involved in service projects scored significantly higher on the VMS than students not involved in service activities (Shelton & McAdams, 1990).

MORAL DEVELOPMENT

89

Convergent Validity A significant correlation (.42) was found between the VMS score and the Interpersonal Reactivity Index (a measure of empathy; Davis, 1983). Discriminant Validity The VMS is not highly correlated with the MAS-R, which may suggest that although both scales measure morality, they each tap into different dimensions (White, 1997). Gender Differences Female respondents scored consistently higher than did male respondents on all aspects of the VMS (Shelton &. MeAdams, 1990). Comments There is strong evidence (e.g., Batson, 1991; Eisenberg & Miller, 1987) to support the critical importance that empathy plays in determining helping behavior and in determining the influence of other variables on altruistic responses. The VMS focuses on empathy and concern for others. Some service learning experiences, especially those involving face-to-face interactions, are likely to have an effect on empathy, and this scale could, therefore, be used as a pretest/posttest in such a course or as a moderator variable. Although the VMS has items that refer to high school, it would be easily adapted to a college-age audience. The types of situations presented are quite realistic and provide a broad sample of potential moral conflicts. The three subscales present some interesting possibilities for research that focuses on how students with different profiles of morality (e.g., high score on social vs. high score on private) respond to different types of service settings. References Batson, C. D. (1991). The altruism question: Toward a social-psychological answer. Hillsdale, NJ: Erlbaum. Davis, M. H. (1983). Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of Personality and Social Psychology, 44, 113-126. Eisenberg, N., &. Miller, P. A. (1987). Empathy and prosocial behavior. Psychological Bulletin, 94, 100-131. Shelton, C. M., & McAdams, D. P. (1990). In search of an everyday morality: The development of a measure. Adolescence, 25, 923-943.

90

RESEARCH SCALES

White, F. A. (1997). Measuring the content of moral judgment development: The Revised Moral Authority Scale (MAS-R). Social Behavior and Personality, 25, 321-334.

Scale Response scale 1

2

I would definitely do what the statement says I do

3

4

5

6

7

I definitely would not do what the statement says I do

1. A local community group has come to my school requesting that students volunteer to take part in a march against world hunger. For every mile walked, a local merchant will contribute one dollar. I agree to take part and walk five miles. 2. I am walking alone and I find a dollar on the street. I pick it up and continue walking. I pass a group of people who are collecting money for muscular dystrophy. I drop the dollar that I found into the basket. 3. I am a member of the history club at school, which consists of fifteen members. The club invites a guest speaker who will speak on current political events. A three-person committee from the club is in charge of arranging details for the event. The committee needs some members to come two hours early on the night of the speech in order to set up chairs and fix refreshments. I am asked by a member of the committee to come two hours early. I tell the person that I will come to the talk but I will not come two hours early to set up.* 4. I overhear two freshman whom I know casually talking and saying that no one ever goes to their games to watch them play, even when they play at home. I realize 1 have a free afternoon after school this next Thursday when the freshman play at home. 1 show up for their game after school on Thursday and I stay for at least half the game. 5. I read in a psychology magazine how people who smile actually help other people to feel better about themselves. The next day when I go to work as a checker at the local grocery store I intentionally make a point of smiling at each customer who comes to my checkout stand. 6. It is a snowy day and I'm off from school. I decide to walk around the block to get some fresh air. As I begin walking I notice a MORAL DEVELOPMENT

91

. 7.

8.

. 9.

10.

11.

12.

13.

14.

92

driver and his car are stuck in the snow. I keep walking and do not stop to help.* I have tickets to go to a local rock concert with some friends (classmates). The concert is sold out. One of my friends who is going with me has to back out because he/she forgot that he/she has to help his/her family this weekend. Although I have several friends in school who would like to buy the ticket, I offer it to a new transfer student in hopes that this will give him/her a chance to meet some people. In our town there is a referendum to raise local taxes by 3%. Supporters of this measure say it is necessary in order to continue providing social welfare services for the needy. Opponents say that taxes are high enough and that people are paying all that they should in taxes. On election day I vote against the 3% tax increase.* I work in a movie theater as an usher with several people my own age. The head of the theater is making out the schedule for the Christmas holidays. I know that one of the ushers wants to go with his family to visit his grandmother for Christmas but he has been put down to work on Christmas Day, whereas I am off that day. I go to the usher and tell him to switch me with this other usher which, in effect, makes me work Christmas Day and allows the other usher to get off. I tell the manager not to tell the other usher I did this. A fellow student whom I know casually asks if I have an hour this Saturday to help him/her with some math problems (I am very good at math). 1 am free on Saturday so I tell the student that I would be happy to help him/her. When cleaning up my room I collect several pieces of clothing that I can no longer use. I can dispose of them or drive five miles to the Salvation Army and drop them in their drop-off box. I dispose of them and do not drive the five miles.* I am asked to write, in class, an essay on why I want to go into a particular career and what influences me to choose this career. When I write my essay I say that the most important influence on any career I choose is the desire to help people. My entire class is going on a weekend outing to the amusement park. One of my classmates whom I know casually loses his/her money at the park. Several student chip in and give the student $5 each. I am one of the students who give this person five dollars. A classmate from my homeroom whom I know casually has been in an accident and there is an announcement that students can

RESEARCH SCALES

give blood if they wish. I am in good health and can give blood and not afraid of needles or blood. I do not volunteer.* 15. A report is published in the newspaper which rates local companies on their efforts to alleviate minority unemployment. These ratings are not a factor when I decide in which stores I will shop.* 16. This weekend, several friends of mine are going to a movie which has gotten good reviews. The movie has also been depicted by several reviewers "as unfortunately supporting and reinforcing sexist and violent attitudes." I go to the film.* 17. I win $50 in the state lottery. I find out in the morning and in the afternoon a volunteer for the annual cerebral palsy drive stops by and requests that I make a contribution if I can afford it. I do not make a contribution.* 18. I have a personality clash with one of my teachers, and simply do not get along with him. I hear from a family friend that this teacher's father has been quite sick recently. Over the next few weeks I make a conscious effort to be respectful in class and I go out of my way to say "hello" to him when I pass him in the halls. 19. A neighbor on my block asks me to take her shopping twice a week while her husband is recuperating from a heart attack (she does not drive). I agree to take her twice a week for the next three weeks. . 20. I read in the paper about a family who has lost all their belonging in a fire. 1 anonymously send a ten dollar check to a fund set up for the family by the town newspaper. .21. I am walking downtown fairly rapidly with a friend so that we can make a movie on time. As I am walking I notice a person standing by a car next to a parking meter. He is holding some change in his hands and looks frustrated. I interrupt my walk to the movies and go over to ask him if he needs correct change for the parking meter. I exchange currency with him so that he will have correct change. . 22. A local civic group has asked over the radio for volunteers to help prepare and serve a free meal on Christmas Day for the needy. I sign up and work four hours on Christmas Day. .23. I am visited by a member of a fraternity/sorority from the college I plan to attend in the fall. After a discussion about the fraternity/ sorority's lifestyle and procedures, I ask about the organization's attitudes toward minorities. I state that I would not be interested in a group that is not open to minority members. . 24- In the upcoming primary election, several candidates have distributed various pieces of literature on issues such as world hunger,

MORAL DEVELOPMENT

93

peace, military spending, and aid to certain foreign governments. In evaluating the candidates, J use their position on these issues as the primary determining factor for my vote. .25. I read an article in a popular magazine urging people to write letters to foreign governments to protest their policy of holding political prisoners. I do not write any letters.* . 26. I hear on a local radio station that the city orphanage is having a paper drive and is requesting residents to bring in their papers. The proceeds received from the paper drive will be used to buy recreation equipment at the orphanage. 1 have the afternoon off so I gather my papers at home and drive the five miles to the orphanage to drop them off. .27. An eighth-grade classmate whom I have not seen in several years calls me up and requests my help for a 5-year eighth-grade reunion that is being planned by this former classmate along with two others from the class. The classmate asks me if I could spare several hours to contact people at my high school and also help set up for the party. I tell the classmate that I'll probably come to the reunion but I will not be able to help with the details.* . 28. I am reading the paper one morning and I come across an article entitled: "How to Become a Better Citizen." Since it is an election year I memorize the main points and use them as criteria for evaluating candidates who are running for office. . 29. In order to make people aware of world hunger, students at my school are requested to restrict their food intake at lunch during the month of March and donate the money they save to a world hunger drive. I take the pledge to be part of this drive and donate my lunch money. .30. I am involved in a heated argument with a classmate about a historical date. I read a few days later in a library book that my classmate was right. I apologize for the argument and admit that he/she is right. .31. It is the end of the semester and everyone seems tired and looking forward to the upcoming vacation. A few of my classmates seem particularly tired of school and start to complain quite regularly about school. I make an effort not to complain and tell them when they do complain that things aren't that bad and point out the good things going on in school. . 32. A local restaurant has discriminated against a racial minority. I am out one night with a group of friends. We are all hungry and my three friends voice a desire to eat at this restaurant. I speak up and say I do not want to eat at the restaurant because of its discriminatory policies. 94

RESEARCH SCALES

. 33. There is a blood drive at school. I am in good health, can give blood and not afraid of the sight of blood or needles. I do not volunteer to give a pint of blood.* .34- As I walk down the street, I notice a blind lady with a cane walking on the other side of the street. She appears confused. I cross the street and ask her if she needs assistance. When I discover her problem, I walk her the two blocks to the store she was trying to find. . 35. While riding to school with several classmates, there is a discussion about some violent crimes reported in the newspaper. Several of my classmates say that "tougher laws are needed" and that "crimi' nals are getting away with too much." I respond that criminals must be punished but the real problem, for the most part, is the inhuman social conditions such as poverty and unemployment that encourage crime, and add that I plan to work to help alleviate these conditions. . 36. I am able to get a part-time job after school. My new employer tells me that he can use one other person on a part-time basis. I know that several of my friends would like to have the job. A student whom I know casually comes from a family that is experiencing some difficult financial problems and also would like to have the job. I mention to one of my friends with whom I would enjoy working that there is a job opening.* .37. I work part time as a checker at the local grocery in my neighborhood. When I receive my next paycheck I also receive a request in my pay envelope to contribute to a local charitable organization. I do not check the box on the request form which says I will contribute ten dollars of my next paycheck to the fund.* . 38. I read where a large company's policies have victimized the lower class of an overseas country. This company makes one of my favorite snack foods. As a way of protest, / give up eating this snack. . 39. The administrator of the school asks all upper-class student (juniors and seniors) to give some serious thought to ways to improve the school for future students. Each junior and senior is requested to spend some time seriously reflecting on their years at the school and then to fill out a questionnaire (anonymously) and to send the form to the school. I take this request seriously and over the next few weeks think of ways to improve the school. / fill out the form and send it in. _ 40. I am going to drive this weekend with two friends to a university I hope to attend when I graduate from high school. The school is roughly 100 miles away. We are leaving on Friday afternoon after school and returning late Sunday evening. Another student MORAL DEVELOPMENT

95

41.

42.

43.

44-

45.

whom I don't know too well asks me if he can come along (I am driving my car). He wants to be dropped off at another school which is on the way. We would have to detour 20 miles off the main highway, however. He/she says, "just drop me off on the way and pick me up Sunday evening on the way back." This student volunteers to chip in some gas money. I agree to take the student along. I am walking home and I pass a woman I barely know (she lives at the other end of my block). She is carrying a large and medium bag of groceries with some difficulty. I continue walking toward my home.* The school I attend needs volunteers who will come two hours early one evening next week to be greeters and parking attendants for the annual freshman parents' night. I volunteer and come two hours early. In order to conserve energy and adopt a simpler lifestyle, I limit myself to only necessary driving. Thus, in good weather, I walk the five blocks to the local shopping center when I need something. A local resident of our township has been critically injured in an accident. A woman calls our home asking my family to volunteer some time either to make posters or to sit at a booth in the local shopping center this Saturday to collect money to help pay the huge hospital costs. My family agrees to help; I do not volunteer.* In order to make better use of natural resources, I have our family save cans. I take responsibility for this project and donate the money (a few dollars a month) to a charitable cause.

* reverse-scored Private (Items 2, 5, 6, 9, 11, 12, 17, 20, 21, 26, 28, 33, 37, 39, 42) Interpersonal (Items 3, 4, 7, 10, 13, 14, 18, 19, 27, 30, 31, 34, 40, 41, 44) Social (Items 1, 8, 15, 16, 22, 23, 24, 25, 29, 32, 35, 36, 38, 43, 45) Note. From "In Search of an Everyday Morality: The Development of a Measure," by C. M. Shelton and D. P. McAdams, 1990, Adolescence, 25, p. 937-941 (Appendix 1). Copyright 1990 by Libra Publishers. Reprinted with permission.

96

RESEARCH SCALES

6 SELF AND SELF-CONCEPT

SELF-ESTEEM SCALE Source Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press. Construct Rosenberg's Self-Esteem Scale (Rosenberg, 1965) measures feelings of self-worth and self-acceptance. Description Originally intended as a measure of self-esteem for adolescents, Rosenberg's Self-Esteem Scale is probably the most widely used measure of selfesteem for adult populations. The scale is composed of 10 items, 5 of which are negatively worded. Although originally constructed as a Guttman-type scale (i.e., items with an ordinal pattern on the attribute), most researchers use a 4-point response format ranging from strongly agree to strong!} disagree. Estimated Time to Administer Five minutes.

97

Reliability Temporal Consistency Fleming and Courtney (1984) reported a 1-week test-retest reliability of .82. Internal Consistency

Fleming and Courtney (1984) reported a coefficient alpha of .88. Factor Structure Hensley (1977) reported a unidimensional factor structure. Validity Known Groups Taylor (1995) found significant differences in self-esteem between students based on class year. Seniors scored the highest on Rosenberg's SelfEsteem Scale, followed by juniors, sophomores, and then freshman. Convergent Validity

Reynolds (1988) found a .44 correlation between Rosenberg's SelfEsteem Scale and the Academic Self-Concept Scale (Reynolds, 1988). Discriminant Validity No relationship was found between grade point average and scores on Rosenberg's Self-Esteem Scale. Scores on the Self-Esteem Scale were also unrelated to locus of control, SAT scores, and social desirability. This is expected because the scale is designed to measure general self-concept and not specific facets (Reynolds, 1988). Gender Differences Hensley (1977) and Reynolds (1988) reported no significant gender differences on the Self-Esteem Scale. Comments Rosenberg's Self-Esteem Scale is the standard measure of self-esteem in psychological research. The scale provides a short, straightforward, and convenient method for measuring global self-esteem. Because self-esteem is a traitlike construct, this scale may be of interest to researchers as a moderator

98

RESEARCH SCALES

variable in research rather than a dependent variable. For example, research could examine how precourse differences in self-esteem are related to student involvement in different types of service settings, how students deal with ambiguities and frustration during service activities, or satisfaction with the service learning course. References Fleming, J. S., & Courtney, B. E. (1984). The dimensionality of self-esteem: II. Hierarchical facet model for revised measurement scales. Journal of Personality and Social Psychology, 46, 404-421. Hensley, W. E. (1977). Differences between males and females on Rosenberg's scale of self-esteem. Psychological Reports, 41, 829-830. Reynolds, W. M. (1988). Measurement of academic self-concept in college students. Journal of Personality Assessment, 5, 223-240. Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press. Taylor, D. L. (1995). A comparison of college athletic participants and nonparticipants on self-esteem. Journal of College Student Development, 36, 444-451.

Scale Rate the items using the following scale: 1 = strongly agree

2 = agree

3 = disagree

4 = strong^ disagree

1. I feel that I am a person of worth, at least on an equal basis with others. 2. I feel that I have a number of good qualities. 3. All in all, I am inclined to feel that 1 am a failure.* 4. I am able to do things as well as most other people. 5. I feel I do not have much to be proud of.* 6. I take a positive attitude toward myself. 7. On the whole, I am satisfied with myself. 8. I wish I could have more respect for myself.* 9. I certainly feel useless at times.* 10. At times I think I am no good at all.* *reverse-scored Note. From the Rosenberg Self-Esteem Scale. Reprinted with permission of Florence Rosenberg.

SELF AND SELF-CONCEPT

99

COMMUNITY SERVICE SELF-EFFICACY SCALE Source Reeb, R. N., Katsuyama, R. M., Sammon, J. A., & Yoder, D. S. (1998). The Community Service Self-Efficacy Scale: Evidence of reliability, construct validity, and pragmatic utility. Michigan Journal of Community Service Learning, 5, 48-57. Construct The Community Service Self-Efficacy Scale (CSSES) assesses the student's confidence in making a clinically significant contribution to the community through service. Description The CSSES presents 10 items, and responses range from 1 (quite uncertain) to 10 (certain). Estimated Time to Administer Five minutes. Reliability

Temporal Consistency Reeb et al. (1998) reported in Study 2 a 14-week test-retest reliability of .62 for a group of students who were not involved in service learning. Internal Consistency The CSSES was found to have coefficients alphas above .90 in two studies (Reeb et al., 1998). Factor Structure A factor analysis of the CSSES revealed a one-factor solution, consistent with the assumption of unidimensionality. Validity Known Groups Study 1 by Reeb et al. (1998) found that students participating in each of the three types of service (extracurricular, summer, and course100

RESEARCH SCALES

related service) during the past year had higher CSSES scores than those not participating in that type of service. In addition, students reporting service participation in all three types of service during the past year had significantly higher CSSES scores than students reporting participation in only two. There was also a linear effect for participation, with significantly lower scores for students participating in only one and in none of the three activities. In Study 2, Reeb et al. (1998) reported that students electing a service learning option within a course had significantly higher CSSES scores than students not selecting the option. Convergent Validity No information has been located. Discriminant Validity No information has been located. Gender Differences No information has been located. Comments This is one of the few scales that has been developed specifically for service learning applications from an area of work that has a rich tradition theoretically and empirically. Bandura's (1977, 1995, 1997) work on selfefficacy has demonstrated the concept's relevance to expectations about an individual's capacity to take action toward a goal. Self-efficacy has relevance to the likelihood that students become involved in service and service learning, persist, and are satisfied with the outcomes of their activities. Reeb et al. (1998) noted Bandura's distinction between self-efficacy and outcome expectations. In Bandura's (1997) words, "self-efficacy is a judgment of one's ability to organize and execute given types of performances, whereas an outcome expectation is a judgment of the likely consequences such performances will produce" (p. 21). Unfortunately, the content of many of the items of the CSSES ignores this distinction, and some of the items focus on the outcomes of the students' actions rather than or in addition to their disposition for taking action. This may explain the failure of Reeb et al. (1998) to obtain significant increases in CSSES scores across a semester for students enrolled in a service learning course. Researchers need to be aware that all items are worded positively, and there is no information available about the scale's correlation with social desirability or acquiescent response bias. Nonetheless, the CSSES is significant as a scale SELF AND SELF-CONCEPT

101

developed for service learning with a good theoretical rationale, promising psychometric characteristics, and potential utility as a moderator variable, mediating variable, and outcome variable. References Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215. Bandura, A. (1995). Self'efficacy bridge University Press.

in changing societies. Cambridge, England: Cam-

Bandura, A. (1997). Self-efficacy:

The exercise of control. New York: Freeman.

Reeb, R. R, Katsuyama, R. M., Sammon, J. A., & Yoder, D. S. (1998). The Community Service Self-Efficacy Scale: Evidence of reliability, construct validity, and pragmatic utility. Michigan Journal of Community Service Learning, 5, 48-57.

Scale Please rate the items on the following scale: 1 quite uncertain

2

3

4

5

6

7

8

9

1 0 certain

1. If I choose to participate in community service in the future, I will be able to make a meaningful contribution. 2. In the future, I will be able to find community service opportunities which are relevant to my interests and abilities. 3. I am confident that, through community service, I can help in promoting social justice. 4. I am confident that, through community service, I can make a difference in my community. 5. I am confident that I can help individuals in need by participating in community service activities. 6. I am confident that, in future community service activities, I will be able to interact with relevant professionals in ways that are meaningful and effective. 7. I am confident that, through community service, I can help in promoting equal opportunity for citizens. 8. Through community service, I can apply knowledge in ways that solve "real-life" problems. 9. By participating in community service, 1 can help people to help themselves. 102

RESEARCH SCALES

10. I am confident that I will participate in community service activities in the future. Note. From "The Community Service Self-Efficacy Scale: Evidence of Reliability, Construct Validity, and Pragmatic Utility," by R. N. Reeb, R. M. Katsuyama, J. A. Sammon, and D. S. Yoder, 1998, Michigan Journal of Community Service Learning, 5, p. 51 (Table 1). Copyright 1998 by OCLS Press, University of Michigan. Reprinted with permission. SELF-EFFICACY SCALE Source Sherer, M., Maddux, J. E., Mercandante, B., Prentice-Dunn, S., Jacobs, B., & Rogers, R. W. (1982). The Self-Efficacy Scale: Construction and validation. Psychological Reports, 51, 663-671. Construct The Self-Efficacy Scale (SES) is designed to measure a generalized expectancy regarding personal mastery across a variety of situations. Description The 30-item scale contains 23 items that constitute the SES and 7 filler items. The scale contains two subscales: General Self-Efficacy subscale (17 items) and a Social Self-Efficacy subscale (6 items). In the original development of the scale, respondents used a 14-point response format ranging from strongfy disagree to strong^ agree (Sherer et al., 1982). Subsequent use has modified the response scale to a 5-point format. Estimated Time to Administer Fifteen minutes. Reliability Temporal Consistency No information has been located. Internal Consistency

The internal consistency, using coefficient alpha, was .86 for the General Self-Efficacy subscale and .71 for the Social Self-Efficacy subscale (Sherer et al., 1982). SELF AND SELF-CONCEPT

103

Factor Structure Factor analysis with varimax rotation of 376 college students produced a two-factor solution that resulted in the two subscales. A second sample of 298 college students replicated the two-factor solution, although there was evidence that the General Self-Efficacy subscale could be divided into two components: initiation/persistence and efficacy in the face of adversity. Validity Known Groups Sherer et al. (1982) reported that scores on the General Self-Efficacy subscale predicted previous success in various vocational, military, and educational goals. Furthermore, scores on the Social Self-Efficacy subscale were negatively correlated with number of jobs quit and number of times fired. Convergent Validity

The General Self-Efficacy subscale had the following significant correlations in the expected direction (Sherer et al., 1982): .43 with the MarloweCrowne Social Desirability Scale (Crowne & Marlowe, 1964), -.29 with Rotter's Internal-External Control of Reinforcement Scale (Rotter, 1966), -.36 with personal control (Gurin, Gurin, Lao, & Beattie, 1969), .29 with ego strength (Baron, 1953), .45 with interpersonal competency (Holland & Baird, 1968), and -.51 with Rosenberg's Self-Esteem Scale (Rosenberg, 1965). The Social Self-Efficacy subscale had significant correlations with Rotter's Internal-External Control of Reinforcement Scale (-.17), personal control (-.13), Marlowe-Crowne Social Desirability scale (.28), Rosenberg's Self-Esteem Scale (-.28), and interpersonal competency (.43). The General Self-Efficacy subscale was found to be significantly correlated with employment status (.28), number of jobs that the respondent had quit (-.24), number of times that the respondent had been fired (-.23), educational level (.27), and military rank (.22). The Social Self-Efficacy subscale was significantly correlated with number of jobs quit (-.20) and number of times fired (-.30) (Sherer et al., 1982). Discriminant Validity The Social Self-Efficacy subscale did not correlate significantly with ego strength, employment status, education level, and military rank (Sherer et al., 1982). Gender Differences No significant differences were found for male and female scores on the SES (May & Sowa, 1994). 104

RESEARCH SCALES

Comments Self-efficacy focuses on the individual's expectations for personal mastery and success. Rather than a situation-specific measure of self-efficacy, this scale measures the broad, generalized, dispositional nature of self-efficacy. Bandura (1977) suggested that efficacy is important to behavior and behavioral change because it influences the decision to act, which is related to the belief that certain behaviors will produce certain outcomes. In addition, self-efficacy is viewed as being related to the effort expended and persistence at a task because of the belief that the individual can produce the behavior with the intended outcomes. To the degree that service learning experiences challenge students to face practical issues that require action, the manner in which students approach their service tasks might be related to generalized disposition of self-efficacy (Eyler, 2000). If the challenge is "getting things done" and one result of effective reflection is influencing the expectancy that action will have beneficial results, then self-efficacy can be an important construct in understanding who persists at community service and why. Morton (1995) described three ways in which service can occur: individual acts of charity, service programs, and advocacy for social change. Self-efficacy can be viewed as a general characteristic that applies to each of these three types of involvement. Further research can target the role self-efficacy plays in recruitment, preparation, behavioral persistence, and other outcomes in service learning experiences. Researchers need to be aware that the scale correlated significantly with social desirability. References Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavior change. Psychological Review, 84, 191-215. Baron, F. (1953). An ego strength scale which predicts response to psychotherapy. Journal of Consulting Psychology, 17, 327-333. Crowne, D., & Marlowe, D. (1964). The approved motive. New York: Wiley. Eyler, J. (2000, Fall). What do we most need to know about the impact of servicelearning on student learning? Michigan Journal of Community Service Learning, 11-17. Gurin, P., Gurin, G., Lao, R. C., & Beattie, M. (1969). Internal-external control in the motivational dynamics of Negro youth. Journal of Social Issues, 25, 29-53. Holland, J. L., &. Baird, L. L. (1968). An interpersonal competency scale. Educational and Psychological Measurement, 28, 503-510. May, K. M., & Sowa, C. J. (1994). Personality characteristics and family environments of short-term counseling clients. Journal of College Student Development, 35, 59-62.

SELF AND SELF-CONCEPT

105

Morton, K. (1995). The irony of service: Charity, project, and social change in service-learning. Michigan Journal of Community Service Learning, 2, 19-32. Rosenberg, M. (1965). Society and the adolescent self'image. Princeton, NJ: Princeton University Press. Rotter, J. B. (1966). Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs, 80(Whole No. 609). Sherer, M., Maddux, J. E., Mercandante, B., Prentice-Dunn, S., Jacobs, B., &. Rogers, R. W. (1982). The Self-Efficacy Scale: Construction and validation. Psychological Reports, 51, 663-671.

Scale This questionnaire is a series of statements about your personal attitudes and traits. Each statement represents a commonly held belief. Read each statement and decide to what extent it describes you. There are no right or wrong answers. You will probably agree with some of the statements and disagree with others. Please indicate your own personal feelings about each statement below by marking the letter that best describes your attitude or feeling. Please be very truthful and describe yourself as you really are, not as you would like to be. A B C D E

= = = = =

disagree strongly disagree moderately neither agree nor disagree agree moderately agree strongly

1. I like to grow house plants/ 2. When I make plans, I am certain 1 can make them work. 3. One of my problems is that I cannot get down to work when I should.* 4. If I can't do a job the first time, I keep trying until I can. 5. Heredity plays the major role in determining one's personality/ 6. It is difficult for me to make new friends.* 7. When 1 set important goals for myself, I rarely achieve them.* 8. I give up on things before completing them.* 9. I like to cook.+ 10. If I see someone I would like to meet, I go to that person instead of waiting for him or her to come to me. 11. I avoid facing difficulties.* 12. If something looks too complicated, I will not even bother to try it. 13. There is some good in everybody/ 14. If I meet someone interesting who is very hard to make friends with, I'll soon stop trying to make friends with that person.* 106

RESEARCH SCALES

15. When I have something unpleasant to do, I stick to it until I finish it. 16. When I decide to do something, I go right to work on it. 17. I like science/ 18. When trying to learn something new, I soon give up if I am not initially successful.* 19. When I'm trying to become friends with someone who seems uninterested at first, I don't give up very easily. 20. When unexpected problems occur, I don't handle them well.* 21. If I were an artist, I would like to draw children.* 22. I avoid trying to learn new things when they look too difficult for me.* 23. Failure just makes me try harder. 24. I do not handle myself well in social gatherings.* 25. I very much like to ride horses/ 26. I feel insecure about my ability to do things.* 27. I am a self-reliant person. 28. I have acquired my friends through my personal abilities at making friends. 29. I give up easily.* 30. I do not seem capable of dealing with most problems that come up in my life.* +

filler item * reverse-scored Note. From "The Self-Efficacy Scale: Construction and Validation," by M. Sherer, J. E. Maddux, B. Mercandante, S. Prentice-Dunn, B. Jacobs, and R. W. Rogers, 1982, Psychological Reports, 51, p. 666 (Table 1). Copyright 1982 by Psychological Reports. Reprinted with permission. CONFIDENCE SUBSCALE OF THE ERWIN IDENTITY SCALE Source Erwin, T. D., & Delworth, U. (1980). An instrument to measure Chickering's vector of identity. National Association of Student Personnel Administrators Journal, 17, 19-24The manual containing general information about the Erwin Identity Scale, including reliability and validity information, is available from: Developmental Analytics P.O. Box 855 Harrisonburg, VA 22801 SELF AND SELF-CONCEPT

107

The manual is available for $5.00, and a copy of the scale may be obtained for $0.50. Construct The Erwin Identity Scale was developed to measure Chickering's (1969) conceptualization of identity, which focuses on the developmental tasks and changes that occur in adults and especially those in college. Chickering identified seven major developmental tasks that form the conceptual basis for the development of this inventory: (a) achieving competence, (b) managing emotions, (c) becoming autonomous, (d) establishing identity, (e) freeing interpersonal relationships, (f) clarifying purpose, and (g) developing integrity. The following dimensions of identity are measured: confidence, sexual identity, and conceptions about body and appearance. Only the Confidence subscale is described. Confidence is defined as An assuredness in one's self and in one's capabilities. Confidence includes a conscious self-reliance while recognizing the necessary dependence on outside sources. This recognition is an awareness and faith in one's own capabilities. The confident person has some understanding of his or her limitations. A self-confident individual feels comfortable about expressing beliefs, making decisions, and behaving competently, even though action may not be taken in these areas. (Erwin & Delworth, 1980, p. 48)

Description The Erwin Identity Scale contains 59 items across three subscales: (a) Confidence, (b) Sexual Identity, and (c) Conceptions About Body and Appearance. The Confidence subscale consists of 24 items and uses a 5point response format ranging from very true of me to not at all true of me. Estimated Time to Administer Fifteen minutes. Reliability Temporal Consistency No information has been located. Internal Consistency

Coefficient alphas for the Confidence subscale were .81 (Erwin, 1979), .81 (Erwin & Delworth, 1980), and .85 for a sample of students in freshman year and .86 in their senior year (Erwin &. Kelly, 1985). 108

RESEARCH SCALES

Factor Structure No information has been located. Validity Known Groups Hood, Riahinehad, and White (1986) found significantly higher scores in confidence for students involved in campus organizations than students who had not been involved. Convergent Validity Erwin and Schmidt (1981) found that the Confidence subscale correlated .46 with the Identity Achievement Scale (Simmons, 1973) and -.21 with Rotter's Internal-External scale (Rotter, 1966). Erwin and Delworth (1980) found that it correlated .81 with personal integration and .67 with lack of anxiety (Heist & Yonge, 1968). Erwin and Kelly (1985) studied the changes in students' self-confidence as they progressed through their college education and found that satisfaction with academic performance and commitment to a vocation were positively related to confidence in the senior year, after removing the effects of confidence in the freshman year. Discriminant Validity The manual reports no significant relationship between scores on the Erwin Identity Scale and grades. Gender Differences Hood et al. (1986) found that male participants scored higher on the Confidence subscale than female participants. Comments The development of identity is viewed as an important developmental task, particularly for young adults. The Erwin Identity Scale is generally based on Chickering's (1969) interpretation of Erikson's (1950, 1968) conceptualization of identity theory. Confidence in one's self is viewed by Erwin as a key component of the development of identity, and in the longitudinal study by Erwin and Kelly (1985), levels of confidence among students in their senior year were found to be related to academic achievement and vocational commitment. Confidence in one's identity is a relevant variable to the experiences students have in service learning classes. Research can be focused on better

SELF AND SELF-CONCEPT

109

understanding the types of experiences that promote a clearer sense of identity. Although this might seem to be more applicable to young adults, college students of all ages are good candidates for increasing confidence as a result of service learning experiences. In addition, it would be useful for practitioners to know how types of reflection activities (written vs. group discussion) promote clarification of issues associated with students' selfconfidence and development. References Chickering, A. W. (1969). Education and identity. San Francisco: Jossey-Bass. Erikson, E. H. (1950). Growth and crisis of the healthy personality. In M. J. E. Seen (Ed.), Symposium on the healthy personality (pp. 91-146, Suppl. II). New York: Josiah Macy Jr. Foundation. Erikson E. H. (1968). Identity: Youth in crisis. New York: Norton. Erwin, T. D. (1979). The validation of the Erwin Identity Scale. Dissertation Abstracts International, 34, 4818A-4819A. Erwin, T. D., &. Delworth, U. (1980). An instrument to measure Chickering's vector of identity. National Association of Student Personnel Administrators Journal, 17, 19-24. Erwin, T. D., & Kelly, K. (1985). Changes in students' self-confidence in college. Journal of College Student Personnel, 22, 395-399. Erwin, T. D., & Schmidt, M. R. (1981). The convergent validity of Erwin Identity Scale. Educational and Psychological Measurement, 41, 1307-1310. Heist, P., & Yonge, G. (1968). Omnibus Personality Inventory manual. New York: Psychological Corporation. Hood, A. B., Riahinehad, A. R., & White, D. B. (1986). Changes in ego identity during the college years. Journal of College Student Personnel, 27, 107-113. Rotter, J. B. (1966). Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs, 80(1, Whole No. 609). Simmons, D. D. (1973). Development of an objective measure of identity achievement status. Journal ofProjective Techniques and Personality Assessment, 34, 241244.

EMOTIONAL EMPATHIC TENDENCY SCALE Source Mehrabian, A., & Epstein, N. (1972). A measure of emotional empathy. Journal of Personality, 40, 525-543.

110

RESEARCH SCALES

Construct The Emotional Empathic Tendency Scale (EETS) measures the degree to which a person vicariously has emotional responses due to the emotional experiences of others. Description The EETS consists of 33 items that refer to empathetic tendencies, 17 of which are negatively worded. Responses are indicated on a scale ranging from +4 (very strong agreement) to -4 (very strong disagreement). Estimated Time to Administer Fifteen to twenty minutes. Reliability Temporal Consistency No information has been located. Internal Consistency The split-half reliability for EETS is .84 (Mehrabian & Epstein, 1972). Validity Known Groups Mehrabian and Epstein (1972) conducted two experiments in which they found that empathetic individuals were less likely to aggress and more likely to help another person. Van Omum, Foley, Burns, DeWolfe, and Kennedy (1981) found that undergraduate students in help-oriented organizations, as opposed to those in self-oriented organizations, scored significantly higher on the EETS. Convergent Validity Chlopan, McCain, Carbonell, and Hagen (1985) reported a correlation of .40 between scores on the EETS and the Social Interest Scale (Crandall, 1975). Furthermore, Mehrabian (1997) reported the EETS to be correlated .77 with the Balanced Emotional Empathy scale (Mehrabian, 1966a) and -.43 with the Risk of Eruptive Violence scale (Mehrabian, 1966b). The EETS positively correlated with the perspective-taking, fantasy, empathic concern, and personal distress subscales of Interpersonal Reactivity Index (Davis, 1983).

SELF AND SELF-CONCEPT

111

Rushton, Chrisjohn, and Fekken (1981) found that self-reported frequencies of helping behaviors (e.g., giving money to charity, donating blood) were correlated with the EETS. Highly empathetic individuals were more likely to volunteer to help (Barnett, Howard, King, & Dino, 1981) and to volunteer more time than low-empathy individuals (Archer, Diaz-Loving, Gollwitzer, Davis, & Foushee, 1981). Barnett, Feighny, and Esper (1983) studied willingness to help neurologically disabled children after students viewed a tape that depicted either a responsive disabled child or an unresponsive disabled child. High-empathy individuals volunteered significantly more frequently than low-empathy individuals after viewing the responsive child. However, the pattern was reversed for the unresponsive child, suggesting that more empathetic individuals value helping someone who clearly benefits from the help. Discriminant Validity Mehrabian and Epstein (1972) reported a correlation of .06 between the EETS and the Marlowe-Crowne Social Desirability scale (Crowne & Marlowe, 1960). Gender Differences Mehrabian, Young, and Sato (1988) identified nine studies in which female participants scored higher than male participants on the EETS. Comments Although there are many reasons why individuals help, one of the prominent motivators for taking action is an empathetic response by an observer. Social psychological research has demonstrated the power of empathy to elicit helping (Batson, 1991; Eisenberg &L Miller, 1987). The concrete experiences that occur in service learning are potentially powerful because they typically involve students in unfamiliar settings and with diverse groups of persons with salient needs. Educative service learning experiences will increase knowledge, understanding, and sensitivity to the circumstances of these people. Batson (1991) argued that the strength of the empathetic response will be increased by the attachment that exists to the needy person and adopting the needy person's perspective. Both of these mechanisms become more likely through personal contact. The EETS allows researchers to investigate empathy as an antecedent to other student outcomes in service experiences (e.g., "Do students who possess high empathy prior to service learning become more engaged in the service activities, contribute more than requested, persist past the end of the course, or obtain higher ratings from community supervisors?") or as an outcome resulting from those service 112

RESEARCH SCALES

experiences (e.g., "Do students who have high-quality service learning experiences develop more empathy?")- The scale will also be useful for tracking changes in empathic tendencies in students because of different types of service experiences. References Archer, R. L., Diaz-Loving, R., Gollwitzer, P. M., Davis, M. H., &. Foushee, H. C. (1981). The role of dispositional empathy and social evaluation in the empathic mediation of helping. Journal of Personality and Social Psychology, 40, 786-796. Barnett, M. A., Feighny, K. M., &. Esper, J. A. (1983). Effects of anticipated victim responsiveness and empathy upon volunteering. Journal of Social Psychology, 119, 211-218. Barnett, M. A., Howard, J. A., King, L. M., & Dino, G. A. (1981). Helping behavior and the transfer of empathy. Journal of Social Psychology, 115, 125-132. Batson, C. D. (1991). The altruism question: Toward a social—psychological answer. Hillsdale, N): Erlbaum. Chlopan, B. E., McCain, M. L., Carbonell, J. L, &. Hagen, R. L. (1985). Empathy: Review of available measures. Journal of Personality and Social Psychology, 48, 635-653. Crandall, J. E. (1975). A scale for social interest. Journal of Individual Psychology, 31, 187-195. Crowne, D. P., & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349-354. Davis, M. H. (1983). Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of Personality and Social Psychology, 44, 113-126. Eisenberg, N., & Miller, P. A. (1987). Empathy and prosocial behavior. Psychological Bulletin, 94, 100-131. Mehrabian, A. (1966a). Manual for the Balanced Emotional Empathy Scale (BEES). (Available from Albert Mehrabian, 1130 Alta Mesa Road, Monterey, CA 93940) Mehrabian, A. (1966b). Manua/ for the Risk of Eruptive Violence Scale (REV). (Available from Albert Mehrabian, 1130 Alta Mesa Road, Monterey, CA 93940) Mehrabian, A. (1997). Relations among personality scales of aggression, violence, and empathy: Validational evidence bearing on the Risk of Eruptive Violence Scale. Aggressive Behavior, 23, 433-445. Mehrabian, A., & Epstein, N. (1972). A measure of emotional empathy. Journal of Personality, 40, 525-543. Mehrabian, A., Young, A. L., & Sato, S. (1988). Emotional empathy and associated individual differences. Current Psychology: Research and Reviews, 7, 221-240.

SELF AND SELF-CONCEPT

]I3

Rushton, J. P., Chrisjohn, R. D., & Fekken, G. C. (1981). The altruistic personality and the self-report altruism scale. Personality and Individual Differences, 2, 293302. Van Omum, W., Foley, J. M., Burns, P. R., DeWolfe, A. S., & Kennedy, E. C. (1981). Empathy, altruism, and self-interest in college students. Adolescence, 16, 799-808.

Sample Items Please rate each statement using the following scale:

-4 -3 very strong disagreement

-2

-1

0

+1

+2

+3

+4 very strong agreement

1. It makes me sad to see a lonely stranger in a group. 8. Sometimes the words of a love song can move me deeply. Note. From the Emotional Empathic Tendency Scale. Copyright 1988 by Albeit Mehrabian. Reprinted with permission of the author.

INTERPERSONAL REACTIVITY INDEX Source Davis, M. H. (1996). Empathy: A social psychological approach. Boulder, CO: Westview Press. Construct The Interpersonal Reactivity Index (IRI) is designed to measure empathy as a set of constructs in addition to a unitary concept. Description The IRI is a self-report measure consisting of 28 items over four subscales. The four subscales are: • Perspective Taking: tendency to naturally examine another's point of view; » Fantasy: tendency to "transpose themselves imaginatively into the feelings and actions of fictitious characters in books, movies, and plays" (Davis, 1983, p. 117); 114

RESEARCH SCALES

• Empathic Concern: feelings of sympathy and concern for others; and • Personal Distress: feelings of anxiety and unease in interpersonal settings. Respondents are to rate each item on a 5-point response scale ranging from 1 = does not describe me very well (0 points) to 5 = describes me very well (4 points). Scores on each subscale are determined by the summation of the item responses. Estimated Time to Administer Ten minutes. Reliability Temporal Consistency Davis (1980) reported test-retest reliabilities from .61 to .81 in a 60-75-day time frame. Internal Consistency Davis (1980) reported coefficient alpha's ranging from .68 to .79 for the subscales. Factor Structure Carey, Fox, and Spraggins (1988) conducted a principal-components analysis and found support for Davis's (1980) four-factor structure. Validity Known Groups Hatcher et al. (1994) studied the changes on scores for the IRI between a group of college students taking a peer-facilitation course and a control group in a behavioral psychology course. Those students in the peer facilitation course scored significantly higher on the Empathic Concern and Perspective-Taking subscales than those in the control group. Convergent Validity Davis (1983) reported perspective taking to be associated with measures of interpersonal functioning and self-esteem, fantasy scores to be generally related to chronic emotionality and intelligence, and empathic concern to be associated with emotionality and concern for others. Personal distress SELF AND SELF-CONCEPT

115

was observed to be related to lower self-esteem and lower interpersonal functioning. Discriminant Validity Perspective-taking scores were found to be unrelated to intelligence measures, whereas fantasy scores were unrelated to measures of social functioning and self-esteem (Davis, 1983). Gender Differences Female students scored higher than male students on all subscales of the measure (Hatcher et al., 1994). Comments The social sciences have confirmed rather regularly the importance that empathy plays in eliciting helping behavior (Batson, 1991; Davis, 1996; Eisenberg & Miller, 1987). Effectively engaging students in educationally meaningful service activities should be related to an enhanced ability to take the other person's perspective, being affectively responsive, empathic concern, and personal distress (Barnett, Howard, King, & Dino, 1981; Batson, 1991). The 1R1 may be of interest to researchers who wish to investigate the construct as an antecedent to other student outcomes in service experiences or as an outcome resulting from those experiences. Davis (1996) reported extensive research on how the different components of the IRI are related to different cognitive and behavior reactions. For example, the Perspective-Taking and Empathetic Concern subscales were found to be more strongly associated with sympathetic reactions to a distressed target than were the Personal Distress and Fantasy subscales. Two studies also found that the Perspective-Taking and Empathetic Concern subscales resulted in more tolerant attitudes toward stigmatized groups compared with the Personal Distress and Fantasy subscales (Sheehan, Lennon, & McDevitt, 1989; Underwood & Briggs, 1980). References Barnett, M. A., Howard, J. A., King, L. M., &. Dino, G. A. (1981). Helping behavior and the transfer of empathy. Journal of Social Psychology, 115, 125-132. Batson, C. D. (1991). The altruism question: Toward a social-psychological answer. Hillsdale, NJ: Erlbaum. Carey, J. C., Fox, E. A., & Spraggins, E. F. (1988). Replication of structure findings regarding the Interpersonal Reactivity Index. Measurement and Evaluation in Counseling and Development, 21, 102-105.

116

RESEARCH SCALES

Davis, M. H. (1980). A multidimensional approach to individual differences in empathy. JSAS Catalog of Selected Documents in Psychology, 20(4), 85. Davis, M. H. (1983). Measuring individual differences in empathy: Evidence for a multidimensional approach, journal of Personality and Social Psychology, 44, 113-126. Davis, M. H. (1996). Empathy: A social psychological approach. Boulder, CO: Westview Press. Eisenberg, N., & Miller, P. A. (1987). Empathy and prosocial behavior. Psychological Bulletin, 94, 100-131. Hatcher, S. L, Nadeau, M. S., Walsh, L. K., Reynolds, M., Galea, J., & Marz, K. (1994). The teaching of empathy for high school and college students: Testing Rogerian methods with the Interpersonal Reactivity Index. Adolescence, 29, 961-974. Sheehan, E. P., Lennon, R., & McDevitt, T. (1989). Reactions to AIDS and other illnesses: Reported interactions in the workplace. Journal of Psychology, 123, 525-536. Underwood, B., & Briggs, S. R. (1980). The influence of perspective-taking and aggressiveness on attitudes toward Iran. Unpublished manuscript. Scale The following statements inquire about your thoughts and feelings in a variety of situations. For each item, indicate how well it describes you by choosing the appropriate rating using the following scale. When you have decided on your answer, fill in the number next to the statement. READ EACH ITEM CAREFULLY BEFORE RESPONDING. Answer as honestly as you can. Thank you. ANSWER SCALE: 1 Does not describe me well

2

3

4

5 Describes me very well

1. I daydream and fantasize, with some regularity, about things that might happen to me. 2. I often have tender, concerned feelings for people less fortunate than me. 3. I sometimes find it difficult to see things from the "other guy's" point of view.* 4- Sometimes I don't feel very sorry for other people when they are having problems.* 5. I really get involved with the feelings of the characters in a novel.

SELF AND SELF-CONCEPT

117

6. In emergency situations, I feel apprehensive and ill-at-ease. 7. I am usually objective when I watch a movie or play, and I don't often get completely caught up in it.* 8. I try to look at everybody's side of a disagreement before I make a decision. 9. When I see someone being taken advantage of, I feel kind of protective towards them. 10. I sometimes feel helpless when I am in the middle of a very emotional situation. 11. I sometimes try to understand my friends better by imagining how things look from their perspective. 12. Becoming extremely involved in a good book or movie is somewhat rare for me.* 13. When I see someone get hurt, I tend to remain calm.* 14. Other people's misfortunes do not usually disturb me a great deal.* 15. If I'm sure I'm right about something, I don't waste much time listening to other people's arguments.* 16. After seeing a play or movie, I have felt as though I were one of the characters. 17. Being in a tense emotional situation scares me. 18. When I see someone being treated unfairly, I sometimes don't feel very much pity for them.* 19. I am usually pretty effective in dealing with emergencies.* 20. I am often quite touched by things that I see happen. 21. I believe that there are two sides to every question and try to look at them both. 22. I would describe myself as a pretty soft-hearted person. 23. When I watch a good movie, I can very easily put myself in the place of a leading character. 24. I tend to lose control during emergencies. 25. When I'm upset at someone, I usually try to "put myself in his shoes" for a while. 26. When I am reading an interesting story or novel, I imagine how I would feel if the events in the story were happening to me. 27. When I see someone who badly needs help in an emergency, I go to pieces. 28. Before criticizing somebody, I try to imagine how I would feel if I were in their place. - reverse-scored Perspective-Taking scale (Items 3, 8, 11, 15, 21, 25, 28) Fantasy scale (Items 1, 5, 7, 12, 16, 23, 26) Empathic Concern scale (Items 2, 4, 9, 14, 18, 20, 22) 118

RESEARCH SCALES

Personal Distress scale (Items 6, 10, 13, 17, 19, 24, 27) Note. From the Interpersonal Reactivity Index. Copyright 1980 by Mark H. Davis. Reprinted with permission of the author.

TEXAS SOCIAL BEHAVIOR INVENTORY—SHORT FORM Source Helmreich, R., & Stapp, J. (1974). Short forms of the Texas Social Behavior Inventory (TSBI), an objective measure of self-esteem., Bulletin of the Psychanomic Society 4(5A), 473-475. Construct The Texas Social Behavior Inventory (TSBI) measures self-esteem and social competence. Description The TSBI gathers data regarding social behavior. The long form (Helmreich, Stapp, & Ervin, 1974) is composed of 32 items and uses a five-category response format (ranging from not at all characteristic of me to very much characteristic of me). The long form has been broken down into two 16-item scales for ease of administration and when alternative forms are desired in test-retest designs. Estimated Time to Administer Ten minutes. Reliability Temporal Consistency No information has been located. Internal Consistency Reliability using coefficient alpha for the long form (32 items) is .92 for male participants and .93 for female participants. Both short forms (Forms A and B) have coefficient alphas of .85 and .86 and .85 and .88 for male and female participants, respectively (Helmrich & Stapp, 1974). SELF AND SELF-CONCEPT

119

Factor Structure

Helmreich and Stapp (1974) reported that the factor structures of the two short forms "for males and females revealed structures similar to those obtained from the original scale" (p. 475). Factor analyses of the long form support the unidimensionality of the scale with a single-factor structure. Oblique rotation yielded four correlated factors: Confidence, Dominance, Social Competence, and Social Withdrawal for male participants and Confidence, Dominance, Social Competence, and Relations to Authority Figures for female participants. Validity Known Groups Parker and Parker (1991) found significant differences in self-esteem scores as measured on the TSB1 between female child abuse victims and nonabused females. Osborne, Hammerich, and Hensley (1998) found significant differences on the TSB1 between students in a service learning sections and students in laboratory-based sections of a pharmaceutical course on communications. Convergent Validity The long form of the TSB1 was found effective in predicting interpersonal attraction in laboratory settings (Ervin, 1969; Helmreich, Aronson, & LeFan, 1970; Kimble & Helmreich, 1972). The two short forms were correlated with the long form of the TSB1. Form A yielded correlations of .97 for male and .97 for female respondents. The Form B scale correlated with the long form .97 and .98 for male and female respondents, respectively. Form A and Form B correlated .89 (Helmreich & Stapp, 1974). Discriminant Validity The TSBI is not related to intelligence. It was not correlated to social desirability for male participants but had a modest correlation (.32) for female participants (Helmreich & Stapp, 1974). Gender Differences Male respondents tend to score higher on the TSBI than female respondents (Davis, Bremer, Anderson, & Tramill, 1983; McGregor et al., 1991). Comments Face-to-face interactions are often a significant component of service learning experiences for students (Eyler & Giles, 1999). Not only do service

120

RESEARCH SCALES

learning students use and develop interpersonal skills at the service site, but they also interact more intensely with other students and their instructors on campus (Eyler & Giles, 1999). Social behaviors are skills that benefit students long after graduation, and the ability to relate to and work with others is among the attributes most highly valued by employers (Appleby, 1997). For example, Osborne et al. (1998) used this scale in research on the role of service learning in a communication class in pharmacy. Eyler and Giles (1999) found that placement quality (as opposed to application, reflection, and diversity) produced the most consistent results on the interpersonal development of students. Subsequent research can clarify when and how each of these components of the service learning experience contributes to the development of interpersonal skills. The TSBI provides a measure that can be used on these research questions. References Appleby, D. C. (1997). The psychology handbook. New York: Addison-Wesley Longman. Davis, S. F., Bremer, S. A., Anderson, B. J., & Tramill, J. L. (1983). The interrelationships of ego strength, self-esteem, death anxiety, and gender in undergraduate college students. Journal of General Psychology, 108, 55-59. Ervin, C. (1969). When being close counts. Unpublished doctoral dissertation, University of Texas at Austin. Eyler, J., & Giles, D. E., Jr. (1999). Where's the learning in service-learning! San Francisco: Jossey-Bass. Helmreich, R., Aronson, E., &. LeFan, J. (1970). To err is humanizing—sometimes: Effects of self-esteem, competence and a pratfall on interpersonal attraction. Journal of Personality and Social Psychology, 16, 269-278. Helmreich, R., & Stapp, J. (1974). Short forms of the Texas Social Behavior Inventory (TSBI), an objective measure of self-esteem. Bulletin of the Psychonomic Society, 4(5A), 473-475. Helmreich, R., Stapp, J., & Ervin, C. (1974). The Texas Social Behavior Inventory (TSBI): An objective measure of self-esteem or social competence. Journal Supplement Abstract Service: Catalog of Selected Documents in Psychology, 4, 79. Kimble, C., & Helmreich, R. (1972). Self-esteem and the need for social approval. Psychonomic Science, 26, 339-342. McGregor, L., Miller, H. R., Mayleben, M. A., Buzzanga, V. L., Davis, S. F., &. Becker, A. H. (1991). Similarities and differences between "traditional" and "nontraditional" college students in selected personality characteristics. Bulletin of the Psychonomic Society, 29, 128-130. Osbome, R. E., Hammerich, S., & Hensley, C. (1998). Student effects of servicelearning: Tracking changes across a semester. Michigan Journal of Community Service Learning, 5, 5-13.

SELF AND SELF-CONCEPT

I 21

Parker, S., & Parker, H. (1991). Female victims of child sexual abuse: Adult adjustment. Journal of Family Violence, 6, 183-197.

Scale FORMA 1. I am not likely to speak to people until they speak to me.* Not at all characteristic of me

b Not very

c Slightly

d Fairly

Very much characteristic of me

2. I would describe myself as self-confident. Not at all characteristic of me

c Slightly

b Not very

d Fairly

Very much characteristic of me

3. I feel confident of my appearance. a Not at all characteristic of me

b Not very

e Slightly

d Fairly

c Slightly

d Fairly

Very much characteristic of me

4. I am a good mixer. a Not at all characteristic of me

b Not very

Very much characteristic of me

5. When in a group of people, I have trouble thinking of the right things to say.* a Not at all characteristic of me

b Not very

c Slightly

d Fairly

Very much characteristic of me

6. When in a group of people, I usually do what the others want rather than or in addition to make suggestions.* Not at all characteristic of me 122

b Not very

RESEARCH SCALES

c Slightly

d Fairly

Very much characteristic of me

7. When I am in disagreement with other people, my opinion usually prevails.

Not at all characteristic of me

b Not very

d Fairly

c Slightly

Very much characteristic of me

8. I would describe myself as one who attempts to master situations. Not at all characteristic of me

b Not very

d Fairly

c Slightly

Very much characteristic of me

9. Other people look up to me. d Fairly

c Slightly

b

Not at all characteristic of me

Not very

Very much characteristic of me

10. 1 enjoy social gatherings just to be with people. a Not at all characteristic of me

b Not very

d

e Slightly

Fairly

e Very much characteristic of me

11. I make a point of looking other people in the eye. a Not at all characteristic of me

Not very

b Slightly

e

d Fairly

Very much characteristic of me

12. I cannot seem to get others to notice me.* Not at all characteristic of me

b Not very

c Slightly

d Fairly

Very much characteristic of me

13. I would rather not have very much responsibility for other people.* Not at all characteristic of me

b Not very

Slightly

d Fairly

Very much characteristic of me

SELF AND SELF-CONCEPT

123

14. I feel comfortable being approached by someone in a position of authority. d Fairly

b

Not at all characteristic of me

Not very

Slightly

Very much characteristic of me

15. I would describe myself as indecisive.* Not at all characteristic of me

b Not very

d Fairly

Slightly

Very much characteristic of me

16. I have no doubts about my social competence. a Not at all characteristic of me

Not very

b Slightly

e

d Fairly

Very much characteristic of me

FORMB 1. I would describe myself as socially unskilled.* a Not at all characteristic of me

Not very

b Slightly

e

d Fairly

Very much characteristic of me

2. I frequently find it difficult to defend my point of view when confronted with the opinions of others.* a Not at all characteristic of me

b e Not very

d Slightly

Fairly

e Very much characteristic of me

3. I would be willing to describe myself as a pretty "strong" personality. a Not at all characteristic of me

b e Not very

d Slightly

Fairly

e Very much characteristic of me

4- When I work on a committee I like to take charge of things. a Not at all characteristic of me 124

b e Not very

RESEARCH SCALES

d Slightly

Fairly

e Very much characteristic of me

5. I usually expect to succeed in the things I do. a Not at all characteristic of me

b e Not very

d Slightly

Fairly

e Very much characteristic of me

6. I feel comfortable approaching someone in a position of authority over me. a Not at all characteristic of me

b e Not very

d Slightly

Fairly

e Very much characteristic of me

7. I enjoy being around other people and seek out social encounters frequently.

a Not at all characteristic of me

b e Not very

d Slightly

Fairly

e Very much characteristic of me

8. I feel confident of my social behavior. a Not at all characteristic of me

b e Not very

d Slightly

Fairly

e Very much characteristic of me

9. I feel I can confidently approach and deal with anyone I meet. a Not at all characteristic of me

b e Not very

d Slightly

Fairly

e Very much characteristic of me

10. I would describe myself as happy. a Not at all characteristic of me

b e Not very

d Slightly

Fairly

e Very much characteristic of me

11. I enjoy being in front of a large audience. a Not at all characteristic of me

b Not very

c Slightly

d Fairly

e Very much characteristic of me

SELF AND SELF-CONCEPT

125

12. When I meet a stranger, I often think that he is better than I am.* a Not at all characteristic of me

b e Not very

d Slightly

Fairly

e Very much characteristic of me

13. It is hard for me to start a conversation with strangers.* a Not at all characteristic of me

b e Not very

d Slightly

Fairly

e Very much characteristic of me

14. People seem naturally to turn to me when decisions have to be made. a Not at all characteristic of me

b e Not very

d Slightly

Fairly

e Very much characteristic of me

15. I feel secure in social situations. a Not at all characteristic of me

b Not very

c Slightly

d Fairly

e Very much characteristic of me

16. I like to exert my influence over other people. a Not at all characteristic of me

b e Not very

d Slightly

Fairly

e Very much characteristic of me

* reverse-scored Note. From "Short Forms of the Texas Social Behavior Inventory (TSBI), An Objective Measure of Self-Esteem," by R. Helmreich and J. Stapp, 1974, Bulletin of the Psychonomic Society, 4(5A), 473-475 (Table 1). Copyright 1974 by the Psychonomic Society. Reprinted with permission.

HOPE SCALE Source Snyder, C. R., Harris, C., Anderson, J. R., Holleran, S. A., Irving, L. M., Sigmon, S.T., Yoshinobu, L., Gibb, J., Langelle, C., & Harney, P. 126

RESEARCH SCALES

(1991). The will and the ways: Development and validation of an individualdifferences measure of hope. Journal of Personality and Social Psychology, 60, 570-585. Construct The Hope Scale measures two interrelated aspects of hope: agency and pathways. Agency refers to a "sense of successful determination in meeting goals in the past, present, and future" (Snyder et al., 1991, p. 570). Pathways refers to feeling that one can devise successful plans to meet goals. Description The Hope Scale is composed of 12 items. Four items tap into the agency element, 4 items measure the pathways element, and 4 items are filler items. Originally, respondents were asked to rate each item on a 4point scale ranging from 1 = definitely false to 4 = definitely true. Recently, however, an 8-point scale ranging from 1 = definitely false to 8 = definitely true has been used (Cheavens, Gum, &. Snyder, in press). Estimated Time to Administer Five minutes. Reliability Temporal Consistency

Snyder et al. (1991) reported test-retest reliabilities for the Hope Scale as .85 for a 3'week interval, .73 for an 8-week interval, and .76 and .82 for a 10-week interval for two different samples. Internal Consistency

Coefficient alpha for the total scale ranged from .74 to .88 (Cramer & Dyrkacz, 1998; Snyder et al., 1991; Sumerlin, 1997). For the Agency subscale, alphas ranged from .70 to .84, and the Pathways subscale alphas ranged from .63 to .86 (Cramer & Dyrkacz, 1998; Snyder et al., 1991; Sumerlin, 1997). Factor Structure

Snyder et al. (1991) used a principal-components factor analysis with oblique rotations to obtain the hypothesized two-factor structure. Furthermore, Babyak, Snyder, and Yoshinobu (1993) and Magaletta and Oliver SELF AND SELF-CONCEPT

127

(1999) factor analyzed the Hope Scale. Results from both of these studies and many others support the two-factor structure as defined by Snyder et al. Validity Known Groups Individuals in psychological treatment scored significantly lower on the Hope Scale compared with college students (Snyder et al., 1991). Convergent Validity The Hope Scale was correlated with two measures of positive outcome expectations: the Life Orientation Test (LOT; Scheier & Carver, 1985) and the Generalized Expectancy for Success Scale (GESS; Fibel & Hale, 1978). Scores on the Hope Scale correlated .60 and .50 with the LOT (Gibb, 1990) and .55 and .54 with the GESS (Holleran & Snyder, 1990). Furthermore, Gibb's (1990) study found a correlation of .58 between scores on the Hope Scale and Rosenberg's Self-Esteem Scale (Rosenberg, 1965). Scores on the Hope Scale were negatively correlated -.51 and -.42 with the Hopelessness Scale (Gibb, 1990) and the Beck Depression Inventory (Beck, Ward, Mendelsohn, Mock, & Erbaugh, 1961). Discriminant Validity Snyder et al. (1991) found no significant relationship between scores on the Hope Scale and scores on the Private and Public Self-Consciousness subscales (Fenigstein, Scheier, &. Buss, 1975). Gender Differences Snyder et al. (1991) found no significant gender differences. Comments This Hope Scale encompasses an optimistic perspective about the respondent's ability to reach desired outcomes. As such, it incorporates aspects of self-efficacy, self-confidence, and self-direction. The scale possesses relatively strong psychometric qualities, and its length should make it convenient to incorporate in research. References Babyak, M. A., Snyder, C. R., & Yoshinobu, L. (1993). Psychometric properties of the Hope Scale: A confirmatory factor analysis. Journal of Research in Personality, 27, 154-169.

128

RESEARCH SCALES

Beck, A. T., Ward, C. H., Mendelsohn, M., Mock, J., & Erbaugh, J. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4, 53-63. Cheavens, J., Gum, A., & Snyder, C. R. (in press). The Hope Scale. In J. Malty, C. A. Lewis, & A. Hill (Eds.), A handbook of psychological tests. Lampeter, Wales, UK: Edwin Mellen Press. Cramer, K. M., & Dyrkacz, L. (1998). Differential prediction of maladjustment scores with the Snyder hope subscales. Psychological Reports, 83, 1035-1041. Fenigstein, A., Scheier, M. F., & Buss, A. H. (1975). Public and private selfconsciousness: Assessment and theory. Journal of Consulting and CJinica! Psychology, 43, 522-527. Fibel, B., & Hale, W. D. (1978). The Generalized Expectancy for Success Scale: A new measure. Journal of Consulting and Clinical Psychology, 46, 924-931. Gibb, J. (1990). The Hope Scale revisited: Further validation of a measure of individual differences in the hope motive. Unpublished master's thesis, University of Illinois at Urbana-Champaign. Holleran, S., & Snyder, C. R. (1990). Discriminant and convergent validity of the Hope Scale. Unpublished manuscript, University of Kansas, Lawrence. Magaletta, P. R., & Oliver, J. M. (1999). The hope construct, will, and ways: Their relations with self-efficacy, optimism, and general well-being. Journal of Clinical Psychology, 55, 539-551. Rosenberg, M. (1965). Society and adolescent self-image. Princeton, NJ: Princeton University Press. Scheier, M. F., &. Carver, C. S. (1985). Optimism, coping, and health: Assessment and implications of generalized outcome expectancies. Health Psychology, 4, 219-247. Snyder, C. R., Harris, C., Anderson, J. R., Holleran, S. A., Irving, L. M., Sigmon, S. T., Yoshinobu, L., Gibb, J., Langelle, C., & Harney, P. (1991). The will and the ways: Development and validation of an individual-differences measure of hope. Journal of Personality and Social Psychology, 60, 570-585. Sumerlin, J. R. (1997). Self-actualization and hope. Journal of Social Behavior and Personality, 12, 1101-1110.

Scale Directions: Read each item carefully. Using the scale shown below, please select the number that best describes YOU and put that number in the blank provided. 1 2 3 4

= = = =

definitely false mostly false somewhat false slightly false

5 6 1 8

= = = =

slightly true somewhat true mostly true definitely true

SELF AND SELF-CONCEPT

129

. . . . .

I.I can think of many ways to get out of a jam. 2. I energetically pursue my goals. 3. I feel tired most of the time/ 4. There are lots of ways around any problem. 5. I am easily downed in an argument/ 6. I can think of many ways to get the things in life that are most important to me. . 7. I worry about my health/ 8. Even when others get discouraged, I know I can find a way to solve the problem. 9. My past experiences have prepared me well for my future. 10. I've been pretty successful in life. 11. I usually find myself worrying about something/ 12. I meet the goals that I set for myself. +

filler item Agency (Items 2, 9, 10, 12) Pathways (Items 1, 4, 6, 8) Note. From "The Will and the Ways: Development and Validation of an Individual'Differences Measure of Hope," by C. R. Snyder, C. Harris, J. R. Anderson, S. A. Holleran, L. M. Irving, S. T. Sigmon, L. Yoshinobu, J. Gibb, C. Langelle, and P. Harney, 1991, Journal of Personality and Social Psychology, 60, p. 585 (Appendix). Copyright 1991 by the American Psychological Association. Reprinted with permission.

DEAN ALIENATION SCALE Source Dean, D. G. (1961). Alienation: Its meaning and measurement. American Sociological Review, 26, 753-768. Construct The Dean Alienation Scale (DAS) measures total alienation, social isolation, normlessness, and powerlessness. Description The 24-item DAS uses a 5-point response format anchored by strongfy agree and strongly disagree. The DAS produces a total score for Alienation 130

RESEARCH SCALES

and the following three subscales: Powerlessness, Normlessness, and Social Isolation. Estimated Time to Administer Fifteen minutes. Reliability Temporal Consistency No information has been located. Internal Consistency Dean (1961) reported split-half reliabilities for the DAS as follows: .78 for the total Alienation score, .78 for the Powerlessness subscale, .73 for the Normlessness subscale, and .84 for the Social Isolation subscale. Factor Structure No information has been located. Validity Known Groups The DAS was used by Tyminski (1995), who studied the effects of a service learning curriculum on the perceptions of immigrant or refugee college students. This population was chosen because of the belief that refugee or immigrant students are considered to be susceptible or at-risk for social isolation and alienation on the college campus. The scale was administered as a pretest and posttest measure of change. The sample consisted of three treatment groups: (a) 10 refugee/immigrant students participating in a work study program during which they were paid for 10 hours of community service, (b) 11 refugee/immigrant students participating in a special topics course that included 3 hours of community service per week, and (c) a control group of 18 refugee/immigrant students. The results indicated that the work study and service learning groups scored lower than the control group on isolation, powerlessness, normlessness, and total isolation (Tyminski, 1995). Convergent Validity No information has been located. SELF AND SELF-CONCEPT

131

Discriminant Validity Calabrese and Schumer (1986) found no significant relationship between scores on the DAS and grade point averages. Gender Differences Tyminski (1995) found no gender differences on the subscales of the DAS. Comments Although the content of the DAS does not appear to have relevance to service learning research, it does provide a counterpoint for the intended outcomes of service learning. That is, effective service learning should increase the engagement of students in their communities and, at the same time, reduce their sense of alienation, powerlessness, normlessness, and social isolation (Eyler & Giles, 1999). Although the supporting psychometric evidence is not yet strong, the scale may be useful when these constructs (or their opposites) are relevant to a research question. References Calabrese, R. L., & Schumer, H. (1986). The effects of service activities on adolescent alienation. Adolescence, 21, 675-687. Dean, D. G. (1961). Alienation: Its meaning and measurement. American Sociological Review, 26, 753-768. Eyler, J., & Giles, D. E., Jr. (1999). Where's the learning in service-learning? San Francisco: Jossey-Bass. Tyminski, C. (1995). Effects of an intergenerational service-learning curriculum on refugee/immigrant students. Unpublished doctoral dissertation, Temple University.

Scale Please consider the following questionnaire from the perspective of yourself in relation to Americans in general. Below are some statements regarding public issues, with which some people agree and others disagree. Please give us your own opinion about these items, i.e., whether you agree or disagree with the items as they stand. Please check in the appropriate blank, as follows:

132

RESEARCH SCALES

. A (Strongly Agree) . a (Agree) . U (Uncertain) . d (Disagree) . D (Strongly Disagree) 1. Sometimes I feel all alone in the world. A a U d D 2. Sometimes I worry about the future facing today's children. A a U d D 3. I don't get invited out by friends as often as I'd really like. A a U d D 4. The end often justifies the means. A a U d D 5. Most people today seldom feel lonely. A a U d D 6. Sometimes I have the feeling that other people are using me. A a U d D 7. People's ideas change so much that I wonder if we'll ever have anything to depend on. A a U d D 8. Real friends are as easy as ever to find.* A a U d D 9. It is frightening to be responsible for the development of a little child. A a U d D 10. Everything is relative, and there just aren't any definite rules to live by. A a U d D 11. One can always find friends if he shows himself friendly.* A a U d D 12. I often wonder what the meaning of life really is. A a U d D 13. There is little or nothing I can do towards preventing a major "shooting" war. A a U d D 14. The world in which we live in basically a friendly place.* A a U d D 15. There are so many decisions that have to be made today that sometimes I could just "blow up." A a U d D

SELF AND SELF-CONCEPT

133

16. The only thing one can be sure of today is that he can be sure of nothing. A a U d D 17. There are few dependable ties between people any more. A a U d D 18. There is little chance for promotion on the job unless a man gets a break. A a U d D 19. With so many religions abroad, one doesn't really know which to believe. A a U d D 20. We're so regimented today that there's not much room for choice even in personal matters. A a U d D 21. We are just so many cogs in the machinery of life. A a U d D 22. People are just naturally friendly and helpful.* A a U d D 23. The future looks very dismal. A a U d D 24. I don't get to visit friends as often as I'd really like. A a U d D * reverse-scored Isolation (Items 1, 3, 5, 8, 11, 14, 17, 22, 24) Powerlessness (Items 2, 6, 7, 9, 13, 15, 18, 20, 21) Normlessness (Items 4, 10, 12, 16, 19, 23) Note. From the Dean Alienation Scale. In the public domain. SELFISM Source Phares, J. E., &. Erskine, N. (1984). The measurement of selfism. Educational and Psychological Measurement, 44, 597-608. Construct Selfism is viewed as a generalized orientation to a wide range of situations that present problems in need satisfaction (e.g., achievement, dependency, love and affection) and characterizes them in an egocentric fashion. 134

RESEARCH SCALES

At the other end of the continuum are individuals who "subsume their own satisfaction in favor of others" (Phares & Erskine, 1984, p. 201). Description The 40-item Selfism scale contains 28 items measuring selfism and 12 filler items. The scale uses a 5-point response format (ranging from strongly agree to strongly disagree). A single score is obtained by summing the 28 selfism items. Estimated Time to Administer Fifteen minutes. Reliability Internal Consistency

The internal consistency as measured by the split-half reliability (with Spearman-Brown correction) of the Selfism scale was .84 for male and .83 for female participants (Phares & Erskine, 1984). Temporal Consistency

The Selfism scale had a 7-week test-retest reliability of .61 and a 4week test-retest reliability of .91 (Phares & Erskine, 1984). Factor Structure

Phares and Erskine (1984) reported: A factor analysis was also conducted using the varimax rotation method. No single factor emerged that accounted for a major portion of the variance. Rather, there were a number of factors each accounting for the variance in approximately the same amount. .. . This rinding is supportive of the concept of an additive scale which samples selfism broadly rather than or in addition to in depth, (p. 604)

Validity Known Groups Phares and Erskine (1984) asked participants to bring a friend to the laboratory. Both completed the Selfism scale for self and in terms of how they thought their friend would respond. The correlation between the actual and the predicted score was .56. It increased to .64 for participants who said they were very close to the partner, and to .66 for those who had known their friend for over 4 years. SELF AND SELF-CONCEPT

135

Convergent Validity

Phares and Erskine (1984) reported a significant correlation (.43) between the Selfism scale and the Narcissistic Personality Inventory (Raskin & Hall, 1979) and a significant negative correlation (-.37) with the Religious Attitudes Scale (Poppleton & Pilkington, 1963). Discriminant Validity

There was no significant correlation between the Selfism scale and social desirability (Crowne &. Marlowe, 1964), Internal-External Control of Reinforcement Scale (Rotter, 1966), Sensation Seeking Scale (Zuckerman, Kolin, Price, & Zoob, 1964), Generalized Expectancy for Success Scale (Fibel & Hale, 1978), and the American College Testing score (Phares & Erskine, 1984). Gender Differences There were no gender differences in two samples of college students (Phares & Erskine, 1984). Comments The opposite of altruistic motives is a selfish or egotistical orientation. This Selfism scale presents a means for measuring changes in selfism across several domains of a person's life. To the extent that service learning experiences develop altruistic motives toward community involvement and persons in general, such experiences would be expected to influence the degree to which students perceive their world in a selfish manner. In addition, selfism might be used in research as a moderator variable to clarify how experiences have different effects on selfish and altruistic persons. Service learning research is typically biased by studying predominantly those with a penchant for community service. But what are the most appropriate types of service learning experiences for students with the opposite inclination? The Selfism scale provides a means for identifying and studying those for whom service learning presents the greatest challenge and the greatest potential for change. References Crowne, D. P., &. Marlowe, D. A. (1964). The approval motive. New York: Wiley. Fibel, B., &. Hale, W. D. (1978). The Generalized Expectancy for Success Scale— a new measure. Journal of Consulting and Clinical Psychology, 46, 924-931. Phares, J. E., & Erskine, N. (1984). The measurement of selfism. Educational and Psychological Measurement, 44, 597-608.

136

RESEARCH SCALES

Poppleton, P., &. Pilkington, G. (1963). The measurement of religious attitudes in a university population. British journal of Social and Clinical Psychology, 2, 20-36. Raskin, R., & Hall, C. S. (1979). A narcissistic personality inventory. Psychological Reports, 45, 590. Rotter, J. B. (1966). Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs, 80(Whole No. 609). Zuckerman, M., Kolin, E. A., Price, L, & Zoob, I. (1964). Development of a Sensation-Seeking Scale. Journal of Consulting and Clinical Psychology, 28, 477482.

Scale Listed below are 40 statements that deal with personal attitudes and feelings about a variety of things. Obviously, there are no right or wrong answers—only opinions. Read each item and then decide how you personally feel. Mark your answers to the left of each item according to the following scheme: 5 4 3 2 I

= = = = =

strongly agree mildly agree agree and disagree equally mildly disagree strongly disagree

1. The widespread interest in professional sports is j ust another example of escapism/ 2. In times of shortages it is sometimes necessary for one to engage in a little hoarding. 3. Thinking of yourself first is no sin in this world today. 4- The prospect of becoming very close to another person worries me a good bit. 5. The really significant contributions in the world have very frequently been made by people who were preoccupied with themselves. 6. Every older American deserves a guaranteed income to live in dignity.* 7. It is more important to live for yourself rather than or in addition to for other people, parents, or for posterity. 8. Organized religious groups are too concerned with raising funds these days.+ 9. I regard myself as someone who looks after his/her personal interests. 10. The trouble with getting too close to people is that they start making emotional demands on you. SELF AND SELF-CONCEPT

137

11. Having children keeps you from engaging in a lot of self-fulfilling activities. 12. Many of our production problems in this country are due to the fact that workers no longer take pride in their jobs/ 13. It's best to live for the present and not to worry about tomorrow. 14. Call it selfishness if you will, but in this world today we all have to look out for ourselves first. 15. Education is too job oriented these days; there is not enough emphasis on basic education.* 16. It seems impossible to imagine the world without me in it. 17. You can hardly overestimate the importance of selling yourself in getting ahead. 18. The difficulty with marriage is that it locks you into a relationship. 19. Movies emphasize sex and violence too much.+ 20. If it feels right, it is right. 21. Breaks in life are nonsense. The real story is pursuing your selfinterests aggressively. 22. An individual's worth will often pass unrecognized unless that person thinks of himself or herself first. 23. Consumers need a stronger voice in governmental affairs.* 24. Getting ahead in life depends mainly on thinking of yourself first. 25. In general, couples should seek a divorce when they find the marriage is not a fulfilling one. 26. Too often, voting means choosing between the lesser of two evils.* 27. In striving to reach one's true potential, it is sometimes necessary to worry less about other people. 28. When choosing clothes, I generally consider style before matters such as comfort or durability. 29. I believe people have the right to live any damn way they please. 30. Too many people have given up reading to passively watch TV.* 31. Owing money is not so bad if it's the only way one can live without depriving oneself of the good life. 32. Not enough people live for the present. 33. I don't see anything wrong with people spending a lot of time and effort on their personal appearance. 34- Physical punishment is necessary to raise children properly.* 35. The Peace Corps would be a good idea if it did not delay one's getting started along the road to a personal career. 36. It simply does not pay to become sad or upset about friends, loved ones or events that don't turn out well. 37. A definite advantage of birth control devices is that they permit sexual pleasure without the emotional responsibilities that might otherwise result. 138

RESEARCH SCALES

. 38. Doctors seem to have forgotten that medicine involves human relations and not just prescriptions/ . 39. I believe that some unidentified flying objects have actually been sent from outer space to observe our culture here on earth.+ . 40. In this world one has to look out for oneself first because nobody else will look out for you. +

filler item

Note. From the Selfism scale. Copyright 1984 by E. Jerry Phares and N. Erskine. Reprinted with permission of the authors.

SOCIAL AVOIDANCE AND DISTRESS SCALE Source Watson, D., & Friend, R. (1969). Measurement of social-evaluative anxiety. Journal of Consulting and Clinical Psychology, 33, 448-457. Construct The Social Avoidance and Distress (SAD) scale measures two components of social anxiety: social avoidance, which is "avoiding being with, talking to, or escaping from others for any reason" (Watson & Friend, 1969, p. 449), and social distress, which encompasses "negative emotions, such as upset, distress, and anxiety, when in social interactions, or the reported lack of negative emotion, such as being relaxed, calm, at ease, or comfortable" (Watson & Friend, 1969, p. 449). Description The 25-item SAD scale was originally based on a true-false format (Watson & Friend, 1969); however, some researchers (e.g., Patterson & Strauss, 1972) have used a 5-point response format ranging from strong^ disagree to strongly agree. Estimated Time to Administer Ten minutes. SELF AND SELF-CONCEPT

139

Reliability Internal Consistency Using the true-false response format, the Kuder-Richardson 20 was .94 for two administrations (Watson & Friend, 1969). Temporal Consistency With the true-false format, the 1-month test-retest reliability of the SAD was reported by Watson and Friend (1969) to be .68 and .79 for two separate samples. Factor Analysis Patterson and Strauss (1972) conducted a factor analysis of the SAD and found support for the two subscales. However, they argued that the SAD represents the social avoidance dimension more strongly than social anxiety. Validity Known Groups Chambless, Hunter, and Jackson (1982) found significantly higher scores on the SAD for socially phobic clients than for college students. Convergent Validity Students who scored high on the SAD "tended to avoid social interactions, preferred to work alone, reported that they talked less, were more worried and less confident about social relationships, but were more likely to appear for appointments" (Watson & Friend, 1969, p. 448). The SAD also significantly correlated (.54) with Taylor's Manifest Anxiety scale (Taylor, 1953), .18 with the Debilitating Anxiety subscale of the Achievement Anxiety scale (Alpert & Haber, 1960), .45 with the Endler-Hunt S-R Inventory of Anxiousness (Endler & Hunt, 1966), .76 with the Audience Sensitivity Index (Paivio, 1965), and -.76 with the Affiliation, -.42 with the Desirability, and -.33 with the Achievement subscales of the Jackson Personality Research Form (Jackson, 1966). Discriminant Validity The SAD is negatively correlated -.25 with the Marlowe-Crowne Social Desirability Scale (Crowne & Marlowe, 1964) and is not correlated with Rotter's Internal-External Control of Reinforcement Scale (Rotter, 1966) and the following subscales of Jackson's Personality Research Form: social approval, autonomy, dependence, aggression, dominance, abasement, exhibitionism, and impulsivity (Jackson, 1966). 140

RESEARCH SCALES

Gender Differences Male students scored significantly higher than female students on the true/false version of the scale (Watson & Friend, 1969). Comments The SAD measures subjective distress and behavioral avoidance. Many service settings place students in unfamiliar circumstances and around persons who may be perceived as different. The net effect of interactions during a semester-long service learning class could mitigate social avoidance. Thus, to the extent that service learning placements develop a student's ability to work with others, it would be expected that social avoidance and distress would decrease. This scale could be used in pretest-posttest comparisons of students in service learning and traditional classes to demonstrate how the service experience produce changes in sociability. References Alpert, R., & Haber, R. N. (1960). Anxiety in academic achievement situations. Journal of Abnormal and Social Psychology, 61, 207-215. Chambless, D. L, Hunter, K., & Jackson, A. (1982). Social anxiety and assertiveness: A comparison of the correlations in phobic and college student samples. Behaviour Research and Therapy, 20, 403-404. Crowne, D. P., & Marlowe, D. (1964). The approval motive. New York: Wiley. Endler, N. S., & Hunt, J. M. (1966). Sources of behavioral variance as measured by the S-R Inventory of Anxiousness. Psychological Bulletin, 65, 336-346. Jackson, D. N. (1966). Personality Research Form. Goshen, NY: Research Psychologists Press. Paivio, A. (1965). Personality and audience influence. In B. Maher (Ed.). Progress in experimental personality research (Vol. 2, pp. 127-169). New York: Academic Press. Patterson, M. L., & Strauss, M. E. (1972). An examination of the discriminant validity of the Social Avoidance and Distress Scale. Journal of Consulting and Clinical Psychology, 39, 169. Rotter, J. B. (1966). Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs, 80(1, Whole No. 609). Taylor, J. A. (1953). A personality scale of manifest anxiety. Journal of Abnormal and Social Psychology, 48, 285-290. Watson, D., & Friend, R. (1969). Measurement of social-evaluative anxiety. Journal 0/Conswiting and CJinicaf Psychology, 33, 448-457.

SELF AND SELF-CONCEPT

141

Scale 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 1415. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28.

I feel relaxed even in unfamiliar situations.* I try to avoid situations which force me to be very sociable. It is easy for me to relax when I am with strangers.* I have no particular desire to avoid people.* I often find social occasions upsetting. I usually feel calm and comfortable at social occasions.* I am usually at ease when talking to someone of the opposite sex.* I try to avoid talking to people unless I know them well. If the chance comes to meet new people, I often take it.* I often feel nervous or tense in casual get-togethers in which both sexes are present. I am usually nervous with people unless I know them well. I usually feel relaxed when I am with a group of people.* I often want to get away from people. I usually feel uncomfortable when I am in a group of people I don't know. I usually feel relaxed when I meet someone for the first time.* Being introduced to people makes me tense and nervous. Even though a room is full of strangers, I may enter it anyway.* I would avoid walking up and joining a large group of people. When my superiors want to talk with me, I talk willingly. I often feel on edge when I am with a group of people. I tend to withdraw from people. I don't mind talking to people at parties or social gatherings.* I am seldom at ease in a large group of people. I often think up excuses in order to avoid social engagements. I sometimes take the responsibility for introducing people to each other.* I try to avoid formal social occasions. I usually go to whatever social engagements I have.* I find it easy to relax with other people.*

* reverse-scored Avoidance (Items 2, 4, 8, 9, 13, 17, 18, 19, 21, 22, 24, 25, 26, 27) Distress (Items 1, 3, 5, 6, 7, 10, 11, 12, 14, 15, 16, 20, 23, 28) Note. From "Measurement of Social-Evaluative Anxiety," by D. Watson andR. Friend, 1969, Journal of Consulting and Clinical Psychology, 33, p. 450. Copyright 1969 by the American Psychological Association. Reprinted with permission. 142

RESEARCH SCALES

7 STUDENT DEVELOPMENT

STUDENT DEVELOPMENTAL TASK AND LIFESTYLE ASSESSMENT Source Winston, R. B., Jr., Miller, T. K., & Cooper, D. L. (1999). Student Developmental Task and Lifestyle Assessment. Athens, GA: Student Development Associates. A copy of this scale may be obtained from: Student Development Associates, Inc. PMB 500 2351 College Station Road Athens, GA 30605 http://www.geocities.com/CollegePark/classroom/3022/ SDTLA.html Student Development Associates, Inc. offers four different licensing agreement options to accommodate institutes, programs, and graduate students. Prices and licensing stipulations can be obtained from the publisher. A specimen set can also be purchased for $30.00 and includes one copy of each of the forms and the Preliminary Technical Manual.

143

Construct The Student Developmental Task and Lifestyle Assessment (SDTLA) is a revision of the Student Development Task and Lifestyle Inventory (SDTLI; Winston, Miller, & Prince, 1995), which was developed as a tool for counselors. The SDTLA measures psychosocial development in traditional-age college students (17-24 years), including behaviors, attitudes, and reports of feelings that are a result of "accomplishing a developmental task or having addressed important life events or issues within the context of higher education" (SDTLA History, 2002). Description The SDTLA is based on the theoretical work of Chickering (1969; Chickering &. Reisser, 1993), which focuses on the developmental tasks and changes that occur in adults and especially those in college. Chickering identified seven major developmental tasks that form the conceptual basis for the development of this inventory: (a) achieving competence, (b) managing emotions, (c) becoming autonomous, (d) establishing identity, (e) freeing interpersonal relationships, (f) clarifying purpose, and (g) developing integrity. The SDTLA is based on a revision of the SDTLI (Winston & Miller, 1987). The tasks and scales assess students' feelings, activities, attitudes, and relationships associated with their development. Scores provide a portrait of students' expected behaviors and perspectives as they reach certain developmental stages. The SDTLA contains 153 questions that measure three developmental tasks consisting of a collection of independent subtasks, two scales, and a measure of response bias: • Establishing and Clarifying Purpose Task. This task is composed of four subtasks: 1. Educational Involvement: well-defined and explored education goals, plans, and self-directed learning 2. Career Planning: commitment to career plans through action 3. Lifestyle Planning: comprehensive plans for family, vocation, and education that incorporate personal, ethical, and religious values 4- Cultural Participation: participation in traditional cultural activities • Developing Autonomy Task. This task is composed of four subtasks: 1. Emotional Autonomy: lack of dependency on friends and parents 144

RESEARCH SCALES

2. Interdependence: recognizes the relationship between the individual and their community and acts as a contributing member 3. Academic Autonomy: good coping skills that allow the individual to meet educational goals 4. Instrumental Autonomy: self-sufficiency in meeting personal needs and goals without directions of others • Mature Interpersonal Relationships Task. This task is composed of two subtasks: 1. Peer Relationships: independent, frank, and trusting peer relationships that allow for individual differences 2. Tolerance: respectful and accepting relationships toward different cultures, races, and backgrounds In addition, there are two scales: (a) Salubrious Lifestyle Scale, which measures good wellness and health practices; and (b) Response Bias Scale, which detects faking good and careless responding An added feature to the SDTLA that was not available on the SDTLI is keyed responses. Each response for each question is coded with a suggested activity (e.g., talk with a counselor about your personal strengths and weaknesses) that the student can engage in to further development on the tasks, subscales, or scales. The student can also be given a list of the most frequently suggested activities. Estimated Time to Administer There are four forms of the SDTLA, including the full assessment, Form 1.99, which takes approximately 25-30 minutes to complete. Form 2.99 consists of 57 questions and measures all subtasks of the Establishing and Clarifying Purpose with the Response Bias Scale and takes approximately 15-20 minutes to complete. Form 3.99 measures all subtasks of the Developing Autonomy Task with 57 questions and takes approximately 15-20 minutes to complete. Form 4.99 measures the Developing Mature Interpersonal Relationships Task with 47 questions and takes approximately 10-15 minutes to complete. Forms 2.99, 3.99, and 4.99 all include the Response Bias Scale. Reliability Winston et al. (1999) summarized the results of several different samples that provide estimates of temporal reliability and internal consistency for the SDTLA. STUDENT DEVELOPMENT

145

Temporal Consistency Four-week test-retest reliabilities for the tasks and scales ranged from .73 for the Peer Relationships Subtask to .89 for the Career Planning Subtask, excluding the Response Bias Scale, which had a correlation of .93. Internal Consistency

Internal consistency for the developmental tasks and scales on the SDTLA were assessed with coefficient alpha: Establishing and Clarifying Purpose Task (.81), Developing Autonomy Task (.88), Developing Mature Interpersonal Relationships Task (.76), Salubrious Lifestyle Scale (.71), and Response Bias Scale (.72). Factor Structure No information has been located. Validity Winston, Miller, and Cooper (1999) summarized the results of samples that provide information about validity evidence for the SDTLA; however, only a portion of that evidence is summarized here. Known Groups As expected, scores on all scales revealed differences related to class standing (freshman, sophomore, junior, senior), indicating higher scores for more advanced students (Winston et al., 1999). Convergent Validity Winston et al. (1999) reported that the Establishing and Clarifying Purpose Task correlated .53, .33, .44, .45, .40, and .47, respectively with the Career Exploration Scale from the Career Development Inventory (Super, Thompson, Lindeman, Jordaan, & Myers, 1981), Classroom Learning Scale from the College Student Experiences questionnaire (Pace, 1983), Experiences With Faculty Scale from the College Student Experiences questionnaire (Pace, 1983), Life Skills Development Inventory (Pickleshimer, 1991), Art, Music and Theater Scale from the College Student Experiences questionnaire (Pace, 1983), and the Problem Solving and Decision Making Scale from the Life Skills Development Inventory (Pickleshimer, 1991). The Developing Autonomy Task correlated .56 with the Georgia Autonomy Scales (Winston, Phelps, Mazzeo, & Torres, 1997). The correlations for the subtasks of the Developing Autonomy Task ranged from .21 (Instrumental Autonomy) to .51 (Emotional Autonomy) with the Georgia Autonomy

146

RESEARCH SCALES

Scales. The Developing Autonomy Task was also correlated with the Family Independence Scale (.37) and the Study Habits Scale (.39) of the College Student Questionnaire (Peterson, 1968). The Developing Mature Interpersonal Relationships Task was correlated .58 with the total score for the Multi-Group Ethnic Identity Measure (Phinney, 1992) and .41 with the Other Group subscale of the same measure. The Salubrious Lifestyle scale correlated .54 with the Baker and Cooper's (2000) Wellness Scale. Finally, the correlation between the Response Bias Scale and the Marlowe-Crowne Social Desirability Scale (Crowne & Marlowe, 1960) was .83. Discriminant Validity No information has been located. Gender Differences Winston et al. (1999) found that women scored higher on all tasks and scales with the exception of the Salubrious Lifestyle Scale. Comments One of the unique contributions of service learning is providing students with opportunities that not only improve understanding of academic content but also provide a basis for personal development (e.g., Astin & Sax, 1998; Eyler & Giles, 1999). The potential to develop these aspects of students is typically undervalued in the academic curriculum and relegated to the purview of student affairs. However, after the academic content is forgotten, it is these areas that will continue to benefit students. This inventory presents a spectrum of outcomes (e.g., increasing educational involvement, clarifying career goals, developing skills for planning and managing one's life, establishing trusting and tolerant relationships) that could potentially be targeted in research on service learning. Establishing the role that service learning plays in developing these outcomes will be an important contribution. Use of this scale and its subscales may be limited by at least two considerations. First, it is designed for young adults. Second, it is intended to measure achievement of developmental tasks that may not be responsive to short-term interventions. Depending on the nature of the course and the service activity, it might be appropriate to use only some of the developmental tasks and scales. For example, McGill (1992) used only the Developing Mature Interpersonal Relationships Task in a pretest-posttest assessment of the effects of community service learning on relationships. This scale may also be useful for tracking students across multiple or intense service learning experiences to determine if the experiences result in clarifying STUDENT DEVELOPMENT

147

issues for the students that may influence their commitment to educational or civic goals. References Astin, A. W., & Sax, L. J. (1998). How undergraduates are affected by service participation. Journal of College Student Development, 39, 251-263. Baker, J., & Cooper, D. L. (2000). A validity study of the Salubrious Lifestyle Scale: Creating a wellness measure. Georgia Journal of College Student Affairs, 14, 11-18. Chickering, A. W. (1969). Education and identity. San Francisco: Jossey-Bass. Checkering, A. W., & Reisser, L. (1993). Education and identity (2nd ed.). San Francisco: Jossey-Bass. Crowne, D. P., & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349-354. Eyler, J., & Giles, D. E., Jr. (1999). Where's the learning in service-learning? San Francisco: Jossey-Bass. McGill, J. (1992). The relationship of community service learning to developing mature interpersonal relationships in a sample of university students. Unpublished doctoral dissertation, American University. Pace, C. R. (1983). College student experiences: A questionnaire (2nd ed.). Los Angeles: University of California, Los Angeles, Higher Education Research Institute. Peterson, R. E. (1968). College Student Questionnaire. Princeton, NJ: Educational Testing Service. Phinney, J. S. (1992). The Multi-Group Ethnic Identity Measure: A new scale for use with diverse groups. Journal of Adolescent Research, 7, 156-176. Pickleshimer, B. K. (1991). The development and evaluation of the Life-Skills Development Inventory-College Form. Unpublished doctoral dissertation, University of Georgia. SDTLA history. (2002). Retrieved August 14, 2002, from http://www.geocities.com/ studevassoc/SDTLAHISTORY.htm Super, D. E., Thompson, A. S., Lindeman, R. H., Jordaan, J. P., & Myers, R. A. (1981). The Career Development Inventory (College and University Form). Palo Alto, CA: Consulting Psychologist Press. Winston, R. B., & Miller, T. K. (1987). Student Development and Lifestyle Inventory manual. Athens, GA: Editorial Services. Winston, R. B., Miller, T. K., & Prince, J. S. (1995). Student Development and Lifestyle. Inventory. Athens, GA: Student Development Associates. Winston, R. B., Jr., Miller, T. K., & Cooper, D. L. (1999). Preliminary technical manual for the Student Developmental Task and Lifestyk Assessment. Athens, GA: Student Development Associates.

148

RESEARCH SCALES

Winston, R. B., Jr., Phelps, R. E., Mazzeo, S., & Torres, V. (1997). A short measure of students' autonomy development: The Georgia Autonomy Scales. College Student Affairs Journal, 17, 4-17. Sample Items Definitions of the tasks, subtasks, and scales follow. Establishing and Clarifying Purpose Task (PUR) Educational Involvement Subtask (El) I have a mature working relationship with one or more members of the academic community (faculty member, student affairs/services staff member, administrator).

A. B. C. D. E.

Yes. No, I don't like dealing with them. No, I have tried to form relationships, but haven't been successful yet. No, I don't know any. No, I don't have time for that kind of thing.

I am purposefully developing intellectual skills and personal habits that will assure that 1 continue to learn after completing my formal education. A. B. C. D.

I haven't thought about this. I rely completely on course requirements to do this. I think about this some times. I do this systematically.

Career Planning Subtask (CP) While in college, I have visited a career center or library to obtain information about a chosen career. A. B. C. D.

No, but I will do that when I find time. No, I don't need career information. No, there is no place or person that deals with career on my campus. Yes.

When thinking about occupations I am considering entering, A. B. C. D.

I I I I

don't know what is required in order to be competitive for a job. haven't decided which occupations interest me most. have a general idea of what is required. can list at least five requirements. STUDENT DEVELOPMENT

149

Lifestyle Planning Subtask (LP)

I am currently involved in one or more activities that I have identified as being of help in determining what I will do with the rest of my life. A. True B. False When thinking about the kind of life I want five years after college, I have. . . A. not come up with a very clear picture. B. a vague picture, but have been unable to identify the specific steps I need to take now. C. a clear enough picture that I can identify the steps that are necessary for me to take now in order to realize my dream, even though I haven't done very much about it yet. D. a clear enough picture and have identified the steps I need to take now. Cultural Participation Subtask (CUP) Within the past twelve months, I have visited a museum or an art exhibit when not required for a class.

A. Yes. B. No, I don't like those kinds of things. C. No, I just haven't gotten around to it. D. No, there aren't such things available here. Within the past six months, I have experienced unfamiliar artistic media or performances. A. True B. False Developing Autonomy Task (AUT) Emotional Autonomy Subtask (EA) If I thought my friends would disapprove of a decision I made, I would most likely .. . A. B. C. D.

try to keep them from finding out (keep it a secret). tell them and pretend I didn't care what they thought. tell them and explain my reasoning for this decision. make up something to mislead them from knowing the truth.

I trust the validity of my values and opinions, even when they aren't shared by my parent(s). 150

RESEARCH SCALES

A. B. C. D.

Never Seldom Sometimes Often

Interdependence Subtask (IND) I participate in community service activities. A. B. C. D.

Never Seldom Sometimes Often

I have made a positive contribution to my community (residence hall, campus, neighborhood, or hometown) within the past three months. A. B. C. D.

No, that isn't important to me. No, 1 don't know that I could do to make a positive contribution. No, but I have tried to find ways. Yes.

Academic Autonomy Subtask (AA) It's hard for me to work intensely on assignments for more than a short time. A. B. C. D.

Never (almost never) true of me Seldom true of me Usually true of me Always (almost always) true of me

I have a difficult time in courses when the instructor doesn't regularly check up on completion of assignments. A. B. C. D.

Never (almost never) true of me Seldom true of me Usually true of me Always (almost always) true of me

Instrumental Autonomy Subtask (1A) I satisfactorily accomplish all important daily tasks (e.g., class assignments, test preparation, room/apartment cleaning, eating, and sleeping). A. B. C. D.

Never (almost never) true of me Seldom true of me Usually true of me Always (almost always) true of me STUDENT DEVELOPMENT

151

Each day, A. I depend on my memory to make sure that I get done what needs to be done, and that works for me. B. I keep a calendar or make a "To Do" list of what needs to be done each day and that works for me. C. I dislike planning what I need to do; I just let things happen and that works for me. D. I don't make detailed plans about what I need to do each day, and as a result I forget important things. Mature Interpersonal Relationships Task (MIR) Peer Relationships Subtask (PR)

Because of my friends' urgings, I get involved in things that are not in my best interest. A. B. C. D.

Never (almost never) true of me Seldom true of me Usually true of me Always (almost always) true of me

I don't socialize with people of whom my friends don't approve. A. B. C. D.

Never (almost never) true of me Seldom true of me Usually true of me Always (almost always) true of me

Tolerance Subtask (TOL) I avoid discussing religion with people who challenge my beliefs, because there is nothing that can change my mind about my beliefs. A. B. C. D.

Never (almost never) true of me Seldom true of me Usually true of me Always (almost always) true of me

A person's sexual orientation is a crucial factor in determining whether I will attempt to develop a friendship with her/him. A. B. C. D. 152

Never (almost never) true of me Seldom true of me Usually true of me Always (almost always) true of me RESEARCH SCALES

Salubrious Lifestyle Scale (SL) I have personal habits that are potentially dangerous for my health. A. True B. False I plan my activities to make sure that I have adequate time for sleep. A. B. C. D.

Never (almost never) true of me Seldom true of me Usually true of me Always (almost always) true of me

Response Bias Scale (RB) I never regret anything I have done. A. True B. False I never say things I shouldn't. A. True B. False

LEARNING FOR SELF-UNDERSTANDING SCALE Source Springer, L., Terenzini, P. T., Pascarella, E. T., & Nora, A. (1995). Influences on college students' orientations toward learning for selfunderstanding. Journal of College Student Development, 36, 5-18. Construct The Learning for Self-Understanding Scale assesses student preferences for clarifying their self-concept and values through education. Description The 5-item Learning for Self-Understanding Scale was identified through factor analysis of a larger pool of items. Responses are given on a 5-point response format anchored by strongly agree and strongly disagree. STUDENT DEVELOPMENT

153

Time of Administration Three minutes. Reliability Temporal Consistency No information has been located. Internal Consistency

Coefficient alpha for the five-item scale was .75 (Springer et al., 1995). Factor Anafysis Springer et al. (1995) based the identification of the items for the scale on a factor analysis. All five items had factor loadings greater than .50. Validity Known Groups Springer et al. (1995) reports analyses that support the conclusion that academic and out-of-class experiences of students are related to scores on the Learning for Self-Understanding scale. Convergent Validity

No information has been located. Discriminant Validity No information has been located. Gender Differences No information was located. Comments Although this scale has had limited use, it provides a relatively short measure of the degree to which students are engaged in their education for the sake of self-understanding. Springer et al.'s (1995) research found that class-related (e.g., instructor effectiveness, experiences with faculty) and outof-class experiences (e.g., socializing with friends, interpersonal experiences, discussions of art, music, and theatre) were associated with scores on the scale. These results further indicate that these experiences had an effect 154

RESEARCH SCALES

even after controlling for pre-college variables. The authors interpreted this finding as indicating that what happens at college has an impact on students' orientation to learning, and that this is more important than what they bring to the campus. Finally, the out-of-class variables were more important than were academic variables measured in the study. This pattern of results suggests that the Learning for Self Understanding scale would be an interesting candidate for service learning research. Regardless of how serious students are when they enter higher education, campus experiences can influence their attitude toward completing a degree and deepen the role that education is seen as playing in their lives. The Learning for Self-Understanding Scale represents one aspect of that deepening which might be sensitive to experiences in a service learning class. References Springer, L., Terenzini, P. T., Pascarella, E. T., &. Nora, A. (1995). Influences on college students' orientations toward learning for self-understanding. Journal ofColkge Student Development, 36, 5-18.

Scale 1 = strongly disagree

5 = strongly agree

I prefer courses in which the material helps me understand something about myself. I prefer reading things that are relevant to my personal experience. I consider the best professors to be those who can tie things learned in class to things that are important to me in my personal life. For me, one of the most important benefits of a college education is an understanding of my self and my values. Developing a clear sense of who I am is very important to me. Note. From the Learning for Self-Understanding Scale. Copyright 1995 by L. Springer, P. T. Terenzini, E. T. Pascarella, and A. Nora. Reprinted with permission of the authors. PROBLEM-SOLVING INVENTORY Source Heppner, P. P. (1988). The Probkm-Solving Inventory (PS1): Manual. Palo Alto, CA: Consulting Psychologists Press. STUDENT DEVELOPMENT

155

3803 East Bayshore Road Palo Alto, CA 94303-0979 1-800-624-1765 PSI Preview Kit $48.00 PSI Item Booklet $34.00 PSI Scoring Key $14.00 PSI Manual $32.00 Construct The Problem-Solving Inventory (PSI) assesses an individual's awareness and evaluation of problem-solving abilities or style and, thus, provides a global appraisal of that individual as a problem solver (i.e., problemsolving appraisal). Description The PSI presents 32 items generated to represent Problem-Solving Confidence, Approach-Avoidance, and Personal Control, across the different stages of problem solving (e.g., generating alternatives). The scale uses a 6-point response format anchored from strongly agree to strongly disagree. The scale does not measure problem-solving but the respondent's perceived problem-solving skills. Based on factor analysis, three subscales are computed: • Problem-Solving Confidence: confidence in a range of problemsolving activities, • Approach-Avoidance: dealing with or avoiding different problem-solving activities, and • Personal Control: elements dealing with self-control. Estimated Time to Administer Fifteen minutes. Reliability Temporal Consistency Heppner and Petersen (1982) initially reported the following 2-week test-retest reliabilities: Problem-Solving Confidence (.85), ApproachAvoidance style (.88), Personal Control (.83), and total inventory (.89). Similar estimates have been reported across subsequent investigations (see Heppner, 1988). 156

RESEARCH SCALES

Internal Consistency Heppner and Petersen (1982) initially reported the following coeffi' cient alphas: Problem-Solving Confidence (.85), Approach-Avoidance style (.84), Personal Control (.72), and total inventory (.90). Similar estimates have been reported across subsequent investigations (see Heppner, 1988). Factor Structure

Heppner and Petersen (1982) conducted a principal-components factor analysis with varimax rotation. The factor analysis did not support the five stages but found that the following three underlying dimensions cut across stages: confidence in one's problem-solving ability, an approach-avoidance style, and personal control. In addition, the total score on the scale can be used as a general indication of perceived problem-solving skills. The factor structure has been replicated in several other investigations (see Heppner, 1988), including samples from Turkey (Sahin, Sahin, & Heppner, 1993) and Black South Africans (Heppner, Pretorius, Wei, Lee, & Wang, 2000). Validity Known Groups

Hepper and Petersen (1982), using a posttest-only, control group design, compared a group of participants who had received problem-solving training with a control group. The training consisted of six 1-hour sessions focused on the problem-solving process. In addition, participants conducted homework assignments "covering a wide range of problem-solving skills" (p. 71). Participants in the group receiving the training perceived more problem-solving skills than did the control group. Convergent Validity

Heppner and Petersen (1982) found that the three factor scores and the total score correlated significantly with the Rotter Locus of Control Scale (Rotter, 1966). Discriminant Validity The three factor scores and the total score did not correlate significantly with the Marlowe-Crowne Social Desirability Scale (Crowne & Marlowe, 1964), a measure of intelligence, high school rank, knowledge of mechanics, written communication, and algebraic skills (Heppner & Petersen, 1982). Only Personal Control correlated significantly (-.27 and -.34) with the fluency and flexibility scores of the Unusual Uses Activity of the Torrance Test of Creative Thinking (Torrance, 1966). The only significant correlation STUDENT DEVELOPMENT

157

(.25) between the PSI and the four dimensions of the Myers-Briggs Type Inventory (Myers, 1962) was between Personal Control and the thinkingfeeling dimension (Heppner & Petersen, 1982). Gender Differences In well over 100 empirical investigations, very few gender differences have been reported on the PSI (P. Heppner, personal communication, May 15, 2002). Comments One of the strengths of service learning occurs when students are faced with real but perplexing problems (Eyler, 2000; Eyler & Giles, 1999; Hatcher, 1997). The dissonance that results from these circumstances provides the impetus to define problems, to determine which knowledge is relevant, to understand what may be and may not be possible, and to explore solutions. High-quality service learning experiences that allow interplay between theory and practice may lead to educational experiences that enhance perceptions of problem-solving skills (Eyler, 2000). Although this scale is not specific for the community service context (for examples, see Batchelder & Root, 1994; Eyler & Giles, 1999), it might still be useful in research on service learning. References Batchelder, T. H., & Root, S. (1994). Effects of an undergraduate program to integrate academic learning and service: Cognitive, prosocial cognitive and identity outcomes. ]oumal of Adolescence, 17, 341-355. Crowne, D., & Marlowe, D. (1964). The approval motive. New York: Wiley. Eyler, J. (2000, Fall). What do we most need to know about the impact of servicelearning on student learning? Michigan Journal of Community Service Learning, 11-17. Eyler, J., & Giles, D. E., Jr. (1999). Where's the learning in service-learningl San Francisco: Jossey-Bass. Hatcher, J. A. (1997). The moral dimensions of John Dewey's philosophy: Implications for undergraduate education. Michigan Journal of Community Service Learn' ing, 4, 22-29. Heppner, P. P. (1988). The Problem Solving Inventory (PSI): Manual. Palo Alto, CA: Consulting Psychologists Press. Heppner, P. P., &. Petersen, C. H. (1982). The development and implications of a personal Problem-Solving Inventory. Journal of Counseling Psychology, 29, 66-75.

158

RESEARCH SCALES

Heppner, P. P, Pretorius, T., Wei, M., Lee, D., & Wang, Y. (August, 2000). Associations between problem solving appraised and psychological adjustment in South Africa. Paper presented at the 108th Annual Convention of the American Psychological Association, Washington, DC. Myers, I. B. (1962). The Myers-Briggs Type Indicator manual. Palo Alto, CA: Consulting Psychologists Press. Rotter, J. B. (1966). Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs, 80(Whole No. 609). Sahin, N., Sahin, N. H., & Heppner, P. P. (1993). Psychometric properties of the Problem Solving Inventory in a group of Turkish university students. Cognitive Therapy and Research, 17, 379-396. Torrance, E. P. (1966). Torrance tests of creative thinking: Norms-technical manual. Princeton, NJ: Personnel Press.

Sample Items

Read each statement, and indicate the extent to which you agree or disagree with that statement, using the following options: 1 1 3 4 5 6

= strongly agree = moderately agree = slightly agree = slightly disagree = moderately disagree = strongly disagree

5. I am usually able to think up creative and effective alternatives to solve a problem. 10. I have the ability to solve most problems even though initially no solution is immediately apparent. 13. When confronted with a problem, I tend to do the first thing that I can think to solve it.* 14. Sometimes I do not stop and take time to deal with my problems, but just kind of muddle ahead.* 18. When making a decision, 1 weigh the consequences of each alternative and compare them against each other. 25. Even though I work on a problem, sometimes I feel like I am groping or wandering, and am not getting down to the real issue.* * reverse-scored Problem-Solving Confidence (Items 5, 10) Approach-Avoidance style (Items 13, 18) Personal Control (Items 14, 25) STUDENT DEVELOPMENT

159

CAREER DECISION-MAKING SELF-EFFICACY SCALE Source Taylor, K. M., & Betz, N. E. (1983). Applications of self-efficacy theory to the understanding and treatment of career indecision. Journal of Vocational Behavior, 22, 63-81. Cost to order the specimen set, which includes use in one study of the Career Decision-Making Self-Efficacy Scale (CDSES) or the short form (CDSES-SF; unlimited copying, though user must provide his/her own answer sheets and computer scoring), the CDSES/CDSES-SF Manual, and the scoring keys, is $75.00. If materials are sent as an email attachment (MS Word), no postage cost is needed. If mailing is desired, then the recipient must enclose the amount for U.S. priority mail, for Express mail, or Global Priority Mail (for mailing to Canada or other international addresses). The materials are available from: Dr. Nancy Betz 478 Whitney Avenue

Worthington, OH 43085 [email protected] Checks only, payable to Nancy E. Betz, PhD. Construct The CDSES is designed to assess self-efficacy expectations of career decision-making tasks based on the five Career Choice Competencies of Crites's (1961, 1965) model of career maturity. Description The CDSES consists of 50 items over five subscales assessing selfefficacy expectations regarding career decision-making tasks. The five subscales of the instrument are Goal Selection, Occupational Information, Problem Solving, Planning, and Self-Appraisal. Respondents are asked to rate each item on a 10-point scale ranging from complete confidence (9) to no confidence (0). Scores can be compiled for each subscale and the total scale by summing the item ratings. A 25-item short form, the CDSES-SF, has also been developed. Estimated Time to Administer Twenty-five to thirty minutes. 160

RESEARCH SCALES

Reliability Temporal Consistency

Luzzo (1993) reports a 6-week test-retest reliability of .83 for the scale. Internal Consistency

Taylor and Betz (1983) obtained a coefficient alpha of .97 for the total scale and the following alphas for each of the subscales: .88 for Self-Appraisal, .89 for Occupational Information, .87 for Goal Selection, .89 for Planning, and .86 for Problem Solving. For the short form, alphas ranged from .73 (Self-Appraisal) to .83 (Goal Selection), and .94 for the 25-item scale (Betz, Klein, & Taylor, 1995). Factor Structure

A principal-components analysis did not fully support the five-factor structure. Individual items load on more than one factor. One large general factor seems to appear which includes items from all five subscales (Taylor & Betz, 1983; Taylor & Popma, 1990). Validity Known Groups Robbins (1985) found that the CDSES correctly placed 70% of cases into the corresponding high or low vocational identity group. Bergeron and Romano (1994) found significant differences among students who were undecided, tentatively decided, and decided about their vocation and scores on the CDSES. Bergeron and Romano (1994) found similar results regarding college major indecision and CDSES scores. Convergent Validity

CDSES and the Career Decision Scale were found to be moderately and negatively related (Taylor & Betz, 1983), indicating that students who were more undecided in their educational/vocational decision report less confidence in decision-making task abilities. Taylor and Popma (1990) reported moderate and strong relationships between scores on the CDSES and Occupational Self-Efficacy Scale and vocational decidedness. Luzzo (1995) found age of participants to be significantly correlated with the CDSES. Robbins (1985) found the CDSES to be moderately correlated to self-esteem (.53) and related to general anxiety (.24). The scale was also found to be moderately rated to the My Vocational Situation Inventory (Taylor & Betz, 1983). Relationships of the CDSES-SF to the Career

STUDENT DEVELOPMENT

161

Decision Scale and My Vocational Situation Inventory were even higher than those using the original CDSES (Betz & Luzzo, 1996). Discriminant Validity CDSES scores were not found to be significantly related to ACT or SAT scores (Taylor & Betz, 1983). Gender Differences Taylor and Betz (1983) and Luzzo (1995) reported no gender differences on scores on the CDSES. Comments One of the potential benefits of service learning experience is clarifying career goals, especially for traditionally aged students in the early years of their college education (Astin & Sax, 1998; Driscoll, Holland, Gelmon, & Kerrigan, 1996; Vogelgesang & Astin, 2000). Interaction with professionals in community settings allows students to gain a more realistic understanding of the demands and rewards of particular types of work. Students also may benefit from being around professionals who model the skills and attributes of a specific career. This scale provides an important tool for measuring outcomes related to career decisions. References Astin, A. W., & Sax, L. J. (1998). How undergraduates are affected by service participation. Journal of College. Student Development, 39, 251-263. Bergeron, L. M., & Romano, J. L. (1994). The relationships among career decisionmaking self-efficacy, educational indecision, vocational indecision, and gender. journal of College Student Development, 35, 19-24. Betz, N. E., Klein, K., & Taylor K. M. (1995). Evaluation of a short form of the Career Decision-Making Self-Efficacy Scale. Journal of Career Assessment, 4, 47-57. Betz, N. E., & Luzzo, D. (1996). Career Assessment and the Career Decisions SelfEfficacy Scale. Journal of Career Assessment, 4, 313-328. Crites, J. O. (1961). A model for the measurement of vocational maturity. Journal of Counseling Psychology, 8, 255-259. Crites, J. O. (1965). Measurement of vocational maturity in adolescence: I. Attitude test of the Vocational Development Inventory. Psychological Monographs, 79(2, Whole No. 595). Driscoll, A., Holland, B., Gelmon, S., & Kerrigan, S. (1996). An assessment model for service-learning: Comprehensive case studies of impact on faculty, students,

162

RESEARCH SCALES

community, and institutions. Michigan Journal of Community Service Learning, 3, 66-71. Luzzo, D. A. (1993). Reliability and validity testing of the Career Decision-Making Self-Efficacy Scale. Measurement and Evaluation in Guidance and Counseling, 26, 137-142. Luzzo, D. A. (1995). The relative contributions of self-efficacy and locus of control to the prediction of career maturity. Journal of College Student Development, 36, 61-66. Robbins, S. B. (1985). Validity estimates for the Career Decision-Making SelfEfficacy Scale. Measurement and Evaluation in Counseling and Development, 18, 16-71. Taylor, K. M., &. Betz, N. E. (1983). Applications of self-efficacy theory to the understanding and treatment of career indecision. Journal of Vocational Behavior, 22, 63-81. Taylor, K. M., & Popma, J. (1990). An examination of the relationships among career decision-making self-efficacy, career salience, locus of control, and vocational indecision. Journal of Vocational Behavior, 37, 17-31. Vogelgesang, L. J., & Astin, A. W. (2000). Comparing the effects of service-learning and community service. Michigan Journal of Community Service Learning, 7, 25-34Sample Items Please indicate your confidence in your ability to successfully complete each task. Rate each item on a 10-point scale ranging from complete confidence (9) to no confidence (0). 4. Choose a career that will fit your preferred lifestyle (Goal Selection) 11. Describe job duties of the career/occupation you would like to pursue. (Occupational Information) 47. Move to another city to get the kind of job you really would like. (Problem Solving) 33. Prepare a good resume. (Planning) 8. Decide what you value most in an occupation. (Self-Appraisal) Note. From the Career Decision-Making Self-Efficacy Scale. Copyright 1983 by Nancy E. Betz and Karen M. Taylor. Reprinted with permission of the authors.

STUDENT DEVELOPMENT

163

8 ATTITUDES

AIDS CAREGIVER SCALE Source Ferrari, J. R., McCown, W., &. Pantano, J. (1993). Experiencing satisfaction and stress as an AIDS care provider: The Caregiver Scale. Evaluation and the Health Professions, 16, 295-310. Construct The AIDS Caregiver Scale assesses the emotional experiences associated with working as a care provider to others. Emotional experiences are measured on two subscales: personal satisfaction and emotional stress. Description The AIDS Caregiver Scale is a 14-item scale assessing one's emotional experiences working as a care provider. The scale is comprised of two subscales: (a) Satisfaction, which measures feelings of personal fulfillment or joy from helping others (seven items); and (b) Stress, which measures feelings of negative affect or depression from helping others (seven items). Respondents are asked to rate their responses on a 7-point scale ranging from strong^ disagree to strongly agree. 165

Although the original scale was developed for use with AIDS caregivers, the scale has been adapted and validated for use with health care providers (Ferrari et al., 1993), community volunteers (Ferrari, Loftus, & Pesek, 1999), and pastoral caregivers (Ferrari, Jason, & Salina, 1995), as well as individuals working with the physically disabled (Ferrari & Jason, 1997), elderly (Ferrari, Dobis, et al., 1999), and homeless (Ferrari, Billows, Jason, & Grill, 1997). Recently, the scale has been used with urban and suburban high school students in service learning classes (Ferrari, Campbell, & DeFilippo, 2002) and college students in experiential and service learning programs (Ferrari, 2002). Time to Administer Five to ten minutes. Reliability Temporal Consistency Ferrari et al. (1993) reported a 6-month test-retest reliability of .68 with AIDS caregivers. Internal Consistency Ferrari et al. (1993) found coefficient alphas of .82 for the Satisfaction subscale, .80 for the Stress subscale, and .86 for the overall scale when administered to volunteer AIDS caregivers. Ferrari, Dobis, et al. (1999) found coefficient alphas of .87 for the Satisfaction subscale and .85 for the Stress subscale using service learning volunteers. Factor Structure When initially developed for AIDS caregivers, Ferrari et al. (1993) conducted a factor analysis with varimax rotation on 16 items. Fourteen of the original 16 items loaded on two factors. The remaining 2 items loaded on their own separate factors. These 2 items were then dropped to form the 14'item scale. Validity Known Groups No information has been located. Convergent Validity Caregiver satisfaction was found to be correlated .57, .65, and .54 with the Values, Esteem, and Protective subscales of the Volunteer Functions 166

RESEARCH SCALES

Inventory (Clary, Snyder, & Ridge, 1992), respectively (Ferrari, Dobis, et al., 1999). Discriminant Validity Ferrari, Dobis, et al. (1999) found no significant correlations between scores on the Caregiver Scale and the Marlowe-Crowne Social Desirability Scale (Crowne &. Marlowe, 1960) with a volunteer population. Gender Differences No significant gender differences were found (Ferrari et al., 1997; Ferrari et al., 1993). Comments The Caregiver Scale is focused specifically on helping and two important components of the experience: satisfaction and stress. The scale provides an excellent tool for tracking the experiences of students who are providing direct service to others. It can also be used to clarify the circumstances at the service site (e.g., good orientation, good feedback from service supervisor, on-site reflection activities) that intensify positive or negative outcomes for students. References Clary, E. G., Snyder, M., & Ridge, R. (1992). Volunteer's motivations: A functional strategy for the recruitment, placement, and retention of volunteers. NonProfit Management and Leadership, 2, 333-350. Crowne, D. P., & Marlowe, D. (1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349-354Ferrari, J. R. (2002). Student community service values: Effects of service day, week, and term durations. Unpublished manuscript, DePaul University, Chicago, IL. Ferrari, J. R., Billows, W., Jason, L. A., &. Grill, G. J. (1997). Matching the needs of the homeless with those of the disabled: Empowerment through caregiving. Journal of Prevention and Intervention in the Community, 15, 83-92. Ferrari, J. R., Campbell, A., & DeFilippo, B. (2002). Caucasian and minority adolescents as community volunteers: Comparing races on sense of community and caregiver experiences. Manuscript submitted for publication. Ferrari, J. R., Dobis, K., Kardaras, E. I., Michna, D. M., Wagner, J. M., Sierawski, S., & Boyer, P. (1999). Community volunteerism among college students and professional psychologists: Does taking them to the streets make-a-difference? Journal of Prevention and Intervention in the Community, 18, 35-51.

ATTITUDES

167

Ferrari, J. R., & Jason, L. A. (1997). Caring for people with chronic fatigue syndrome: Perceived stress versus satisfaction. Rehabilitation Counseling Bulletin, 40, 240250. Ferrari, J. R., Jason, L. A., & Salina, D. (1995). Pastoral care and AIDS: Assessing the stress and satisfaction from caring for persons with AIDS. Pastoral Psychology, 44, 99-110. Ferrari, J. R., Loftus, M. M, & Pesek, J. (1999). Young and older caregivers at homeless animal and human shelters: Selfish and selfless motives in helping others. Journal of Social Distress and the Homeless, 8, 37-49. Ferrari, J. R., McCown, W., & Pantano, J. (1993). Experiencing satisfaction and stress as an AIDS care provider: The Caregiver Scale. Evaluation and Health Professions, 16, 295-310.

Scale Please rate each statement using the scale below on how well it reflects your opinion about your experience(s) in your school's community service or service learning program. (Please review the scale and circle the corresponding number as appropriate.) 1 2 3 4

= = = =

strongly disagree moderately disagree disagree neither agree or disagree

5 = agree 6 = moderately agree 1 = strongly agree

1. Helping people through the volunteer program is worthwhile t o m e personally. 2. Helping people through the volunteer program h a s "burned m e out." 3. My volunteering with people through the program makes me feel 1 am contributing to the community. 4. I sometimes feel very sad when I think of the people I serve in volunteer program. 5. My volunteering has been psychologically or spiritually fulfilling. 6. Volunteering in the program has exhausted me. 7. Helping people through the volunteer program c a n prolong m y life. 8. I am very discouraged because of the demands on me by volunteering in the program. 9. I feel that through my volunteer efforts, the school program has genuinely helped someone. 10. I wonder if it is worth the personal commitment to help someone or cause in the community. 168

RESEARCH SCALES

1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7

11. I have done so much through the volunteer program that I feel that I can't help anyone else. 12. My volunteering in the school's program has added to other areas of my life (spirituality, family, friends, emotional). 13. Volunteering in the program is adding meaning t o m y life. 14. I've experienced too many difficult situations through the volunteer program to want to help anyone else in the future.

1 2 3 4 5 6 7

1 2 3 4 5 6 7 1 2 3 4 5 6 7

1 2 3 4 5 6 7

Satisfaction (Items 1, 4, 6, 7, 9, 12, 13) Stress (Items 2, 3, 5, 8, 10, 11, 14) Note. From "Experiencing Satisfaction and Stress as an AIDS Care Provider: The Caregiver Scale," by J. R. Ferrari, W. McCown, and J. Pantano, J., 1993, Evaluation and the Health Professions, 16, p. 309. Copyright 1993 by Sage Publications. Reprinted with permission.

CIVIC ACTION Source Moely, B. E., Mercer, S. H., Ilustre, V., Miron, D., & McFarland, M. (2002). Psychometric properties and correlates of the Civic Attitudes and Skills Questionnaire (CASQ): A measure of students' attitudes related to service-learning. Michigan Journal of Community Service Learning, 8(2), 15-26. Construct The Civic Action scale measures "intentions to become involved in the future in some community service or action" (Moely, Mercer et al., 2002, p. 15). Description The Civic Action scale has eight items for which respondents indicate their agreement or disagreement on a 1 (disagree completely) to 5 (agree completely) response scale. Moely, Mercer et al. (2002) stated that the items for the Civic Action scale are similar to items used by Astin and Sax (1998) on their Civic Responsibility Scale. The Civic Action scale is one of six subscales of the Civic Attitude and Skills Questionnaire. The other subscales ATTITUDES

169

are (a) Interpersonal and Problem-Solving Skills, (b) Political Awareness, (c) Leadership Skills, (d) Social Justice Attitudes, and (e) Diversity Attitudes. Estimated Time to Administer Five minutes. Reliability Temporal Consistency Three-month test-retest reliabilities for the Civic Action scale were .74 and .71 for two separate samples of students not enrolled in service learning classes, after controlling for Social Desirability (Kirby, 1995) measured on the pretest (Moely, Mercer et al., 2002). Internal Consistency

Internal consistency using coefficient alpha was .86 and .88 for two samples (Moely, Mercer et al., 2002). Factor Structure

Factor analyses were conducted with two samples (ns = 761, 725) using principal-components analysis and varimax rotation with Kaiser normalization on the Civic Attitudes and Skills Questionnaire, which contains the Civic Action scale. According to Moely, Mercer et al. (2002), the pattern of factor loadings was consistent for the two samples and demonstrated convergence on how items loaded on the six factors. Validity Known Groups Moely, McFarland, Miron, Mercer, and Ilustre (2002) compared students in service learning and traditional classes on pretest-posttest administration of the Civic Action scale given at the beginning and the end of the semester. The results indicated that service learning students significantly increased on civic action but students in traditional classes did not. Convergent Validity A composite index of students' estimates of the number of hours of community service that they had engaged in during high school, during college, and through religious volunteering was correlated .35 and .29 in two separate samples with scores on the Civic Action scale, after controlling 170

RESEARCH SCALES

for Social Desirability. Civic Action scale scores were negatively correlated -.32 and -.31 in two samples with scores on the Modern Racism Scale (McConahay & Hough, 1976), positively correlated .36 and .35 with scores on the Value of College (Moely, Mercer et al., 2002), and positively correlated .44 with Mastery Orientation (Moely, Stantulli, & Obach, 1995). The Civic Action scale was also found to be related to students' feelings of having made a contribution to the community through service (Moely, McFarland et al., 2002). Discriminant Validity The Civic Action scale was correlated .27 and .28 with Social Desirability (Kirby, 1995) in two samples. Gender Differences Female students scored higher than male students on the Civic Action scale in two samples, after controlling for Social Desirability (Moely, Mercer et al., 2002). Comments Service learning aspires not only for students to learn academic lessons through service but also for students to learn to serve. Indeed, one of the unique niches of service learning is educating students for civic participation (Barber & Battistoni, 1994; Boyte & Hollander, 1999; Ehrlich, 1997; Kahne, Westheimer, &. Rogers, 2000). The Civic Action scale is among a number of scales that have been developed to measure this outcome (e.g., Astin &. Sax, 1998; Eyler & Giles, 1999) and has good psychometric evidence to date. The scale can be useful as an outcome measure, as a measure of change over a period of time, or as a pretest measure to covary out preexisting differences due to students' self-selection into service learning and traditional classes (Eyler & Giles, 1999). References Astin, A. W., & Sax, L. J. (1998). How undergraduates are affected by service participation. Journal of College Student Development, 39, 251-263. Barber, B., & Battistoni, R. (1994). A season of service. PS: Political Science and Politics, 26, 235-262. Boyte, H., & Hollander, E. (1999). Wingspread declaration on the civic responsibility of research universities. Providence, RI: Campus Compact. Ehrlich, T. (1997, Summer-Fall). Civic learning: Democracy and education revisited. Educational Record, 57-65.

ATTITUDES

171

Eyler, J., & Giles, E. E., Jr. (1999). Where's the learning in service-learning? San Francisco: Jossey-Bass. Kahne, J., Westheimer, J., & Rogers, B. (2000, Fall). Service-learning and citizenship: Directions for research. Michigan Journal of Community Service Learning, 42-51. Kirby, J. (1995). Influences on academic achievement: Goal orientation, strategy use, and self-regulation in university students. Unpublished master's thesis, Tulane University, New Orleans, LA. McConahay, J., & Hough, J., Jr. (1976). Symbolic racism. Journal of Social Issues, 32(2), 23-45. Moely, B. E., McFarland, M., Miron, D., Mercer, S., & Ilustre, V. (2002). Changes in college students' attitudes and intentions for civic involvement as a function of service-learning experiences. Michigan Journal of Community Service Learning, 9, 18-26. Moely, B. E., Mercer, S. H., Ilustre, V., Miron, D., & McFarland, M. (2002). Psychometric properties and correlates of the Civic Attitudes and Skills Questionnaire (CASQ): A measure of students' attitudes related to service-learning. Michigan Journal of Community Service Learning, 8(2), 15-26. Moely, B. E., Santulli, K. A., & Obach, M. S. (1995). Strategy instruction, metacognition, and motivation in the elementary school classroom. In F. Weinert & W. Schndeider (Eds.), Memory performance and competencies: Issues in growth and development (pp. 301-321). Hillsdale, NJ: Erlbaum. Scale 1 = disagree completely

5 = agree completely

I plan to do some volunteer work. I plan to become involved in my community. I plan to participate in a community action program. I plan to be become an active member of my community. In the future, I plan to participate in a community service organization. I plan to help others who are in difficulty. I am committed to making a positive difference. I plan to become involved in programs to help clean up the environment. Note. From "Psychometric Properties and Correlates of the Civic Attitudes and Skills Questionnaire (CASQ): A Measure of Students' Attitudes Related to Service-Learning," by B. E. Moely, S. H. Mercer, V. Ilustre, D. Miron, and M. McFarland, 2002, Michigan journal of Community Service Learning, 8(2), p. 19 (Table 2). Copyright 2002 by OCLS Press, University of Michigan. Reprinted with permission.

172

RESEARCH SCALES

COMMUNITY SERVICE INVOLVEMENT PREFERENCE INVENTORY Source Payne, C. A., & Scott, N. A. (1993). Investigating involvement preferences to understand the impact of service-learning. Unpublished manuscript. Contact: Christopher A. Payne Director of Housing and Residential Education CB #5500, 110 Carr Building Chapel Hill, NC 27599 Construct The Community Service Involvement Preference Inventory (CSIPI) was designed to assess the cognitive, behavioral, and affective changes that students experience as a result of service learning or campus/community service experiences. The instrument's design was based on Delve, Mintz, and Stewart's (1990) service learning model, which focuses on values clarification and development through different types of involvement in community service and commitment to community service. However, rather than or in addition to measuring values, the scale focuses on the nature of the student's involvement and experiences that are consistent with Delve et al.'s model. Description The CSIPI consists of 48 items, separated into four subscales that correspond to Delve et al.'s (1990) service learning model. The CSIPI is a revision of a scale Payne (1992) developed through his doctoral work. The CSIPI measures the following four preferences for involvement in service: • Exploration: "The exploration score reflects the affective nature of apprehension common in new experiences. Students are unsure how to best help others and tend to become involved because of the self-satisfaction associated with helping others. From a behavioral perspective the commitment is short term and is usually at the convenience of the helper. Cognitively, a high score reflects a desire to learn about oneself. Because of the lack of previous exposure to community service, there may not be an awareness of the variety of individuals ATTITUDES

173

and organizations in the community that need help" (Payne & Scott, 1993, p. 8). • Affiliation: "Items for this involvement preference elicit a preference for participating in community service with a group of peers. A high score implies that the person is more comfortable helping others when working in a group. Recognition by peers is a common behavioral motivation for involvement. A variety of short-term commitments are preferred to involvement that is longer in duration. Understanding the motives for involvement in community service is achieved by experiencing a variety of ways to help people overcome social issues and concern" (Payne & Scott, 1993, pp. 8-9). • Experimentation: "Items written for the Experimentation preference focus on the personal challenge of becoming involved in community service. A feeling of making a difference in the lives of others and a desire to learn about the needs of the individual or community served are dominant affective qualities. Behaviorally there is a tendency to move away from selfsatisfaction as a reason to become involved. Especially as the individual begins to spend more time with a specific issue or concern, direct interaction provides the student with insights into the needs of the individual or community served. As with the SLM [service learning model] (Delve et al., 1990), the reciprocal nature of the service effort provides the helper with a deeper understanding of how to empower others to help themselves" (Payne & Scott, 1993, p. 9). • Assimilation: "The preference for Assimilation reflects a feeling of lifelong commitment and frequent contact with the individual or community served. Behaviorally the student may take an active role in helping other volunteers become involved in community service. The deeper understanding of the needs of those served parallels a personal investment in long term commitments. Cognitively, the student begins to make career and lifestyle decisions based on the service experience as a way to understand what it is to be a responsible citizen" (Payne & Scott, 1993, pp. 9-10). Responses are based on a 5-point response format ranging from strongly disagree to strongly agree. Estimated Time to Administer Fifteen minutes.

174

RESEARCH SCALES

Reliability Temporal Consistency No information has been located. Internal Consistency According to Payne (1992), coefficient alphas for the four subscales of the CSIPI were .53 (Exploration), .62 (Affiliation), .76 (Experimentation), and .73 (Assimilation). Factor Structure No information has been located. Validity Known Groups Payne and Scott (1993) found that service learning and nonservice learning students differed on the four subscales. Nonservice learning students scored significantly higher on the Exploration and significantly lower on the other three subscales. Those respondents reporting eight or fewer service contacts during the past year also scored significantly higher on the Exploration and lower on the other three subscales, when compared with those students reporting more than eight contacts. Payne (1997) found that students in Greek organizations scored significantly higher on Exploration and Affiliation than did students enrolled in a service learning class. Payne and Bennett (1996) reported significant difference scores across the semester on the Affiliation, Experimentation, and Assimilation subscales for students enrolled in a service learning class. Convergent Validity No information has been located. Discriminant Validity No information has been located. Gender Differences No information has been located. Comments Delve et al.'s (1990) service learning model for student involvement in service is important for providing researchable questions about motives

ATTITUDES

175

for and the impact of student involvement in service activities on their development. This is particularly the case for service learning experiences that exceed a semester in duration. Research can provide important contributions by studying how intensity of experiences, different types of experiences, and different types of reflection are related to changes in how students approach their service. Payne's CSIPI is an important measure of the model, although the coefficient alpha's are marginal and nothing is known about the factor structure of the scale. The results of Payne's research are encouraging, and this scale and the service learning model deserve more research and evaluation. References Delve, C. I., Mintz, S. D., & Stewart, G. M. (1990). Promoting values development through community service: A design. In C. I. Delve, S. D. Mintz, & G. M. Stewart. (Eds.), New directions for student services as values education (No. 50, pp. 7-29). San Francisco: Jossey-Bass. Payne, C. A. (1992). Construction of an instrument to assess the service learning model: Establishing concurrent validity and internal reliability. Unpublished doctoral dissertation, University of Northern Colorado. Payne, C. A. (1997). Comparing community service learning involvement preferences for two groups: Students enrolled in a service-learning course and members of Greek organizations. Unpublished manuscript. Payne, C. A., & Bennett, E. B. (1996). Service-learning and changes in involvement preferences among undergraduates. Unpublished manuscript. Payne, C. A., & Scott, N. A. (1993). Investigating involvement preferences to understand the impact of service-learning. Unpublished manuscript.

Sample Items This inventory is designed to identify feelings, actions, and thoughts about how you experience and understand community service and helping others. Serving others is defined as the tendency to help someone, either directly or indirectly, other than family members or close friends. Defined in this way, the terms "serving" and "helping others" are used interchangeably. For the purpose of this administration, please use your involvement in community service over the past year as a reference point. Circle the degree to which you agree or disagree with each statement. Please respond to each statement. Do not skip any items. 1 = strong^ disagree 2 = disagree 3 = neutral 4 = agree 5 = strongfy agree 1 2 3 4 5

176

I am more aware of how I have stereotyped others in the past as a result of my involvement in community service. (Exploration)

RESEARCH SCALES

1 2 3 4 5 1 think most people in need rely too heavily on the social service system. (Experimentation) 1 2 3 4 5 Involvement in community service provides an opportunity to socialize with my friends. (Affiliation) 1 2 3 4 5 1 belong to a group or organization that expects members to be involved with activities designed to serve others. (Assimilation)

COMMUNITY SERVICE ATTITUDES SCALE Source Shiarella, A. H., McCarthy, A. M., & Tucker, M. L. (2000a). Development and construct validity of scores on the Community Service Attitudes Scale. Educational and Psychological Measurement, 60, 286-300. Construct The Community Service Attitudes Scale (CSAS) is designed to measure college students' attitudes about community service based on Schwartz's (1977) model of helping behavior. Description The scale is designed to measure attitudes contained within each step described in Schwartz's (1977) model. The Schwartz model consists of the following steps: • Phase I. Activation Steps: Perception of a need to respond. • Awareness that others are in need, • Perception that there are actions that could relieve the need, • Recognition of one's own ability to do something to provide help, and • A feeling of responsibility to become involved based on a sense of connectedness with the community or the people in need. • Phase II. Obligation Step: Moral obligation to respond. • Feeling a moral obligation to help generated through (a) personal or situational norms to help and (b) empathy. • Phase III. Defense Steps: Reassessment of potential responses. • Assessment of (a) costs and (b) probable outcomes (benefits) of helping. ATTITUDES

177

• Reassessment and redefinition of the situation by denial of the reality and seriousness of the need and the responsibility to respond. • Phase IV. Response Step: Engagement in helping behavior. • Intention to engage in community service or not. (Shiarella et al., ZOOOa) The survey consists of 46 items assessing community service attitudes on eight subscales. These eight subscales, derived from factor analysis, consist of varying aspects of the steps in Schwartz's model. All items use a 7-point response format. Estimated Time to Administer Twenty to thirty minutes. Reliability Temporal Consistency Test-retest reliabilities for the eight factors are as follows (Shiarella, McCarthy, & Tucker, ZOOOb): Normative Helping (.72), Connectedness (.70), Costs (.61), Awareness (.71), Benefits (.57), Seriousness (.64), Career Benefits (.52), and Intentions (.74). Internal Consistency

Shiarella et al. (2000a) reported coefficient alphas for each of the factors as follows: Normative Helping (.92), Connectedness (.93), Costs (.85), Awareness .(85), Benefits (.79), Seriousness (.84), Career Benefits (.72), and Intentions (.86). Factor Structure A principal-components analysis with varimax rotation yielded eight factors (Shiarella et al., 2000a). All factor loadings were greater than .40. Items were assigned to the factor on which the loading was the largest. Five items had approximately equal loadings on two different factors and therefore were assigned to the factor that was most consistent with the theory. Normative Helping contained items from the actions, ability, and norms scales of the Schwartz model. Connectedness consisted of items from the connectedness scale. Costs contained items from the costs scale, and Awareness was made up of items from the awareness and empathy scales. Intentions consisted of the intention-to-participate items, and Seriousness contained items from the seriousness scale. Benefit items split with career items loading together on career benefits and the remaining items on benefits.

178

RESEARCH SCALES

Validity Knotun Groups Shiarella et al. (ZOOOa) expected that nonbusiness majors would score higher on all of the scales than business majors because students may be drawn to the liberal arts and social sciences out of an interest in helping others. As expected, nonbusiness majors scored higher on most of the scales than did business majors. Convergent Validity

Previous community service experience and amount of previous involvement were positively related to scores on all of the scales (Shiarella et al., 2000a). Discriminant Validity

Shiarella et al. (ZOOOa) found no significant relationships between scores on the CSAS subscales and age, race, or college standing. Gender Differences Female students scored higher on all of the scales than male students (Shiarella et al., ZOOOa). Comments The CSAS was developed with a theoretical context that offers researchers an excellent opportunity to explore various aspects of students' perceptions of, attraction to, changes during and outcomes from service learning and community service. Schwartz's model describes a linear sequence of stages through which a person progresses toward action. The components of the model also represent a set of motives and a set of beliefs about engaging in service. The scale could be useful for providing descriptive information on students at a particular campus, comparing campuses, or a longitudinal study of how students change on a campus. This would supply useful information for designing programs, publicity, and recruitment strategies. In addition, the scale could be used as a moderator, mediating, or outcome variable in service learning classes. Researchers will need to take note of the possibility that scale items may not align with the specified factor structure. References Schwartz, S. H. (1977). Normative influences on altruism. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 10, pp. 221-279). New York: Academic Press.

ATTITUDES

179

Shiarella, A. H., McCarthy, A. M., & Tucker, M. L. (2000a). Development and construct validity of scores on the Community Service Attitudes Scale. Educational and Psychological Measurement, 60, 286-300. Shiarella, A. H., McCarthy, A. M., & Tucker, M. L. (2000b). Increasing college students' intentions to participate in community service with service-learning college courses. Manuscript in preparation, Colorado State University.

Scale We are trying to understand your willingness to donate your time regularly to a community service project. By community service, we mean a project in which you would volunteer at least twice a month for a couple of hours and use your skills and knowledge. In other words, this is more than just volunteering time stuffing envelopes or picking up trash. These types of community service projects require a long-term commitment (i.e., at least one semester) and offer you the opportunity to share you skills, as well as develop new ones. Examples include tutoring a child, organizing a fund-raising campaign, or designing a computer application for a nonprofit organization. Please answer the following questions about your feelings regarding community service projects using the definition provided previously. Some of the questions might appear similar, but each one measures a unique set of information. If some of the questions do not apply to you (e.g., you do not have a job), please skip those questions. Now, pretend you are going to volunteer for a community service project sometime in the next year. Questions 9-20 ask you about possible outcomes associated with volunteering. Use the following scale to rate how likely you feel these outcomes are to occur.

1

2

Extremely unlikely

Quite unlikely

3 Slightly unlikely

4 Neither likely nor unlikely

5 Slightly likely

6 Quite Likely

7 Extremely likely

9. I would be contributing to the betterment of the community. 10. I would experience personal satisfaction knowing that I am helping others. 11. I would be meeting other people who enjoy community service. 12. I would be developing new skills. 13. I would make valuable contacts for my professional career. 14. I would gain valuable experience for my resume. 15. I would have less time for my schoolwork. 16. I would have forgone the opportunity to make money in a paid position. 180

RESEARCH SCALES

17. . 18. . 19. . 20.

I would have I would have I would have I would have

less energy. less time to work. less free time. less time to spend with my family.

Again, pretend you are going to volunteer for community service, described earlier, sometime in the next year. Use the following scale to rate how you feel about Questions 21-50: 1

2

Strongly disagree

Disagree

21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32. 33. 3435. 36. 37. 38. 39. 40.

3

4 Slightly disagree

Neither agree nor disagree

5 Slightly agree

Agree

6

7 Strongly agree

I want to do this activity. Community groups need our help. It is important to help people in general. Improving communities is important to maintaining a quality society. I can make a difference in the community. Our community needs good volunteers. There are people in the community who need help. All communities need good volunteers. Volunteer work at community agencies helps solve social problems. Volunteers in community agencies make a difference, if only a small difference. College student volunteers can help improve the local community. Volunteering in community projects can greatly enhance the community's resources. I am responsible for doing something about improving the community. Contributing my skills will make the community a better place. It's my responsibility to take some real measures to help others in need. It is important to provide a useful service to the community through community service. It is important to me to have a sense of contribution and helpfulness through participating in community service. It is important to me to gain an increased sense of responsibility from participating in community service. When I meet people who are having a difficult time, I wonder how I would feel if I were in their shoes. I will participate in a community service project in the next year. ATTITUDES

181

41. I feel bad that some community members are suffering from a lack of resources. 42. I feel bad about the disparity among community members. 43. I feel an obligation to contribute to the community. 44. There are needs in the community. 45. Lack of participation in community service will cause severe damage to our society. 46. Without community service, today's disadvantaged citizens have no hope. 47. Other people deserve my help. 48. Community service is necessary to making our communities better. 49. It is critical that citizens become involved in helping their communities. 50. Community service is a crucial component of the solution to community problems. Almost done. Use the following scale to rate how you feel about Questions 51-54:

1

2

Strongly disagree

Disagree

51. 52. 53. 54-

3

4 Slightly disagree

Neither agree nor disagree

5 Slightly agree

Agree

6

7 Strongly agree

The more people who help, the better things will get. There are people who have needs which are not being met. My contribution to the community will make a real difference. Would you seek out an opportunity to do community service in the next year?

Normative Helping attitudes (Items 23, 24, 25, 26, 28, 29,30,31,32,34, 53) Connectedness (Items 33, 35, 36, 37, 38, 43, 47, 49) Costs (Items 15, 16, 17, 18, 19, 20) Awareness (Items 22, 27, 39, 41, 42, 44, 52) Benefits (Items 9, 10, 11, 12) Seriousness (Items 45, 46, 48, 50, 51) Career Benefits (Items 13, 14) Intentions (Items 21, 40, 54) Note. From "Development and Construct Validity of Scores on the Community Service Attitudes Scale," by A. H. Shiarella, A. M. McCarthy, and M. L. Tucker, 2000, Educational and Psychological Measurement, 60, pp. 291-293, Table 3. Copyright 2000 by Sage Publications. Reprinted with permission. 182

RESEARCH SCALES

GLOBAL BELIEF IN A JUST WORLD SCALE Source Lipkus, I. (1991). The construction and preliminary validation of a Global Belief in a Just World Scale and the exploratory analysis of the Multidimensional Belief in a Just World Scale. Personality and Individual Differences, 12, 1171-1178. Construct The Global Belief in a Just World Scale (GBJWS) assesses the general belief that "people get what they deserve and deserve what they get" (Lipkus, 1991, p. 1172). Description The GBJWS is a 7-item scale administered in booklet form. Responses are based on a 6-point scale with scores ranging from strong disagreement to strong agreement. Respondents are asked to indicate their level of agreement based on how well the statement applies to others and themselves. The GBJWS differs from its predecessor, the Multidimensional Belief in a Just World Scale (MBJWS) in that it collapses Rubin and Peplau's (1975) multidimensional conceptualization: personal efficacy, interpersonal control, and sociopolitical control. Previous factor analysis was critical of the unidimensionality of the MBJWS. In addition, gender differences have been shown to be a potential confounding factor in the MBJWS. The GBJWS is an attempt to provide an alternative measure. Estimated Time to Administer Five minutes. Reliability

Temporal Consistency No information has been located. Internal Consistency

Coefficient alpha for the overall scale is .82 (Lipkus, 1991). ATTITUDES

183

Factor Structure Maximum likelihood factor analysis was performed with primary factors of individual gender as well as collapsed across gender. Collapsing across gender produced one large single factor (eigenvalue = 4-83), accounting for 69% of the total variance prior to rotation. All of the items showed moderate to high loadings on the primary factor. O'Connor, Morrison, and Morrison (1996) studied the factor structure of the GBJWS. Their results support the one-factor solution for female but not for male participants. On deletion of the second item on the scale, a one-factor solution was also found for male participants. Validity Known Groups Glennon, Joseph, and Hunter (1993) found that scores on the GBJWS were significantly lower for Catholic-Nationalists than Protestant-Unionist groups in Northern Ireland, indicating that the disadvantaged Catholic group viewed the world as less just compared with their Protestant counterparts. Convergent Validity Lipkus (1991) found scores on the GBJWS were highly correlated with scores on the MBJWS. Scores on the GBJWS were also correlated (.46) with internal locus of control (Levensen, 1974). Discriminant Validity Crozier and Joseph (1997) found the GBJWS not to be correlated with the Francis Scale of Attitudes Towards Christianity (Francis & Stubbs, 1987). Gender Differences Male participants typically score higher than female participants on the GBJWS (Lipkus, 1991). Comments The just world hypothesis raises interesting issues for the attributions that persons make about those who need help. Bringle and Velo (1998) provided an attributional analysis of helping situations. They noted that the just world hypothesis is a deeply engrained belief system that allows individuals to assume that the world is a predictable place in which there 184

RESEARCH SCALES

are causal relationships between events. One of the implications of the just world hypothesis is that observers will stigmatize the person needing help (i.e., bad things happen to bad people). This is an understandable defensive inference that permits them to maintain the belief that good people, like themselves, can expect good things to continue to happen. Not believing in the just world hypothesis presents the more threatening possibility that events happen at random and there is no relationship between their actions today and their future outcomes. Exploring the nature of these inferences as students experience contact with various groups of individuals needing help will be interesting, as will better understanding for whom the just world hypothesis is heightened or reduced as a result of service experiences. The just world hypothesis and the associated attribution theories in social psychology provide a rich resource for developing research projects focused on better understanding the nature of community service, volunteering, and service learning experiences (see Bringle & Velo, 1998; Nadler, 2002). This scale provides a reasonable means for measuring belief in the just world hypothesis. References Bringle, R. G., & Velo, P. M. (1998). Attributions about misery. In R. G. Bringle & D. K. Dufiy (Eds.), With service in mind: Concepts and models for servicelearning in psychology (pp. 51-67). Washington, DC: American Association for Higher Education. Crozier, S., & Joseph, S. (1997). Religiosity and sphere-specific just world beliefs in 16- tol8-years-olds. Journal of Social Psychology, 137, 510-513. Francis, L. J., &. Stubbs, M. T. (1987). Measuring attitudes towards Christianity: From childhood to adulthood. Personality and Individual Differences, 8, 741-743. Glennon, F., Joseph, S., & Hunter, J. A. (1993). Just world beliefs in unjust societies: Northern Ireland. Journal of Social Psychology, 133, 591-592. Levensen, H. (1974). Activism and powerful others: Distinctions within the concept of internal-external locus of control, journal of Personality Assessment, 38, 377383. Lipkus, I. (1991). The construction and preliminary validation of a Global Belief in a Just World Scale and the exploratory analysis of the Multidimensional Belief in a Just World Scale. Personality and Individual Differences, 12, 11711178. Nadler, A. (2002). Inter-group helping relations as power relations: Maintaining or challenging social dominance between groups through helping, journal of Social Issues, 58, 487-502. O'Connor, W. E., Morrison, T. G., &. Morrison, M. A. (1996). The reliability and factor structure of the Global Belief in a Just World Scale. Journal of Social Psychology, 136, 667-668.

ATTITUDES

185

Rubin, Z., & Peplau, L. A. (1975). Who believes in a just world? Journal of Social Issues, 31, 65-88. Scale Please indicate your level of agreement on the following scale with respect to how well each statement applies to others and yourself.

1

2

3

4

strong disagreement

5

6

strong agreement

1. 2. 3. 4-

I feel that people get what they are entitled to have. I feel that a person's efforts are noticed and rewarded. I feel that people earn the rewards and punishments they get. I feel that people who meet with misfortune have brought it on themselves. 5. I feel that people get what they deserve. 6. I feel that rewards and punishments are fairly given. 7. I basically feel that the world is a fair place. Note. From "The Construction and Preliminary Validation of a Global Belief in a Just World Scale and the Exploratory Analysis of the Multidimensional Belief in a Just World Scale," by I. Lipkus, 1991, Personality and Individual Differences, 12, p. 1174 (Table 3). Copyright 1991 by Elsevier Science. Reprinted with permission.

LIFE ORIENTATION TEST REVISED Source Scheier, M. F., Carver, C. S., &. Bridges, M. W. (1994). Determining optimism from neuroticism (and trait anxiety, self-mastery, and self-esteem): A reevaluation of the Life Orientation Test. Journal of Personality and Social Psychobgy, 67, 1063-1078. Construct The Life Orientation Test Revised (LOT-R) assesses individual differences in dispositional optimism along the optimism-pessimism dimension. The LOT-R detects individual differences in relation to a general sense of optimism toward life goals.

186

RESEARCH SCALES

Description The LOT-R is a 10-item self-report measure assessing general expectations regarding positive versus negative outcomes. Although the original LOT contained 12 items and possessed a moderate degree of reliability and predictive and discriminant validity, the authors sought to improve the test by omitting weaker items. In the current version, 6 items address the construct of optimism with 4 additional distracter items. A 5-point response scale ranging from strongfy disagree to strong^ agree is used to determine the individual's degree of agreement. Estimated Time to Administer Five minutes. Reliability Temporal Consistency

Test-retest reliabilities of the LOT-R for four groups were as follows: 4 months (.68), 1 year (.60), 2 years (.56), and 28 months (.79) (Scheier et al., 1994). Internal Consistency

Internal consistency of the LOT was established using coefficient alpha (.82). Item-scale correlations for the LOT-R range from .43 to .63. Coefficient alpha for the six construct items was .78 (Scheier et al., 1994). Factor Structure

Principal-components analysis with varimax and oblique rotations of the LOT-R yielded one distinct factor with all item loadings .58 or higher. Furthermore, factor analysis for the LOT-R yielded a distinct Optimism factor when combined with items on accompanying measures. Validity Known Groups No information has been located. Convergent Validity Scores on the LOT-R correlated .48 with the Self-Mastery Scale (Pearlin &. Schooler, 1978), .50 with Rosenberg's Self-Esteem Scale (Rosenberg, 1965), and -.43 with the neuroticism subscale of the Guilford-

ATTZTUDES

187

Zimmerman Temperament Survey (Guilford, Zimmerman, &. Guilford, 1976). Scheier et al. (1994) suggested that these modest correlations indicate that the LOT-R shares some variance with these constructs but does measure a distinct component. Discriminant Validity No information has been located. Gender Differences No gender differences were found for scores on the LOT-R (Scheier et al., 1994). Comments Students enter service settings with a variety of attitudes about themselves, the world, and the service activities. Optimism, as measured by the LOT-R scale, incorporates a positive attitude about one's self, one's outcomes, and one's life. An interesting research question would be to understand the role that variations on this type of attitude play in how students approach their service, interact with staff and clients, and are affected by the service learning class (e.g., sense of self-efficacy). References Guilford, J. S., Zimmerman, W. S., & Guilford, J. P. (1976). The GuilfordZimmerrrum Temperament Survey handbook: Twenty-five years of research and application. San Diego, CA: EDITS. Pearlin, L. I., & Schooler, C. (1978). The structure of coping. Journal of Health and Social Behavior, 19, 2-21. Rosenberg, M. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press. Scheier, M. F., Carver, C. S., & Bridges, M. W. (1994). Determining optimism from neuroticism (and trait anxiety, self-mastery, and self-esteem): A reevaluation of the Life Orientation Test. Journal of Personality and Social Psychology, 67, 10631078.

Scale Please rate the statements using the following scale: 188

RESEARCH SCALES

0 1 2 3 4 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.

= = = = =

strong^ disagree disagree neutral agree strongly agree

In uncertain times, I usually expect the best. It's easy for me to relax/ If something can go wrong for me, it will.* I'm always optimistic about my future. I enjoy my friends a lot/ It's important for me to keep busy.* I hardly ever expect things to go my way.* I don't get upset too easily/ I rarely count on good things happening to me.* Overall, I expect more good things to happen to me than bad.

+

filler/distracter item * reverse-scored Note. From "Determining Optimism From Neuroticism (and Trait Anxiety, Self-Mastery, and Self-Esteem): A Reevaluation of the Life Orientation Test," by M. F. Scheier, C. S. Carver, and M. W. Bridges, 1994, Journal of Personality and Social Psychology, 67, p. 1073 (Table 6). Copyright 1994 by the American Psychological Association. Reprinted with permission. UNIVERSAL ORIENTATION SCALE Source Phillips, S. T., & Ziller, R. C. (1997). Toward a theory and measure of nonprejudice. Journal of Personality and Social Psychology, 72, 420-434. Construct The Universal Orientation Scale (UOS) measures nonprejudical attitudes, which are defined as "a universal orientation in interpersonal relations whereby the actor selectively attends to and accentuates the similarities between the self and diverse others" (Phillips & Ziller, 1997, pp. 420-421). Description The UOS is a 20-item scale on which respondents are asked to rate their perceptions of self-other similarities. Ratings are given on a 5-point scale ranging from does not describe me well to describes me very well. ATTITUDES

189

Estimated Time to Administer Ten to fifteen minutes. Reliability Temporal Consistency Phillips and Ziller (1997) reported a 6-week test-retest reliability of .75. Internal Consistency Phillips and Ziller (1997) reported a coefficient alpha of .76. Factor Structure

Phillips and Ziller (1997) conducted a series of factor analyses to determine the factor structure. Principal-components analysis with orthogonal rotation indicated that a two-factor structure best fit the data. Confirmatory factor analysis also suggested that although a one-factor solution fit the data, a two-factor solution was a better fit. However, the two factors were comprised of all positively worded and all reverse scored items, respectively. Therefore, Phillips and Ziller (1997) believed the distinction is unimportant for their purposes and used the scale unidimensionally. Validity Knoum Groups No information has been located. Convergent Validity The UOS correlated -.38 with the Dogmatism Scale (Rokeach, 1960), indicating that universally oriented individuals appear less close-minded than those not universally oriented. Moderate negative relationships (-.18 and -.19) were also found between the scores on the UOS and scores on the Modern Racism Scale (McConahay, Hardee, & Batts, 1981) and the Anti-Black Scale (Katz & Hass, 1988), respectively. Discriminant Validity The UOS was uncorrelated (-.05) with the Marlowe-Crowne Social Desirability Scale (Crowne & Marlowe, 1964). No relationship was found between age and scores on the UOS (Phillips & Ziller, 1997). J90

RESEARCH SCALES

Gender Differences Female participants tended to score higher on the UOS than male participants (Phillips & Ziller, 1997). Comments The UOS, which measures "we-ness" or a communal perspective that minimizes group inequality, is the complement of the Social Dominance Orientation Scale (Pratto, Sidanius, Stallworth, & Malle, 1994), which measures "us-them" characterizations that maximize distinctions between self and out-groups. Although one might expect that the UOS would be susceptible to socially desirability response tendencies, Phillips and Ziller (1997) reported a nonsignificant correlation. The UOS will be useful for measuring changes in perspective that can be expected from educative service learning experiences that follow principles of good practice. Particularly when service learning engages students in service settings that involve interactions with different persons, there is an important opportunity for the students' perspective to become more tolerant and understanding. As knowledge and familiarity with diverse groups increase, there may be a decrease in the degree to which differences are perceived and an increase in the perception of similarities. This scale provides a useful tool for tracking these changes. References Crowne, D. P., & Marlowe, P. (1964). The approval motive. New York: Wiley. Katz, I., & Hass, R. G. (1988). Racial ambivalence and the American value conflict: Correlational and priming studies of dual cognitive structures. Journal of Personality and Social Psychology, 55, 893-905. McConahay, J. B., Hardee, B.B., & Batts, V. (1981). Has racism declined in America? It depends upon who is asking and what is asked. Journal of Conflict Resolution, 25, 563-579. Phillips, S. T., &. Ziller, R. C. (1997). Toward a theory and measure of nonprejudice. Journal of Personality and Social Psychology, 72, 420-434. Pratto, F., Sidanius, J., Stallworth, L. M., & Malle, B. F. (1994). Social dominance orientation: A personality variable predicting social and political attitudes. Journal of Personality and Social Psychology, 67, 741-763. Rokeach, M. (1960). The open and closed mind. New York: Basic Books.

Scale Please read each statement carefully. Please rate each statement using the following scale: ATTITUDES

191

1

2

Does not describe me well

3

4

5 Describes me very well

1. The similarities between males and females are greater than the differences. 2. I tend to value similarities over differences when I meet someone. 3. At one level of thinking we are all of a kind. 4- I can understand almost anyone because I'm a little like everyone. 5. Little differences among people mean a lot.* 6. I can see myself fitting into many groups. 7. There is a potential for good and evil in all of us. 8. When I look into the eyes of others I see myself. 9. I could never get accustomed to living in another country.* 10. When I first meet someone I tend to notice differences between myself and the other person.* 11. "Between" describes my position with regard to groups better than does "in" and "out." 12. The same spirit dwells in everyone. 13. Older persons are very different than I am.* 14. I can tell a great deal about a person by knowing their gender.* 15. There is a certain beauty in everyone. 16. I can tell a great deal about a person by knowing his/her age.* 17. Men and women will never totally understand each other because of their inborn differences.* 18. Everyone in the world is very much alike because in the end we all die. 19. I have difficulty relating to persons who are much younger than I. 20. When I meet someone I tend to notice similarities between myself and the other person. * reverse-scored Note. From "Toward a Theory and Measure of Nonprejudice," by S. T. Phillips and R. C. Ziller, 1997, Journal of Personality and Social Psychology, 72, p. 422 (Table 1). Copyright 1997 by the American Psychological Association. Reprinted with permission.

192

RESEARCH SCALES

SOCIAL DOMINANCE ORIENTATION SCALE Source Pratto, F., Sidanius, J., Stallworth, L. M., & Malle, B. F. (1994). Social dominance orientation: A personality variable predicting social and political attitudes. Journal of Personality and Social Psychology, 67, 741-763. Construct The Social Dominance Orientation (SDO) scale measures individual differences in the degree of preference for inequality among social groups. Description The SDO scale measures the extent that one prefers in-group dominance and superiority over out-groups. Pratto et al. (1994) considered the SDO scale "to be a general attitudinal orientation towards intergroup relations" (p. 742). The scale consists of 16 items rated on a 7-point scale ranging from extremely positive to extremely negative. Estimated Time to Administer Five to ten minutes. Reliability Temporal Consistency

A 3-month test-retest reliability of .81 was found by Pratto et al. (1994). A test-retest reliability of .84 was found for another sample with "several months" between administrations of the scale (Pratto et al., 1994, p. 747). Internal Consistency Across 13 samples, coefficient alphas ranged from .80 to .89 (Pratto et al., 1994). Factor Structure Using principal-components analysis, a single factor structure was found using each sample in the study individually. Pratto et al. (1994) subjected the largest sample (n = 446) to confirmatory factor analysis using

ATTITUDES

193

maximum-likelihood estimation. Each item related significantly to the latent factor thereby supporting the initial analysis that the SDO assesses a single construct. Validity Known Groups Pratto, Stallworth, Sidanius, et al. (1997) reported that members of various occupations (clustered together) differed on the SDO scale. Pratto (1999) found that in college populations, White Americans scored higher than Asian Americans, Hispanics, and Blacks, and nongay people scored higher than gays and lesbians. Also, Levin (1996) presented results showing that Israeli Ashkenazim scored higher than Israeli Mizrachim, who scored higher than Israeli Arabs and Palestinians. These findings are consistent with predicting that the SDO scale is stronger in dominant than subordinate groups. Convergent Validity The SDO scale was found to be correlated with a variety of hierarchylegitimizing myth scales. The SDO scale was related most heavily with "ideologies concerning group prejudices against other nations, ethnic groups, and women" (Pratto et al., 1994, p. 748). Scores on the SDO scale were found to be correlated an average of-.46 across 10 samples with the Concern for Others subscale of the Interpersonal Reactivity Index (Davis, 1983) as well as negatively related to Super and NevilPs (1985) Altruism scale. Furthermore, the SDO scale was found to be correlated -.34 with Katz and Hass's (1988) Humanitarian-Egalitarian Scale and correlated -.27 to -.36 with the Jackson Personality Inventory Tolerance subscales (Jackson, 1976). Discriminant Validity Pratto et al. (1994) found that the scores on the SDO scale were uncorrelated to self-monitoring, self-consciousness, extraversion, neuroticism, and conscientiousness. An average correlation of .03 across the 10 samples was found between the California Personality Inventory Dominance subscale (Gough, 1987) and the SDO scale scores, and a correlation of-.006 was found between the SDO scale and the Jackson Personality Research Form Dominance subscale (Jackson, 1965). Gender Differences Male participants scored higher on the SDO scale than female participants (Pratto, Stallworth, & Sidanius, 1997; Pratto, Stallworth, Sidnaius, & Siers, 1997). 194

RESEARCH SCALES

Comments Heightened "us-them" thinking is a component of prejudicial thinking. Pratto (1999) found that high SDO scores are associated with strong ingroup identification for those in dominant groups. Service learning experiences that provide out-group contact (e.g., cultural, race, class) are assumed to be educationally beneficial when they mitigate preservice prejudice and promote a sense of "we-ness." SDO scores have also been found to be associated with hierarchical views of society (e.g., occupations and views of greater inequality in dominant and subordinate social groups; Pratto, 1996). This scale has good psychometric characteristics and provides a basis for tracking attenuation of we-they thinking in students across successful service learning experiences (Sidanius & Pratto, 1999). References Davis, M. H. (1983). Measuring individual differences in empathy: Evidence for a multidimensional approach. Journal of Personality and Social Psychology, 44, 113-126. Gough, H. (1987). California Psychological Inventory: Administrator's guide. Palo Alto, CA: Consulting Psychologists Press. Jackson, D. N. (1965). Personalty Research Form. Goshen, NY: Research Psychologists Press. Jackson, D. N. (1976). Jocfcson Personality Inventory. Goshen, NY: Research Psychologists Press. Katz, I., & Hass, R. G. (1988). Racial ambivalence and American value conflict: Correlational and priming studies of dual cognitive structures. Journal of Personality and Social Psychology, 55, 893-905. Levin, S. L. (1996). A socialf>s;yc/ioJogicaI approach to understanding intergroup attitudes in the United States and Israel. Unpublished doctoral dissertation, Department of Psychology, University of California, Los Angeles. Pratto, F. (1996). Sexual politics: The gender gap in the bedroom, the cupboard, and the cabinet. In D. M. Buss & N. Malamuth (Eds.), Sex, power and conflict: Evolutionary and feminist perspectives (pp. 179-230). New York: Oxford University Press. Pratto, F. (1999). The puzzle of continuing group inequality: Piecing together psychological, social, and cultural forces in social dominance theory. Advances in Experimental Social Psychology, 32, 191-263. Pratto, F., Sidanius, ]., Stallworth, L. M., & Malle, B. F. (1994)- Social dominance orientation: A personality variable predicting social and political attitudes. Journal of Personality and Social Psychology, 67, 741-763.

ATTITUDES

195

Pratto, F., Stallworth, L. M., & Sidanius, J. (1997). The gender gap: Differences in political attitudes and social dominance orientation. British Journal of Social Psychology, 36, 49-68. Pratto, F., Stallworth, L.M., Sidanius, J., & Siers, B. (1997). The gender gap in occupational role attainment: A social dominance approach. Journal of Personality and Social Psychology, 72, 37-53. Sidanius, J., & Pratto, F. (1999). SociaJ dominance. New York: Cambridge. Super, D. E., & Nevill, D. D. (1985). The values scale. Palo Alto, CA: Consulting Psychologists Press.

Scale Which of the following objects or statements do you have a positive or negative feeling towards? Beside each object or statement, place a number from " 1" to "7" which represents the degree of your positive or negative feeling. 7 6 5 4

= = = = 1. 2. 3. 45. 6. 7. 8.

9. 10. 11. 12. 13. 14. 15. 16.

extremely positive somewhat positive slightly positive neutral

We should strive to make incomes as equal as possible.* Group equality should be our ideal.* It's OK if some groups have more of a chance in life than others. To get ahead in life, it is sometimes necessary to step on other groups. We should do what we can to equalize conditions for different groups.* It's probably a good thing that certain groups are at the top and others are at the bottom. Inferior groups should stay in their place. We would have fewer problems if groups were treated more equally.* It would be good if groups could be equal.* In getting what you want, it is sometimes necessary to use force against other groups. All groups should be given an equal chance in life.* If certain groups stayed in their place, we would have fewer problems. We should strive for increased social equality.* Sometimes other groups must be kept in their place. Some groups of people are simply inferior to other groups. No one group should dominate in society.*

* reverse-scored 196

3 = slightly negative 2 = somewhat negative 1 = extremely negative

RESEARCH SCALES

Note. From "Social Dominance Orientation: A Personality Variable Predicting Social and Political Attitudes," by F. Pratto, J., Sidanius, L. M. Stallworth, and B. F. Malle, 1994, Journal of Personality and Social Psychology, 67, 760 (Appendix A). Copyright 1994 by the American Psychological Association. Reprinted with permission.

CIVIC ATTITUDES Source Mabry, J. B. (1998). Pedagogical variations in service-learning and student outcomes: How time, contact, and reflection matter. Michigan Journal of Community Service Learning, 5, 32-47. Construct The Civic Scales scale assesses civic attitudes as they relate to community service. Description The Civic Attitudes scale is composed of five items that assess attitudes toward the responsibility to help others and solve societal problems. Respondents are asked to rate the items on a 5-point scale ranging from strongly disagree to strongly agree. Estimated Time to Administer Five minutes. Reliability Temporal Consistency No information has been located. Internal Consistency Coefficient alphas for pre- and postcourse were .80 and .81, respectively. Factor Structure Factor analysis with varimax rotation supported the finding of a single factor (Mabry, 1998). ATTITUDES

197

Validity Known Groups Male respondents showed significant increases in civic attitudes from pre- to postservice learning experience. Although there were no significant changes for female respondents, the findings demonstrated a positive change trend (Mabry, 1998). Convergent Validity

No information has been located. Discriminant Validity No information has been located. Gender Differences Female respondents scored significantly higher on the precourse Civic Attitudes scale than male respondents (Mabry, 1998). Comments Enhancing civic attitudes is one of the primary objectives of service learning courses (Barber & Battistoni, 1994; Boyte & Hollander, 1999; Ehrlich, 1997). Mabry's Civic Attitudes scale provides a short measure of civic attitudes. The scale, at this point, has limited psychometric evidence. In addition, the scale contains beliefs about others (e.g., "adults should") and self (e.g., "I feel"). Nonetheless, there is good evidence from Mabry's research that the scale is unidimensional. References Barber, B., & Battistoni, R. (1994). A season of service. PS: Political Science and Politics, 26, 235-262. Boyte, H., & Hollander, E. (1999). Wingspread declaration on the civic responsibility of research universities. Providence, RI: Campus Compact. Ehrlich, T. (1997, Summer-Fall). Civic learning: Democracy and education revisited. Educational Record, 57-65. Mabry, J. B. (1998). Pedagogical variations in service-learning and student outcomes: How time, contact, and reflection matter. Michigan Journal of Community Service Learning, 5, 32-47.

198

RESEARCH SCALES

Scale To what extent do you agree or disagree with the following statements: 1

2

Strongly disagree

3 Disagree somewhat

Neither agree nor disagree

4 Agree somewhat

5 Strongly agree

1. Adults should give some time for the good of their community or country.b 2. People, regardless of whether they've been successful or not, ought to help others.b 3. Individuals have a responsibility to help solve our social problems.0 4. I feel that I can make a difference in the world.b 5. It is important to help others even if you don't get paid for it.c Items in the above scale come from the following sources as indicated: Markus, G., Howard, ]., & King, D. (1993). Integrating community service and classroom instruction enhances learning: Results from an experiment. Educational Evaluation and Policy Analysis, 15, 410-419. c Myers-Lipton, S. (1994). The effects of service-learning on college students' attitudes toward civic responsibility, international understanding, and racial prejudice. Unpublished doctoral dissertation, University of Colorado, Boulder, CO. b

Note. From "Pedagogical Variations in Service-Learning and Student Outcomes: How Time, Contact, and Reflection Matter," by J. B. Mabry, 1998, Michigan Journal of Community Service Learning, 5, p. 46 (Appendix 1). Copyright 1998 by OCLS Press, University of Michigan. Reprinted with permission.

ATTITUDES

199

9 CRITICAL THINKING

WATSON-GLASER CRITICAL THINKING APPRAISAL Source Watson, G. B., & Glaser, E. M. (1980). Watson-Glaser Critical Thinking Appraisal. San Antonio, TX: Psychological Corporation. A catalog that contains the Watson-Glaser Critical Thinking Appraisal (WGCTA) is available from: Harcourt Brace Educational Measurement 1-800-211-8378 (phone) 1-800-232-1223 (fax) [email protected] The manual contains information regarding preparation, administration, and scoring of the test as well as norms, reliability, and validity information. The manual is available for $32.00 Construct The WGCTA scale assesses five aspects of critical thinking ability. A single score indicating critical thinking ability is obtained from the five components. 201

Description The WGCTA has two forms: Form A and Form B. There is evidence that the two forms are equivalent (Watson & Glaser, 1980, pp. 7-9). Each form consists of 80 items that address problems, statements, arguments, and interpretations similar to those found in daily life (Watson & Glaser, 1980, p. 2). The test is not a timed assessment but a power test. The appraisal consists of the following five subscales: • Inference: "discriminating among degrees of truth or falsity of inferences drawn from given data" (Watson & Glaser, 1980, p-2); • Recognition of Assumptions: identifying unstated assumptions or presuppositions in a statement; • Deduction: concluding whether certain conclusions "necessarily follow from information in a given statement" (Watson & Glaser, 1980, p. 2); • Interpretation: examining evidence and deciding if conclusions based on data are justified; and • Evaluation of Arguments: distinguishing between strong and relevant and weak or irrelevant arguments (Watson & Glaser, 1980). The scale requires responses on items with two different types of content: Items having "neutral" content deal with the weather, scientific facts or experiments, and other subject matter about which people generally do not have strong feelings or prejudices. Items having "controversial" content, although approximately parallel in logical structure to neutral items, refer to political, economic, and social issues that frequently provoke very strong feelings. As has often been shown, strong attitudes, opinions, and biases affect the ability of some people to think critically. (Watson & Glaser, 1980, p. 2)

Estimated Time to Administer Forty to fifty minutes. Reliability Temporal Consistency Watson and Glaser (1980) reported a 3-month test-retest reliability for the overall measure of .73. 202

RESEARCH SCALES

Internal Consistency The WGCTA measured as a single score produced split-half reliabilities with Spearman-Brown correction ranging from .69 to .85 (Watson & Glaser, 1980). Factor Structure Westbrook and Sellers's (1967) results did not support a five-factor structure. The results of a centroid factor analysis resulted in one general factor. However, when varimax rotation was conducted the items loaded on two factors. Follman, Miller, and Hernandez (1969) also found that the subscales loaded highly on a single factor. Validity Known Groups Watson and Glaser (1980) reported studies by Sorenson (1966) and Agne and Blick (1972) that found laboratory-centered and data-centered instruction were associated with larger change scores on the WGCTA than were courses using the traditional lecture format. Pascarella (1989) reported significantly higher scores on the overall WGCTA score, the Interpretation subscale, and the Evaluation of Arguments subscale for individuals who attended a first year of college compared with a matched set of individuals who did not go to college. Convergent Validity Watson and Glaser (1980) summarized studies that demonstrated significant correlations between the WGCTA and various measures of intelligence and scholastic aptitude. Ennis, Millman, and Tomko (1985) reported correlations ranging between .41 and .48 between the WGCTA subscales and the Cornell Critical Thinking Test, Level X. For Level Z of the Cornell Critical Thinking Test and the WGCTA, a correlation of .48 was found using an undergraduate student sample and .79 utilizing a student sample comprised of undergraduate and graduate students. Discriminant Validity Scores on the WGCTA were not significantly correlated to high school grade point averages (Watson & Glaser, 1980). Gender Differences Converting raw scores to normed percentile scores does not have different norms for male and female students. Pascarella (1989) found no gender differences in scores on the WGCTA.

CRITICAL THINKING

203

Comments Service learning experiences, at their best, promote the intellectual development of students (Eyler, 2000). Some research has used self-report measures of critical thinking (e.g., Astin & Sax, 1998). Other research has focused on critical thinking skills as they apply to social problems (e.g., Batchelder & Root, 1994; Eyler & Giles, 1999). The WGCTA provides a means for measuring critical thinking as a general attribute. Pascarella (1989) identified the WGCTA as "the most commonly used measure of critical thinking with post-secondary student samples" (p. 21). It is advantageous for longitudinal research that there are two equivalent forms of the WGCTA. References Agne, R., & Blick, D. A. (1972). A comparison of earth science classes taught by using original data in a research-approach technique versus classes taught by conventional approaches not using such data. Journal of Research in Science Teaching, 9, 83-89. Astin, A. W., & Sax, L. J. (1998). How undergraduates are affected by service participation. Journal of College Student Development, 39, 251-263. Batchelder, T. H., & Root, S. (1994). Effects of an undergraduate program to integrate academic learning and service: Cognitive, prosocial cognitive and identity outcomes, journal of Adolescence, 17, 341-356. Eyler, J. S. (2000, Fall). What do we most need to know about the impact of service-learning on student learning? Michigan Journal of Community Service Learning, 11-17. Eyler, J., & Giles, D. E., Jr. (1999). Where is the learning in service-learning! San Francisco: Jossey-Bass. Ennis, R., Millman, J., & Tomko, T. (1985). Manual, Corned Critical Thinking Test. Pacific Grove, CA: Midwest. Follman, ]., Miller, W., & Hernandez, D. (1969). Factor analysis of achievement, scholastic aptitude, and critical thinking ability. Journal of Experimental Education, 38, 48-53. Pascarella, E. T. (1989). The development of critical thinking: Does college make a difference? Journal of College Student Development, 30, 19-26. Sorenson, L. (1966). Watson-Giaser Critical Thinking Appraisal: Changes in critical thinking associated with two methods of teaching high school biology (Test Data Report No. 51). New York: Harcourt Brace &. World. Watson, G. B, & Glaser, E. M. (1980). Watson-Gloser Critical Thinking Appraisal, manual: Forms A and B. San Antonio, TX: Psychological Corporation. Westbrook, B. W., & Sellers, J. R. (1967). Critical thinking, intelligence, and vocabulary. Educational and Psychological Measurement, 27, 443-446.

204

RESEARCH SCALES

SCALE OF INTELLECTUAL DEVELOPMENT Source Erwin, T. D. (1981). The Scale of Intellectual Development Manual. Harrisonburg, VA: Developmental Analytics. The manual containing general information about the Scale of Intellectual Development (SID), including reliability and validity information for Versions I through IV, is available from: Developmental Analytics P.O. Box 855 Harrisonburg, VA 22801 The manual is available for $5.00, and a copy of the scale may be obtained for $0.50. Construct The SID was developed to measure Perry's (1970) scheme of intellectual development. Description Erwin (1981) developed the original 119-item scale from Roberts's (1977) Scale of Ethical and Intellectual Development. A 4-point format was used with items written in a positive and negative direction, offsetting response bias. Four factors were identified on the basis of factor analysis: • Dualism: issues are seen clear-cut and authorities are consulted for answers. • Relativism: recognize alternative views but moral positions are influenced by other people and situations. • Commitment: tolerance of other views but have own set of morals and beliefs. • Empathy: sensitivity toward others and feel responsible to help improve society. The first three factors are equivalent to those of Perry (1970); however, Empathy differs in that it is interpreted directly from the cluster of items obtained in the factor analysis. The SID has been revised several times. The most current version, the SID-IV, consists of 115 items including a fifth scale, "Faking," designed to determine social desirability effects in responses. Ratings for all versions are on a 4-point scale ranging from strong^ agree to strongly disagree. CRITICAL THINKING

205

Estimated Time to Administer Twenty to thirty minutes. Reliability Temporal Consistency No information has been located. Internal Consistency

Coefficient alphas were .81 for Dualism, .70 for Relativism, .76 for Commitment, and .73 for Empathy for the SID-I (Erwin, 1983). Factor Structure Erwin (1983) conducted a factor analysis of the SID-I that produced four factors: Dualism, Relativism, Commitment, and Empathy. Validity Known Groups

Erwin (1983) reported students who had greater participation and responsibility in community groups and clubs obtained higher scores on Commitment than those with lesser participation and responsibility. Convergent Validity

The four factors of the SID-I were correlated with two other measures of student development: the Perceived Self Questionnaire (Heath, 1968) and the Erwin Identity Scale (Erwin & Delworth, 1980). The SID-IV correlated with the Batson and Ventis Religious Life Inventory (Batson & Ventis, 1982). Erwin (1981) reported a moderate negative relationship between the Relativism subscale and an end orientation toward religion. There was also a moderate positive relationship between the Dualism subscale and a mean orientation toward religion (Erwin, 1981). The SID-IV was also correlated the Action Control Scale (Kuhl, 1982), suggesting that action-oriented people are more committed whereas state-oriented people are more dualistic in their thinking (Erwin & Marcus-Mendoza, 1988). Discriminant Validity

No information has been located. Gender Differences No information has been located. 206

RESEARCH SCALES

Comments The SID has undergone several revisions to increase its reliability and validity. The SID has been shown to be related to several measures of development and provides a good measure of Perry's (1970) scheme of intellectual development, which can be viewed as a precursor for developing critical thinking. References Batson, C. D., & Ventts, W. L. (1982). The religious experience. New York: Oxford University Press. Erwin, T. D. (1981). The Scale of Intellectual Development manual. Harrisonburg, VA: Developmental Analytics. Erwin, T. D. (1983). The Scale of Intellectual Development: Measuring Perry's scheme. Journal of College Student Development, 24, 6-12. Erwin T. D., & Delworth, U. (1980). An instrument to measure Chickering's vector of identity. National Association of Student Personnel Administrators Journal, 17, 19-24. Erwin, T. D., & Marcus-Mendoza, S. (1988). Motivation and student's participation in leadership and group activities. Journal of College Student Development, 29, 356-361. Heath, D. (1968). Growing up in college: Liberty and maturity. San Francisco: Jossey-Bass. Kuhl, J. (1982). Action vs. state-orientation as a mediation between motivation and action. In W. Hacker, W. Volpert, & M. von Cranach (Eds.), Cognitive and motivational aspects of action (pp. 67-85). Amsterdam: North-Holland. Perry, W. G., Jr. (1970). Forms of intellectual and ethical development in the college years. New York: Holt, Rinehart, & Winston. Roberts, M. J. (1977). Construction and validation of the scales of ethical and intellectual development. Unpublished manuscript.

CALIFORNIA CRITICAL THINKING SKILLS TEST Source Facione, P. A., Facione, N. C., Blohm, S. W., Howard, K., & Giancarlo, C. A. F. (1998). Manual, the California Critical Thinking Skuls Test, Form A and Form B revised edition. Millbrae, CA: California Academic Press. The specimen kit containing one copy of Form A, Form B, and Form 2000, a sample answer sheet, a Spanish version of Form A, and the test manual may be obtained from:

CRITICAL THINKING

207

California Academic Press 217 La Cruz Ave. Millbrae, CA 94030 1-650-697-5628 (phone and fax) www.calpress.com The specimen kit is available for $60.00. Construct The California Critical Thinking Skills Test (CCTST) instrument is designed to measure five cognitive skill dimensions of critical thinking associated with core skills needed in college education. Description The CCTST consists of 34 multiple-choice items and contains the following five subscales: • Analysis: to comprehend and express meaning or significance of a variety of issues and to identify intended and actual inferential relationships; • Evaluation: to assess claims and arguments and to state results, justify procedures, and present arguments; • Inference: to obtain needed information to draw conclusions and speculate hypotheses; • Deductive Reasoning: to determine if the assumed truth of the premises necessitates the truth of the conclusion; and • Inductive Reasoning: to determine if the conclusion is warranted but not necessitated by the assumed truth of the premises. Forms A and B of the CCTST are conceptually equivalent (Facione et al., 1998). Estimated Time to Administer Forty-five minutes. Reliability Temporal Consistency

No information has been located.

208

RESEARCH SCALES

Internal Consistency Kuder-Richardson (K-R 20) values ranging from .68 to .70 were found by Facione (1990). Facione et al. (1998) reported an alternative form reliability of .78. Factor Structure

Jacobs (1995) conducted a principal-components analysis that did not support the factor structure. Validity Known Groups Facione et al. (1998) reported an experiment using pre-post measures conducted on students finishing a critical thinking course and a control group of similar students not enrolled in the course. They found significantly higher postscores on the CCTST for the students enrolled in the course. Convergent Validity

Scores on the CCTST were correlated (.58 to .72) with scores on each subscale of the ORE. Facione et al. (1998) also found CCTST scores were correlated (.42 to .55) with SAT scores and correlated (.41 to .54) with the WGCTA (Watson & Glaser, 1980). Discriminant Validity

Scores on the CCTST were unrelated to age, ethnicity, and academic major (Facione et al., 1998). Gender Differences No significant gender differences have been found for scores on the CCTST (Facione et al., 1998). Comments The ability of service learning to become institutionalized in higher education will, in part, depend on its capacity to improve the cognitive skills of students (Eyler, 2000). The CCTST provides a measure of core educational skills for college students. Although the scale is a general measure, rather than a context-specific measure of community problem solving, it provides a direction of inquiry that research on service learning needs to approach (Eyler, 2000).

CRITICAL THINKING

209

Jacobs (1995) reported that Forms A and B are not statistically equivalent, and that Form B is slightly more difficult (0.5 point) than Form A, resulting in lower scores on Form B. Furthermore, a principal-components analysis conducted by Jacobs (1995) did not support the item classifications. However, this statistical analysis may be inappropriate for this construct, because the factors are theorized to be interactive in actual judgment formation. Researchers should be aware of these potential limitations as they use the scale. Form 2000 was not examined in this research. References Eyler, J. (2000, Fall). What do we most need to know about the impact of servicelearning on student learning? Michigan Journal of Community Service Learning, 11-17. Facione, P. A. (1990). The California Critical Thinking Ski/is Test—college level: Technical report #1. Experimental validation and content validity. Millbrae, CA: California Academic Press. Facione, P. A., Facione, N. C., Blohm, S. W., Howard, K., &. Giancarlo, C. A. F. (1998). Manual, the California Critical Thinking Skills Test, Form A and Form B revised edition. Millbrae, CA: California Academic Press. Jacobs, S. (1995). Technical characteristics and some correlates of the California Critical Thinking Skills Test, Forms A and B. Research in Higher Education, 36, 89-108. Watson, G. B., & Glaser, E. M. (1980). Watson-Glaser Critical Thinking Appraisal. San Antonio, TX: Psychological Corporation.

CORNELL CRITICAL THINKING TEST Source Ennis, R., Millman, J., & Tomko, T. (1985). Manual, Cornell Critical Thinking Test. Pacific Grove, CA: Midwest. A catalog and order form can be obtained from: Critical Thinking Books and Software P.O. Box 448 Pacific Grove, CA 93950 1-800-458-4849 1-831-393-3277 (fax) www.criticalthinking.com The test manual containing administration and scoring procedures as well as psychometric properties of the test is available for $8.99. A specimen set

2 J0

RESEARCH SCALES

(consisting of Cornell Level Z, Cornell Level X, the manual, and an answer sheet) is available for $21.99. Construct The Cornell Critical Thinking Test (CCTT) Level Z measures the following aspects of critical thinking: induction, deduction, observation, credibility, assumptions, and meaning. Description Level Z consists of 52 test items aimed at advanced and gifted high school students, college students, and adults. Respondents are presented with statements and must choose an appropriate response from among three alternatives. Scoring instructions are available in the test manual. Estimated Time to Administer Fifty minutes. Reliability Temporal Consistency No information has been located. Internal Consistency Split-half reliability for Level Z ranged from .55 to .75. KuderRichardson reliability estimates for Level Z ranged from .50 to .77. Factor Structure For Level Z, the test manual reports that Follman, Brown, and Burg (1970) concluded that critical thinking ability is not a function of general ability but a combination of specific factors. Ennis et al. (1985) concluded that the results are inconclusive but support the belief that critical thinking ability is heterogeneous. Validity Known Groups Groups that were judged to have varying amounts of critical thinking ability on the basis of varying amounts of critical thinking instruction believed to be effective differed significantly (e.g., Allegrette & Frederick, CRITICAL THINKING

211

1995). Groups at higher levels of education were found to score significantly higher than groups with lower levels of education (e.g., graduates vs. undergraduates; e.g., Frisby; 1992; Mines, King, Hood, & Wood, 1990). Convergent Validity

The test manual describes several relationships between Level Z of the CCTT and other variables. For Level Z, an undergraduate student sample correlated .48 with the WGCTA (Watson & Glaser, 1980), whereas a sample of undergraduate and graduate students correlated .79 with the WGCTA (Watson & Glaser, 1980). Discriminant Validity

Level Z has relationships that fit expectations from underlying critical thinking theory combined with educational and sociological theory. For example, there are negative correlations with dogmatism, materialistic economic values, and teaching attitudes and low positive correlations with socioeconomic status, internal locus of control, and independence (FlorishaTheil, 1983; Phelps, 1987). Furthermore, there are high positive correlations with tests that contain items measuring critical thinking (e.g., ACT, GRE) but lower positive correlations with other tests that do not (e.g., SAT, LSAT; Florisha-Theil, 1983; Frisby, 1992; Mines et al., 1990). Gender Differences No gender differences have been found for Level Z (Ennis et al., 1985). Comments As with other measures of critical thinking, this scale provides a general measure for tracking a central aspect of cognitive development for students. Ennis et al. (1985) noted that Level Z has a higher level of difficulty than Level X in the language used within the scale. Therefore, Level Z would not be appropriate for less sophisticated high school students. Thus, the test content should be reviewed to determine its appropriateness for the intended audience. References Allegrette, C. L., & Frederick, J. N. (1995). A model for thinking critically about ethical issues. Teaching of Psychology, 22, 46-48. Ennis, R., Millman, J., &. Tomko, T. (1985). Manual, Cornell Critical Thinking Test. Pacific Grove, CA: Midwest.

212

RESEARCH SCALES

Follman, J., Brown, L., &. Burg, E. (1970). Factor analysis of critical thinking, logical reasoning, and English subtests. Journal of Experiential Education, 38, 11-16. Florisha-Theil, A. (1983). The relationship of selected personality and cognitive factors to critical thinking among graduate students in educational psychology. Unpublished doctoral dissertation, University of Southern California. Frisby, C. L. (1992). Construct validity and psychometric properties of the Cornell Critical Thinking Test (Level Z): A contrasted groups analysis. Psychological Reports, 71, 291-303. Mines, R. R., King, P. M., Hood, A. B., & Wood, P. K. (1990). Stages of intellectual development and associated critical thinking skills in college students. Journal o/College Student Development, 31, 538-547. Phelps, P. H. (1987). The effects of participation in reflective thinking on preservice teachers' critical thinking. Unpublished doctoral dissertation, Michigan State University. Watson, G. B., & Glaser, E. M. (1980). Watson-Glaser Critical Thinking Appraisal. San Antonio, TX: Psychological Corporation.

CRITICAL THINKING

213

GLOSSARY Alternative form reliability: The correlation between two separate administrations to the same individuals using two equivalent measurement procedures (e.g., similar sets of items) that measure the same construct. Also referred to as parallel forms reliability. Coefficient alpha: An estimate of reliability based on inter-item correlations; an estimate of the correlation between the scale and a hypothetical scale of the same length; an estimate of the correlation between the scale score and true score; the average of all possible split-half reliabilities; a measure of the homogeneity of the items. Also referred to as Cronbach's coefficient alpha. Concurrent validity: The correlation between a measurement procedure and a standard that is available at the same time; a type of criterion-based validity. Construct: An abstract or hypothetical property or attribute that organizes, labels, and otherwise makes sense out of a diverse set of phenomena. Also called concept. Construct validity: The degree to which a particular measurement of a construct confirms the expectations contained in the theory, including the nature of the construct, its manifestations, and its relationships and lack of relationships to other constructs. Content validity: The degree to which a content domain of a construct (e.g., civic skills) has been clearly defined and sampled by a measurement procedure (e.g., a civic skills scale). Convergent validity: Empirical evidence that scores from a measurement procedure are correlated with scores from measures of the same or a related construct using different measurement procedures. Cronbach's coefficient alpha: See Coefficient alpha. Deduction: Drawing specific inferences from general principles; the process of going from general, abstract constructs to specific, concrete manifestations of the construct. Discriminant validity: Empirical evidence that scores from a measurement procedure are uncorrelated with scores from measures of another construct that is posited to be unrelated and that uses the same measurement procedure (heterotrait, monomethod) or a different measurement procedure (heterotrait, heteromethod). External validity: The degree to which inductive inferences can be made about the generalizability (e.g., to other operationalizations, settings, conditions, times, groups) of research results. Face validity: The degree to which a measurement procedure appears to be an appropriate and accurate measure.

215

Factor analysis: A statistical procedure that evaluates the degree to which a set of data (e.g., items on a scale) is unidimensional or multidimensional (i.e., multiple subsets of items that are internally consistent but uncorrelated). Factor loading: The correlation between an item and a factor. Induction: Drawing general principles from specific inferences; the process of going from specific, concrete manifestations of the construct to broad conclusions or theoretical propositions. Kuder-Richardson 20 (KR-20): An estimate of reliability based on inter-item correlations of items with dichotomous response formats (e.g., correct/ incorrect; true/false; like me/not like me); KR-20 is a special case of coefficient alpha, which can be used for items with continuous or dichotomous response formats. Measurement: Rules used to assign labels to qualitative variations or numbers to quantitative variations in a variable. Mediating variable: An intervening variable that explains in part or in whole the relationship between two variables. Moderator variable: When the effect of one variable on an outcome variable depends on the level of a third variable, the third variable is referred to as a moderator variable. Also referred to as an interaction effect. Oblique factor structure: Used to identify correlated, as opposed to independent, factors. Operationalization: The specific way in which a variable is measured or manipulated. Parallel forms reliability: See Alternative forms reliability. Predictive validity: The correlation between a measurement procedure and a standard (e.g., behavior, performance, cognitive outcome) that is available later in time; a type of criterion-based validity. Qualitative variable: A variable for which numbers or labels are assigned to represent differences in kind or categories. Quantitative variable: A variable for which numbers are assigned to represent differences in intensity or amount. Questionnaire: A paper-and-pencil assessment instrument that is composed of single items and/or one or more multiple-item scales. Reciprocity: A characteristic of service learning relationships between the community and the campus in which both invest and benefit, and both serve as teacher and learner. Reflection activities: Activities (e.g., journals, small group discussions, directed writing) in a service learning class that (a) intentionally link the service experience to course-based learning objectives, (b) are structured, (c) occur regularly, (d) allow feedback and assessment, and (e) include the clarification of values (Bringle & Hatcher, 1999; Hatcher & Bringle, 1997).

216

GLOSSARY

Reliability: The degree to which a measurement procedure produces scores that are "free from errors of measurement" (American Psychological Association, 1985, p. 9); reliability coefficients range from .00 to 1.00. Scale: A multiple-indicator measurement procedure. Service learning: A "course-based, credit-bearing educational experience in which students (a) participate in an organized service activity that meets identified community needs and (b) reflect on the service activity in such a way as to gain further understanding of course content, a broader appreciation of the discipline, and an enhanced sense of civic responsibility" (Bringle & Hatcher, 1995, p. 112). Social desirability response bias: The tendency for respondents to portray themselves in a manner that is socially acceptable, likeable, and well adjusted; this type of systematic response bias detracts from the validity of the measurement procedure. Spearman-Brown prophecy formula: A method for estimating how the reliability of a measure (e.g., scale) is related to length. Specificity: The degree to which a particular indicator (e.g., item on a scale) measures qualities that are unique to that item, rather than or in addition to the construct of interest. Split-half reliability: The correlation between scores from two nonoverlapping subsets of items from a scale. Test—retest reliability: A correlation between two sets of scores from the same measurement procedure administered on two separate occasions to the same group of individuals. Test-retest reliability presumes that the attribute being measured is constant across the time span and is inappropriate if that assumption is tenuous. Theory: A set of interrelated propositions about the nature of constructs, their manifestations, and the relationships between constructs. Unidimensionality: When a construct is assumed to be a single, coherent attribute; measurement of one coherent construct. Validity: The degree to which a measurement procedure accurately measures the construct it is supposed to measure. Variables: Concrete manifestations of a construct that can vary in intensity or kind. Varimax rotation: Factor-analytic procedure used to identify orthogonal or uncorrelated factors.

GLOSSARY

217

INDEX Academic performance, and scales, 27 Academic Self-Concept Scale, and SelfEsteem Scale, 98 Achievement Anxiety scale, and SAD scale, 140 Achievement scales, 27 Action Control Scale, and SID-IV, 206 ACT (American College Testing) scores and CCTT, 212 and Selfism scale, 136 Affiliation, and CSIPI, 174 Agency, in Hope Scale, 127 AIDS Caregiver Scale, 165-169 Alienation Scale, Dean (DAS), 130-134 Alternative form reliability, 18, 215 Analysis, in CCTST, 208 Anti-Black Scale, and UOS, 190 Assertiveness, in Goal and Mode Values Inventories, 54 Assimilation, and CSIPI, 174 Assumptions, in WGCTA, 202 Audience Sensitivity Index, and SAD scale, 140 Autonomy, in SDTLA, 144-145, 150-152 Avoidance, and PS1, 156 and SAD scale, 139, 141

Campbell, JoAnn, x Campus-Based Workshop Curriculum Guide (Foos and Hatcher), xi Campus Compact Research Advisory Council, 26 Career choices, motivation for (PSMS), 45 Career Decision Scale, and CDSES, 161-162 Career Decision Self-Efficacy Scale (CDSES), 160-164 Career Development Inventory, and SDTLA, 146 Career planning, in SDTLA, 149 Caregiver Scale, AIDS, 165-169 Careless responding, and SDTLI, 145 CCTST (California Critical Thinking Skills Test), 207-210 CCTT (Cornell Critical Thinking Test), 210-213 and WGCTA, 203 CDSES (Career Decision Self-Efficacy Scale), 160-164 Civic Action scale, 169-172 Civic Attitudes scale, 197-199 Civic Attitudes and Skills Questionnaire (CASQ), 169-170 Civic Responsibility Scale, and Civic Action scale, 169 Coefficient alpha, 18, 215 Cronbach's, 19 Cognitive apprenticeship, in learning process, 4 Collaboration, in learning process, 4 Collaborative learning, and service learning, 7 College Student Experiences questionnaire, and SDTLA, 146 College Student Questionnaire, and SDTLA, 147 Colleges and Universities as Citizens (Bringle, Games and Malloy), x Colorado State University service learning listserv, 26 Commitment to public interest, and PSMS, 44

Balanced Emotional Empathy scale, and EETS, 111 Beck Depression Inventory, and Hope Scale, 128 Blood donation sample story, 80 Boundary conditions, and moderator variables, 27 Boyer, Ernest L, ix-x Bringle, Robert, x

California Critical Thinking Skills Test (CCTST), 207-210 California Personality Inventory, and SDO scale, 194 Cambridge, Barbara, xiv

219

Communal perspective, and UOS, 191 Community impact, and scales, 27 Community service, CSAS definition of, 180 Community Service Attitudes Scale (CSAS), 177-182 Community Service Involvement Preference Inventory (CSIPI), 173177 Community Service Self-Efficacy Scale (CSSES), 100-103 Compassion, and PSMS, 44, 45 Competence, in Goal and Mode Values Inventories, 53-54 Concurrent validity, 21, 215 Confidence, and PSI, 156 Confidence Subscale of Erwin Identity Scale, 107-110 Consistency internal, 18, 21 temporal, 18, 21 Constructs, 8, 13-14, 15, 215 determining appropriateness of, 27-

28,30 and scales, 28 and theories, 14, 20-21 Construct validity, 8, 20, 215 factors contributing to, 21-22 and scale construction, 26 Content validity, 21, 215 Control group, 22 Controversial content, 202 Convergent validity, 21, 215 Cornell Critical Thinking Test (CCTT), 210-213 and WGCTA, 203 Critical thinking and California Critical Thinking Skills Test, 207-210 as construct, 13-14 as heterogeous, 211 Watson-Glaser Appraisal of, 201-204 Cronbach's coefficient alpha, 19 CSAS (Community Service Attitudes Scale) 177-182 CSIPI (Community Service Involvement Preference Inventory), 173-177 CSSES (Community Service SelfEfficacy Scale), 100-103

220

INDEX

Dean Alienation Scale (DAS), 130-134 Deduction, 13-14, 215 in CCTST, 208 in WGCTA, 202 Defining Issues Test (DIT), 63-67 and EPQ, 86 and MAS, 83 and MMO, 74, 75 Developmental tasks, 144 Dewey, John, 3-4 Discriminant validity, 21, 215 DIT. See Defining Issues Test Diversity education, and service learning, 7 Dogmatism Scale, and UOS, 190

Education, higher and democratic participation (undergraduate), 3 and research on service learning, ix, 8-9 and Universities as Citizens project, x Educators, information collecting by, 11 Edwards Personal Preference Schedule, and SIV, 57 EETS (Emotional Empathic Tendency Scale), 110-114 Efficacy. See Self-efficacy Egocentrism, and Selfism scale, 134 Emotional Empathic Tendency Scale (EETS), 110-114 Empathy, 90 and EETS, 111, 112 helping elicited by, 112, 116 andlRI, 114, 115, 116 and service learning, 112-113 EPQ (Ethics Position Questionnaire), 85-88 Erwin Identity Scale Confidence Subscale of, 107-110 and S1D-I, 206 Establishing Universities as Citizens: Towards the Scholarship of Engagement (Rothman), x Ethics Position Questionnaire (EPQ),

85-88 Ethnic groups, and SDO scale, 194 Evaluation of arguments in CCTST, 208 in WGCTA, 202

Experimental manipulation, 22 Experimentation, and CSIPI, 174 Exploration, and CSIPI, 173-174 External validity, 8, 16, 215

Face-to-face interviews, 17 Face validity, 21, 215 Factor analysis, 21, 216 Factor loadings, 22, 216 Factor structure, 21-22 Faculty, and scales, 27 Faking of results for DIT, 65 and SDTLI, 145 and SID-IV, 205 Family Independence Scale, and SDTLA, 147 Fantasy, and IRI, 114, 115, 116 Feedback, in learning process, 4 Foos, Cathy Ludlum, x Francis Scale of Attitudes Towards Christianity, and GBJWS, 184 Functional approach, to volunteering motives, 38

Games, Richard, x, xiii GBJWS (Global Belief in a Just World Scale), 183-186 General Aptitude Test Battery, and SIV, 57 Generalizability, 14, 16 Generalized Expectancy for Success Scale (GESS) and Hope Scale, 128 and Selfism scale, 136 Georgia Autonomy Scales, and SDTLA, 146-147 Getting ahead, in Goal and Mode Values Inventories, 54 Global Belief in a Just World Scale (GBJWS), 183-186 Goal and Mode Values Inventories, 47-55 GRE scores and CCTST, 209 and CCTT, 212 Guilford-Zimmerman Temperament Survey, and LOT-R, 187-188

Hatcher, Julie, x, xiii Hopelessness Scale, and Hope Scale, 128 Hope Scale, 126-130 Humanitarian-Egalitarian Scale, and SDO scale, 194

Idealism, in EPQ, 85, 86 Identity Chickering's conceptualization of, 108 development of, 109 Identity Achievement Scale, and Confidence Subscale of Erwin Identity Scale, 109 Immigrant college students, and service learning or work study, 131 Indiana Campus Compact, ix, xi, xiii Campus Compact Research Advisory Council, 26 Indiana University—Purdue University Indianapolis (IUPUI) Center for Service and Learning, xiii Induction, 14, 16, 216 in CCTST, 208 Inference in CCTST, 208 in WGCTA, 202 Information collecting, by educators, 11 Inquiry, 12-13 Inquiry-based learning, Dewey's conditions for, 3-4 Institutional attributes, and scales, 27 Instruction methods, and WGCTA scores, 203 Instructor, in service learning, 7 Intellectual Development, Scale of (SID), 205-207 Intergroup relations, and SDO scale, 193-197 Internal consistency, 18, 21 Internal-External Control of Reinforcement Scale and SAD scale, 140 and Selfism scale, 136 and SES, 104 Internal-External scale, and Confidence Subscale of Erwin Identity Scale, 109 Internal validity, 8, 216

INDEX

221

International Conferences on Research on Service Learning, xi International harmony and equality, in Goal and Mode Values Inventories, 51, 55 Internships, 6 Interpersonal Reactivity Index (IRI), 114-119 andEETS, 111 and SDO scale, 194 and VMS, 90 Interpersonal Reactivity Questionnaire, and PROM, 78-79 Interpersonal relationships in Goal and Mode Values Inventories, 52 in SDTLA, 145, 152 Interpersonal skills, and service learning, 120-121 Interpretation, in WGCTA, 202 Interviewers, paper-and-pencil scales as eliminating, 17 Interviews, 16—17 Ireland, Northern, GBJWS scores in, 184 IRI. See Interpersonal Reactivity Index

Jackson Personality Inventory, and SIX) scale, 194 Jackson Personality Research Form and SAD scale, 140 and SDO scale, 194 Junior Scholastic Aptitude Test and S1V, 57 Just world hypothesis, 184-185. See also Global Belief in a Just World Scale; Multidimensional Belief in a Just World Scale

Keyed responses, in SDTLI, 145 Known groups comparisons, 22 Kohlberg's theory and Defining Issues Test, 66, 74 and PROM-R, 79 and SROM, 67, 68, 69, 70-72 See also Moral dilemmas Kuder-Richardson (K-R) formulas, 19 Kuder-Richardson 20 (KR-20), 19, 216

222

INDEX

Lankford, Kendall, x Learning, elements to enhance depth of, 4 Learning for Self-Understanding Scale, 153-155 Leary Interpersonal Checklist, and SIV, 57 Lecture method, and WGCTA scores, 203 Life Orientation Test (LOT), and Hope Scale, 128 Life Orientation Test Revised (LOT-R), 186-189 Life Skills Development Inventory, and SDTLA, 146 Locus of Control Scale, and PSI, 157 LOT-R (Life Orientation Test Revised), 186-189 LSAT, and CCTT, 212

Manifest Anxiety scale, and SAD scale, 140 Marlowe-Crowne Social Desirability scale and AIDS Caregiver Scale, 167 andEETS, 112 and PSI, 158 and SAD scale, 140 and SDTLA, 147 and SES, 104 and UOS, 190 MAS-R (Revised Moral Authority Scale), 81-84, 90 Mastery Orientation scale, and Civic Action scale, 171 Measurement, 17-18, 23, 216 through interviews, 16-17 multiple forms of, 22-23 quantitative, 30 of service learning, 25 through standardized scales, 17-18 reliability in, 18-20 validity in, 20-22 See also Scales Measure of Moral Orientation (MMO),

72-77 Mediating variable, 27, 216 Michigan Journal of Community Service Learning, xi MMO (Measure of Moral Orientation), 72-77

Moderator variable, 27, 216 traitlike construct as, 28 Modern Racism Scale and Civic Action scale, 171 and UOS, 190 Moral Authority Scale, Revised (MAS-R), 81-84 and VMS, 90 Moral conflicts, and VMS, 90 Moral development, 66 Moral dilemmas in DIT, 64 in MMO, 72, 74, 76 in PROM-R, 77-81 in SROM, 67-68, 70-72 Moral Judgment Interview, and SROM, 68 Moral Maturity Index, of Kohlberg, 74 Motivation to Volunteer (MTV) scale, 40-44 Multidimensional Belief in a Just World Scale (MBJWS), 183, 184 Multi-Group Ethnic Identity Measure, and SDTLA, 147 Multi-item measures (scales), ix, xii, 8, 22, 23, 28, 30 Multiple indicators, 22-23 Myers-Briggs Type Indicator Feeling score from (and MMO), 74 and PSI, 158 My Vocational Situation Inventory, and CDSES, 162

Narcissistic Personality Inventory, and Selfism scale, 136 National strength and order, in Goal and Mode Values Inventories, 55 Neutral content, 202 Norms, in SROM, 67-68 Northern Ireland, GBJWS scores in, 184

Oblique factor structure, 21, 216 Occupational groups, and SDO scale, 194 Occupational Self-Efficacy Scale, and CDSES, 162 Operationalizations, 14, 15, 216 Optimism, 188 and LOT-R, 186-189

Orientation to learning, and college experiences, 154-155 Orientation to others, in Goal and Mode Values Inventories, 51, 53 Outcome expectations, vs. self-efficacy, 101

Paper-and-pencil scales, 17, 22 Parallel forms procedures, 18 Parents dilemma, 76 Pathways, and Hope Scale, 127 Payton Philanthropic Studies Library, Indiana University—Purdue University Indianapolis campus, 29 Pedagogy, service learning as, 5-7 Perceived Self Questionnaire, and SID-1, 206 Personal development, and service learning, 147 Personal distress, in IRI, 115, 116 Personal growth and inner harmony, in Goal and Mode Values Inventories, 52 Personal Social Values scale, 59-61 Perspective taking, and IRI, 114, 115, 116 Pessimism, and LOT-R, 186-189 Philanthropic habits, 3 Physical well-being, in Goal and Mode Values Inventories, 52 Placement quality, and interpersonal development, 121 Practica on experiential education continuum, 6 in goals or psychology education, 4 Practical application, in learning process, 4 Predictive validity, 21, 216 Prejudicial thinking, 195 Presidents' Declaration on the Civic Responsibility of Higher Education (Boyte and Hollander), xi Principal-components analysis, 21 Problem-based learning, and service learning, 7 Problem-Solving Inventory (PSI), 156-160

INDEX

223

PROM-R (Prosocial Reasoning Objective Measure—Adult Version, 77-81 Propriety in dress and manners, in Goal and Mode Values Inventories, 54 Prosocial Reasoning Objective Measure— Adult Version (PROM-R), 77-81 and adolescent PROM, 78 PSl (Problem-Solving Inventory), 156-160 PSMS (Public Service Motivation Scale), 44-47 Psychology education, common goals for, 4 Public policy making, and PSMS, 44 Public Service Motivation Scale (PSMS), 44-47 Purpose, in SDTLA, 144, 146, 149-150

Qualitative variables, 14, 216 Quality of supporting evidence, as scale criterion, 28-29 Quantitative measurement, 30 Quantitative variables, 14, 216 Questionnaires, 17, 216

Racial groups, and SDO scale, 194 Racism. See Modern Racism Scale Reciprocity, 216 Reflection activities, 5-6, 216-217 as mediating variable, 27 in moderator-variable example, 27 and students' self-confidence or development, 110 Reflection in service learning, and Personal Social Values scale, 60 Refugee college students, and service learning or work sudy, 131 Relativism, in EPQ, 85, 86 Relevance to service learning, as scale criterion, 26—28 Reliability 8, 18-20, 217 alternative form, 18, 215 split-half, 18, 217 test-retest, 18, 217 Religiosity, in Goal and Mode Values Inventories, 51, 52

224

INDEX

Religious Attitudes Scale, and Selfism scale, 136 Religious commitment, in Goal and Mode Values Inventories, 51, 54 Religious Life Inventory, and SID-1V, 206 Replication, of operationalizations, 14 Research, 12, 23 on constructs, 23 and deduction, 13-14 and induction, 14, 16 standardized scales in, 11 (see also Scales) and theories, 12-13, 15 (see also Theory) Research scales. See Scales Research on service learning, ix, xi, 7-8, 9, 175-176 Revised Moral Authority Scale (MAS-R), 81-84, 90 Risk of Eruptive Violence scale, and EETS, 111 Rokeach Values Survey, and Goal and Mode Values Inventories, 47, 48,50

SAD (Social Avoidance and Distress) scale, 139-142 Samples, and generalizability, 16 Satisfaction, in AIDS Caregiver Scale, 165, 167 SAT scores and CCTST, 209 and CCTT, 212 Scale of Ethical and Intellectual Development, 205 Scale of Intellectual Development (SID), 205-207 Scales, 17, 217 determining appropriateness of, 27-

28,30 as existing vs. modified vs. developed, 25-26 standardized, 11, 17-18, 22 reliability of, 18-20 validity of, 20-22 in this volume, xi-xii, 29-30 inclusion and exclusion criteria for, 26-29

Scales (as sections of chapters in this volume) AIDS Caregiver Scale, 165-169 California Critical Thinking Skills Test (CCTST), 207210 Career Decision Self-Efficacy Scale (CDSES), 160-164 Civic Action, 169-172 Civic Attitudes, 197-199 Community Service Attitudes Scale (CSAS), 177-182 Community Service Involvement Preference Inventory (CSIPI), 173-177 Community Service Self-Efficacy Scale (CSSES), 100-103 Confidence Subscale of Erwin Identity Scale, 107-110 Cornell Critical Thinking Test (CCTT), 210-213 and WGCTA, 203 Dean Alienation Scale (DAS), 130-134 Defining Issues Test (DIT), 63-67 and EPQ, 86and MAS, 83 and MMO, 74, 75 Emotional Empathic Tendency Scale (EETS), 110-114 Ethics Position Questionnaire (EPQ), 85-88 Global Belief in a Just World Scale (GBJWS), 183-186 Goal and Mode Values Inventories, 47-55 Hope Scale, 126-130 Interpersonal Reactivity Index (IRI), 114-119 and EETS, 111 and SDO scale, 194 and VMS, 90 Learning for Self-Understanding Scale, 153-155 Life Orientation Test Revised (LOT-R), 186-189 Measure of Moral Orientation (MMO), 72-77 Motivation to Volunteer (MTV), 40-44 Personal Social Values, 59-61

Problem-Solving Inventory (PSI), 156-160 Prosocial Reasoning Objective Measure—Adult Version (PROM-R), 77-81 and adolescent PROM, 78 Public Service Motivation Scale (PSMS), 44-47 Revised Moral Authority Scale (MAS-R), 81-84, 90 Scale of Intellectual Development (SID), 205-207 Self-Efficacy Scale (SES), 103-107 Self-Esteem Scale, 97-99 and Hope Scale, 128 and LOT-R, 187 and SES, 104 Selfism, 134-139 Social Avoidance and Distress, 139-142 Social Dominance Orientation (SDO), 193-197 and DOS, 191 Sociomoral Reflection Objective Measure (SROM), 67-72 Short Form of, 69 Student Developmental Task and Lifestyle Assessment (SDTLA), 143-153 Survey of Interpersonal Values (SIV), 55-59 Texas Social Behavior Inventory (TSBI)-Short Form, 119-126 Universal Orientation Scale (UOS), 189-192 Visions of Morality Scale (VMS), 88-96 and MAS-R, 82 Volunteer Functions Inventory (VFI), 35-40 and AIDS Caregiver Scale, 166-167 Watson-Glaser Critical Thinking Appraisal (WGCTA), 201-204 and CCTST, 209 and CCTT, 211-212 and SIV, 57-58 Scholarship of engagement, x Schwartz model of helping behavior, and CSAS, 177, 178, 179 Science, 30

INDEX

225

Scientific research. See Research SDO (Social Dominance Orientation) scale, 193-197 and UOS, 191 SDTLA (Student Developmental Task and Lifestyle Assessment), 143-153 SDTLI (Student Development Task and Lifestyle Inventory), 144 Self-efficacy, 101, 105 and CDSES, 160-164 and Hope Scale, 128 Self-Efficacy Scale (SES), 103-107 Self-esteem as construct, 28 andTSBI, 119 Self-Esteem Scale, 97-99 and Hope Scale, 128 and LOT-R, 187 and SES, 104 Selfism scale, 134-139 Self-Mastery Scale, and LOT-R, 187 Self-sacrifice, and PSMS, 44, 45 Self-understanding, and Learning for Self-Understanding Scale, 153-155 Sensation Seeking Scale, and Selfism scale, 136 Service learning, 5, 217 and career goals, 162 confidence from, 109-110 and empathy, 112-113 on experiential education continuum, 6 growth in, xi, 7-8, 9 for immigrant or refugee college students, 131 and Indiana Campus Compact, ix and intellectual development, 204, 209 and interpersonal skills, 121 as involving students in unfamiliar situations, 112 and knowledge base development, 12 and learning to serve, 171 and measurement or design issues, 25 as a pedagogy, 5-7 and personal development, 147 and prejudice vs. tolerance, 191, 195 and problem solving, 158

226

INDEX

relevance to (as scale criterion), 26-28 research on, ix, xi, 7-8, 9 and sociability, 141 for students with selfish inclinations, 136 and Universities as Citizens project, ix-x Service Learning Tip Sheets: A Faculty Resource Guide (Hatcher), xi Service site, and AIDS Caregiver Scale, 167 SES (Self-Efficacy Scale), 103-107 SID (Scale of Intellectual Development), 205-207 SIV (Survey of Interpersonal values), 55-59 Social Avoidance and Distress (SAD) scale, 139-142 Social behaviors, 121 Social competence, and TSBI, 119 Social Desirability scale, MarloweCrowne. See Marlowe-Crowne Social Desirability scale Social desirability response bias, 217 Social Dominance Orientation (SDO) scale, 193-197 and UOS, 191 Social Interest Scale, and EETS, 111 Socially responsive knowledge, 3 goals for, 6 Social Responsibility Scale, 29 Social standing, in Goal and Mode Values Inventories, 52 Social stimulation, in Goal and Mode Values Inventories, 53 Sociomoral Reflection Measure, and SROM, 68 Sociomoral Reflection Measure—Short Form, 69 Sociomoral Reflection Objective Measure (SROM), 67-72 Short Form of, 69 Spearman-Brown prophecy formula, 19, 217 Specificity, 19-20, 217 Split-half reliability, 18, 19, 217 S-R Inventory of Anxiousness, 140 SROM (Sociomoral Reflection Objective Measure), 67-72 Short Form of, 69

Standardized scales. See Scales State-dependent constructs, 28 Stress, in AIDS Caregiver Scale, 165, 167 Student Developmental Task and Lifestyle Assessment SDTLA), 143-153 Student Development Task and Lifestyle Inventory (SDTLI), 144 Students, in service learning, 7 Supporting evidence, quality of (as scale criterion), 28-29 Survey of Ethical Attitudes, and EPQ, 86 Survey of Interpersonal Values (SIV), 55-59 Temporal consistency, 18, 21 Test-retest procedure, 18 Test-retest reliability, 217 Texas Social Behavior Inventory (TSBI)—Short Form, 119-126 Theory(ies), 12-13, 14, 217 and construct, 20-21 and research, 7-8, 12-13, 15 Torrance Test of Creative Thinking, and PSI, 158 Traditional religiosity, in Goal and Mode Values Inventories, 51, 52 Traitlike constructs, 28 TSBI (Texas Social Behavior Inventory)—Short Form, 119-126 Understanding, elements to enhance depth of, 4 Unidimensionality, 217 Universal Orientation Scale (UOS), 189-192 Universities as Citizens Higher Education Series, xi Universities as Citizens project, ix-x UOS (Universal Orientation Scale), 189-192

Validity, 20-22, 217 concurrent, 21, 215 construct, 8, 20, 21-22, 26, 215 content, 21, 215 convergent, 21, 215 discriminant, 21, 215 external, 8, 16, 215 face, 21, 215 internal, 8, 216 predictive, 21, 216 Value of College scale, and Civic Action scale, 171 Variables, 14, 15, 217 mediating, 27, 216 moderator, 27, 216 qualitative, 14, 216 quantitative, 14, 216 Varimax rotation, 21-22, 217 VFI (Volunteer Functions Inventory), 35-40 and AIDS Caregiver Scale, 166-167 Visions of Morality Scale (VMS), 88-96 and MAS-R, 82 Volunteer activities, 6 Volunteer Functions Inventory (VFI), 35-40 and AIDS Caregiver Scale, 166-167

Watson-Glaser Critical Thinking Appraisal (WGCTA), 201-204 and CCTST, 209 and CCTT, 211-212 and SIV, 57-58 Wellness Scale, and SDTLA, 147 "We-ness" and SDO, 195 and UOS, 191 WGCTA. See Watson-Glaser Critical Thinking Appraisal Work Environment Preference Schedule, and SIV, 57 World View Questionnaire, and MMO, 74

INDEX

227

ABOUT THE AUTHORS

Robert G. Bringle, PhD, earned his doctorate in social psychology at University of Massachusetts in 1974. He serves as Chancellor's professor of psychology and philanthropic studies as well as director of the Center for Service and Learning at Indiana University-Purdue University Indianapolis (IUPUI). Dr. Bringle has been involved in the implementation and evaluation of educational programs and is recognized for his research on jealousy in close relationships. His interests also include institutionalizing service learning and civic engagement. He is editor of With Service in Mind: Concepts and Models for Service-Learning in Psychology (with D. Duffy) and Colleges and Universities as Citizens (with R. Games &. E. Malloy). Dr. Bringle was awarded the Ehrlich Award for Service Learning from Campus Compact and the Hiltunen Award from Indiana Campus Compact. He is a member of the Campus Compact Consulting Corps, the National Review Board for the Scholarship of Engagement, and a consultant on the Community-Higher Education-Service Partnership project in South Africa. Mindy A. Phillips, ABT, is project manager and lead training instructor at Foresight International. Ms. Phillips has been involved in research on jealousy, service learning, employee surveys, and 360° surveys. She is currently employed as a project manager for a survey software company where she also is involved in the training and development of survey software. Ms. Phillips graduated summa cum laude from Saint Mary's College, Notre Dame, Indiana, and is currently finishing her thesis for an MS in industrial/organizational psychology at Indiana University-Purdue University Indianapolis. Michael Hudson, MS, received his master's degree in clinical rehabilitation psychology at Purdue University in 2001. Mr. Hudson is program manager 229

in the office of Adaptive Educational Services at Indiana University-Purdue University Indianapolis. He has worked in the field of rehabilitation and disability since 1998 and currently coordinates disability accommodations at the post-secondary level. In addition to his work with service learning research, Mr. Hudson has been involved in a number of research projects assessing the impact of disability perception on post-secondary educational success. He has made professional presentations on the topic of education and disability and is a member of the Association of Higher Education and Disability (AHEAD). Mr. Hudson is currently developing a comprehensive program of academic, social, and cultural transition to post-secondary education for students with disabilities.

230

ABOUT THE AUTHORS