Developing Quality Dissertations in the Social Sciences: A Graduate Student's Guide to Achieving Excellence

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page b

COPYRIGHT © 2009 BY STYLUS PUBLISHING, LLC Published by Stylus Publishing, LLC 22883 Quicksilver Drive Sterling, Virginia 20166-2102 All rights reserved. No part of this book may be reprinted or reproduced in any form or by any electronic, mechanical or other means, now known or hereafter invented, including photocopying, recording and information storage and retrieval, without permission in writing from the publisher. Library of Congress Cataloging-in-Publication-Data Lovitts, Barbara E., 1960– Developing quality dissertations in the social sciences : a graduate student’s guide to achieving excellence / Barbara E. Lovitts and Ellen L. Wert.— 1st ed. p. cm. Includes bibliographical references and index. ISBN 978-1-57922-261-1 (pbk.) 1. Dissertations, Academic—Handbooks, manuals, etc. 2. Social sciences—Study and teaching (Graduate)—Handbooks, manuals, etc. I. Wert, Ellen L., 1953– II. Title. LB2369.L685 2009 808'.0663—dc22 2008031382 13-digit ISBN: 978-1-57922-261-1 (paper) Printed in the United States of America All first editions printed on acid free paper that meets the American National Standards Institute Z39-48 Standard. First Edition, 2009

10 9 8 7 6 5 4 3 2 1

Acknowledgments Credit for the idea of translating the book version of this work into a handbook for graduate students and faculty goes to Louis Sherman, Purdue University, one of the members of my advisory committee. Ted Greenwood, of the Alfred P. Sloan Foundation, provided moral and financial support for the project and wholeheartedly endorsed creating not one generic handbook but three, one each for the sciences, social sciences, and humanities. My publisher and editor, John von Knorring, Stylus Publishing, ably orchestrated all the people and pieces in what turned out to be a longer than anticipated process. Many, many thanks go to Ellen Wert, my co-author, and the consultants who worked with her, Chris Golde, of Stanford University and a member of my advisory committee; Mary Huba, Iowa State University; and Dannelle Stevens, Portland State University. Last but not least is Karen Klomparens, of Michigan State University and an advisory committee member, who contributed intellectually to this project and provided an institutional home for the funds that supported it.

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page i

Developing Quality Dissertations in the Social Sciences A Graduate Student’s Guide to Achieving Excellence

Barbara E. Lovitts and Ellen L. Wert

S T E R L I N G , V I RG I N I A

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page ii

Contents

List of Tables

iii

Preface To the Faculty

iv

To the Student

vii

1 2 3 4 5

Identifying the Purpose of the Dissertation

1

Understanding Originality and Significance

000

Aiming for Excellence in the Dissertation

000

Maintaining Consistent Quality Within the Dissertation

000

Achieving Excellence

000

Appendix A: Tasks of the Social Sciences Dissertation

000

Appendix B: Advice for Writing a Social Sciences Dissertation

000

Making the Implicit Explicit: About the Study

000

ii

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page iii

Tables

Table 1.1.

The Purpose of the Dissertation: Descriptions from Faculty in the Social Sciences

000

Table 1.2.

The Purpose of the Dissertation: Descriptions from Faculty in Economics, Psychology, and Sociology

000

Original Contribution: Characterizations from Faculty in Economics, Psychology, and Sociology

000

Significant Contribution: Characterizations from Faculty in Economics, Psychology, and Sociology

000

Table 3.1.

The Economics Dissertation at Different Quality Levels

000

Table 3.2.

The Psychology Dissertation at Different Quality Levels

000

Table 3.3.

The Sociology Dissertation at Different Quality Levels

000

Table 4.1.

The Tasks of a Dissertation in the Social Sciences

000

Table 4.2.

Introduction in the Social Science Dissertation at Different Quality Levels

000

Table 4.3.

Literature Review in the Social Science Dissertation at Different Quality Levels

000

Table 4.4.

Theory in the Social Science Dissertation at Different Quality Levels

000

Table 4.5.

Method in the Social Science Dissertation at Different Quality Levels

000

Table 4.6.

Results or Data Analysis in the Social Science Dissertation at Different Quality Levels

000

Discussion and Conclusion in the Social Science Dissertation at Different Quality Levels

000

Table 2.1. Table 2.2.

Table 4.7.

Table A.1. Tasks of the Economics Dissertation by Quality Level

000

Table A.2. Tasks of the Psychology Dissertation by Quality Level

000

Table A.3. Tasks of the Sociology Dissertation by Quality Level

000

Table B.

000

Writing the Social Science Dissertation

iii

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page iv

To the Faculty

on your own dissertation, you probably now see it as a major exercise that demonstrated your expertise at the moment when you were ready to make the transition from being a student to being a professional. Most of your faculty colleagues would say the same thing, as I discovered through a study of graduate faculty across 10 disciplines. Again and again, the faculty described the dissertation as both an opportunity to develop and sharpen skills and knowledge and evidence of a student’s readiness to work independently at a professional level.1 This understanding of the dissertation, it seems, comes with some distance from the process. However, while students are anticipating or engaged in the process, the dissertation is a bit of a mystery and a source of great anxiety. Many students spend a great deal of time (perhaps too much time) wrestling with process questions such as how to identify a topic and how to organize time and material. They try to make sense of a vague demand that the dissertation be “original” and “significant.” Important questions about expectations of quality—What does it mean to make an original or significant contribution? What constitutes an outstanding or very good dissertation?—often go unasked because it is assumed, by both faculty and students alike, that students should simply know. Of course, they do not know. That they do not is evident from the varying levels of quality, both from student to student in a department and within individual dissertations. Some faculty spend a great deal of time coaching and guiding students through the process and do so for a longer period of time than should be necessary. Further, the dissertation is the point of attrition for nearly half of aspiring PhDs who make it this far. Yet it is the rare department that ex-

plicitly and publicly states its expectations about this capstone project.

L

OOKING BACK

Shared Expectations When I asked experienced dissertation advisors to describe their expectations for originality, significance, and quality in their students’ dissertations, the responses I heard were clear and consistent both within and across the disciplines and fields. The faculty offered similar views on the purpose of the dissertation and what it means to make an original and a significant contribution. They also expressed similar views on what constitutes outstanding, very good, acceptable, or unacceptable work. It seems, then, that students spend a great deal of valuable time and energy trying to guess something that the faculty implicitly agree upon but have not articulated in any formal way, either to the students or each other.

Why Explain the Dissertation? Although the dissertation represents the transition from student to professional and should display the capacity for self-directed work, students need and deserve the benefit of guidance about the process and clear expectations about quality. The department or program has a responsibility to communicate clearly both its standards of quality and those of the discipline. The faculty advising dissertations have a particular responsibility to explain the process, work with students to set firm but realistic deadlines, and provide timely, constructive feedback at every step.

iv

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page v

P R E FA C E There are ethical, professional, and practical reasons for making expectations about the dissertation explicit, the process of review transparent, and the production efficient: It is only right and fair to give students guidance about standards and expectations and then allow them to make choices about how to use it. Members of the profession who prepare doctoral students should communicate the norms and skills of the field to those who seek to enter it.2 The process of dissertation advising will move more quickly and efficiently if students know what is expected of them. Some faculty members may argue that this call for greater clarity and transparency is a plea for handholding or coddling. Actually, I am suggesting something quite the opposite. If advisors provide guidelines about standards and expectations from the start, students’ responsibilities for meeting them become immediately apparent. Students will also have benchmarks against which they can judge and revise their work, reducing the amount of work for advisors and committee members.

Making Expectations Explicit When it became clear that the findings of my study could be put to very specific and practical use, the Alfred P. Sloan Foundation, which funded the study, offered to underwrite a set of booklets for graduate students in the sciences, engineering, and mathematics; social sciences; and humanities. The booklets summarize faculty members’ descriptions of the purpose of the dissertation, the role and nature of an original and a significant contribution, and levels of performance. The specific examples throughout this booklet, many of them drawn from economics, psychology, and sociology, represent the observations of 86 faculty from 21 social science departments.

Using This Booklet: Benefits to the Faculty and Program Although, as I discuss below, my primary hope is that you and your students use this booklet together, I also hope you and your colleagues will find it valuable in your ongoing conversation about guiding your program’s students. Over time, a collective effort to artic-

v ulate expectations will help your program achieve a consistent level of quality. In addition, with a transparent process in place and a body of evidence around student achievement, you will also be better equipped to provide information about program quality as part of internal and external accountability processes.

Using This Booklet: Benefits to the Student This booklet is intended primarily as an entry point for discussion with the students you advise. I do not pretend to represent what you, as advisor, expect or what your program requires. Instead, you will find, throughout the booklet I urge the student to talk with you and your colleagues about the particular emphases and expectations of your discipline, subdiscipline, department and program. The information in this booklet is offered as a way to start the discussion. Indeed, the earlier students are introduced to the standards and expectations of their disciplines, the better. You might consider providing this booklet to students long before they start the dissertation, for example, in a research methods course or in a journal club. Whether you bring this booklet to your students’ attention or they bring it to yours, I hope that you will use it with them to plan, set goals, and provide feedback on work in progress. For example, the sections about the purposes of the dissertation, originality, and significance would be important to discuss with students who are preparing proposals and planning their projects. The descriptions of quality might be of use as you discuss the specific goals of a student’s project. I believe that you and your students will find the process of identifying expectations for the dissertation, as a whole and in its parts, very useful. Your students will be better able to assess their own work and address deficiencies before they submit drafts to you. When you receive drafts, you can use the expectations to provide focused feedback. This puts the conversation about quality directly in the context of professional standards and expectations. Finally, your efforts to articulate quality will help students learn to judge quality in their own and others’ work. They can share what they have learned with colleagues and peers; ultimately they will become better

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page vi

P R E FA C E

vi professionals and, for those who become faculty members, better advisors. I do not, however, suggest that these descriptions of quality be casually given quantitative values and used to “score” dissertations for decisions about passing or failing. In fact, I strongly resist this impulse. I hope that you will use the descriptors with students to set goals and monitor progress while they are in the process of researching and writing their dissertations so that they can make improvements along the way.

Focusing Students on Responsibility More clearly defining expectations is a way to inspire students and spur them to higher levels of achievement. I suggest offering a set of clear expectations about the dissertation, articulating goals, and providing feedback and guidance because I believe doing so challenges students to meet their responsibility as aspiring scholars and researchers, to check their progress

toward professional excellence, and to work more efficiently and independently. Barbara E. Lovitts

Notes 1. The study and the findings are described in detail in Barbara E. Lovitts, Making the Implicit Explicit: Creating Performance Expectations for the Dissertation (Sterling, VA: Stylus Publishing, 2007). 2. On this topic, faculty and graduate students alike might be interested in the observations of senior faculty from neuroscience, education, and the history of science, along with others from the humanities, mathematics and science disciplines, found in C. M. Golde, G. E. Walker, and associates, Envisioning the Future of Doctoral Education: Preparing Stewards of the Discipline. Carnegie Essays on the Doctorate (San Francisco: JosseyBass, 2006).

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page vii

To the Student

A Question of Quality

just beginning your graduate studies or have advanced standing in your program, it is highly likely the dissertation is much on your mind—and with good reason. This capstone project of your graduate education is not like anything else you have written before. It is unlike anything you will write afterward. The dissertation is a unique, hybrid project—both a professional-level report on research and a crucial piece of evidence of your qualification for a credential. It is a point of exit from student status and a point of entry to professional and scholarly life. In short, your dissertation is a process and a product. Writing a dissertation is a process of developing expert knowledge: the understanding, skills, and thought processes of a professional researcher. In this sense, you began your dissertation as soon as you entered your graduate program. Through classes, supervised research, participation in journal clubs, teaching assistantships, and conversations with fellow students, faculty members, mentors, and advisors, you are acquiring skills and sorting information, learning about issues, making connections, and developing ideas for work to be done in your field—all of which is necessary for writing the dissertation. As a product, the dissertation is an external representation that you have achieved the expertise necessary to be a professional in your field. It demonstrates that you know how to approach and master complex ideas and information; you have mastered your discipline’s knowledge base, acquired its professional skills and competencies, and are capable of doing independent research. And so, as a product, the dissertation serves as a “union card” or credential for admission into the scholarly and professional community.

W

HETHER YOU ARE

Faculty in the social sciences are currently giving serious thought to the nature and meaning of the dissertation. Many departments and programs are moving away from single-study dissertations, long the traditional form, and toward a dissertation compiled from a set of articles or research studies. Your dissertation might focus on a single study, report on several studies, or consist of a collection of published or publishable papers. Judgments about the quality of a dissertation, however, are independent of its form. As you talk with your advisor and dissertation committee members about the form your dissertation will take and the departmental requirements you should meet, you should be asking questions about their expectations: What does it mean to make an original or significant contribution? What criteria will be used to judge the quality of my work?

Starting the Conversation About Quality Knowing that questions about quality and expectations often go unasked, and that few programs or departments have explicit expectations, I decided to ask a representative group of faculty who have extensive experience advising dissertations to describe what they expect in a dissertation: What is the purpose of the dissertation? What constitutes an original or significant contribution? What makes a dissertation outstanding, very good, or acceptable? Under what conditions is work unacceptable? The responses of 86 faculty from 21 economics, psychology, and sociology departments are the foundation of this booklet. As a way to start the conversation

vii

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page viii

P R E FA C E

viii about quality, I offer these advisors’ general descriptions of standards and expectations for the dissertation in the social sciences, with specific examples drawn from psychology, economics, and sociology. It is important to keep in mind that these descriptions are a starting place. After reading this booklet, talk with your advisors and other faculty members to understand the expectations specific to your field, department, and project. Ask them what in your department and field is considered to be an original or significant contribution and what constitutes an outstanding or very good dissertation. Read recent dissertations from your department. Ask your advisors to recommend an outstanding or very good dissertation in your field—and study it. Look at your own work critically and ask for feedback in terms of the descriptors of quality presented here. I also suggest that you discuss this booklet and its information with your fellow students. It is an opportunity to develop colleagues and a way to combat isolation and the feeling that you are the only person wrestling with tough questions. In fact, as you will see throughout this booklet, the key to quality in the dissertation is communication. If nothing else, I hope that reading this booklet will prompt you to have many conversations with your dissertation advisor, to seek more than one mentor among the faculty, and to talk with other students to compare experiences. I hope you will map out your

dissertation with your advisor and committee and check in frequently with them. Finally, I hope you will ask questions, ask for feedback, and follow up with your program’s administrators if you are not getting the help you need.

A Question of Professional Responsibility As someone pursuing advanced training, you are learning what constitutes credible work in the field. Your increasing understanding of the standards of the field is part of what makes you a professional. It also means that you have to take responsibility for your work. Whether your faculty mentors, dissertation advisor or advisors, committee members, or department heads take the initiative to make their expectations of quality explicit or you take the initiative to bring this booklet to their attention and ask for their thoughts, once their expectations are made clear, it is your responsibility to act on them. As someone who aspires to thePhD, you are responsible for setting goals for yourself, checking your progress, asking for feedback, and taking feedback in the context of professional standards. I wish you well in your work. Barbara E. Lovitts

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 1

1 Identifying the Purpose of the Dissertation

on the title page of nearly every dissertation, “Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy,” underscore that the dissertation is part of a process. The requirements of your program and the many informal opportunities for learning are designed to move you from student to professional, from someone largely dependent on others for guidance in learning to an independent, expert learner and producer of knowledge. Consider the observations of the faculty who participated in the study on which this booklet is based. The various purposes they ascribe to the dissertation,

T

summarized in Table 1.1, point to the dissertation as a product that provides evidence that you have mastered professional skills and knowledge: your discipline’s theories and methods and a vast array of facts, principles, concepts, and paradigms. The dissertation is also evidence that you have developed informed opinions about various issues, learned how to approach problems, and how to judge others’ work. Whether you plan a career in academia, government, business, industry, or in the nonprofit sector, the successful completion of your dissertation will signify your ability to conduct high-level inquiry and to create new knowledge.

HE WORDS

Table 1.1 The Purpose of the Dissertation: Descriptions From Faculty in the Social Sciences The purpose of the dissertation is to prepare the student to be a professional in the discipline. Through this preparation the student learns and demonstrates the ability to conduct independent, original, and significant research. The dissertation thus shows that the student is able to • • • • • • • • • •

identify/define problems, generate questions and hypotheses, review and summarize the literature, apply appropriate methods, collect data properly, analyze and judge evidence, discuss findings, produce publishable results, engage in a sustained piece of research or argument, think and write critically and coherently.

The dissertation shows mastery of the field, that the student is ready to be a professional in and contribute to the discipline. The dissertation prepares the student for a career. It is the capstone of the graduate education and research experience, a rite of passage from student to professional. It is a “union card” or credential for admission to the profession.

1

17221-DQD-SocialSciences

2

12/1/08

4:47 PM

Page 2

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S O C I A L S C I E N C E S Table 1.2 provides descriptions that emerged from the discussions with the faculty about the dissertation in their discipline. As you consider the discipline-specific examples in Table 1.2, think about your particular field and program. Ask your advisor and other faculty members what they consider the purpose of the dissertation to be. What aspects of the dissertation does your field or program emphasize? What are the expectations, for example, about the quality of writing? As you discuss your dissertation with your advisor and committee members, ask about ways your particular project best lends itself to serving these purposes. What, specifically, must your dissertation demonstrate?

What, exactly, will give evidence that you have mastered the expected knowledge and skills? What will demonstrate your capacity to independently produce professional-level work in the future? From these conversations, you might want to draw up a summary of the purposes you and your advisors agree on. Ask your advisor, committee, and other faculty mentors to suggest recent dissertations from your department that might serve as good examples. Look at how the students assembled their dissertations. Put yourself in your committee’s shoes and consider how the students fulfilled the purposes of the dissertation that your advisors have described.

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 3

Identifying the Purpose of the Dissertation Table 1.2 The Purpose of the Dissertation: Descriptions From Faculty in Economics, Psychology, and Sociology Economics The purpose of the dissertation is to allow students to • practice the habits of professional economists; • learn to be researchers who do original, creative, significant work; • learn how to do research—generate questions, apply the appropriate methods, and solve problems independently at a professional level. The dissertation demonstrates the student’s ability to do original research, use tools appropriately, and produce an independent piece of work. Its function is to launch a career and help the student get a job; to credential the individual as a professional economist. Psychology The purpose of the dissertation is to allow students to learn how to be an independent researcher. The dissertation demonstrates the student’s skills and ability to independently conceive and conduct original, significant, scholarly research: develop important researchable problems, define an experiment, generate and address hypotheses, conduct the experiment, and produce publishable results. The dissertation is a process that encourages and fosters creativity, a test of whether the student can do research independently, the preparation for and launching of a career, on-the-job training, a capstone, a rite of passage, the culmination of graduate education and the start of a career as an investigator, and a transition from being a student to being an independent scholar. Sociology The dissertation trains scholars; the student learns how to do independent research, learns certain useful skills, learns to synthesize the literature, analyze unformed data, and reach conclusions and write them up; the student learns what the professional standard of scholarship is. The dissertation demonstrates that the student can mount an original, independent research project and take it from conceptualization to completion independently, formulate problems or questions, review the literature, collect and analyze data, and discuss findings. It demonstrates that the student can do sociology, knows the area at a sufficient level of depth to teach it, and can be an independent, productive academic/scholar. The process of the dissertation moves the students from thinking of themselves as research assistants to thinking of themselves as independent researchers/sociologists. It develops independence and the ability to do original and significant research and put together a product that has internallogic, is coherent, and makes an innovative, significant contribution to the field. The dissertation gives the student a product that can be published; it teaches the skills necessary to get a tenure-track job. It is a rite of passage, a test for establishing a person’s claim to be a scholar, a certificate for admission into the profession.

3

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 4

2 Understanding Originality and Significance

the terms original and significant in the examples in chapter 1. Maybe you have heard or been told that you must make an original and significant contribution to the field through your dissertation. Graduate students across all disciplines spend a great deal of time worrying about these expectations: What, exactly, does it means to make an original or significant contribution in my field? Is my project or its results important enough to be considered original and significant? What, specifically, about my dissertation must be original or significant? In talking with the faculty who participated in the study, it became evident that the qualities of originality and significance are elusive and difficult to define. Moreover, the terms are often shorthand for “the capacity to make an original or significant contribution.”

It is important to understand that the original contribution is not necessarily your entire dissertation but something that is part of it. The faculty in the study explained that an original contribution may result from

Y

OU PROBABLY NOTICED

• asking or identifying new questions, topics, or areas of exploration; • applying new ideas, methods, approaches, or analyses to an old question, problem, issue, idea, or context; • developing or applying new theories, theorems, theoretical descriptions, or theoretical frameworks, or reinterpreting old ones; • inventing, developing, or applying new methods or techniques; creating, finding, or using new data or data sets; • applying old ideas, methods, approaches, or analyses to new data; • developing, modifying, or applying new analyses, analytic approaches, frameworks, techniques, models, or statistical procedures; • coming up with new ideas, connections, inferences, insights, interpretations, observations, perspectives; • producing new conclusions, answers, findings, or results; • combining or synthesizing things (facts, knowledge, models of inquiry, problems, sources, theoretical constructs) from other fields or disciplines.

Originality An original contribution offers a novel or new perspective. The faculty in the social sciences who participated in the study described an original contribution as something that has not been done, found, proved, or seen before. It is publishable because it adds to knowledge, changes the way people think, informs policy, moves the field forward, or advances the state of the art. To achieve this goal, you might develop an original insight or advance, or you might borrow a contribution from another discipline and apply it to your field for the first time.

Your original contribution might appear in almost any part of the dissertation. It can be a question, theory, data, data source, method, analysis, or result.

4

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 5

Understanding Originality and Significance Degrees of Originality There are, of course, levels or degrees of originality. At the lowest level, originality typically involves applying known methods or techniques to new data, materials, or the like. Originality in this sense might make a small or incremental change to the knowledge base and be considered to have little consequence. At the higher levels, making an original contribution typically involves asking new questions or applying new methods to old or new problems and achieving results that are immediately recognizable as having consequence. Contributions such as these advance the field, are publishable, and often appear in top-tier journals. At the very highest level, the contribution has the potential to change the field. It is rare for a graduate student to make an original contribution at the highest level. Indeed, few faculty make contributions at this level. Table 2.1 provides some examples, drawn from economics, psychology, and sociology, of what it means to make an original contribution. As you consider these descriptions, think about the questions you might ask your advisor and committee members about their expectations and the potential of your project to create new knowledge, take a new approach to inquiry, or explain or solve a social problem. The expectations for originality also vary by subiscipline. In experimental or empirical fields, originality usually involves generating novel information or data. By contrast, in theoretical or historical fields, it could be a new idea. However, your role in making an original contribution might be difficult to identify, especially if you are in a field that relies on a high degree of teamwork, or if you are one of several students collaborating on an advisor’s funded research project. It is, therefore, important to talk with your advisor about your role in the project and how to identify your contribution to the field. Understand Expectations Because expectations for originality vary across disciplines, subdisciplines, and even advisors, it is important to ask your advisor and committee, from the start, for clarification about what is expected of you. Ask to see examples of what they mean when they—and others in the field—call something original. In your ongoing conversations, discuss specifically where, in your project, you are most able to make an original contribution. Is it through, for example, your research question, theory, data, data source, method, analysis, or

5 result? Are you using an existing concept or method in a new way? Discuss, as well, how you might best bring your original contribution to the reader’s attention. Significance A significant contribution is defined primarily by its consequences. As suggested by the descriptions provided by the faculty from economics, psychology, and sociology who participated in the study (summarized in Table 2.2), a significant contribution is of interest and importance to the community and influences the field by changing the way people think. It is important to understand that significance is typically determined, over time, by the disciplinary community, not the individual advisor or committee. Moreover, understanding and appreciation of a contribution often emerges, sometimes many years after the fact. Sometimes a topic, issue, or approach is before its time; it does not immediately fit with prevailing knowledge or thought, but later it is recognized as significant. Although significance is frequently stated as a requirement for thePhD, graduate students rarely make significant contributions—or are rarely recognized at the time for making significant contributions. Faculty do not typically expect them to do so. They look, rather, for the capacity to produce a significant contribution in the future. The faculty who participated in the study described a significant contribution as “something that is useful and will have impact, and is therefore publishable in top-tier journals” because it

• offers a nontrivial to very important original breakthrough at the empirical, conceptual, theoretical, or policy level; • is useful and will have an impact; • causes those inside, and possibly those outside, the community to see things differently; • influences the conversation, research, and teaching; • has implications for and advances the field, the discipline, other disciplines, or society. As with originality, there are degrees of significance. At the highest level, significance is a function of the field’s long-term interest in the problem, the difficulty involved in solving the problem, the influence of the results on further developments in the field, as well

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 6

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S O C I A L S C I E N C E S

6

Table 2.1 Original Contribution: Characterizations From Faculty in Economics, Psychology, and Sociology Economics An original contribution is something nontrivial that has not been done before, a deviation from or a new way of thinking about an economic feature, issue, or problem. It advances knowledge and is publishable. It might result from • • • • • • • • • • • •

identifying a gap in the literature; redefining or reconceptualizing old problems; asking a new question; making new connections among existing data; using a new source of data or data set; constructing or amalgamating new data; applying old econometric ideas to new data; applying new econometric ideas to old data; applying, modifying, or developing a model or technique to solve a problem; proving a new result; proving a theorem using weaker assumptions; challenging existing theories or policies.

Psychology An original contribution is something that has not been done, shown, or made available before that creates new knowledge and is publishable. It might be • a new question, idea, insight, perspective, theory, model, technology, method, or finding; • a novel twist or approach to an old question; • an empirical or theoretical advance. It might result from • • • • • • • •

applying new, innovative, cutting-edge methods to existing theory; synthesizing knowledge; using or integrating something from another discipline; generating new data; analyzing existing data sets in new ways; clarifying someone else’s findings; resolving issues or clearing up some confusion in the field; making an empirical or theoretical contribution.

Sociology An original contribution goes beyond what is known, offers new questions or contexts, opens new areas of exploration or a new angle on an old area of exploration, provides a fresh empirical focus to some key theoretical puzzle or debate in the literature, takes an important next step, brings things together in a new way, or extends a debate or the current thinking on a particular topic. It changes the way people think about a certain topic, leads to further research, is a meaningful contribution, and adds to the literature. It might result from • identifying an unanswered question that resonates with a larger theoretical issue; • applying an established theory in a new context; • reframing existing data and shedding new light on them theoretically, substantively, or methodologically; • challenging or reinterpreting existing theory or methods; • developing a set of concepts or ideas; • developing a new theory or method.

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 7

Understanding Originality and Significance as the degree to which the results affect other fields, disciplines, and even society. At the lower levels, the contribution is a small or incremental improvement. A contribution is of little or no significance if people say, “So what?” However, the most significant contributions are often ones that initiate a new trend or destabilize a conservative area, thus creating new questions and new research agendas. Table 2.2 summarizes the descriptions of a significant contribution provided by the faculty from economics, sociology, and psychology who participated in the study.

Understand Expectations Talk to your advisor and committee members about their expectations for significance. Do so early in the process, and continue this discussion as you progress. Do they want to see a significant contribution in your dissertation or do they want you to demonstrate that you have the capacity—the knowledge and skills—to make a significant contribution in the future? What specific aspect of your project has the potential to make a significant contribution? How can you best present this contribution?

Table 2.2 Significant Contribution: Characterizations From Faculty in Economics, Psychology, and Sociology Economics Surprising, impressive, important, and useful, a significant contribution is worthy of publication in toptier journals because it causes people to see things in a different way and makes progress at the empirical, theoretical, or policy level. A big, useful, or relevant idea, it increases understanding of an economic problem; challenges existing theory or policy; advances methodology and pushes the empirical frontier; extends data or methods in a nontrivial way; provides greater validation of existing results; will have wide applicability; cuts across many fields or disciplines; will be used by other people; or advances or provides greater insight into the discipline or the world. Psychology A significant contribution is something that advances knowledge. It addresses or distinguishes between competing hypotheses; influences theory development and research; leads to the modification of existing theory or hypotheses; eliminates a theory and provides support for another theory; offers interesting, meaningful, or counterintuitive results; has an application; is of interest to others and affects their research; changes the discourse in the field; or affects future research. Sociology Surprising or unexpected, a significant contribution extends knowledge and pushes an area forward. It does so by • • • • • • • • • • • • •

studying something no one has studied before; offering new findings, formulations, arguments, sets of comparisons, or methods; using a richer and more extensive data set; filling in some missing piece; unlocking a term or phase and captures people’s imagination; conceptualizing or articulating something better; providing various confirmations or amendments; clarifying a point in a debate; shedding light on an issue; identifying an important theoretical puzzle; addressing an emerging social problem or question that has implications for the larger society; educating people in the field; opening up a new field.

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 8

3 Aiming for Excellence in the Dissertation

tances, some provided deoutstanding, n the followhat they said e these sumr, evaluating es as you disd committee goal for excelfort to develop e the quality?

d by originalonsequences. ure and masa richness of rtant breakdissertations ds, the exper. The quality hniques and s are rare— that often— ovide a very bed an outphysical scithe higher it mportant

the scholarship of others to learn. But in the process, you also make judgments about the quality of their work. In the same way that you make judgments about the scholarship of others, your advisor and committee members make holistic judgments about the quality of their students’ dissertations. However, your faculty advisors also read student work with another purpose: They read to teach. They must read carefully to see where they can suggest improvements in students’ conceptualization of the topic, in their methods, in their presentation of results, and in their writing. Moreover, your advisors and committee read to certify quality. That is, they must make sure their students’ dissertations demonstrate professional competence and capacity for future professional-quality contributions. In short, your advisors and committee are reading your drafts and final version to determine whether your dissertation is at a level of quality that demonstrates your readiness to make the transition from student to professional.

Y

The faculty participating in the study provided descriptions of what makes a dissertation outstanding, very good, acceptable, or unacceptable. In the sections following, you will find summaries of what they said about quality at these different levels. Use these summaries as a way to start planning and, later, evaluating your own work. They are also useful guides as you discuss your project with your advisors and committee members: Am I making progress toward my goal for excellence? Where do I need to make a special effort to develop my dissertation? What might I do to improve the quality?

OU READ

Outstanding Outstanding dissertations are characterized by originality, high-quality writing, and compelling consequences. They show deep knowledge of a massive amount of complicated literature and mastery of the subject matter. They display a richness of thought and insight, and make an important breakthrough. The body of work in outstanding dissertations is deep and thorough. The student demonstrates a sophisticated grasp and use of theory. In experimental fields, the experiments are well designed and well executed. The quality and care put into the measurement techniques and analyses instill confidence in the results. The data are rich and come from multiple sources. Even though outstanding dissertations are rare— faculty see them once or twice a decade, if that often— the faculty in the study were able to provide a very consistent set of descriptors. They described an outstanding dissertation in the social sciences at the higher levels of originality or significance in that it

Degrees of Quality Like published articles, completed dissertations have been written and rewritten. The ideas and presentation have been subjected to expert criticism and honed through repeated drafts, feedback, and editing. And, like published research articles and books, most dissertations are very good. A few dissertations are remarkable or outstanding in some aspect. On the other hand, some dissertations are, for a variety of reasons, just within the boundaries of the profession’s standards of quality. They are good enough. In rare instances, some dissertations are unacceptable.

• asks new questions; • addresses an important question or problem;

8

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 9

Aiming for Excellence • uses or develops new tools, methods, approaches, or new types of analyses; • pushes the discipline’s boundaries and opens new areas for research; • has practical and policy implications; • is of interest to a larger community and changes the way people think.

9 Other terms the faculty used to describe outstanding dissertations were “compelling,” “concise,” “counterintuitive,” “creative,” “elegant,” “engaging,” “exciting,” “insightful,” “surprising,” and “thoughtful.”

• is very well written and very well organized; • exhibits mature, independent thinking; • displays deep understanding of a massive amount of complicated literature; • exhibits command and authority over the material; • challenges the literature and strongly held traditional views; • is thoroughly researched; • is synthetic and interdisciplinary; • clearly states the problem and explains why it is important; • has a brilliant research design; • has well-planned and well-performed experiments (if experimental); • is theoretically sophisticated and shows a deep understanding of theory; • has rich data from multiple sources • has a comprehensive, complete, sophisticated, and convincing analysis.

Very Good The very good dissertation is very good indeed. It fulfills the purposes of the dissertation requirement and establishes the student as a capable social scientist. The majority of the dissertations that faculty see are very good, and this is the level they expect of most graduate students. The faculty in the study explained that a very good dissertation displays the student’s mastery of the field, addresses a meaningful question or problem, and is executed competently. Although it might not hold the promise of altering the field, it has the potential to contribute to the field by expanding its knowledge and thinking. The dissertation contains material for two or three papers that could be published in top-tier professional journals. More specifically, the faculty described a very good dissertation as “original or significant,” making a “modest contribution to the field.” A very good dissertation has a good question or problem. It shows understanding and mastery of the subject matter; uses appropriate, standard theory, methods, and techniques; includes well-executed research; demonstrates technical competence; presents solid, expected results/answers; and is well written and well organized.

The faculty also described the outstanding dissertation as having the potential to “illuminate an entire area,” “startle the field,” or “stimulate a lot of activity in the profession.” Indeed, the results or conclusions of an outstanding dissertation push the discipline’s boundaries and are publishable in the top-tier journals. Along with offering new and significant knowledge, an outstanding dissertation is a pleasure to read. It has a point of view and a strong, confident, independent, and authoritative voice. Each part of the outstanding dissertation, from introduction through conclusion, is excellent, and the pieces are integrated seamlessly. The writing is clear and persuasive. The ideas are set out very clearly and concisely. The writer anticipates—and answers—the reader’s questions. Outstanding dissertations were described as “page turners,” surprising and edifying the reader. Readers often react with, “Wow! Why didn’t I think of that?”

Acceptable A dissertation that meets the basic criteria for the award of the PhD is considered acceptable. Such a dissertation contains a sufficient amount of solid work to demonstrate that the student can do research. It might result in some conference papers, but it has little in the way of publishable material, and what is publishable is likely to be accepted by lower-tier journals. The faculty in the study explained that an acceptable dissertation demonstrates technical competence and shows the student’s ability to do research, use standard methods, and competently apply theory to a problem. However, they noted, a student might display a narrow understanding of the field. For instance, the student might present an uncritical review of the literature that does not show insight or understanding of what is important. The analysis might be unsophisticated or limited.

They explained that in its execution, the outstanding dissertation

17221-DQD-SocialSciences

10

12/1/08

4:47 PM

Page 10

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S O C I A L S C I E N C E S The acceptable dissertation shows little promise of adding much to the field. It is not very original or significant because it is narrow in scope. It typically focuses on a question or problem that is not interesting or has predictable results. It might be a highly derivative, small extension of someone else’s work. Sometimes a dissertation that is adequate may reflect circumstances. In some studies, the hypotheses turn out to be wrong or the results are not statistically significant, meaningful, or important. Sometimes an acceptable dissertation is the result of choices and compromises: The student has accepted a job or post-doc position and needs to sprint to finish. In such instances, the student has achieved a primary purpose of graduate education, which is getting a professional position. More typically, however, an adequate dissertation is the product of poor communication between student and advisor, or inadequate advising. Because much of this guidance should take place before you begin to write the dissertation, it is important to talk soon and in detail with your advisor and other faculty members about your topic, your research question or problem, your plan for researching it, the methods you will use to collect and analyze data, the results you are getting, and your interpretation of the results. Get early feedback on your plan for organizing your dissertation and presenting your results. In addition, be sure to get feedback on the quality of your writing as you begin to draft the chapters. Because strong skills in organizing and writing are a critical professional attribute, even if your dissertation is very good or even outstanding in other respects, it will be considered only acceptable if you cannot communicate your ideas clearly and effectively. Just as excellent writing enhances a solid piece of scholarship, weak writing undermines otherwise excellent ideas and research. Unacceptable It is your responsibility to produce professional-quality work, and it is your advisor’s responsibility to prevent unacceptable work from advancing. As the faculty who participated in the study concurred, faculty advisors should provide the guidance necessary to ensure that the dissertation meets professional standards. The advisor should make sure that the student is working with a clearly defined question or problem and must make sure that the student is using proper methods. The advisor should also provide prompt and constructive feedback. It is also your responsibility to follow through on your advisor’s and committee’s advice and guidance.

Work that is poorly written and full of errors and mistakes or has other serious flaws is not of adequate quality. The faculty in the study were clear that they would turn back a draft if the question or problem is trivial, weak, or unoriginal. Work that does not demonstrate that the student understands the relevant literature and basic concepts and the key processes or conventions of the discipline is unacceptable when • the literature review is weak or missing. • methods are used inappropriately, or incorrect methods are used. • theory is missing, wrong, or not handled well. • hypotheses are inconsistent, do not flow from theory, or are missing. • the data are flawed, misrepresented, “fudged,” or wrong. • the results presented are obvious, already known, unexplained, or misinterpreted. • the analysis is wrong, inappropriate, incoherent, or confused. • the conclusions drawn from the data are invalid or oversold. The faculty also described the unacceptable dissertation in terms of its presentation: The organization is confusing and the writing is filled with spelling and grammatical errors. They said that a dissertation that violates standards of academic integrity through plagiarism, falsification of data, or misrepresentation of data is unacceptable. These problems should be caught early. Use the criteria in this booklet as a starting point for identifying problems or as a way to plan improvements. If your advisor and other members of your dissertation committee ask you to revise and rewrite, make sure you understand specifically what you need to do to improve—and make the improvements.

Examples From the Disciplines The faculty in economics, psychology, and sociology who participated in the study offered detailed descriptions of the dissertation in their discipline at these four quality levels. As you consider the summaries of their descriptions in tables 3.1 through 3.3, develop questions that you might raise with your advisor and committee about their expectations for quality in general and for your particular project.

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 11

Aiming for Excellence Table 3.1 The Economics Dissertation at Different Quality Levels Outstanding Extremely well written; thoughtful, elegant, clever, compelling, and surprising; internally coherent; chapters have a lot of substance; has the beauty of simplicity; has a lot of creativity, insight, originality, and independence; exhibits command and authority over the material; makes you look at something differently; addresses a very interesting question in a solid way; exhibits a deep understanding of the literature and theory; challenges the literature and strongly held traditional views; contains a very interesting theoretical component; integrates theory across fields; makes a theoretical contribution; constructs a new data set; develops or applies new tools, methods, models, or analytical techniques; exhibits incredibly good data collection and analysis skills; has a complete, comprehensive, and convincing sophisticated analysis; solves a controversy or answers questions of interest to the field; very practically useful for a lot of people; will fundamentally and radically change things. Very Good Original, insightful, integrated; may involve new and creative ideas and be well executed, but the problem is not that interesting; some elements are surprising; has a good question or problem, but the topic may not be of central interest to the field; good question but poor empirical execution; the question is good, but the answer is not; the answer is good, but the question is not; an extension of someone else’s work; the hypotheses have not been developed in a theoretical way, are mainly statements about how the world behaves; lacks the theory needed to conceptualize the work; uses appropriate data collection and analytic techniques; uses advanced mathematical or econometric techniques to analyze the data but cannot really interpret the results; provides convincing evidence, but the analysis does not live up to the idea; has consistent results derived from well-formed assumptions but not contained in the assumptions; the outcomes are more or less predictable; does not push the frontiers; does not really make a significant contribution; contributions are not as deep as they might be; may readjust the way people think but does not fundamentally change things. Acceptable Competent, useful; adequately organized, investigated, and communicated but not particularly clever or original; plausible but not compelling; shows some gaps in reasoning; directed, as opposed to being independently researched; asks an old question; highly derivative, a correct, small, not exciting, extension of someone else’s work; synthesizes other people’s work; has an interesting question that is poorly addressed; does not exhibit a broad understanding of the subfield; does not place the work in context; displays competence in applying theory to a specific problem, in collecting data, and in managing and analyzing the data; applies an existing model to new data; uses a very restrictive model to narrowly answer a question; has not really answered the question; results are believable but the conclusions are not; student does not realize the depth or the significance of what he or she has done; contribution is of minimal value to the profession. Unacceptable Trivial and poorly written; not thorough; lacks careful thought; too ambitious or idealistic; contains errors or mistakes; misleading; has no introduction or literature review; does not understand the relevant literature; the question is not posed well; asks an inappropriate or unoriginal question; student does not see that the question has an obvious answer; has a series of hypotheses not rooted in theory; uses the wrong data set or analytic techniques; uses inappropriate or incorrect methods; uses a system of equations that does not have a solution; cannot solve the problem; the problem has already been solved; misinterprets the data; unable to explain the results; results are self-contradictory; overinterprets the results; student does not understand the implications of the results; cannot demonstrate the conclusion.

11

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 12

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S O C I A L S C I E N C E S

12

Table 3.2 The Psychology Dissertation at Different Quality Levels Outstanding Scholarly; solid, clever, coherent; interesting, persuasive, exciting; counterintuitive; paradigm shifting; well organized and well written; driven by intense curiosity; has an elegant, creative, and original idea and method; identifies a gap that no one else has identified that needs to be done; builds a case in a linear, logical way; anticipates questions; very synthetic; interdisciplinary; looks at the issue from multiple perspectives; eradicates the boundaries among the introduction, literature review, and theory, and integrates them in a seamless, fluent, articulate way; has multiple components or multiple studies; draws on diverse literatures from different disciplines; presents the literature well and shows the gaps; theoretically sophisticated; shows depth of understanding of theory and methods; involves a set of programmatic, linear experiments; experiments are designed to falsify a hypothesis; uses very good, sound methods; develops complex stimuli; involves a new or massive data collection; has an innovative analytical strategy; has compelling data and amazing results; the general discussion ties it all together; publishable in a top-tier journal; changes the way people think; shifts the thinking in the field; has policy implications. Very Good Novel, creative, coherent, independent but less ambitious and crisp; not highly unusual or exciting; the obvious next step in a research program; done confidently but lacks sparkle; technically well presented; the introduction is tight but not particularly new or different; what is original is clear and laid out persuasively; the model and hypotheses are well laid out and well tested; the methods are very good but is not conceptually exciting; explores the range with which two variables interact and affect the dependent variable; uses conventional methods and analyses; uses a new method or statistical application; the results are less crisp and clear; the experiment did not work out as planned; publishable; does not reform the discipline. Acceptable Yeomanlike; correct; technically competent, meets the standard, not very interesting or exciting; executes what is planned, but the research may not work out as expected; an extension of the advisor’s work or an in-depth examination of a single case; the questions are simple and reasonable but not exciting; lacks an introduction; has reviewed the literature and identified a gap, but the gap is not very important; not critical of the literature and what needs to be done; hypotheses are strong enough and consistent with the proposed model; the research design is simple; uses a reasonable method to answer the question; analysis is clear and appropriate, fits the hypothesis, and answers the question; analysis does not explore all the possibilities present in the data; does not do a lot of exploratory or additional analyses to clarify the interpretation of the main analysis; results are not important; discussion does not return to the original question, restates the results, does not identify what has been learned; may not get published; will not change the way people approach the issue. Unacceptable Poorly conceived, wrong, sloppy; has logical flaws; does not do what student said he or she was going to do; says things that readers believe are false; introduction is shoddy; misses major aspects of the literature; shows little understanding of the core processes; the model and hypotheses are inconsistent; the methodological techniques are poor or flawed; fails to implement the necessary controls; data analysis and results are inappropriate, inadequately reported, unexplained, misinterpreted, or contain errors; the general discussion reiterates the major findings; explanations are not thought through clearly; jumps to conclusions; has no synthesis or big picture context; does not see the next step; requires major revisions; student fails to complete the revisions.

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 13

Aiming for Excellence Table 3.3 The Sociology Dissertation at Different Quality Levels Outstanding Well written; clear and concise; rhetorically very solid; fresh, novel, original, insightful, intellectually engaging, creative, imaginative; well crafted and well executed; coherent, has a watertight argument; the different components are connected in a seamless way; has a point of view and a distinct voice; looks at some aspect of the field in a new way; tells a story and is a good read; grounded in a debate but goes beyond it; a deep investigation that brings empirical focus to something hitherto unclear; shows a unique, rich, in-depth understanding of the topic that leads to the generation of an important question; thoroughly and creatively conceptualized; theoretically sophisticated, brings together theories or concepts in a creative way; has a brilliant research design; examines the questions through the creative and innovative collection and analysis of rich data; uses several kinds of data to build the case; presents results in a convincing and articulate manner; offers a meaningful or cultural interpretation of the results; brings things together that had not been brought together before; student demonstrates the ability to be independent; teaches the reader something; publishable in its present form; speaks to a broader audience; has the potential to change the way people think about the problem. Very Good Solid; done correctly; follows the rules well; demonstrates technical competence; well written and well designed; thorough; has a lesser degree of originality; not overwhelmingly brilliant; lacks a creative, broadly synthetic, innovative spark; seams are not quite as clear cut; the point of view is laid on it; does not have its own voice; may be too rhetorical; may lack a core message or sense of what is critical; addresses an important and interesting question or problem but does not produce excitement or surprise; theoretically and methodologically solid; integrates but does not advance theory; has a good data set and a good set of observations; has enough appropriately analyzed data; weaves in the data; offers useful findings; a good, normal science contribution to the field. Acceptable Demonstrates technical competence; all parts are there; shows student could be a professional in the field; less well written; not much originality, passion, or excitement; less thorough; usually small scale; not broad, focused, or integrated enough; not well conceptualized; arguments are spongy; consistent across sections but has some loose ends; does not have all the subtleties and connections; literature review is very mechanistic; does not have command or grasp of the critical literature; shows poor comprehension of key areas of theory; has minimal or mundane theory; theoretical propositions are not fully worked out; does not typically involve original data collection; the data set is inadequate for the proposition; the right data are not collected; theory and data are not connected; the data are pedestrian; student is not in command of the data; the evidence is not plausible; the quantitative analysis is very unsophisticated and elementary; has predictable results; student cannot step outside position of participant and develop a critical, sociological perspective on the data; student exhibits trouble thinking like a researcher; peters out at the end; cannot answer the “So what?” question. Unacceptable Has major flaws; terrible writing; sloppy presentation; lacks depth of thought and coherent logic; inadequate or incorrect comprehension of basic concepts, poor conceptualization, execution, and methodology; pure replication of another study; does not clearly define a problem or issue; weak or no literature review; inconsistent use of references; student is unable to link bodies of theory to make a theoretical argument; materials were poorly chosen; problem is poorly researched; data collection is flawed; analysis is fundamentally flawed and poorly executed; uses inappropriate statistics; findings are not relevant; evidence does not support the interpretation; interpretation is exaggerated; does not link findings to the broader field.

13

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 14

4 Maintaining Consistent Quality Within the Dissertation

N A SUCCESSFUL DISSERTATION ,

the parts work together to form an integrated whole. Each piece has a clear function and relates to the others. Taken together, all the parts connect to form a coherent argument. Indeed, as the faculty members in the Making the Implicit Explicit study talked about quality in the dissertation, they consistently talked about the quality of the argument, using words like “coherent” and “convincing.” They described high-quality dissertations as “well organized”: the student uses all the parts of the dissertation to build and support a clear, overarching idea. Because the expectations for the parts of the dissertation, their relationship, and importance vary not only from discipline to discipline but within fields and among faculty, it is critical to understand what your committee expects or recommends regarding the tasks and organization of your dissertation. But first, understand the difference between the form of the dissertation and the tasks of the dissertation. The form is the structure, the parts or divisions. Think of the form as the presentation. You accomplish the tasks, regardless of the dissertation’s structure. Think of them as the substance.

called “papers,” “articles” or “essays,” depending on the field); some of these papers may even have been already published. The student is expected to connect the articles in a meaningful way. Faculty in some fields or departments ask that the dissertation be organized specifically by tasks, with section titles such as introduction or problem statement, literature review, theory, methods, results or analysis, and discussion and conclusion. And some dissertations are structured thematically, with the tasks woven throughout the dissertation in a less clearly delineated way. Thus it is very important to understand what, exactly, your advisors and committee mean when they talk about chapters, literature review, or methods. Not only do disciplines differ in their approaches, but each department and university has different requirements— that may be under review or in the process of being revised—about the structure of the dissertation.

I

Understand the Tasks While there is no set formula for organizing the dissertation, there are, however, essential tasks that must be completed regardless of how the dissertation is organized or whether the research is empirical, theoretical, historical, qualitative, or quantitative. Through these tasks, the student demonstrates his or her professional skills and knowledge. For example, at this point in your education you should know the important literature in the field and the state of current thinking about your topic. Your task is to present this knowledge in the context of the research question, to synthesize it, to do so succinctly,

Understand the Form In your early conversations with your advisors about your dissertation topic, it is important to be clear about the requirements for the form of your dissertation. In some social science disciplines or department, students present a collection of papers (they might be

14

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 15

Maintaining Consistent Quality

15

Table 4.1 The Tasks of a Dissertation in the Social Sciences Introduction/problem statement Literature review Theory Method Results/data analysis Discussion and conclusion

and to offer this synthesis in service of your argument. You might be required to present this information in a section called “literature review.” If your dissertation is a compilation of papers, you might not have a literature review section, but you will incorporate a review and synthesis of the literature at appropriate points within the papers. Be aware that ideas about and requirements for the various tasks of the dissertation vary not only by discipline, department, and university, but among faculty members in the same department. Table 4.1 lists the tasks that the social sciences faculty in the Making the Implicit Explicit were asked to discuss. Not only should all the parts of the dissertation fit together and each task be addressed, but they should also be of consistent quality. As we discuss each of the tasks, we provide, in Tables 4.2 through 4.7, summaries of the descriptions of faculty from economics, psychology, and sociology. The Introduction The introduction motivates the work and makes the case for the research. The student’s task is to state the problem, set the project in context, present the research question, and let the reader know the general strategy for the argument. As a distinct chapter, the introduction often includes the literature review and theory, and provides an overview of the entire dissertation. Many students write or rewrite the introduction after all else is completed. In the paper-style dissertation, the introduction typically connects the papers and identifies a common theme. Literature Review In the traditional-style dissertation, the literature review is typically part of the introductory chapter. In the paper-style dissertation, each substantive chapters starts with an introduction and a literature review. The task of the literature review is not simply to summarize

the literature but rather to connect the problem to the research on the topic, provide a context by synthesizing the history and controversies of the field, and build a story that leads to the hypotheses. The process of choosing the appropriate works to cite and of crafting an argument is not easy. However, doing so is an important professional skill. Theory Theory is often part of the introduction and literature review, but it may be its own chapter. The student’s research question(s) should be linked to theory, and theory should be used to select appropriate methods. Method The social sciences faculty made few general remarks about the nature or role of the methods section in a social science dissertation, in part because different fields use different methods. Indeed, different fields have different standards depending on whether students are collecting their own data or are using an archival data set. In all cases, the methods used should be aligned with the study’s research question(s) and theory. Results or Data Analysis In the social sciences, data analysis and results involve taking raw material—quantitative or qualitative data—and trying to make sense out of them. The goal is usually to infer causality from nonexperimental or correlational data. Consequently, the quality of the tasks is a function of how complete the analysis is, the techniques used, and the robustness of the results. The section should tell the reader why a particular analysis is being conducted and what the limitations are. Discussion and Conclusion The discussion and conclusion is typically the last section or chapter of a dissertation. Some faculty draw a

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 16

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S O C I A L S C I E N C E S

16

Table 4.2 Introduction in the Social Science Dissertation at Different Quality Levels Outstanding • • • • • • • • • •

well written brief, interesting, surprising, and compelling motivates the work has a hook provides a clear statement of the problem explains why the problem is important and significant places the problem in context presents an overview of the theory, methods, results, and conclusions lays out the study’s implications provides a road map of the dissertation

Very Good • • • • • • •

well written interesting has breadth, depth, and insight motivates the work poses a good question or problem explains why the problem is important and significant provides an overview of the dissertation

Acceptable • • • • • •

not well written or well organized lacks or provides minimal motivation for the work makes a case for a small problem does not do a good job of explaining why it is interesting or important provides minimum or poor context for the problem presents minimal overview of the work

Unacceptable • • • • • • •

poorly written and organized provides little or no motivation for the problem problem is not stated well, is wrong, or trivial does not make the case for the importance of the topic does not provide or does not put problem in a clear context does not present an outline or overview of the research contains extraneous material

distinction between the discussion and the conclusion: The conclusion summarizes and wraps things up, whereas the discussion, which is more important, should tie in to the introduction and put the work in a larger perspective. Some disciplines and faculty consider the discussion and conclusion task the creative part of the dissertation, the place where the student has a chance to draw independent conclusions about the

project and show what it means in the larger perspective of the discipline. The economics, psychology, and sociology faculty who participated in the Making the Implicit Explicit study had many insightful and specific comments about the quality of the tasks of the dissertation in their fields. You can find summaries of their descrip-

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 17

Maintaining Consistent Quality Table 4.3 Literature Review in the Social Science Dissertation at Different Quality Levels Outstanding Comprehensive, thorough, complete, coherent, concise, creative, up to date The student • • • • • • • • • • •

shows critical and analytical thinking about the literature synthesizes a body of literature integrates literature from other fields displays understanding of the history and context of the problem identifies problems in and limitations of the literature selective (discriminates between important and unimportant works) identifies and organizes analysis around themes or conceptual categories student adds own insights uses the literature to build an argument and advance the field like a good review article makes reader look at the literature differently

Very Good Comprehensive but not exhaustive The student • • • •

provides a thoughtful, accurate critique of the relevant literature shows student’s understanding of and command over the most relevant literature student selects literature wisely and judiciously sets the problem in context, uses literature to build a case for the research

Acceptable The student • • • • • • • •

provides adequate coverage of the literature demonstrates that student has read and understood the literature lacks critical analysis and synthesis not selective (does not distinguish between more- and less-relevant works) misses some important works cites some works that are not relevant an undifferentiated list (“This person said this, this person said that.”) does not put problem in context

Unacceptable Missing, inadequate, or incomplete The student • has not read enough and does not cite enough sources. • misinterprets or does not understand the literature. • misses, omits, or ignores important studies, whole areas or literature, or people who have done the same thing. • has not read the source or has only read the abstract. • cites articles that are out of date. • does not provide a context for the research.

17

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 18

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S O C I A L S C I E N C E S

18

Table 4.4 Theory in the Social Science Dissertation at Different Quality Levels Outstanding • • • • • • • • • •

original, creative, insightful, innovative simple and elegant well conceived, logically consistent, and internally coherent identifies and critically analyzes strengths and weaknesses uses more than one theory or model compares or tests competing theories advances concepts develops, adds to, revises, or synthesizes theory(ies) aligns with research question, methods, and observations has broad applicability

Very Good • • • •

complete and correct uses existing theory well informs the research question and measures identifies where it works and where it does not work

Acceptable The student • • • •

understands theory. uses theory appropriately. does not specify or critically analyze the theory’s underlying assumptions. offers hypotheses that do not derive logically from theoretical premises.

Unacceptable Absent, omitted, or wrong, misunderstood, or misinterpreted The student • cannot explain it or why it is being used. • uses theory inappropriately. • does not align theory with research question, literature review, or methods.

tions in Appendix A. Ask your advisors to be similarly specific, if not more specific, about what constitutes high-quality work in the tasks of the dissertation. Above all, understand the form you are required to use for your dissertation and the tasks you must address within it. Take time to map out your argument and dis-

cuss in detail with your advisors and committee how you will make the dissertation work as a coherent whole. Discuss with them how each of the parts will contribute to the argument. Study recent dissertations from your department and see how the students made the pieces work together.

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 19

Maintaining Consistent Quality Table 4.5 Method in the Social Science Dissertation at Different Quality Levels Outstanding • • • • • • •

original, clear, creative, and innovative provides a thorough and comprehensive description identifies strengths and weakness/advantages and disadvantages flows from question and theory uses state-of-the-art tools, techniques, or approaches applies or develops new methods, approaches, techniques, tools, or instruments uses multiple methods

Very Good • • • •

appropriate for the problem uses existing methods, techniques, or approaches in correct and creative ways discusses why method was chosen describes advantages and disadvantages

Acceptable • • • •

appropriate for the problem uses standard or less-sophisticated methods correctly provides minimum or sufficient documentation shows basic competence

Unacceptable • • • • • •

lacks a method uses wrong method for the problem uses method incorrectly methods do not relate to question or theory confounded or fatally flawed does not describe or describes poorly (insufficient detail)

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 20

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S O C I A L S C I E N C E S

20

Table 4.6 Results or Data Analysis in the Social Science Dissertation at Different Quality Levels Outstanding • • • • • • • • • • • • • •

multidimensional original, insightful, surprising uses advanced, powerful, cutting-edge techniques analysis is sophisticated, robust, and precise provides justifications for each analysis aligned with question and theory sees complex patterns in the data iteratively explores questions raised by analyses results areusable, meaningful, and unambiguous presents data clearly and cleverly makes proper inferences provides plausible interpretations discusses limitations refutes or disproves prior theories or findings

Very Good Analysis is thorough, appropriate, and correct. The student • • • • •

uses standard methods. produces rich, high-quality data. links results to question and theory. substantiates the results. provides plausible arguments and explanations.

Acceptable Objective, routine, and correct but not sophisticated The student • • • • • •

aligns data and results with question and theory. produces a small amount of thin data. provides results that are correct but not robust. includes extraneous information and material. has difficulty making sense of data. offers an interpretation that is too simplistic.

Unacceptable • • • • •

analysis is wrong, inappropriate, or incompetent data are wrong, insufficient, fudged, fabricated, or falsified data or evidence do not support the theory or argument data do not answer the question interpretation is not objective, cogent, or correct

The student • • • • • •

cannot distinguish between good data and bad data. cannot discern what is important or explain the results. does not explain counterintuitive results. explains away or buries results that are inconsistent with expectations. makes improper inferences. overstates the results.

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 21

Maintaining Consistent Quality Table 4.7 Discussion and Conclusion in the Social Science Dissertation at Different Quality Levels Outstanding Short, clear, concise, and creative; interesting, surprising, and insightful The student • • • • • • • • •

summarizes the work. refers to the introduction. ties everything together. explains what has been accomplished. underscores and explains major points and findings. discusses strengthens, weaknesses, and limitations. identifies contributions, implications, applications, and significance. places the work in a wider context. raises new questions and discusses future directions.

Very Good The student • • • • • • • •

provides a good summary of the results. refers to the introduction. states what has been done. ties everything together. states its contribution. identifies possible implications. discusses limitations. identifies some future directions.

Acceptable The student • • • • •

summarizes what has been accomplished. repeats or recasts the results or major points. does not address the significance or implications of the research. does not place the work in context. identifies a few, nonspecific next steps.

Unacceptable Inadequate or missing The student • • • • • •

summarizes what has already been said. repeats the introduction. does not tie things up. does not understand the results or what has been done. claims to have proved or accomplished things that have not been proved or accomplished. does not draw conclusions.

21

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 22

5 Achieving Excellence

HROUGHOUT THIS BOOKLET, we have offered suggestions for achieving quality in your dissertation. The final table of the booklet, Appendix B, provides practical advice gathered from the social sciences faculty in the study. We close with some frank recommendations about the work ahead of you.

choice, structure of the sentences) and the quality of your argument. Why is writing so important? The quality of your ideas will not shine through poor or ineffective writing. Regardless of how brilliant the ideas and how outstanding your research, if you cannot convey your ideas and data clearly, concisely, and coherently, the reader will not be able to appreciate their import. Likewise, if your presentation is not well planned, if you cannot effectively map out and sustain an argument or make a case, you miss an opportunity to share your knowledge and contribute to the field. Moreover, the mechanics of grammar and the style of prose affect the perception of a work’s quality. Many faculty believe that unclear writing reflects unclear thinking, and that a good researcher is also a competent writer. Once you are out in the field, your writing will be an indicator of the quality of your thought and your attention to the details of research. If your work is error laden, or you did not take the time to plan and organize the presentation of the material, the reader will wonder, with good reason, how careful you were in your research. The faculty in the study spoke candidly about the surprising amount of poor writing they see among their graduate students, and some even suggested that writing should be among the essential tasks of the dissertation. For these reasons, learning to write and communicate well is one of your professional responsibilities. Furthermore, an expert is expected to be able to communicate effectively about his or her field to a variety of audiences. Your dissertation demonstrates

T

Practice Academic Honesty By passing the dissertation, your advisor, committee, department, and university are certifying that you have the skills, knowledge, and disposition expected of a professional in your field, which includes a commitment to integrity. Honest use of data and sources is the fundamental expectation of academic work. None of the levels of quality described here matter if you plagiarize, deliberately misuse or misrepresent sources, or falsify data. Such serious betrayal of professional standards puts your degree in jeopardy. It could even be rescinded if the dissertation is passed and then found to have these abuses.

Develop Professional-Level Writing Skills You probably noticed that “writing” is a constant item in the summaries of faculty descriptions of quality. The quality of your writing plays a large role in faculty members’ assessment of the quality of your dissertation. They are looking at the quality of your prose (word

22

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 23

Achieving Excellence your skill in communicating with your peers in the discipline.

Take Action to Improve Your Writing Although some graduate students, domestic and international, have problems with grammar, many also have problems with composition—basic structure and rhetoric. Some faculty report spending a great deal of time working with students to improve the quality of writing in the dissertation—time that should be focused on the larger ideas. You have made a considerable investment of time and energy learning about your discipline. If the feedback on your papers for your course work, dissertation proposal, or chapter drafts suggests that you need to pay more attention to this skill, here is a plan of action. • Know good scholarly writing. Ask your advisor or other faculty members to recommend the work of a scholar whose writing is very clear; read it not for the subject but for how it is written. • Plan your dissertation. While you are working on the dissertation, it is important to be clear about what you want to say before you begin to shape chapter sections, paragraphs, and sentences. For example, make a map or outline of your dissertation and plot out the argument. Try it out by presenting it to other students and faculty mentors. Use the map as you draft the various pieces of the dissertation. • Plan the pieces of the dissertation. Decide how you will organize each chapter. What are the main points? In what order does the reader need to know the information? What supporting data will you use, including charts, graphs, tables, equations and other illustrations? • Write and revise. Understand the process of writing. Approach the tasks of writing and revising as separate activities, for they each demand different skills. Although writing and polishing at the same time might seem efficient, it is not. After you have drafted a piece, set it aside for a while, and then review what you have written and reshape it. Then go back again, at another time, and polish the grammar

23 and prose by reading aloud what you have written. You will be surprised at what you see. • Follow conventions. Find and use a handbook of grammar and style; your university’s bookstore and library will have many. Ask other students and faculty members to recommend books about academic writing. In addition, know the style and formatting conventions of your discipline. Many disciplines publish style guides, with instructions for everything from charts and tables to citation format and manuscript preparation. Some of these guides include discussions of grammar and punctuation or suggest good sources for this information. Major journals also provide guides for preparing manuscripts. • Get feedback. Although there are many things you can do as your own editor, it is important to seek and consider the feedback of others. Go to the campus writing center for tutoring. Join a writing group and work with your peers. Ask a faculty member you respect to work over a short piece of text with you; even a half an hour in which someone edits your work and explains his or her suggestions can be extremely enlightening. • Practice writing and presenting your research. Take advantage of opportunities to present at conferences or campus research colloquia. Seize opportunities to write and speak about your field or project. The more varied the audience the better.

Set the Bar Having a clear target makes it easier for you to ask questions, seek guidance, and make corrections. It also makes it easier for your advisors to answer questions, provide guidance, and suggest revisions. The descriptions of quality presented in this booklet are benchmarks of excellence. You and your advisor and committee might find it helpful to develop specific expectations of quality for your dissertation. Doing so will help them guide your work and will help you use their feedback. If you and your advisors are interested in putting expectations for your dissertation in writing, you might start with the general examples

17221-DQD-SocialSciences

24

12/1/08

4:47 PM

Page 24

D E V E L O P I N G Q U A L I T Y D I S S E R T AT I O N S I N T H E S O C I A L S C I E N C E S in this booklet and customize them for your discipline and project. A larger question, though, is where to set the bar. You and your advisors need to talk about finding the balance between challenging yourself and setting realistic goals. Discuss the level of quality they expect in the dissertation. Talk about your goals for quality and your professional aspirations (you might find it helpful to put these thoughts in writing first, before the conversations). As you set the bar together, keep in mind that the “very good” that the faculty in the study discussed is the equivalent of an A. Achieving that level of excellence is an accomplishment that you can be proud of. Engage Your Advisors Above all, we recommend that you take the initiative to speak with your advisors, committee members, and other faculty members who mentor you, and that you do so early in the process and frequently throughout it. You are entitled to guidance about your dissertation and to prompt and constructive feedback throughout the process. Know what you are aiming for and what you need to do to meet your goals. Be clear about the tasks of the dissertation and the expectations for quality for each. Work with your advisor and committee to set a realistic schedule that includes time to respond to drafts, especially early in the process. Meet the deadlines you and your committee agree to. When you meet, use their time and expertise wisely; ask questions and make sure you understand their directions. Make the revisions your advisor and committee members ask you to make—or provide them with reasoned justifications for not making them. If their feedback seems contradictory or confusing, ask for clearer or more consistent guidance. Be proactive about solving any problems. That is an important professional skill. If you feel stuck or unsure about something, talk to your advisor, a faculty member you respect, or other students. If you are not getting the

guidance or feedback that you need, bring it to the attention of someone who can help you, such as the program head, department chair, or graduate dean. Don’t withdraw and hope the problem will go away. Don’t give up. Persistence is also an important professional attribute.

Engage Your Peers Use the process of writing the dissertation to develop colleagues for the future. Sharing work with peers in order to get honest and expert feedback is a habit— and skill—you should learn early. Form a group of peers and meet regularly to check drafts, discuss problems and questions, and savor accomplishments. Meeting regularly to discuss your work will help keep you on track as you prepare to meet interim deadlines.

Applaud Yourself No doubt, at every milestone in the process, you will want to thank the friends and family members who are supporting and encouraging you. They take pride in what you’ve accomplished and the role they have played in your success. But remember, there is someone else who should be congratulated: you. Even when you sometimes feel like you are taking small steps up a big mountain, pause to recognize how far you have come; you will be surprised at the distance you have covered. By pursuing the PhD, you are making an enormous investment in yourself. Take time along the way to appreciate what your commitment to this effort says about you, personally and professionally. The dissertation is a symbol of the hours and commitment that you have devoted to the process. And when you have finished it and the dissertation is submitted and defended, be sure to celebrate!

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 25

Appendix A Tasks of the Social Science Dissertation

25

Introduction Acceptable

Well written; very clearly states what the study is about; raises a question without context; clearly identifies the techniques that will be used but does not provide a good sense of the economics

Not well organized or compelling; lacks motivation; does not do a good job of explaining why the problem is important, or it oversells its importance; provides a general discussion of the question or issues, but is insufficiently specific about the techniques and results to come; has not pulled the pieces together

Does not or cannot lay out the question; shows a fundamental lack of understanding of the problem; fails to outline the research; overstates what was done; makes huge claims about the results

Page 26

Well motivated, very brief, and very well written; compelling, penetrating and insightful; motivates and prompts the reader; sets up and articulates an interesting question; provides a clear statement of the problem; identifies why the problem is important; relates this body of work to other bodies and identifies how it is different; provides an overview of the dissertation; summarizes the methods, data, and results; like an executive summary

Unacceptable

4:47 PM

Very Good

12/1/08

Outstanding

Literature Review Outstanding Short, concise, complete, coherent, and comprehensive; focuses on the most directly relevant works; very analytical; provides new insights into the literature; insightfully synthesizes the literature; shows how the literature relates to the question; identifies problems and limitations in the literature; shows how the research will advance the field

17221-DQD-SocialSciences

26

Table A.1 Tasks of the Economics Dissertation by Quality Level

Very Good Coherent, thoughtful, and accurate critique of the literature; sufficiently comprehensive to set the context for the research question; shows understanding of the literature and where the research fits

Acceptable

Unacceptable

Cites most of the key literature; may not be on top of recent literature, no new references; may cite papers that are not relevant to the question; does not sufficiently synthesize the literature; a listing, “X found this, Y found this”; puts the problem in context; identifies the student’s contribution

Sloppy mind dump; uses chronology as an organizing principle; neglects an important paper that contains the result; fails to cite important, relevant articles; does not cite enough sources; student cites sources he or she has not read (or student has read only the abstract); shows lack of understanding of important articles; just a list of names, “This person did this and this person did that”; does not clearly relate the literature to the student’s contribution

Theory Very Good

General, comprehensive, creative, original, simple, elegant, tidy; logically consistent and internally coherent; has

Appropriate; uses existing methods or models well

Acceptable Mediocre, very loose, or vacuous theory; uses or regurgitates existing theory; not well articulated, implicit

Unacceptable Wrong, has errors, or not properly adapted to the situation; student cannot explain the theory; does not convince

APPENDIX A

Outstanding

Acceptable rather than explicit; does not make predictions based on the theory; does not understand the limitations of the theory

the reader that it is cogent or logically consistent; clearly an afterthought

4:47 PM Page 27

Methods Outstanding Well done; original, novel; carefully and comprehensively documented, lays out every step; indicates why the method or technique was used; used appropriately; makes and justifies judgments about the procedures and the trustworthiness of the data; identifies limitations and potential weaknesses; flows naturally from the theory; integrates the theory and empirical work; gathers own data; adds a new twist or application to existing methods; uses the best state-of-theart techniques; develops innovative new methods or estimators; introduces an estimator from another area; contributes to the theory of methodology

12/1/08

clever arguments; aligns well with the question; provides intuition for the results; identifies the assumptions under which the model works best and the internal limitations; borrows from outside the field; applies in an area where no one thought to apply it before; builds on prior theoretical models; a new, more abstract theory; a new theoretical method for solving complicated economic problems or policies; a unified model; a model that yields consistent information

Unacceptable

17221-DQD-SocialSciences

Very Good

APPENDIX A

Outstanding

Very Good Important for that problem but does not have applications beyond that

Acceptable

Unacceptable

Adequately uses the right methods for the problem; uses existing, less sophisticated, or inferior methods reasonably well; documented sufficiently so that the reader can follow and replicate the method; methods do not closely relate to the theory

Uses the wrong methodology or tools to address the problem; methods do not relate to the theory; data are not handled carefully; has mistakes

27

(continued)

Results/Data Analysis Unacceptable

Well executed; robust, complete, meaningful, interesting, surprising; adds to knowledge; has high-quality data; understands the data set inside and out, including the shortcomings and limitations of the data; choice of model, statistical, and/or econometric tools is convincing; uses a sophisticated or innovative technique to approach the question; analysis is multidimensional, interesting, and important; analyzes the data in different ways and explains the differences in results; develops a theory to explain anomalies; provides a very plausible interpretation of results; refutes important prior findings

Well executed but not robust; demonstrates something believed to be true; shows good understanding of statistical and econometric methods; attempts to link theory and methods; applies established methods to a new data set; goes beyond the basic data set and makes a convincing case; points out causal relationships

Results do not seem very robust; uses a scattershot approach to statistical analysis that is not closely aligned with the theory; not clear that a causal relationship has been found; cannot clearly state the limitations of the results

Data are wrong; full of errors; obvious things are missing; ignores other available data; wrong, poor, or scattershot analysis; accepts incorrect data coming out of the computer at face value; magnitudes of the estimates are not convincing; results are incomplete or contradictory; results are wrong or trivial; results are not explained; counterintuitive results are not explained; results that are inconsistent with expectations are explained away or buried; makes improper statistical inferences; overstates results; empirical findings cannot be compared across chapters

Page 28

Acceptable

4:47 PM

Very Good

12/1/08

Outstanding

Discussion and Conclusion Very Good

Acceptable

Unacceptable

Short and concise; completes the argument; puts the work in context; summarizes and brings the work together—sets out the problem, methods, theorems, and data; indicates what has been learned; conclusions are connected to the introduction; student shows keen understanding and appreciation of the limitations of the work and what can be done to strengthen it; identifies the significance and applications for other audiences and fields; has policy ramifications; talks about future directions

Very short and well synthesized but is not as strong as it could be; ties in with the introduction and literature review; compares what was known before with what is known now; conclusions are supported by the evidence; shows an appreciation of the shortcomings of the research

Absent or a little too thin; conclusions are not supported by the results; overinterprets or oversells the results

Does not have a discussion; simply takes the introduction and changes future tense to past tense; insufficient discussion of results; misinterprets the findings; claims to prove things that have not been proved

APPENDIX A

Outstanding

17221-DQD-SocialSciences

28

Table A.1

Outstanding

Acceptable

Less breadth, depth, and insight; presents well-articulated, interesting, and important questions about gaps in knowledge

No hook, poorly written, incomplete; lacks structure; approach is formulistic; does not make the case for the importance of the topic; premise fails to take into account something that is already known

4:47 PM

Gap argument for a derivative, mundane project; has legitimate questions about gaps in knowledge, but they may not be interesting; does not try to make the case for or explain why the question is original, interesting, or the next logical step; does a poor job of connecting the question with the literature and putting it in context

Unacceptable

Literature Review Creative, incisive, comprehensive; sparkles; shows critical thinking about the literature; has breadth and depth; uses the primary literature, including classic papers, to make important points and generate hypotheses; has a lot of theory in it; expansive, brings in different points of view; not limited to the particular substantive area, integrates material from related fields; shows where all the pieces of the hypothesis come from; places the work within a larger context; makes reader look at the literature in a different way

Very Good

Acceptable

A very critical review of the relevant literature; shows insight; has a theme or perspective; points out methodological flaws in studies; compares studies and draws connections between them; integrates things in a new way; draws conclusions; explains its relevance for the problem; demonstrates that the student can use the material, apply it to a problem, and develop hypotheses

Adequate coverage of the literature; mentions everything; talks about what others have said; student does not put himself or herself into it; a laundry list of prior findings; lacks critical analysis and synthesis; critiques are derived from other people; makes obvious points

Unacceptable

Page 29

Outstanding

Incomplete; misses or omits important studies or whole areas of literature; does not go back far enough in the literature; leaves out the most recent literature; does not make clear distinctions between theory and methods, talks about them as if they were the same; the literature’s relevance to the question and methods is unclear

Theory Outstanding Creative, original; has a theory; discusses and works with more than one theory or model; articulates and compares competing theories; shows how competing theories are complementary; uses competing ideas to make hypotheses and develop studies; identifies and critically analyzes key theoretical assumptions and boundary conditions; identifies the theories’ implications for the study; advances theory

Very Good

Acceptable

Student has a sophisticated knowledge of and ability to use relevant theories; figures out where the gaps are in the theories and extracts what is useful; uses theory to inform the research questions and measures; discusses how observations are consistent or inconsistent with prevailing theory; suggests how diverse observations can be pulled together; makes some progress

No clear theoretical framework; provides a laundry list of relevant theories; question is not integrated into a theoretical perspective; does not critically analyze the theories’ underlying assumptions or boundary conditions; accepts theories at face value; hypotheses are not logical deductions from theoretical premises; hypotheses do not synthesize multiple theories or test competing theories

Unacceptable Has no theory; does not have a good guiding theory; theory is misunderstood, misclassified, or undeveloped; overlooks a certain body of theory; theory is unrelated to the literature review

Methods Outstanding

Applies methods in correct and creative ways; describes why the

Acceptable Shows basic level of competence; method fits the problem; follows the

Unacceptable Uses wrong or poor methods to answer the question; has a major confound;

29

High quality; a well-designed experiment with proper controls; has a level of complexity that goes beyond the

Very Good

12/1/08

Interesting, comprehensive, coherent, engaging, exciting, surprising; has a hook; draws the reader in; well organized; states the problem and shows why it is important; makes a persuasive, convincing case for the study; leads to the hypotheses; provides an overview of the answers; exhibits depth and breadth of understanding; puts forth implications

Very Good

APPENDIX A

Introduction

17221-DQD-SocialSciences

Table A.2 Tasks of the Psychology Dissertation by Quality Level

(continued)

Methods Outstanding

Acceptable

student is using a particular task, what it does, and how it fits with the study; creates new tasks; uses multiple measures of the same constructs; shows interest in convergent and divergent validity issues

rules for samples, measures, and analyses; uses one measure for each construct

Unacceptable uses an inappropriate population to test a theory; does not have appropriate controls or control groups; does not have controls

12/1/08

obvious; has done some pilot testing to nail down the characteristics of the methods; creatively applies an existing method to a new question; uses a new method; comes up with useful measures

Very Good

4:47 PM

Results/Data Analysis Outstanding

Acceptable

Appropriate; clear; does not conduct supplemental analyses; leaves open data analysis opportunities

Meets the standard of thoroughness or comprehensiveness; has done the minimum analysis required to address the original question; results go back to the hypotheses; does not develop a meaningful story

Unacceptable

Page 30

Creative; uses proper, defensible statistical and analytical methods; uses best, most powerful and most sensitive analytic procedures to address the experimental question; uses cutting-edge techniques; takes existing commercial software and develops new models; applies newer and different models to the data set; provides information about why each analysis is being conducted; analysis is thorough and seamless; integrates among and across levels of analysis; develops new ways to look at the data and makes the most of the data; tells a story; makes a theoretical argument; analyses map back to the hypotheses and answers the questions; shows curiosity through relentless exploration of the data; iteratively explores questions raised by each analysis; pays attention to detail; communicates analyses very clearly; discusses the limitations of the analysis

Very Good

Analyses are wrong, inappropriate, or not well matched to the research question; analyses are not reported completely enough; presents the results poorly; does not follow up on alternative interpretations allowed by the analyses

Discussion and Conclusion Outstanding

Less of the same; does not close the circle, does not come back to the beginning and address the problem

Acceptable Summarizes the results; provides a superficial interpretation of the findings; references to the literature simply state that the findings are consistent with other people’s findings; has a rote discussion of strengths and limitations; provides some very general directions for future research that do not provide structure for the next study; makes wild speculations that have nothing to do with the topic

Unacceptable Shows lack of understanding and careful thought; the discussion and conclusion do not adequately reflect the journey; a disconnect between data and conclusions; restates the results without providing any interpretation; misinterprets the results; interprets the results beyond what the data allow; generalizes too broadly

APPENDIX A

Deep, accurate, creative, enthusiastic; goes beyond summarizing the findings; draws things together; goes back to the introduction; states the hypotheses and answers each one; provides an in-depth account of the findings; develops a novel framework or explanation for unanticipated results or results that have internal contradictions; goes back to the literature and discusses the differences between the student’s findings and other people’s findings; discusses big surprises and the strengths and limitations of the current design or research; puts the study in a larger context; says what it means for the rest of the field; identifies future directions; speculates on why and how the field might need to change; moves the field forward

Very Good

17221-DQD-SocialSciences

30

Table A.2

Acceptable

Well written but less eloquent; poses a clear research question; expresses clarity of purpose; focuses on the key issues; good, solid but not surprising;

Workmanlike; reasonably clear and focused; has a marginal hook but is not exciting; conveys what the research is about; shows understanding of the topic; provides an inkling of the theoretical and methodological approach; may leave something out but does not say anything absolutely wrong

Not grounded in anything; very defensive; tone is very politicized; takes inappropriate stances; goes off on incomprehensible tangents

Page 31

Short, focused, creative, and very synthetic; has a hook; states the problem and shows why it is interesting and important; explains the significance of the study; introduces the literature review; sets the context; locates the project in what has been done before; lays out a thesis and an organizational structure; provides a preview and a road map of where the research is going and what is in the coming chapters

Unacceptable

4:47 PM

Very Good

Literature Review Outstanding

Very Good

Acceptable

Unacceptable

Demonstrates a grounded understanding of the literature; provides reasons for looking at the literature differently; draws on literature in a convincing and supple way; brings together and summarizes a broad body of material and makes meaningful distinctions without being exhaustive; knows what needs to be cited and what does not; analysis is organized around themes; succinct; indicates the significance of the research

Provides meaningful summary of the literature; includes both classic and recent citations; not a laundry list of “Smith said this” and “Jones said that”; demonstrates a nuanced understanding of the literature; takes a body of material and leans it toward a particular direction; brings various intellectual resources to bear on the topic; builds a case for the research and for the hypotheses

Is ill conceived or seems wrong; not analytical, integrated, or synthesized; a stacked annotation, “This person said this” and “This person said this”; just regurgitates material; confusing; not clear why some literature is being cited and other literature is not

Omits people who have done the same thing the student is doing; has not looked at commonly understood bodies of relevant literature; cites articles that are out of date; misinterprets the literature; misquotes major theorists; shows lack of understanding of the literature and where this research fits in the field

Theory Outstanding Not discussed

Acceptable Weakly understood; does not specify assumptions; shows slippage between the conceptual apparatus and the problem

Unacceptable No theory; completely unclear; ideas, theory, and material are not aligned

31

Provides a good, logical, sensible, coherent argument; clearly indicates understanding of the major perspective; shows up in the introduction, literature review, and in the substantive parts of the dissertation; is in the student’s own language; relates to other traditions and

Very Good

12/1/08

Outstanding

17221-DQD-SocialSciences

Introduction

APPENDIX A

Table A.3 Tasks of the Sociology Dissertation by Quality Level

Theory Outstanding

Very Good

Acceptable

Unacceptable

Acceptable

Unacceptable

12/1/08

other ideas; evaluates a specific problem through a theoretical lens; evaluates different theories; sees multiple levels and multiple relationships; links observations to theory; uses conceptual ideas in a creative way; synthesizes theories; develops or creates theory

4:47 PM

Outstanding

Very Good

Appropriate, clever, original, thorough; very well done; has basic validity; student exhibits good judgment about what needs to be said and what can go in an appendix; connects questions and theory with methods; does something that ordinarily cannot be done; uses a novel method or multiple methods (triangulation); uses cutting-edge statistical techniques

More workmanlike; does not provide lengthy definitions of techniques already in the literature; use of a different technique might have been more appropriate or made it more interesting

Appropriate, competent; no fatal flaws; a rubber-stamped use of a textbook method; appropriate for the problem; has basic validity; sample is large enough but barely; uses a very unusual group that does not represent the average; yields a reasonably accurate answer; a different method might have been better

Page 32

Methods Fatally flawed; mismatch between method and problem; student does not seem to understand the method; uses method improperly; the implementation is inappropriate; no clear relationship between hypotheses and variables; variables do not capture the concept; no variance in one of the major variables; measures are not valid or reliable; statistical techniques are inappropriate or poorly explained

Results/Data Analysis Outstanding

Very Good Data rich; provides plausible arguments; student sees interrelations that are not obvious; has rich illustrations

Acceptable Analyses are well executed but not sophisticated or substantial; data are not rich; does not have enough substance; is not clear that the data are really evidence of the concepts; findings are null; provides too much information; looses significant and important find-

Unacceptable Marginal analysis of the data; student does not know why the technique is being used; uses advanced techniques but sees nothing in the data; has obvious misinterpretations of the data; shows every iteration of the model but cannot discern what is important; mindless

APPENDIX A

Appropriate; uses advanced techniques; interprets data properly; sees complex patterns in the data; does a high-level iterative analysis of the data; uses tables, figures, charts, and maps to display the data cleverly; makes clear links between the conceptual apparatus and results;

17221-DQD-SocialSciences

(continued)

32

Table A.3

Very Good

ings in the midst of endless discussions of insignificant ones; includes every regression equation

presentation of data without interpretation; uses graphic displays to create misleading perceptions; evidence does not support the argument; results do not follow from the analysis and are interpreted incorrectly; oversells or over generalizes the results

4:47 PM

Unacceptable

Very Good

Acceptable

Unacceptable

Briefly summarizes what was done and reaches into new areas and different ways of seeing things; ties the whole study together; shows that the questions, methods, analyses, and findings are consistent; connects to the theoretical puzzles or debates the student started with and takes them to another level; underscores the findings; discusses what is interesting and surprising about the results; recognizes the study’s strengths, weaknesses, and limitations; sees the big picture significance of the work; speculates on and provides an astute discussion of future directions; has implications for the subfield, sociology, or social science

Discusses what is now known that was not known before; shows the limits of the research; indicates where future research might improve upon what was done; proposes logical follow-on research; focuses on very specific findings and neglects to bring out the general implications

Restates what has already been said; summarizes rather than analyzes; overstates the results; does not see or generalizes to the big picture; indicates that further research is necessary but does not provide specifics

Just a summary; no conclusion; takes a section out of the introduction and puts it in the conclusion; oversells the results

Page 33

Discussion and Conclusion Outstanding

12/1/08

highlights the most important, original, and significant contributions; goes beyond supporting the argument and disproves common theories

Acceptable

17221-DQD-SocialSciences

Outstanding

APPENDIX A

Results/Data Analysis

33

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 34

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 35

Appendix B Advice for Writing a Social Science Dissertation Table B Writing the Social Science Dissertation Introduction • • • • • • •

State your question or problem clearly and concisely. Discuss the importance of your problem. Put your problem in context. Discuss your methods, how your results were obtained, and how broadly applicable they are. Identify your contribution and its significance. Provide an overview of your entire dissertation. Review and rewrite your introduction as necessary after writing your conclusion.

Literature Review • • • • • • • • • • • • • •

Model your literature review on a good review article. Be selective, critical, and comprehensive in your review of the literature. Read the primary literature, especially classic or commonly cited books and papers. Include and cite the most important and relevant works. Bring in relevant literature from other fields and disciplines. Connect the literature to your problem. Provide a context for your problem by summarizing and synthesizing the history and controversies that surround your problem. Organize your literature review around themes. Focus on concepts and ideas, not on who said what. Present an analytic and synthetic discussion of the literature. Include your own insights on the literature. Use the literature to show what is missing in the literature and why your study needs to be done. Tell a story or build a story that leads to your hypotheses, if you have them. Discuss how your study will advance the field.

Theory • • • •

Use theories that are appropriate for your problem. Articulate your theory. Link your theory to your research question(s), methods, and observations. Understand and be able to explain the theory(ies) you are working with or developing.

35

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 36

APPENDIX B

36 Methods • • • • • •

Use the best, most appropriate cutting-edge methods or techniques for your problem. Align and integrate your methods with your theory and research question(s). Where possible and appropriate, use multiple methods to research your problem. Document your methods thoroughly. Provide a justification for each method or technique you use. Discuss the pros and cons, strengths and weaknesses of your methods.

Results and Analysis • • • • • • • • • •

Explain why each analysis is being conducted and what its limitations are. Map your analyses back to your hypotheses and answer the research question(s). Be thorough and correct. Ask penetrating questions of your data. Engage in supplementary analyses. Explore questions raised by your analyses. Look for and explain complex patterns in your data. Use the data to make an argument, tell a story, or prove a point. Provide a coherent introduction to and summary of your results. Provide plausible explanations and interpretations of your results.

Discussion and Conclusion • • • • • • • • • •

Take a deep breath and find the time and energy to write a solid conclusion. Connect your conclusion to your introduction. Tie all the pieces of your dissertation together. Highlight the major points and findings of your work. Draw independent conclusions about your research. Discuss the significance and implications of your work for theory, research, or practice. Place your conclusion in the context of the larger perspective of the discipline or society. Identify the shortcomings of your research. Anticipate and respond to criticism. Identify new questions or next steps.

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 37

APPENDIX B

37 DOS

DON’TS The Dissertation as a Whole

• • • • • • • • •

• • •

Master your discipline’s subject matter. Learn and internalize the professional standards of your discipline. Come up with a clearly defined researchable question or problem whose answer/results will be important to your field. Know the literature in your field and in related fields, especially as it relates to your dissertation topic. Take the initiative to explore new ideas and new literatures. Look at other fields and other disciplines for ideas. Take the time necessary to fully develop your ideas. Present your ideas clearly, concisely, and persuasively. Do what you say you are going to do in your dissertation proposal or something very close to it, but remember, ideas evolve and so should your research. Use proper research methods appropriately. Anticipate and answer readers’ questions and criticisms. Make the revisions your advisor and committee members ask you to make or provide them with good justifications for not making them.

• •

• • • •

Think of your dissertation as the biggest or best thing you will ever do in your life. Set an unreasonable quality level for your dissertation. Outstanding dissertations are rare. Faculty expect most dissertations to be very good. You can always expand and polish your work after you receive your degree. Expect your advisor to do your research or write your dissertation for you. Submit a manuscript containing spelling and grammatical errors Misinterpret or oversell your results. Push for a defense if your advisor does not feel you are ready to defend.

Introduction • • • • • • •

State your question or problem clearly and concisely. Discuss the importance of your problem. Put your problem in context. Discuss your methods, how your results were obtained, and how broadly applicable they are. Identify your contribution and its significance. Provide an overview of your entire dissertation. Review and rewrite your introduction as necessary after writing your conclusion.

• •

Put in a lot of extraneous material. Go off on tangents.

Literature Review • • • • • • •

• • • • • • •

Model your literature review on a good review article. Be selective, critical, and comprehensive in your review of the literature. Read the primary literature, especially classic or commonly cited books and papers. Include and cite the most important and relevant works. Bring in relevant literature from other fields and disciplines. Connect the literature to your problem. Provide a context for your problem by summarizing and synthesizing the history and controversies that surround your problem. Organize your literature review around themes. Focus on concepts and ideas, not who said what. Present an analytic and synthetic discussion of the literature. Include your own insights on the literature. Use the literature to show what is missing in the literature and why your study needs to be done. Tell a story or build a story that leads to your hypotheses, if you have them. Discuss how your study will advance the field.

• • • • • • • •

Discuss everything that has been written on your topic. Overlook relevant parts of the literature. Cite literature that is out of date. Cite only books or review articles. Take the literature at face value. Present a descriptive summary of the literature devoid of critical analysis of it. Cite papers you have not read or do not understand. Plagiarize or misuse your sources.

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 38

APPENDIX B

38 DOS

DON’TS Theory

• • • •

Use theories that are appropriate for your problem. Articulate your theory. Link your theory to your research question(s), methods, and observations. Understand and be able to explain the theory(ies) you are working with or developing.

• • •

Omit theory. Use the wrong theory(ies). Use the theory(ies) or equations inappropriately.

• • •

Use the wrong method(s) for your problem. Use methods inappropriately. Omit necessary details.

Methods • • • • • •

Use the best, most appropriate cutting-edge methods or techniques for your problem. Align and integrate your methods with your theory and research question(s). Where possible and appropriate, use multiple methods to research your problem. Document your methods thoroughly. Provide a justification for each method or technique you use. Discuss the pros and cons, strengths and weaknesses of your methods.

Results and Analysis • • • • • • • • • •

Explain why each analysis is being conducted and what its limitations are. Map your analyses back to your hypotheses and answer the research question(s). Be thorough and correct. Ask penetrating questions of your data. Engage in supplementary analyses. Explore questions raised by your analyses. Look for and explain complex patterns in your data. Use the data to make an argument, tell a story, or prove a point. Provide a coherent introduction to and summary of your results. Provide plausible explanations and interpretations of your results.

• • • • • •

Falsify, fudge, fabricate, or hide data. Present data or analyses that are not relevant to the topic. Present figures and tables without discussing them. Use graphic displays to create misleading perceptions. Make improper inferences. Overstate your results.

Discussion and Conclusion • • • • • • • • • •

Take a deep breath and find the time and energy to write a solid conclusion. Connect your conclusion to your introduction. Tie all the pieces of your dissertation together. Highlight the major points and findings of your work. Draw independent conclusions about your research. Discuss the significance and implications of your work for theory, research, or practice. Place your conclusion in the context of the larger perspective of the discipline or society. Identify the shortcomings of your research. Anticipate and respond to criticism. Identify new questions or next steps.

• • •

Omit the conclusion. Blow it off and write an insufficient conclusion. Repeat your introduction.

17221-DQD-SocialSciences

12/1/08

4:47 PM

Page 39

Making the Implicit Explicit: About the Study

is based in large part on a study conducted during 2003–2004 at nine doctoralgranting universities. The study included focus groups with 276 faculty in 74 departments across 10 disciplines representing the sciences (biology, electrical engineering and computer engineering, physics/physics and astronomy, mathematics), social sciences (economics, psychology, sociology), and humanities (English, history, philosophy). The faculty were selected because they had advised many doctoral students and served on many dissertation committees. The average faculty member chaired 13 dissertations and served on 36 dissertation committees. Faculty from the following universities participated in the study:

T

• • • •

• • • • •

HIS BOOKLET

Syracuse University University of Colorado at Boulder University of Illinois at Urbana-Champaign University of Kansas University of Southern California

The faculty were asked to describe dissertations in their disciplines, and, specifically, to describe dissertations and the tasks of dissertations at four different quality levels: outstanding, very good, acceptable, and unacceptable. They were also asked to explain the purpose of the dissertation and what it means to make an original and significant contribution in their discipline. At the time of the focus group conversations, the 86 social sciences faculty whose responses are featured in this booklet had chaired about 1,297 dissertations and served on about 3,355 dissertation committees across 21 economics, psychology, and sociology departments. On average, they chaired 14 dissertations and served on 39 dissertation committees.

Duke University Michigan State University Northwestern University Stony Brook University

39